"Connection refused" in nginx when doing Load testing



  • I carry out load testing of my API by YandexTank.

    Load: 100RPS for 60 seconds POST request with a small body
    I don’t understand what’s going on: for about the first 1600-1900 requests I get a successful response with a code 200. The rest of the requests are error 111 Connection refused.
    If you run the test again immediately after the first run, 111 Connection refused errors will fly from the first request, and if you wait 10 minutes after launch, the situation will be as in the beginning: 1600-1900 successful and then only 111 Connection refused.

    If the load is reduced to 30RPS and the test is extended to 100 seconds. Then the Situation will repeat itself, everything will break again after 1600-1900 requests.
    The feeling that somewhere there is some kind of limit on the quantity and lifetime of "something". I would be glad to have any suggestions on how to fix this! What parameters to tweak or where to look?

    Played with different parameters: worker_processes, worker_rlimit_nofile, worker_connections, keepalive_timeout. The last remaining configuration is shown below. Nothing has helped yet.

    ulimit -n 200000
    Enviroment: Ubuntu 18.04.1, DigitalOcean: RAM 2Gb + 2CPU

    Nginx config:

    user  www-data;
    worker_processes  4;
    
    error_log  logs/error.log;
    error_log  logs/error.log debug;
    
    worker_rlimit_nofile 33000;
    
    events {
        worker_connections  4000;
        use epoll;
        multi_accept on;
    }
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
    
        reset_timedout_connection on;
        sendfile        on;
        keepalive_timeout 15;
    
        include '/etc/nginx/sites-enabled/*';
    }
    

    File (server + location)

    ...
        location ~ ^/api {
            content_by_lua '
                ngx.say("OK")
                return ngx.exit(200)
            ';
        }
    ...
    

    YandexTank Ammo

    [Host: myserver.com]
    [Content-Type: application/x-www-form-urlencoded]
    80 /api
    xxxxxx=777777777&yyyyyyy=666666666&zzzzzzzz=ce96c7c3-87e5-4b90-b1e5-9664697aa5b6
    


  • The problem was with my home router
    He simply could not handle such a load.
    The problem was solved by replacing the router



Suggested Topics

  • 3
  • 3
  • 2
  • 3
  • 2