Raju Ginni

wordpress tutorials seo hosting etc

You are here: Home / nginx server tutorials (installation, configuration, performance tuning, security) / nginx open file limit connections ulimits sysctl

nginx open file limit connections ulimits sysctl

1st set at system level then user level then process level.

nginx open file limit rlimit should twice than connection limit

UNIX offers several ways to set the number of FDs per process:

  • The ulimit command if you start NGINX from a shell
  • The init script or systemd service manifest variables if you start NGINX as a service
  • The /etc/security/limits.conf file

There is also a system‑wide limit on the number of FDs, which you can set with the OS’s sysctl fs.file-max command. It is usually large enough, but it is worth verifying that the maximum number of file descriptors all NGINX worker processes might use (worker_rlimit_nofile * worker_processes) is significantly less than fs.file‑max.

 

worker_processes auto equals number cores, 4 core 4*1024= 4086 files = 2048 connections.

check nginx rmlit nofiles

finding process id, check existing and set in daemon , after adding to etc secuirty limit conf file.

check the current file limit on the master process

cat /proc/$(cat /var/run/nginx.pid)/limits | grep open.files

root@srv443573:~# cat /proc/$(cat /var/run/nginx.pid)/limits | grep open.files
Max open files 30000 30000 files

check the current limit on the nginx worker process

ps --ppid $(cat /var/run/nginx.pid) -o %p|sed '1d'|xargs -I{} cat /proc/{}/limits|grep open.files

root@srv443573:~# ps –ppid $(cat /var/run/nginx.pid) -o %p|sed ‘1d’|xargs -I{} cat /proc/{}/limits|grep open.files
Max open files 30000 30000 files
Max open files 30000 30000 files
Max open files 30000 30000 files
Max open files 30000 30000 files

worker_rlimit_nofile directive in {,/usr/local}/etc/nginx/nginx.conf

Not works limits by system.

set nolimit without Systemd in ect security limits

# /etc/security/limits.conf
# /etc/default/nginx (ULIMIT)
$ nano /etc/security/limits.d/nginx.conf
nginx soft nofile 65536
nginx hard nofile 65536

 

set nolimit with Systemd ubuntu

nano /lib/systemd/system/nginx.service

LimitNOFILE=30000

 

check existing open file limit

process level  file descriptor limit

cat /proc/$(cat /var/run/nginx.pid)/limits

root@instance-1:~# cat /proc/491113/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 63948 63948 processes
Max open files 1024 524288 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 63948 63948 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
root@instance-1:~#

nano /lib/systemd/system/nginx.service

LimitNOFILE=30000

systemctl daemon-reload

nginx -s reload

cat /lib/systemd/system/nginx.service

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g ‘daemon on; master_process on;’
ExecStart=/usr/sbin/nginx -g ‘daemon on; master_process on;’
ExecReload=/usr/sbin/nginx -g ‘daemon on; master_process on;’ -s reload
ExecStop=-/sbin/start-stop-daemon –quiet –stop –retry QUIT/5 –pidfile /run/nginx.pid
TimeoutStopSec=5
KillMode=mixed
LimitNOFILE=30000
[Install]
WantedBy=multi-user.target

root@instance-1:~# cat /proc/493972/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 63948 63948 processes
Max open files 30000 30000 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 63948 63948 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0

 

User Level FD file descriptors Limits

cat /etc/security/limits.conf

#@student – maxlogins 4
nginx soft nofile 100000
nginx hard nofile 300000
www-data soft nofile 200000
www-data hard nofile 500000
mysql soft nofile 30000
mysql hard nofile 50000

* soft nofile 60000  (any user)
* hard nofile 60000
* soft core 0

root limits comments

nano /etc/security/limits.conf

After you have changed the ulimit value, restart the computer, or below.

reload etc security limits without restart

pam_limits.so in /etc/pam.d/common-session

set dynamically user limits  with ulimits command

ulimit -n 100000

System Max Open Files Limit

also check sysctl,conf
to know system wide limits

# Controls the maximum number of open files in oeratng system
fs.file-max=65536

root@srv443573:~# cat /proc/sys/fs/file-max
500000

ideal should be equal to 10% of memory.

nginx connection dropped error

 

error open socket #3442 left in connection 3412

alert open socket ## left in connection # happens when a socket is left open after a nginx process thread is shutdown. There is a debugging note about this on the nginx wiki. This type of problem can exhaust your available TCP sockets, preventing new connections. Make sure your processes running under nginx are closing their sockets properly, if you believe they are already supposed to be doing that then something is interrupting the proper closing of sockets.

 

solution 1:

timeout setting should be equal in php-fpm and nginx fascgi timeout

 

Warning: The unit file, source configuration file or drop-ins of nginx.service changed on disk. Run ‘systemctl daemon-reload’ to reload units.
root@instance-1:~# systemctl daemon-reload
root@instance-1:~# tail -f /var/log/nginx/error.log
2023/09/01 07:30:58 [alert] 489857#489857: *161898 open socket #6669 left in connection 9913
2023/09/01 07:30:58 [alert] 489857#489857: *141616 open socket #50 left in connection 9923
2023/09/01 07:30:58 [alert] 489857#489857: *162218 open socket #971 left in connection 9924
2023/09/01 07:30:58 [alert] 489857#489857: *135246 open socket #4006 left in connection 9942
2023/09/01 07:30:58 [alert] 489857#489857: *150263 open socket #3764 left in connection 9950
2023/09/01 07:30:58 [alert] 489857#489857: *148091 open socket #1065 left in connection 9960
2023/09/01 07:30:58 [alert] 489857#489857: *158398 open socket #1459 left in connection 9966
2023/09/01 07:30:58 [alert] 489857#489857: *131779 open socket #6085 left in connection 9977
2023/09/01 07:30:58 [alert] 489857#489857: *146189 open socket #1021 left in connection 9983
2023/09/01 07:30:58 [alert] 489857#489857: aborting

 

also ngincx worker process high cpu

 

 

soltion increaeing rmlimt no files

 

in nginx.conf not worked

 

worker_rlimit_nofile 4096;

^[[A^[[A^[[A^[[A^C

a. find the nginx process ID

$ ps aux -P | head -1
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root@instance-1:~# ps aux |grep nginx
root 491113 0.0 0.0 261488 1844 ? Ss 07:30 0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 491114 19.7 0.4 325600 70908 ? S 07:30 1:08 nginx: worker process
www-data 491115 38.6 0.8 391384 137060 ? S 07:30 2:14 nginx: worker process
www-data 491116 2.8 0.1 274932 20188 ? S 07:30 0:09 nginx: worker process
www-data 491117 8.5 0.2 289024 34456 ? S 07:30 0:29 nginx: worker process
www-data 491118 0.0 0.0 261636 3252 ? S 07:30 0:00 nginx: cache manager process
root 492266 0.0 0.0 5132 700 pts/0 S+ 07:36 0:00 grep nginx

cat /proc/{PID}/limits

cat /proc/493972/limits

cat /proc/518/limits