php fpm pool manager configuration settings based on server spike high cpu wordpress

Table of Contents

php fpm pool options (must understand this part)

Php fpm pool manager Static vs dynamic vs ondemand

Understanding Php resource limits for wordpress

memory_limit = 128M
upload_max_size = 8M
post_max_size = 8M
upload_max_filesize = 8M
max_execution_time = 180
max_input_time = 500

follow errors. at var/log/php_error.log

request terminate timeout

Deciding max Children Limit

Present max childrens are 25 frequently getting max children reached errors.

So Visitor >> nginx>> php-fpm >> mysql>>

1 visitor 1 connection = 1 process per second.

let’s say 50%-75% of visitors at caching / static files.

25 connection /second = 25 visitors/second

Average memory usage

128MB is max but average maybe 40MB

total RAM 8046/40= 201 Child process But its a theory.

added 60 by assuming average memory 40MB for process and reduced max execution time 5 mins to 3 mins.

; static –

a fixed number (pm.max_children) of child processes;

; dynamic –

the number of child processes are set dynamically based on the

; following directives. With this process management, there will be

; always at least 1 children.

; pm.max_children – the maximum number of children that can

; be alive at the same time.

; pm.start_servers – the number of children created on startup.

; pm.min_spare_servers – the minimum number of children in ‘idle’

; state (waiting to process). If the number

; of ‘idle’ processes is less than this

; number then some children will be created.

; pm.max_spare_servers – the maximum number of children in ‘idle’

; state (waiting to process). If the number

; of ‘idle’ processes is greater than this

; number then some children will be killed.

;

ondemand –

no children are created at startup. Children will be forked when

; new requests will connect. The following parameter are used:

; pm.max_children – the maximum number of children that

; can be alive at the same time.

; pm.process_idle_timeout – The number of seconds after which

; an idle process will be killed.

pm = Static, Dynamic, on demand

NOTE: apache mpm module configuration similar to this.

php fpm calculator / calculation process

php fpm Static pool manager

pm.max_children = 5

php – fpm pool manager Dynamic

Default www.conf file

pm.max_children = 5

pm.start_servers = 2

pm.min_spare_servers = 1

pm.max_spare_servers = 3 (1+2)

pm.starts servers

min_spare_servers + (max_spare_servers – min_spare_servers) / 2

1+(3-1)/2=2

2/2=1 +1 =2.

//note less than min spare

php fpm pool ondemand settings best for beginners low memory consumption;

pm.process_idle_timeout = 10s;

pm.max_children = 60 // just increase it if pool manager busy.

limit upto available memory.

ex: max memory php limit = 128M*60=7260 (max possible usage php alone only incase 0f brute force attacks)

memory leaks safety

; Default Value: 0

;pm.max_requests = 500

Process has to dire or respawn after handling above request this way we avoid memory leaks.

respawning frequently also utilizes more memory so set it 10K.

ondemand 10s limit to kill the process upon idle.

; Set open file descriptor rlimit.

; Default Value: system defined value

;rlimit_files = 1024

; Set max core size rlimit.

; Possible Values: ‘unlimited’ or an integer greater or equal to 0

; Default Value: system defined value

;rlimit_core = 0

pm max children reached

fixing php-fpm pool high cpu usage

available memory in LEMP stack

4GB

1GB for MySQL, 512MB for linux, 1GB for nginx

2.5GB (1.5GB,

php max memory limit (128MB or 256MB)

max execution time 60s or 300s 1 minute or 5 minutes

slow query log threshold 1s.

starart severs +idle servers

all are max child processes. handles only one connection per second.

30 connection per second.

30*128MB=3048M (but all requests not consumes 128M)

4MB or 8MB average.

monitor slow query logs.

static: pool manager

when the limit is 30.

30*128M = 3GB Max dedicated needed.

minimum memory ==?

each process occupies memory to handle requests even though no request there

dynamic for mid range request

On demand for (free memory)

dynamic is better holds 3 max spare & total 5 spare

we have increase upto 10. If you get message like

max children reached.

php fpm max requests

by deafault 500

Applicable only when using: dynamic

increase upto 5000

Why: process has to kil upon completing 500 or specified requested to avoid memory leakage.

Determining Average memory usage per process

root@instance-1:~# ps –no-headers -o “rss,cmd” -C php-fpm7.4 | awk ‘{ sum+=$1 } END { printf (“%d%s\n”, sum/NR/1024,”M”) }’
283M

root@instance-1:~# systemctl restart php7.4-fpm
root@instance-1:~# ps –no-headers -o “rss,cmd” -C php-fpm7.4 | awk ‘{ sum+=$1 } END { printf (“%d%s\n”, sum/NR/1024,”M”) }’
29M

root@instance-1:~# ps –no-headers -o “rss,cmd” -C php-fpm7.4 | awk ‘{ sum+=$1 } END { printf (“%d%s\n”, sum/NR/1024,”M”) }’
96M

root@instance-1:~# ps -ef | grep ‘[f]’pm
root 216323 1 0 15:48 ? 00:00:00 php-fpm: master process (/etc/php/7.4/fpm/php-fpm.conf)
www-data 216333 216323 0 15:48 ? 00:00:01 php-fpm: pool www
www-data 216334 216323 0 15:48 ? 00:00:02 php-fpm: pool www
www-data 216335 216323 0 15:48 ? 00:00:01 php-fpm: pool www
www-data 216336 216323 0 15:48 ? 00:00:00 php-fpm: pool www
www-data 216337 216323 0 15:48 ? 00:00:00 php-fpm: pool www
www-data 216338 216323 0 15:48 ? 00:00:02 php-fpm: pool www
www-data 216339 216323 0 15:48 ? 00:00:00 php-fpm: pool www
www-data 216340 216323 0 15:48 ? 00:00:00 php-fpm: pool www

ps -ylC php-fpm –sort:rss

ps -ylC php7.3-fpm –sort:rss

RSS contains the average memory usage in kilobytes per process.

S UID PID PPID C PRI NI RSS SZ WCHAN TTY TIME CMD
S 0 24439 1 0 80 0 6364 57236 – ? 00:00:00 php-fpm
S 33 24701 24439 2 80 0 61588 63335 – ? 00:04:07 php-fpm
S 33 25319 24439 2 80 0 61620 63314 – ? 00:02:35 php-fpm

61588 = 60mb

enable php fpm slow query log

slowlog = /var/log/php-slow.log
request_slowlog_timeout = 10s

slowlog = /var/log/$pool.log.slow (no need to worry about wordpress plugin update requests)
equest_slowlog_timeout = 10s

uncomment the lines in /etc/php/7.3/fpm/pool.d/www.conf

Deciding the Slow query threshold if you updates wordpress plugins it takes times,

if you upload 8MB files its takes time.

Best php query time for wordpress is ….? (wordpress load in 1-3 secs) (php execution time time , request terminate timeout ).

Monitor the script and adjust the value.

restart or reload php fpm

root@instance-1:~#

systemctl restart php7.3-fpm

service reload php7.3-fpm

enable php from status page to better understand

Step 1: uncomment pm /status

step 2: add nginx server block

for understanding php-fpm status page here.

Monitoring Requests in with php fpm log

[16-Dec-2020 09:30:28] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 38 total children
[16-Dec-2020 09:30:29] WARNING: [pool www] server reached pm.max_children setting (40), consider raising it

Php Fpm 7.3 On dynamic Settings

pm = dynamic

pm.max_children = 60 (max child process based on memory max memory if 128*60=7680MB in upper limit)

40*128=5120MB if every request uses 128MB memory.

max children value:

pm.start_servers value

; Default Value: min_spare_servers + (max_spare_servers – min_spare_servers) / 2

8+(10-6=4)/2 = 12/2=6

pm.start_servers = 8 (4*number of cores on server = 8)

pm.min_spare_servers = 6 (2*Cpu core =4

pm.max_spare_servers = 10 (4*Cpu cores =8) // Generally idea 2x memory required for cpu cores general tasks.

pm.max_requests = 5000

upgrading

pm.start_servers = 10

pm.min_spare_servers = 12

pm.max_spare_servers = 20

4+(20-4=16)/2=10 (pm.start_servers)

;pm.process_idle_timeout = 10s; //on demand only

check php fpm syntax errors without reload or restart

php-fpm7.3 -t

php-fpm7.4-t

[18-Dec-2020 12:21:15] NOTICE: configuration file /etc/php/7.3/fpm/php-fpm.conf test is successful

Restart / reload php fpm to see the effects

service php7.3-fpm reload

systemctl reload php7.3-fpm

systemctl reload php7.4-fpm

systemctl restart php7.4-fpm

also configure php fpm status page

php fpm fool dynamic vs ondemand vs static

on demand spawns process only when required and kills ater 10s or idle timeout (only one idle process = consumes less moery ex:40MB)

dynamic starts servers =5 (5 child process started on startup 50*40MB =200 MB allocated to php regardless of handling request or not, max servers if any spike it can spare to 10child,

Static : regardless ondemand or dynamically spawing rpcoess , there are fixed amount of process ex: 40*40=1600MB allocated to memory.

Ondemand = spawning upon requests causes some delay, & killing every process after 10sce causes cpu usage high.

saves memory best for low traffic sites.

dynamic: midrange sites based on traffic spikes.

static: regardless of memory performance is the key. processes are ready handle to requests.

php.ini settings

max execution time 30s to 300s 5 minutes

same fastcgi execution time, nginx buffer

memory limit: 128MB mper script

max input time for uploads

php fpm pool high cpu wordpress cpu panel

this is due to long time running process, you can mitigate by php fpm status page

php memory limit for per process * number of process = memory & high cpu usage.

Average cpu memory usage by process

ps aux | grep php

Solution: increase workers or decrease php workers

or decrease process execution time based on environment

mitigate: ddos attacks install WAF firewall to protect layer 7,

php fpm status last request cpu time

1285.35

447.43

– the %cpu of the last request consumed (it’s always 0 if the
process is not in Idle state because CPU calculation is done when the request
processing has terminated)

check source code for better understanding – php fpm source code.

php fpm pool error

WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 1 idle, and 39 total children