Nginx – PHP-FPM and continuos “exited on signal 11”

dumpnginxphp-fpm

I've a Hp web server with Xeon E5504 2ghz and 8gb ram, with freebsd 8.2-p9
amd64, nginx 1.2.1, PHP 5.3.14, mysql 5.5.25, apc, memcached and other
package installed using freebsd ports. (conf file at the end of that message)

My trouble are simple, I've a lot of "exited on signal 11" in
/var/log/messages and when php-fpm child die I've "502 gateway error" on my
website, that's really boring because I've a lot of that errors in a day,
in that php-fpm conf I've forced it to an heavy configuration with a 150 of
php-fpm's child in static mode, and pm.max_requests = 10000000, but in my
php-fpm log I found for each child:

[19-Jul-2012 18:58:14.666913] NOTICE: pid 84563, fpm_children_make(), line
421: [pool www] child 84717 started
[19-Jul-2012 18:58:14.666984] DEBUG: pid 84563, fpm_event_loop(), line 409:
event module triggered 1 events
[19-Jul-2012 18:58:17.403217] DEBUG: pid 84563, fpm_event_loop(), line 409:
event module triggered 2 events
[19-Jul-2012 18:58:17.407442] DEBUG: pid 84563, fpm_got_signal(), line 72:
received SIGCHLD
[19-Jul-2012 18:58:17.407552] WARNING: pid 84563, fpm_children_bury(), line
252: [pool www] child 84563 exited on signal 11 (SIGSEGV) after 39.849444
seconds from start

and in /var/log/message for the same pid:

Jul 19 18:58:14 web1 kernel: pid 84563 (php-fpm), uid 1001: exited on signal 11

to be sure that isn't a configuration files (php.ini and php-fpm.conf) troubles, I restored it with the original one that I found in the archive, but nothing, than I tried to use spawn-fcgi, but same trouble!

Than I tried to generate a php-fpm.core running "gdb /usr/local/sbin/php-fpm", when the child exit with "exited on signal 11" php-fpm main process exit with "exited on signal 5 (core dumped)" (that only when I run it throw GDB!), here logs and gdb backtrace results:

As you can see in last link, that are last rows:

#659 0x0000000801534de0 in ?? () from /lib/libc.so.7
#660 0x0000000000000001 in ?? ()
#661 0x00007fffffffec08 in ?? ()
#662 0x000000000000000f in ?? ()
Cannot access memory at address 0x800000000000

Any ideas?

Thanks!

/var/log/messages:

Jul 24 17:58:41 web1 kernel: pid 3887 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:42 web1 kernel: pid 3998 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:42 web1 kernel: pid 3895 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:42 web1 kernel: pid 3892 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:43 web1 kernel: pid 3889 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:43 web1 kernel: pid 3898 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:44 web1 kernel: pid 3886 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:44 web1 kernel: pid 3999 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:45 web1 kernel: pid 3896 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:45 web1 kernel: pid 4000 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:45 web1 kernel: pid 3893 (php-fpm), uid 1001: exited on signal 11
Jul 24 17:58:55 web1 kernel: pid 3885 (php-fpm), uid 0: exited on signal 5 (core dumped)

php.ini (only difference from original file)

[PHP]
memory_limit = 128M

apc.enabled=1
apc.shm_size=128M
apc.ttl=0
apc.mmap_file_mask=/tmp/apc/apc.XXXXXX

error_reporting = E_ALL & ~E_DEPRECATED & ~E_NOTICE
display_errors = On
display_startup_errors = On
track_errors = On
html_errors = On

upload_max_filesize = 4M

[Date]
date.timezone = Europe/Rome

[browscap]
browscap = /home/serverweb/etc/php/browscap.ini

php-fpm.conf (only difference from original file)

[global]
error_log = /home/serverweb/log/php-fpm/error.log
log_level = debug

emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 10s

process.max = 0

[www]
user = web
group = web

listen = 127.0.0.1:9999
listen.allowed_clients = 127.0.0.1

pm = static
pm.max_children = 16

pm.max_requests = 10000000

slowlog = /home/serverweb/log/php-fpm/$pool.log.slow

Best Answer

PHP 5.3.14 has a serious bug that leads to segfault in php-fpm. It was fixed in 5.3.15 and 5.4.5.