Linux – Ubuntu’s garbage collection cron job for PHP sessions takes 25 minutes to run, why

linuxperformancePHPUbuntu

Ubuntu has a cron job set up which looks for and deletes old PHP sessions:

# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -x /usr/lib/php5/maxlifetime ] \
   && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 \
   -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) ! -execdir \
   fuser -s {} 2> /dev/null \; -delete

My problem is that this process is taking a very long time to run, with lots of disk IO. Here's my CPU usage graph:

CPU usage graph

The cleanup running is represented by the teal spikes. At the beginning of the period, PHP's cleanup jobs were scheduled at the default 09 and 39 minutes times. At 15:00 I removed the 39 minute time from cron, so a cleanup job twice the size runs half as often (you can see the peaks get twice as wide and half as frequent).

Here are the corresponding graphs for IO time:

IO time

And disk operations:

Disk operations

At the peak where there were about 14,000 sessions active, the cleanup can be seen to run for a full 25 minutes, apparently using 100% of one core of the CPU and what seems to be 100% of the disk IO for the entire period. Why is it so resource intensive? An ls of the session directory /var/lib/php5 takes just a fraction of a second. So why does it take a full 25 minutes to trim old sessions? Is there anything I can do to speed this up?

The filesystem for this device is currently ext4, running on Ubuntu Precise 12.04 64-bit.

EDIT: I suspect that the load is due to the unusual process "fuser" (since I expect a simple rm to be a damn sight faster than the performance I'm seeing). I'm going to remove the use of fuser and see what happens.

Best Answer

Removing of fuser should help. This job runs a fuser command (check if a file is currently opened) for every session file found, which can easily take several minutes on a busy system with 14k sessions. This was a Debian bug (Ubuntu is based on Debian).

Instead of memcached you can also try to use tmpfs (a filesystem in memory) for session files. Like memcached this would invalidate sessions on reboot (this can be worked around by backing up this directory somewhere in shutdown script and restoring in startup script), but will be much easier to setup. But it will not help with fuser problem.

Related Topic