mac-osx-server – How to Find Phantom Storage Usage on Mac OS X Server

disk-space-utilizationmac-osx-server

I've got a Mac OS X 10.6.4 Snow Leopard Server file server (AFP) which has been running out of storage space on the boot volume for the past few weeks. It takes about two days for the remaining 42GB on the 80GB boot volume to be eaten up, even though a sudo du -chsx -I dev / still shows only 29GB used.

I've run into this in the past w/a Linux server who's Apache logs were deleted after N days, but the log was still kept open by Apache, causing the storage to not be freed. I had been able to track it down relatively easily in that case w/a sudo lsof, but I'm not easily finding the culprit in this case (being a file server, there are a ton of open files and sockets). How can I sort lsof output by file size (and show sized in a human friendly format) so I can find the culprit?

The server has 3GB of RAM. After being up 4hrs, Activity Monitor shows 700MB free, 1.5GB inactive, and 200GB of VM. mds has the largest VM usage at 1.8GB, AppleFileServer in 2nd place w/500MB, and everything else is using 10MB-75MB of VM. That said, /private/var/vm is only 128MB.

Rebooting the system clears the issue, hence my belief that it's free storage space that's still being held open by some process or processes.

Any other hypothesis, suggestions, etc., greatly appreciated.

Best Answer

You can find the biggest open files with:

sudo lsof -s | awk '$5 == "REG"' | sort -n -r -k 7,7 | head -n 50

This will list the regular files (not pipes, sockets, etc) sort by size in descending order, and take the top 50.

You might also look at what processes have the most files open, with something like

sudo lsof | awk '$5 == "REG" {freq[$2]++ ; names[$2] = $1 ;} END {for (pid in freq) print freq[pid], names[pid], pid ; }' | sort -n -r -k 1,1