Unable to write to disk but disk isn’t full

diskmanagementfilesystemshard drive

I am using Ubuntu 12.04 and can't write to any file, even as root, or do any other operation that requires writing. Neither can any process that needs to write, so they're all failing. df says I've got plenty of room:

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       30G   14G   15G  48% /
udev            984M  4.0K  984M   1% /dev
tmpfs           399M  668K  399M   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            997M     0  997M   0% /run/shm

All of the results I find for "can't write to disk" are about legitimately full disks. I don't even know where to start here. The problem appeared out of nowhere this morning.

PHP's last log entry is:

failed: No space left on device (28)

Vim says:

Unable to open (file) for writing

Other applications give similar errors.

After deleting ~1gb just to be sure, the problem remains. I've also rebooted.

df -i says

Filesystem      Inodes   IUsed  IFree IUse% Mounted on
/dev/xvda1     1966080 1966080      0  100% /
udev            251890     378 251512    1% /dev
tmpfs           255153     296 254857    1% /run
none            255153       4 255149    1% /run/lock
none            255153       1 255152    1% /run/shm

Best Answer

You are out of inodes. It's likely that you have a directory somewhere with many very small files.