Linux – Disk space not freed on ext3/raid1 after deleting


I have two 120 GB disks in a software raid1 array (i.e. mirrored), showing as /dev/md2. There's an ext3 filesystem on top of this, mounted at /.

# uname -a
Linux svnserv 2.6.26-2-amd64 #1 SMP Sun Jun 21 04:47:08 UTC 2009 x86_64 GNU/Linux

Recently, writes to the disk started failing with "write error: No space left on device". df showed that there's no more disk space. I removed some unused large files, approx. 5 GB, yet error persists.

df shows this:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md2              109G  104G     0 100% /
tmpfs                 471M     0  471M   0% /lib/init/rw
udev                   10M  100K   10M   1% /dev
tmpfs                 471M     0  471M   0% /dev/shm
/dev/md0               44M  9.8M   32M  24% /boot

Note that 104 out of 109 GB is used, yet there's 0 GB available.

I'm not running out of inodes:

# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/md2             7208960   55273 7153687    1% /

mdadm seems to be happy (both disks are active and in sync):

# mdadm --detail /dev/md2
        Version : 00.90
  Creation Time : Mon Jul 20 18:53:01 2009
     Raid Level : raid1
     Array Size : 115218112 (109.88 GiB 117.98 GB)
  Used Dev Size : 115218112 (109.88 GiB 117.98 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Thu Jul  8 14:34:08 2010
          State : active
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 1bb0fff8:38e78897:34693ac2:f3157c61
         Events : 0.71

    Number   Major   Minor   RaidDevice State
       0      22        3        0      active sync   /dev/hdc3
       1      22       67        1      active sync   /dev/hdd3

How can I make the unused(?) space available?

Best Answer

Ah, the wonders of the SO/SF/SU trilogy: this question was kind of similar, but the answers didn't solve it. There was this comment however:

Reserved space for root user. Free space is free space for the unpriviledged user, not total free space. Linux doesn't display negative free space, unlike some other UNIX systems, when you start eating up root's reserved space. – pehrs Apr 16 at 8:42

According to this article, default space reserved for root is 5% of disk size.

By issuing tune2fs -m 0 /dev/md2 , the system now reports free space correctly (this system doesn't need root's reserved space).

The problem was not related to RAID at all.