Redhat – Increasing the size available to, or changing location of, /tmp

large-dataredhattmp

I am currently in charge of a server running Red Hat and a bioinformatics webapp that deals with huge files, some of which are over 100GB when uncompressed. The act of decompressing these files is done by several different programs, all of which use the system temporary directory, /tmp. When a huge file is being decompressed, /tmp fills up and halts the operation in progress, causing downstream errors in the webapp. I have to go in and delete the problem file from /tmp.

The server's filesystem is setup like this:

/*output of df-h follows*/
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_root-LogVol01 83G   34G   45G  43% /
tmpfs                  95G     0   95G   0% /dev/shm
/dev/sda1             485M   81M  379M  18% /boot
/dev/sdb1             8.0T  6.2T  1.5T  82% /data

/*output of blkid follows*/
/dev/sda1: UUID="5f489589-0678-46c1-9f0f-e4e66c6a9e04" TYPE="ext4" 

/dev/sda2: UUID="k2rP0k-1YhK-72D9-fRrQ-BxUW-3gaC-6KF0nh" TYPE="LVM2_member" 

/dev/sdb1: UUID="4e54bee3-6450-446f-af80-70ca6268e12f" TYPE="ext4" 

/dev/mapper/vg_root-LogVol01: UUID="b6f228dc-fa6c-43c1-b88e-3b43a75e980b" TYPE="ext4" 

/dev/mapper/vg_root-LogVol00: UUID="80c6863d-c9f8-4abe-a353-a6a6818dc82d" TYPE="swap"

/*output of fstab*/
/dev/mapper/vg_root-LogVol01 /   ext4    defaults   1 1
UUID=5f489589-0678-46c1-9f0f-e4e66c6a9e04 /boot   ext4   defaults   1 2
/dev/mapper/vg_root-LogVol00 swap    swap    defaults    0 0
/dev/sdb1   /data   ext4    defaults    1 1
tmpfs        /dev/shm   tmpfs   defaults        0 0
devpts      /dev/pts    devpts  gid=5,mode=620  0 0
sysfs       /sys        sysfs   defaults        0 0
proc        /proc       proc    defaults        0 0

I know the basics of Linux, but I don't know much about filesystems. What I would like to do is enact a permanent solution to allocate more space to the temporary directory. I would be open to setting the TMPDIR environment variable to point to another filesystem with more space. Or, if it's possible, I'd be equally satisfied allocating more of the existing space to LogVol-01.

My question is how, given my current filesystem details, to permanently allocate more space to the temporary directory. As you can see by the df output, I have disk space to spare. I didn't set the server up, but I am now in charge of it and have root access, for better or worse!

Best Answer

Update:

Since you've now confirmed that /tmp is not a symbolic link and that there are no mounts at /tmp. It's basically saying that / is filling up. How do you fix that? 2 options.

  1. I'd insert another hard disk and mount it at /tmp by inserting an appropriate entry in /etc/fstab.
  2. Or as a temporary measure move /tmp to another drive like /dev/sdb1 which currently has hundreds of GB free. It's simply a matter of creating a tmp directory on /dev/sdb1

i.e.

sudo mkdir /data/tmp
sudo chmod 1777 /data/tmp
sudo rm /tmp
sudo ln -s /data/tmp /tmp

If you'd prefer to add more storage (which is a better long term solution), there are lots of tutorials on the net on how to go about adding extra drives to Linux.

How to add an extra hard drive to Linux

Original answer...

tmpfs could be the culprit. It's ram based and it'll use swap when it needs to as well. That means it'll keep growing and growing until no more ram can be used for tmpfs. You're ending up with an incomplete decompress.

What you need to do then is edit /etc/fstab and comment out (with a leading '#') the line beginning with '/tmp' and reboot or unmount /tmp (with 'sudo umount /tmp')

Looks like you've got the whole drive using LVM so it's safe to not mount a replacement for /tmp.

Edit:

Wish I could see a default fstab file from redhat, or yours. Anyhow... It's possible that /tmp doesn't exist in fstab because it could be a symbolic link.
To find out run "ls -ld /tmp". If it's a symbolic link you'll see it listed with /tmp -> /dev/shm or something.

If it is actually a link to /dev/shm or to a directory that is mounted as tmpfs, Fix that with:

sudo rm /tmp
sudo mkdir /tmp
sudo chmod 1777 /tmp

EDIT2: On second thought, it could other reasons.

  1. Either /tmp is filling up, or
  2. The decompression tools aren't working too well on your large archives. 100GB uncompressed data is probably a large archive! What tools are you using to extract the archive. What's it's format? I know some versions of tar have issues, or
  3. The filesystem type does not allow for some files to to be extracted because they exceed the filesystem limit. What type is the target filesystem? is it ext3 or 4 or something else?
  4. Or the filesystem is really just full.
Related Topic