CentOS default installation gave about 60% of memory space to the tmpfs partition – Is this an issue

centostmpfs

I installed a CentOS server which will be used for xen hypervisor. The server has two Intel(R) Xeon(R) CPU E5620 and 148G memory. The OS was installed on a 120G SSD drive. After the installation, I found that the tmpfs partition occupied about 60% of the drive. Even though I don't need much space for the OS, will there be any problem with 71G tmp partition? Thanks for any comment.

[root@cloud ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on 
/dev/mapper/VolGroup00-LogVol00
                       55G  1.1G   51G   3% /
/dev/sda1              99M   13M   82M  14% /boot
tmpfs                  71G     0   71G   0% /dev/shm

EDIT:

[root@cloud ~]# fdisk -l

Disk /dev/sda: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14       14593   117113850   8e  Linux LVM

[root@cloud ~]# vgdisplay
  --- Volume group ---
  VG Name               VolGroup00
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               111.69 GB
  PE Size               32.00 MB
  Total PE              3574
  Alloc PE / Size       3573 / 111.66 GB
  Free  PE / Size       1 / 32.00 MB
  VG UUID               37dcLa-zJv6-i2j3-3Xvk-keXO-HJDh-UDgQ8l

  [root@cloud ~]# lvdisplay
    --- Logical volume ---
    LV Name                /dev/VolGroup00/LogVol00
    VG Name                VolGroup00
    LV UUID                Kj5LoJ-kCOq-MWhV-Wc3s-kds1-AsNE-cs9Ibh
    LV Write Access        read/write
    LV Status              available
    # open                 1
    LV Size                56.50 GB
    Current LE             1808
    Segments               1
    Allocation             inherit
    Read ahead sectors     auto
    - currently set to     256
    Block device           253:0

    --- Logical volume ---
    LV Name                /dev/VolGroup00/LogVol01
    VG Name                VolGroup00
    LV UUID                4h7uvI-as7G-OTWE-ufea-CqqO-ws76-WolJCt
    LV Write Access        read/write
    LV Status              available
    # open                 1
    LV Size                55.16 GB
    Current LE             1765
    Segments               1
    Allocation             inherit
    Read ahead sectors     auto
    - currently set to     256
    Block device           253:1

Best Answer

The tmpfs partition you see listed is the /dev/shm shared memory "ramdisk". It's equal to 50% of your physical RAM by default. This is unrelated to your OS drive (/dev/mapper/VolGroup00-LogVol00) mounted at /. So there's nothing to worry about here. Your system drive is 55G at present, and you're only using 1.1G of it.

See a thorough description at: http://www.cyberciti.biz/tips/what-is-devshm-and-its-practical-usage.html

The Linux kernel

At boot time, Linux sets default limits to the init (or systemd) process, which are then inherited by all the other (children) processes. To see these limits: cat /proc/1/limits.

For example, the kernel default for maximum number of file descriptors (ulimit -n) was 1024/1024 (soft, hard), and has been raised to 1024/4096 in Linux 2.6.39.

The default maximum number of processes you're talking about is limited to approximately:

Total RAM in kB / 128

for x86 architectures (at least), but distributions sometimes change default kernel values, so check your kernel source code for kernel/fork.c, fork_init(). The "number of processes" limit is called RLIMIT_NPROC there.

PAM

Usually, to ensure user authentication at login, PAM is used along with some modules (see /etc/pam.d/login).

On Debian, the PAM module responsible for setting limits is here : /lib/security/pam_limits.so.

This library will read its configuration from /etc/security/limits.conf and /etc/security/limits.d/*.conf, but even if those files are empty, pam_limits.so might use hardcoded values that you can check within the source code.

For example, on Debian, the library has been patched so that by default, the maximum number of processes (nproc) is unlimited, and the maximum number of files (nofile) is 1024/1024:

  case RLIMIT_NOFILE:
      pl->limits[i].limit.rlim_cur = 1024;
      pl->limits[i].limit.rlim_max = 1024;

So, check your CentOS' PAM module source code (look for RLIMIT_NPROC).

However, please note that many processes will not go through PAM (usually, if they are not launched by a logged in user, like daemons and maybe cron jobs).

systemd

Nowadays, systemd is widely used, it replaces init and can also configure specific limits values, especially to the processes/daemons it manages and creates itself.

Some limits it uses by default can be manually configured in /etc/systemd/system.conf. There is more information available in the documentation.

Qemu don’t start quest with disk image on tmpfs

Apparently, if you set cache=NONE for a disk image file on any host filesystem that does not support Direct IO, Virt-Manager will currently give a not very helpful error message saying "Something something... Invalid Argument" and refuse to start the Guest VM.

One example of such a filesystem --that does not support Direct IO-- is the tmpfs. Another such filesystem where maybe you can optionally disable Direct IO is GlusterFS (which I had not heard of before I also hit the same problem as Sergei did and was researching the error.) For tmpfs, not supporting Direct IO seems to be a technical limitation or design conflict at present and I don't know whether it will/can be rectified in future.

In my case, I had put a 3.6 GB Ramdisk for a Win7Pro Guest VM running on CentOS7 and had set cache=NONE for the ramdisk in Virt-Manager. The other options that the tmpfs img used were virtio and raw. The VM would refuse to start with the same/similar error saying "... Invalid Argument".

For technical details and notes directly from the Redhat Engineer and Developer that coded the patch for the feature cache=NONE & maintains (?) Virt-Manager (Daniel Berrange), see the discussion at the following link:

https://bugs.launchpad.net/nova/+bug/959637

To quote Daniel from the URL above: " > could not open disk image /mnt/vmstore/instances/instance-0000001a/disk: Invalid argument

probably means that the filesystem does not support Direct IO. AFAIK, all filesystems except tmpfs should support this.."

Also, Daniel continues: "- So what we'll want todo then, is to [....] do a check whether the storage volume supports Direct IO. If it does, then use cache=none, otherwise fallback to cache=writethrough which does not use Direct I/O, but is still crash safe."

In my case, I was able to verify that setting cache=NONE for the tmpfs img file was NOT starting the VM and showed the error as discussed. It WAS able to successfully start the VM if cache was set to either "Default" or explicitly "Write-through". There is no sense in going with "Write-back" since this filesystem both expendable and was anyways entirely in RAM, so obviously I did not use Write-back.

Hope that helps!

Best Answer

Related Solutions

Linux – where are the default ulimit values set? (linux, centos)

The Linux kernel

PAM

systemd

Qemu don’t start quest with disk image on tmpfs

Related Topic