Difference Between Buffers and Cache in Linux Free Command

linuxmemoryvirtual-memory

This is an old question that I've seen from time to time. My understanding of it is rather limited (having read about the differences a long time ago, but the factoid(s) involved never really stuck).

As I understand it,

Buffers

Are used by programs with active I/O operations, i.e. data waiting to be written to disk
Cache

Is the result of completed I/O operations, i.e. buffers that have been flushed or data read from disk to satisfy a request.

Can I get a clear explanation for posterity?

Best Answer

The "cached" total will also include some other memory allocations, such as any tmpfs filesytems. To see this in effect try:

mkdir t
mount -t tmpfs none t
dd if=/dev/zero of=t/zero.file bs=10240 count=10240
sync; echo 3 > /proc/sys/vm/drop_caches; free -m
umount t
sync; echo 3 > /proc/sys/vm/drop_caches; free -m

and you will see the "cache" value drop by the 100Mb that you copied to the ram-based filesystem (assuming there was enough free RAM, you might find some of it ended up in swap if the machine is already over-committed in terms of memory use). The "sync; echo 3 > /proc/sys/vm/drop_caches" before each call to free should write anything pending in all write buffers (the sync) and clear all cached/buffered disk blocks from memory so free will only be reading other allocations in the "cached" value.

The RAM used by virtual machines (such as those running under VMWare) may also be counted in free's "cached" value, as will RAM used by currently open memory-mapped files (this will vary depending on the hypervisor/version you are using and possibly between kernel versions too).

So it isn't as simple as "buffers counts pending file/network writes and cached counts recently read/written blocks held in RAM to save future physical reads", though for most purposes this simpler description will do.

Related Solutions

Difference Between ‘unlink’ and ‘rm’ in Linux

Both are a wrapper to the same fundamental function which is an unlink() system call.

To weigh up the differences between the userland utilies.

rm(1):

More options.
More feedback.
Sanity checking.
A bit slower for single calls as a result of the above.
Can be called with multiple arguments at the same time.

unlink(1):

Less sanity checking.
Unable to delete directories.
Unable to recurse.
Can only take one argument at a time.
Marginally leaner for single calls due to it's simplicity.
Slower when compared with giving rm(1) multiple arguments.

You could demonstrate the difference with:

$ touch $(seq 1 100)
$ unlink $(seq 1 100)
unlink: extra operand `2'

$ touch $(seq 1 100)
$ time rm $(seq 1 100)

real    0m0.048s
user    0m0.004s
sys     0m0.008s

$ touch $(seq 1 100)
$ time for i in $(seq 1 100); do rm $i; done

real    0m0.207s
user    0m0.044s
sys     0m0.112s

$ touch $(seq 1 100)
$ time for i in $(seq 1 100); do unlink $i; done

real    0m0.167s
user    0m0.048s
sys     0m0.120s

If however we're talking about an unadulterated call to the system unlink(2) function, which I now realise is probably not what you're accounting for.

You can perform a system unlink() on directories and files alike. But if the directory is a parent to other directories and files, then the link to that parent would be removed, but the children would be left dangling. Which is less than ideal.

Edit:

Sorry, clarified the difference between unlink(1) and unlink(2). Semantics are still going to differ between platform.

Meaning of Buffers/Cache Line in Free Command Output – Linux Memory Usage

Meaning of the values

The first line means:

total: Your total (physical) RAM (excluding a small bit that the kernel permanently reserves for itself at startup); that's why it shows ca. 11.7 GiB , and not 12 GiB, which you probably have.
used: memory in use by the OS.
free: memory not in use.
shared / buffers / cached: This shows memory usage for specific purposes, these values are included in the value for used.

The second line gives first line values adjusted. It gives the original value for used minus the sum buffers+cached and the original value for free plus the sum buffers+cached, hence its title. These new values are often more meaningful than those of first line.

The last line (Swap:) gives information about swap space usage (i.e. memory contents that have been temporarily moved to disk).

Background

To actually understand what the numbers mean, you need a bit of background about the virtual memory (VM) subsystem in Linux. Just a short version: Linux (like most modern OS) will always try to use free RAM for caching stuff, so Mem: free will almost always be very low. Therefore the line -/+ buffers/cache: is shown, because it shows how much memory is free when ignoring caches; caches will be freed automatically if memory gets scarce, so they do not really matter.

A Linux system is really low on memory if the free value in -/+ buffers/cache: line gets low.

For more details about the meaning of the numbers, see e.g. the questions:

Changes in procps 3.3.10

Note that the output of free was changed in procps 3.3.10 (released in 2014). The columns reported are now "total", "used", "free", "shared", "buff/cache", "available", and the meanings of some of the values changed, mainly to better account for the Linux kernel's slab cache.

See Debian Bug report #565518 for the motivation, and What do the changes in free output from 14.04 to 16.04 mean? for more details information.