Linux – CentOS server. What does it mean when the total used RAM does not equal the sum of RES

centosjavalinuxmemorytop

I'm having a problem with a virtual hosted server running CentOS. In the past month a process (java based) that had been running fine started having problems getting memory when the JVM was started.

One strange thing I've noticed is that when I start the process, the PID says it is using 470mb of RAM while the 'used' memory immediately drops by over a 1GB. If I run 'top', the total RES used across all processes falls short of the 'used' listed at the top by almost 700mb.

The support person says this means I have a memory leak with my process. I don't know what to believe because I would expect a memory leak to simply waste the memory that the process is allocated- not to consume additional memory that doesn't show up using 'top'.

I'm a developer and not a server guy so I'm appealing to the experts. To me, if the total RES memory doesn't add up to the total 'used' it indicates that something is wrong with my virtual server set-up. Should I suspect a memory leaking java process in this case?

If I use free before:

             total       used       free     shared    buffers     cached  
Mem:       2097152     149264    1947888          0          0          0  
-/+ buffers/cache:     149264    1947888  
Swap:            0          0          0

free after:

             total       used       free     shared    buffers     cached  
Mem:       2097152    1094116    1003036          0          0          0  
-/+ buffers/cache:    1094116    1003036  
Swap:            0          0          0

So it looks as though the process is using (or causing to be used) nearly 1GB of RAM. Since the process (based on top is only using 470mb, does that mean that the kernel is all of a sudden using an additional 500mb?

Edit: Added output of cat proc/meminfo – with the java app running:

MemTotal:      2097152 kB
MemFree:       1112672 kB
Buffers:             0 kB
Cached:              0 kB
SwapCached:          0 kB
Active:              0 kB
Inactive:            0 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2097152 kB
LowFree:       1112672 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:           0 kB
Mapped:              0 kB
Slab:                0 kB
PageTables:          0 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:         0 kB
Committed_AS:        0 kB
VmallocTotal:        0 kB
VmallocUsed:         0 kB
VmallocChunk:        0 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Best Answer

The "used" field in top reflects the total amount of RAM being used, including that by processes, file buffers, and cache. But the RES number shows only the amount of RAM being used by a process, exclusive of the file buffers.

To really see how much RAM is being used without the buffers, use the free command and look at the -/+ buffers/cache line. The number in the "used" column shows how much RAM is actually being used by processes.

Here's an example:

$ free
             total       used       free     shared    buffers     cached
Mem:       2057196    1812352     244844          0     344768     833660
-/+ buffers/cache:     633924    1423272
Swap:      2097148          0    2097148

This output shows that the system has 2GB of RAM, of which 1.8GB is being used. However, 344MB of that is used by buffers and 833MB is used by cache. Subtracting that out leaves only 633MB being used by processes and kernel.

Run the free command before and after launching your Java app to get a better idea of what's going on.

Edit: removed references to the kernel memory, which is not reported by these tools.

Related Topic