In managing a server with 512GB of RAM I encountered a process that keeps consuming swap until it hits 100% of the swap space then stops consuming more (6GB of swap) but keeps working fine (albeit, when a request enters the process, it takes a long time (20+ min) to get to the required performance).
Even setting swappiness to 0
doesn't prevent this proces from swapping.
Swapping happens while free
shows this:
# free -h --giga
total used free shared buff/cache available
Mem: 515G 16G 2.3G 30M 497G 496G
Swap: 5.8G 1.0M 5.8G
The process in question:
# smem -s swap -t -n -k
PID User Command Swap USS PSS RSS
(...)
36776 1000 java -XX:+UseG1GC -Xms1G -X 1.6M 13.4G 13.4G 13.4G
-------------------------------------------------------------------------------
148 11 2.0M 15.1G 15.2G 15.9G
It keeps growing and growing over time (at a rate of ~20 MB / hour) until 100% swap is consumed. It might be worth mentioning it is running in a docker container but I don't know if that influences anything.
Swappiness:
# cat /proc/sys/vm/swappiness
0
I really want to disable swap completely at this point but this answer strongly recommends against it. What are my options to keep this programs' memory completely in RAM?
Best Answer
Since your question is about process inside Docker container, it is worth checking if you are not missing
vm.overcommit_memory=1
configuration as described here: Node using swap memory instead of host memory