Docker – After disabling swap memory for Docker container, but I can still see lots of swap used in the process, why

docker

I am running some pods (Solr & Zookeeper) on AWS EC2.

Though k8s does not promote swap by default, I still enabled it by setting –fail-swap-on to false cause I want my system to be able to use swap. And then I found some strange behaviors on swap usage which I hope you could help shed some light on 🙂

According to Docker doc, we can restrict Docker containers from using swap memory by doing the following.

If –memory-swap is set to the same value as –memory, and –memory is
set to a positive integer, the container does not have access to swap.
See Prevent a container from using swap.

With docker inspect, I can see:

  • MemorySwappiness is set to null, so it just uses the params set on
    the OS.

    Memory equals MemorySwap, so no swap memory could be used.

    $ sudo docker inspect 30309d07aa95 | grep Mem
    "Memory": 1073741824,
    "CpusetMems": "",
    "KernelMemory": 0,
    "MemoryReservation": 0,
    "MemorySwap": 1073741824,
    "MemorySwappiness": null,

However, after running my container for a while, I can see my Solr is using a lot of swap.

$ grep --color VmSwap /proc/9549/status
VmSwap:   261812 kB

and meanwhile, about 517820 kB physical memory is used.

Does anyone know why this doesn't work as described in Docker's document?

Other Info

  • OS: Amazon Linux 2 AMI (HVM), SSD Volume Type – ami-0cbc6aae997c6538a (64-bit x86) / ami-09172771b47695ce2

  • Docker version

    Client:
    Version: 18.06.3-ce
    API version: 1.38
    Go version: go1.10.3
    Git commit: d7080c1
    Built: Wed Feb 20 02:26:51 2019
    OS/Arch: linux/amd64
    Experimental: false

    Server:
    Engine:
    Version: 18.06.3-ce
    API version: 1.38 (minimum version 1.12)
    Go version: go1.10.3
    Git commit: d7080c1
    Built: Wed Feb 20 02:28:17 2019
    OS/Arch: linux/amd64
    Experimental: false

  • Some system settings about memory

    vm.overcommit_kbytes = 0

    vm.overcommit_memory = 1

    vm.overcommit_ratio = 50

    vm.swappiness = 60

    vm.vfs_cache_pressure = 100

Best Answer

Because not all swap is created equal. VmSwap accounts for things other than what you might be thinking of as "swap", including dirty pages from memory-mapped files -- and IIRC, Solr does like itself some mmap'd files. /proc/<pid>/smaps should give more detail as to exactly what's going on.

Related Topic