Troubleshoot Docker Failed to Start on Linux

dockerlinuxsystemdulimit

Today I have an issue with docker, It stopped and I cannot start it

 systemctl status docker

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2020-01-28 09:59:03 +0330; 1min 51s ago
     Docs: https://docs.docker.com
  Process: 2327 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=205/LIMITS)
 Main PID: 2327 (code=exited, status=205/LIMITS)

Jan 28 09:59:03 example.com systemd[1]: Starting Docker Application Container Engine...
Jan 28 09:59:03 example.com systemd[1]: docker.service: main process exited, code=exited, status=205/LIMITS
Jan 28 09:59:03 example.com systemd[1]: Stopped Docker Application Container Engine.
Jan 28 09:59:03 example.com systemd[1]: Unit docker.service entered failed state.
Jan 28 09:59:03 example.com systemd[1]: docker.service failed.

I see that I have status=205/LIMITS I remember that 3 day ago I changed ulimit configs

# /etc/security/limits.d/custom.conf

root soft nofile 1000000
root hard nofile 1000000
* soft nofile 1000000
* hard nofile 1000000

# /etc/sysctl.conf

fs.file-max = 1000000
fs.nr_open = 1000000

I increased fd to 1 million , after undo changes docker started successfully

What is problem here exactly??

Best Answer

status=205/LIMITS mean one of these limit hits LimitCPU=, LimitFSIZE=, LimitDATA=, LimitSTACK=, LimitCORE=, LimitRSS=, LimitNOFILE=, LimitAS=, LimitNPROC=, LimitMEMLOCK=, LimitLOCKS=, LimitSIGPENDING=, LimitMSGQUEUE=, LimitNICE=, LimitRTPRIO=, LimitRTTIME=

I am not sure about your OS version but giving one example for Ubuntu 16 create this file /etc/systemd/system/docker.service.d/customexec.conf if doesn't exists with below content.

[Service]
LimitNOFILE=1000000

Do reload systemctl daemon-reload

The docker package set a limit higher than our new limit, so the process was failing to start. So you can try below if above option doesn't work for you.

sysctl -w fs.nr_open=1048576

Related Topic