Docker – Fix Docker-ce Reinstallation Hanging in Systemd Service Enable

dockersystemd

I know my question may be too generic but please bear with me for a moment:

I use ubuntu 18.04 LTS, and had docker-ce installed (using the official install procedure described in https://docs.docker.com/engine/install/ubuntu/) on a modern thinkpad. Nothing fancy about my setup.

A day or two ago, my system hanged when trying to poweroff my machine. Then, when I tried to use docker, the daemon was unresponsive, just hanged. systemd unit was loaded, but had not started. I assumed at that point that the poweroff hang was due to this docker daemon issue.

So I tried to purge and reinstall docker from the system.
sudo apt-get purge docker-ce docker-ce-cli containerd.io
Even this command hanged.

So I tried to manually unlock dpkg by:

sudo systemctl disable docker
pgrep -a apt
sudo kill -9 <apt process>
sudo rm /var/lib/apt/lists/lock
sudo rm /var/cache/apt/archives/lock
sudo rm /var/lib/dpkg/lock

also deleted all docker and containerd related files + disabled and removed systemd unit:

sudo systemctl disable docker
sudo rm -rf /etc/docker
sudo rm -rf /run/containerd/containerd.sock
sudo rm -rf /usr/bin/dockerd
sudo rm -rf /lib/systemd/system/docker.service
sudo rm -rf /var/lib/docker-engine
sudo rm -rf /var/lib/docker
sudo rm -rf /var/run/docker/
sudo rm -rf /var/run/docker.pid
sudo rm -rf /var/run/docker.sock
sudo rm /etc/apt/sources.list.d/docker.list
sudo rm -rf /var/lib/containerd
sudo rm -rf /opt/containerd
sudo systemctl daemon-reload

and restarted which then allowed me to finally run the purge command successfully:
sudo apt-get purge docker-ce docker-ce-cli containerd.io

Then when I try to reinstall docker (using again the official docs method, stable channel), it hangs at enabling the systemd service:

...
Preparing to unpack .../5-docker-ce_5%3a19.03.8~3-0~ubuntu-bionic_amd64.deb ...
Unpacking docker-ce (5:19.03.8~3-0~ubuntu-bionic) ...
Setting up aufs-tools (1:4.9+20170918-1ubuntu1) ...
Setting up containerd.io (1.2.13-1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.
Setting up cgroupfs-mount (1.4) ...
Setting up docker-ce-cli (5:19.03.8~3-0~ubuntu-bionic) ...
Setting up pigz (2.4-1) ...
Setting up docker-ce (5:19.03.8~3-0~ubuntu-bionic) ...
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.

which I need to suspend (ctrl+z) and then clean up everything again as described above.

Here's also a screenshot on how the systemd service looks while installation is stuck: enter image description here

I have also tried to do apt clean and/or apt autoclean then reboot and try again, but nothing has worked so far, it still hangs when installing.

Best Answer

I am not sure what kernel version you are running, but I noted this exact problem today and noticed that running docker ps just hanged the computer and like you I couldn’t deinstall or reinstall without killing the processes and manually remove files (when trying to reinstall the progress bar even stopped at 94% - exactly as in your screenshot).

What I found was that on all servers this problem arose, they were running kernel 5.3.0-52-generic (from the linux-virtual-hwe-18.04 package).

After downgrading the kernel by installing the linux-virtual package with kernel 4.15.0.100.90 and rebooting with that kernel, the docker installation/uninstallation was successful.

I also tried installing kernel 5.6.7 from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6.7/ on the same server and reinstalling docker with success. So, my guess is that it is something related to the kernel. And my solution was just to install an older or newer kernel.

Related Topic