Linux – Zombie processes still alive and working fine, but can’t be killed

linuxprocesszombie

Am I misunderstanding something, or should this not be possible?

All my daemon processes are in zombie state after I tried to stop the control service:

# ps ax | grep controller
13768 pts/11   S+     0:00 grep controller
26866 ?        Zl    18:56 [controller] <defunct>
26870 ?        Zl    18:57 [controller] <defunct>
26871 ?        Zl    18:45 [controller] <defunct>
26876 ?        Zl    13:17 [controller] <defunct>
26877 ?        Zl    10:28 [controller] <defunct>
26880 ?        Zl    18:18 [controller] <defunct>
26881 ?        Zl    12:01 [controller] <defunct>
26882 ?        Zl    18:18 [controller] <defunct>

And yet ports are still open (although netstat can't find the process name)

# netstat -tlpn | sort
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1180/sshd
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      11882/httpd
tcp        0      0 10.0.0.50:8890              0.0.0.0:*                   LISTEN      -
tcp        0      0 10.0.0.50:8891              0.0.0.0:*                   LISTEN      -
tcp        0      0 10.0.0.50:8892              0.0.0.0:*                   LISTEN      -
tcp        0      0 10.0.0.50:8896              0.0.0.0:*                   LISTEN      -
tcp        0      0 10.0.0.50:8897              0.0.0.0:*                   LISTEN      -
tcp        0      0 10.0.0.50:8900              0.0.0.0:*                   LISTEN      -

Although lsof can see the process names

# lsof -i -n -P | grep 10.0.0.50 | grep LISTEN
controlle 26866  devuser   82u  IPv4    323641      0t0  TCP 10.0.0.50:8890 (LISTEN)
controlle 26870  devuser   82u  IPv4    323629      0t0  TCP 10.0.0.50:8891 (LISTEN)
controlle 26871  devuser   82u  IPv4    323635      0t0  TCP 10.0.0.50:8892 (LISTEN)
controlle 26876  devuser   82u  IPv4    323643      0t0  TCP 10.0.0.50:8896 (LISTEN)
controlle 26877  devuser   82u  IPv4    323615      0t0  TCP 10.0.0.50:8897 (LISTEN)
controlle 26880  devuser   82u  IPv4    323647      0t0  TCP 10.0.0.50:8900 (LISTEN)
controlle 26881  devuser   82u  IPv4    323649      0t0  TCP 10.0.0.50:8901 (LISTEN)
controlle 26882  devuser   82u  IPv4    323631      0t0  TCP 10.0.0.50:8902 (LISTEN)

And weirdest of all, these zombie processes appear to be working fine:

# curl http://10.0.0.50:8892/status
{"status": "ok"}

But killing the processes to make them stop (I need to upgrade them, hence trying to stop them in the first place) doesn't have any effect.

I can probably reboot to kill the processes in order to upgrade them, but it would be nice to figure out WTF is happening here with invincible running-dead processes first…

Best Answer

kill -9 will exterminate those zombies.

Typically, zombies happen when the parent dies and the child processes are not properly shut down by the parent before it exits. This happens more often if you kill the parent and it doesn't gracefully shut down (and take all the children with it). This is similar to an Orphan process.

Related Topic