Am I misunderstanding something, or should this not be possible?
All my daemon processes are in zombie state after I tried to stop the control service:
# ps ax | grep controller
13768 pts/11 S+ 0:00 grep controller
26866 ? Zl 18:56 [controller] <defunct>
26870 ? Zl 18:57 [controller] <defunct>
26871 ? Zl 18:45 [controller] <defunct>
26876 ? Zl 13:17 [controller] <defunct>
26877 ? Zl 10:28 [controller] <defunct>
26880 ? Zl 18:18 [controller] <defunct>
26881 ? Zl 12:01 [controller] <defunct>
26882 ? Zl 18:18 [controller] <defunct>
And yet ports are still open (although netstat can't find the process name)
# netstat -tlpn | sort
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1180/sshd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 11882/httpd
tcp 0 0 10.0.0.50:8890 0.0.0.0:* LISTEN -
tcp 0 0 10.0.0.50:8891 0.0.0.0:* LISTEN -
tcp 0 0 10.0.0.50:8892 0.0.0.0:* LISTEN -
tcp 0 0 10.0.0.50:8896 0.0.0.0:* LISTEN -
tcp 0 0 10.0.0.50:8897 0.0.0.0:* LISTEN -
tcp 0 0 10.0.0.50:8900 0.0.0.0:* LISTEN -
Although lsof can see the process names
# lsof -i -n -P | grep 10.0.0.50 | grep LISTEN
controlle 26866 devuser 82u IPv4 323641 0t0 TCP 10.0.0.50:8890 (LISTEN)
controlle 26870 devuser 82u IPv4 323629 0t0 TCP 10.0.0.50:8891 (LISTEN)
controlle 26871 devuser 82u IPv4 323635 0t0 TCP 10.0.0.50:8892 (LISTEN)
controlle 26876 devuser 82u IPv4 323643 0t0 TCP 10.0.0.50:8896 (LISTEN)
controlle 26877 devuser 82u IPv4 323615 0t0 TCP 10.0.0.50:8897 (LISTEN)
controlle 26880 devuser 82u IPv4 323647 0t0 TCP 10.0.0.50:8900 (LISTEN)
controlle 26881 devuser 82u IPv4 323649 0t0 TCP 10.0.0.50:8901 (LISTEN)
controlle 26882 devuser 82u IPv4 323631 0t0 TCP 10.0.0.50:8902 (LISTEN)
And weirdest of all, these zombie processes appear to be working fine:
# curl http://10.0.0.50:8892/status
{"status": "ok"}
But kill
ing the processes to make them stop (I need to upgrade them, hence trying to stop them in the first place) doesn't have any effect.
I can probably reboot to kill the processes in order to upgrade them, but it would be nice to figure out WTF is happening here with invincible running-dead processes first…
Best Answer
kill -9
will exterminate those zombies.Typically, zombies happen when the parent dies and the child processes are not properly shut down by the parent before it exits. This happens more often if you
kill
the parent and it doesn't gracefully shut down (and take all the children with it). This is similar to an Orphan process.