Nfs – How to kill ostensibly immortal process

killnfsrhel6

I had some huge file transfers operating on an NFS mount. The server on which the mount point resided was carelessly rebooted, and now the server from which these large transfers were initiated seems to be bogged down by them.

If I run top, I see the following:

enter image description here

The first thing I tried was to run kill with each the -1 -2 -9 and -15 flags, and each of the process ids shown above in turn. This allowed me to proceed, but didn't kill the processes. The next thing I attempted was to reboot the server, but neither reboot nor shutdown -r now worked. When I ran shutdown -r now the standard broadcast message was sent out, but the sever did not reboot. I confirmed this by looking at the server uptime, which was 25 days.

So now I'm a little stuck. I'm running these commands as root.

EDIT: Here's another interesting tidbit:

enter image description here

In top, I don't see that any other processes are using more than a fraction of a percent of memory or more than 5% of CPU.

EDIT 2: output of /var/log/messages

enter image description here

Best Answer

OK, time for something even more drastic! 

echo 1 >/proc/sys/kernel/panic

This instructs the kernel that when you kernel panic, to reboot the host in 1 second.

echo c >/proc/sysrq-trigger

This forces the kernel to panic. So hopefully you'll end up rebooting the host.

Related Topic