Linux – How to know what the server is doing when crashing

centoskernellinux

I have a server running on Centos 5.2 and is there any a better way to know why the server crashed or what it's doing at that time?

I am sorry I am a newbie and any help is appreciated~ Thanks

Best Answer

If you have experienced a kernel panic, you can set up a remote kernel console to capture all the data that might be lost on the local console (especially if the crash is from a non-maskable interrupt, which tends to reboot the system).

On the system that you expect might crash:

/sbin/modprobe netconsole netconsole=6666@10.1.1.16/eth0,6666@10.1.1.17/00:19:BB:31:B8:0E
  • 6666 is an arbitrary port number
  • 10.1.1.16 is the IP address of the local interface to send via
  • eth0 is the name of the local interface to send via
  • 10.1.1.17 is the IP address of the remote interface to send to
  • 00:19:BB:31:B8:0E is the MAC address of the remote interface to send to

On the remote system, run (this requires that you have netcat installed):

nc -l -p 6666 -u | tee capture.file

This will capture all kernel output on the remote system. This runs at a much lower level (the same point in the kernel that writes to /dev/klog), so you may see the very last bit of information that the kernel outputs when it panics even if syslog et. al have stopped operating.