FreeBSD 10.1 how to autoreboot on kernel panic

freebsdkernel-panicrootfs

Recently I installed FreeBSD 10.1 on machine that for years was working fine with FreeBSD 6.2. Since machine is located in datacenter I configured remote access via serial console.

After installation I encountered a weird problem on boot.
Every reboot the rootfs cannot be mounted.

uhub4: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/raid/r0p2 [rw]...
mountroot: waiting for device /dev/raid/r0p2 ...
Mounting from ufs:/dev/raid/r0p2 failed with error 19.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/raid/r0p2
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
  Mount <device> using filesystem <fstype>
  and with the specified (optional) option list.

  eg. ufs:/dev/da0s1a
  zfs:tank
  cd9660:/dev/acd0 ro
  (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

  ?  List valid disk boot devices
  .  Yield 1 second (for background tasks)
  <empty line>  Abort manual input

mountroot>

but when I press enter (via attached keyboard or serial console) I can see the following, the system reboots just fine

mountroot>
panic: mountroot: unable to (re-)mount root.
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff80963000 at kdb_backtrace+0x60
#1 0xffffffff80928125 at panic+0x155
#2 0xffffffff809c554f at vfs_mountroot+0x1eaf
#3 0xffffffff808d7533 at start_init+0x53
#4 0xffffffff808f8b6a at fork_exit+0x9a
#5 0xffffffff80d0acbe at fork_trampoline+0xe
Uptime: 12s
Automatic reboot in 15 seconds - press a key on the console to abort

now system reboots and everything goes fine

Trying to mount root from ufs:/dev/raid/r0p2 [rw]...
Setting hostuuid: 530c9cf5-a7e5-11e4-97b5-00e08146563c.
Setting hostid: 0xe4218f4f.
Entropy harvesting: interrupts ethernet point_to_point swi.
Starting file system checks:
/dev/raid/r0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/raid/r0p2: clean, 7232943 free (983 frags, 903995 blocks, 0.0% fragmentation)
/dev/raid/r1s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/raid/r1s1f: clean, 41630678 free (104646 frags, 5190754 blocks, 0.1% fragmentation)
/dev/raid/r0p4: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/raid/r0p4: clean, 49146170 free (8498 frags, 6142209 blocks, 0.0% fragmentation)
Mounting local file systems:.
Writing entropy file:.

I just thought that if the system auto rebooted when it cannot mount root (which I guess is due to kernel panic), I could have this system working fine, it would just reboot twice.

I found similar thread but related to linux, where I could set /etc/sysctl variable and have system autoreboot on kernel panic

kernel.panic = 20

Automatic Reboot after Kernel Panic

It doesn't seem to work on FreeBSD though.
My question is is there a way I can make this system to autoreboot when rootfs cannot be mounted? I hope it might reboot and mount root just fine on the second run?

I will also appreciate any other suggestions.

Thanks

comment1:

I was requested to output some data from server settings, here is it:

nn@d02:~ % uname -a
    FreeBSD hostname 10.1-RELEASE-p19 FreeBSD 10.1-RELEASE-p19 #0: Sat Aug 22 03:55:09 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

nn@d02:~ % freebsd-version -k
10.1-RELEASE-p19

nn@d02:~ % freebsd-version -u
10.1-RELEASE-p19

nn@d02:~ % gpart show
=>       34  488390589  raid/r0  GPT  (233G)
         34       1024        1  freebsd-boot  (512K)
       1058   62914560        2  freebsd-ufs  (30G)
   62915618    8388608        3  freebsd-swap  (4.0G)
   71304226  417086396        4  freebsd-ufs  (199G)
  488390622          1           - free -  (512B)

=>       63  488390593  raid/r1  MBR  (233G)
         63  488375937        1  freebsd  [active]  (233G)
  488376000      14656           - free -  (7.2M)

=>        0  488375937  raid/r1s1  BSD  (233G)
          0    1048576          1  freebsd-ufs  (512M)
    1048576    8324192          2  freebsd-swap  (4.0G)
    9372768    6258688          4  freebsd-ufs  (3.0G)
   15631456    1048576          5  freebsd-ufs  (512M)
   16680032  471695905          6  freebsd-ufs  (225G)

nn@d02:~ % graid show
graid: Unknown command: show.

nn@d02:~ % graid status
   Name   Status  Components
raid/r0  OPTIMAL  ada0 (ACTIVE (ACTIVE))
                  ada1 (ACTIVE (ACTIVE))
raid/r1  OPTIMAL  ada2 (ACTIVE (ACTIVE))
                  ada3 (ACTIVE (ACTIVE))

Best Answer

If you are using the GENERIC kernel in FreeBSD, the KDB option is enabled, alongside the sysctl debug.debugger_on_panic, which defaults to dropping into the kernel debugger on a panic for troubleshooting.

To stop the debugger from being invoked, change the sysctl debug.debugger_on_panic to 0 per the recommendation in the Glossary of Kernel Debug Options. That should solve your problem of needing to be present to interact with the server on panic.

Hope that helps you out. Good luck. =)