Debian – OpenNebula can’t start vm after reboot

debiankvm-virtualizationlibvirtopennebula

I installed a fresh install of Debian-7-opennebula-4.4.0-1.tar.gz on a fresh Debian and got a VM to start via sunstone.

The output of libvirt was:

2013-12-26 12:48:12.919+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.1 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name one-30 -uuid f12869a4-52e0-0568-9a04-53986c8c73c9 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/one-30.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/one//datastores/0/30/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/var/lib/one//datastores/0/30/disk.1,if=none,id=drive-ide0-0-1,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive file=/var/lib/one//datastores/0/30/disk.2,if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=20,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=02:00:0a:0a:00:04,bus=pci.0,addr=0x3 -vnc 0.0.0.0:30 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
qemu: terminating on signal 15 from pid 13422
2013-12-26 13:13:30.448+0000: shutting down

But after a reboot of the host, OpenNebula/Sunstone/KVM changed their doings and the VM does not start anymore, the output is:

2013-12-27 09:39:06.511+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin HOME=/ QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.1 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name one-30 -uuid 1dcb4efc-4915-5685-b2d0-4a66e12c81c9 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/one-30.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/one//datastores/0/30/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/var/lib/one//datastores/0/30/disk.1,if=none,id=drive-ide0-0-1,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive file=/var/lib/one//datastores/0/30/disk.2,if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=20,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=02:00:0a:0a:00:04,bus=pci.0,addr=0x3 -vnc 0.0.0.0:30 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
kvm: -drive file=/var/lib/one//datastores/0/30/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none: could not open disk image /var/lib/one//datastores/0/30/disk.0: Permission denied
2013-12-27 09:39:06.917+0000: shutting down

When comparing these two, the first thing I notice is the difference of parameters. Before the reboot, it was HOME=/root USER=root LOGNAME=root and now it is just HOME=/.

What did change and how can I get it back, working? I did not change anything in the configuration files itself, I just used sunstone to work with opennebula.

For more details, here the datastore content:

 # ls -ahl /var/lib/one//datastores/0/30/
total 388K
drwxr-xr-x 2 oneadmin     oneadmin     4.0K Dec 27 10:47 .
drwxr-xr-x 5 oneadmin     oneadmin     4.0K Dec 27 10:36 ..
-rw-r--r-- 1 oneadmin     oneadmin     1.1K Dec 26 13:48 deployment.0
-rw-r--r-- 1 oneadmin     oneadmin     1.1K Dec 27 10:39 deployment.1
-rw-r--r-- 1 oneadmin     oneadmin     1.1K Dec 27 10:47 deployment.2
lrwxrwxrwx 1 oneadmin     oneadmin       18 Dec 26 13:48 disk.0 -> /dev/vg0/lv-one-10
lrwxrwxrwx 1 oneadmin     oneadmin       58 Dec 26 13:48 disk.1 -> /var/lib/one/datastores/1/7793795d8fa6b6e82aecec2b129c5ba3
-rw-r--r-- 1 libvirt-qemu libvirt-qemu 364K Dec 26 13:48 disk.2
lrwxrwxrwx 1 oneadmin     oneadmin       35 Dec 26 13:48 disk.2.iso -> /var/lib/one/datastores/0/30/disk.2

# ls -ahl /dev/vg0/lv-one-10
lrwxrwxrwx 1 root root 7 Dec 26 14:15 /dev/vg0/lv-one-10 -> ../dm-6

# ls -ahl /dev/dm-6
brw-rw---T 1 root disk 253, 6 Dec 26 14:15 /dev/dm-6

Best Answer

It seems that OpenNebula daemon was started as root the first time. OpenNebula is meant to run as oneadmin user but needs some permissions set to be able to do some operations. The first time it worked as it was running VMs as root and could access the disks.

In an standard installation VMs are run as oneadmin user and to access LVM LVs that user needs to be in the group disk. As you can see LV has RW permissions to group disk (/dev/dm-6).

You can find the requirements for LVM in the documentation page.

Related Topic