Debian – How to boot the Debian 10 image with qemu/kvm

debiankvm-virtualizationlibvirtqemu

I'm attempting to boot the openstack image of Debian 10 using qemu and encountering an error detecting the harddrive, the end of the boot up sequence is showing:

[    0.989085] Run /init as init process
Loading, please wait...
Starting version 241
[    1.068365] SCSI subsystem initialized
[    1.073933] cryptd: max_cpu_qlen set to 1000
[    1.085586] AVX2 version of gcm_enc/dec engaged.
[    1.085699] PCI Interrupt Link [LNKA] enabled at IRQ 10
[    1.086342] AES CTR mode by8 optimization enabled
[    1.094169] scsi host0: ata_piix
[    1.095524] scsi host1: ata_piix
[    1.096120] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc2c0 irq 14
[    1.097198] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc2c8 irq 15
[    1.108170] PCI Interrupt Link [LNKB] enabled at IRQ 11
[    1.120402] virtio_blk virtio0: [vda] 736 512-byte logical blocks (377 kB/368 KiB)
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
... line repeats ...
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  UUID=77e3f255-2ef2-47bc-ad89-7cdbd65f5fbc does not exist.  Dropping to a shell!


BusyBox v1.30.1 (Debian 1:1.30.1-4) built-in shell (ash)
Enter 'help' for a list of built-in commands.

From the initramfs prompt, I can see the following:

(initramfs) cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.19.0-5-cloud-amd64 root=UUID=77e3f255-2ef2-47bc-ad89-7cdbd65f5fbc ro biosdevname=0 net.ifnames=0 console=tty0 console=ttyS0,115200 earlyprintk=ttyS0,115200 consoleblank=0 systemd.show_status=true

(initramfs) cat /proc/modules
ata_generic 16384 0 - Live 0xffffffffc00ba000
crc32c_intel 24576 0 - Live 0xffffffffc00b3000
virtio_blk 20480 0 - Live 0xffffffffc00ad000
aesni_intel 200704 0 - Live 0xffffffffc0156000
ata_piix 36864 0 - Live 0xffffffffc0147000
aes_x86_64 20480 1 aesni_intel, Live 0xffffffffc0135000
crypto_simd 16384 1 aesni_intel, Live 0xffffffffc0130000
libata 245760 2 ata_generic,ata_piix, Live 0xffffffffc00da000
cryptd 28672 2 aesni_intel,crypto_simd, Live 0xffffffffc00d2000
glue_helper 16384 1 aesni_intel, Live 0xffffffffc00cb000
scsi_mod 237568 1 libata, Live 0xffffffffc0072000
virtio_pci 28672 0 - Live 0xffffffffc0066000
virtio_ring 28672 2 virtio_blk,virtio_pci, Live 0xffffffffc005b000
virtio 16384 2 virtio_blk,virtio_pci, Live 0xffffffffc0053000

(initramfs) ls /dev
block               tty18               tty5
char                tty19               tty50
console             tty2                tty51
core                tty20               tty52
cpu_dma_latency     tty21               tty53
disk                tty22               tty54
fd                  tty23               tty55
full                tty24               tty56
hpet                tty25               tty57
input               tty26               tty58
kmsg                tty27               tty59
mem                 tty28               tty6
memory_bandwidth    tty29               tty60
network_latency     tty3                tty61
network_throughput  tty30               tty62
null                tty31               tty63
psaux               tty32               tty7
ptmx                tty33               tty8
pts                 tty34               tty9
random              tty35               ttyS0
snapshot            tty36               ttyS1
stderr              tty37               ttyS2
stdin               tty38               ttyS3
stdout              tty39               urandom
tty                 tty4                vcs
tty0                tty40               vcs1
tty1                tty41               vcsa
tty10               tty42               vcsa1
tty11               tty43               vcsu
tty12               tty44               vcsu1
tty13               tty45               vda
tty14               tty46               vga_arbiter
tty15               tty47               zero
tty16               tty48
tty17               tty49

(initramfs) ls /dev/disk/by-label/
cidata

(initramfs) ls -al /dev/disk/by-uuid/
total 0
drwxr-xr-x    2 0        0               60 Jul 22 00:07 .
drwxr-xr-x    5 0        0              100 Jul 22 00:07 ..
lrwxrwxrwx    1 0        0                9 Jul 22 00:07 2019-07-21-20-02-09-00 -> ../../vda

To recreate this VM, first I setup the qcow2 image off of the downloaded base image (downloaded from the openstack image site):

~/vm/deb10-test$ sha256sum ../base-debian-10/base.qcow2
d4c2966d996a3e08c198be41640d54b5d0c038cfc21b4d05e4b769824974daaf  ../base-debian-10/base.qcow2

~/vm/deb10-test$ qemu-img create -f qcow2 -o "backing_file=../base-debian-10/base.qcow2" image.qcow2 20G
Formatting 'image.qcow2', fmt=qcow2 size=21474836480 backing_file=../base-debian-10/base.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16

Next I setup a seed.img for cloud-init using cloud-localds:

~/vm/deb10-test$ cat meta-data
instance-id: iid-deb10-test-20190721-193118
hostname: deb10-test
local-hostname: deb10-test

~/vm/deb10-test$ cat network-config
---
version: 1
config:
- type: physical
  name: eth0
  subnets:
  - type: static
    address: 192.168.237.42
    netmask: 255.255.255.0
    routes:
    - network: 0.0.0.0
      netmask: 0.0.0.0
      gateway: 192.168.237.1
- type: nameserver
  address: [192.168.234.10, 192.168.234.254, 8.8.8.8]
  search: []

~/vm/deb10-test$ cat user-data
#cloud-config
users:
  - default
chpasswd:
  list: |
    debian:passw0rd
  expire: False
ssh_pwauth: True
package_update: true
packages:
- python
bootcmd:
# disable automatic dhcp
- sed -e '/^#/! {/eth0/ s/^/# /}' -i /etc/network/interfaces

~/vm/deb10-test$ cloud-localds -v ./seed.img --network-config network-config user-data meta-data
wrote ./seed.img with filesystem=iso9660 and diskformat=raw

And lastly I ran virt-install with the following:

~/vm/deb10-test$ virt-install \
  --os-variant auto --virt-type kvm \
  --name deb10-test --graphics none --import \
  --disk path="./image.qcow2",format=qcow2,bus=scsi \
  --disk path="./seed.img",bus=virtio \
  --cpu host --vcpus 2 --memory 2048 \
  --network network=routed

A similar procedure works for Debian 9 images, and changing to bus=virtio works for many other Linux images (CentOS and Ubuntu). I'm at a loss for why the harddrive device is not showing while the rest of the initramfs appears to be working. Are there different options I need to pass to work with Debian 10?

Edit: Attempting the following did not solve the issue:

--machine q35: no visible difference, from the documentation this doesn't appear to be needed with KVM.
--disk path=./seed.img,device=cdrom,bus=sata: no visible difference
--controller scsi,model=virtio-scsi: this actually broke the boot further, just hanging on a blank console, no grub, kernel booting, or initramfs prompt. Using model=auto got back to the initramfs prompt.

Best Answer

Download this netboot/initrd.gz
Download this netboot/vmlinuz
Create an empty disk image with:

qemu-img create -f qcow2 hda.qcow2 10G
Run the debian-installer in QEMU with:

qemu-system-??????? -M virt -m 1024 -kernel vmlinuz -initrd initrd.gz -drive if=none,file=hda.qcow2,format=qcow2,id=hd -device virtio-blk-device,drive=hd -netdev user,id=mynet -device virtio-net-device,netdev=mynet -nographic -no-reboot

The debian-installer will run, ask you some basic questions with some text menus, detect virtual devices, do its job and successfully install a fully functional Debian10 OS into the hda.qcow2 disk image. Ignore its cries about the missing boot-loader...

Now, to run this image, you have to copy 2 files out of the hda.qcow2 image to your Host OS' file system (e.g. to your QEMU directory), namely: vmlinuz-4.19.???????? and initrd.img-4.19.????? ...using libguestfs tool on Linux or PeaZip on Windows
Finally execute the freshly installed Virtual Machine with:

qemu-system-??????? -M virt -m 1024 -kernel vmlinuz-4.19.??????? -initrd initrd.img-4.19.??????? -append root=/dev/vda2 -drive if=none,file=hda.qcow2,format=qcow2,id=hd -device virtio-blk-device,drive=hd -netdev user,id=mynet,restrict=off,net=192.168.0.0/24,hostfwd=tcp:127.0.0.1:10022-192.168.0.15:22 -device virtio-net-device,netdev=mynet -nographic -no-reboot

Make sure to change the ?????? to the specific filenames on your system and change the -append root=/dev/vda2 to some other device if you chose not to format the entire virtual disk for the Linux partition, during the Debian's installation.

Finally, boot the VM and login into it from the console.

In order to login into this virtual machine as root through SSH at 127.0.0.1:10022, you must edit the /etc/ssh/sshd_config file and add the line: PermitRootLogin yes.

You might also want to take a look at this answer of mine if the SSH server misbehaves.

Related Solutions

Debian Lenny as Xen domU fails to boot

mount: cannot read /etc/fstab: No such file or directory

That's a pretty clear sign that your initramfs is borked. Probably because your install is borked.

How did you get past partitioning in debian-installer? Last time I did it (ummm... yesterday) I had to export something that I could partition, so your disk being hda1 is rather weird. Mine looks like:

disk = ['phy:/dev/GLaDOS/xen-portaltest,xvda,w']

its on a logical volume, not a file, but that shouldn't matter. Giving it xvda1 or whatever didn't work; it wanted to partition that, which is fairly silly.

With xvda, I went ahead and partitioned that (into a xvda1 for /boot and an xvda2 for a LVM physical volume, but you could certainly just use that for root). The installer then completed normally, and it works after dealing with the bootloader not executable error documented on the Debian Wiki's Xen entry.

Difference between KVM and QEMU

Qemu:

QEmu is a complete and standalone software of its own. You use it to emulate machines, it is very flexible and portable. Mainly it works by a special 'recompiler' that transforms binary code written for a given processor into another one (say, to run MIPS code on a PPC mac, or ARM in an x86 PC).

To emulate more than just the processor, Qemu includes a long list of peripheral emulators: disk, network, VGA, PCI, USB, serial/parallel ports, etc.

KQemu:

In the specific case where both source and target are the same architecture (like the common case of x86 on x86), it still has to parse the code to remove any 'privileged instructions' and replace them with context switches. To make it as efficient as possible on x86 Linux, there's a kernel module called KQemu that handles this.

Being a kernel module, KQemu is able to execute most code unchanged, replacing only the lowest-level ring0-only instructions. In that case, userspace Qemu still allocates all the RAM for the emulated machine, and loads the code. The difference is that instead of recompiling the code, it calls KQemu to scan/patch/execute it. All the peripheral hardware emulation is done in Qemu.

This is a lot faster than plain Qemu because most code is unchanged, but still has to transform ring0 code (most of the code in the VM's kernel), so performance still suffers.

KVM:

KVM is a couple of things: first it is a Linux kernel module—now included in mainline—that switches the processor into a new 'guest' state. The guest state has its own set of ring states, but privileged ring0 instructions fall back to the hypervisor code. Since it is a new processor mode of execution, the code doesn't have to be modified in any way.

Apart from the processor state switching, the kernel module also handles a few low-level parts of the emulation like the MMU registers (used to handle VM) and some parts of the PCI emulated hardware.

Second, KVM is a fork of the Qemu executable. Both teams work actively to keep differences at a minimum, and there are advances in reducing it. Eventually, the goal is that Qemu should work anywhere, and if a KVM kernel module is available, it could be automatically used. But for the foreseeable future, the Qemu team focuses on hardware emulation and portability, while KVM folks focus on the kernel module (sometimes moving small parts of the emulation there, if it improves performance), and interfacing with the rest of the userspace code.

The kvm-qemu executable works like normal Qemu: allocates RAM, loads the code, and instead of recompiling it, or calling KQemu, it spawns a thread (this is important). The thread calls the KVM kernel module to switch to guest mode and proceeds to execute the VM code. On a privileged instruction, it switches back to the KVM kernel module, which, if necessary, signals the Qemu thread to handle most of the hardware emulation.

One of the nice things of this architecture is that the guest code is emulated in a posix thread which you can manage with normal Linux tools. If you want a VM with 2 or 4 cores, kvm-qemu creates 2 or 4 threads, each of them calls the KVM kernel module to start executing. The concurrency—if you have enough real cores—or scheduling—if not—is managed by the normal Linux scheduler, keeping code small and surprises limited.

Best Answer

Related Solutions

Debian Lenny as Xen domU fails to boot

Difference between KVM and QEMU

Related Topic