Linux – Migrate raw disk image over LAN

linuxlocal-area-networkmigrationraid

Here is my situation:

  • Two dedicated servers in the same datacenter with gigabit ethernet between them.
  • Both dedicated servers booted into a rescue environment based on Debian Squeeze with extra tools and utilities added. Also plenty of tmp space (32GB of RAM on both boxes) for downloading software, installing packages, and/or compiling as needed.
  • Both dedicated servers have approximately 3TB of usable space.
  • The "source" server has 4 x 1.5TB disks in Hardware RAID-10 with an Adaptec 4 port controller.
  • The "destination" server has 2 x 3TB disks in Hardware RAID-1 with an Adaptec 2 port controller — same generation as the other, but different number of ports.
  • The number of usable blocks on /dev/sda differs by less than 10 MB, but the destination server's array is for some reason a few megs smaller.
  • Both RAID arrays are configured to use the entire disk surface of all constituent disks to create one, single RAID volume.
  • The operating system boots in MBR mode; no UEFI booting is used.

What I want to do:

  • Copy, at the block layer, the entire OS image (this only consists of GRUB2 bootloader in the GPT partition table, /boot partition, and / partition) from the "source" server to the "destination" server.
  • If possible, the copy should take place "live": this means I don't have enough space to store a proper file of the disk image on the destination side, unless I'm unpacking the disk image onto the hard disk as the copy is taking place. The gigabit ethernet connection between the servers is reliable enough that I'm comfortable with this, and I will of course run fsck on both ends (source and destination) to verify the filesystem is OK before and after the transfer.
  • If possible, do not transfer blocks over the network, which are not used by the constituent filesystems in each partition (all partitions are formatted as ext4). This is because more than 50% of the "source" disk is free space in the / partition.
  • Adjust the size of the / partition so that when it is copied, it is resized to fit within the just barely smaller size of the destination disk.
  • Once the copy is successful, mount each volume and fix up references to static IPs to reflect the IPs of the new server. (Can do this just fine without any further help)

My questions:

  • Should I first calculate the difference (in bytes) between the size of /dev/sda on each server, and then use e2resize to non-destructively reduce the size of the / partition on the source side so that it will fit into the space of the destination side?
  • Should I run dd on the raw block device, /dev/sda from the source to the destination (over ssh), or should I create an equivalent partition layout on the destination and run dd on each partition? Note that handling a partition at a time leaves me the problem of the bootloader, but if I don't do it a partition at a time, then dd needs to know to stop transferring data once it has written as many bytes as the destination can hold (which hopefully will "close out" the very end of the / partition on the last block, which is logically "to the right of" all other partitions in the partition layout of the source).

A few misc. specifics:

  • The host OS on the source box is Ubuntu Server 12.04 running several OpenVZ guests
  • Since both boxes are booted into rescue, direct disk access is possible without expecting any change to the underlying data by the running operating system.

Best Answer

This is messy, but doable.

I presume here that / is on /dev/sda3 and that /boot is on /dev/sda1.

  1. Shrink the filesystem on the old server to its minimum possible size.

    oldserver # resize2fs -M /dev/sda3
    
  2. Partition the new server's disk with an identically sized /boot, swapspace, and new / partition (and anything else you need).

    newserver # parted /dev/sda
    
  3. Copy the / and /boot filesystems.

    oldserver # dd if=/dev/sda1 | ssh root@newserver "dd of=/dev/sda1"
    oldserver # dd if=/dev/sda3 | ssh root@newserver "dd of=/dev/sda3"
    

    Because the partition on the new server will be slightly smaller than the one on the old server, you'll receive a spurious No space left on device message at the end of this. However, since you shrank the filesystem at step 1, this doesn't matter.

  4. Resize the filesystem on the new server to the size of the partition.

    newserver # resize2fs /dev/sda3
    
  5. Install GRUB on the new disk.

    newserver # mount /dev/sda3 /mnt
    newserver # mount /dev/sda1 /mnt/boot
    newserver # mount -o bind /dev /mnt/dev
    newserver # mount -o proc proc /mnt/proc
    newserver # chroot /mnt /bin/bash
    
    newserver(chroot) # grub-install /dev/sda
    newserver(chroot) # exit
    
  6. Finish the rest of your fixups (IP address, etc.).

You can probably find a way to avoid copying the partition's free space, but it'll probably take you longer to research than to just copy it all...