Edit 13/12/12 – At this stage it would appear to be related to Western Digital (and possibly others) Advanced Format 4K block sizes.
This link goes some way into describing the issue, however the disk I have reports 512 logical and 4096 physical blocks (as opposed to 512/512).
http://johannes-bauer.com/linux/wdc/?menuid=3
I will continue testing and if I ever find a solution I'll be sure to post it.
I have a requirement to write large amounts of data to a vmdk disk located a USB3 disk attached to a Debian Linux server. I am creating partition mappings using kpartx and then mounting the created loop devices e.g.
The USB disk is /dev/sdb4 mounted at /mnt/backup.
> kpartx -av /mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk
add map loop2p1 (254:3): 0 273042 linear /dev/loop2 63
add map loop2p2 (254:4): 0 16787925 linear /dev/loop2 273105
add map loop2p3 (254:5): 0 8401995 linear /dev/loop2 17061030
add map loop2p4 (254:6): 0 184219648 linear /dev/loop2 25495552
add map loop2p5 : 0 227328 linear 254:6 2048
add map loop2p6 : 0 15998976 linear 254:6 231424
add map loop2p7 : 0 3059712 linear 254:6 16232448
add map loop2p8 : 0 2942976 linear 254:6 19294208
add map loop2p9 : 0 161978368 linear 254:6 22239232
Then I mount one of the partitions.
> mkdir /mnt/loop2p2
> mount /dev/mapper/loop2p2 /mnt/loop2p2
And run a test writing data to the mount point, aka the second partition in the vmdk.
> dd if=/dev/zero of=/mnt/loop2p2/zerofile.tst bs=1k count=1700000
1700000+0 records in
1700000+0 records out
1740800000 bytes (1.7 GB) copied, 135.871 s, 12.8 MB/s
A test to the USB3 disk itself runs well.
> dd if=/dev/zero of=/mnt/backup/zerofile.tst bs=1k count=1700000
1700000+0 records in
1700000+0 records out
1740800000 bytes (1.7 GB) copied, 19.5485 s, 89.1 MB/s
As a comparison I created a raw image file and ran the same test.
> qemu-img create -f raw /mnt/backup/raw.img 4G
After creating an ext3 partion on raw.img I mounted it and ran the test to it.
> kpartx -av /mnt/backup/raw.img
add map loop2p1 (254:3): 0 8386560 linear /dev/loop2 2048
> mkdir /mnt/loop2p1
> mount /dev/mapper/loop2p1 /mnt/loop2p1
> dd if=/dev/zero of=/mnt/loop2p1/zerofile.tst bs=1k count=1700000
1700000+0 records in
1700000+0 records out
1740800000 bytes (1.7 GB) copied, 23.3126 s, 74.7 MB/s
Good result! slightly slower than writing direct to the disk but that's to be expected.
So then I decided t convert the vmdk to a raw image using qemu-img as another comparison.
> qemu-img convert /mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk -O raw /mnt/backup/3MSYDDP01/3MSYDDP01_3.raw
> kpartx -av /mnt/backup/3MSYDDP01/3MSYDDP01_3.raw
add map loop2p1 (254:3): 0 273042 linear /dev/loop2 63
add map loop2p2 (254:4): 0 16787925 linear /dev/loop2 273105
add map loop2p3 (254:5): 0 8401995 linear /dev/loop2 17061030
add map loop2p4 (254:6): 0 184219648 linear /dev/loop2 25495552
add map loop2p5 : 0 227328 linear 254:6 2048
add map loop2p6 : 0 15998976 linear 254:6 231424
add map loop2p7 : 0 3059712 linear 254:6 16232448
add map loop2p8 : 0 2942976 linear 254:6 19294208
add map loop2p9 : 0 161978368 linear 254:6 22239232
> mount /dev/mapper/loop2p2 /mnt/loop2p2
dd if=/dev/zero of=/mnt/loop2p2/zerofile.tst bs=1k count=1700000
1700000+0 records in
1700000+0 records out
1740800000 bytes (1.7 GB) copied, 129.653 s, 13.4 MB/s
Poor performance with the converted disk as well.
The fdisk of each disk as follows:
> fdisk -l /dev/sdb
Note: sector size is 4096 (not 512)
Disk /dev/sdb: 3000.6 GB, 3000558944256 bytes
255 heads, 63 sectors/track, 45599 cylinders, total 732558336 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x000246c6
Device Boot Start End Blocks Id System
/dev/sdb1 256 131327 524288 83 Linux
/dev/sdb2 131328 2228479 8388608 83 Linux
/dev/sdb3 2228480 2490623 1048576 82 Linux swap / Solaris
/dev/sdb4 2490624 732558335 2920270848 83 Linux
raw.img
> fdisk -l /mnt/backup/raw.img
Disk /mnt/backup/raw.img: 4294 MB, 4294967296 bytes
43 heads, 32 sectors/track, 6096 cylinders, total 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xd48075ee
Device Boot Start End Blocks Id System
/mnt/backup/raw.img1 2048 8388607 4193280 83 Linux
3MSYDDP01_3-flat.vmdk
> fdisk -l /mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk
Disk /mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders, total 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0005cbed
Device Boot Start End Blocks Id System
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk1 * 63 273104 136521 83 Linux
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk2 273105 17061029 8393962+ 83 Linux
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk3 17061030 25463024 4200997+ 82 Linux swap / Solaris
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk4 25495552 209715199 92109824 5 Extended
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk5 25497600 25724927 113664 83 Linux
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk6 25726976 41725951 7999488 83 Linux
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk7 41728000 44787711 1529856 83 Linux
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk8 44789760 47732735 1471488 83 Linux
/mnt/backup/3MSYDDP01/3MSYDDP01_3-flat.vmdk9 47734784 209713151 80989184 83 Linux
Edit: The Block size of the partitions in the vmdk are:
> dumpe2fs /dev/mapper/loop2p1 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 1024
> dumpe2fs /dev/mapper/loop2p2 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 4096
> dumpe2fs /dev/mapper/loop2p5 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 1024
> dumpe2fs /dev/mapper/loop2p6 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 4096
> dumpe2fs /dev/mapper/loop2p7 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 4096
> dumpe2fs /dev/mapper/loop2p8 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 4096
> dumpe2fs /dev/mapper/loop2p9 | grep -i 'block size'
dumpe2fs 1.42.5 (29-Jul-2012)
Block size: 4096
/dev/mapper/loop2p3 is swap.
/dev/mapper/loop2p4 is an extended partition.
Could the sector size of the USB disk have an impact? If so why would the raw image be fine? I will look at testing with the USB disk having a 512 sector size but that will take time as I have to copy the data off, format and copy back.
Best Answer
Don't back up the entire VM. Depending on the application, you should be able to do some application-level replication. Also just filling a VMDK file via rsync is not the same as backing up the VM. There are intangibles like locked files in use, databases (if applicable) and configuration files that may need to be copied over separately.
Overall this is a poor idea and will probably not provide the recovery plan your employers are expecting. Also, external USB drives do not belong in production.