Mdadm won’t create an array larger than 10TB

mdadmpartitionraid

  • I have a server with six 4TB drives running Ubuntu 14.04 LTS.
  • One of the drives is used for the system/OS
  • I wanted to use the other 5 drives to get as close to 20TB of storage as possible
  • It's a short term data analysis project, so RAID 0 is fine
  • I used fdisk to create a single ~4TB partition on each of the 5 drives

After running mdadm to create a RAID 0 array, the result was just a 10TB device on /dev/md0. Half the size that I was expecting!

Here is the command that I used to create the array:

mdadm --create --verbose /dev/md0 --level=stripe --raid-devices=5 /dev/sd[bcdef]1

Below is an fdisk -l output. Any ideas on why I can't get more space out of the drives?

Disk /dev/sda: 4000.8 GB, 4000787030016 bytes
255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1  4294967295  2147483647+  ee  GPT

Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes
90 heads, 3 sectors/track, 28940878 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xda4f04e2

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc9e57616

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sdd: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x53bed1ab

Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sde: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbc5c1069

Device Boot      Start         End      Blocks   Id  System
/dev/sde1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sdf: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4801480c

Device Boot      Start         End      Blocks   Id  System
/dev/sdf1            2048  4294967294  2147482623+  83  Linux

Disk /dev/md0: 10995.1 GB, 10995108413440 bytes
2 heads, 4 sectors/track, -1610614656 cylinders, total 21474821120 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 2621440 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Best Answer

Drop the 1 from the end of your command:

mdadm --create --verbose /dev/md0 --level=stripe --raid-devices=5 /dev/sd[bcdef]

Should work fine. I'm assuming you want to use all the space available on all 5 of the RAID disks.

Related Topic