Is calculating IOPS for ZFS RAIDZ different then calculating IOPS for RAID5 & RAID6

performancestoragezfs

When calculating IOPS for traditional RAID arrays, one could use the following formula (Borrowed from Getting The Hang Of IOPS v1.3 on Symantec Connect):

Ieffective = (n * Isingle) / (READ% + (F * WRITE%))

Where:

  • Ieffective is effective number of IOPS
  • Isingle is a single drive's average IOPS.
  • n is number of disks in the array
  • READ% is the fraction of reads taken from disk profiling
  • WRITE% is the fraction of writes taken from disk profiling

  • F is the RAID write penalty:

    RAID Level      Write Penalty
RAID-0          1
RAID-1          2
RAID-5          4
RAID-6          6
RAID-10         2
RAID-DP         2

The formula is essentially a function of:

  • IOPS for each individual drive in the array
  • The number of disks. More disks means more IOPS
  • The RAID penalty for each write operation.
    • RAID5 & RAID6 require 4+ disk operations for every write. The controller must read the block and then read the parity data (two operations), calculating the new parity data and then update the parity block and update the data block (Two more operations). RAID6 has two parity blocks and therefore requires three reads and three writes. RAID5 & RAID6 arrays are therefore capable of fewer IOPS then RAID1.
    • RAID1 & RAID10 only require 2 writes, one to each disk in the mirror.

And to be clear, this all provides an estimate of theoretical performance. Various controllers and RAID methods have tricks to speed some of this up.

ZFS's equivalent of RAID5 & RAID6 is RAIDZ and RAIDZ2. When calculating IOPS for RAIDZ arrays, can I use the same formula that I use for RAID5 & RAID6, or does ZFS have special tricks to reduce the number of operations requited for write operations.

Is there a different formula to use when calculating IOPS for RAIDZ arrays?

Best Answer

This is easier to answer...

It's all distilled here: ZFS RAID recommendations: space, performance, and MTTDL and A Closer Look at ZFS, Vdevs and Performance

  • RAIDZ with one parity drive will give you a single disk's IOPS performance, but n-1 times aggregate bandwidth of a single disk.

So if you need to scale, you scale with the number of RAIDZ vdevs... E.g. with 16 disks, 4 groups of 4-disk RAIDZ would have greater IOPS potential than 2 groups of 8-disk RAIDZ.

Surprising, right?

I typically go with striped mirrors (RAID 1+0) on my ZFS installations. The same concept applies. More mirrored pairs == better performance.

In ZFS, you can only expand in units of a full vdev. So while expansion of a RAID 1+0 set means adding more pairs, doing the same for RAIDZ sets means adding more RAIDZ groups of equal compositon.

Related Topic