Linux – How to interpret iozone values

iopslinux

I ran a test to measure my I/O IOPS on Linux:

iozone -s 4g -r 2k -r 4k -r 8k -r 16k -r 32k -O -b /tmp/results.xls

iozone claims that output is in operations per second yet the numbers are too big for that to be plausible. I'm observing some 320 CMDs/s maximum on vmware esx console (esxtop, then v).

    File size set to 4194304 KB
    Record Size 2 KB
    Record Size 4 KB
    Record Size 8 KB
    Record Size 16 KB
    Record Size 32 KB
    OPS Mode. Output is in operations per second.
    Command line used: iozone -s 4g -r 2k -r 4k -r 8k -r 16k -r 32k -O -b tmpresults.xls
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                        random  random    bkwd   record   stride
          KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
     4194304       2   19025    5580    27581    29848     284     198     415  1103217     1498    18541     4340   24245    25618
     4194304       4   15650   21942    18962    21068     252    1198     193   976164     1677    22802    23093   21089    21232
     4194304       8   11121   11638    10273    10165     247    1196     202   625020^C

The test ran for 15 hours before I pressed ^C. Is that ordinary expectation for such command line (dedicated 4 drive RAID10 LUN, 10k RPM SAS drives in EMC CX300)?

Best Answer

It's strange that there doesn't seem to be a standard for IOPS measurement in Linux. These questions come up often when people are asked to provide an estimate of IOPS for capacity planning or storage sizing. I end up using multiple tools to gain this info. First, you'll need to generate a load using the method of your choice; either a representative application load, or something like the iozone test you had above.

In another window, track IOPS using iostat -dxk 1, summing the r/s and w/s columns OR using IBM's nmon tool with the D option to track the Xfers column (which is essentially the sum of iostat's r/s and w/s).

Here's nmon output from a heavy sequential read using bonnie++ -u root -n 64:100000:16:64 (run from within the target directory). At this point, we're showing ~3200 IOPS while reading at about 258 Megabytes/second on a 6-disk RAID 1+0 array of 15k RPM SAS drives.

- Disk I/O -----(/proc/diskstats)------- all data is Kbytes per second ---------------------------------------------
|DiskName Busy    Read    Write       Xfers   Size  Peak%  Peak-RW    InFlight                                     |
|iss/c0d0 100% 264571.1    112.3KB/s 3238.7  81.0KB  100%  340584.5KB/s   2                                        |
|s/c0d0p1   0%      0.0      0.0KB/s    0.0   0.0KB    0%       2.0KB/s   0                                        |
|s/c0d0p2   0%      0.0      0.0KB/s    0.0   0.0KB   99%    1022.4KB/s   0                                        |
|s/c0d0p3   0%      0.0      0.0KB/s    0.0   0.0KB  100%    3636.5KB/s   0                                        |
|s/c0d0p4   0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0                                        |
|s/c0d0p5   0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0                                        |
|s/c0d0p6   0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0                                        |
|s/c0d0p7   0%      0.0     41.9KB/s    7.5   5.0KB  100%   16103.5KB/s   0                                        |
|s/c0d0p8   0%      0.0      0.0KB/s    0.0   0.0KB   79%     147.8KB/s   0                                        |
|s/c0d0p9 100% 264571.1     64.4KB/s 3230.2  81.0KB  100%  340538.5KB/s   2                                        |
--------------------------------------------------------------------------------------------------------------------
Related Topic