MD3200i Slow Performance and Queue Depth

dell-powervaultiscsistorage-area-networkwindows-server-2008-r2

Read performance on our SAN is slow under certain workloads. When we compare this to some local storage, we find the local storage performing 2x as fast. The SAN performs well with a high Queue Depth, and poorly with a low queue depth. However, the local storage performs well with a low Queue Depth. I'd like to know the reason for this occurring and find out what the specific limiting factor is in this situation.

MD3200i iSCSI SAN ($15,000)

  • 6 x 600GB 15k SAS RAID5
  • 6 x 2TB 7.2k NLS RAID5

XCOPY /j Benchmark: (Slow)

  • 15k Array – 71MB/s (Queue Depth 1)
  • 7.2k Array- 71MB/s (Queue Depth 1)

Robycopy /MT:32 Benchmark: (Fast)

  • 15k Array – 171MB/s (Queue Depth ~12)
  • 7.2k Array- 128MB/s (Queue Depth ~12)
    ,

,
Read Performance on a Local controller is fast under the workload the SAN is slow at.

,

HighPoint 2230 RAID Controller ($600)

  • 4 x 1TB 7.2k SATA RAID5

XCOPY /j Benchmark:

  • 7.2k Array – 145MB/s (Queue Depth 1) (appears to max out the SATA bus)

Best Answer

You don't mention how the networking for iSCSI is setup but if I had to guess you'll only be getting a maximum of 1Gbps of theoretical bandwidth right? So that's ~100-125MBps MAX, on top of that you have both the IP protocol and the SCSI protocol - both of which eat up both bandwidth and latency - so you're not actually doing too badly although I'd have expected more like 80MBps. Now compare that to the 300MBps minimum bandwidth you'll be seeing via SATA and PCIe - there's SO much more bandwidth and virtually no protocol translation, even then it'll be done as much lower latencies.

I think that's you issue, hope it helps.