Dell VRTX – slow cluster shared storage

blade-serverdell-percfailoverclusterhyper-v-server-2012windows-server-2012

I have a brand new Dell VRTX box set up as a Failover Cluster running HA Hyper-V virtual machines. This is my first time setting up clustering, and my first time with one of these boxes, so I'm sure I've missed something.

The virtual machines are experiencing high disk latency and bad performance when accessing their VHD(x) files located on a Cluster Shared Volume.

The VRTX has 10 x 900 GB 10K SAS drives in RAID 6 configuration, and the VRTX has the redundant Shared PERC 8 controllers. Both blades have full access to the virtual disks. There are two M520 blades installed, each with 128 GB RAM. MPIO is configured for the PERC 8 controllers. Operating system on the blades is Server 2012 (NOT R2).

The RAID 6 array is split into a small (8 GB) volume for cluster quorum witness and a large (6.5 TB) volume for a Cluster Shared Volume (mounted on the nodes as C:\ClusterStorage\Volume1)

An example of slow disk access: logging into a Server 2012 VM and having Server Manager come up automatically. Disk access goes to 100%, with write speeds at 20 MB or so, read speeds of 500 KB or so, and Average Response Time of over 1000 ms, sometimes spiking at 4000-5000 ms or so. It's the latency that really worries me.

Is there something specific I should look at in my configuration? It doesn't seem to matter whether I use VHD or VHDX, dynamic or static.

Best Answer

I have experienced the exact same performance issue with a VRTX with the Dual SPERC8. What I have done to work around this at the moment is, change the dual config to a single config. This way I am able to use write-back, which performs way way better.

  • The exact steps:
    • Remove the second SPERC 8 controller
    • Remove the second expander
    • Re-cable the internal SAS connections
    • Downgrade chassis to 1.25 (works the same as upgrading, no special steps needed)
    • Delete all VD's (backup data/VM's if needed)
    • Power cycle entire VRTX (remove and reconnect power cables to be sure)
    • create VD('s) with write-back enabled

To see the performance difference check this/my thread at: http://en.community.dell.com/support-forums/servers/f/906/t/19587459.aspx

Update:
Test results:
Dual PERC / RAID6 / Write Through: Read 2500 MB/s Write 200 MB/s
Dual PERC / RAID10 / Write Through: Read 2500 MB/s Write 400 MB/s
Single PERC / RAID6 / Write Back: Read 2500 MB/s Write 2700 MB/s

As long Dual PERC is bound to Write Through policy, I would stick with a Single PERC setup

Related Topic