Slow storage spaces direct S2D setup

storage-spaceswindows-server-2016

I have two Dell R710's with 8 Cores 64+ GB RAM, Flashed Dell H200 with 9211 IT mode, 4 teamed gbe nics, Server 2016
One server has:
2 x 480 GB SSD's
3X 2TB HDD
1x 3 TB HDD
The other
2 x 240 GB SSD
4 x 2 TB HDD

I have an external server with 2012 R2 SS with just 2x4TB Drives, and another with 6+ HDD (no SSDs).

Transfers from the external servers to each other top out the 1GB connections they have, 100MB/s file transfers. From those to my S2D cluster I top out around 14MB/s or so.

I can't find anything to help me diagnose what is soooo slow. Column count 4, 4k, Mirrored, Cluster validion report check, What can i do to figure out why this is soo slow? I know 10Gbe with RDMA is 'recommended' but I'm not sure why it is this bad really. This was going to be my new cluster @ home but Its not really usable like this. Help? Desperate…..

Also windows VM shows 500ms average response time from the disk. Yikes

Best Answer

You really need RDMA-capable 10 GbE for your backbone connectivity. 4x1GbE is simply not enough for Storage Spaces Direct. Get Mellanox ConnectX3 or similar cards, you can get used/refurb cheap from eBay.

P.S. S2D doesn't shine on a two-node config...