QoS (802.1p) for iSCSI Traffic on Cisco Nexus 3048 – Configuration Guide

ciscocisco-nx-oscosiscsiqos

We're building a new core for our University and one thing that bugs me is the iSCSI network.

We already considered Jumbo Frames and Flow Control on the iSCSI ports, but one thing remains for debate, and it's the QoS Feature.

Our new core consist of two Cisco Nexus 3048. There will be two different iSCSI VLAN's for MPIO, one VLAN will be in the first switch and the other one on the the second. The Jumbo Frames and Flow Control seems trivial, but QoS does not appear to be simple.

I was unable to find any guidelines for iSCSI on the Nexus 3048 models. There are a lot of information for the 5000 and 7000 series but many features are missing on the 3000 series, like the QoS traffic match with the iSCSI protocol.

Another thing that I'm unsure is about CoS (Class of Service) Classification. Is this a requirement requirement? I don't know where iSCSI should be placed.

Thanks in advance,
V.

EDIT with Answers:

  1. A couple of design questions: – Are you mixing iSCSI and other data
    in this cluster (I assume you will user VPC)? – Are you mixing iSCSI
    and other data in the same port with tagged frames?

Data is mixed on the switch. VM's Network and Access will reach the
core. But the iSCSI traffic has dedicated ports with larger MTU in
mind and specific VLANs with untagged data, complete Layer 2
segregation. VPC will not be used for iSCSI since we're deploying
MPIO, so there's no need. The storage is connected via SFP+ on the
10GbE port and the hypervisor hosts on the downlink GbE ports. There's
5 hypervisor hosts in total, each one with two cards, one for each
switch. VPC is only used in the VM's Network and the access uplink.

  1. What is the actual issue you are experiencing? Is this a performance problem? Generally, it is not advised to have your ISCSI network on the same hardware that will provide connectivity to the rest of your network. You should really be using a dedicated switch (2 for redundency) for your ISCSI network and segrate that from the normal user data.

We don't have any issues at this moment since this core is not in
production, we just want to make it right before putting it on
production. I'm aware that separate fabrics should be deployed for SAN
traffic. With cheaper and common switches this is true and we rolled
iSCSI this way on older equipment. But this switch is a unified fabric
model, so it's developed to have mixed SAN and LAN traffic on the same
box. The model we choose, the Nexus 3048 is capable of delivering
extremely low latency and lossless iSCSI and here is where we need to
configure it properly to make everything work as expected.

Thanks for the questions guys. Hope this helps.

Best Answer

If the clasification isn't available on the switch model, then you need to look at something like ACL-based QoS classification (Layers 2, 3, and 4) so that you can mark what traffic, by IP address range for example and tell the hardware how you want to prioritise the traffic with mappings.

CoS is for traffic that is layer 2 only. If you need to prioritise traffic that is going to get routed, then you would use DSCP which is layer 3.

So CoS will put the marking on an Ethernet frame header and DSCP would do it within the space on a IP header. If the traffic is getting routed, the Ethernet frame get's stripped so you would loose your CoS marking and would rely on DSCP within the IP header.

CoS and DSCP Marking on Ethernet Frame Header and IP Header

You achieve more granuality with DSCP compared to CoS as you can set more options because of the difference in bit size since CoS uses only 3 bits to set markings while DSCP uses 6 bits.

All in all, I would advise you seperate your ISCSI network from your core infrastrucure and then you wouldn't even need to worry about QoS.

Related Topic