SATA and SAS are point-to-point links. So the total distance is irrelevant, only the link lengths.
The SAS Protocol is used to connect to SAS devices. SATA is used to talk to SATA devices. SAS HBAs and Active Backplanes (aka SAS Expanders) must be able to speak SATA, so it can communicate to SATA drives; and this SATA connection is sent back over the SAS backhauls via the SATA Tunneling Protocol.
I haven't checked the specs yet, but it's likely that the expanders are SAS 1.0 or 1.1 and don't support 6Gb SATA (they would only support 1.5Gb SATA). I have further seen issues with these "older" generation SuperMicro Expanders when using SATA disks the whole chip slows to 1.5Gb instead of just the one lane. This would mean you could only get 1.5Gbx4 to all the drives in the rear of the chassis, and only under ideal conditions.
First, stop and think about your realistic goals...
The short answer to your question is that SAS is a switching protocol. The drives connected via expanders are all accessible and can be addressed transparently by a single controller.
However, the rest of your questions are going to be a function of your particular controller, your storage architecture and the backplanes in the storage enclosure(s) you end up using.
It's not possible to give a specific answer without more information.
- How many disks do you need?
- What RAID controller or SAS HBA are you using?
- Will these disks be internal or external?
- What specific SAS expander card are you looking to use (it's not like there are many on the market)?
- What server and enclosure hardware are you considering?
- What is the application/reason for this setup? (ZFS? Storage Spaces?)
Let's use your example of a controller with 4 x SAS SFF-8087 ports. These are meant for communication inside of a chassis. Each SFF-8087 port is a 4-lane connector that can accommodate 4 drives. 16 drives total, in your case... Servers typically only have 4, 8, 12, 16, or 24/25 internal drive bay capacities.
At the 4 and 8-drive level, the internal server drive backplane will have a 1:1 lane-to-drive ratio and typically have one or two SFF-8087 ports. No oversubscription!
For 12-port servers, the expander is usually built into the backplane, and the input will be a single SFF-8087 port. Oversubscription!
For servers with 16 internal bays, you may see four separate SFF-8087 ports intended to be served by a single or multiple SAS/RAID controllers. But at 16-ports, you may have an expander on the backplane instead.
Almost every 24 or 25-bay server I've encountered has expanders built onto the drive backplane and accommodate one or two SFF-8087 ports. The exception is something like the HP ProLiant DL370 G6, which could accept three 8-bay drive cages, each requiring 2 SFF-8087 ports.
So when do people use PCIe card SAS expanders?
Note: HP makes a great PCIe SAS expander. There's an entire community built around its use for storage projects.
The most common case I see is a motherboard or PCIe controller with TWO SFF-8087 ports and a need to connect more than 8 drives split across multiple drive cages. I can't envision something like your scenario where you have 16 ports available on a controller and would want to multiply that via expanders to accommodate more internal drives. If you're working with a server with more than 16 disks, chances are that you'll be encountering a backplane expander somewhere. This would make the PCIe expander unnecessary.
This is all different when you look at external storage, though. The transport there would be a SAS SFF-8088 external cable. Most external drive shelves/JBODs have expanders built-in.
Example RAID configuration output from an HP Smart Array P410i controller connected to two drive cages inside of an HP ProLiant DL370 G6 server through an HP SAS expander card. Note the card is represented as an enclosure (Vendor ID HP, Model HP SAS EXP Card) and has its own SAS WWN.
Smart Array P410i in Slot 0 (Embedded) (sn: 5001438006503640)
array A (Solid State SATA, Unused Space: 0 MB)
logicaldrive 1 (223.5 GB, RAID 0, OK)
physicaldrive 5C:2:8 (port 5C:box 2:bay 8, Solid State SATA, 240.0 GB, OK)
array B (SAS, Unused Space: 0 MB)
logicaldrive 2 (68.3 GB, RAID 1, OK)
physicaldrive 7C:2:1 (port 7C:box 2:bay 1, SAS, 72 GB, OK)
physicaldrive 7C:2:2 (port 7C:box 2:bay 2, SAS, 72 GB, OK)
array C (SATA, Unused Space: 2651025 MB)
logicaldrive 3 (1000.0 GB, RAID 1+0, OK)
logicaldrive 4 (500.0 GB, RAID 1+0, OK)
physicaldrive 2C:1:1 (port 2C:box 1:bay 1, SATA, 1 TB, OK)
physicaldrive 2C:1:2 (port 2C:box 1:bay 2, SATA, 1 TB, OK)
physicaldrive 2C:1:3 (port 2C:box 1:bay 3, SATA, 1 TB, OK)
physicaldrive 2C:1:4 (port 2C:box 1:bay 4, SATA, 1 TB, OK)
physicaldrive 3C:1:5 (port 3C:box 1:bay 5, SATA, 1 TB, OK)
physicaldrive 3C:1:6 (port 3C:box 1:bay 6, SATA, 1 TB, OK)
Expander 250 (WWID: 50014380133153E6, Box: 1)
Enclosure SEP (Vendor ID HP, Model HP SAS EXP Card) 248 (WWID: 50014380133153E5, Box: 2)
SEP (Vendor ID PMCSIERA, Model SRC 8x6G) 249 (WWID: 500143800650364F)
Best Answer
Yes, that's all correct. I suppose it's not clear from the Supermicro product pages.
SAS SFF-8087 is a 4-lane multi-connection transport. It's typically used by RAID controllers and disk backplanes. You probably want to avoid anything that requires fanout cables. Expander backplanes can have some problems, depending on what types of disks and controllers will be in use.
Also this: How exactly does a SAS SFF-8087 breakout cable work? + RAID/connection questions