I'm trying to figure out how to set up a simple storage system which exports block storage over Infiniband, using either SRP or iSER.
I'm very early in the process, and at the moment I'm basically just looking for a tutorial on the level of, "You have two servers on your fabric: here's how to export a block storage device from one to the other." Preferably on RHEL or variants thereof. Any pointers?
Best Answer
Well, to be frank I went the simple route and happily used iSCSI over IP over IB and it worked easily and performed well:
Infiniband IP setup ultra-quick primer.
first...
performance settings:
YMMV with the IB controller model, driver, etc.
IP Settings :
Some documentations :
http://support.systemfabricworks.com/lxr/#ofed+OFED-1.5/ofed-docs-1.5/ipoib_release_notes.txt
http://www.mjmwired.net/kernel/Documentation/infiniband/ipoib.txt
iperf, 4 threads :
Total aggregated bandwidth 1.3 GB/s, definitely better than 10 GigE.