Freebsd – How to setup PCI passthrough for the HighPoint 1144A USB 3.0 card under ESXi 5.1

freebsdpassthroughpci-expressusbvmware-esxi

I have an Apple Mac Pro (MacPro5,1) running VMware ESXi 5.1.0 (799733) with a HighPoint RocketU 1144A USB 3.0 PCI-express card installed. Given that VMware ESXi 5.1 doesn't support USB 3.0 for host-attached devices, I am trying to attach the USB 3.0 PCIe card to a virtual machine using DirectPath I/O PCI passthrough.

It took me a number of different configurations, but I finally managed to configure the HighPoint RocketU 1144A USB 3.0 PCI Express card for DitectPath passthrough:

DirectPath Configuration Screenshot

DirectPath Status Screenshot

The first few times I tried, after rebooting I still received the "The host needs to be rebooted to apply configuration changes" warning, even after rebooting the host. Finally I found that by moving the USB 3.0 card to the 3rd PCIe slot, I was able to get the passthrough configuration to stick.

I also note that the device shows up as a "ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller" when it actually should have 4xASM1042 chips on it. I also had to pass-through the PLX Technology, Inc PEX 8609 8-lane, 8-Port PCI Express Gen 2 Switch to make the configuration changes stick.

Now, however, I am stuck trying to attach the ASMedia Technology ASM1042 device to a FreeBSD (FreeNAS) virtual machine. When I add the device as a PCI passthrough device to the virtual machine, I am unable to power on the virtual machine:

VM Power On Error Screencapture

VMware ESX unrecoverable error: (vcpu-0) PCIPassthruChangeIntrSettings: 09:00.1 failed to register interrupt (error code 195887105)

I tried passing through only the ASMedia ASM1042, only the PLX PEX 8609, and both the two devices. None of these three configurations worked.

Googling for that error code led me to this forum post in German which by using Google Translate I was able to extract the following information:

You have to disable MSI and forcing the card to get a INTx interrupt access, we go.
Can you read here on page 5. http: / / www.vmware.com/pdf/vsp_4_vmdirectpath_host.pdf you
following parameters have to manually add to your VM config:
pciPassthru0.msiEnabled = "FALSE" I had to run the whole thing also virtualisert to ESXi
and then I walked the closed system of ESXi but with the time on your nerves because you
problems but not actually really engage. can I have now migrated to KVM (Proxmox VE) where
the rich and Cine S2 V5.5 also successful at the next VM (with MSI). The whole runs super
stable.

Following those instructions I added

pciPassthru0.msiEnabled = "FALSE"
pciPassthru1.msiEnabled = "FALSE"

to my .vmx file, and this did enable me to boot the FreeBSD 8.3 (FreeNAS 8.3.1) VM… however the virtual machine hung at boot here:

FreeBSD Hang Screencapture

I expect that I need to make some other change. The VMware PDF document they linked to describes "linked devices", and I expect there's some sort of linking going on whereby the 4xASM1042 chips (because the 4 ports on the USB 3.0 card each have their own channel) need to be linked through the switch… or something… but I am unsure of technically what is going on or how to proceed.

How can I resolve these problems and get passthru working for the HighPoint 1144A USB 3.0 card under EXSi 5.1?

Best Answer

Honestly, you're wayyyy outside of the norms for an ESXi server setup. I'm certain that this would not be a supported configuration in any way.

I understand that you're probably trying to get to an all-in-one (ZFS?) storage solution, but I doubt that you'll find a lot of mindshare on this type of arrangement.

The two things I can recommend are checking the VMware Hardware Compatibility List (HCL) for notes on the HighPoint USB card and also upgrading your ESXi to the current build, #914609.