Electronic – USB Camera to GPU Memory Directly

gpupcieusbusb-host

There may be several points in this question which may not make sense to a knowledgeable person, so I do thank you for your understanding in advance.

For a computer vision project I wish to process images from a camera array (10s of cameras) simultaneously and we wish to jam image data to GPU memory as fast as possible, with RDMA or NVIDIA's gpuDirect. There exist cameras with SDI outputs and SDI capture pci-e cards with said direct GPU memory access, however, a setup like this may end up being quite expensive.

From the cost aspect USB cameras we are considering seem to be a solution, however, I am yet to find a USB host pci-e card that can support RDMA or gpuDirect.

(1) Have any of you seen a product that can support such direct access schemes? (Maybe something that emulates driver of the camera one end and can directly write to GPU memory via pci-e)

(2) Is what I am looking for even technologically possible?
When I contacted a USB host pci-e card manufacturer I have received the following response that I cannot even interpret:

All our USB 3 controllers follow the xHCI specification published by
Intel:
https://www.intel.com/content/www/us/en/io/universal-serial-bus/extensible-host-controler-interface-usb-xhci.html

When data is transferred from a usb device, the controller is given a
list of address/length pairs where the data should be written. The
the controller will issues a PCIe MEM_WR to the address it is given
with the data. I hope that helps you decide if you can do what you
want with our host controller

So do you guys think it is possible or not (By possible I do not mean designing my own custom usb host capable of only doing what have described but with what's available already in market)?

Best Answer

Haven't heard of this, but looking in GPUdirect description and developer site, it seems like it is needed that the device driver for the 3rd party device, should be capable of assign GPU address to the device, in order to perform DMA operations (sections 1.2 and 1.3).

I am not used to PCIe, but I guess that as you plan to use USB cameras, they will be communicating with the USB driver in your system. So 2 options I see are:

  1. The one you stated, build your own USB host with RDMA capabilities, or
  2. modify/create a USB driver for your cameras including Nvidia Kernel driver features, to allow these features to be used from a CUDA program.
Related Topic