Electronic – Streaming camera solutions for RaspberryPi MCU

cameradevicemicrocontrollerraspberry pi

I'm trying to get a Raspberry Pi MCU to stream an A/V feed over the internet to a web server where I'll have some homegrown software capturing each frame and doing analysis with it.

My general strategy here is:

  • Get a camera and microphone sending audio and video signal to one of the Pi's onboard USB ports
  • If my choice of technology is correct, the Pi (which would be running GNU/Debian linux) will have the right drivers to ingest the streaming frames and send them off to a tool that would then be able to forward them on to a WiFi or Ethernet adapter
  • The adapter sends the frames on their way over TCP/IP and then I have to figure out how to handle them server-side

So as far as getting the frames to the MCU, there's two main issues here: (a) the choice of camera and microphone, and (b) the drivers for connecting the cam/mic to the Pi's USB port. There's also the issue of A/V encoding as well as synchronizing the video and the audio feeds together.

I'm beginning to think that a "web cam" might be my best bet, because most web cams have built in microphones, automatically synchronize the audio and video feeds together, encode the A/V frames, and come with drivers – so on the outset they appear to be perfect solutions. But I'm open to anything that gets the job done efficiently and in keeping with best practices.

Here's the kicker – I might (doubtful) want to try and sell this device, so I can't simply take some Logitech web cam and use it due to obvious licensing issues. So that's a huge constraint.

I should also mention that price is sort of a factor as well – I'm comfortable in the $20 – $40 USD range, but over $50 and I'll start to cringe.

The best web cam I've been able to find is from Jameco.

But I am wondering how hardcore, real electronic wizards would attack this problem given the constraints I've mentioned. I'm a software developer so I'm out of my element here in the electronics arena.

Unless I'm missing something, then unless I find a camera that has integrated/synchronized audio, performs A/V encoding for me, and has device drivers all ready to go, then I'll have to do all this myself. If that's the case, I'd be interested in hearing what some solutions could be. I've looked into MPEG encoder chips, so I know it's possible to do this, I just don't want to work any harder then I have to here.

Thanks in advance from a newbie who's fumbling around in the dark here.

Best Answer

I might (doubtful) want to try and sell this device, so I can't simply take some Logitech web cam and use it due to obvious licensing issues. So that's a huge constraint.

Basically you will be creating an IP Camera. The Raspberry Pi is probably the cheapest and easiest way to prototype this, however there are other boards like the Beagle Boards.

Once you get a prototype going, then you may consider creating your own all in one device that uses an ARM or DSP processor. For example I would probably use some type of Serial JPEG Camera Module, a cheap microphone, and the cheapest ARM processor that will fit these requirements. But for a prototype, the Raspberry Pi and a cheap usb web cam is probably the cheapest and quickest way to get started and get your software going. You may even to be able to find a cheap no-name usb camera from china that will work for this so you can resell it in small quantities.

So as far as getting the frames to the MCU, there's two main issues here: (a) the choice of camera and microphone, and (b) the drivers for connecting the cam/mic to the Pi's USB port.

Here is the huge list of devices that work with the Raspberry Pi: RPi Verified Peripherals. The USB Webcams section lists both working and problem units. Along with a bit of extra info. After you find a unit that fits your requirements (price, etc.) I would try to double check via Google to verify someone has used it and it does indeed work; although this is probably unnecessary.

I'm sure there are other units that work that haven't been tested, the two things that will help you is to make sure it is Linux compatible, and that there are ARM drivers available.

There's also the issue of A/V encoding as well as synchronizing the video and the audio feeds together.

As I mentioned in my comment, the RP really won't have any issues handling this part. It has more than enough processing power to handle most all A/V formats.

the Pi (which would be running GNU/Debian linux) will have the right drivers to ingest the streaming frames and send them off to a tool that would then be able to forward them on to a WiFi or Ethernet adapter

Basically the RP is just going to be a linux computer that is connected to the internet and has the camera and microphone plugged into it. You will install and configure libasound2-dev (for the audio) and FFMPEG to stream everything.

Then this will just be like any other server online (you may need to configure your router and port forwarding etc. to get it to be visible on the web,) and according to this, you and other computers will just access it by going to h**p://YOUR_WEBCAM_SERVER/webcam.mjpeg

Helpful links:

Related Topic