Electronic – Need to use decoded image as reference image, not original. Why

Okay, this is more of an opinion than a typical definitive stackoverflow type answer, but here it goes.

I tried using a Raspberry Pi for image processing for my capstone project. Perhaps it was just the libraries, or having a bunch of stuff on USB, but I was extremely unsatisfied with its performance. It had a hard time streaming VGA images at any appreciable framerate, let alone doing any sort of image processing with them and then streaming it over.

I mean, there's a lot of variables (especially wrt code optimization and implementation), and I can't speak objectively and quantitatively, but you get the performance you would expect out of a computer you would only paid $30 for. I would suggest taking a look at Odroid X2 which has far more hardware kick, although I haven't used one myself.

You could buy one and try it and see if it meets your demands; you'd be out only $30 if it didn't. But based on my own personal experience with this thing I really doubt this thing can do what you want at 120x a second.

If you have a bit of budget, the Xilinx Spartan 6 FPGA is quite well suited to video processing. You can even buy a complete kit for about $5K. The link has some reference designs which you can compare to what you are trying to accomplish.

Since you don't know exactly what amount of processing is required, it might be better to err on the side of more capability than you think you'll need, so you don't run out of room on a given approach. You can always downsize to a smaller FPGA.