I've recently completed a PCIe design using Altera's PCIe block. I was using the soft core on a Cyclone IV E part. I ended up in the same state and a number of things contributed:
- not enough filtering on the PHY power supplies
- a timing error caused by one of the SSTL-2-II signal traces being significantly longer than the others
- The PCIe trace pair lengths being far too long
In the end the filtering was easy to fix, as was adjusting the SDC file so that P&R could adjust the I/O delays. The nasty part was correcting the excessively long (and out of spec, I think it worked out to something like 11 inches!) PCIe trace pairs. Once this was done through a re-route the core came up immediately and performed flawlessly.
The PHY we were using also seemed to be transmitting "quietly" -- the vendor claimed they were transmitting as per spec but analysis and measurement with some very expensive rental equipment could not verify their claim. If they were transmitting to spec we should have been able to get away with the long PCIe traces.
The video is 1080p and 30fps.
So, assuming 8 bit per color channel, 3 color channels, that's
$$1920\cdot 1080\frac{\text{px}}{\text{frame}}\cdot 3\frac{\text{channel}}{\text{px}}\cdot 8\frac{\text{b}}{\text{channel}}\cdot30\frac{\text{frame}}{\text s}\approx 1.5 \,\frac{\text{Gb}}{\text s}\text.$$
That's more than Gigabit Ethernet can carry.
So, no, that's not possible with what you'd call "Ethernet", unless you compress the video, but then you need to do HD video decompression (and probably compression) on the FPGA, and that is not what one would call "light processing".
1.5 Gb/s is also faster than USB2 can work.
Thus, you're either up for USB3 or 10 Gigabit Ethernet. Both very complex buses, and I can't recommend integrating either in the first FPGA design you do – and based on the lack of concept in how you're trying to specify your project, I guess this is your first serious project.
HDMI/DVI/DP isn't all that complex a bus, if you restrict yourself to only one frame format. There's open source projects that implement that, so having an FPGA with such an interface built yourself sounds more promising than going for an universal bus.
Best Answer
It can be made to work, certainly.
There appears to be a PCIe image processing reference design listed on the product page at Terasic, which bodes well for you!