Let's get some terms straight first: An Ethernet interface is typically made from two parts: a MAC and a PHY. The MAC, Media Access Controller, handles all of the packet assembly, transmission, reception, and error checking. A PHY handles all of the PHYsical transport stuff like modulating the signal, managing the DC balancing, tracking baseband wander, etc.
There are some things that both sides do, to some extent. Both MAC and PHY do some level of data error detection. This is not redundant error detection, but just error detection that is related directly to the types of things that the MAC and PHY do. Also, both MAC and PHY are dependent on the packet nature of Ethernet. The MAC because it is using the packet nature to filter, route, and manage the data. The PHY because there are certain signal modulation/demodulation functions that require packets (and the space between packets) to function correctly.
The point is: You cannot get away from packets even if you just use the PHY. Of course, the packet headers do not have to be "standard" headers. And the CRC does not have to be a standard CRC. But you are still limited to the maximum packet length and inter-packet-gap that standard Ethernet requires. (Note: You might be able to do "jumbo" packets if both PHYs support it.)
There are many benefits to using standard Ethernet packet headers, however. We would refer to this as a "Layer 2" protocol. The main benefit is that you can use standard Ethernet switches to help connect different devices together.
You mention just connecting a "TDM stream" directly (more or less) to the PHY. Every time someone has said that to me they have been talking about running multi-channel digital audio over Ethernet. If that is the case then you have a bunch of other issues, like clock synchronization and error detection that will prevent you from doing it the easy way. I won't cover audio over Ethernet more in this answer, but tell me if that is what you want to do because I can add a lot more info in that case.
Historically there have been many products that have taken some sort of data stream and ran it over Cat-5 using Ethernet PHYs and FPGAs, but without traditional MACs. Some of them have used the proper Ethernet Layer 2 or Layer 3 packets, and some of them have not. Some have also used non-Ethernet technology like ATM or FDDI. Some of them have used FPGA's, but inside the FPGA is a more traditional CPU and MAC.
I hope that at this point you have realized that what you want to do (use an FPGA and PHY to transfer a data stream over Cat-5) is difficult. Not impossible, but difficult. Let me try to explain how difficult.
First, you will have to master FPGA logic design. Of all the professional FPGA logic designers I know of, this project is beyond the ability of maybe 95% of them. These are people who have been designing FPGAs for several years or even several decades. It will take you a long time to learn FPGAs enough to design this logic. Probably years if you are doing this as a hobby.
Next, you need to learn exactly what a MAC and PHY do, and how they interface. This is not as hard as learning FPGAs, but it isn't easy either. There are a lot of basic concepts that are important, but not easily learned.
Now you'll have to design a PCB to do all of this. Designing a reliable PCB that uses FPGAs, PHYs, and does all of the proper Ethernet signal integrity stuff is also not easy. Not super hard either. But on a scale of 1-10, with 1 being super easy, this PCB would be about a 6. Not hard for an experienced professional, but definitely hard for a non-professional-EE.
At this point you probably noticed that I didn't directly answer your questions. This was on purpose. I could answer your questions, but honestly that wouldn't help you. It would be like telling you how to build the second story of a house when you haven't figured out how to build the first story or even the foundation.
Start by learning everything about designing FPGAs that you can. Also learn everything about Ethernet that you can. There are lots of online resources from app notes, datasheets, and how-to's. Go to opencores.org and study their Ethernet MAC cores. Do this diligently and in a year you might be ready. And when you are ready then you will likely know the answers to 75% of your questions-- and you will be able to put the other 25% into proper context so when someone does give you an answer it will actually be useful to you.
The USB port can be used to communicate with a design running on the FPGA, as well as for programming it. The Digilent Adept software can be used to interact with a design which implements this protocol; it's also possible to use the Digilent libraries to write your own software which uses this protocol. (I've also written a Perl module, Device::Digilent, if you'd prefer that to C.)
Information on the protocol is available at:
https://www.digilentinc.com/Data/Products/ADEPT/DpimRef%20programmers%20manual.pdf
In short, though: there is a parallel bus between the USB interface chip and FPGA consisting of:
- An 8-bit bidirectional bus, used for both addresses and data.
- Address and data strobes, and a write flag, all signalled from the host
- A "wait" signal used by the FPGA to indicate when it has serviced a read/write
The protocol used treats the FPGA as having 256 byte-wide "registers", each of which can be read or written by the host at any time. The order of events for a read is:
- Host asserts the address strobe with the write flag on, and drives the data bus with the index of the register it will be writing.
- Host asserts the data strobe with the write flag off, and the FPGA drives the data bus with the value of the register.
A write works similarly, except the second transfer has the write flag on, and the host drives the data bus with the value to be written.
Best Answer
OK, after the first couple of lines you've, correctly, ruled out microcontrollers. So, you've ruled out microcontrollers!
What you need is something that, as you've noticed, "speaks" a fast interconnect on one side, and multiple SPI buses on the other.
The only way I see that happening is an FPGA.
Attaching FPGAs to Gigabit ethernet is pretty standard, and it's very likely that whatever FPGA you pick, its manufacturer has a ethernet MAC block ready to use that you "just" have to connect to your SPI controllers that you implement inside the FPGA, and write a bit of state machine to actually stream the data (many designs even implement a small CPU like a picoRV32 in the FPGA itself to control the streaming/framing). All you then need to do is attach a PHY, and an ethernet jack that contains the magnetics. Many FPGAs come with Evalboards that carry exactly that. An ECP5 might be a cheap entry here.
You're under the wrong impression that the FPGA would bring an SGMII interface by itself - no, the whole points of FPGAs is that you can configure their logic such that you build such an interface.
If you want to go the USB route, there's also (less common) USB2 controller blocks for FPGAs, and I know of at least one free&open source USB3 controller in development (Luna by Kate Temkin). However, the usual route would be attaching your FPGA to a specialized USB controller IC thing; Cypress' FX2 and FX3 are the "classics" here.