Let's get some terms straight first: An Ethernet interface is typically made from two parts: a MAC and a PHY. The MAC, Media Access Controller, handles all of the packet assembly, transmission, reception, and error checking. A PHY handles all of the PHYsical transport stuff like modulating the signal, managing the DC balancing, tracking baseband wander, etc.
There are some things that both sides do, to some extent. Both MAC and PHY do some level of data error detection. This is not redundant error detection, but just error detection that is related directly to the types of things that the MAC and PHY do. Also, both MAC and PHY are dependent on the packet nature of Ethernet. The MAC because it is using the packet nature to filter, route, and manage the data. The PHY because there are certain signal modulation/demodulation functions that require packets (and the space between packets) to function correctly.
The point is: You cannot get away from packets even if you just use the PHY. Of course, the packet headers do not have to be "standard" headers. And the CRC does not have to be a standard CRC. But you are still limited to the maximum packet length and inter-packet-gap that standard Ethernet requires. (Note: You might be able to do "jumbo" packets if both PHYs support it.)
There are many benefits to using standard Ethernet packet headers, however. We would refer to this as a "Layer 2" protocol. The main benefit is that you can use standard Ethernet switches to help connect different devices together.
You mention just connecting a "TDM stream" directly (more or less) to the PHY. Every time someone has said that to me they have been talking about running multi-channel digital audio over Ethernet. If that is the case then you have a bunch of other issues, like clock synchronization and error detection that will prevent you from doing it the easy way. I won't cover audio over Ethernet more in this answer, but tell me if that is what you want to do because I can add a lot more info in that case.
Historically there have been many products that have taken some sort of data stream and ran it over Cat-5 using Ethernet PHYs and FPGAs, but without traditional MACs. Some of them have used the proper Ethernet Layer 2 or Layer 3 packets, and some of them have not. Some have also used non-Ethernet technology like ATM or FDDI. Some of them have used FPGA's, but inside the FPGA is a more traditional CPU and MAC.
I hope that at this point you have realized that what you want to do (use an FPGA and PHY to transfer a data stream over Cat-5) is difficult. Not impossible, but difficult. Let me try to explain how difficult.
First, you will have to master FPGA logic design. Of all the professional FPGA logic designers I know of, this project is beyond the ability of maybe 95% of them. These are people who have been designing FPGAs for several years or even several decades. It will take you a long time to learn FPGAs enough to design this logic. Probably years if you are doing this as a hobby.
Next, you need to learn exactly what a MAC and PHY do, and how they interface. This is not as hard as learning FPGAs, but it isn't easy either. There are a lot of basic concepts that are important, but not easily learned.
Now you'll have to design a PCB to do all of this. Designing a reliable PCB that uses FPGAs, PHYs, and does all of the proper Ethernet signal integrity stuff is also not easy. Not super hard either. But on a scale of 1-10, with 1 being super easy, this PCB would be about a 6. Not hard for an experienced professional, but definitely hard for a non-professional-EE.
At this point you probably noticed that I didn't directly answer your questions. This was on purpose. I could answer your questions, but honestly that wouldn't help you. It would be like telling you how to build the second story of a house when you haven't figured out how to build the first story or even the foundation.
Start by learning everything about designing FPGAs that you can. Also learn everything about Ethernet that you can. There are lots of online resources from app notes, datasheets, and how-to's. Go to opencores.org and study their Ethernet MAC cores. Do this diligently and in a year you might be ready. And when you are ready then you will likely know the answers to 75% of your questions-- and you will be able to put the other 25% into proper context so when someone does give you an answer it will actually be useful to you.
Although the lines are routed as differential pairs, it's not necessarily a killer to your kind of application. The amount of crosstalk is actually still likely to be quite small, as diffpairs are often routed closer to the groundplane than to each other. It's not like they are a twisted wire with no shield, where the signals are effectively much better coupled to each other. And the traces are likely to be relatively short I imagine (if you are contemplating parallel transmission).
I would suggest you could happily set up a 25MHz parallel bus, with a series-terminated clock to signal the device at the far end to latch the data. Drive the data on one edge, latch it on the opposite edge, you'll have 20ns of time for crosstalk and reflections to settle out). You could drive the negative side of the clock driver to '0' and place it away from the databus to keep that signal as clean as possible. It doesn't sound that different from how we used to route asynchronous SRAM databusses (all the wires run very close together with an edge triggered write-enable line) and that used to work fine :)
Best Answer
TL;DR: The terms you need to know are PHY, SERDES, and serial transceiver.
I don't think the answer to "what kind of PLL" would be helpful, since anyone wanting to use USB 2 in their design wouldn't be ordinarily building a clock recovery circuit from scratch, and thus there's no need to bother with PLLs or anything such unless you're designing silicon or otherwise need/want to implement the physical layer from scratch.
Typically, the function that implements the physical layer of some communication protocol is called a PHY. You can get PHYs as stand-alone chips, or built into FPGAs, ASICs and microcontrollers. So, if you have an FPGA without a built-in USB 2 PHY, you can get an external one for a couple bucks (e.g. TUSB1210). You can use such PHYs without implementing the entirety of the USB stack, of course. Same goes for Ethernet PHYs, although those don't support bidirectional communication over a single twisted pair. On the other hand, Ethernet twisted pair cabling in your speed range is so ubiquitous and cheap that using it may actually be cheaper than something custom, especially if you need potentially long cables.
The big advantage of using a PHY for a widely used consumer serial protocol like USB is that it's cheap. You can go faster by using specialized single-chip SERDES functions. SERDES stands for SERializer/DESerializer. They are faster but not as cost efficient. For example, DS92LV16 would do what you want: you can connect the transmitter and receiver together on each end, RS-485 style, and use the transmitter/receiver power-down controls in half-duplex mode. The minimum data rate on that device, though, is 50MByte/s, i.e. it needs a 16-bit word clock of at least 25MHz, up to 80MHz. The bit rate is up to 2.56Gbps. This particular one needs a "rough" reference clock on both ends, i.e. if you decide to use 25MHz word clock, you will need a 25MHz reference clock on both ends of the connection from a local source, but it doesn't need to be terribly accurate: the PLL compensates for up to +/-5% absolute clock frequency error between transmitter and receiver.
And finally, you'd definitely want to consider the on-board serial transceivers that may be available in the FPGA proper. Don't let the name mislead you: they are usually gigabit speed devices, not much related to asynchronous UARTs or anything like that. These days, there plenty of affordable FPGAs with built-in transceivers that can be used exactly for what you intend. Using an external PHY or SERDES would make sense if you wanted to limit yourself to FPGAs without built-in serial transceivers (for whatever reason), and/or wanted to add a bit of a retro feel to the device. Finally, using a USB 2 HS PHY with the board laid out for OTG operation may allow you to use some ad-hoc protocol for the time being, but eventually support actual OTG operating mode once you get it implemented (in the firmware of the CPU/core and the bitstream). This may be useful in some cases to extend product functionality while keeping the same board design, but would require careful attention to the requirements of USB 2 OTG spec to ensure that the board itself would support necessary functionality (power switching and such).