The AD9862 has an input impedance of 200 ohms typical and that is of some interest but not of major importance when it comes to the outside world interfacing. Generally speaking, a chip input impedance of infinity is easier to work with - in this way it can be ignored providing the chip doesn't sit more than a few inches away from the resistor/components that terminate the incoming line.
I say a few inches, but that really does depend on the frequency(ies) you are receiving. Let's say max frequency of interest is 300MHz - it has a wavelength of 1 metre and a rule of thumb says that if your pcb tracking is less than one-tenth of the wavelength then you are not going to have problems feeding 10cm (4 inches) to the chip from the line terminator.
Other folk may say less but it is just a rule of thumb. So the chip PCB tracks being matched to a certain impedance are not that critical either providing the rule of thumb is met. The fact that the chip has an input impedance of 200 ohm slightly helps this - a distributed load termination (instead of a single 50 ohm or 75 ohm termiantor) is allowable too (rule of thumb etc).
Now the balun. Yes it says it is a 75ohm balun but at the end of the day it's a transformer with nothing normally inherently 75ohm or 50 ohm ish about it. It says it is a 1:1 impedance device which means to me that if there is 50 ohms (or 75 ohms) on one side of the transformer, this impedance is reflected to the other side for the normal range of frequencies that it is intended for.
The impedance on the chip side of the balun is 200 ohms (chip) + 50 ohms (R4) + 50ohms (R5) = 300 ohm. Again, this is not going to work as well as an impedance of 75 ohm but it probably won't make a massive deal - it's not optimum but it's very difficult to tell from the balun spec how far off optimum it will be. My guess is that it's not perfect but you probably won't deteriorate signals by more than a couple of dB.
This 300 ohm is reflected onto the primary side of the balun and becomes in parallel with 50 ohms (R3). The net impedance looking in to the circuit is now about 43 ohms. I have to say that clearly this would be nicer if it were closer to 50 ohms BUT, I don't know the impedance of the cable this circuit is intended for. It could be 50 ohms and in which case there will be a tendency for standing waves and reflections up and down the cable but nothing so severe it will kill operations. The cable could be 45 ohms cable (not unheard of).
If you are making a circuit, I'd use a 62 ohm for R3 and the impedance presented at the input would be about 51.4 ohms.
Remember, the most important part of this design is to match the impedance of the cable to prevent serious reflections. It doesn't matter if the matching impedance is distributed between R3, R4, R5 and the chip providing the PCB traces are not excessively long AND the PCB traces needn't be designed to be exactly 50 ohms providing the lengths are short.
Analog Devices offers this part, the ADUM4160, which offers complete isolation between USB input and output, while providing a data path. However, it only supports slow speed 12 Mbps speeds, which won't be enough for wide bandwidth operation.
There are several pre-built isolation boards based on this part, which might be worth looking at.
Best Answer
I would recommend using a PLL chip that is capable of generating a 28.8 MHz output and can do so with a 10 MHz input.
Take a look at http://www.analog.com/en/products/clock-and-timing/plls.html . The ADF4360 could be a good choice.