A high impednace passive scope probe treats the cable as a capacitor rather than treating it as a transmission line. The compensation capacitance in the probe balances (with the appropriate scale factor) the capacitance of the cable and the capacitance of the scope input.
The cable on a high quality high impedance scope probe is special, it's not normal 50 ohm coax. The special cable along with the relatively short lengths of probe leads means they can get away with treating it as a capacitor at up to 100 MHz or so, much beyond that and traditional high impedance passive scope probes don't work too well.
Using a 10x probe designed for a 1 megohm scope input on a 50 ohm scope input doesn't make much sense.
The alternative to high impedance scope probing is to run a 50 ohm line to the scope and run the scope in 50 ohm mode (or use an inline terminator if your scope is too cheap to have a 50 ohm option). Compensation capacitors are no longer needed.
If 50 ohms is too low for your application then you can add a series resistor at the point of probing. For example adding a 450 ohm series resistor would give you an x10 probe with a 500 ohm input impedance. Adding a 4950 ohm series resistor would give you an x100 probe with a 5 kilohm input impedance.
The great thing about low impedance probing is you don't need compensation capacitance and the line back to the scope is a regular 50 ohm line. So it's much easier to integrate low impedance probing into your design than it is to integrate high impedance probing.
When probing high frequency signals, the standard way to allow an arbitrary length of cable between the device under test (DUT) and the scope, is to make the scope 50\$\Omega\$ input impedance, and use 50\$\Omega\$ cable.
In the ideal world, that will be good enough, Because the cable is terminated by the scope correctly, no reflections will occur at the scope, so no reflections will make it back to the driven end of the cable. The input to the cable will present a 50\$\Omega\$ load to the device being measured. We can choose to drive that load how we like.
However, in the real world, both scope and cable have a tolerance, and there will be some reflection. At very high frequencies, that could be quite large. Making the drive to the cable approximately 50\$\Omega\$ absorbs whatever does come back, improving the frequency response dramatically.
The 'tidiest' way to make this happen is to arrange for your DUT to have a 50\$\Omega\$ output impedance, to a connector. If the source of signals is low impedance, like the output of a power supply for instance, then a series 50\$\Omega\$ resistor will do nicely. If it's not convenient to use a connectered jig, then solder a 50\$\Omega\$ in line at the end of the cable.
Knowing what I did about matching, I was then surprised on my first day in a microwave lab to be shown how they probed circuits. A 50\$\Omega\$ cable, with a 470\$\Omega\$ carbon resistor soldered to the end. This was the -20dB probe.
Remember I said the input to a cable properly terminated by the scope looks like 50\$\Omega\$. The 470\$\Omega\$ resistor in series with this gives a roughly 10:1, or -20dB pot-down. It doesn't need to be matched at the sending end. It would have a flatter frequency response if it were, but another 50\$\Omega\$ resistor at the probe end would complicate the probe (obviously the cable ground is grounded to the circuit at the 'same' point, size matters!), and decrease signal or increase circuit loading for the same pickoff. For most measurements it was flat enough, and was the right price!
Best Answer
Probe cable is lossy coax.
Achieving a matched condition with an oscilloscope probe is virtually impossible because the source impedance of the circuit under test is unknown and generally different from the scope's 1MΩ or 50Ω input impedance.
On top of that the input impedance of the oscilloscope has a reactive component as do most circuits under test, which makes it impossible to achieve impedance matching over the full BW of the scope (eg 100MHz). To dampen the catastrophic frequency response as a result of the reflection on the probe cable, the latter is made of lossy coax. If you measure the core resistance between the tip and the BNC connector with a multimeter you can observe this.
You may want to read "Tektronix ABC's of Probes" primer and "The Secret World of Probes" from Doug Ford. Both are excellent primers on a correct understanding of how probes work.