Electronic – System Clock Ringing

The waveform isn't square as your scope only has 50MHz of bandwidth.

A 10MHz square wave has significant frequency content at well above that. All you are seeing is the result of low-pass filtering at ~50MHz which means you are only getting the fundamental and a couple (at most) of the odd-harmonics.

Why is changes with two probes is also similar - you are adding more capacitance across the signal, which also acts as a low-pass filter. You may also be loading the oscillator more than it can drive "nicely". Check the impedance of the probes.

Rule-of-thumb says you have to calculate transmission line effects in if your connection length is longer than 1/10th of the signal's wavelength.

Transmission lines will cause reflections where they show a sudden change in impedance. The reflected signal adds to the original, may reflect again at the transmitter side, and that way go back and forth. The result is shown in the graph: the overshoot you're talking about, and some ringing.

A 12" wire length (30cm) is 1/10 of a 3m wavelength, or 100MHz. 1MHz, and even 4MHz shouldn't give much problems here. Terminating the line, like Wouter says, could be a first measure, though. The input is probably high impedance, and that's never a good matching. Since you have three different pieces in your connection (PCB, wire, PCB) finding the characteristic impedance will be difficult. (It will also be different for the PCB and the wire, so at very high frequencies you'd also get reflections at the cable's connections.) Given the short length and the low frequency the value of your terminating resistor is not critical. 100\$\Omega\$ may be too low for the microcontroller, most of them can't supply 33mA (or even 50mA, at 5V). I would try a 1k\$\Omega\$ resistor to start with, and see how far that gets us.

edit (re question update)
Rise time seems to be 7ns. That's fast, like Kortuk says, it means you have a spectrum at least to 400MHz, and those harmonics will indeed suffer transmission line effects, even if your clock is only 1MHz. Try to filter them out, a 20MHz bandwidth (80MHz for the 4MHz clock) gives you more than enough rise time. This is a 1MHz square wave filtered with a brick wall LPF at 20MHz:

Placing a series resistor will form a first order LPF with the line's capacitance. If we estimate that at 50pF then

\$ R = \dfrac{1}{2 \pi \cdot 100 MHz \cdot 50 pF} =32 \Omega \$

gives you cutoff frequency of 100MHz. So a 33\$\Omega\$ series resistor should decrease your rise time, but leave more than enough to have a good signal at 4MHz.