The probe is connected directly at the pin of the oscillator. I soldered a small wire onto the ground pad and I connect the ground clip there. This is VERY near where I'm probing.
No. No. NO! If you are using a long ground clip (and even connect it to a wire?), it does not matter how close the point is, you still add a huge amount of inductance. To be sure, use something like this:
This also could be a probe compensation fault. Be sure to check that.
Image sources:
1
2
Talking about signal termination is like opening a can of worms. This is a HUGE subject that is difficult to summarize in just a couple hundred words. Therefore, I won't. I am going to leave a huge amount of stuff out of this answer. But I will also give you a big warning: There is much misinformation about terminating resistors on the net. In fact, I would say that most of what's found on the net is wrong or misleading. Some day I'll write up something big and post it to my blog, but not today.
The first thing to note is that the resistor value to use for your termination must be related to your trace impedance. Most of the time the resistor value is the same as your trace impedance. If you don't know what the trace impedance is then you should figure it out. There are many online impedance calculators available. A Google search will bring up dozens more.
Most PCB traces have an impedance from 40 to 120 ohms, which is why you found that a 1k termination resistor did almost nothing and a 100-ish ohm resistor was much better.
There are many types of termination, but we can roughly put them into two categories: Source and End termination. Source termination is at the driver, end termination is at the far end. Within each category, there are many types of termination. Each type is best for different uses, with no one type good for everything.
Your termination, a single resistor to ground at the far end, is actually not a very good. In fact, it's wrong. People do it, but it isn't ideal. Ideally that resistor would go to a different power rail at half of your power rail. So if the I/O voltage is 3.3v then that resistor will not go to GND, but another power rail at half of 3.3v (a.k.a. 1.65v). The voltage regulator for this rail has to be special because it needs to source AND sink current, where most regulators only source current. Regulators that work for this use will mention something about termination in the first page of the datasheet.
The big problem with most end-termination is that they consume lots of current. There is a reason for this, but I won't go into it. For low-current use we must look at source termination. The easiest and most common form of source termination is a simple series resistor at the output of the driver. The value of this resistor is the same as the trace impedance.
Source termination works differently than end termination, but the net effect is the same. It works by controlling signal reflections, not preventing the reflections in the first place. Because of this, it only works if a driver output is feeding a single load. If there are multiple loads then something else should be done (like using end termination or multiple source termination resistors). The huge benefit of source termination is that it does not load down your driver like end termination does.
I said before that your series resistor for source termination must be located at the driver, and it must have the same value as your trace impedance. That was an oversimplification. There is one important detail to know about this. Most drivers have some resistance on it's output. That resistance is usually in the 10-30 ohm range. The sum of the output resistance and your resistor must equal your trace impedance. Let's say that your trace is 50 ohms, and your driver has 20 ohms. In this case your resistor would be 30 ohms since 30+20=50. If the datasheets do not say what the output impedance/resistance of the driver is then you can assume it to be 20 ohms-- then look at the signals on the PCB and see if it needs to be adjusted.
Another important thing: when you look at these signals on an o-scope you MUST probe at the receiver. Probing anywhere else will likely give you a distorted waveform and trick you into thinking that things are worse than they really are. Also, make sure that your ground clip is as short as possible.
Conclusion: Switch to source termination with a 33 to 50 ohm resistor and you should be fine. The usual caveats apply.
Best Answer
Rule-of-thumb says you have to calculate transmission line effects in if your connection length is longer than 1/10th of the signal's wavelength.
Transmission lines will cause reflections where they show a sudden change in impedance. The reflected signal adds to the original, may reflect again at the transmitter side, and that way go back and forth. The result is shown in the graph: the overshoot you're talking about, and some ringing.
A 12" wire length (30cm) is 1/10 of a 3m wavelength, or 100MHz. 1MHz, and even 4MHz shouldn't give much problems here. Terminating the line, like Wouter says, could be a first measure, though. The input is probably high impedance, and that's never a good matching. Since you have three different pieces in your connection (PCB, wire, PCB) finding the characteristic impedance will be difficult. (It will also be different for the PCB and the wire, so at very high frequencies you'd also get reflections at the cable's connections.) Given the short length and the low frequency the value of your terminating resistor is not critical. 100\$\Omega\$ may be too low for the microcontroller, most of them can't supply 33mA (or even 50mA, at 5V). I would try a 1k\$\Omega\$ resistor to start with, and see how far that gets us.
edit (re question update)
Rise time seems to be 7ns. That's fast, like Kortuk says, it means you have a spectrum at least to 400MHz, and those harmonics will indeed suffer transmission line effects, even if your clock is only 1MHz. Try to filter them out, a 20MHz bandwidth (80MHz for the 4MHz clock) gives you more than enough rise time. This is a 1MHz square wave filtered with a brick wall LPF at 20MHz:
Placing a series resistor will form a first order LPF with the line's capacitance. If we estimate that at 50pF then
\$ R = \dfrac{1}{2 \pi \cdot 100 MHz \cdot 50 pF} =32 \Omega \$
gives you cutoff frequency of 100MHz. So a 33\$\Omega\$ series resistor should decrease your rise time, but leave more than enough to have a good signal at 4MHz.