Electronic – Looking for good articles about reducing crosstalk and other noise in PCB design

In the AVR Appnote on hardware design considerations on page 5:

To protect the RESET line further from noise, it is an advantage to connect a capacitor from the RESET pin to ground. This is not directly required since the AVR internally have a low-pass filter to eliminate spikes and noise that could cause reset. Applying an extra capacitor is thus an additional protection. However, note that this capacitor cannot be present if DebugWIRE or PDI is used. If not using High Voltage Programming it is recommended to add an ESD protecting diode from RESET to Vcc, since this is not internally provided due to High Voltage Programming. Alternatively, or in addition, a Zener diode can be used to limit the RESET voltage relative to GND. The Zener diode is highly recommended in noisy environments. The components should be located physically close to the RESET pin of the AVR. Figure 2-1 shows the recommended circuit on the RESET line.

I suggest you try an external capacitor of 0.1uF to ground, and a schottky diode to from RESET to VCC to act as an ESD clamp. Add a 5.1V Zener from RESET to GND, which comes highly recommended in the manufacturer's text.

A signal with less than 10 ohms impedance will be susceptible to magnetic mode interference, and although the Ethernet pair will not radiate much common mode energy, there will still be some, and most of this energy is magnetic mode. See this previous question and answer for more details on common mode radiation, the causes and some solutions.

Apart from separating the analogue signal by some distance (which is an effective method of reducing interference), it may be necessary to add a guard track between the Ethernet pair and the analogue signal (which could probably be considered DC relative to the Ethernet pair). See this excellent article for guidance on guard traces.

That is just the starting point. Do you have any simulation tools available? Some permit proper simulation for both pre- and post-layout; if you have such tool(s) available, I strongly suggest using them.

HTH