Electronic – Interference from well pump pressure switch and microcontroller

Its not a result of the high voltage but rather a by product of how that high voltage is generated. Very high frequency switching is used, such devices often emit a ton of RF around 25Mhz depending on the design. This is what throws off the USB scope and other digital electronics.

As for the whining your heard, plasma can actually transfer energy to the air very effectively. If fact you can even build a tweeter from plasma generation.

Incidently be careful with that stuff, ionizing the air into plasma generates a fair amount of ozone which isn't good for the lungs.

If the bulk of interference is coming from circuit connections (a schematic would help), you can either add inductance to the connections to filter out high-frequency feedback or attempt to isolate the discharge circuit and control & monitoring circuit. Adding inductance can be as simple as wrapping wire around a ferrite bead. Care must be taken to ensure the feedback is sufficiently attenuated while not impeding circuit operation (ie: slower rise times). Optical and physical isolation are common methods of separating high and low voltage circuits. Safely separating grounds may be too much trouble, but you can still keep the return paths of each circuit apart for most of their journey. The spiking voltage return path should be unimpeded (least inductance). If isolation efforts don't do the trick, one can lower the input impedance of the troublesome digital inputs using pull-up or pull-down resistors and capacitors. The resistor value should be high enough such that the line's regular operation is not impeded -- that is, the driver can support this lower resistance; the capacitor shorts high-frequency content to ground -- start with 100nF ceramic and work up to 10uF if needed (try it first with nothing, of course!). If voltage at any point is exceeding a part's maximum, one can clamp it to below a chosen value using something as simple as a zener diode, though other (superior and more expensive) TVS systems/parts are available. This just protects from damage, though.

If the bulk of interference is radiating from the capacitor discharge connections, one approach would be to reduce the radiation at the source. I'm guessing that slowing or otherwise modifying the cap discharge rate (TVS) is not an option, as it would affect measurements. The next best thing is to reduce the propagation properties of the wires and traces powering the capacitor(s): minimize all connection lengths including ground, and minimize ground loop areas (keep the return as close as possible to signal/power). Of course, physical distance between the controller and DUT will help.

I have no experience with EMI shielding layers (mu-metal, etc.).

A strategy to skip all this is to temporarily shut down the controller during the discharge, a few hundred milliseconds, saving state in the meantime.