Electronic – Amplified electret mic gives a lot of low frequency noise

For starters, you haven't decoupled the microphone power at all. You should split R1 into two resistors (e.g., 4700Ω each), and put a fairly sizable ceramic capacitor (0.1 to 10.0 µF; 1.0 µF gives you a cutoff frequency of 34 Hz) from the middle junction to ground. This is the most direct path by which power supply noise can get into the audio input chain.

The next way would be if the opamp has poor PSRR, in which case, you should decouple its supply with a dedicated resistor and capacitor. The problem here is that you don't have a lot of headroom to work with, which will limit the size of the resistor you can use (voltage drop based on the average current that the opamp requires), which in turn will require a relatively large capacitor in order to get a low cutoff frequency.

I think you've misinterpreted the circuit shown on the web page. It applies to any electret. Basically it shorts out only the AC (signal) by using a large capacitor but avoids switching the DC bias voltage which is the source of the pop.

When the mic is plugged in C1 experiences a step voltage (from the 5V supply/R2) which brings both terminals of the capacitor to the DC bias level of the electret.

The negative terminal of the capacitor then discharges through R1 (100k) back to (nearly) 0 volts leaving no voltage across R1 (after more than 5 time constants). From the microphone's perspective it 'sees' a parallel load of 100k (the impedance of the large capacitor being negligible). This is so large it has no effect on the mic's output.

When the mute switch is operated (closed) there is no large DC step because the voltage across R1 is 0V and hence no large POP. The microphone now 'sees' only the 100uF capacitor as a load because the 100k has been shorted out. This effectively shorts out any AC (signal) and hence mutes the mic. The DC bias circuit (R2) is not shorted out because C1 is effectively an open circuit to the DC signal.