Several ways to measure peak-to-peak voltage with a MCU:

• undersampling: Directly sample with an ADC, even though you know you are getting less than once sample per cycle of the signal. This requires (a) the signal repeats over and over the same waveform, and any changes to that pattern occur relatively slowly, and (b) the actual sample-and-hold part of the ADC has a high enough bandwidth, even though you only get one sample once every few cycles of the signal, and (c) the exact sampling instant of the sample-and-hold doesn't always hit the same point on the repeating signal, but drifts along it, eventually hitting the highest peak and later the lowest valley of the signal. (Often this drift happens without you having to do anything; but to guarantee it will always work, you may need to lower your sampling rate with some signals). High-speed digital oscilloscopes often use this undersampling technique to inspect amazingly high frequency signals.

• use maximum peak detector and minimum valley detector to hold the maximum and minimum voltages on sampling capacitors long enough for you to sample them. Either use a resistor on each capacitor with a "long enough" RC time constant that the change in voltage from the actual maximum to your ADC sample is insignificant, or use a transistor on each capacitor to reset the voltage immediately after you sample it. If you can estimate the diode voltage drop in software, the simplest hardware is a diode-capacitor-resistor peak detector and a diode-capacitor-resistor valley detector ... but using an op-amp precision peak detector rather than a single diode may be simpler than trying to estimate the diode voltage drop.

• use a DC blocking capacitor, followed by a diode to ground (to clamp the minimum voltage to 0) and a diode to a peak detect sampling capacitor. The voltage on that capacitor gives you the peak-to-peak voltage directly. Somehow estimate and compensate for the diode voltage drops in software, or else use op-amps to compensate for them in hardware, similar to the peak detector.

• use a frequency mixer (analog multiplier) to shift high-frequency signals down to an easier-to-handle frequency well inside your ADC's capability. (Like undersampling, this assumes the signal repeats over and over the same waveform, and any changes to that pattern occur relatively slowly).

I'm not sure why you want a precision rectifier. I assume you really meant "precision peak detector" (which is admittedly almost the same circuit).

A RC filter with a long RC time constant will give you the average voltage of the signal. As pingswept says, it's probably best to use a fixed cutoff frequency (for the peak detect, valley detect, and the average detect) to handle the high frequencies with analog circuits, and deal with low frequencies in software.

As long as the MCU is sampling the highest peak voltage and lowest valley voltage, consider also reporting the midway voltage halfway between the two -- this is not always the same as the true average voltage of the signal.

Couple of theories here:

• How sure are your power supplies are coming up symmetrically?
  If one rail comes up before the other, you may have non-zero output voltages from the op-amp for a very short period of time.
• Have you implemented all the PCB-layout practices needed for such high-impedances? At minimum, you're going to need guard-rings on all the ultra-high-impedance nodes.
  The National (Now TI) LMC6082 datasheet has a good discussion of what is required to get board-leakage currents low enough to not be a problem.

This will likely not address the possibility that you have dielectric soakage issues, as discussed in @RocketSurgeon's answer.
A good and easy way to test his answer would be to desolder one of the caps on a bad board, and reverse it. If the offset is flipped in the other direction, it's a dielectric soakage issue (because the persistant charge in the cap will have a single polarity). If the offset voltage does not change, the problem is not the capacitor.

One thing I don't see the dielectric soakage issue explaining is why the charge seems to come back when the circuit is unpowered, and go away when it is powered. Since the element that discharges the capacitor is connected continuously across the cap, (e.g C1||R2, C2||R1), the contribution of any current leaking out of the cap should be a constant, and not affected by the supply voltage.

The only thing that comes to mind for me would be that there is something hygroscopic somewhere, and it injects an offset current. When you power the board, it warms up, and drives the moisture out over time. Turn the board off, and it begins to resorb moisture.

One comment I do have is I don't see why you have both SW1A, and SW1B. You can entirely dispose of SW1B. Just tie both of the R/C pairs together, and to the output of the op-amp. When one of the cap/resistor sets is selected, the other will just slowly discharge. As long as one end is floating (which is accomplished by SW1A), the voltage on the other end is irrelevant.