Electrical – PMOS High Side Switch

Yes, it will work. @Peter's comments on bypass capacitors are valid- adding to the (already significant) gate-drain capacitance (as in C3 on the below schematic) will more closely define the value and allow the slew rate to be better controlled. This is a Miller effect slowing of the slew rate- a one-transistor integrator. A much larger reservoir capacitor (than the bypass capacitors on the switched side) will also help. I have shown 47K for R2 since it will waste power continuously when the switched 3.3V is off.

The MOSFET may have to be a bit larger than otherwise, since it will switch more slowly, depending on the current, causing a blip of heat which may be significant at 0.5A. Ideally use transient thermal numbers for the MOSFET and calculate the junction heating and at least test it with much slower rates to ensure you have sufficient safety margin.

An excellent application note on load switching from ONSemi is here with calculations for the slew rate control R1/C3 parts.

^{simulate this circuit – Schematic created using CircuitLab}

The I2C pullup resistors should only be connected to the switched 3.3V or they will probably conduct into the switched 3.3V via the protection diodes on the unpowered micro.

My first impression was, that R1 and R2 are way oversized. 10M may be considered as an open circuit in some applications.

Try lowering those resistances and see if your oscillations persist.

If the oscillations stop you could raise the resistance again but bypass them with small value ceramic capacitors to provide a low impedance return for transients.

Designing low power systems is not easy and you will be tempted to use huge resistance values but they often cause problems like the one you describe.