What are the effects of running a PIC μC slightly above its rated working voltage

(the numbers are given for LiCoO2 chemistry, for LiFePO4 they should be lower) Short answer: yes, this is OK. I charge lithium cells with a bench supply set to 4.2 V and whatever C/10 figure is for the cell I'm charging. If you are not in a hurry, slow charging is better, even for cells claimed to be capable of withstanding higher charging currents.

However (quoting you):

charging at a constant voltage (say 4.2V) so long as the maximum current is limited to a reasonable value for the cell

means you will have constant current charger till your cell is at ~95%. Up to this point the voltage across the battery will be less than 4.2V if you measure it. Only when your charger starts outputting 4.2V it will become constant voltage.

What you are about to build is CC/CV charger and this is the right thing to do. "Constant voltage only" charger will be set to 4.2 V with no current limiting and it will charge the lithium cell very slowly. You can check it youself, just construct var.voltage circuit and measure the current into (discharged) cell at 3.5, 3.7, 4.0, 4.2, and 4.5 V. Cheap Chinese chargers are constructed like that, they restrict the voltage to 4.2 V so the cell won't ignite after being charged but the consumer would have to wait longer. I once bought a portable emergency charger which could be emptied in a couple of hours then took 3 days to recharge its internal cells.

There are other precautions to observe while charging lithium, you can learn them from any modern charge controller IC datasheet (my favorite is Linear Tech., their literature is very high quality). If you don't implement these precautions in your design never leave it unattended while charging, otherwise it may ruin your morning one day.

The short answer is yes, but your voltage is going to track with the voltage of the battery. A slightly more elegant approach is the basic linear regulator shown below.

edit: just to be clear, this will still track with the battery voltage when the battery voltage is less than the breakdown voltage of the zener. If you can get away with a Vout of about 3V, you could select a zener with a breakdown voltage of 3.6V and get a steady output voltage over a relatively large section of your battery SOC.