STM32 – Hardfault on SystemClock_Config()

First the programming voltage, it is given in the data sheet under the operating conditions of the memory (page 63):

Vprog Programming voltage 2 - 3.6 V

So if I read your suggestion correct it might have been powered only by that coin cell during programming? This might be very very bad indeed. The controller draws a maximum current during write operation (just the flash) of 7 mA. A small coin cell really doesn't cope well with higher currents and it's voltage as well as the capacity dropped probably really fast. Given the datasheet of a CR1220, the voltage drop on a full battery would be at least 0.5 V if you draw 7 mA from it, so it starts out at about 2.35 V, not much headroom.

But I also wonder if your circuit is fine now - have you measured the current it draws and checked all the voltages and their stability if you draw current (connect a resistor decade and vary it a bit)?

Now a possibility on how you might be able to salvage the part. I guess it won't work as you have no trouble connecting to the device, but anyways:

The F103 has a built in bootloader which is accessible if you pull the BOOT pin high during power up. You will also need access to UART1 to be able to communicate with the bootloader. Have a read of section 3.4 of the reference manual. Another good resource is this document.

If you can connect to it, there is a tool (STSW-MCU005STM32 and STM8 Flash loader demonstrator (UM0462)) available from ST which allows to interface with the bootloader and erase or program the chip.

booting into an erased flash leads to lockup quickly

This may or may not happen - depending what was written and what was not. Note that depending on how the flash is updated and timing there might be no erased flash page - just some with new and some with old data.

initial PC/SP [...] The critical window is now only the write of these two words

True, but there will still be a chance for failure - not good enough for safety critical applications.

Note that users might be tempted to use a standard STM32 flashing tool instead of your software - it would work just fine with ROM bootloader but defeats your migitations completely.

Write a custom bootloader into Flash

I strongly recommend this approach if the results could be catastrophic in failure conditions.