It depends on where the currents are flowing. In general, if current flows from the power supply rails (your +10V/0V) through the load resistor to your virtual ground, I'd use three capacitors, with the one across the 10V supply before the rail splitter and two equal capacitors from the +5 and -5 rails to the virtual ground.
From the datasheet figure, you might want to consider a dummy load resistor to improve the stability if power consumption isn't a big deal to you.
At first, the principle of "virtual ground" can be applied during DESIGN of opamp-based amplifiers. This simplifies calculations - and the error is in most cases acceptable. Error? Yes - because there is always a differential voltage between both opamp inputs, which is exactly Vdiff=Vout/Aol. (Aol=open-loop gain of the opamp). Because of the large values for Aol (1E4...1E6 for lower frequencies) this diff. voltage Vdiff is in the µV range.
However, because this is not true for larger frequencies, the closed-loop gain will deviate from the calculated value for rising frequencies.
Regarding your last sentence: Yes - introducing additional delay in the feedback path will cause additional phase shift - and this can lead to instability/oscillations.
EDIT: "...until the virtual ground is re-established and the cycle repeats."
I suppose, with the above cited sentence you are asking for something like a "sequence" which leads to the steady-state conditions after applying an input signal, correct? This is, indeed, a question which deserves some explanations.
Example: Inverting opamp-based amplifier with a gain of "-2". Input: +1V step (t=0).
At the very beginning (t>0), the feedback is not yet active and the output will jump to the maximum negative voltage (supply rail). Now the feedback network causes the inverting terminal to become negative - and the output starts to go to positive voltages. However, this will not continue again and again because the opamp has internal delay elements (causing bandwidth limitations and phase shift). That means: The output does not "jump" to other values but it takes some time to reach the upper rail. But, in reality, the output will NOT reach the upper rail because on the way to the maximum positive output the output voltage crosses some finite negative values - and for an output value of app. Vout=-1.999V there will be an equilibrium between input and output. Explanation:
Vout=-1.999V and Vin=+1V cause a very small voltage between both resistors (at the inv. input terminal) which - when multiplied with Aol - is exactly the assumed output voltage (in the example: Vout=-1.999V.) This equilibrium state is stable.
Best Answer
You have a rather weak connection to the midpoint voltage. (50kΩ.) In the first circuit this is fine, but in the second it is not. The midpoint voltage will drift a lot, because the load current is being returned through the same 50kΩ impedance. You even get coupling from output back to input through the Rload to the bottom of the input voltage. This can cause oscillations, though probably not for a Av=1 buffer like you have here.
My suggestions:
If you want to AC-couple input and output, use circuit 1. Circuit 2 gives no advantage.
If you want to avoid AC-coupling, use a rail-splitter circuit (e.g. buffer the midpoint voltage with another op-amp.) Then you can dispense with the caps because you have a good +/- 4.5V supply.