Here's one mental picture of a battery:
The electro-chemical reactions inside the battery happen only when there's a closed circuit.
When you place a voltmeter across the poles, then you create a closed circuit (with very low current) – the electro-chemical reactions start, and you now have a voltage between the poles. The placement of the voltmeter changes the thing you are trying to measure. Without the voltmeter, there's no voltage between the poles, because there's no closed circuit.
But the above seems completely inconsistent with how people speak of open-circuit voltage, as existing independent of actually measuring it.
My reasoning must be wrong somewhere. But where?
(I know this is a very simple question, but I'm genuinely confused by it.)
Here's an example of the typical use of open-circuit voltage (see 24:00), from an MIT lecture.
Edit: replaced voltage-drop with voltage.
For an ideal battery, this is true. However, all batteries have some, hopefully very small, leakage current, and will eventually discharge. Hopefully it takes years for this to happen, but that is not always the case.
The voltage between the poles existed prior to connecting the voltmeter or closing the circuit. It is the voltage between the poles that keeps the electro-chemical reactions from proceeding. If the battery is not connected to anything, the voltage between its poles exactly matches the electro-chemical potential of the reaction.
This is mistaken. There is a voltage difference between the poles whenever the battery is holding a charge. There is no current when there is no closed circuit.
Open circuits may have voltage, and this voltage exists independently of attempts to measure it.