Most, if not all, input pins on AVR controllers are internally protected by clamping diodes. These diodes prevent that an input voltage can be higher than the supply voltage.
Why these internal diodes are there:
It is important to realize that when an input pin does rise above the supply voltage and these diodes weren't there, the chip might act as an SCR, latch, short the battery, and the chip itself would probably get damaged by overcurrent/overheating.
Why your Arduino runs from the battery
Take a look at the circuit diagram below. When Vcc is detached from the normal power source, there is a path from the battery, via D1, to Vcc. The battery voltage is just high enough for the controller to operate.
simulate this circuit – Schematic created using CircuitLab
How to solve it
The solution is simple, add a high value resistor in series between the battery and the controller's input. Most AVR's have a maximum input leakage current specified (IIL) of 1μA, so you want to make sure the voltage drop is acceptable with that current, say 100kΩ.
But beware!
You do have to realize that the input may draw this current from your button cell and shorten its life time.
What you are missing is the voltage. You're paying for energy storage, not for mAh. If you put n cells in series, the mAh does not change, but the energy stored is n times as much, because that's how much more voltage you have. Since they're in series the current (mA) through each is the same.
Some batteries are rated in W-h (watt hours) which is directly proportional to the energy stored (1 W-h = 3,600J),
Best Answer
The PR48 may be rated for 1.4 volts "nominal", but if you inspect the data sheet closely, the actual voltage is closer to 1.15v to 1.2v. Not sure if your bluetooth could run with 1.2 + 1.2 = 2.4 volts. Doesn't look too good as far as voltage is concerned. The CR2032 does show about 3 volts under load.
I was surprised at the ampacity of the zinc air (even though I use them and wear a hearing aid, I had never looked up the ampacity of zinc air.
EDIT : Added link to CR2032 : http://na.industrial.panasonic.com/sites/default/pidsa/files/crseries_datasheets_merged.pdf