Electronic – Why do battery/power supply supply voltages drop under a load

Your alternator has a non-zero impedance, which is higher than the impedance of the partly-charged battery in this case. That's what causes the voltage to drop when the battery is connected.

Or, in other words: The alternator cannot deliver as much current as would be required to put the battery's terminals to 14V, so it delivers as much as it can and the voltage drops. - With higher rpm the picture (voltage) will likely change. Besides, note that an alternator -as the name suggests- produces (rectified) alternating current (AC) where peak voltage and RMS may differ, so be sure what you are measuring. (This is, by the way, one of the reasons why you should never run your car's engine without connected battery; the battery, like a giant capacitor, smoothes the system's voltage ripple and spikes away which would otherwise put significant stress on other electric components.)

You decide what the "low" threshold voltage level when the battery has its load on it. This way you will know when the battery gets to critical level of capacity corresponding to what load it is expected to power. Trying to do this when the battery is unloaded will not give you a realistic in system usage scenario.

Think about your car. Its battery stays connected to the alternator / voltage regulator all the time -- unless you remove the cable to do service work.