Electronic – Will there be any voltage drop when connecting load to batteries

That's how batteries work. As you draw current from them, you deplete the chemical reactants near the plates, and it takes time for new reactants to take their place. This creates a gradient in the reactant concentration within the volume of the battery.

The terminal voltage is a function of both the reactant concentration at the plates and the voltage drop across the internal resistance of the battery (which rises a bit with reactant depletion). If you let it run long enough, the terminal voltage should stabilize at some lower value, although the overall depletion of the battery will eventually become a factor.

When you switch off, the voltage drop across the internal resistance disappears immediately, but it takes some time for the reactant concentration gradient to disappear, which is why you see the sudden jump up followed by the slow recovery to the original terminal voltage.

This is a characteristic of any old and withering battery. As a battery nears the end of its life, it gets harder and harder to source the active charges. In the 'off' state, the battery will have a set of charges ready to go so you'll have a relatively high voltage. As you apply the load, the battery resistance will actively increase rapidly near the end of life so by the time you measure it with your DVM, the voltage will have dropped much more than in the 25ms that your ADC is working on it.

This is a phenomenon I've seen many times on NiCd's, NiMh, Alkaline's, Li-ion's so it's definitely not unique to lithium batteries.

A battery this close to the edge everyone else would have thrown away by now so manufacturers won't have any data on this.