currentohms-lawresistancevoltage
Say we have the following setup
simulate this circuit – Schematic created using CircuitLab
Input is pulled high and current flows through it as long as the switch is open. As soon as it closes current switches to the path of least resistance and goes through the switch to ground.
Imagine a similar setup with two paths with different resistances
In this case current will flow through both paths, and will not simply go through the path of least resistance.
The idea you describe can work, but it is very crude to remove the power from the coil by shorting the battery. At the very least you should insert a resistor to limit the current somewhat, as is you rely on the internal resistance of the battery, which can be OK for a 9V radio battery, but disastrous for 12V car battery.
The more common way to achieve roughly the same is to use a 'relay' with a Normally Closed (NC) contact, so you can open the circuit instead of shorting the battery. This is in fact how the common electro-mechanical doorbell works.
Many power supply have built in short circuit protection, therefore it is not always the case that shorting the output of the power supply will harm it. Though it is not advisable to do this even when the short circuit protection is in place.
Your conclusion about the current being steered into the zero resistance path is correct, but you should not conclude that all other connections are open circuits, or that this can't harm devices connected in parallel to short.
Simple example:
simulate this circuit – Schematic created using CircuitLab
We are charging some very big capacitor with a power supply having \$50 \Omega\$ internal output impedance. This impedance limits the current which can be supplied to the cap and the charging process completed fine.
Now you are closing the switch, shorting both the power supply and the capacitor. Lets assume that the supply is fine - it has SC protection in place. However, due to very low resistance of the switch, the discharge current of our big capacitor is huge. The capacitor has some low Equivalent Series Resistance and gets very hot due to high discharge current. This heat causes the capacitor to be destroyed.
A single capacitor is the simplest example I could think of, but there are many more.
Summary:
Shorting the output of power supply to the ground can damage both the supply and the equipment connected in parallel to the short. The potential damage to other equipment depends on the equipment's internal implementation.
Best Answer
'Path of Least Resistance' is a phrase that only really applies where you have alternative paths for, for instance, a walker who chooses to go through a gap in the wall next to the gate, rather than open the gate. Wikipedia says this:
In electronics, the current always divides between several paths in the inverse ratio of their resistances (or impedances if it's AC).
In your first example, the switch when closed has orders of magnitude less resistance than the input. The current division ratio will be so close to 100% to the switch, that we engineers approximate it to 'all through the switch'.
In your second example, it's a reasonable ratio.
Note that the input current doesn't 'divide' as such, looking at all the paths and deciding how to split. The node where the paths split from has a specific voltage V. Current flows through each path I=V/R, V is the same for each path, hence the current is in the inverse ratio of resistances. The input current to that node is then the sum of all the output currents.
Looking at the comments about water channels, and thinking about the origins of the phrase, it's also often used for lightning. Who hasn't been told, when perhaps visiting a tower with a 'lightning conductor' fixed to the outside, that it takes the path of least resistance. Just like water eroding its own channel, ionisation in the air creates its own low resistance channel. This happens to such a degree that lightning may well not strike the most obvious high point in the local area, but proceed by ionisation steps to create an alternative low resistance path to ground through which the main strike then occurs.