Many power supply have built in short circuit protection, therefore it is not always the case that shorting the output of the power supply will harm it. Though it is not advisable to do this even when the short circuit protection is in place.
Your conclusion about the current being steered into the zero resistance path is correct, but you should not conclude that all other connections are open circuits, or that this can't harm devices connected in parallel to short.
Simple example:
simulate this circuit – Schematic created using CircuitLab
We are charging some very big capacitor with a power supply having \$50 \Omega\$ internal output impedance. This impedance limits the current which can be supplied to the cap and the charging process completed fine.
Now you are closing the switch, shorting both the power supply and the capacitor. Lets assume that the supply is fine - it has SC protection in place. However, due to very low resistance of the switch, the discharge current of our big capacitor is huge. The capacitor has some low Equivalent Series Resistance and gets very hot due to high discharge current. This heat causes the capacitor to be destroyed.
A single capacitor is the simplest example I could think of, but there are many more.
Summary:
Shorting the output of power supply to the ground can damage both the supply and the equipment connected in parallel to the short. The potential damage to other equipment depends on the equipment's internal implementation.
Best Answer
Why do you think so? I don't understand where the idea that Ohm's Law is "violated" by an ideal wire (or ideal short-circuit) comes from.
Ohm's Law:
$$V = IR$$
Now, if \$R=0\$, as is the case for an ideal wire, there is zero voltage across for any current through.
Consider the I-V characteristic for an ideal resistor with a large resistance:
Note that the slope of the characteristic is \$\frac{1}{R}\$ and thus, as \$R \rightarrow \infty\$, the slope approaches zero, i.e., the I-V characteristic becomes horizontal through the origin. This is an ideal open circuit; the current is zero for any voltage across.
Now, consider the I-V characteristic for an ideal resistor with a small resistance:
As \$R \rightarrow 0\$, the slope approaches infinity, i.e., the I-V characteristic becomes vertical through the origin. This is an ideal short circuit; the voltage is zero for any current through.
There is no violation of Ohm's Law - the open circuit and short circuit are simply the limits of \$R \rightarrow \infty\$ and \$R \rightarrow 0\$ respectively.