The voltage across a capacitor discharging into a fixed resistance decays exponentially. The time constant is RC, where C is the capacitance, and R is the resistance between the terminals of the resistor.
$$
V(t)=V(0)e^{\frac{-t}{RC}} \\
$$
You're right that the capacitor starts at 20V, because that's the voltage across the 200 ohm resistor after the capacitor is charged. So you know V(0), and you know C. All you're missing is R. To analyze that properly, think of the switch as a resistor of 0 ohms when closed, and don't worry about the current source. The parallel combination of a 200 ohm resistor and a 0 ohm resistor is 0 ohms. The series combination of a 50 ohm resistor and a 0 ohm resistor is 50 ohms. So the 50 ohm resistor is the only one that matters when determining your discharge constant.
The current through the 200 ohm resistor depends on the voltage across the 200 ohm resistor. The voltage across the 200 ohm resistor is the same as the voltage across the 0 ohm resistor (the closed switch). V=IR, so what's the voltage across that pair of resistors when the switch is closed?
And yes, the voltage across an ideal resistor can change instantaneously. Keep in mind though, there's no such thing as an ideal resistor in the physical world. Everything has capacitance to everything else.
You are confused about what the concept of infinity means. Infinity isn't a number that can ever actually measure a quantity of something, like resistance, because it's not a real number. As Wikipedia aptly puts it:
In mathematics, "infinity" is often treated as if it were a number (i.e., it counts or measures things: "an infinite number of terms") but it is not the same sort of number as the real numbers.
When we talk about an "infinite" resistance, what we are really considering is this: as the resistor gets arbitrarily large, what does something (current, voltage, etc) approach?
For example, we can say that as the resistance gets arbitrarily large, current gets arbitrarily small. That is, it approaches zero:
$$ \lim_{R\to\infty} \frac{15\mathrm V}{R} = 0\mathrm{A} $$
That's not the same as saying the current is zero. We can't ever increase R all the way to infinity, so we can't ever decrease current to zero. We can just get arbitrarily close. That means you can't now do this:
$$ \require{cancel} \cancel{0\mathrm A \cdot \infty \Omega = ?}$$
This is a bit of a mathematical contradiction by most definitions of infinity, anyhow. Most numbers, when multiplied by an arbitrarily large number, approach infinity. But, anything multiplied by zero is zero. So when you multiply zero by an arbitrarily large number, what do you get? I haven't a clue. Read more about it on Mathematics.SE: Why is Infinity multiplied by Zero not an easy Zero answer?
You could ask, as the current becomes arbitrarily small, what does the resistance approach?
$$ \lim_{I\searrow 0} \frac{15\mathrm V}{I} = \infty \Omega $$
However, if you look closely, you will notice that if \$I = 0\$, then you are dividing by zero, which is your hint you are approaching something that can't happen. This is why we must ask this question as a one sided limit.
Leaving the realm of mathematics, and returning to the realm of electrical engineering, what do you really get if you remove the resistor from that circuit, and leave it open? What you have now is more like this circuit:

simulate this circuit – Schematic created using CircuitLab
C1 represents the (extremely small) capacitance between the two wires that aren't connected. Really, it was there all along but wasn't significant until the resistance went away. See Why aren't wires capacitors? (answer: they are) and everything has some capacitance to everything else.
Best Answer
Many power supply have built in short circuit protection, therefore it is not always the case that shorting the output of the power supply will harm it. Though it is not advisable to do this even when the short circuit protection is in place.
Your conclusion about the current being steered into the zero resistance path is correct, but you should not conclude that all other connections are open circuits, or that this can't harm devices connected in parallel to short.
Simple example:
simulate this circuit – Schematic created using CircuitLab
We are charging some very big capacitor with a power supply having \$50 \Omega\$ internal output impedance. This impedance limits the current which can be supplied to the cap and the charging process completed fine.
Now you are closing the switch, shorting both the power supply and the capacitor. Lets assume that the supply is fine - it has SC protection in place. However, due to very low resistance of the switch, the discharge current of our big capacitor is huge. The capacitor has some low Equivalent Series Resistance and gets very hot due to high discharge current. This heat causes the capacitor to be destroyed.
A single capacitor is the simplest example I could think of, but there are many more.
Summary:
Shorting the output of power supply to the ground can damage both the supply and the equipment connected in parallel to the short. The potential damage to other equipment depends on the equipment's internal implementation.