An ideal inductor would not behave like a capacitor, but in the real world there are no ideal components.
Basically, any real inductor can be though of an ideal inductor that has a resistor in series with it (wire resistance) and a capacitor in parallel with it (parasitic capacitance).
Now, where does the parasitic capacitance come from? an inductor is made out of a coil of insulated wire, so there are tiny capacitors between the windings (since there are two sections of wire separated by an insulator). Each section of windings is at a slightly different potential (because of wire inductance and resistance).
As the frequency increases, the impedance of the inductor increases while the impedance of the parasitic capacitor decreases, so at some high frequency the impedance of the capacitor is much lower than the impedance of the inductor, which means that your inductor behaves like a capacitor. The inductor also has its own resonance frequency.
This is why some high frequency inductors have their windings far apart - to reduce the capacitance.
How does this relationship proves that a discontinuous change in
voltage requires an infinite current?
First, note that the equation given in your question defines an ideal (non-physical) capacitor and so this is the context of my answer.
Second, note that if the capacitor voltage is discontinuous at some instant(s) of time, the time derivative of the voltage does not exist there.
However, one can approximate a discontinuity by, e.g., letting the voltage change linearly with time over some short interval. For example, let the capacitor voltage change linearly from \$0\mathrm V\$ to \$1\mathrm V\$ in \$\Delta t\$ seconds.
Then, according to the ideal capacitor equation, the capacitor current during the transition is
$$i(t) = C\frac{1 \mathrm V}{\Delta t}$$
In the limit as \$\Delta t \rightarrow 0\$, the capacitor voltage becomes discontinuous (finite change in zero time) and the capacitor current goes to an infinity large, infinitesimally short pulse; a current impulse.
But this is academic since physical capacitors obey the ideal capacitor equation only approximately and over a relatively narrow region of operation.
Best Answer
In an ideal world, where a capacitor has no series inductance and an inductor has no parallel capacitance, and voltage and current sources can provide voltages and currents with a step-shaped profile, the current into a capacitor and the voltage over an inductor can change abruptly.
Note that the reverse is not true: the voltage over a capacitor, and the current through an inductor, can not change abrubtly (unless you allow for non-finite currents or voltages, like a Dirac-shaped pulse).
Note that this ideal world is an mathematical abstraction, you can't buy such components.