You are confused about what the concept of infinity means. Infinity isn't a number that can ever actually measure a quantity of something, like resistance, because it's not a real number. As Wikipedia aptly puts it:
In mathematics, "infinity" is often treated as if it were a number (i.e., it counts or measures things: "an infinite number of terms") but it is not the same sort of number as the real numbers.
When we talk about an "infinite" resistance, what we are really considering is this: as the resistor gets arbitrarily large, what does something (current, voltage, etc) approach?
For example, we can say that as the resistance gets arbitrarily large, current gets arbitrarily small. That is, it approaches zero:
$$ \lim_{R\to\infty} \frac{15\mathrm V}{R} = 0\mathrm{A} $$
That's not the same as saying the current is zero. We can't ever increase R all the way to infinity, so we can't ever decrease current to zero. We can just get arbitrarily close. That means you can't now do this:
$$ \require{cancel} \cancel{0\mathrm A \cdot \infty \Omega = ?}$$
This is a bit of a mathematical contradiction by most definitions of infinity, anyhow. Most numbers, when multiplied by an arbitrarily large number, approach infinity. But, anything multiplied by zero is zero. So when you multiply zero by an arbitrarily large number, what do you get? I haven't a clue. Read more about it on Mathematics.SE: Why is Infinity multiplied by Zero not an easy Zero answer?
You could ask, as the current becomes arbitrarily small, what does the resistance approach?
$$ \lim_{I\searrow 0} \frac{15\mathrm V}{I} = \infty \Omega $$
However, if you look closely, you will notice that if \$I = 0\$, then you are dividing by zero, which is your hint you are approaching something that can't happen. This is why we must ask this question as a one sided limit.
Leaving the realm of mathematics, and returning to the realm of electrical engineering, what do you really get if you remove the resistor from that circuit, and leave it open? What you have now is more like this circuit:
simulate this circuit – Schematic created using CircuitLab
C1 represents the (extremely small) capacitance between the two wires that aren't connected. Really, it was there all along but wasn't significant until the resistance went away. See Why aren't wires capacitors? (answer: they are) and everything has some capacitance to everything else.
Ohm's law applies to resistors, not to diodes. When you place a diode in a simple series circuit like yours, you cannot apply Ohm's law to find the current.
To be clear, if you know the voltage across a resistor, you can calculate the current through using Ohm's law.
If you know the voltage across a LED, you cannot calculate the current through using Ohm's law.
This is because the diode current is an exponential function of the voltage across:
$$I_D = I_S(e^{\frac{V_D}{nV_T}} - 1)$$
So, for example, if you change the voltage across the diode, the change in current will not be proportional, i.e, the ratio of the change in voltage to the change in current is not constant.
Best Answer
This answer is probably inherently displeasing to the feeling of natural order for some :-) :
A law of nature is simply a statement of observed results under defined conditions.
Ohms law is essentially a statement that the ratio of the two two variables V & I is typically observed to remain approximately constant as the variables vary.
It is arguably saying the opposite of what it may seem - ie not so much
"R is the ratio between ..." but
it is more "if the ratio between V & I is constant then we call this constant resistance" and "this approximates typical behaviour of a significant proportion of real world products".
At any given moment R IS the ratio between V and I. If this ratio has changed then R has changed. So V/I never changes for specific values of V & I with all other conditions held constant, whereas dV/dI typically does change in the real world.
So R = V/I is an accurate statement
R = dV/dI is usually an approximation and
where it all falls apart it just means that the observation does not apply under thos e conditions.
That's woolier than I'd like but seems to convey what I'm trying to say. I hope :-).