You are confused about what the concept of infinity means. Infinity isn't a number that can ever actually measure a quantity of something, like resistance, because it's not a real number. As Wikipedia aptly puts it:
In mathematics, "infinity" is often treated as if it were a number (i.e., it counts or measures things: "an infinite number of terms") but it is not the same sort of number as the real numbers.
When we talk about an "infinite" resistance, what we are really considering is this: as the resistor gets arbitrarily large, what does something (current, voltage, etc) approach?
For example, we can say that as the resistance gets arbitrarily large, current gets arbitrarily small. That is, it approaches zero:
$$ \lim_{R\to\infty} \frac{15\mathrm V}{R} = 0\mathrm{A} $$
That's not the same as saying the current is zero. We can't ever increase R all the way to infinity, so we can't ever decrease current to zero. We can just get arbitrarily close. That means you can't now do this:
$$ \require{cancel} \cancel{0\mathrm A \cdot \infty \Omega = ?}$$
This is a bit of a mathematical contradiction by most definitions of infinity, anyhow. Most numbers, when multiplied by an arbitrarily large number, approach infinity. But, anything multiplied by zero is zero. So when you multiply zero by an arbitrarily large number, what do you get? I haven't a clue. Read more about it on Mathematics.SE: Why is Infinity multiplied by Zero not an easy Zero answer?
You could ask, as the current becomes arbitrarily small, what does the resistance approach?
$$ \lim_{I\searrow 0} \frac{15\mathrm V}{I} = \infty \Omega $$
However, if you look closely, you will notice that if \$I = 0\$, then you are dividing by zero, which is your hint you are approaching something that can't happen. This is why we must ask this question as a one sided limit.
Leaving the realm of mathematics, and returning to the realm of electrical engineering, what do you really get if you remove the resistor from that circuit, and leave it open? What you have now is more like this circuit:
simulate this circuit – Schematic created using CircuitLab
C1 represents the (extremely small) capacitance between the two wires that aren't connected. Really, it was there all along but wasn't significant until the resistance went away. See Why aren't wires capacitors? (answer: they are) and everything has some capacitance to everything else.
You can't specify the current AND the voltage. Either you are applying 5V or you are applying 1A. Since you have a batery symbol drawn, I will assume you are applying 5 volts.
This 5v is applied across two 1 ohm resistors in series. Total resistance of two 1 ohm resistors in series is 1 + 1 = 2 ohms. V = I * R tells us that 5 = I * 2 where I = 2.5 A. Then the voltage across each resistor is V = 2.5 * 1 = 2.5 volts.
How about applying 1 A to two series 1 ohm resistors? Well, that 1 A is going to produce V = 1 A * 1 ohm = 1 volt across each resistor. Since there are two 1 ohm resistors in series, the voltage across the pair is 1 + 1 = 2 volts.
The current must be the same at all points along that path as charges cannot be created or destroyed ('what goes in must come out'). The voltages around the loop must also add up to zero ('what goes up must come down'). In this case, you go up 5 volts in the battery, then you come down 2.5 volts in each resistor, ending up at zero right where you started.
Best Answer
Yes. All materials under normal conditions and at fixed temperature follow* ohm's law, though it becomes less useful in good insulators where breakdown occurs before any substantial amount of current can flow.
Non-ohmic effects occur at boundaries between different materials, such as pn junctions, schottky junctions, thermocouples, electrochemical cells, et cetera. They can also be observed in discharge phenomena, where the flow of current causes ionization and chemical changes in the conducting material.
*Here, "follow" means "behave in a way closely approximated by". Depending on how precisely you're measuring things, it may matter that it's not quite exact.
Edit: it's worth mentioning that the presence of (changing) magnetic fields can complicate things. Transformers and inductors are not generally considered to obey ohm's law under dynamic conditions, for instance.
For further information on where it gets murky, see this question.