Resistors are the subject of Ohm's Law
The resistance, current and voltage are all tied together by the formula:
\$I=\frac{V}{R}\$
The current flowing through the resistors is determined by the voltage across the resistors divided by the total resistance.
So, in a voltage divider, you have a known voltage across the resistors - say 5V.
If the total resistance is 20KΩ (two 10KΩ resistors in series), that is a total current of \$\frac{5}{20000}\$ which equals 250µA.
Now, if the resistors in the voltage divider are the same value then it stands to reason that the voltage applied across the whole divider is split in half across the two resistors, as, according to the same formula (turned around for voltage):
\$V=IR\$
Which is \$0.00025 \times 10000\$ - or 2.5V.
If the resistors were 15KΩ and 5KΩ then it would be \$0.00025 \times 15000\$ for one, and \$0.00025 \times 5000\$ for the other - that's 3.75V and 1.25V respectively.
I know you said you don't want to run outlets everywhere, but I think a pile of wall warts is exactly what you want. Jameco has a large selection. 9 V sounds pretty good and that is one of the standard voltages, but you might want to consider 5 V (see below). Get something in the 500 mA to 1 A range.
These supplies are inherently isolated from the line, are usually short-circuit protected (check to make sure, you definitely want that), and draw so little AC power that you can string a bunch of outlet strips together without harm.
In the end, I think this will cost less and provide a better experience for the kids. I remember when I was a kid tinkering with this stuff how frustrating it was to have batteries run down, especially when you're not aware of when you are asking a lot from them. You also feel a lot less guilty abusing a power supply than running down consumable batteries.
You can start a fire with almost anything. If you do just the right thing, even a 9 V 500 mA supply can catch something on fire, but no more so than a 9 V battery and without the chance of the battery itself doing something bad and causing chemical burns.
If you are worried about LEDs getting damaged by them getting hooked up backwards, maybe you should get 5 V or 6 V supplies. Most LEDs can handle 5 V in reverse, and some 6 V. With 5 V supplies, you can eventually run logic cicuits directly without having to use a linear regulator. Lots of stuff will run well from 5 V. If you get 5 V supplies, get at least 1 A capability. That will be useful for running small motors. 5 W total power really isn't all that much.
Best Answer
In my experience, resistors and diodes burn, but they don't explode.
The only components that I've experienced exploding are tantalum capacitors when placed with the wrong polarity and transient voltage suppressors ("tranzorbs") when exposed to ~2x their rated voltage.
To determine which resistors or diodes to use, you can check their power rating. For example a typical through-hole resistor is rated for 0.25 W. Depending on how you're using the resistor, there are three equations that are useful to calculate the power dissipation:
For example, suppose you have a 10k resistor that you're using as a pull-up resistor for a pin on a 5 V microprocessor. The maximum power drawn by the chip through the resistor (by equation #2) will be 5 * 5 / 10000 = 0.0025 W, or 2.5 mW. That's fine even the tiniest surface mount resistors.