What you want is called a power supply amplifier or PSA in the business. You are essentially looking for a power amplifier with a gain of 2.4, although it only needs to drive in one direction but has to be isolated.
Think of how a power supply works. At some point there is a feedback signal indicating how high or low the output is and a reference that indicates what that feedback signal should be. The controller looks at the difference and tweaks the output up or down. You can control the output by changing the reference signal.
Isolation makes things more difficult. Sometimes this comparison to the reference is performed on the isolated side with a simple digital high/low indication transmitted back thru a opto to the hot side. The system still works as I described, but the reference is on the isolated side which is not where you want it.
This leaves two possibilities. Communicating the reference to the isolated side and doing the comparison there, or communicating the actual output level to the hot side and comparing it to the reference there. Both schemes have some merit, but I'd probably pick the first in a DIY project.
The best solution is to simply buy a PSA if you can find one with the right characteristics.
I know you said you don't want to run outlets everywhere, but I think a pile of wall warts is exactly what you want. Jameco has a large selection. 9 V sounds pretty good and that is one of the standard voltages, but you might want to consider 5 V (see below). Get something in the 500 mA to 1 A range.
These supplies are inherently isolated from the line, are usually short-circuit protected (check to make sure, you definitely want that), and draw so little AC power that you can string a bunch of outlet strips together without harm.
In the end, I think this will cost less and provide a better experience for the kids. I remember when I was a kid tinkering with this stuff how frustrating it was to have batteries run down, especially when you're not aware of when you are asking a lot from them. You also feel a lot less guilty abusing a power supply than running down consumable batteries.
You can start a fire with almost anything. If you do just the right thing, even a 9 V 500 mA supply can catch something on fire, but no more so than a 9 V battery and without the chance of the battery itself doing something bad and causing chemical burns.
If you are worried about LEDs getting damaged by them getting hooked up backwards, maybe you should get 5 V or 6 V supplies. Most LEDs can handle 5 V in reverse, and some 6 V. With 5 V supplies, you can eventually run logic cicuits directly without having to use a linear regulator. Lots of stuff will run well from 5 V. If you get 5 V supplies, get at least 1 A capability. That will be useful for running small motors. 5 W total power really isn't all that much.
Best Answer
First there's the obvious, like good line and load regulation. Line regulation tells you how the output voltage varies with varying input voltage. Load regulation shows how the output voltage varies is you vary the load. If you use an integrated regulator like the typical LM317 you should have nothing to worry about.
Ripple rejection tells you how input ripple (like from the smoothed rectifier voltage) passes through the regulator and shows at the output.
Do you need to go as low as 0V? Most regulators start at a bit higher voltage, like 1.2V for the LM317. The datasheet shows an application to start from 0V, but notes that full output current are not available at high input-output voltages.
Then there's power management. If you build a linear regulator like with the LM317 internal dissipation can be high. A 0-30V supply with 0-5A output dissipates at least 150W if you need the 5A at very low voltages. One way to circumvent this is to use a transformer with several taps, where the control logic switches to a lower transformer output voltage is less output is required.
Related is short-circuit current. A classical power supply will just limit the current to the 5A, giving the high dissipation I mentioned. This high current may damage your circuit too. Solution is a foldback current limiter, which decreases the output current when a short-circuit has occurred.
Switchers (SMPS, short for Switched-Mode Power Supply) are much more efficient than linear regulators and are more and more used in circuit, now that advanced controllers have simplified their design. But switchers are best at a given input and output voltage and output current. Their design for wide input voltage/output current range is more complicated. You can find them commercially, however, and you can recognize them by their light weight.