It would be possible to build such and adapter. It would depend on the rating of your drills and how you want to plug in the power.
Either way a cordless drill has a DC motor so it will require a step down transformer and a full wave bridge rectifier. Probably also need a regulator and a few capacitors. You might be able to use the power transformer from a laptop or some other device. But the voltage must match the drill and the ampere requirement must be very similar (more you can fry the drill less you can fry the power supply.)
As for how you attach the power supply I would recommend building the interface out of an old battery pack so you don’t have to modify the drill. But it should be possible to add an auxiliary power jack to the drill as well.
White LEDs need a voltage of about 3 V to 3.2 V, YMMV. You place a series resistor to control the LED's current. A 0.5 V drop across the resistor will not give you good control, I'd rather go for at least a volt. So if you place the LEDs in parallel you'll need 4.5 V minimum. For parallel LEDs place one series resistor for each of them.
If your power supply can deliver a higher voltage you can l=place the LEDs in series. Then they will need 9V to 9.6 V, or thereabout. Again a series resistor for at least 1 V drop, and you come up close to 12 V.
To calculate the value of the series resistor you need the LED's current requirement. Suppose this is 20 mA, then for the 12 V supply with the 3 LEDs in series you need
\$ R = \dfrac{12 V - 9.6 V}{20 mA} = 120 \Omega \$
If the LEDs' voltage is somewhat lower you'll have a higher current: 25 mA at 3.0 V LEDs.
If you want to use one of the 5 V supplies you get
\$ R = \dfrac{5 V - 3.2 V}{20 mA} = 90 \Omega \$
Again, at 3 V per LED you'll get a higher current: 22 mA. Note that the difference in current is smaller, which is because voltage tolerance compared with the resistor's voltage drop is smaller. That's why a minimum voltage drop of 1 V to 2 V is recommended.
edit
You added a link to the product, which says "coloured 12V LED". There are no 12 V LEDs. So this is either a LED with the required series resistor built-in, or an incandescent bulb. In the former case you can probably simply apply 12 V to it, as the site also suggests for the replacement LED. I'd love to have seen a polarity indication on it though.
Best Answer
The internal PWM control on the cordless drill consumes current in trapezoidal pulses most of the time. The low impedance NiCad battery is no longer there to soak things up. What I did was place low ESR caps inside the drill along with some status LEDs. I then used a bridge made from Schottkys and huge 20000 \$ \mu \$F total electrolytic caps. I used a 12V transformer that was thermally protected with a self resetting fuse. I know that not all cordless drills are equal but what worked for me should work for you.