Power supply 5V

I don't know if these are inexpensive enough, but the simplest - and safest - way would be to use a commercial pcb mount power supply module like these:

RECOM Power RAC01-05SC (Mouser)

MYRRA 47122 (Newark)

Or else build a wall wart into your device. I've purchased commercial equipment that had a wall wart mounted internally by means of cable ties, with wires soldered to the wall wart's prongs.

Firstly, why <3 dB? This is a ridiculously strict requirement for general use, so do you have a specific need for zero noise?

This aside, the lowest-noise power supplies are linear. This is a basic circuit that pretty much looks like:

transformer -> rectifier -> smoothing capacitors -> regulator -> more smoothing capacitors

See this article for more info.

You can buy these off the shelf. However, they are declining in popularity as switchmode supplies are more compact and efficient. I would just build one yourself, as this way you can find your own specific requirements and parts. For part selection, keep in mind that:

• Toroidal transformers tend to have the lowest noise
• The regulator is specific to your voltage, power requirements and features (e.g. variable)
• These are the only components you really need to choose based on noise, as capacitors and diodes are silent!
• As @AndrejaKo quite rightly pointed out some capacitors (e.g. ceramics) are indeed not silent, so you should probably steer clear of them. Thanks AndrejaKo!

Look online for example circuits, calculators, and tutorials.