Do Standard NiMh AA or AAA Batteries Have Built-in BMS?

LiPo is MUCH easier to manage well than NimH.
Energy densities for top capacity NimH are about the same as LiPo nowadays.
(That was written in 2012. In 2021 LiPo energy densities are now typically somewhat higher).

NimH is a relatively hard battery chemistry to manage well. Charging at low rates is not usually advised and negative voltage deflection under charge or temperature rise are the usual end-of-charge detection methods. In contrast, LiPo is charged at constant-current until a set voltage is reached and then at constant-voltage until current falls to a preset level. LiPo will accept any lower-than-maximum rate of charge if desired, and can be recharged from any state-of-charge with no special conditions. (Handling very low voltage cells is slightly more complex, but all sensible charger ICs handle this - and very low voltage should never be allowed to happen.)

The ONLY reason I would think of using NimH in your context is safety - and if it was my son, I'd consider that I could make LiPo safe enough for him to use. LiPo can "melt down" very enthusiastically with flame, BUT it is extremely rare in practice and taking quite usual precautions should allow a safe result. I would have no personal concerns over LiPo safety in a competently engineered system.

HOWEVER, NEVER use unprotected LiPo cells if you care about safety. The in-battery protection IC DOES NOT serve the same roles as the charger ICs do. The in-battery ones are just to stop people from doing stupidly dangerous things to the battery. That said, IF your charger is properly implemented, and if there is no chance of short or fire potential then most of the protection circuitry is not needed. I say "most" because, if there is e.g. a catastrophic equipment failure and e.g. a short circuit occurs, the in-cell circuitry will usually open-circuit the cell and prevent a fire.

Using the proper charger ICs should allow a very safe and reliable charger to be implemented.

You do not need gas gauging per se - just low voltage cut-out. If you can stop operation at say 3V / cell, that should be enough.

Protected cells should not cost vastly more. If they do, it MAY indicate that the cheap ones are bad ones. You can get utter junk LiIon batteries (and you'd hope to get a price advantage when buying junk :-) - if you were silly enough to buy them. There are enough reputable brand cells around that buying them probably does not cost vastly more. Ensuring that the cells are genuine is another matter. As a working position I suggest you start by assuming that anything bought from a low cost Chinese supplier is fake or out of spec and THEN try and prove otherwise. (NB: Racism? - definitely not!. It's based on experience - many visits to China and time in factories, etc. China is very, very large and has a vast range of sellers in a very competitive market place. In a casual sale, expect a certain portion of the sellers to be 'dodgy' at best.)

Added:

I was going to come back and mention LiFePO4 - AndreKr beat me to it.

Compared to LiPo, LiFePO4 (Lithium Ferro Phosphate) are safer, longer life and have lower energy density. You can buy RCR123A LiFePO4 batteries with 450 mAh x 3.2V capacity. (Some claim up to about 700 mAh but are suspect.) Tenergy LiFePO4 RC123A are widely advertised on ebay and should be good. Tenergy are AFAIK a "rebadger" BUT seem to sell good product. LiFePO4 MUST be charged properly, but are as easy as LiPo to manage. A very simple charger can be built using a constant-current regulator followed by a 3.6V constant-voltage regulator. This setup charges at constant current until Vlimit is reached, and then at constant V. Setting to 3.5V is better.

Here is a randomly found seller of Tenergy LiFePO4 RCR123A batteries. They also sell chargers. NOTE:
Do NOT use Lithium Ion RC123 (3.6V nominal).
Do not use 3.0V Lithium Primary RC123.

The terms RC123, RC123A, RCR123, RCR123A etc are used somewhat interchangeably by sellers. Just be sure of what you are getting.

Frankly, if you want to use rechargeable batteries, I'd suggest to go for Li-ion batteries. A single-cell (3.7V nominal) or a double-cell (7.4V nominal) sound like good choices. For the single-cell version, you'd need a DC-DC boost step-up converter to get 5V, for the double-cell you can either use a "normal" dissipative power regulator (something similar to the 7805, but with a lower dropout voltage, since the minimum output voltage of a double-cell Li-ion battery is about 6V, and that's not enough for the 7805), or, if power loss is an important factor (I'd guess it is), you should use a DC-DC buck step-down converter. Also, if you don't want to prematurely kill your Li-ion batteries, you should use a power converter which shuts down when the minimum input voltage is reached (3V for a single-cell, 6V for a double-cell).

So I'd look for a battery with the proper capacity (you should do a rough calculation for the expected minimum milliamp-hours: Pout / efficiency / battery voltage * run time), and a circuit designed for converting from that battery into 5V. Ebay is for example full of these components (both Li-ion batteries and circuits designed for this purpose).