Electronic – Battery Management Controller

LiPo is MUCH easier to manage well than NimH.
Energy densities for top capacity NimH are about the same as LiPo nowadays.
(That was written in 2012. In 2021 LiPo energy densities are now typically somewhat higher).

NimH is a relatively hard battery chemistry to manage well. Charging at low rates is not usually advised and negative voltage deflection under charge or temperature rise are the usual end-of-charge detection methods. In contrast, LiPo is charged at constant-current until a set voltage is reached and then at constant-voltage until current falls to a preset level. LiPo will accept any lower-than-maximum rate of charge if desired, and can be recharged from any state-of-charge with no special conditions. (Handling very low voltage cells is slightly more complex, but all sensible charger ICs handle this - and very low voltage should never be allowed to happen.)

The ONLY reason I would think of using NimH in your context is safety - and if it was my son, I'd consider that I could make LiPo safe enough for him to use. LiPo can "melt down" very enthusiastically with flame, BUT it is extremely rare in practice and taking quite usual precautions should allow a safe result. I would have no personal concerns over LiPo safety in a competently engineered system.

HOWEVER, NEVER use unprotected LiPo cells if you care about safety. The in-battery protection IC DOES NOT serve the same roles as the charger ICs do. The in-battery ones are just to stop people from doing stupidly dangerous things to the battery. That said, IF your charger is properly implemented, and if there is no chance of short or fire potential then most of the protection circuitry is not needed. I say "most" because, if there is e.g. a catastrophic equipment failure and e.g. a short circuit occurs, the in-cell circuitry will usually open-circuit the cell and prevent a fire.

Using the proper charger ICs should allow a very safe and reliable charger to be implemented.

You do not need gas gauging per se - just low voltage cut-out. If you can stop operation at say 3V / cell, that should be enough.

Protected cells should not cost vastly more. If they do, it MAY indicate that the cheap ones are bad ones. You can get utter junk LiIon batteries (and you'd hope to get a price advantage when buying junk :-) - if you were silly enough to buy them. There are enough reputable brand cells around that buying them probably does not cost vastly more. Ensuring that the cells are genuine is another matter. As a working position I suggest you start by assuming that anything bought from a low cost Chinese supplier is fake or out of spec and THEN try and prove otherwise. (NB: Racism? - definitely not!. It's based on experience - many visits to China and time in factories, etc. China is very, very large and has a vast range of sellers in a very competitive market place. In a casual sale, expect a certain portion of the sellers to be 'dodgy' at best.)

Added:

I was going to come back and mention LiFePO4 - AndreKr beat me to it.

Compared to LiPo, LiFePO4 (Lithium Ferro Phosphate) are safer, longer life and have lower energy density. You can buy RCR123A LiFePO4 batteries with 450 mAh x 3.2V capacity. (Some claim up to about 700 mAh but are suspect.) Tenergy LiFePO4 RC123A are widely advertised on ebay and should be good. Tenergy are AFAIK a "rebadger" BUT seem to sell good product. LiFePO4 MUST be charged properly, but are as easy as LiPo to manage. A very simple charger can be built using a constant-current regulator followed by a 3.6V constant-voltage regulator. This setup charges at constant current until Vlimit is reached, and then at constant V. Setting to 3.5V is better.

Here is a randomly found seller of Tenergy LiFePO4 RCR123A batteries. They also sell chargers. NOTE:
Do NOT use Lithium Ion RC123 (3.6V nominal).
Do not use 3.0V Lithium Primary RC123.

The terms RC123, RC123A, RCR123, RCR123A etc are used somewhat interchangeably by sellers. Just be sure of what you are getting.

Batteries are tricky nonlinear devices. If you want to build a safe and reliable lithium battery charger, you need to know more about your batteries and your battery charger circuit. In general, lithium batteries are not interchangeable and not all chargers will work with all batteries.

There are also significant differences between lithium (probably not what you mean), lithium ion (probably what you mean, sometimes written Li+ or Li-ion), and lithium polymer (sometimes written LiPo) batteries, and significant differences within the battery chemistries of these categories.

The datasheet I found for TP4056 does not say that this device includes undervoltage protection. It only provides charging. Perhaps other circuitry does on that evaluation board does. The appropriate discharge protection is a function of battery chemistry, and the threshold should set according to the battery manufacturer's datasheet (as a baseline, anyway).

A circuit that tests for undervoltage is probably measuring the battery under load, and will need to compensate for the battery's internal resistance \$R_{internal}\$ reducing the voltage at the terminals \$V_t\$. That is

$$ V_t = V_{oc} - IR_{internal} $$

So to answer your second question it's quite possible that \$V_t\$ = 2.5V is an appropriate cutoff for a battery with \$V_{oc(min)}\$ = 3.0V, if \$ IR_{internal} \$ ~ 0.5V.