LiPo is MUCH easier to manage well than NimH.
Energy densities for top capacity NimH are about the same as LiPo nowadays.
(That was written in 2012. In 2021 LiPo energy densities are now typically somewhat higher).
NimH is a relatively hard battery chemistry to manage well. Charging at low rates is not usually advised and negative voltage deflection under charge or temperature rise are the usual end-of-charge detection methods. In contrast, LiPo is charged at constant-current until a set voltage is reached and then at constant-voltage until current falls to a preset level. LiPo will accept any lower-than-maximum rate of charge if desired, and can be recharged from any state-of-charge with no special conditions. (Handling very low voltage cells is slightly more complex, but all sensible charger ICs handle this - and very low voltage should never be allowed to happen.)
The ONLY reason I would think of using NimH in your context is safety - and if it was my son, I'd consider that I could make LiPo safe enough for him to use. LiPo can "melt down" very enthusiastically with flame, BUT it is extremely rare in practice and taking quite usual precautions should allow a safe result.
I would have no personal concerns over LiPo safety in a competently engineered system.
HOWEVER, NEVER use unprotected LiPo cells if you care about safety. The in-battery protection IC DOES NOT serve the same roles as the charger ICs do. The in-battery ones are just to stop people from doing stupidly dangerous things to the battery. That said, IF your charger is properly implemented, and if there is no chance of short or fire potential then most of the protection circuitry is not needed. I say "most" because, if there is e.g. a catastrophic equipment failure and e.g. a short circuit occurs, the in-cell circuitry will usually open-circuit the cell and prevent a fire.
Using the proper charger ICs should allow a very safe and reliable charger to be implemented.
You do not need gas gauging per se - just low voltage cut-out. If you can stop operation at say 3V / cell, that should be enough.
Protected cells should not cost vastly more. If they do, it MAY indicate that the cheap ones are bad ones. You can get utter junk LiIon batteries (and you'd hope to get a price advantage when buying junk :-) - if you were silly enough to buy them. There are enough reputable brand cells around that buying them probably does not cost vastly more. Ensuring that the cells are genuine is another matter. As a working position I suggest you start by assuming that anything bought from a low cost Chinese supplier is fake or out of spec and THEN try and prove otherwise. (NB: Racism? - definitely not!. It's based on experience - many visits to China and time in factories, etc. China is very, very large and has a vast range of sellers in a very competitive market place. In a casual sale, expect a certain portion of the sellers to be 'dodgy' at best.)
Added:
I was going to come back and mention LiFePO4 - AndreKr beat me to it.
Compared to LiPo, LiFePO4 (Lithium Ferro Phosphate) are safer, longer life and have lower energy density. You can buy RCR123A LiFePO4 batteries with 450 mAh x 3.2V capacity. (Some claim up to about 700 mAh but are suspect.) Tenergy LiFePO4 RC123A are widely advertised on ebay and should be good. Tenergy are AFAIK a "rebadger" BUT seem to sell good product. LiFePO4 MUST be charged properly, but are as easy as LiPo to manage. A very simple charger can be built using a constant-current regulator followed by a 3.6V constant-voltage regulator. This setup charges at constant current until Vlimit is reached, and then at constant V. Setting to 3.5V is better.
Here is a randomly found seller of Tenergy LiFePO4 RCR123A batteries. They also sell chargers.
NOTE:
Do NOT use Lithium Ion RC123 (3.6V nominal).
Do not use 3.0V Lithium Primary RC123.
The terms RC123, RC123A, RCR123, RCR123A etc are used somewhat interchangeably by sellers. Just be sure of what you are getting.
The calculation is straightforward. The capacitor size is simply a question of how much voltage drop you can tolerate over the duration of the pulse. The average current from the battery is a function of the duty cycle.
ΔV = I × Δt / C
Solving for C gives:
C = I × Δt / ΔV
Let's assume you can allow ΔV = 0.1V. For your first example, this works out to:
C = 25 mA × 25 ms / 0.1 V = 6.25 mF
The average current draw is 25 mA * 25 ms / 2.5 s = 0.25 mA.
For the second example, the numbers work out to:
C = 50 mA × 100 ms / 0.1 V = 50 mF
Average current = 50 mA * 100 ms / 1.0 s = 5 mA.
Best Answer
I want to preface this with the following: I'm by no means an expert in the field of batteries and/or their chemistry. This is just what I know/have picked up over the years.
Depending what kind of autonomy you need, you need to decide what chemistry provides the best performance.
If it is truly a very long-life product, you pretty much rule out nickel-cadmium (NiCd) and nickel-metal-hydride (NiMH). They have quite high self-discharge, and it is possible the self-discharge may deplete more of your capacity than your actual circuit does. In recent years, this has gotten a lot better with techologies like the Sanyo (now Panasonic) Eneloop type batteries. However, they are still outperformed by lithium-ion technology in most cases.
You have to also consider if you truly need rechargeable batteries. I'm not very familiar with the cost of modern (reliable) lithium-based technologies, but unless things have changed dramatically, it is very possible that it will be more expensive to use a rechargeable battery instead of replacing the battery every few months/years.
Regarding discharge rates: This depends on the technology, but also the design. However, as lithium batteries tend to have lower ESR, they (generally) perform better in this regard.