Good question. Big question. Partial answer ...
Reputable manufacturers provide specification sheets (yes, even for batteries) and these will provide recommended maximum continuous currents and may provide peak allowable discharge currents.
The maximum value is NOT a hard and fast limit which may not be exceeded, and how much it matters depends on the battery chemistry, the specific implementation and on how much you care about the result. Slight but continuous over current discharge may led to reduced cycle life at a rate disproportionately high compared to the amount of over discharge.
There are many Li (Lithium) chemistry based systems. Some are primary (non rechargeable), and some secondary (rechargeable).
Starting with LiIon (Lithium Ion) which is probably what you meant. These are the most common Li secondary cells available and have related "spinoffs' such as LiPo (Lithium Polymer). They have close cousins in eg Lithium Ferro Phosphate (LiFePO4) whch is a lower capacity but MUCH better behaved variant.
LiIon have a charming "feature" known euphemistically as "vent with flame" (VWF) (to which you can append :-) :-( !!!! )
ie when used in modes outside spec (or sometime just because they can) they will self destroy with heat flame smoke and general hilarity.
LiIon are generally rated at 1C max charge rate and 1C to 2C max discharge rate depending on manufacturer, model etc. Exceeding the max recommended discharge rate modestly is not liable to cause problems. 10% or 20% is probably OK and maybe even 50% or 100% MAY be OK . YMMV and you can have no complaint if it does VWF.
Charging LiIon above their specified rate is a really bad idea [tm]. As above, it may work OK but certainly may result in "vent with flame". Again, I'd hazard that 10% or 20% is liable to be fine and maybe double may be OK. Or not.
If you use LiIon at rates beyond rated values you will generally degrade their cycle life by accelerated amounts. eg I'd guess that a consistent 20% overcharge may halve cycle life. Informed guess only. Similarly, by running LiIon at somewhat below spec the cycle life can be usefully extended.
LiIon also have a very tightly specified upper charge voltage - usually 4.2V with some variation specified across temperature. Exceeding this by 0.1 volt is "unwise" and by 0.2 v is very very unwise. eg 4.2V std, 4.3V hazardous, 4.4V stupid. BUT lowering the max charge voltage slightly to say 4.1V or 4.0V will greatly improve the cycle life and also lower the charge capacity. eg 4.1V max charge voltage may be 80% - 90% of capacity.
At the bottom end, lifetime is also affected by Vmin. There is very little energy left below about 3.0V* and stopping discharge at 3V or even above can be a very good idea for lifetime purposes. (* Discharge curves not to hand - look at manufacturer's graphs. Note that voltage depends heavily on load. Heavy load will drop acceptably lower than light load.
There are numerous "new" versions of liIon being announced regularly. Few have yet got to market. These may have charge or discharge rates of 10C or even 100C. ie at the top end of claims, charging in under 1 minute is claimed.
Lithium Polymer (LiPo - NOT to be confused with LFP / LiFePo) uses "plastic" materials for electrolyte retention and generally have somewhat superior electrical characteristics and somewhat greater resistance to VWF destruction. Somewhat.
A very worthwhile variant of LiIon is LiFePO4 / Lithium FerroPhosphate. Sometimes referred to as LiFe which is OK enough as long as this is not taken to be the chemistry. I'll use LFP. LFP allows charging at 1C to 2C (some manufacturers 0.5C) but discharging at 10C or more (some eg 30C)
Energy contant is low wrt LiIon (about 60% or less) but cycle life is vastly superior and performance at high and low temperatures may be superior. Properly managed LFP offers 2000 deep discharge cycles (against 300-500 for LiIon) and vastly greater figures are claimed by some for larger batteries with good management.
As always - see spec sheets. Top manufacturers provide a large amount of information re rates, voltages, cycle life etc.
LiIon is a joy to manage charge wise.
NimH (see below) is an ornery pig. You can get OK results with NimH using simple methods but best results need rocket science or necromancy.
Reputable manufacturers equip LiIon cells with internal protection circuitry. When a single cell MUST have electronics inside to make it half safe you know you have a fun product. Very low voltage LiIon need to be coaxed into normal range with great care. Very very low LiIon are usually declared dead by their controllers. Insisting on charging such (bypass protection( may result in death (usually just the cell) but can work with due care. People make special bags for charging liIon cells in. What does this tell you?
All that said, an excellent technology. Treat with due care.
Lightly:
NimH: Charging up to 1C Ok with monitoing of negative delta V or delta temperature or absolute temperature for termination. Some allow 2C with speial batteries. Smart monitoring may allow 2C+ with care. Radio control model fans charge NimH at 4C or more using capacity x 1xx% overcharge as charge termination. eg they may charge a 4Ah pack at 20 A for 15 minutes. This is 5C and 125% energy input. Lifetimes suffer. They don't care.
NimH may be more or less discharged at whatever rate they will bear. Internal cell resistance drops voltage increasingly at high current making battery less useful unless designed accordingly. Discharge should be stopped at say 1V at lowish loads and no less than say 0.9V at very high loads. I'd err on the high side. You can discharge them to utterly empty (0.8-0.9V ) but you gain little and will very severely impact lifetimes.
NimH is good for 300-500 deep discharge cycles but can be taken to 2000 or so by taking 10% off top and bottom (stop discharge early, terminate charge early).
The calculation is straightforward. The capacitor size is simply a question of how much voltage drop you can tolerate over the duration of the pulse. The average current from the battery is a function of the duty cycle.
ΔV = I × Δt / C
Solving for C gives:
C = I × Δt / ΔV
Let's assume you can allow ΔV = 0.1V. For your first example, this works out to:
C = 25 mA × 25 ms / 0.1 V = 6.25 mF
The average current draw is 25 mA * 25 ms / 2.5 s = 0.25 mA.
For the second example, the numbers work out to:
C = 50 mA × 100 ms / 0.1 V = 50 mF
Average current = 50 mA * 100 ms / 1.0 s = 5 mA.
Best Answer
Your math has failed you. :-)
You mention that the duty cycle is 2% at 1/3 Hz. The period is therefore 3 seconds, and only during 2% of those 3 seconds does your device deliver 10 A. 2% of 3 seconds is 0.06 seconds.
While your device may consume 60 W while delivering 10 A, it does so for only 0.06 seconds, then "rests" for 2.94 seconds. This means that the energy delivered per pulse is only 3.6 J, not 1000 J:
$$60\text{ W}\times0.06\text{ s} = 3.6\text{ W}\cdot\text{s} = 3.6\text{ J}$$
This amount of energy can definitely be stored in a relatively non-exotic capacitor.
You may not even require a huge power supply. If the device was designed by someone competent, it would already have its own specialized power supply to deliver those 10 A pulses 2% of the time, so while the device can deliver 10 A pulses, it doesn't consume anywhere near 10 A.