Electronic – Do batteries last longer with constant current draw or spiky current draw

Some power supplies are designed to safely 'fold back' the voltage when the current demand goes too high. This means that once you reach the max current, if the load resistance decreases so as to demand more current, the supply voltage just decreases so as to keep the current from exceeding the maximum.

If you have that kind of supply, you should be fine. If you don't have that kind of supply, then it's simply under-rated and you need one that either (a) has the fold-back feature, or (b) can supply the 2.1A, with a comfortable safety margin.

And unless my math has failed me, the bursts are 1000J so I would need about a 55F capacitor, which is out of the question.

Your math has failed you. :-)

You mention that the duty cycle is 2% at 1/3 Hz. The period is therefore 3 seconds, and only during 2% of those 3 seconds does your device deliver 10 A. 2% of 3 seconds is 0.06 seconds.

While your device may consume 60 W while delivering 10 A, it does so for only 0.06 seconds, then "rests" for 2.94 seconds. This means that the energy delivered per pulse is only 3.6 J, not 1000 J:

$$60\text{ W}\times0.06\text{ s} = 3.6\text{ W}\cdot\text{s} = 3.6\text{ J}$$

This amount of energy can definitely be stored in a relatively non-exotic capacitor.

You may not even require a huge power supply. If the device was designed by someone competent, it would already have its own specialized power supply to deliver those 10 A pulses 2% of the time, so while the device can deliver 10 A pulses, it doesn't consume anywhere near 10 A.