I'm going to address the larger issue of making the system robust and reliable, rather than just focusing on the batteries. The main issues that I see are: ruggedness, waterproofness, battery/charging, and the "chassis".
If I were building these things myself, I would use PVC pipe as the chassis. But more on this in a moment.
To increase ruggedness, I would encase the PCB in "casting resin". Just google "casting resin". Essentially it is an epoxy that you can pour into the PVC pipe to encase the PCB's to both waterproof them and support the PCB against shock and vibration. Casting resin is available from many hobby/craft stores like Michaels and Hobby Lobby. Just put your electronics in the PVC pipe, mix up the resin, and pour it into the pipe. Important Note: casting resin comes in 2 parts and the ratio of the two parts effects how long it takes to harden. The faster it hardens the HOTTER it gets during the curing process. You want it to harden as slowly as possible, otherwise it might get hot enough to damage the electronics. Experimentation with the resin is important to getting this right.
Casting resin will work best if your batteries are rechargeable and fully encased in the resin. However, I wouldn't do that. Batteries behave weirdly when charged. Best case your batteries could get hot and not be able do dissipate the heat due to the resin. Worst case, your batteries build up some internal pressure that can't be dealt with due to being encased. As an alternative, you could use some super-capacitors. The usefulness of super-caps will depend on your power consumption and a variety of other issues, but I've used them for several applications and they work quite nicely. Essentially, supercaps behave like rechargeable batteries except that they don't hold as much power but they can be charged and discharged almost an unlimited number of times.
If you can't use supercaps, then rechargeable batteries with tabs/pins/wires already attached would be your 2nd best choice. 3rd choice would be standard or rechargeable AA's. With AA's, I personally wouldn't spend much time making the spring keep good contact. That is a massive waste of time because whatever you do, it won't be good enough! Instead, your design should take that into account. The best way to do this is to simply put large-ish caps in your circuit so that if the batteries do momentarily loose contact then the circuit will remain powered up.
If you use supercaps or rechargeable batteries then next comes the charging system. You could simply have a connector that goes to a charger. Of course that isn't very waterproof. A cool way would be to have a non-contact inductive charger. Imagine a transformer with two coils of wire. AC goes into one coil and comes out the other coil. An inductive charger is the same, except that one coil is in the base of your baton and the other coil is in the "charging station".
In the best case everything-- including super caps, charging coil, and PCB's-- could be encased in casting resin. With no seams there is no way for water to get into the circuitry! And with everything encased and fully supported the whole thing is very mechanically robust. With a little bit of work, you wouldn't even need end-caps on the PVC pipe. That way your baton would be a simple and smooth rod. I'd bet that you could take this and throw it out of a low-flying plane and it would survive.
Also, Casting resin is water-clear. Your circuit can have status LED's that can be seen through the resin.
It looks like the Intel computer is really sensitive to overvoltage. You could use a two stage power supply for it - a buck converter to drop the voltage down to, for example, 7 volts and then a linear regulator to provide clean and spike-free 5V. The intermediate voltage (7V in my example) should be just slightly higher than the minimal requirement for the linear regulator for best efficiency (though efficiency won't be that great anyway).
I used LM338K to make a car power supply for my Viliv N5 palmtop which has the same problem.
Best Answer
A lot of clips on the market support more than one battery thickness. Try looking at ones that only support 2016 (and maybe 2012 size as well). For example, I was able to find a 2016-only clip from Digikey that is 2.16mm thick (nominal).
That extra mm on a 2016 holder is there primarily for the spring contacts that hold it together. You could take it away completely, and use a 2016 with SMT tabs attached. Depending on your device needs, you could just solder it down, and you'd be good. From Digikey, I found a 2016 cell with leads on it that is 1.87 mm thick.
If you wanted to make it removable, I could imagine some clip or screw holding down each lead flush to the PCB, and then you could have a bit more vertical space to get the spring/screw in there.
Alternatively, if you want to use a plain 2016 and since you said you were game for some fabrication, you could try making the the traditional metal style clip without a spring. That should save a fraction of a mm. To get good contact, you may need to put a piece of shim metal between the battery and the clip, or have some sort of ridge built into the clip to provide pressure on the battery. The caveat here is that you might not be saving much thickness and you may be introducing a source of unreliability.