A diode will do very little harm - but probably not much good either. I'd put the diode in the fan side so that you get max possible power to the TEC.
5 Watts is unlikely to give much cooling at all. Maximum possible attention to insulation is needed. You can get more cooling initially with evaporative cooling and using this as well as the Peltier would help.
Because the source is driving two things, are there any special considerations I need to think about such as a diode on the TEC so that when it loses power it doesn't dump heat back into the cooler by powering the fan? Or do I want it do run the fan after it loses power to remove heat from the hot side to reduce the differential faster?
I read that to mean that with all external power removed the TEC will act as a TEG using the cool to external temperature differential.
A standard consumer grade Peltier does not make a good TEG (Thermoelectric generator) and even a purpose built TEG is not vastly better. The thermal differential is small compared to what you need to run the TEG on to get useful power. I'd be surprised (it happens) if you could run any sensible sized fan on the delta temperature between cold box and ambient. However, a TEG shunts most of its energy input to the cold side by conduction, which is the main reason for their poor efficiencies. (Much research is being directed at reducing electrical resistance and increasing thermal conductivity.)
The following is probably true but needs some thought:
A hot heatsink will allow heat to flow to the interior via a turned off TEC much better than the TEC can remove heat energy when operating. As the heatsink has been heated by both TEC thermal transfer AND electrical wasted energy you may be worse off if you run this for a while and then turn it off with a hot heatsink. So, if PV output starts to fall (eg end of day) you may be advised to divert energy to the fan to get the heatsink temperature down before power fails fully.
This isn't going to work.
Metal is an excellent conductor of heat, so its temperature quickly becomes the same throughout. The figure of merit for a thermoelectric device is crippled by high heat conductivity [\$\kappa\$].
$$ZT = {S^2GT \over \kappa}$$
Additionally, copper has a Seebeck coefficient [P] of 1.5 μV/K. The most common material used, bismuth telluride, has a coefficient of -297 μV/K. Notice this is the squared term in the numerator, it's very relevant. This is why peltier devices are not made of metal. Increasing the current will only waste more energy.
Result, you get a heater.
Ok, you say, I want my wire to be made of the same material inside a typical junction.
The problem there is the thermoelectric effect has a limited efficient distance for any material. As the thickness of the junction material increases the electron mean free path doesn't. Instead of heat transfer you get heat generation (see Fourier's Law).
Result, you make a heater.
Ok, you say, I will make many small stacks of peltier junctions in my wire passing heat along like a bucket brigade.
The problem here is each device generates its own heat while trying to pass along heat from cold to hot sides. Stacking devices quickly becomes very inefficient and things get hot very quickly.
Result, you make a heater.
Best Answer
Aside from being a potential trip hazard, Peltier devices are very inefficient. They pump some heat from one side to the other, and introduce a relatively large heat source (from the inefficiency).
So if you don't have an effective (big, or forced convection from a fan, or liquid cooled) heat sink on the hotter side, you may end up making both sides hotter, just one a bit hotter than the other.