Buck boost converter

As OP has not clarified what 6V depletable source is to be used, this answer predicates a battery which has a nominal full charge voltage of 6 Volts, and is operable down to 3 Volts.

As discussed in comments, there are few if any options for an integrated buck-boost device in a 0.1 inch pin pitch DIP form factor. Further, with most manufacturers DIP is rapidly going out of production to be replaced by SMD, so it is inadvisable to use a DIP part for a new design, even if one finds such an option.

A parametric search for SEPIC / buck-boost ICs with the stated criteria, on a major vendor site such as DigiKey yields several options in SOIC packages. These are leaded packages with a 0.05 inch pin pitch, not extremely difficult to hand-solder. One DDPAK-5

Examples:

• Linear Technology LT1572:

  1.25 Ampere monolithic switching regulator with inbuilt switching MOSFETs, in SOIC-16. Operates from 3 Volts to 30 Volts. Supports most switching topologies including SEPIC.

• On Semiconductor CS5171:

  1.5 Ampere switching regulator with inbuilt switching MOSFETs, in SOIC-8. Operates from 2.7 Volts to 30 Volts. Similar to above, supports several topologies including SEPIC.

Converters get hot due to the non-ideal efficiency of the devices; without knowing the load you are using I can only generalise. Note that the junction temperature of these devices can be quite high and not degrade performance (just what temperature is device dependent).

Will that degrade the overall efficiency?

Yes. The overall efficiency will be \$\eta_1 \cdot \eta_2\$ where \$\eta\$ is the efficiency of each converter: Assuming them to be 85% efficient, you will have a total efficiency of 72.25% for the pair in series.

Apart from eventually making them run cooler, is there any benefit?

Generally, running things cooler extends the life of the component(s)

Is there any other reason why it makes no sense to do that?

Unless efficiency is the overriding concern, no, as you do not appear to have regulators that can handle the load and remain cool.

Note that it is not that unusual to generate an intermediate voltage rail at the expense of some efficiency; this can make power distribution more effective and spreads heat around so it does not concentrate at a single converter.

Update: comment on power consumption of the regulators.

At 20mA for 20V input and 5V out, then the power dissipation is 15 x 0.02 = 300mW. A temperature rise of 30C (which you would notice) implies a thermal resistance of almost 100C / watt.

That said, this is apparently the static dissipation and any extra dissipation will be due only to ineffiencies, so when fully loaded the junction temperature is still safe (probably when operated at room temperature).