Electrical – 7805 voltage regulator reverse polarity

If I understand you correctly, you are trying to adjust the output voltage of your 7805 regulator to output 5V - 10V by adjusting the voltage at it's ground pin between 0V and 5V.
If you are doing this, the pin needs to be able to sink current in order to maintain regulation, so whatever the power source used, it needs to be capable of this. Many supplies will only source current, and if this is the case what you are seeing will occur.

A simple resistor divider with potentiometer can be used, or an opamp to drive the 7805 ground pin is maybe the best solution, preferably with a output including its negative rail if you want to go all the way down to 5V. You will find many example adjustable supply circuits on the web, and in the datasheets of the regulators also.

Here are a couple of example circuits:

Pot adjust, from this page with more info:

Opamp adjust, from datasheet pg.24 (don't use a 741 if you can possibly avoid it, as they are obsolete):

The 7805 will drop out typically around 1.6V above the output voltage. At 1A it's guaranteed to not drop out with 2V of input-output differential. Most likely you'll not be running anywhere near 1A or you'd be using a switching regulator, but even at low current the dropout is not so low- that's because the 7805 is not an LDO regulator and there are Vbe drops in there.

One could guess that the input ripple rejection probably deteriorates as you get close to the dropout voltage and the gain drops. The datasheet specification is at 5V input-output differential, so they sidestep that issue. If you have a sensitive analog circuit like an RF module you may wish to use a higher input voltage than the absolute minimum.

If you're using a 7805 with unregulated (transformer, rectifier and filter) input voltage probably needs to be something like 10V to be safe and account for line voltage tolerance, ripple and so on). If you're using it with a regulated supply (like a switching wall wart) 9V is good, 7.5V is okay, but 6V is not high enough. There are LDO regulators that have very low dropout (so 6V would be fine) but they have other disadvantages (they are only conditionally stable- pay careful attention to the output capacitor value, ESR and type), they are more expensive, less sources, and generally have much lower input voltage capability so they're easier to fry with input transients. Much modern electronics uses LDO regulators and/or switching regulators, there are literally thousands of possible parts to use, but none yet has quite the staying power of the 7805/78M05/78L05.

I would say that if you need a heat sink on the 7805 it's time to move to a switching regulator in most cases. There's no problem using the 7805 or 78M05 at 10, 50 or 100mA, and it's better than a 78L05 (more expensive, but the circuit is different and has better guaranteed performance). The trade-off of an LDO vs. a 78xx regulator is a bit more complex and it is heavily dependent on the input voltage and how much control you have over it.