Using half wave rectifier for low power applications

Either can work correctly if designed properly. If you have a dumb rectifier supply feeding a 7805, then all the rectifier part needs to do is guarantee the minimum input voltage to the 7805 is met.

The problem is that such a power supply only charges up the input cap at the line cycle peaks, then the 7805 will drain it between the peaks. This means the cap needs to be big enough to still supply the minimum 7805 input voltage at the worst case current drain for the maximum time between the peaks.

The advantage of a full wave rectifier is that both the positive and negative peaks are used. This means the cap is charged up twice as often. Since the maximum time since the last peak is less, the cap can be less to support the same maximum current draw. The downside of a full wave rectifier is that it takes 4 diodes instead of 1, and one more diode drop of voltage is lost. Diodes are cheap and small, so most of the time a full wave rectifier makes more sense. Another way to make a full wave rectifier is with a center tapped transformer secondary. The center is connected to ground and there is one diode from each end to the raw positive supply. This full wave rectifies with only one diode drop in the path, but requires a heavier and more expensive transformer.

A advantage of a half wave rectifier is that one side of the AC input can be directly connected to the same ground as the DC output. That doesn't matter when the AC input is a transformer secondary, but it can be a issue if the AC is already ground-referenced.

Adding the capacitor smooths the rectified AC voltage and not having one probably accounts for you measuring different voltages when you applied different loads. Pure half wave, as a waveform looks like this: -

If you add a capacitor the red waveform is transformed into the blue waveform for normal loads. This is the first benefit you get - the dc voltage remains largely at the peak value of the AC waveform and is therefore more predictable when you measure it with your meter.

When you connected DC motors this has a similar effect in that between half wave pulses, the back emf from the motor "free-running" holds the dc voltage and prevents it falling to zero (like in the red waveform).

The voltage you measured was 7.1V off-load and this largely corresponds to what a half-wave rectifier produces. Mathematically it is \$ \frac{V_{ACpeak}}{\Pi}\$ = \$ \frac{16.3\times\sqrt{2}}{\Pi}\$ = 7.34V.

A better option is to use a full bridge rectifier as this fills in the gaps on the negative cycle of the AC waveform: -

This has a better average dc voltage without a capacitor of twice what the half wave circuit gives i.e. about 14.7V. Also the smoothing capacitor can be smaller (when using one) because it gets charged-up to peak voltage twice as often.

There are a whole load of other things I could go into but this document from wiki says most of them and covers the basics above.