Ground loops are important and safety is always a major consideration BUT from a technical point of view, use of the ground AT the probe tip is essential in many circuits.
The oscilloscope "sees" a voltage between probe tip and some reference point.
If the reference point is the probe-ground the ground lead can be connected very close to the point under measurement at a point where "ground" is meaningful for the signal concerned.
If the circuit is operating at say 100 MHz and involves other than perfect infinite ground planes then local ground for a signal may be quite different in potential than the "ground" level on the oscilloscope itself. Failure to use probe ground will often produce a signal that would be ideal for use as a source of random numbers.
So, for technical reasons, in complex situations, use of a probe ground connected AT the point of relevance is utterly essential. Safety and ground loop and other considerations are not made any less important by this need, but must be worked out AFTER the technical ground point requirement is met.
Achieving anything worthwhile was never meant to be easy :-)
How real men (and women) do it - a picture from a Jim Williams application note:
As far as I know, this effect has nothing to do with antennas and electromagnetic waves, unless you consider capacitive coupling as an antenna effect (which is unusual). Here is the complete explanation:
If you are sufficiently close to an electric wire connected to the main, there exists a small capacitance between the wire and your body. The closest you are, the more capacitance.
Let us assume, for example, you are at about 1m of the main wire. This may produce about 10pF of capacitance. So, the main is connected in series with a capacitance of 10pF, 9MOhm resistance of the tip of your probe, and 1MOhm resistance of the oscilloscope, to the earth (I've assumed you set the probe at 1:10).
Now, 10pF gives an impedance of 320 MOhm at 50Hz (approx.). Together with the resistances of the probe+oscilloscope, this gives a 10 MOhm : 330 MOhm = 1:33 voltage divider.
So, the voltage indicated by the oscilloscope is 220V / 33 = 6V approx. (if your main is 220V as in the European standard).
This answers the question of the OP.
I add the following remark.
If you touch the ground wire of the probe with one finger, and the tip of the probe with another finger, you'll still see a 50 Hz signal in the oscilloscope, but less stronger. The explanation is simple too:
The finger touching the ground wire offers a resistance of about 1 MOhm. So, now, the main is connected to the ground via a capacitance of 10pF = 320 MOhm impedance and a resistance of 1MOhm in series, hence we have a 1:321 voltage divider, which is about 10 times lesser than the previous voltage divider. So, the voltage seen by the oscilloscope will be of 0.5V approximately.
On the other hand, if you touch the ground wire with your tongue, then the resistance between your body and the earth becomes negligible and you observe no more signal if you touch the tip of the probe with your finger.
Of course, these computations were performed under the assumption that the capacitance between the main and your body is 10pF, but this may vary greatly according to the proximity of your body to the main or to a large conductive plate.
This gives some insight though.
Best Answer
You can see some reviews at EEVBlog: http://www.eevblog.com/tag/rigol/
Specifically Full review of the Rigol DS1052E (a cut down version of this review was included in EEVblog #1 – Rigol DS1052E Oscilloscope Reviwed) and EEVblog #37 – Rigol DS1052E Oscilloscope Teardown.