The neutral wire is usually at 0V relative to earth right? Or 0V RMS?
The 0V is relative to earth, in theory. When the neutral wire carries some current from devices, the resistance causes a small voltage between neutral and earth.
Besides, RMS does not mean "relative to hot wire". It just tells how voltage is expressed.
HYPOTHETICALLY, if one were to stick a knife into the neutral hole, would they be fine?
HYPOTHETICALLY, yes. Though, don't to this!
Is the neutral hole socket on the left or the right side? Is this universal to all countries and most houses?
Some outlets have a distinct orientation, some have not, like this Type-F used in Germany:
There is absolutely no rule which one is neutral and which is hot. It is sometimes said to connect neutral to the left when the outlet is mounted with the holes aligned horizontally, since then it's the same as Type-E, which is used in France, for example:
I would expect that the outputs of your wall-wart supplies have no defined relation to the AC power Neutral.
You MUST connect the low side (what you are calling "neutral") of the wall-wart power supplies together and to the Arduino Ground, to get a common voltage reference.
Best Answer
I just had one of those last week. The plug was hot, and the socket was hotter. I'm the electrical guy so I popped it off. #12 stranded wire shoved into a smaller #14 backstab hole. Two thirds of the strands had missed the hole!
Dumb things like that happen all the time, and you have to nip 'em in the bud the moment you see them.
Don't use this outlet until it's fixed. It probably feeds thru to other outlets on the same circuit, so check this outlet while using those. Get it fixed ASAP.
If I had used the next outlet over, I would've never found it. It can start a fire. (We use steel junction boxes, that helps.)