Electronic – Shouldn’t LED blink 25 (30) times per second on AC

No way. Given just a differential, just a pair of connections, there's no way for a circuit to tell which is which.

If you had access to phase information about the grid, you might be able to do it, but as the mains frequency isn't terribly constant, you would not be able to rely on predicting this. Your device would have to get information about the present phase angle of the mains from someplace.

However, if you do have a real earth-ground reference from some place, you could build a circuit to look at the voltage of each supply line with respect to ground. One will be a small drop below the full 120 (or whatever the local mains supply) whereas the other will be slightly above ground. (Neutral return current vs your neutral run's resistance). This would just let you know which leg was 'hot', and you could swap your output polarity accordingly.

A Second Thought

If the power supply has only four connections with the world, two from the mains plug, and two to output the DC, there's no way I can think of that you could deterministically set output polarity based on plug orientation.

BUT

if you're willing to make your otherwise simple supply a lot more complex, you could conceivably make it so that if you pulled the plug and flipped it, and plugged it back in straight away, the output would flip.

Here's the idea: You stuff some kind of micro-controller in the box, that monitors the line voltage, and determines when rising-crossing-zero (or some other phase point) happens on one of the legs. You'd have to reference this to the midpoint of the two supply legs via a voltage divider. The micro could then anticipate when the next such phase point would occur. Now you'd also have to put some kind of super-cap in the box and pick a low power micro that could live off the super-cap long enough for the user to flip the plug. When power comes back, either the anticipated phase change happens when you expect, or half-way between. Half-way between means the user flipped the plug, so your micro flips the output polarity.

Of course, that still would be problematic. If the thing had been unplugged for a while, the micro would be dead, and you'd have to make an assumption about what the output polarity should be. Finding a micro that could last maybe 10 seconds on a cap's worth power while actively chugging away could be easier said than done. Last and not least, this would really only be a novelty gadget.

Believe it or not, that black lump is epoxy covering an integrated circuit, probably a small microprocessor. These are often caseless chips, just the IC die glued to the PCB with fine wires between the pads on the chip and PCB.

Many microcontrollers can be put into a deep-sleep state where they draw just a few microamps, so little so as not to significantly affect battery life. They spend all of their time in deep sleep until someone presses the button, which is connected to an interrupt line to wake it up.