Electrical – Can a voltage regulator work below minimum Vin

Yes, the dropout voltage is the headroom the regulator needs to work with. This is the minimum amount the input voltage must be above the output voltage for the regulator to work properly. So as you say, a 3.3 V regulator with 1.1 V dropout requires at least 4.4 V input.

Stuff happens, so it's usually a good idea to give the regulator a little more room. However, going too far will be less efficient, so there is a tradeoff. The range you quote of 4.75 to 5.25 volts could well be a reasonable tradeoff. Basically, that's a "5 V" supply.

The output current of a linear regulator is its input current (minus the small current on the ground pin, but that is small enough to ignore for the point I'm trying to make). Since the input voltage is higher than the output but the currents are the same, the output has less power. The difference in input power and output power is wasted in the regulator as heat. Another way to look at this is that the regulator will dissipate the difference between the input and output voltages times the current as heat. For example, consider a case where the input is 5.0 V, the output 3.3 V, and the current 500 mA. The voltage accross the regulator is 5.0V - 3.3V = 1.7V. That times 500 mA is 850 mW, which will go into heating the regulator. That would be fine for something like a TO-220 package, but a SOT-23, for example, would burn up at 850 mW.

The regulator will work perfectly if you keep within the datasheet specs. If you supply it with less than 7V it will lose regulation.

Things to be aware of are that if you supply power with a 9V battery and try and draw too much current, the battery voltage will eventually sag below the 7V required (this is likely what was happening to the first commentor)

Also, the higher your input voltage, the more power is dissipated in the regulator so you may need a heatsink. There are many answers on here that go through all this. To tell you whether you would need one we would need to know how much current you are planning on drawing from it at what input voltage.
If it's 9V, then assuming a maximum ambient temperature of 50°C, a maximum operating temp of 125°C:

(125 - 50) / 19 = 3.95W maximum.
at 9V:
3.95W / (9V - 5V) = ~1A maximum

If it's just the microcontroller you are powering though, then it's almost certainly no problem. As we can see over an amp would be needed to needed to reach maximum operating temperature (even if reached, it's unlikely to break - it will just shut down) Your kit will probably only draw a few milliamps, maybe up to 100mA with all pins driving heavyish loads.