LDO or switching Regulator

Switching regulator design is complicated indeed, and it even requires a proper PCB layout thanks to the high frequencies and EMI involved. But it's by no means impossible when building it around a controller IC and following its datasheet's recommendations.

But consider that for about the same 3 euros, you could buy a ready-made switching regulator module on its own little PCB from eBay.

And you don't necessarily even need a switching regulator circuit capable of 2A or more, if those spikes are short enough. You could buck down to let's say 8V and put that into a suitably large capacitor, and further regulate to 5V using a linear regulator. The 2A spikes would make the capacitor voltage dip down, but if it doesn't dip too much (thanks to a big capacitor), the output will remain a steady 5V.

I figured out the cause. The inrush current to the new processor board was much greater than the old processor board. It exceeded maximum current of the LDO regulator, causing the odd startup behavior.