Electronic – a 2 to 4 line decoder

Congrats on your CPU design. Yes, the FPGA is definitely your next step. But you're not very clear what it is that makes your head spin. I presume it's the large number of different devices on offer. You also don't say what simulator you're using.
When you synthesize a design the synthesizer should give you a report on the resources used, like number of gates and RAM. This should give you an idea what parts are suitable for your design. Make sure you have enough headroom. Pick a part which has some extra RAM, which you'll need to execute programs on your CPU.

edit (re your comment)
You need a development system for a specific manufacturer. Altera and Xilinx are the big players, and both have their believers. Which one you choose is not that important, both have enough different parts to satisfy your needs.
If you would choose for Xilinx, it has its ISE WebPACK Design Software, which is free (large download at 3.4GB). The limitations compared to more advanced versions of the software shouldn't bother you at this time.
Next you need a development board. You need to have an idea of what FPGA you'll need to select one. If you go for Xilinx I would pick a Spartan, maybe a Spartan-6, the Virtex is already too high-end IMO. There's still a great many too choose from, mainly different in the extras on the board. You'll want a board with a bit more than a few switches and LEDs. I would look out for a keyboard connector, and a display module.
Xilinx has a number of boards, and for Xilinx FPGAs there's also Digilent, Avnet, Xess and many others.

So compare a number of boards (price will also play a part, I presume) and download the development software for the FPGA you plan to use. Synthesize your design to verify that it will fit in the chosen FPGA, and then you're ready to purchase a board around this FPGA.

I forgot to tell a bit about FPGAs. An FPGA is basically a large collection of gates with programmable connections between them, so that you can create almost any logic function/system with them. Over several generations FPGAs became more advanced and now have optimized blocks to create efficiently functions like RAM. Your development software's synthesizer creates the connection diagram between the gates. This isn't permanently stored in the FPGA, but must be loaded on power-up from an external configuration Flash memory. That's where your design will be stored. Like any other Flash memory it can be erased and rewritten a great number of times.

Your basic synchronous digital design is a sort of discrete time feedback loop.

One or more registers made of flip-flop like storage elements hold values. These flow onto buses, possibly through path selectors called multiplexers and various combinatorial logic circuits which perform a logical or arithmetic operation on the value represented by some number of signals (typically each signal being a binary digit, or bit). Over a short period of time, the result flowing through the paths and logic to circle back to the inputs of the registers stabilizes. After an interval calculated to safely allow the worst case time needed for this, another active clock edge occurs, which causes one or more of the registers to replace the current values they have been holding with the updated values being fed back to their inputs.

In the case of something like a (synchronous) a counter, the logic circuit sitting between the outputs of the registers and the inputs would add one, so each active clock edge would see the stored value increase by one.

More complicated operations such as performed by a CPU might select two source numbers from a register array from which two locations can be simultaneously accessed through two separate read ports, add them together, and write them back to somewhere in that same register array through a third writing port.