Big names in EDA software are Synopsys, Cadence, Mentor, and Magma.
Here is some of the software typically involved in a standard-cell ASIC flow:
HDL simulators read an RTL description of the design (typically written in Verilog or VHDL) and mimic the behavior of the hardware described by the RTL. Wikipedia has a list of Verilog simulators; the list notes which ones also support VHDL.
Synthesis tools read the RTL description and map it onto the cells available in your target library. The cell library is usually described in .db format, and may be provided either by your foundry or by a third-party library provider. The output of the synthesis tool may also be Verilog, but it won't have any high-level constructs, only cell instantiations and wires. This is called a netlist. Wikipedia has a list of tools.
Place and route (P&R) tools take the netlist and search for a physical implementation of that netlist. This involves placing all of the cells in two dimensions and figuring out how to route the connections between the cells. Examples are IC Compiler, Encounter, and Blast Fusion.
As a design goes through P&R there will be additional quality checks including static timing analysis (using e.g. PrimeTime), layout vs. schematic, and DRC.
If your budget is less then 10-20 thousand dollars (or more realistically 100K+), you have no hope of getting an ASIC made.
The common device used instead of an ASIC, in situations where you cannot afford the NRE (non-returnable expenses - basically the cost for producing the masks for etching your asic, as well as the design costs), is to use a FPGA.
There is a list of papers related to using FPGAs for hash-cracking here.
A lot depends on what you mean when you say your budget is "limited". Big FPGAs can be several hundred dollars a piece, and that's just for the IC (integrated circuit)! The great majority of FPGA-based hash crackers that have been built by amateurs are done by reverse-engineering existing products from e-bay that use FPGAs, commonly real-time video encoding/processing devices.
Really, if you're serious about this project, you should spend some time researching and understanding HDLs. FPGAs (as well as ASICs) are not "programmed" in C, or any other common language you may be familliar with. Instead, they use a language (called a 'Hardware description language") that describes a series of logic operations and registers, that all operate in parallel. It's quite different then C.
Then, you should purchase an inexpensive FPGA development board (digilent makes some nice ones), and get your hashing algorithm working and tuned. Then you could look at scaling up.
It's worth noting that none of this would run linux at all. Everything ASICs do (as well as FPGAs) is so-called "bare metal" the way a ASIC/FPGA works is fundamentally different then a microcontroller, and there is no such thing as a FPGA "operating system". The extent to which linux (or any operating system) is likely to be involved is simply communicating with the special-purpose hardware and feeding it things to hash/reading back hashing hits. This would probably occur over USB, or if you're using a platform like the Zynq, through shared memory.
It's interesting that you mention the parallela people, as that's a pretty good example of the costs involved in doing a fairly complex ASIC. They needed about ~$750,000 to get their design produced as hardware.
Best Answer
For basic info check out the link:
http://www.msc.rl.ac.uk/europractice/vendors/date2010_imec_foundries.pdf.
Europractice is similar organization to MOSIS with discount for European academic/research institutions. As a bonus from the link you will get a feeling what does it cost to make prototype.
For gate count per square mm rule of thumb is that each generation of process will double gate density with .18u having around 100k/mm^2 (so .13u should have around 200k/mm^2 and .25u around 50k/mm^2).
However I would never make such important decision as technology choice based on such rudimentary info. You will have to contact prototyping service (mosis/europractice) and get as detailed info as possible. As it is "digital" circuit what would be most important is as detailed info on standard cells as possible. To make things more complicated each technology will have several standard cells libraries available with different options/tradeoffs.
Also 100 picoseconds per pipeline stage sound like typo - this would translate to 10GHz clock frequency, which is out of reach even for most advanced techs. If you meant 100 nanoseconds (10MHz) I believe this is well within 350nm tech capabilities.
Side note:
I admire boldness of you endeavor and wish you success but you have to prepare for significant cost and getting some external expertise. The IC design is orders of magnitude more expensive than PCB design (both engineering and prototyping). Unless you are with academic/research facility and have prospect for sizable grant or VC backing (or VC grade money for the project) most likely you will not be able to complete it.