Security – ESXi hosted on public IP without firewall

Advantages of firewall:

1. You can filter outbound traffic.
2. Layer 7 firewalls (IPS) can protect against known application vulnerabilities.
3. You can block a certain IP address range and/or port centrally rather than trying to ensure that there is no service listening on that port on each individual machine or denying access using TCP Wrappers.
4. Firewalls can help if you have to deal with less security aware users/administrators as they would provide second line of defence. Without them one has to be absolutely sure that hosts are secure, which requires good security understanding from all administrators.
5. Firewall logs would provide central logs and help in detecting vertical scans. Firewall logs can help in determining whether some user/client is trying to connect to same port of all your servers periodically. To do this without a firewall one would have to combine logs from various servers/hosts to get a centralized view.
6. Firewalls also come with anti-spam / anti-virus modules which also add to protection.
7. OS independent security. Based on host OS, different techniques / methods are required to make the host secure. For example, TCP Wrappers may not be available on Windows machines.

Above all this if you do not have firewall and system is compromised then how would you detect it? Trying to run some command 'ps', 'netstat', etc. on local system can't be trusted as those binaries can be replaced. 'nmap' from a remote system is not guaranteed protection as an attacker can ensure that root-kit accepts connections only from selected source IP address(es) at selected times.

Hardware firewalls help in such scenarios as it is extremely difficult to change firewall OS/files as compared to host OS/files.

Disadvantages of firewall:

1. People feel that firewall will take care of security and do not update systems regularly and stop unwanted services.
2. They cost. Sometimes yearly license fee needs to be paid. Especially if the firewall has anti-virus and anti-spam modules.
3. Additional single point of failure. If all traffic passes through a firewall and the firewall fails then network would stop. We can have redundant firewalls, but then previous point on cost gets further amplified.
4. Stateful tracking provides no value on public-facing systems that accept all incoming connections.
5. Stateful firewalls are a massive bottleneck during a DDoS attack and are often the first thing to fail, because they attempt to hold state and inspect all incoming connections.
6. Firewalls cannot see inside encrypted traffic. Since all traffic should be encrypted end-to-end, most firewalls add little value in front of public servers. Some next-generation firewalls can be given private keys to terminate TLS and see inside the traffic, however this increases the firewall's susceptibility to DDoS even more, and breaks the end-to-end security model of TLS.
7. Operating systems and applications are patched against vulnerabilities much more quickly than firewalls. Firewall vendors often sit on known issues for years without patching, and patching a firewall cluster typically requires downtime for many services and outbound connections.
8. Firewalls are far from perfect, and many are notoriously buggy. Firewalls are just software running on some form of operating system, perhaps with an extra ASIC or FPGA in addition to a (usually slow) CPU. Firewalls have bugs, but they seem to provide few tools to address them. Therefore firewalls add complexity and an additional source of hard-to-diagnose errors to an application stack.

The answer to this is most likely: don't do that. Whatever it was you were doing to break pfSense, figure out what is was and don't do that any more.

I've worked with a plethora of linux and BSD-based router distributions, and pfSense is by far the most stable and flexible one of the bunch.

While there are a few pfSense folks here on Serverfault, I'd recommend taking specific questions to the pfSense mailing list. The project founders and many of the core devs are active on the list, along with a much larger group of active pfsense users than we have here.