Firewall – Determining cause of TCP/IP connect issue without traceroute

Advantages of firewall:

1. You can filter outbound traffic.
2. Layer 7 firewalls (IPS) can protect against known application vulnerabilities.
3. You can block a certain IP address range and/or port centrally rather than trying to ensure that there is no service listening on that port on each individual machine or denying access using TCP Wrappers.
4. Firewalls can help if you have to deal with less security aware users/administrators as they would provide second line of defence. Without them one has to be absolutely sure that hosts are secure, which requires good security understanding from all administrators.
5. Firewall logs would provide central logs and help in detecting vertical scans. Firewall logs can help in determining whether some user/client is trying to connect to same port of all your servers periodically. To do this without a firewall one would have to combine logs from various servers/hosts to get a centralized view.
6. Firewalls also come with anti-spam / anti-virus modules which also add to protection.
7. OS independent security. Based on host OS, different techniques / methods are required to make the host secure. For example, TCP Wrappers may not be available on Windows machines.

Above all this if you do not have firewall and system is compromised then how would you detect it? Trying to run some command 'ps', 'netstat', etc. on local system can't be trusted as those binaries can be replaced. 'nmap' from a remote system is not guaranteed protection as an attacker can ensure that root-kit accepts connections only from selected source IP address(es) at selected times.

Hardware firewalls help in such scenarios as it is extremely difficult to change firewall OS/files as compared to host OS/files.

Disadvantages of firewall:

1. People feel that firewall will take care of security and do not update systems regularly and stop unwanted services.
2. They cost. Sometimes yearly license fee needs to be paid. Especially if the firewall has anti-virus and anti-spam modules.
3. Additional single point of failure. If all traffic passes through a firewall and the firewall fails then network would stop. We can have redundant firewalls, but then previous point on cost gets further amplified.
4. Stateful tracking provides no value on public-facing systems that accept all incoming connections.
5. Stateful firewalls are a massive bottleneck during a DDoS attack and are often the first thing to fail, because they attempt to hold state and inspect all incoming connections.
6. Firewalls cannot see inside encrypted traffic. Since all traffic should be encrypted end-to-end, most firewalls add little value in front of public servers. Some next-generation firewalls can be given private keys to terminate TLS and see inside the traffic, however this increases the firewall's susceptibility to DDoS even more, and breaks the end-to-end security model of TLS.
7. Operating systems and applications are patched against vulnerabilities much more quickly than firewalls. Firewall vendors often sit on known issues for years without patching, and patching a firewall cluster typically requires downtime for many services and outbound connections.
8. Firewalls are far from perfect, and many are notoriously buggy. Firewalls are just software running on some form of operating system, perhaps with an extra ASIC or FPGA in addition to a (usually slow) CPU. Firewalls have bugs, but they seem to provide few tools to address them. Therefore firewalls add complexity and an additional source of hard-to-diagnose errors to an application stack.

Is this a set of hand-crafted rules?

One thing that jumps out: the network 192.168.0.0/24 is on two interfaces: eth1 and eth2 - which is almost certainly not what you want.

For general debugging, you can also use this command:

watch -d iptables -L -v -n

(or some variant of that.) Then you can watch packets increase as traffic flows. This is only really good for low traffic networks where you don't have a lot of extraneous traffic.

Another thing to do is to turn on logging for some of the firewall entries: use the LOG target extension.

Another thing: reset the counts and see what is getting dropped - or better yet, use that LOG target combined with rules that DROP. In your list, there are only two rules that have been matched and have dropped packets (pulling rules out of context):

Chain INPUT (policy DROP 223 packets, 9229 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   88  3768 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           state INVALID 
   18  1602 DROP       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp flags:!0x17/0x02 state NEW 

Perhaps these should be investigated?

To get away from that mess entirely, I'd suggest using Firewall Builder, an excellent GUI-based multi-firewall designer. You can design the firewall anywhere and plunk it down where it will do the work - fwbuilder even has ways of pushing the firewall out to the remote host.