Firewall – Tracking changes to firewall configs

Advantages of firewall:

1. You can filter outbound traffic.
2. Layer 7 firewalls (IPS) can protect against known application vulnerabilities.
3. You can block a certain IP address range and/or port centrally rather than trying to ensure that there is no service listening on that port on each individual machine or denying access using TCP Wrappers.
4. Firewalls can help if you have to deal with less security aware users/administrators as they would provide second line of defence. Without them one has to be absolutely sure that hosts are secure, which requires good security understanding from all administrators.
5. Firewall logs would provide central logs and help in detecting vertical scans. Firewall logs can help in determining whether some user/client is trying to connect to same port of all your servers periodically. To do this without a firewall one would have to combine logs from various servers/hosts to get a centralized view.
6. Firewalls also come with anti-spam / anti-virus modules which also add to protection.
7. OS independent security. Based on host OS, different techniques / methods are required to make the host secure. For example, TCP Wrappers may not be available on Windows machines.

Above all this if you do not have firewall and system is compromised then how would you detect it? Trying to run some command 'ps', 'netstat', etc. on local system can't be trusted as those binaries can be replaced. 'nmap' from a remote system is not guaranteed protection as an attacker can ensure that root-kit accepts connections only from selected source IP address(es) at selected times.

Hardware firewalls help in such scenarios as it is extremely difficult to change firewall OS/files as compared to host OS/files.

Disadvantages of firewall:

1. People feel that firewall will take care of security and do not update systems regularly and stop unwanted services.
2. They cost. Sometimes yearly license fee needs to be paid. Especially if the firewall has anti-virus and anti-spam modules.
3. Additional single point of failure. If all traffic passes through a firewall and the firewall fails then network would stop. We can have redundant firewalls, but then previous point on cost gets further amplified.
4. Stateful tracking provides no value on public-facing systems that accept all incoming connections.
5. Stateful firewalls are a massive bottleneck during a DDoS attack and are often the first thing to fail, because they attempt to hold state and inspect all incoming connections.
6. Firewalls cannot see inside encrypted traffic. Since all traffic should be encrypted end-to-end, most firewalls add little value in front of public servers. Some next-generation firewalls can be given private keys to terminate TLS and see inside the traffic, however this increases the firewall's susceptibility to DDoS even more, and breaks the end-to-end security model of TLS.
7. Operating systems and applications are patched against vulnerabilities much more quickly than firewalls. Firewall vendors often sit on known issues for years without patching, and patching a firewall cluster typically requires downtime for many services and outbound connections.
8. Firewalls are far from perfect, and many are notoriously buggy. Firewalls are just software running on some form of operating system, perhaps with an extra ASIC or FPGA in addition to a (usually slow) CPU. Firewalls have bugs, but they seem to provide few tools to address them. Therefore firewalls add complexity and an additional source of hard-to-diagnose errors to an application stack.

This isn't a complete answer, but hopefully some useful thoughts:

inotifywait will work happily with multiple files and can recursively set watchpoints on directories. For example, I can run:

inotifywait -m -r -e close_write /etc

And get the following log after editing /etc/passwd and /etc/postfix/main.cf:

/etc/ CLOSE_WRITE,CLOSE .passwd.swpx
/etc/ CLOSE_WRITE,CLOSE .passwd.swp
/etc/ CLOSE_WRITE,CLOSE 4913
/etc/ CLOSE_WRITE,CLOSE passwd
/etc/ CLOSE_WRITE,CLOSE .passwd.swp
/etc/postfix/ CLOSE_WRITE,CLOSE .main.cf.swx
/etc/postfix/ CLOSE_WRITE,CLOSE .main.cf.swp
/etc/postfix/ CLOSE_WRITE,CLOSE 4913
/etc/postfix/ CLOSE_WRITE,CLOSE main.cf
/etc/postfix/ CLOSE_WRITE,CLOSE .main.cf.swp

You could very easily work this into a script that, upon each close_write event, would commit the file to the local repository and push the changes to a remote server.

Also note that incron is a nifty tool for automating this sort of inotify-based workflow (but wouldn't substantially change the nature of the solution).

You're going to run into difficulties if your admins will be editing the same files that get updated by the server at runtime. This suggests that you're going to have to set up some sort of automatic conflict resolution on the server, which will invariably result in losing some information (well, apparent loss, at least, you can obviously preserve conflicting changes in two distinct branches of the repository but the server only gets to see one branch).

I don't think any parts of this problem or solution are particular to git. You're going to run into these issues with any sort of distributed, asynchronously synchronized shared maintenance of files.