Firewalls can be difficult to manage. Often times rules lack granularity due to a lack of understanding of the application traffic or trying to keep up with the speed of business. Firewalls get more and more rules added to their policies to the point of becoming "Swiss cheese". Don't give up hope because there are methods and tools available to help you gain control of your firewall policies.
A firewall simply implements the security policy of an organization. It is pointless to implement a firewall without strong policies defining traffic types that are permitted or denied. Good firewall design policy specifies the rules used to implement the minimum permissions required to allow the application to function. This policy must be designed with the capabilities and weaknesses of firewall and the application in mind. Typically firewall policies are based on one of two strategies. Either they permit any service unless it is expressly denied or they deny all services unless expressly allowed. The later is the preferred industry best practice fail-safe stance. However, during security assessments, we still find firewall policies where the first rule is "permit ip any any".
You want to configure a stateful firewall with a policy that provides maximum security. That means configuring rules that have extreme granularity in terms of connection direction, source and destination IP address and application port number. Often this is difficult because this information is sometimes not provided to the security administrators to be entered into the firewall policy. Application owners may not know the comprehensive list of IP addresses and port numbers for their applications and the data flow between physical/virtual servers. Therefore, there are no details to provide to the firewall administrator to facilitate that granular firewall policy
It is a common problem that firewall rules get added but there is no decommissioning of those rules. Firewall policies are a bit like a "roach motel"; rules check in but they don't check out. Over time firewall policies continue to grow until they are too large to be effectively managed. If you are not documenting why firewall rules were added and if your organization has turn-over of experienced personnel then no one in your organization can explain what is in the firewall policy.
We find that many organizations have no documented policies and procedures for how the security teams operate. Most of the procedures of firewall policy management are tribal knowledge among the security teams. This type of vague policy can change over time and is subject to each individual's interpretation of the spoken rules. An individual may behave based on how they were trained. Overtime the procedures used can change and warp so that the security teams are not operating consistently or as desired.
Some organizations have a general process of making firewall changes. The process starts by someone within the company who needs a firewall modified making an entry in the service ticket. Sometimes there is a form attached to the ticket that describes the firewall rule/object request/change. The requests can sometimes contain an attached document if there are many rule changes requests. Otherwise, the request simple asks for src/dest/ports and src/dest IP addresses be permitted through the firewall. If this policy isn't documented then the organization should define this process and look for ways that this process can be improved.
Another important component of firewall policy standards for organizations that have many people making changes to many firewalls is the concept of common object and rule creation. Everyone working on creating or modifying firewall policy should follow a for common method for object/rule naming/numbering. It is easy to anticipate the problems that could arise if every security engineer had their own naming conventions and ways that they liked to create policy. A benefit to the organization occurs when people generally adhere to the naming and commenting of rules. Regardless, your organization needs these policies to be documented and all groups making firewall and ACL changes must adhere to the naming conventions.
The issues arise when, due to the speed of doing business, people in the company are driving the firewall policy request and the changes have to be made immediately. Another similar problem is when there is a very large the number of TCP/UDP ports or a large number of source/destination IP addresses that are listed on the request. If the security team questions these coarse firewall policy requests and stops to educate the requestor on firewall policy granularity, then they can help the requestor strive for a more secure policy change. The security engineering team should be able to push back on the policy granularity to help ensure the security of the entire company. The security engineers should be able to decline requests when the firewall changes being requested are "too loose". There should also be policies that allow security engineering to defend their decision to push back on unreasonable "requests".
IT leaders should support security engineers when they push back on overly permissive firewall policy additions. After all, the security engineers are only trying to help the organization be more secure. If the security engineers are not supported and are overruled often then the security engineers will stop trying to make the system more secure. If these good security practitioners do not get management support then they will stop pushing back and accept the firewall change requests "as is". The culture will then allow for overly permissive firewall rules and the security will be compromised. At some point it will take a lot of effort to go back and correct many of the overly-permissive policies that have made their way into the firewall configurations.
One suggestion on the writing of this firewall configuration policy is that it can describe the idea of a "policy spectrum". This is the idea that the decision to be cautious or optimistic about a request has to do with the risk that the request represents to your business. For the firewall rule/object addition policy to be effective it must address the issues of having consistency of granularity. For example, if a connection is going from a completely untrusted network to a very trusted network then the objects will need to be well defined and the rules should allow only specific TCP/UDP port numbers between specific hosts. And if a connection is going from a trusted management system to another management system then the policy can be less granular to make administration easier. The firewall policy should document these different regions of the network environment and give examples as to what constitutes a good firewall rule and what a bad rule looks like. Once this firewall administration policy is documented there will be improved firewall policy granularity, rule changes can be accelerated, and the configuration will be consistent among various groups making requests and among the security engineers making firewall policy changes.
Eventually the firewall policies become such a mess that the organization ends up launching an effort to cleanup of old rules and objects that are overly permissive. Organizations need a method of cleaning up of old rules that should be removed. One method to accomplish this is to look at rule hit-counts and disable the rule if it hasn't been hit in a week. If no one complains for another week then remove the rule and objects. It is possible to create auto-expiring rules for exceptions and then wait a week and delete the rule. This can be very time intensive to perform a full policy review of all the firewalls in place. The one problem with this approach is that some of the firewall logs don't go back far enough to determine if rules have had any hits. Rule cleanup is time intensive and is often performed "as time permits". Because the security engineering teams don't typically have time for this then this work is never performed. Because it is difficult to remove any rules at all on the firewalls the engineers have given up on the task and this important task is not performed.
In extreme situations when the firewalls have literally turned into "Swiss cheese" then one drastic option is to flip the policy. The organization may deem that the firewalls have essentially degraded to a "permit all" policy because of all the rules in the policy today. The organization can then start to add deny rules further up in the policy for those things that must be protected. Rules could be added higher up in the policy that block connections to the critical devices to prevent anyone from communicating with them. Therefore, a short-term quickly deployed method would be to add some rules to the policies that deny specific access to resources that should not be communicated to. However, if many of these block rules are added then it will really confuse any security engineer in the future who is truly tasked with rule cleanup.
Some organizations may look to automated tools to help them maintain their ACLs and firewall policies. There are many firewall operations and management products available on the market today. These products can help audit and verify the current rulebase and give advice on improvement or compare them to best practices or compliance guidelines. These tools can perform revision control, change control, and change management functions for firewalls. These tools can help you re-order the entries in the rule-base, improve granularity of rules, and find unused, contradictory or overlapping rules. Some tools can even perform "what if" scenarios and model the firewalls behavior or trace packets through the policy. Some tools also tie into trouble-ticket systems, other SIM/SIEM products, vulnerability scanning tools, notification and alerting methods
A year ago, Rob Smithers wrote an article titled "Extreme firewall makeover" where he covered five firewall management tools. Since that article was written it seems that the firewall management market has become populated by many companies that offer products to help you gain control of your firewall policies. A year ago Neil Roiter also wrote an article titled "Firewall audit dos and don'ts" which gives some good advice on how to chose a firewall audit product.
Here is a list of firewall operations and management products that I am aware of. Please let me know if there are others that you know of and would recommend.
Skybox Security - Firewall Assurance, Network Assurance, Risk Control Tufin - SecureTrack and SecureChange Workflow AlgoSec - Firewall Analyzer (AFA), FireFlow NetCitadel - Firewall Builder 4 Red Seal Systems - Network Advisor & Vulnerability Advisor Secure Passage - FireMon Athena Security - FirePAC, Firewall Grader, Firewall Browser Cisco - Cisco Security Manager (CSM) v4.1 Juniper - Network and Security Manager (NSM) ManageEngine - FWanalyzer Cyber Operations - Cyber ACL
If your organization doesn't have budget for a firewall management product then you can solve these same problems with discipline and process. Your firewall administrators should have the ability to push back on unreasonable requests for adding rules. If the group requesting the new firewall rule can't granularly document how their application works then the rule shouldn't be added to the firewall until more details are known. Firewall administrators can use other methods to document their changes to the policy using simple documentation practices or a team Wiki page. A trouble ticketing system or a workflow automation tool can be used to document who requested the firewall policy change and the technical background behind the request. Creating an audit trail for firewall rules doesn't have to cost a lot or take a lot of time.
Firewalls are just like any other IT project that requires a healthy mix of people, process, and technology. Frequently organizations put too much emphasis on the technology and forget about supporting that technology with high-trained and disciplined personnel that are following universally accepted and followed procedures. Many organizations deploy a firewall and think that they are safe without doing any maintenance. However, to effectively secure a networked environment you must take a methodical approach to proactively manage that firewall so it doesn't turn into Swiss cheese.