Palo Alto earns short list status

App-aware firewall proves especially useful for controlling outbound traffic

Palo Alto Networks has injected excitement and innovation into the firewall market with its "next-generation" appliances that combine traditional firewalls, threat mitigation technologies such as anti-malware and intrusion prevention, and the new magic dust of application identification.

We first tested Palo Alto in late 2008 and found the PA-4020 to be an interesting product that still needed work. This time around, we tested Palo-Alto's newest high-end appliance, the PA-5060 and found plenty to love.

The product clocked multi-gigabit speeds even with all threat mitigation and identification features enabled, proving that it's capable of conducting deep session analysis in an enterprise setting. In fact, using the exact same test scenario, the PA-5060 forwarded traffic 10 times faster than the product we tested in 2008 (see story, "Palo Alto PA5060 is one fast firewall").

With a solid basic firewall feature set and UTM protections such as anti-malware and intrusion-prevention system (IPS), the PA-5060 can be used for inbound traffic. And its application awareness makes it even better suited as an outbound firewall, giving extended visibility into what is happening, and fine-grained control over what is allowed.

Of course, no product is perfect. Palo Alto Networks is a relatively new company with limited resources, and features such as centralized management, Web-based GUI, VPN and network access control-like user identification and host scanning could be improved upon.

Next-gen firewalls: Off to a good start

Fast-forwarding firewall faceoff

However, none of these rough spots should stop network managers from looking carefully at the PA-5060, especially when tackling the thorny problem of outbound access control. The PA-5060 is also able to replace some Web security gateways, with the advantage of combining firewall and gateway in a single device.

Effective outbound traffic control

Security-conscious network managers have long known that port number is not the same as application. For example, two applications can share the same port, such as Skype and Web browsing over TCP Port 80. And, an application can change ports. For example, some network managers run SSL VPN servers on TCP Port 53, normally reserved for DNS, to tunnel through many pay-per-use Wi-Fi hotspots that allow DNS, but not much else.

A firewall rule that allows incoming traffic to specific ports is generally sufficient to control traffic, since you control your own servers and know what applications are running on them — in theory, at least. While the PA-5060 can be used for inbound traffic to enterprise networks, we focused most of our evaluation on outbound traffic, such as Web browsing.

Outgoing traffic has long ignored the idea of specific port numbers, with applications of all types running over whatever port seemed good at the time. Network managers using port restrictions to control applications such as Amazon Cloud Drive or Google Talk File can't easily do so, because those applications are happy to run over the traditional port for encrypted Web traffic, 443.

Controlling outbound mail traffic by redirecting everything sent to ports 25, 587 and 495, the common SMTP ports, to enterprise mail servers works only as long as no one out on the Internet is running SMTP on yet another alternative port.

Net results

Even without the effects of crafty applications and crafty users, enterprise network managers want greater granularity than allowing or denying all Web browsing. Because so many applications are now run through Web browsers, allowing general Web browsing while blocking specific applications is one of the reasons to use a next-generation firewall. Or, you may want to go deeper, such as letting people read blogs, but not let them post to blogs. Or read Facebook, but not run Facebook applications or use Facebook chat.

We tested the PA-5060 to determine if it could do what Palo Alto says it can: effectively control application traffic. With about 1,300 application signatures, we knew that we couldn't test every application. So, we picked a set of 100 common applications that we thought network managers might want to use as policy building blocks.

Our first challenge came because many applications can be effectively blocked using URL filtering, and the PA-5060 handles URL filtering separately from application identification. This requires some rethinking of security policy, because you may use URL filtering for some parts and application identification for others. For example, to block access to investing Web sites, we used "financial services" URL filtering, but to block people from posting to financial Web sites, we created an application group with applications such as "motleyfool-posting" and "google-finance-posting" to control that traffic. In some cases, such as blocking Internet proxies, we used both URL filtering and application identification.

In the end, we were able to block 83% of the applications we set out to, and we could probably have improved that score by adding our own custom signatures.

Our overall conclusion is that the PA-5060 has more than most managers would need to effectively control outbound application usage in enterprise networks. While splitting application controls between URL filtering and application identification can be error-prone and time-consuming, we didn't find the additional work to be a show-stopper.

We did discover that working with the PA-5060 requires some new ways of thinking. In theory, since you're writing rules based on application identification, you actually don't have to put port numbers in your firewall rules, at least in the outbound direction.

However, by not putting port numbers in, the PA-5060 allows all outbound connections, only cutting off traffic once something prohibited is found. That can have other side-effects, such as letting short bits of traffic out that you weren't expecting. We found a bug in the beta version we first evaluated using an outbound policy without most port numbers. In this case, the PA-5060 allowed entire connections that should have been prohibited. Palo Alto Networks fixed the bug quickly, but it's an example of the kind of issue that can come up.

Since the PA-5060 must look deep into outbound traffic to identify applications, some type of man-in-the-middle SSL decryption is required. Configuring and using SSL decryption is not difficult, and the PA-5060 has a very well thought-out model for handling the subtle issues of SSL man-in-the-middle. For example, a proper configuration gives the PA-5060 two different certificates to use to re-encrypt traffic: one considered valid by end-users and one considered invalid.

This lets the PA-5060 properly pass back to the end-user cases where an invalid certificate was offered by an upstream Web site. SSL decryption is handled in a separate policy from normal firewall rules, another well-designed separation that helps to reduce confusion.

The last piece of next generation firewall wizardry we looked at was the ability to identify users, and write rules based on user, rather than just zone or IP address. While the PA-5060 definitely has this feature, there are significant limitations.

Gathering user identification information is hard. One option is the built-in captive portal, which may be useful in a guest wireless network or an environment where network users are accustomed to such gateways, such as a university network. But we don't think that enterprises would find this acceptable.

The other is via a Windows application which captures domain authentications and IP addresses out of the event logs from Active Directory servers, and sends this information to the PA-5060. That would work fine in a very constrained, very homogeneous environment, but would miss a lot of users in others.

This difficulty isn't actually the PA-5060's fault: no one has come up with a particularly good way to capture this information short of deploying 802.1X and some type of NAC on their LAN. Unfortunately, if you did enable 802.1X, the PA-5060 has no ability to gather that information from NAC products.

With user identification fully integrated into the normal policy rules, it's easy to constrain outbound access based on who the user is. Unfortunately, getting that information is so difficult that we think this will have limited usefulness in large networks and work best in small environments, such as branch offices, which have simpler and more consistent architectures. The PA-5060 is definitely not the right firewall for that environment, but Palo Alto has other models sized for the branch office.

User identification is the incomplete piece that would let the PA-5060 take on another significant enterprise security market, the Web security gateway or Web proxy. The PA-5060 does everything that enterprise products from vendors such as Blue Coat Systems, Cisco, McAfee and Websense do.

If Palo Alto Networks wants to fully take on these giants in all market niches, it will need to work harder on the user identification part of the PA-5060. For now, the application control and filtering features of the PA-5060 are all you need to get rid of a standalone Web security gateway — if you're happy with the user identification options Palo Alto Networks gives you today.

Even if you don't choose to write access control rules based on user identification, there is a good reason to collect that information: it appears in the log files. Since every session through the PA-5060 firewall is logged with extensive information, adding user identification just makes the job of the security analyst that much easier. For this reason, we'd recommend turning on user identification, even if it doesn't properly identify 100% of network users.

Eye-opening visibility features

Traditionally, firewalls have been control points for security, and network managers who want to know what is happening on their networks turn to other devices and other vendors. We expect that next-generation firewalls will begin to be more than control points, and will start to also answer the question "what traffic is passing over the network?"

This visibility and reporting function won't obviate the need for IDSs, Netflow tools and protocol analyzers. Instead, it will supplement them by giving detailed information on what is, and is not, passing through the firewall itself.

The main reason for adding visibility, a radical increase in functionality, is the need for matching context between reporting and rule writing. The additional functionality of a next-generation firewall, including application identification and user identification, requires reporting to use the exact same terms as the firewall controls.

For example, our PA-5060's reports broke traffic to our Web servers into five separate categories: web-browsing, web-crawlers, http-audio, http-video, and flash. Without knowing exactly how the firewall was going to categorize the traffic, we couldn't effectively write rules to control the traffic. In other words, without knowing the context the firewall would use, we couldn't know how to use the firewall properly.

Adding a visibility function also makes sense because the firewall has already done most of the work. Next-generation firewalls are constantly engaging in deep session inspection to identify applications, threats, URLs, and other attributes of the session, in order to apply their rules. Therefore, it's reasonable to re-use that work by reporting on what is passing through the firewall.

The Palo Alto Networks PA-5060 has so many good reporting and visibility tools that it's easy to forget that it's a firewall. But that's important to keep in mind, because although the tools work great, they only take you so far — and the PA-5060 is not a complete replacement for other visibility-focused products.

The main starting point for viewing your traffic is the PA-5060's Application Command Center, which shows a constantly updated list of top applications, source and destination hosts and countries, URL filtering categories, IPS rules, data types (such as ZIP, PDF, Excel file, etc.), and a few other categories. This view alone will likely be an eye-opener for most network managers, as most of these data are not available in traditional visibility tools.

For example, the PA-5060 told us that the top application (both in terms of sessions, and byte counts) running on our networks was not what we expected -- Web traffic of some kind, or maybe email -- but something completely different: DNS. More importantly, by clicking through the Web-based reports, we were able to drill-down further into the summary information to identify the misbehaving hosts.

The Application Command Center only has summary information in it, giving you lists of up to 500 items (whether applications, threats, hosts, URL categories, and so on) for any timeframe you choose. That sounds good, although we found that asking for more than the past 24 hours worth of data was slow going. Still, if you wanted to get those top lists for the past month and were willing to wait a few minutes, the PA-5060 could give it to you.

The Application Control Center is great for what it is, but it's not re-summarizing the firewall logs every time you click. This means that trying to filter the data on something that the PA-5060 is not reporting on can't be done. For example, since there is no report on "port number," you can't summarize all traffic on Port 80.

Hacker 'handshake' hole found in common firewalls

To get deeper, the PA-5060 has a pastiche of additional tools. The most useful include log analyzers and periodic reporting tools. Jumping between the Application Control Center and the detailed log analysis tools is easy, because once you've narrowed down what you want to look at in the Application Control Center, the filter is automatically passed over into the log analyzer.

The log analyzers are one place where it's easy to get carried away. For example, once you've spent a while building a great filter to identify a particular subset of traffic, it would be nice to be able to dump that filter into a reporting tool and get a summary report. Unfortunately, you can't — although we found it hard to hold this against the PA-5060, because it already was telling us more information about our network, faster, and in more detail, than most of the other visibility tools we had.

Not everything is perfectly thought-out. A series of flashy tools collected under the generic title "App Scope" are just silly, ranging from the poorly designed Summary page, which mixes graph types and time frames into a confusing jumble of misdirection, to the "why is this useful" traffic map, which draws different sized and colored dots on a map in an attempt to show where your traffic is going, at the country level.

Still, the visibility tools are so good that it's difficult to find serious fault with the PA-5060. We certainly had a higher level of visibility than any other firewall has given us. We don't think that you'd want to buy the PA-5060 as a visibility tool all on its own, and Palo Alto Networks isn't selling it that way. But having such sophisticated and powerful ways to look into the network traffic crossing your firewall could easily sway the decision to buy the PA-5060 instead of a traditional firewall, even if you didn't think you wanted next generation firewall functionality.

Basic firewall features are solid

Before a firewall can be a next-generation firewall, it has to cover the basics of a plain-old firewall. In our 2007 tests of UTM firewalls, we identified characteristics that any firewall must have to be considered a proper enterprise product, including firewall, VPN, and NAT functionality, advanced networking support (such as link aggregation and virtual LANs), high availability and high performance, global management, extended IP functionality (routing, QoS and IPv6), and global management.

We started by testing basic firewall functionality: writing rules to allow and block traffic, implement source and destination NAT policies, and build site-to-site VPNs. Network managers familiar with enterprise products from Check Point, Cisco and Juniper will find the Palo Alto firewall user interface both intuitive and familiar.

The main configuration interface is a Web-based GUI, although a command-line interface (CLI) is also available. Network managers who prefer the easy simplicity of Cisco IOS or Juniper's ScreenOS CLI will discover that the Palo Alto's CLI is more like editing raw XML, and not nearly as simple or straightforward to learn or use.

The productivity-killing part of the Web-based GUI is the commit model. After making changes in the user interface, you have to "commit" the changes to the firewall. There's nothing wrong with that, except that a commit takes about 25 seconds. When you're making occasional changes, the commit delay is a mild annoyance. When you're debugging and making rapid small changes, slow commits become a significant speed bump. Don't think that using the CLI will save you -- the same commit delay applies.

We took our lab's existing main production Juniper ScreenOS firewall policy, with 182 rules in it, and tried to convert it to fit into the Palo Alto firewall. The job was easier than we had imagined, because Palo Alto has fixed one of our long-standing design complaints about the Juniper firewall: the inability to put more than one security zone in a single firewall rule. The PA-5060 supports rules with more than one security zone, which let us shrink our policy down by a third. Smaller policies simplify firewall management, and reduce the risk that human error will introduce a security hole. The Palo Alto Networks' firewall won points for both transparency and simplicity.

We were also able to move our NAT configuration, with both source and destination NAT rules, easily. We had less success, though, in trying to move our site-to-site VPN configuration. The Palo Alto firewall has an extensive site-to-site IPsec VPN capability, and we were successfully able to build tunnels and pass traffic with Cisco, Juniper and SonicWall firewalls.

We think that network managers with large VPN deployments will not want to move to Palo Alto quite yet. Panorama doesn't build or manage VPNs across firewalls, meaning that any large deployment would have to be built entirely by hand. At the same time, Palo Alto firewalls only support route-based VPNs, meaning that traffic is pushed into tunnels by routing tables rather than the Security Policy Database called for by the IPsec standards. That meant we had to do some re-engineering of policy and our NAT configuration to fully re-build our five-site VPN. For small VPNs, or even large VPNs that don't change very often, Palo Alto firewalls will work just fine. But the product isn't yet at the same level of power and flexibility that other enterprise firewalls support.

The PA-5060 firewall passed all our other enterprise firewall functionality tests with no problems and a minimum of unexpected behavior. We were successfully able to configure and use dynamic routing, QoS features, and networking features, such as link aggregation and VLAN tagging. You shouldn't buy the PA-5060 firewall to use primarily as a WAN router or a bandwidth management device, but that's no different a conclusion than we'd make about any of the enterprise firewalls on the market today. The PA-5060 supports both active/passive and active/active high availability, but we did not test this feature.

We were also impressed that the PA-5060 came out-of-the-box with a full set of IPv6 capabilities, including firewall rules, application identification and control, static routing, and management and monitoring tools. The only missing piece is dynamic routing. Our testing did exercise some IPv6 bugs, though, as we managed to lock up the IPv6 side of the firewall, requiring a system reboot.

Panorama makes it easy to transition from single firewall to multiple firewall management. Security policies and policy objects can be built in Panorama and pushed to multiple firewalls, which covers the biggest use case for centralized management.

The Panorama management model will be attractive to network managers who want to share management between a central authority and individual network managers, such as in branch offices. Panorama doesn't take over the entire firewall; instead, the Panorama-created security policy is merged with individual policies on each firewall.

Even if you don't use Panorama for configuration of firewalls, it still brings a benefit by collecting and reporting on log information for multiple firewalls at the same time.

As an enterprise firewall, the PA-5060 can certainly be a credible competitor to products from Check Point, Cisco, Juniper and SonicWall. While we found some weaknesses, network managers should definitely consider Palo Alto firewalls for enterprise deployments.

UTM features are broad, granular

Next-generation firewall vendors don't like the term "UTM" (Unified Threat Management) very much because UTM products have been unfairly painted as only appropriate for small businesses. However, next-generation firewalls need threat mitigation features just as much as UTM firewalls do. While the buzzword police fight out the differences and split hairs, we tested the PA-5060's UTM features, including intrusion prevention (IPS), anti-malware and URL filtering.

The PA-5060 lumps threat management and mitigation into a set of seven policies collected together as Security Profiles. The seven policies include the traditional anti-malware (split into anti-virus and anti-spyware policies), vulnerability protection (Intrusion Prevention policy), and URL filtering, as well as the slightly-more-unusual file blocking (which lets you block certain file types from upload or download or both), data filtering (data leak protection), and DoS protections.

For every rule that lets traffic through the firewall, you can apply a separate Security Profile. This would let you apply, for example, one set of DoS protections to seldom-used Web servers and a different set to heavily-used ones. Or, you could apply different IPS signatures for incoming traffic than for outgoing. Since PA-5060 rules can also include user identification, you could even have different sets of URL filtering rules depending on whether the user is identified or not. We tried all of these things and were able to successfully show a high level of granularity when applying UTM protections on traffic flowing through the PA-5060.

Overall, the configuration of UTM features is easy and flexible. Unlike some firewalls where the UTM features are system-wide or apply to all traffic, we found the ability to tie different threat protection profiles to different sets of traffic both intuitive and useful. The PA-5060 has adopted an easy-to-use model with the right amount of flexibility.

When we looked in detail at some of the UTM features, we found that the anti-malware was good, but not great. We tested using a set of fresh viruses that had been caught by our enterprise anti-malware scanner over the 24 hours prior to our test and had a 75% capture rate. The PA-5060 does not use a third-party anti-malware engine; Palo Alto has its own engine that combines multiple threat protections (IPS and anti-malware) into a single über-engine. This suggests that the PA-5060 is good as a secondary anti-malware protection device, but does not obviate the need for other gateway and desktop anti-malware.

With URL filtering, we also had typical results for all engines: our testing showed a few false positives and a few false negatives, along with the usual mis-categorizations.

We configured, but did not rigorously test, file blocking, data filtering and DoS protection capabilities. File blocking lets you identify certain file types that can then be blocked for either upload or download or both. We found that the file blocking was easily fooled. For example, putting a file into a zip archive effectively hid the file type, as did changing the first few bytes of the file (by adding blank lines) and, in one case, changing the filename — which we didn't expect to work. Data filtering, a type of data leak protection, successfully let us search for strings and wildcards in various applications flying by, but really isn't powerful enough to qualify as a data leak protection solution.

To test intrusion prevention, we fed the PA-5060 a live Internet feed of approximately 40Mbps for several weeks and watched what it told us. As with most IPS-in-a-firewall products, the PA-5060 doesn't match the flexibility and power of dedicated IPS products.

However, the policy management system is exceptionally good for this class of device, and most network managers will find it easy to configure policies and examine events. Policies select threats from Palo Alto's own library of about 1,900 threats, and are applied to firewall traffic rules. This integration of IPS and firewall rule is an important one, because it lets you select very different IPS policies on a granular basis for different types of traffic based on zone, IP address, port number and the all-important application identification.

When defining policies, Palo Alto encourages you to use the "simple" settings, which offer a list of severities (critical, high, medium, low, and informational) and an option for each severity, essentially "block" or "allow." You can also take Palo Alto's advice and select "default," which will pick whatever mysterious default Palo Alto shipped with the IPS signatures.

However, you do have the option to select "custom" settings, which lets the network manager pick a specific action for each signature. In this mode, more options are available, including simply dropping packets, resetting connections and even blocking IPs (a common request after detected brute force attacks).

Each policy, whether "simple" or "custom," also has a list of exceptions -- threats that should be ignored. One critical and valuable feature of the PA-5060 GUI is the ability to go directly from a log entry to the exception list with just a few clicks, and without losing your place. This lets you handle false positives quickly and get back to the difficult work of interpreting IPS events.

Analyzing events from the PA-5060 IPS is easy, and the GUI has some nice reporting tools built-in to simplify the task of managing IPS events. For example, we defined an IPS report to show us the top events for our critical systems. Once we ran the report, we were very pleased to see that all of the elements were "hot links" that let us drill down into the actual IPS event logs very quickly. These reporting and analysis tools are scattered around the management system, intermixed with other tools for other parts of the firewall (such as traffic logs or anti-malware logs).

Palo Alto chose to group log analysis tools by the type of tool, rather than the part of the firewall generating the log information, which makes it somewhat cumbersome to just concentrate on IPS events without, for example, stumbling over "top 10 URL categories" along the way. Still, we found ourselves relatively expert at flying between reports, logs, log filters, and events after only a few hours of practice. Network managers for whom IPS analysis is a part-time job will find the PA-5060 offers considerable power without a lot of complexity.

The IPS management system in the PA-5060 will not replace a dedicated IPS console, but it represents one of the most sophisticated IPS event analysis tools we've ever seen in a firewall. The IPS console in Panorama offers the same capabilities as the firewall GUI, except that Panorama is able to report on events from multiple devices at once. We found some bugs in Panorama's reporting that blocked us from generating full reports, so we concentrated our testing on our PA-5060 by itself.

As with all IPSs, the PA-5060 required some tuning before we were ready to set it loose on our production network. We focused on IPS events marked as "high" and "critical" severity, and immediately found a number of important things going on in our network--and no high or critical false positives.

The top 50 events, which the PA-5060 marked as high severity, were all brute force login attempts to SSH servers, Windows Terminal Server servers, FTP servers, and mail servers. Fair enough, and accurate. Same with Conficker, which was hitting our network on average every 30 seconds, along with someone looking for cross-site scripting vulnerabilities, and a variety of other break-in attempts for common exploits and vulnerabilities to all our Web servers.

When we were doing our testing, we quickly discovered an important detail: if you ever want to understand what your firewall is telling you, it's critical to enable not just threat logging, but also URL logging and traffic logging. Without all three pieces of information, the IPS logs themselves are vague enough that many events cannot be tracked or understood. This has important implications for enterprises deploying the PA-5060 with IPS features enabled -- you also have to enable logging on everything else.

The "no false positive" rule didn't hold quite so well as we moved down the severity category to "low" and "informational" events. In those areas, we found a number of false positives. Since the PA-5060 is being sold as an in-line device, we didn't find these false positives a big issue, as network managers deploying the PA-5060 shouldn't be depending on anything other than "high" and "critical" events.

When we were investigating these top events, we found a common frustration: poor documentation on the threat and vulnerability database. For example, one threat that caught our eye was something the PA-5060 called "PDF Exploit Evasion." Looking that up on Palo Alto's threat database portal gave us no useful information: no CVE number, no description or threat analysis other than "PDF exploit evasion has been found on your network." Enterprise IPS products generally include extensive documentation on threats to help the network manager understand criticality and impact, and the PA-5060 hasn't met this standard.

We also tested the PA-5060 IPS using a Mu Dynamics test chassis and their published vulnerabilities tests. When we tested UTM firewalls using the same test chassis in 2007, most firewalls using recommended settings blocked only about a third of the attacks (the average score across all products was 32% block rate, with a low of 14% and a high of 75% using recommended settings). The PA-5060 did better in this test, blocking 90% of the attacks in the client-to-server direction and 93% of the attacks in the server-to-client direction. Since our tests are four years apart, it's difficult to draw any conclusions from these results, other than to say that the IPS in the PA-5060 seems to do a good job on the 1,954 vulnerabilities in our Mu Dynamics tester.

Enterprise network managers looking for better control and higher security in their firewalls need to pay attention to Palo Alto Networks. The PA-5060 goes beyond legacy products from the big three enterprise firewall vendors - Cisco, Juniper and Check Point -- and has earned its place on evaluation short lists.

Snyder, a Network World Test Alliance partner, is a senior partner at Opus One in Tucson, Ariz. He can be reached at

You Might Like
Join the discussion
Be the first to comment on this article. Our Commenting Policies