How we tested Palo Alto's PA-4020 firewall

We spent two weeks putting the Palo Alto Networks PA-4020 through a series of tests designed to measure its capabilities. During our testing, the PA-4020 was connected to the Internet and was able to download virus, threat, and URL filtering updates. We also updated the software on the PA-4020 once during the test, from 2.0.1 to 2.0.3. We did encounter problems during the update, and had to have Palo Alto's technical support team apply fixes to our configuration to make it compatible with the new software version.

We spent two weeks putting the Palo Alto Networks PA-4020 through a series of tests designed to measure its capabilities. During our testing, the PA-4020 was connected to the Internet and was able to download virus, threat and URL filtering updates. We also updated the software on the PA-4020 once during the test, from 2.0.1 to 2.0.3. We did encounter problems during the update, and had to have Palo Alto's technical support team apply fixes to our configuration to make it compatible with the new software version.

We started by installing the PA-4020 as a tap on an existing connection used to serve about 1,000 DSL users. During the initial installation, we looked at the management interface for the PA-4020 and evaluated the policy definition and visibility tools available in the PA-4020. Because the PA-4020 supports layer 2, layer 3 and tap mode, we were able to evaluate the capabilities of the system without interfering with existing traffic.

Once we were confident that we understood the operation, we installed the PA-4020 in-line as a layer 2 firewall with the same DSL connection, applying threat protections and some application blocking. We also waited for the phone to ring, possibly indicating that the PA-4020 was improperly blocking traffic. We did catch a few complaints and false positives at this point.

At the same time, we put the PA-4020 in-line with our live antispam/antivirus gateway to see how well it would catch viruses "in the wild". We let it run for a week, and then compared the logs of the PA-4020 to the logs of the antivirus scanner on the e-mail gateway to see which viruses the PA-4020 had caught, and which it had missed.

Next, we moved the PA-4020 to a more controlled environment, our own wireless network, and began to explore each of its capabilities in depth, including application identification, specific virus testing, SSL man-in-the-middle decryption, network address translation, firewall policy definition, URL filtering and intrusion-prevention/detection system (IDS/IPS) signatures. We ran numerous small tests to determine how well the PA-4020 performed each of these tasks. This part of the test also contributed to our evaluation of the policy definition tools and visibility tools in the PA-4020.

For antivirus testing, we took 15 recent (June 2008) viruses and packaged them using six vectors: e-mail via SMTP on standard and non-standard ports, FTP, HTTP on Port 80, HTTPS on Port 443, and HTTP again on a non-standard port. We used a client to transfer the viruses across the PA-4020 and looked to see which viruses were identified and blocked. As in our UTM test last year, we did not make any specific identification of the non-standard HTTP or SMTP ports. In our UTM test, we did not test for secure-HTTP blocking, but we did with the PA-4020.

To get a more rigorous view of the IPS capabilities, we turned to the Mu-4000 Security Analyzer appliance from Mu Dynamics, an attack generation and reporting tool. For the Mu-4000 testing, we focused on published vulnerability attacks. We broke up our testing into two directions: client to server, and server to client. Because the PA-4020 seems to be most suited to protecting clients, we wanted to separate out client protections (which had a higher score) from server protections. This also paralleled our UTM test from last year. In the client protecting case, we looked for the PA-4020 to protect users who are browsing the Internet or downloading files and thus are susceptible to certain types of attacks focused on client applications, such as Web browsers and PDF readers. In the server case, our tests were focused on attacks on Web, e-mail and other types of servers.

Like most IPSs, the PA-4020 has multiple levels of protections, dropping attacks into buckets labeled "Critical," "High," "Medium," "Low" and "Informational." The PA-4020 also separates out attack protections specifically into client and server attacks. For the server-to-client test, we set the PA-4020 to block Critical, High and Medium attacks on clients; for the client-to-server test, Critical, High and Medium attacks on servers. Then, we tested each profile using the Mu-4000 to see the percentage of attacks blocked by the IPS. The client profile had approximately 550 attacks, while the server profile had approximately 630. We believe that Palo Alto has had access to the Mu-4000 analyzer as well, so it is possible that the very high results achieved in this test (higher than any in our UTM firewall test) are a result of adjusting the signature database specifically to match the Mu-4000 attacks.

< Return to test: Palo Alto provides great visibility into network threats >

Learn more about this topic

 
Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2008 IDG Communications, Inc.