AV's place is not in the all-in-one security box

There is no real agreement about whether antivirus software is required in or even a good idea for an enterprise-class firewall.

Some consider antivirus software protection irrelevant in a unified threat management (UTM) firewall deployment, because a desktop antivirus application and an e-mail security appliance are doing a good job scanning for viruses. Others consider sitting inside the UTM a huge bonus, because they want every possible "defense-in-depth" feature to block viruses at other places on the network.

The former attitude proved the most defensible one based on our testing. Not only did we see incredible performance problems when antivirus scanning was included in the UTM mix, but we also found that these firewalls don't do a very good job of finding viruses in any event.

We ran into flawed implementations, and bugs and hidden features that were needed to make antivirus scanning work properly.

We started our testing knowing that most vendors feel that UTM-based antivirus scanning is useful in the small-to-midsize business sector, but not necessarily in gigabit-speed enterprise firewall deployments. Exactly where antivirus stops being useful is not clear.

We discovered quickly that few of these participating vendors take antivirus software seriously. Some don't even include it in their high-end boxes. For example, Juniper Networks' ISG-1000 makes you pick between virus and intrusion-prevention protection. The Cisco ASA5540 doesn't give you any antivirus-management options.

Some vendors do give antivirus a fighting chance, though. Secure Computing's Sidewinder gives network managers tight control of antivirus scanning parameters. For every rule that allows traffic through the firewall using the HTTP, FTP and SMTP protocols it supports, you can specify what to scan, and what Multi-purpose Internet Mail Extensions types to scan.

The Sidewinder got a perfect score in blocking all our FTP, SMTP and HTTP viruses. However, when we tried to send viruses through the firewall using a nonstandard HTTP port, the Sidewinder missed them all. That scanning comes at a moderate performance cost, though, with antivirus scanning dropping the throughput of the Sidewinder 2150D by about 50%.

Sidewinder fans might protest this as an unfair test, because we had to create an "un-Sidewinder-like" policy to allow traffic out on nonstandard ports. Maybe so, but the bottom line is that you can't cover all traffic with virus protection unless you know about it ahead of time.

When we tried to configure SMTP scanning, we discovered why Secure Computing added to its portfolio Ironmail, an antispam-antivirus messaging-security gateway. Although Sidewinder is sold as an appliance firewall, there are multiple dead ends, where the firewall ends and you're suddenly managing a Unix system. E-mail is one of those. If you want to scan e-mail for viruses, you have to configure Sendmail on the Sidewinder. You're limited in what direction you can scan e-mail. In addition, performance was miserable across SMTP, with the antivirus scanner taking 64 seconds to handle 20 e-mails.

Fortinet certainly knows how to set up antivirus scanning. Every rule passing traffic through the FortiGate 3600A firewall can call for a protection profile that enables antivirus scanning. In addition, the FortiGate box supports a wide number of protocols, adding messaging and Network News Transfer Protocol to standard mail (SMTP, POP, IMAP), Web and FTP protocols. FortiGate's logs were complete, and the performance was incredible, scanning at 500Mbps.

Unfortunately, the FortiGate 3600A initially caught only 60% of the viruses we sent through it. It also had an annoying configuration that had the box pass on virus-containing e-mail after it had stripped away the virus. Because most viruses are not attached to legitimate e-mails, sending through junk mail without a virus is no longer a best practice in enterprise e-mail management. We were disappointed there was no way just to tell a system to scan all traffic on all ports. If there's any product in this entire test that might be used for enterprise-speed antivirus scanning, FortiGate was it, except that you can't scan all traffic on all ports.

Additionally, it turns out that FortiGate's secret to success is hidden in its command-line interface (CLI). While SMB users of smaller FortiGate firewalls may be happy to use the Web-based GUI, enterprise managers will have to become proficient at the CLI to have the control needed. By turning on a heuristic virus-scanning feature, we increased FortiGate's catch rate by as much as 100% on mail, FTP and SMTP.

Juniper's high-speed ISG-1000 and ISG-2000 firewalls don't support antivirus. We tested Juniper's SSG-520M, which does. With a fairly standard policy configuration covering FTP, HTTP and mail protocols SMTP, POP and IMAP, the SSG-520M missed two viruses it should have caught, and -- like most other firewalls -- didn't catch anything on nonstandard ports. With antivirus scanning enabled, the SSG-520M dropped to a speed of about 160Mbps.

Our multiple Check Point Software submissions behaved quite differently when antivirus was put into play. The IBM System x3650 server included Check Point's brand-new CoreXL technology, which lets the Check Point software take advantage of multiple-CPU or multicore systems, but does not include antivirus scanning or URL filtering.

For platforms that do support antivirus scanning (including the Check Point UTM-1 2050, Crossbeam Systems C25 and Nokia IP290), the Check Point management system's levels of detail and control are disappointing. Whatever antivirus policy is set up is applied to all gateways managed within the same SmartCenter. That might be fine if you were managing 500 branch-office devices, because you probably would want the same policy for every single branch. That said, antivirus scanning causes an 87% minimum performance hit on the Check Point gateways we tested. That kind of performance degradation calls for greater specificity in configuration than "one policy for all."

Check Point platforms that do support antivirus scanning cover the SMTP, POP3, FTP and HTTP protocols. Again, nonstandard ports, such as Web servers running other than on Port 80, can't be scanned for viruses automatically. On Check Point's own UTM-1 2250 hardware and on Crossbeam's C25 system, the firewall let through three viruses on covered ports and all viruses on noncovered ones. That, combined with the massive slowdown (our Crossbeam C25 dropped to about 60Mbps, a very slow rate for such high-speed hardware) and the policy-definition problems mentioned earlier suggest that the antivirus function on Check Point firewalls should be left turned off.

We had a different problem with the Nokia IP290 firewalls, also running CheckPoint VPN-1 software. Using the same policy, they blocked the three viruses the C25 and UTM-1 missed. However, Nokia's firewalls can't do antivirus scanning on virtual LAN ports, a strange configuration restriction. We had to reengineer our test bed to get performance numbers on the IP290 firewalls.

SonicWall also originally threw us for a loop. Although SonicWall scans a long list of protocols, including HTTP, FTP, SMTP, POP, IMAP, CIFS and generic TCP streams, something was seriously wrong in the customer-release software we tested: The Pro 5060 caught 42% of the viruses, even though the configuration should have covered 100% of the traffic. SonicWall engineers shipped us a special bug-fix build that solved the problem, taking SonicWall to the highest antivirus-catch score, 85%, of the entire test. Performance in antivirus scanning was about 210Mbps, down from nearly 600Mbps without antivirus scanning.

We also expected great things from WatchGuard Technologies, another veteran of the SMB space, but were disappointed. Like SonicWall, WatchGuard covers all protocols easily, and it was the only vendor to catch our nonstandard HTTP viruses. Unfortunately, although WatchGuard caught most of the viruses we sent through e-mail, it missed all the viruses we sent through FTP, giving a total coverage of 45%. The performance hit for turning on virus protection was heavy, bringing total throughput down to less than 200Mbps from more than 1Gbps.

This strange behavior is likely a side effect of the way the WatchGuard Firebox Peak works. As a proxy-based firewall, Firebox Peak applies antivirus to a proxy, and only three proxies are supported: SMTP, HTTP and generic TCP. This means that because FTP uses its own proxy, FTP traffic won't be covered by antivirus scanning. We were impressed by the wide variety of controls in the Firebox Peak proxy for antivirus scanning. No one will say the Firebox Peak proxies have insufficient controls for what and how to scan viruses.

Our only complaints are about the defaults. For example, by default, Firebox Peak blocks all Java, ZIP and EXE files and archives. That might be safe for a small business, but enterprise network managers probably will need to spend time ripping out some of the default configuration before Firebox Peak meets their needs.

Two other products that have enjoyed success in the SMB and branch-office environments and that gave us enterprise-class antivirus support are Astaro Internet Systems' ASG 425a and IBM Internet Security Systems' Proventia MX5010, which blocked most -- but not all -- viruses on standard ports and missed out on nonstandard port traffic.

Cisco's ASA5540 has a single slot, which means that you're forced to choose between its IPS and its antivirus modules -- and Cisco says its enterprise customers generally pick the IPS module. For this reason, we couldn't test antivirus features in the ASA5540 firewall. In our final scorecard, we reported on our scores for antivirus coverage and management for products with antivirus scanning capabilities, but did not include the antivirus scoring in the bottom-line final score for each product.