WatchGuard Firebox V200 firewall/VPN

SOHO vendor moves into high-end arena

WatchGuard Technologies, a leader in small office/home office firewall/ VPN appliances, is targeting the big boys with a high-end device aimed squarely at Cisco's PIX 535 and NetScreen Technologies' NetScreen-5200.

WatchGuard Technologies, a leader in small office/home office firewall/ VPN appliances, is targeting the big boys with a high-end device aimed squarely at Cisco's PIX 535 and NetScreen Technologies' NetScreen-5200.

On the plus side, the $60,000 device is a lot less expensive than competing products from Cisco, NetScreen or Nokia, and its management graphical user interface will be familiar to users of other WatchGuard products.


How we did it

Interactive Scorecard and NetResults

Archive of Network World reviews

Subscribe to the Product Review newsletter


And in our tests, WatchGuard's V200 set up an impressive 42,000 concurrent IP Security (IPSec) tunnels, a useful capability when dealing with huge numbers of dial-up users in an corporate setting.

But on the minus side, the latest beta unit bore out the adage that security always comes with a performance cost. Even with a much-reduced number of IPSec tunnels in place, the V200's latency and throughput were much degraded compared with its performance when configured as a firewall.

The V200 offers firewall, VPN and network address translation via two Gigabit Ethernet interfaces. The V200 also offers Border Gateway Protocol routing and two out-of-band interfaces for high-availability applications.

We assessed the V200 with seven different performance measurements (see how we conducted our test). Besides determining IPSec tunnel capacity, we also measured latency and throughput with IPSec configured and with two and 1,000 firewall rules in place.

IPSec tunnel capacity

We established 42,000 tunnels using Spirent Communications' SmartBits analyzer running TeraVPN test software. These were fully formed tunnels that dial-up users would build when connecting through a V200. Each tunnel consisted of an Internet Key Exchange (IKE) session and pair of one-way security associations.

It's important to apply this three-element definition of tunnels - one IKE session plus two one-way security associations - when assessing VPN gear for dial-up use. A common trick in VPN specsmanship is to set up impressively large numbers of security associations but neglect to mention that all security associations were set up with one IKE session.

The issue is that many IPSec devices employ high-speed silicon for encryption but not for key exchange. The V200 has eight ASICs for acceleration of key exchange, encryption and firewall rule processing.

The V200 offered impressive tunnel capacity, but the beta version shone a bit less brightly when it came to moving packets through those tunnels. We measured throughput with 42,000 pairs of security associations between a pair of V200 devices, and we also tried a few test runs with 4,096 concurrent tunnels, but there wasn't any traffic level we could offer where packet loss was zero.

This isn't to say that the V200s can't forward traffic through all 42,000 pairs of security associations. In some cases (such as with 1,440-byte frames, the optimal length for IPSec testing), the amount of packet loss was trivial. Nonetheless, the Internet Engineering Task Force defines throughput as the highest rate with zero loss.

Our workaround for the latency and throughput tests was to scale back to 64 pairs of security associations (all done with one IKE session, a limitation of the back-to-back device configuration we used). In that less-stressful configuration, the V200 forwarded traffic without loss.

Dealing with delay

We measured latency and throughput in three configurations: with eight pairs of IPSec security associations configured between V200s; with IPSec disabled and two firewall rules in place; and with IPSec disabled and 1,000 firewall rules in place.

Latency is higher with IPSec enabled than without it - about four to six times higher (see Figure 1 ). The most pronounced increase was for 1,518-byte frames, the maximum length allowed in Ethernet. We observed average latency of 818 microsec across a pair of V200s.

Added delay for maximum-length frames is not surprising, considering that IPSec's encapsulating security payload method fragments and then reassembles these frames. Given the added processing involved, an extra 100 microsec of latency - 818 microsec vs. 725 microsec for unfragmented 1,440-byte frames - is not a huge increase.

Latency with IPSec disabled, but with firewall rules enabled, was far lower. More impressive still was that latency was essentially the same with two firewall rules configured or 1,000 rules. That's because the V200 loads all firewall rules onto its ASICs.

Testing throughput

Throughput, like latency, is lower on the V200 when IPSec is enabled than when it's acting purely as a firewall. In this test, there was a significant throughput difference depending on whether the V200 had to fragment frames (see Figure 2 ).

With maximum-size 1,518-byte frames, which get fragmented, throughput was equivalent to about 23% of line rate. With 1,440-byte frames, which IPSec doesn't fragment, throughput more than doubled to the equivalent of 54% of line rate, or nearly 533M bit/sec.

The V200's throughput compares favorably with the 440M bit/sec claimed by Cisco for its flagship PIX-535 in a similar configuration, but it's less than the line-rate numbers NetScreen cited for its flagship NetScreen-5200 line.

The throughput picture was brighter when we configured the V200 strictly as a firewall. For all frame lengths, throughput with 1,000 rules in place is virtually identical to throughput with just two rules configured. WatchGuard credits the V200's ASICs for the fact that firewall rules have essentially zero performance cost.

However, there is a substantial performance cost when it comes to pushing medium- and short-length frames through the V200. When handling 1,440- and 1,518-byte frames, a pair of V200s configured as firewalls will forward traffic at line rate without loss. When forwarding 256-byte frames, the highest rate at which the same configuration will move traffic without loss is equivalent to about 54% of line rate. With minimum-length 64-byte frames, throughput falls to the equivalent of 10% of line rate.

Given that the average frame length for TCP/IP traffic is close to 256 bytes, and that TCP packets must receive 64-byte acknowledgement frames in return, it can be said that the V200 will deliver more than adequate throughput on networks with light to medium loads. Users on heavily utilized segments might experience packet loss.

 WatchGuard Firebox V200
 Company: WatchGuard Technologies Cost: $60,000. Pros: New hardware supports very large numbers of IPSec tunnels; no performance hit for large number of firewall rules. Cons: Late beta hardware and software exhibited elevated packet loss and latency with IPSec configured.

Learn more about this topic

Newman is president of Network Test in Westlake Village, Calif., an independent benchmarking and network design consultancy. He can be reached at dnew man@networktest.com.

Network Test and Network World thank Spirent Communications for supplying its SmartBits test system and TeraVPN software, and for helping develop and troubleshoot the scripts used. Special recognition goes to Spirent's Bob Anderson, Eddie Arrage, Paul East, Debbie Landis, and Davison Zhang. Thanks, too, to Extreme Networks' Huy Nguyen, who supplied Summit workgroup switches for the test bed.

NW Test Alliance

Global Test Alliance

Newman is also a member of the Network World Global Test Alliance, a cooperative of the premier reviewers in the network industry, each bringing to bear years of practical experience on every review. For more Test Alliance information, including what it takes to become a member, go to www.nwfusion.com/alliance.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Now read: Getting grounded in IoT