Mapping SSL VPN manageability

Our Clear Choice Test expert evaluates SSL VPN for manageability.

While the bulk of our SSL VPN evaluation was anchored in very objective test results, we also had to make some tough subjective calls about the products’ management interfaces. In most cases, the devices offer very similar management systems, but we ran into some unusual pieces that are worth considering as part of the buying decision.

Management, of course, is not just how pretty or ugly the interface is. We evaluated eight areas of management, ranging from the GUI and online help to SNMP support, partitioned management capabilities and status reporting and auditing features.

We didn’t simply check the presence of a feature, but, rather evaluated how well it was done. For example, although every vendor but SonicWall offered SNMP management, Array, Aventail, Caymas, F5 Networks, Juniper and Nortel impressed us by collecting statistics and alerts specific to the SSL VPN functionality of their devices.

One feature that enterprises commonly require is partitioned or delegated management. The cleanest implementations, in the Array SPX-5000, Caymas 525 and F5 FirePass 4160, let the device manager grab entire pieces of the SSL VPN deployment and give someone else complete control. (Nortel’s VPN Gateway 3070 also offers this as part of its separately licensed VPN virtualization feature, which we did not test.) More common in the service provider space, this also can be useful when different groups within the same enterprise are sharing a common infrastructure. With most of the SSL VPN devices we tested running about $60,000, the temptation is to build a single large, high-availability cluster and share it, rather than try and give every project its own device. Fortinet’s Fortigate-3600 and Juniper’s SA-6000 also offered either delegated or partitioned management that might be appropriate for an enterprise deployment.

Configuration management is certainly an important part of an SSL VPN device, and we looked at configuration interfaces in great detail. But once a device is configured, operations management can be just as important. Features such as the ability to see who is logged on and terminate a session (missing in Check Point Connectra, Fortinet Fortigate-3600 and Nortel VPN Gateway 3070) from the management interface can be important.

Areas where SSL VPN vendors do not offer apples to apples features sets are logging and auditing. Enterprises should have the option to get extensive and complete logging, as much as for debugging purposes as for accounting reasons. Better devices also have log management tools, such as automated rollover and export of log files to an archive server. And at the high end, we expect devices to offer special debugging logging (such as the ability to track a single user session) and RADIUS-based accounting. The Caymas 525, F5’s FirePass 4160, Juniper’s SA-6000 and Nokia’s Secure Access System 500s scored highest by covering most or all of these bases, while AEP’s Netilla Security Platform and SonicWall’s SSL-VPN 2000 don’t have as much flexibility and control over logging.

Auditing comprises its own category nowadays, and we found some seriously good and some seriously bad auditing capabilities among the products tested. Caymas, Juniper and Nortel all have auditing down pat, enough to impress any compliance officer. With auditing enabled, you get sufficient data to know exactly who did what in the management and control interface. Fortinet’s Fortigate-3600, Array’s SPX-5000, SonicWall’s SSL-VPN 2000 and AEP’s Netilla Security Platform don’t offer capabilities such as separation of auditing from usage logs and persistent storage of auditing information that we believe will be acceptable to a persnickety auditor. If you have special auditing requirements, you should carefully consider capabilities of these products to see if they meet your needs.

Reporting is not an area where any of these devices are particularly strong. Because there are so many ways to slice and dice the usage data, that’s not too surprising. Generally, we would expect that actual usage reporting would be handled via an external Web log analysis tool, and the on-device reporting would be limited to capacity-planning support. F5 and Juniper go to some length to try and generate useful graphs to help in capacity planning, with most other devices also having some capabilities. The only devices we found wanting in this were Check Point’s Connectra and Nortel’s VPN Gateway 3070, neither of which had any graphing capabilities.

Finally, we looked at documentation, both printed and online, as part of our management evaluation. Array, Check Point, Fortinet and Nortel all have failed to really meet basic standards for documentation. While Array and Nortel certainly meet the requirement for pure bulk, having a lot of bad documentation isn’t really what managers need with these products. We were also disappointed that AEP, Array, Caymas, Juniper, Nokia and SonicWall haven’t figured out how to provide context-sensitive help for their management GUIs. Yes, it’s hard, but it’s a lot easier than most of the other software they had to write to build an SSL VPN device.

After looking at each of these areas, F5 and Juniper offer the best overall management based on our criteria, followed by Aventail, Caymas and Nokia. AEP, Check Point, Nortel and SonicWall were either missing important management features or had done a poor job implementing some of them.

GUIs as windows to the management soul

One product stood out for having a GUI we just couldn’t handle: Array’s SPX-5000. In addition to poor design and multiple bugs in some very basic areas, the GUI is largely useless when it comes to appliance configuration. Array aims at the service provider and service providers don’t use GUIs for configuration, which explains why its interface is so weak. For basic monitoring and continuing administration operations, the Web-based GUI is just fine. However, when we wanted to configure security policy on the Array device, we absolutely had to use the command-line interface (CLI).

Nortel’s VPN Gateway 3070 management GUI is only a few notches above Array. We only had to resort to using the CLI a few times to get the system working, but it’s pretty clear that the CLI is the fastest way to get things done, overall. The weakness here, as with Array, is on the configuration side. Monitoring and basic operations were acceptable, although with Nortel’s extensive SNMP support, you might expect much of that to be handled via a separate SNMP-based management station.

Another user interface for management of note came with our Caymas 525 system. Unlike all other SSL VPN vendors, Caymas uses a Java-based application to manage its appliance. While the application is nice, the crippling problem is that it takes 4 to 5 minutes to get the full GUI up and running. We haven’t seen a GUI that took that long to start since we ran Lotus Notes on a 90MHz Pentium – and that was last century.

Caymas has other severe interface weaknesses in that it packed thousands of intrusion-detection system (IDS) signatures into the Caymas 525 that you could apply signatures to any service. However, the Java interface is so poorly designed and implemented in handling these IDS signatures that we’d be astonished if anyone took advantage of the feature for long.

By in large, the other products had fairly similar management systems. The most obvious differences in the GUIs were a consequence of product complexity. Check Point’s Connectra GUI is easy to use and intuitive, requiring very little training or documentation. However, Check Point’s product doesn’t have the amazingly large set of SSL VPN features that are in Juniper’s Secure Access 6000 platform. Most of the complexity in the Check Point management interface lies in the area of protective services, namely Check Point’s SmartDefense, that run on the Connectra device, and even that area is not as complex as a normal IDS.

Juniper’s SA-6000 GUI has been widely criticized as complex. Juniper could have done a much better job. It could have made life easier for network managers configuring the SA-6000 system. But it’s not impossible to learn once you start to figure out which parts you can ignore. For example, when setting access control rules, the options for caching, Java, rewriting, single sign-on, SAML, Web Proxy, JSAM and compression are seldom touched.

Juniper deserves some credit for incredible manageability in other areas. For example, when we were doing our high-availability testing, we had to bind a second Juniper SA-6000 to the existing SA-6000 under test. With a serial cable attached to a console, we gave the new box the bare minimum of information it needed to find the existing cluster and that was as much work as we had to do. The new box found the existing one, detected that the software versions were out of sync, automatically downloaded the package from the existing system, upgraded itself and formed a cluster. Someone at Juniper spent a lot of time and effort making that easy to do, even though it’s not the kind of thing you do very often. To the credit of Aventail, Fortinet and Nortel, clustering their boxes was also a painless process. Contrast this to our AEP, Array and Caymas cluster exercises, which took hours along with technical support calls to accomplish.

Aventail and F5 also have management systems in need of a warning. Both are elegantly designed and deceptively easy to use, unless you’re a security nut. This is because one of the ways that both achieve such elegance is by pushing some difficult security configurations into the fringes of the product. With Aventail’s “you never know how someone is going to access this resource” approach, we had a great deal of trouble coming up with a security policy that matched our requirements yet took advantage of Aventail’s technology for simplifying the end-user experience. F5’s GUI has a similar issue: It’s beautiful and generally easy to use, until you realize that you’re spending most of your time configuring how and where things appear on the portal, and getting fine-grained access control is hard.

Authentication interoperability | Next test: Portal Control and Virtualization >