How we did it

Secure Sockets Layer VPN testing turned out to be a fairly complex task. We started by building a test network consisting of client systems, the SSL VPN device and servers running different enterprise applications. Each SSL VPN device would be used to connect clients to servers, and we’d record the results of interoperability tests.

This immediately raised two questions: which clients to use and which servers. To determine which clients were important, we analyzed the HTTP Web server log files for a recent one-month period to see which clients are used most commonly. Because SSL VPN users might not be working from company-owned and controlled systems, we let the general Internet distribution of browsers guide us. We analyzed approximately 3 million unique visitors to find which browsers account for at least 1% of the systems, and came up with five browsers: Internet Explorer versions 5 and 6, Netscape versions 4.7 and 7, and Apple’s Safari browser, spread across various versions of Macintosh and Windows operating systems.

We installed several client Windows systems running Windows 2000; some with the most recent patch kits and others patched only up to Service Pack 3. The Windows systems ran two versions of Internet Explorer (Version 5 and an up-to-date Version 6) and two versions of Netscape (Version 4.7 and an up-to-date Version 7.1). We also borrowed a PowerBook G4 from Apple to run three browsers on Macintosh OS X (Internet Explorer, Netscape and Safari).

On the server side, we identified 20 typical enterprise applications for SSL VPNs, including some simple Web applications in pure HTML, applications using JavaScript, iNotes from IBM, Outlook Web Access from Microsoft, WhatsUp from Ipswitch, several test Macromedia Flash applications, Java-based applications from Altio, Microsoft’s Terminal Services, Citrix Systems’ MetaFrameXP, Windows file servers, Network File Systems file servers, FTP file servers, terminal emulation using Telnet and SSH, NetScreen Technologies’ Global Pro Firewall management system, and mail services using the standard Post Office Protocol, Internet Message Access Protocol and Simple Mail Transfer Protocol. Although we wanted to include some more complex enterprise applications, such as SAP, the time requirements and expense of installing them proved too much for our test team.

We set up a Windows 2000 server running VMware’s GSX server with multiple virtual machines and installed each application onto a different virtual machine. The GSX host machine had dual 2.4-GHz CPUs and 3G bytes of memory, which was plenty for our test because we weren’t looking at performance.

On the outside network, where the clients were located, we also installed a small Linux system to handle DNS and Dynamic Host Configuration Protocol queries. Our outside network was disconnected at all times from the Internet.

To test interoperability, we configured each SSL VPN device to connect to our client and server networks. We issued certificates using our OpenSSL Certification Authority and set the usual parameters for tasks such as routing, DNS and time service. Then we added in the 20 applications to each device and used each client to attempt to connect to each application. We collected 140 data points for each SSL VPN device to determine its interoperability and application support scores.

We also evaluated each device using a set of 12 evaluation criteria that had been distributed to all SSL VPN manufacturers before the review. A copy is available here. These criteria helped guide our testing and organize the rest of our evaluation of SSL VPN products.