Speed traps

While our services tests assessed how well Web front-end devices handled application traffic, our scalability tests can help properly size one of these products for a particular network's needs.

The scalability tests demonstrated the limits of system performance in terms of maximum concurrent-connection capacity, TCP multiplexing ratios and maximum forwarding rates. In all these areas, the test results show big differences among devices.

In the maximum concurrent-connections test, our goal was to determine how many client connections a device could handle. There were major differences among vendors in this test.

To determine connection count, we configured Spirent's Avalanche to emulate as many as 4 million clients running Internet Explorer. Each client opened a TCP connection and requested a 1KB Web object from the Web front-end device's virtual one or more IP addresses (just as a single IP address for, say, www.amazon.com hides dozens or hundreds of servers, all these devices used one virtual IP address as a proxy for the back-end servers on the test bed).

After receiving the object, clients sat idle for 60 seconds before requesting another object over the same connection. This long "think time," how long a client waits before requesting the next page, allowed us to build up connection count. For all vendors, we kept adding new connections until the device failed to complete some transactions, or until we reached 4 million connections, the limit of our test bed. Even though our goal was a Layer-4 measurement - the number of established TCP connections - we used Layer-7 switching in this and all other tests.

Citrix's NetScaler Application Delivery System set up 4 million concurrent TCP connections, the limit of our test bed (see graphic at right). F5's BIG-IP was next, setting up about 3.5 million connections, followed by Foundry's ServerIron 450 with about 2.7 million connections.

Juniper's DX 3600 topped out at 2 million connections with four systems working together and 500,000 concurrent connections on a single box. The vendor says its appliance has a hard-coded limit of 500,000 connections per system, something reflected in our test results.

The Crescendo and Array systems each sustained fewer than 1 million connections. Crescendo says its device has a hard-coded limit of 1 million connections, a few of which are reserved for internal use.