Feeding the need for speed

Web front-end accelerators use a variety of techniques to speed delivery of content, including application-layer switching, HTTP compression and TCP multiplexing. That's the major conclusion of the industry's first comprehensive performance tests of Web front-end devices.

Web front-end accelerators use a grab bag of techniques to speed delivery of content, including application-layer switching, HTTP compression and TCP multiplexing.

The problem is that Web front-end devices either make traffic go very fast for a limited number of users or handle a very large number of users - but they can run into trouble doing both at the same time.

That's the major conclusion of the industry's first comprehensive performance tests of Web front-end devices. For nearly a year, we benchmarked devices from leading vendors Array, Citrix, Crescendo Networks, F5 Networks, Foundry Networks and Juniper Networks.

Among our findings in this inaugural test:

• The benefits of application acceleration are real. Properly implemented, devices can speed delivery of content to users while simultaneously lightening the load for data-center servers.
• HTTP compression doesn't always reduce response time, and can increase it in some situations, even with highly compressible text objects and very low client access rates.
• TCP multiplexing can dramatically reduce network processing overhead on servers, by nearly 350-to-1 in one case.
• Some devices don't offload much TCP connection processing when users request pages quickly (as on e-commerce sites).
• Web front-end devices exhibit big variations in the maximum number of connections they handle and the maximum rates at which they move traffic.

As is often the case with any new market category, there are substantial differences in terms of form factor, topological requirements and supported features (see "One size doesn't fit all").

Because of those differences, and because this is a relatively new product category and no device aced all tests, we're not scoring products this time around. As our results clearly show, different Web front-end vendors have put their development dollars in different places.

Two products deserve special mention. Crescendo's CN5080-E is a screamer of a performer, turning in top results in many of our tests because of its use of custom hardware for content switching. And Citrix's NetScaler Application System combines high scalability with a rich application-acceleration feature set.

To the test

Although an earlier test compared Web front-end features, the emphasis this time was on Web front-end performance.

Tests fell into two main areas. Services tests measured transaction rates and response times for a given number of users, both with and without HTTP compression applied, and with and without access control lists in place and distributed denial-of-service attacks underway (see services test results). Scalability tests demonstrated the limits of system performance in terms of maximum concurrent connection capacity, TCP multiplexing ratios and maximum forwarding rates (see scalability test results).

It's important to note that we're not claiming either set of tests represents "real-world" behavior for all users. There are too many variables in application load testing for a one-size-fits-all definition of that term to be meaningful (see "Your mileage will vary"). Still, these tests offer useful comparisons in that we offered the same traffic at the same levels to each device tested.

Our test bed emulated as many as 2.35 million unique clients and 16 Web servers (see "How we did it"). In some events, we pounded the devices with traffic at gigabit Ethernet LAN rates, pushing traffic at rates approaching 4Gbps. In other tests, we emphasized scalability in terms of large numbers of users or high transaction rates rather than focusing on high forwarding rates.

No device aced every event; rather, the results suggest different vendors optimized for different aspects of device performance. Some dramatically offload servers from the burden of TCP processing overhead, while others use HTTP compression to reduce response time. Devices also vary widely in terms of the scalability of connection counts and "goodput" rates, which reflect how quickly a device transmitted request HTTP object back to the client.

Faced with all these differences, the best advice we can offer in choosing a Web front-end device for your network is to assess which of these metrics matters most, and go for the box that delivers the biggest benefit in that area.

Newman is president of Network Test, an independent engineering services consultancy in Westlake Village, Calif. He can be reached at dnewman@networktest.com.

Newman is also a member of the Network World Lab Alliance, a cooperative of the premier reviewers in the network industry, each bringing to bear years of practical experience on every review. For more Lab Alliance information, including what it takes to become a member, go to www.networkworld.com/alliance.

Next test: Services >