We tested application-acceleration devices for performance, features, manageability and usability.

To assess performance, we constructed a test bed modeling an enterprise hub-and-spoke network with five sites: a headquarters in Boston and branch offices in Portsmouth, N.H.; Newton, Mass.; El Segundo, Calif.; and San Francisco (see graphic “The Logical Test Bed”).

We used a Spirent Converged Network Impairment Emulator to emulate WAN rates and delays. The Newton and San Francisco remote links ran at T1 (1.5-Mbit/s) rates, while the other two ran at T3 (45-Mbit/s) rates (see graphic “The Physical Test Bed”).

The Newton and Portsmouth links used 16-millisecond round-trip delays, while the other two used 100-millisec round-trip delays, representing all permutations of low and high bandwidth and delay.

We measured application acceleration performance with CIFS/SMB Windows file transfers, Outlook/Exchange, HTTP and SSL traffic. In a separate test, we assessed devices’ QOS capabilities by generating VoIP traffic while simultaneously loading the network with HTTP traffic. We developed custom software for this project to generate CIFS/SMB and Outlook/Exchange traffic.

To measure bandwidth reduction, we used a ClearSight hardware-based analyzer with taps in both the Boston LAN and WAN sides of the test bed. To measure application-response time, our custom software measured CIFS and MAPI transfers.

For the CIFS file transfers, two clients at each remote site simultaneously sent and received Microsoft Word documents from the Boston site. Clients on T3 links transferred 750 files in each direction, while clients on T1 links transferred 25 files each way. We ran each CIFS test three times: a “cold” run with empty device data stores, a “warm” run once the data store had been populated and a “modified” run in which we altered the contents of 10% of the files.

To measure MAPI/Exchange performance, Outlook 2007 on each client created 1,000 or 34 messages for T3 or T1 circuits, respectively. Some of the messages had Microsoft Word files as attachments, and some were simple text. Each client sent messages to all other clients, but always through and Exchange 2003 server at the Boston site.

To measure HTTP performance, we configured the Spirent Avalanche and Reflector 2500 appliances to emulate Web clients and servers, respectively. As many as 2,048 clients at remote sites requested 11KB objects from servers at the Boston site. We measured HTTP response time and transfer rates. We repeated these tests twice, once with 256 clients across all remote sites, and again with 2,048 clients.