Supersizing existing WAN connections

Squeezing 10 pounds of data into an existing 5-pound connection can save a lot of money. Network data compression is becoming so commonplace that router vendors have begun including limited compression features - such as TCP/IP header compression - in most models. However, for maximum compression performance that can be tailored to fit your specific traffic patterns, you should consider dedicated network compression devices.

We tested five systems - from BoostWorks, Expand Networks, ITWorx, Packeteer and Peribit Networks - to see how they compressed data through a limited network connection and what the deployment and management costs amount to.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

Squeezing 10 pounds of data into an existing 5-pound connection can save a lot of money. Network data compression is becoming so commonplace that router vendors have begun including limited compression features - such as TCP/IP header compression - in most models. However, for maximum compression performance that can be tailored to fit your specific traffic patterns, you should consider dedicated network compression devices.

We tested five systems - from BoostWorks, Expand Networks, ITWorx, Packeteer and Peribit Networks - to see how they compressed data through a limited network connection and what the deployment and management costs amount to.

All products go well beyond shrinking packet headers by also significantly compressing the packet payload. In our FTP testing, we achieved as much as a 400% performance gain, which represents a best-case compression scenario.

The performance gains from these products heavily depend on the traffic types and flows running across a network (see how we set up our lab.) Your mileage definitely will vary.

Within the broad category of "network compression," the vendors found many ways to squeeze data into a standard pipe. The major differences lie in how compression is defined and what traffic is compressed.

The algorithms these products use depend, broadly, on one of two techniques: redundant string compaction and replacement dictionary lookup. Redundant string compaction replaces strings of repeated characters, or strings with regular patterns, with smaller replacements and reconstruction instructions. Dictionary lookup also targets repeating strings and patterns, but instead of inserting compacted replacement strings, lookup keys pointing to dictionary entries are inserted into the traffic stream.

Compression algorithms have a significant effect on performance, but larger differences center on whether compression is applied to all traffic or only traffic from a particular source, and how much of the overall network traffic stream already is compressed.

All products except BoostWorks' attempt to compress all network traffic, unless a setup parameter has exempted a particular type of traffic or traffic from a particular address.

The BoostWorks product attempts to compress only particular types of traffic, such as HTTP, Simple Mail Transfer Protocol (SMTP) or specific application transactions. Its designers are banking on the assumption that most enterprise traffic falls into one of the targeted patterns.

It also makes a difference on complexity whether compression is unidirectional (from server to client, for example, where a network compression device acts as a server and compresses specific types of traffic, and decompression occurs on the client) or bidirectional (end-to-end, where a compression device sits on both ends of the network segment to handle compression). All products in this review support bidirectional compression. BoostWorks also supports unidirectional compression.

Most devices pay close attention to the TCP port from which the traffic originates, although Packeteer also considers information from the packet header.

Most products - except those from BoostWorks and ITWorx, which don't compress User Datagram Protocol (UDP) traffic - could compress voice-over-IP (VoIP) traffic we threw at them by 20% to 25%. In our FTP tests, a 146M-byte uncompressed copy of the Linux kernel took 39 minutes to transfer. When we used compression devices, transfer times shrank to between less than 10 minutes up to just over 19 minutes (see graphic).

If data passing across the network is repetitive, caching effectively can reduce the amount of traffic passing through the network - Expand and Peribit support caching. When we threw a pre-compressed 2M-byte file at Peribit's box, the first and second times yielded a transfer time of 34 seconds. The third time around, the same file took two seconds. Expand's box exhibited similar behavior. This sort of caching/reduction would benefit an enterprise link, where copies of the same file are being sent around through e-mail or FTP transfers.

Packet aggregation effectively joins small LAN packets into jumbo packets to reduce header overhead costs. This can be important on satellite links, where network latency on one side of the connection is a factor. While packet aggregation had a significant effect in our UDP performance tests (changing packet aggregation timing values increased performance greatly), network managers should consider whether to apply packet aggregation to UDP traffic because the process can have an adverse effect on latency. Most of the products tested support packet aggregation. Packeteer and Expand let you configure their devices to support this feature while ITWorx and Peribit support it transparently to the user.