Understanding delta compression

While WAN-compression solutions have been around for years, new compression advances have resulted in previously unheard of gains in bandwidth savings. Delta compression, commonly referred to as segment caching or byte caching, leverages pattern-matching techniques and large persistent dictionaries to dramatically reduce the amount of data sent across the WAN.

Delta-compression systems are symmetric, which means they require components, either software or hardware, on both ends of the network. In almost all cases, the server-side component is a dedicated appliance and the client-side component is a software module installed on the PC or an appliance deployed in the data path.

Software client-based systems have the advantage of requiring hardware on only one side of the link, making the approach suited for deployments in which there are only a few users per location. However, this flexibility comes at a cost. Client-based compression systems are limited in that they operate only on the data sent to that particular client. A file downloaded by a client therefore provides no benefit to other users. Furthermore, client-based compression systems require an additional download during initial application access. This download degrades first-access performance.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

While WAN-compression solutions have been around for years, new compression advances have resulted in previously unheard of gains in bandwidth savings. Delta compression, commonly referred to as segment caching or byte caching, leverages pattern-matching techniques and large persistent dictionaries to dramatically reduce the amount of data sent across the WAN.

Delta-compression systems are symmetric, which means they require components, either software or hardware, on both ends of the network. In almost all cases, the server-side component is a dedicated appliance and the client-side component is a software module installed on the PC or an appliance deployed in the data path.

Software client-based systems have the advantage of requiring hardware on only one side of the link, making the approach suited for deployments in which there are only a few users per location. However, this flexibility comes at a cost. Client-based compression systems are limited in that they operate only on the data sent to that particular client. A file downloaded by a client therefore provides no benefit to other users. Furthermore, client-based compression systems require an additional download during initial application access. This download degrades first-access performance.

While appliance-to-appliance delta compression requires hardware at both ends of the network, it offers significant performance advantages over client-based deployments. First, appliance-to-appliance delta compression allows cross_-user benefit. When one user downloads a file, the transferred bytes can be used to compress the same file when it is requested by a second user. Additionally, symmetric appliance deployments have no first-transfer penalty because no client code has to be installed. Finally, symmetric appliance deployments provide benefits not found in client-based systems, such as QoS capabilities.

In addition to hardware and software deployment techniques, delta-based compression can be achieved at different network layers. Some systems operate at lower layers of the OSI model, while others operate at higher layers. The layer of operation has a significant impact on compression effectiveness.

IP- or TCP Packet-based (Layer 3 or Layer 4) compression systems buffer packets that are then compressed one at a time or as a group and sent to the decompressor. The primary problem with Layer 4 compression is that it compresses packets individually or mixes different data types together. Both of these options negatively affect the final compression ratio.