Data reduction speeds WANs

Various technologies have emerged aimed at improving application performance over enterprise WANs. Though helpful, solutions such as compression, QoS and wide-area file services don't offer large enough performance gains across a wide spectrum of applications, making their costs difficult to justify for larger deployments.

Disk-based data reduction is the newest WAN-acceleration technology that has emerged to help solve the performance, breadth and scale limitations that have plagued earlier technologies. It works on a simple premise - the most efficient way to accelerate the transfer of information across the WAN is to not send it in the first place. This provides significant benefits in the form of increased WAN bandwidth efficiency and reduces application response time.

To employ disk-based data reduction, a WAN-acceleration appliance is deployed in each location, such as a branch office or data center. The appliances examine all information traveling in and out of a WAN, "fingerprinting" the information and storing a copy, or instance, of the data on local hard drives.

During the fingerprinting process, pattern-matching technology is used to see whether the data being transferred matches data stored on a local drive at the destination. If the remote appliance has already stored the information, there is no need to resend it over the WAN. Instead, instructions are sent to deliver the data locally. This entire process takes place independent of normal client/server communications, ensuring that the most up-to-date data is always delivered in real time.

On the surface, disk-based data reduction resembles traditional caching, but there are several major differences, which include:

Application breadth: Data reduction detects patterns across many types of traffic. Caches, on the other hand, work at the object level, and are therefore applicable only to a specific application.

Application transparency: There are no client/server modifications when deploying data reduction. In some caching environments, clients need to be reconfigured to point to proxy devices.

Coherence: By preserving all client/server communications, there is no chance to deliver stale or inconsistent information in a data-reduction environment.

Effectiveness: Data-reduction fingerprints at the byte level, not the object level. This provides a higher hit rate when looking for duplicate data, including the detection of similar information, such as files that have been renamed or data that has changed slightly.

While results may vary, disk-based data reduction can eliminate more than 99% of WAN traffic under the right circumstances. For example, in a typical office environment, file transfers and e-mail traffic can be reduced five to 20 times on average, with peak reductions exceeding 100 times. In addition, backup and replication data volumes will be routinely reduced by 10 to 20 times, with peaks of 50 to 100 times. Many variables affect how well data reduction will improve application performance, including the amount of redundancy within WAN traffic. Often the best (and only) way to determine gains is to evaluate the technology in a live network.

Disk-based data reduction is the first WAN-acceleration technology to provide order-of-magnitude performance benefits across a broad set of enterprise applications. Because it relies on the fact that a large percentage of WAN traffic is repetitive, it often is implemented in conjunction with other techniques to provide performance improvements across a wide variety of WAN conditions and traffic patterns.

With data reduction as a core technology, enterprises have a full arsenal of tools to overcome WAN limitations. This paves the way for strategic server centralization projects and improves business continuity by ensuring successful data backup and recovery.

Aaron is director of product marketing, and Hughes is CTO and founder of Silver Peak Systems. They can be reached at and, respectively.

A WAN acceleration appliance uses data reduction to deliver information from local data stores, thus improving application performance.

Learn more about this topic

Vendors, customers hash out application acceleration


WAN optimizers stretch budgets


Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2006 IDG Communications, Inc.