In our last newsletter we discussed how the traffic that goes between data centers (a.k.a., east-west traffic) is undergoing tremendous growth. In this newsletter, we will discuss how the increase in this traffic is impacting WAN optimization.
One way to support the huge and growing east-west traffic flows is to connect the data centers with WAN links running at speeds of 10Gbps or higher. Unfortunately, a critical limitation of this approach is that these WAN links are not always available when and where they are needed. Another more fundamental limitation is that in most cases WAN links at this speed are prohibitively expensive.
A far more practical way to support these large inter-data center traffic flows is to implement techniques that reduce the amount of data that gets transferred over the WAN and that also guarantee sustained acceptable performance for critical traffic. Over the last few years, many IT organizations have deployed WAN optimization controllers (WOC) to perform these tasks. However, most traditional WOCs were designed to support traffic between branch offices and a data center; a.k.a., north-south traffic. North-south traffic is typically comprised of tens, if not hundreds, of slow-speed connections. East-west traffic is typically comprised of a small number of very high-speed connections.
LAN design guidelines are changing
What is needed is a WOC that was purpose-built to support east-west traffic. This new class of device will be referred to in this newsletter as a Hyper-scale WAN Optimization Controller (Hyper-scale WOC). As the name implies, a Hyper-scale WOC must be capable of saturating a multi-gigabit per second WAN link.
However, the distinction between a traditional WOC and a Hyper-scale WOC is more than the link speeds that each supports. One example of the distinction is that a Hyper-scale WOC must have very large buffers in order to shield the end systems at each data center from the effects of WAN propagation latency and any WAN packet loss. Another example is that a Hyper-scale WOC has to be able to perform de-duplication on a much more granular level than is done by the traditional WOC.
To illustrate this consider one of the key drivers of the increase in east-west traffic: storage replication. Storage replication applications typically only send those blocks of data that have changed since the previous transfer. In these cases, achieving good de-duplication ratios depends on the WOC being able to identify patterns that are far smaller than the typical data block addressed by a disk system. For maximal data reduction, the de-duplication provided by the WOC should be able to find repetitions all the way down to sub-10 byte packet segments - both within and across individual streams or flows.
The Infineta Data Mobility Switch (DMS) is an example of a Hyper-scale WOC. The architecture of the DMS reflects the fact that it was designed to support east-west traffic. For example, each of the core functions of the DMS (i.e., QoS, TCP Transport Optimization, De-duplication, and Compression) is supported in a separate hardware complex. These hardware complexes are arranged in a pipelined, distributed-processing architecture and are interconnected with a high speed, non-blocking switch fabric.
We would like to hear from you on this topic. Has your company been experiencing significant growth in its east-west traffic? If so, what have you done, or what are you planning to do, to support that traffic