Cisco Subnet An independent Cisco community View more

Knocking down the “I need a full mesh between my locations” MPLS myth

While enterprises need any-any connectivity, neither traffic patterns nor VoIP require a fully meshed WAN.

As we began to cover last time, until recently, if you were an enterprise WAN manager responsible for a serious WAN, you probably needed MPLS.  Thanks to the Next-generation Enterprise WAN (NEW) architecture, this is no longer the case.  Yet many myths persist as to why MPLS is necessary for the enterprise WAN.

If my main point in this series is that going forward MPLS need not be the guts of an enterprise WAN strategy, and certainly is not the service to which companies should be putting additional resources as me move towards the age of the cloud, my secondary point is that WAN managers almost surely made the right choice in going with MPLS, despite how expensive it is, given the choices and technologies available at the time.

Today, we'll tackle a couple of commonly held myths around MPLS: "I need a full mesh between all my locations because we have any-any traffic patterns," and "I need a full mesh for VoIP." And if I need a full mesh, it follows that I basically need MPLS.

The historical truth here is that MPLS came along after Frame Relay, right around the same time that enterprises were looking to deploy VoIP on their WAN. Frame Relay required paying for Permanent Virtual Circuits (PVCs) between any pair of locations (Switched Virtual Circuits – SVCs – were sometime available as well, but these too cost money). Given the cost of the PVCs, creating a full mesh was quite expensive under Frame Relay. Additionally, paying twice for expensive private WAN bandwidth at the data center to handle traffic like VoIP, or any other site-to-site traffic, was also undesirable.

Telecom service providers were eager to upgrade customers from Frame Relay to MPLS, frequently getting customers to upgrade smaller sites from fractional T1 links to the full T1 connections that in practice are the minimum for MPLS in the process. The fact that migrating customers from Frame Relay to MPLS could also lower the SP's internal costs by coalescing around a single WAN architecture was icing on the cake. 

And so the SPs sold "full mesh" MPLS as a less expensive way to deploy VoIP in the enterprise WAN. (MPLS also had better QoS capabilities than Frame Relay, something we'll return to in a future column.) But the fact is that MPLS was superior to Frame Relay from a price perspective, not that a full mesh was necessary to effectively support applications.

Apart from better pricing then Frame Relay offered, full mesh capability does have some advantages from the enterprise customer's perspective. It is easier to manage than a system with a bunch of PVCs.

And it does indeed generally – although not always – deliver lower latency than a hub-and-spokes approach for that traffic which really is from one spoke site to another spoke site. The practical benefits of this lower latency, however, are not well understood. The issue is that lower latency generally only matters for so-called interactive applications.

Interactive applications such as web-based applications, desktop virtualization, even traditional client-server applications are where lower latency unquestionably leads to better application performance. The reality, however, is that the servers for such applications are almost entirely based in data centers or headquarters locations, and not at the spokes of the network.

For bulk data transfer applications, bandwidth available is a much more important factor than latency. For real-time traffic like voice, avoidance of packet loss and large latency variability (i.e. jitter) are what matters. But 10 to 30 additional milliseconds of fixed latency, which is what results from a typical hub-and-spoke approach, has no real impact on the quality of a voice call. It effectively makes the "distance" between the two callers a few hundred miles longer than it might otherwise be. But a voice call between Los Angeles and Denver versus a call between Los Angeles and Dallas doesn't actually feel, or sound, any different. Only if the call were going halfway around the world in the first place, and so perhaps on the edge of the ~250 millisecond end-to-end delay where voice conversations start to "feel" materially different than across-town conversations, might the additional delay make any appreciable difference, and even then the difference is likely to be small indeed.

Other than real-time applications like voice and perhaps the occasional large file transfer, the data center consolidation trend of the last decade means the old 80-20 rule is probably more like 90-10 now in terms of most WAN traffic going to/from one or a small number of data centers. And so a full mesh isn't needed to deal with real-life traffic patterns, nor is it needed to deal with real-time traffic either.

Now, if there are no downsides to having a full mesh, then by all means an enterprise should take advantage of it, which is part of why enterprise WAN managers made the correct decision last decade in switching from Frame Relay to MPLS. But there are in fact some disadvantages. From a technical perspective, it is much harder to do optimum inbound traffic management of last-mile links when faced with a full mesh, as opposed to handling traffic from a much smaller number of locations. And to support public and hybrid cloud computing, doing a full mesh from every public cloud location is likely to be prohibitively complex.

More importantly, by leveraging the vastly lower cost, more competitive public Internet, non-meshed enterprise WAN approaches this decade can be far less expensive. Done properly, they can offer far more bandwidth and so deliver better application performance. 

"Done properly" is exactly what the NEW architecture provides today. While you do "pay twice" for some low percentage of traffic going through hub locations between "spokes", when that bandwidth costs only between $1 and $20 per Mbps per month, overall costs go way down with these approaches even as bandwidth goes up.

So while there may be benefits to MPLS, and while some WAN managers will be more comfortable with it than alternatives for some time to come, the "need" for a full mesh is definitely not a legitimate justification for deploying MPLS.

A twenty-five year data networking veteran, Andy founded Talari Networks, a pioneer in WAN Virtualization technology, and served as its first CEO, and is now leading product management at Aryaka Networks. Andy is the author of an upcoming book on Next-generation Enterprise WANs.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: Hidden Cause of Slow Internet and how to fix it
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.