It\u2019s no secret that the public Internet is a quagmire of latency and packet loss problems. No wonder, many of clients are reluctant to trust Internet-based SD-WANs with VoIP and business-critical applications. After all, how can an SD-WAN running over Internet provide a predictable user experience if the underlying transport is so unpredictable?\nTo answer that question, SD-WAN Experts recently evaluated the performance and stability of long-distance Internet connections. Our goal: to determine the source of the Internet's performance problems by measuring variability and latency in the last and middle miles.\nWhat we found was by swapping out the Internet core for a managed middle mile makes an enormous difference. Case in point is Amazon. The latency and variation between our AWS workloads was significantly better across Amazon\u2019s network than the public Internet (see figure). Why that\u2019s the case and how we tested is explained below and in greater depth from this post on our site.\n SD-WAN Experts \n\nFig. 1: Median results for latency and variation (standard deviation from median latency) for all paths between AWS instance on-net (AWS-AWS) and off-net across the Internet (Backbone-AWS). Note: lower is better. \n\n\nThe why and how we tested\nIf I was an old MPLS hand (which I am) clinging to my MPLS past (which I\u2019m not) I might be inclined to dismiss our research from the outset. After all, if my MPLS performance declines what difference does it make where it happens in my service provider\u2019s network? The service has a problem and either the provider fixes the problem, or I ultimately switch MPLS services.\nBut because of the way the Internet works we have more options when it comes to selecting providers. Precisely because Internet connections consist of three components \u2014 \u00a0the last mile from the customer\u2019s premises to the ISP\u2019s premises (sometimes called the \u201cfirst mile\u201d), the middle mile (sometimes called the Internet core), and the last mile from the destination ISP\u2019s premises to the customer\u2019s premises \u2014 \u00a0we have some additional flexibility. We can select a different last mile provider, if it\u2019s the source of our problems. And if the problem is in the middle mile, we can choose a different middle-mile provider either explicitly or by switching to an ISP connected to a different middle mile.\nTo figure out the source of the Internet\u2019s performance problems, our testing measured the time to first byte (TTFB) when sending test files from AWS workloads between one another and to locations in six cities. TTFB is a more accurate measure of latency than a simple PING. It looks at the time needed to send a packet and wait for an acknowledgment, eliminating the time for setup and connection negotiation. Tests were conducted with various Internet tools \u2014 \u00a0Catchpoint, SpeedTest, and Cedexis. We repeated the tests, in part for verification purposes,\u00a0 and reported on each path\u2019s median latency and variation (the standard deviation from median latency). Last and middle mile performance were compared in absolute numbers and relative to the segment or overall latency.\n Catchpoint \n\u00a0\nMiddle mile vs. last mile: which is more stable?\nWe\u2019ve often spoken about the unpredictability of the Internet middle mile. A major source of the problem is that ISPs route packets based on economics not application performance. They dump traffic on one another to maximize their infrastructure investments. As such, Internet traffic might take the fewest possible hops to reach a destination one day and bounce around the world the next.\nAnd while most people I speak with think the middle mile is more unstable than the last mile, that\u2019s not exactly true. Our testing shows, last mile latency fluctuates by a far greater percentage than latency in the middle mile,\u00a0reaching 196% in some cases. By contrast, the middle mile fluctuated by no more than 143%.\nSo why focus on the middle mile? Simple: The middle mile\u2019s impact on the overall connection is far more significant. The above mentioned 196% number, for example, described the last mile variation for four paths to Bangalore from San Jose, London, Tokyo and Sydney. The actual number, though, was a matter of 5.88ms (3ms was the median last mile latency). By contrast, the middle miles varied from 36% to 85% \u2014 \u00a0\u00a092ms to 125ms \u2014\u00a0 a 20x greater impact on the connection.\n SD-WAN Experts \n\nFig 3: Although last miles showed greater relative change, latency variation still stems primarily from the middle mile.\n\n\nAnd that aligns with what you probably intuitively knew all along. Last miles are comparatively short, extending from the customer premise to the local ISP\u2019s network. The middle miles in our case, though, stretched across North America, the Atlantic, or the Pacific depending on the path. Latency should be far greater across the middle mile if only because of the distance not to mention the routing issues.\nMoreover, the middle mile issues might be more prominent on an international Internet routes, but they\u2019re not limited to them. Even within well-developed Internet regions, such as in the US, middle mile issues occur. In our testing, we found that latency variation in the middle mile from Virginia to San Francisco, for example, to be the highest of all paths (103 ms or 143% from the path\u2019s median latency).\nThe answer: privatize the middle mile\nBack to our original question: Can you trust an SD-WAN to deliver a predictable user experience if the basis of that SD-WAN is the unpredictable public Internet? Based on our testing, the answer is a qualified \u201cyes.\u201d\nLet\u2019s take apart the problem \u2014 \u00a0middle mile and last mile. As we\u2019ve seen, any concerns around latency will lie in the Internet core, at least when long distances separate the source and destination. In those cases, as we\u2019ve seen, fluctuations in the last mile become negligible relative to the middle mile. \u00a0\nAs such, latency and variability problems will likely arise when delivering VoIP and other applications requiring low-latency, predictable connections. It\u2019s a crapshoot; some days VoIP will be fine operating over the Internet middle mile, other days unsuitable. (To better understand why running enterprise VoIP across the Internet is less than ideal, see this in-depth blog by longtime VoIP expert, Phil Edholm.) The impact of the last mile, though, from a latency standpoint is nominal. As such, you can use the last mile for access to a middle mile without concern that last mile latency will be problem. (Packet loss rates and availability are issues in the last mile, which we haven\u2019t spoken about here but can be largely addressed by a combination of SD-WAN features and selecting the right ISP.)\nAll of which is why providers of global SD-WAN services can claim \u201cMPLS-like\u201d performance even though customers must access their global networks using the Internet last mile. Swapping the Internet core for a managed network significantly reduces latency even when that managed network is not MPLS. How much better? Consider this: the median variation of the tested paths across the Amazon network was just 9.91ms vs. 83.6 for the public Internet \u2014 \u00a0a nearly 10x difference.