A dozen extensions to TCP/IP that optimize internet connections

Do you remember when we used multi-protocol routing for IPX, AppleTalk, and TCP/IP running on the same network? In the 1980s and early 1990s many enterprises had multiple protocols running on the physical network infrastructure as “ships in the night”. Cisco routers became highly adept at multi-protocol routing and the company grew in prominence as a result. Then in the early 1990s, TCP/IP won out and the internet took shape as the global network we enjoy today.

The TCP/IP protocol uses a very simple four-layer model (Application, Transport, Internet, and Link Layer). TCP/IP’s layers are much simpler to remember than the seven-layer Open Systems Interconnection model (OSI model) conceived in the 1980s. To remember all those layers some use the mnemonic device (All People Seem To Need Data Processing) to remember (Application, Presentation, Session, Transport, Network, Data Link, Physical). IPv4 was the only protocol at the narrow waist of the TCP/IP hour glass, but now IPv6 has added a second protocol to the internet layer.

The internet has been under constant development since its inception in the 1970s. TCP/IP has continued to be optimized from its original design. Protocol designers have continued to create new internet transport layer protocols improve how internet applications perform. These protocols try to improve upon TCP and UDP and add congestion control and optimized throughput forwarding performance.

Some of these protocols are esoteric and handle rare corner cases. Some of these protocols may already be in use for your internet connections and you don’t even realize it. This article will break down these new Internet Protocol optimizations and show which ones are most impactful for your organization and can improve your Internet application experience.

Multipath TCP (MPTCP)

Most networks do not have any redundancy and, therefore, there is only one available path from the source of a connection to the destination. However, the internet is very “meshy” and there are many different possible paths between any two nodes. Network equipment has historically handled multiple equal-cost paths in one of two ways: either packets are broken up without regard for connection and sent over both paths, or all the packets sent for a single connection takes one path and different connections are sent over diverse paths. The first method can cause problems for traditional TCP connections, while the second method works well with TCP.

+ MORE ON NETWORK WORLD 25 years of TCP/IP  +

Multipath TCP (MPTCP) (RFC 6824) addresses the first problem of having a single TCP connection use multiple paths where the packets can be split up randomly among the different paths. In this way, all the resources of the multiple links contribute to throughput and connection resiliency is a byproduct of having multiple paths. You can think of this as the TCP equivalent of the transport independence concept in many of the Software-Defined WAN (SD-WAN) products on the market. MPTCP is implemented in Apple iOS and Mac OS X as well as several popular server load balancers.

Data Center TCP (DCTCP)

Traditional TCP congestion control tolerates packets getting dropped in transit and reduction in the TCP congestion window size which reduces the transmission rate. Over the years, the TCP has been the focus of many research projects to optimize the congestion control algorithms. Explicit Congestion Notification (ECN) (RFC 3138) is a method of signaling congestion encountered in transit before packets actually get dropped.

As the source sends packets toward the destination, if congestion is encountered, the network device marks the ECN bits in either the low-order 2 bits after the DSCP marking in the IP header, or flags within the TCP header. When the destination receives the packets indicating that congestion is imminent, the destination relays that alert back to the source to slow its transmission to avoid future packet loss and resulting retransmission and throughput reduction.

Implementation of ECN has been problematic on the internet due to middle boxes not properly handling the ECN marking process. ECN is being applied to other protocols, such as RTP/RTCP (RFC 6679) and for SCTP (see below). The protocol designers at the IETF are looking at other ways to use ECN to help avoid congestion. IETF RFCs such as “Congestion Exposure (ConEx) Concepts and Use Cases” (RFC 6789) and “Problem Statement and Requirements for Increased Accuracy in ECN Feedback” (RFC 7560) document some potential options.

This has led to the development of a new protocol called Data Center TCP (DCTCP) that makes improvements on congestion notification for data center switching environments. DCTCP alerts early on congestion and tries to quantify the amount of congestion to signal to the source to reach the optimal forwarding rate. DCTCP has been documented in an IETF draft and it has been researched by Stanford University and Microsoft with Windows Server 2012. Cisco’s Nexus 3548 switches use DCTCP and there is interest by other data center switch manufacturers.

TCP AnyCast

You may already be familiar with the concept of anycast, where the principle of uniqueness of IP addresses is violated and multiple systems use the same address. Those identical systems advertise that single IP address into a dynamic routing protocol. Clients wanting to communicate with that service communicate with the “closest” instance of that synchronized content. If one of the services fails, it withdraws its route advertisement, the routing tables converge, and service is restored. Whereas unicast communications is one-to-one, anycast is more like one-to-nearest. The internet already uses anycast extensively with the root DNS name servers.

Anycast has historically worked well for quick stateless communications like DNS queries, but hasn’t worked so well for long-lived TCP sessions that might be affected by a server failure and route convergence. There have been discussions for several years about how to extend this anycast concept to TCP communications. Back in 2006 there was a NANOG 37 presentation titled “TCP Anycast – Don’t believe the FUD”, by Matt Levine (CacheNetworks), Barrett Lyon (BitGravity), and Todd Underwood (Renesys). More recently, Matt Levine, from CacheFly, wrote about “Measuring Throughput Performance: DNS vs. TCP Anycast Routing”. LinkedIn has been testing TCP Anycast and last year they published a paper titled “TCP over IP Anycast – Pipe dream or Reality?”.

Hypertext Transfer Protocol Version 2.0 (HTTP/2)

Hypertext Transfer Protocol (HTTP) has been the de facto Internet protocol since the late 1990s because of its ability to carry connections across IPv4 NAT middleboxes. People have tried to improve HTTP by adding persistent connections and connection reuse (HTTP version 1.1 [RFC 2068]) and security (HTTP Secure (HTTPS) with SSL, and then TLS).

One such effort to reduce the latency of HTTP connections was SPDY (pronounced as “SPeeDY”). SPDY was a research effort by Google to improve the speed of the Chrome browser. SPDY has multiplexed streams to allow for simultaneous fetches, request prioritization, HTTP header compression, and server push. This work on SPDY has now led to the newest optimized version of HTTP.

Hypertext Transfer Protocol Version 2.0 (HTTP/2) is the most current version of HTTP that supports the SPDY features and is now an industry standard. In 2015, the HTTP/2 protocol was standardized with IETF RFC 7540. HTTP/2 has gained substantial industry support across web browsers and adopted by numerous content providers. Content delivery networks such as CloudFlare and Akamai now offer this to their customers. You may be using HTTP/2 today and not even realize it.

Quick UDP Internet Connections (QUIC)

Most web connections are established over TCP HTTP connections using TLS. This method of creating connections to fetch web page contents adds overhead and increased latency. In this fast-paced online world, every millisecond counts against excellent end-user experience and long page load times drive your customers to your competitors.

Quick UDP Internet Connections (QUIC) offers a way to create web connections of UDP that results in higher performance of UDP, but the same reliability of TCP. QUIC allows for multiplexed connections that do not require TCP’s connection establishment. At last year’s NANOG64 conference there was a good session titled “QUIC: Next generation multiplexed transport over UDP” by Brian Rogan and Ian Swett of Google that reviewed how QUIC worked and its benefits.

Google has been the primary proponent of QUIC and has implemented it into Chromium and the open source code is available for review. Google has published a good QUIC Geek FAC and has been working on integration into the Chrome web browser and creating a web server with QUIC integration. Google is even dog-fooding it on some of their own projects. QUIC is being proposed to the IETF and is currently in draft form, but may become a standard soon.

HTTP Strict Transport Security (HSTS)

In recent years, there has been increased awareness of Internet eavesdropping and more companies have a desire to use encryption on their primary customer-facing web sites. Despite the Heartbleed OpenSSL vulnerability in 2014, there is a greater percentage of sites using HTTP (TLS) than ever before. Initiatives such as the Let’s Encrypt movement help organizations learn about how to enable HTTPS and encourage them to use TLS. Another way to help web sites negotiate HTTPS connections is to enforce this behavior through another protocol.

HTTP Strict Transport Security (HTTPS) is a policy mechanism that helps prevent web connections from falling back and downgrading to using unencrypted HTTP and for web servers to encourage web browsers to connect with HTTPS. Several years ago, HSTS was defined and standardized in an IETF RFC 6797. The web server can signal to the client that only HTTPS is allowed to interact. This is done when the web server sends a HTTP Response Header field “Strict-Transport-Security” to the web browser (user agent) indicating a preference for HTTPS. The client behaves by automatically turning any “http://” links into “https://” links. While, HSTS is not a TCP/IP transport protocol, it has the ability to influence and control how Internet connections are established and help them be more secure.

Datagram Congestion Control Protocol (DCCP)

Some Internet architects feel that TCP has too much overhead for some applications, and UDP has too few options for controlling congestion. This leaves room for protocols that occupy the middle-ground between TCP and UDP to provide options for applications that need the speed of a streaming protocol and some congestion notification, but don’t need in-order packet delivery and acknowledgements.

The Datagram Congestion Control Protocol (DCCP) hopes to be this protocol that hits the sweet spot between TCP and UDP. The DCCP IETF working group created the primary DCCP (RFC 4340) and specified the DCCP service codes (RFC 5595) along with the TCP Friendly Rate Control (TFRC) (RFC 5348). DCCP has been implemented into the Linux 2.6 kernel, but has struggled to gain further adoption by vendors.

Structured Stream Transport (SST)

Just like other transport protocols aim to improve upon TCP, Structured Stream Transport (SST) is no exception. SST is an experimental protocol that adds stream management on top of TCP to allow systems to effectively perform streaming reception and interactive connections simultaneously. SST started development in 2006 and 2007, but in almost 10 years, hasn’t garnered significant support to become a mainstream transport protocol. SST is an example of how a protocol has a tough road to join the likes of TCP and UDP in mainstream adoption.

Stream Control Transmission Protocol (SCTP)

SCTP is yet another transport-layer protocol that combines some of the features of TCP with some of the features of UDP. In 2012, I wrote an article titled “What About Stream Control Transmission Protocol (SCTP)?” that described SCTP and its message-oriented approach compared with TCP’s stream-oriented approach. There are many implementations of SCTP and seemingly a lot of potential for this protocol. However, TCP and UDP have considerable momentum and there is gravity for developers to consider these two traditional transport layer protocols. Without major operating system support by Microsoft and Apple, this protocol is destined to be another footnote in the Internet’s history book.

Datagram Transport Layer Security (DTLS)

Transport Layer Security (TLS) (previously known as SSL) has been used for many years to secure web sites using HTTPS. There was an industry-wide desire to provide privacy for connections datagram protocols like UDP. Datagram Transport Layer Security (DTLS) in an IETF protocol (originally RFC 4347) that provides for confidentiality and non-repudiation for UDP streams. DTLS is used with the Cisco AnyConnect SSL VPN client. DTLS version 1.2 was originally conceived for use as a way to secure UDP streams (RFC 6347), but DTLS can also be used with other protocols such as DCCP (RFC 5238), with SCTP (RFC 6083), and with SRTP (RFC 5764).

Secure Real-time Transport Protocol (SRTP)

Similar to DTLS, SRTP has the goal of providing encryption and message authenticity for real-time communications using UDP or multicast. SRTP is an extension of the Real-time Transport Protocol (RTP) that is frequently used for streaming media transport and frequently accompanies the RTP Control Protocol (RTCP) that provides for management of those streams. SRTP also provides a secure form of RTCP called Secure RTCP (or SRTCP). The Secure Real-time Transport Protocol (SRTP) and SRTCP were defined in 2004 by the IETF with RFC 3711. SRTP has been augmented with the RFCs titled “Encryption of Header Extensions in the SRTP” (RFC 6904) and “Support for Reduced-Size RTCP: Opportunities and Consequences” (RFC 5506).

Reliable User Datagram Protocol (RUDP)

RUDP aimed to provide a compromise between the connectionless lightweight UDP transport protocol and the connection-oriented heavyweight TCP protocol. RUDP provided for ordered packet delivery and acknowledgement of receipt and retransmissions. It is very likely that you haven’t heard of RUDP. That is because it was originally created by Bell Labs as a new addition to their Plan 9 operating system. Eventually, Plan 9 was released by Lucent Technologies as a free open source operating system, but it didn’t gain the industry-wide adoption of Linux and Android. Cisco integrated RUDP into their SS7 Cisco PGW 2200 Softswitch node Signalling Link Terminal (SLT)

Summary

It is human nature to try to improve upon inventions of others and to try to optimize systems and improve efficiency. IPv4 and the protocols above the Internet layer continue to develop, despite the steamily ubiquitous use of the Internet Protocol. TCP/IP will never be finalized; it will continue to evolve and improve as we think of new ways to communicate.

These optimizations to TCP and UDP and IP have potential to improve the efficiency and performance of internet communications. However, each of these new transport layer protocols need to be implemented into host operating systems, and integrated into applications such as web browsers and web server software. Wikipedia has an excellent table that compares and contrasts many of these transport layer protocols. It remains to be seen which of these protocols will dominate. Some of these protocols have potential to gain mainstream adoption, while others will wither away as research projects. Other protocols have already gained momentum and you may already be using them today.