Let's hear it for latency

Editor's note: This is a corrected version of a column originally posted March 24.

Let me be the first to admit test labs have gotten something very basic very wrong: We put too much emphasis on throughput because it's simple and sexy. What could be cooler than seeing how fast the latest network widget runs?

Plenty, as it turns out. In our recent test of 10G Ethernet switches, there were big differences in throughput. The paradox is that the faster the network, the less important throughput becomes.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

Editor's note: This is a corrected version of a column originally posted March 24.

Let me be the first to admit test labs have gotten something very basic very wrong: We put too much emphasis on throughput because it's simple and sexy. What could be cooler than seeing how fast the latest network widget runs?

Plenty, as it turns out. In our recent test of 10G Ethernet switches, there were big differences in throughput. The paradox is that the faster the network, the less important throughput becomes.

Throughput is meaningful, but only when a network is heavily loaded. Another metric, delay, is meaningful for all traffic, all the time, on all networks.

Devices that add high delay slow your network regardless of whether it runs at 1% utilization or at 100%. What's more, applications may start conking out well before delays creep into the dozens of milliseconds.

Delay with Gigabit or 10G Ethernet interfaces is usually in the dozens to hundreds of microseconds. When I write about results like this, I usually add boilerplate text saying, "applications don't suffer until delay reaches into the dozens of milliseconds."

Sorry, but I was wrong. Even delays in the microseconds can degrade TCP performance dramatically on Gigabit and 10G Ethernet networks. The reason has to do with the way TCP works.

Normally, we do throughput tests using connectionless traffic like plain IP or UDP/IP packets. With connectionless traffic, delay doesn't have any practical impact on throughput (provided delay is constant, which is a whole other issue).

But even constant delay can have a huge impact on connection-oriented protocols like TCP. Considering that at least 80% of Internet traffic uses TCP, this is a rather large oversight in the context of throughput testing (see http://ipmon.sprint.com for recent Internet backbone observations).

In TCP, a transmitter can send only a limited amount of data, called a window, before the receiver must send an acknowledgement. A window may include multiple packets, but if the transmitter doesn't get acknowledgements for any of the data, it can't send anything further.

Here's where delay comes into the picture. Let's say we're using a 10-gigabit switch to receive 1,518-byte Ethernet frames. Let's further assume network utilization is light. At 10% utilization, we would receive 81,274 frames per second, or one frame every 12 microsec.

If the sending TCP window size is 16K bytes, no more than 11 frames' worth of data can be unacknowledged at any one time. For 11 frames at 12 microsec each, any delay of 132 microsec or more will prevent the transmitter from sending more data until any of the outstanding data is acknowledged.

Of course, the same is true for delays of less than 132 microsec too, but it's delays greater than this level that can have an adverse impact on throughput.

Two switches in our 10G Ethernet test - from Avaya and Force10 - delayed each 1,518-byte frame by more than 40 microsec. Add up 11 delays like that, and we're well beyond the point where throughput can be degraded.