Mailbag: Jumbo roadblocks

I received some excellent responses to last week’s newsletter asking just how relevant Jumbo Frames are today – so I’d like to share them with you now.

Many readers brought up this downside of Jumbo Frames: They can get in the way of convergence. If you are sending voice traffic, or something else that is time-sensitive, over your network, you don’t want a Jumbo Frame getting in the way of that time-sensitive data. This reader put it most succinctly:

“Bad idea. The problem is that Jumbo Frames create latency. Once a ‘big guy’ gets on the wire, he occupies it for a relatively longer period of time, preventing access to others, creating latency. This is one area where ATM had an advantage, giving access to the wire in short order, to high-priority users when needed.”

Another reader suggested that Jumbo Frames could work in a network that is separated from the rest, so they don’t get in the way of other traffic. One example might be a storage-area network.

However, there may no longer be an advantage to using Jumbo Frames in the first place. A couple of readers at universities indicated they have tested Jumbo Frames to see if they actually increase performance. One writes:

“After extensive testing, we’ve concluded that there is very little performance gain with modern PCs and Gigabit NICs. The primary advantage in years past was reducing the impact of the high rate of interrupts. But 3-GHz CPUs have enough power to handle Gigabit streams, and NIC cards and drivers no longer require one interrupt per packet. We agree that Jumbo Frames sound like a good idea on paper, but provide little utility in practice at 1 Gigabit. 10 Gigabit may be another issue.”

Another writes:

“We found that it is not the protocol processing overhead that kills the performance but the latency and impact of data movement between the CPU and the NIC buffers. Because of larger DMA size the CPU spends less time setting up DMA and other stuff. Also the latency is reduced. With increasing CPU speeds, protocol-processing overhead matters less and less. Our conclusion, if a standard commodity NIC allows huge DMA transfers in one shot you can have more performance gains. At the same time, you don’t need to tweak the MTU size and break the standards.”

Lastly, one reader (at a vendor) brought up several downsides to using Jumbo Frames. First, bigger frames mean that if you lose a frame, it’s a more significant network event, and retransmitting lost packets becomes a more time-consuming deal. Second, everything in the network must support Jumbo Frames for it to work. And thirdly, Internet connections don’t support Jumbo Frames:

“A frame bigger then the one supported by the Internet connection will be fragmented before it’s sent, highly reducing the performance and reliability of the Internet connection. This results in a requirement that every workstation must know which packets go to the local network and which go to the Internet. To detect the maximal packet size on a connection, the IP implements the MTU path discovery algorithm, but this is not standardized, and many firewalls don’t allow the ICMP packets related to this algorithm to pass through, due to denial-of-service attacks. So, Jumbo Frames can’t be implemented in a network connected to the Internet.”

Big thanks to everyone who wrote in with such thoughtful comments on this topic.