Routers/Switches /

Time for Jumbo Frames - again!

By Kevin Tolly
Network World, 01/07/02

If nothing else, 2002 is going to be the year that storage over IP and 10G Ethernet switching become reality for brave souls on the cutting edge. Both could deliver more if Jumbo Frames were part of the industry-standard features available. Yes, it's time for Jumbo Frames - again.

I know, I've written about this before. It's been several years and the arguments in favor of Jumbo Frames are stronger than ever. The needs of storage and speeds of 10G Ethernet make frame size an issue.

For those just tuning in, the issue is this: While the speed of Ethernet has increased 1,000-fold over the past decade - from 10M bit/sec to 10G bit/sec - the maximum frame size hasn't grown at all. With the grand exception of 4-bytes added to accommodate virtual LANs (VLAN) and priority, which in any case can't be used for payload, the size has remained at 1,518 bytes.

Relative to the transport capacity, then, the maximum frame size has shrunk to almost microscopic proportions. Indeed, at 1G bit/sec it is like having a maximum frame size of 15 bytes. Divide by 10 again when cranking up to 10G bit/sec and we'd have 1.5 bytes per frame. And these are "best-case" examples using the maximum-allowable Ethernet frame size.

This low ceiling on frame size causes stations to slice up data into pitifully small chunks. Imagine loading an 18-wheeler carrying sugar - one cube at a time.

The processing load on end stations is enormous. And the load also increases on switches as they peer into and process each frame.

The alternative, Jumbo Frames, increases the frame size maximum by a factor of about six up to the 9K realm. Data streams better, and processing load is reduced dramatically. Tests have proven that.

Just consider a few numbers. At gigabit speeds, it takes only about 14,000 frame/sec to fill a pipe with 9K Jumbos compared with more than 81,000 frame/sec for 1,518-byte frames. These numbers jump to 140,00 and about 812,000 frame/sec at 10G bit/sec.

But for years, the committee to preserve and to protect Ethernet, IEEE 802.3, has refused to expand the frame size. While a technical smokescreen is often used, I think it is purely political.

There is ample precedent for large frames - token ring increased its maximum frame size from 4K to around 18K when it moved from 4M bit/sec to 16M bit/sec way back in 1988.

As noted above, Ethernet enlarged the frame size - a mere four bytes - when the 802.1Q VLAN standard was ratified. Backward compatibility can be considered.

Storage-over-IP vendors may come to the rescue. Storage is the "killer app" for Jumbo Frames. Storage over IP is little more than a torrent of the largest-sized packets possible. Look closely at storage over IP benchmarking and you'll likely see that Jumbos were used. Without them, processor resources get eaten alive and effective throughput suffers.

And, wanting to offer support for high-throughput storage over IP deployments, infrastructure switch vendors have seen the light. In recent weeks, several have said pressure from storage vendors has made them decide to add support for still-proprietary Jumbo Frames.

It's ironic - We struggle to deploy ever-more-efficient technologies and ignore one that have been staring us in the face. The IEEE 802.3 committee would do well to pick this up again, and the 10-Gigabit Alliance should also help promote users interests by pushing for Jumbos.