Full speed ahead

If you want to beef up your backbone, you're going to have to pump some iron - or you can leave the heavy lifting to us. We invited 12 vendors to send us modular, multilayer (Layer 2 and Layer 3) switches capable of supporting at least 96 10/100M bit/sec Ethernet or 100Base-FX ports and a minimum of four Gigabit Ethernet ports.

Four accepted our invitation: Nortel Networks' Bay Networks division, Cabletron Systems, Extreme Networks and Packet Engines (which Alcatel last week announced it will purchase).

But don't expect to find a typical throughput shootout here. Instead, our Review takes a look at more significant benchmarks. We studied how each switch controls buffer delays and packet loss rates, checked out quality-of-service (QoS) capabilities, looked for evidence of head-of-line (HOL) blocking and evaluated the effects of latency.

We didn't try to saturate the switches. We measured forwarding rates and packet loss rates for two streams of bidirectional traffic over full-duplex Gigabit Ethernet and 10 streams of bidirectional traffic over full-duplex Fast Ethernet.

What we found is that each of the switches can forward packets at line speed even when configured with Gigabit Ethernet ports, irrespective of whether traffic is forwarded at Layer 2 or Layer 3. Each turned in a perfect forwarding rate under loading conditions ranging from 60% to 100%; each displayed consistently low latency; and none suffered from HOL blocking. Essentially, these switches have successfully assumed a new layer of intelligence without sacrificing performance.

Capacity

Extreme's BlackDiamond 6800 features the largest capacity of the switches we tested. The BlackDiamond 6800 is a 10-slot, modular backbone switch capable of providing up to 48 Gigabit Ethernet ports or 256 10/100M bit/sec ports in various combinations. The chassis can hold as many as eight line cards and two Management Switch fabric Modules (MSM). Only a single MSM is required, but if you install a second MSM, it may be used for load-shared redundant backplane switching, network management and protocol processing. A second MSM also doubles the backplane capacity from 32G to 64G bit/sec.

Packet Engines' PowerRail 5200 is a 14-slot modular backbone switch that can handle any combination of up to 120 100Base-FX ports or 240 autosensing 10Base-T/100Base-TX ports. The model we tested can be configured with up to 25 Gigabit ports. After our tests, Packet Engines began shipping a new six-port Gigabit module that enables a maximum of 73 Gigabit Ethernet ports.

Cabletron's 16-slot SmartSwitch Router 8600 (SSR-16) is a modular backbone switch that supports up to 30 Gigabit ports or 120 10/100M bit/sec ports, or a combination of the two. And finally, Bay's Accelar 1200 is an eight-slot modular backbone switch that can be fitted with up to 12 Gigabit Ethernet ports or 96 autosensing 10/100M bit/sec Ethernet ports, or a combination of the two technologies.

Speed plus finesse

Extreme's BlackDiamond has a 64G bit/sec nonblocking switch fabric with an aggregate Layer 3 forwarding capacity rated by the vendor at over 48 million packet/sec. The switch tackles QoS using the IP TOS field (originally the Type of Service field, now being applied as a QoS signaling field) and standard Ethernet 802.1p Layer 2 priority signaling mechanisms.

Packet Engine's PowerRail 5200 routing switch has a nonblocking backplane capacity rated by the vendor at 52G bit/sec and a switching throughput of over 37 million packet/sec. The PowerRail routing switch protects applications with a combination of application-based filtering and forwarding, IP TOS-based routing and precedence assignment, priority tagging according to the IEEE's 802.1p standard and Resource Reservation Protocol. Predefined templates are provided to make policy-based routing simpler, but they look like a first-generation feature.

Cabletron's SSR-16 can route or switch packets based on TCP header information - commonly called Layer 4 switching, which really amounts to Layer 3 switching with information from the Layer 4 header factored into routing decisions. We're not convinced this approach will enhance routing protocol performance, but time will tell.

The SSR-16 is capable of routing 30 million IP packet/sec via its nonblocking switch fabric. The switch provides dedicated, independent packet buffers on each output port, and space is allo-cated for hundreds of maximum-size Ethernet packets on each Fast Ethernet and Gigabit port. Separate buffer space is allocated to each of four classes of traffic, and forwarding is done on a prioritized basis, ranking the four classes from highest to lowest priority.

Cabletron takes an excellent approach to QoS, and we believe four to be an appropriate number of levels. Many vendors offer too few or too many classes; two is too few, and sixteen is far too many.

Bay's Accelar 1200, which is one of the first products to offer true Layer 3 switching, has an aggregate Layer 3 forwarding capacity rated by the vendor at over 14 million packet/sec. This capacity rating was fine a year ago, but today the Accelar is lagging in terms of aggregate switch capacity. Each module in the Accelar 1200 supports multiple QoS levels, separate queuing and priority for IP multicast traffic.

All four contenders support port trunking and load-sharing trunk groups with failover. Of the four, only Extreme's BlackDiamond does not perform load balancing.

Performance

We didn't attempt to saturate these switches and test aggregate throughput, but instead concentrated on the performance of individual ports.

The Gigabit and Fast Ethernet port densities of our test configuration came nowhere near saturating the rated backplane capacities of these backbone switches. The Gigabit test generated a maximum of 4G bit/sec of traffic, compared with up to 32G bit/sec of switching capacity among the switches tested. In fact, all of the switches performed well in these tests, with each turning in perfect forwarding rate scores.

In all cases, we found the forwarding performance was identical for Layer 2 and Layer 3 switching. This impressed us because it is much more difficult to build a wire-speed Layer 3 router than a Layer 2 switch. It's only in the past year that the majority of Layer 2 switches have become capable of running at wire speeds. So Layer 3 switches are showing signs of early maturity in their ability to route back-to-back packets coming in at wire speed and under heavy loads without introducing additional network delays.

The current generation of multilayer switches takes advantage of the latest ASICs to perform both Layer 2 and Layer 3 switching. By exploiting the pattern-matching that is common to both Layer 2 and Layer 3 table lookups, switch designers can make packet processing latencies for Layer 2 and Layer 3 forwarding nearly identical. Indeed, all the switches we tested display essentially the same low latency for both Layer 2 and Layer 3 traffic in both Gigabit and Fast Ethernet tests. In all four switches we tested, we expect latency to have virtually no perceivable effect on applications running over a network.

Nonetheless, it is worth noting that the Bay Accelar 1200 is the only switch in the group that has essentially the same low latency (less than 5 microseconds) for all packet sizes, while the other three switches have latencies that are roughly proportional to packet size. This difference reflects the fact that the Accelar 1200's shared memory architecture stores each packet only once before forwarding it to its destination. In contrast, all the other switches store a packet several times in the process of moving it from one internal location to another before forwarding. Given Bay's architecture, the Accelar 1200 could be better suited than the other three switches to handle some latency-sensitive applications, such as voice over IP.

We used a congestion test to measure the maximum burst of back-to-back packets that an output port can handle before it begins to lose packets. The test was designed to determine buffer capacity for the congested port and the ways in which buffer space is allocated to congested ports.

In our test results for Gigabit Ethernet, maximum burst sizes ranged from 250 1,518-byte packets for Extreme's BlackDiamond to over 12,000 for Packet Engine's PowerRail 5200. The corresponding Fast Ethernet burst sizes ranged from 60 packets for the BlackDiamond to over 1,100 packets for Bay's Accelar switch.

Clearly, Packet Engines' PowerRail 5200 has unusually large buffers, while Extreme's Black- Diamond is on the shallow side. This could reflect a philosophical difference with respect to the suitability of using local flow control to fight congestion or just different preferences in terms of buffer size. End users should be aware of these differences because buffer size can influence ap-plication performance and equipment costs.

We found that a fairly robust design will allow three to five bursts of approximately 40 1,518-byte packets to coincide on a server or backbone switch port without packet loss. Be sure to take into account the time it takes for a full buffer to be drained, which will be much faster at Gigabit Ethernet speeds than at Fast Ethernet speeds. Combining these two factors, we found that burst capacities for Gigabit Ethernet ports of between 200 and 500 large packets appear to provide adequate protection against packet loss without creating potentially excessive buffer delays. Bay and Packet Engines looked the best in this regard.

If a product offers much larger buffer capa-city, as in the case of the PowerRail, you can avoid excessive buffer delay by either enabling QoS or using the network management interface to reduce the maximum buffer allocation for each port.

HOL blocking can occur when one congested output port stalls or blocks traffic destined for uncongested ports. In our tests, all four switches proved to be free of HOL blocking.

Network management

Each card in Extreme's BlackDiamond features a self-contained Web server that may be used to configure, observe and manage the switch remotely. We enjoyed working with the BlackDiamond's Web management interface - it is one of the slickest we've seen. Hover buttons, Java applets and dynamic HTML features are used extensively to create an easy-to-use configuration tool.

Bay's Accelar features Web-based configuration management, which enables managers to access the routing switch using port-based out-of-band management. Bay's routing switch software operates on Solaris and HP-UX workstations, as well as on PCs running Windows 95 or NT. The graphical user interface provides integrated switching and routing configuration services, and a simple, drag-and-drop virtual LAN man-ager supports simultaneous configuration of multiple switches.

In general, we found Accelar's Web-based management utility adequate. However, philosophically, the Bay product line seems to have been designed more for a command-line interface, which many network managers prefer using for element management functions.

Cabletron's Java-based network management software provides easy-to-use templates and wizards that assist in creating and maintaining QoS policies on the SSR-16. We found them easy to use but somewhat immature as management tools.

To manage the PowerRail, Packet Engines offers a Java-based browser interface to its configuration and policy management application. We found the Web management interface to be adequate for performing basic configuration tasks.

Wrap-up

QoS capabilities will be the next major vendor battleground. Users need simple, easily administered QoS capabilities that allow select traffic to be protected from potential delays and packet loss associated with port congestion. Configuring an appropriate amount of buffering is an adequate way of dealing with congestion if all traffic has the same delay and packet loss requirements. How-ever, in real networks where all traffic is not generated by a homogeneous set of delay-insensitive applications, using windowed protocols - FTP/ TCP, for example - and simple first in, first out (FIFO) queuing can have serious shortcomings.

Switched infrastructures need to deal with congestion by manipulating multiple logical queues to ensure that overloaded conditions have a minimal impact on network applications with quite different service requirements. If developments in switch architecture continue at their present pace, in a year or two we expect to see multiprotocol backbone routing switches with 10G bit/sec interswitch links, capable of handling voice, video and data with equal aplomb. RELATED LINKS How we did it
Our testing methodology for this review.

Interactive buyer's guide
See how 15 switches stack up against each other in several categories, or download a spreadsheet with all their specs.

Bigger, better backbones
Issues to consider before investing in a switch.

Testing switch performance: A primer

Bell is president and CEO of Silicon Valley Networking Lab. Joe Bullock, senior test engineer, and Chris Kraemer, lab engineer, conducted the testing for this review.

Bell is a member of the Network World Test Alliance, a cooperative of the premier reviewers in the network industry, each bringing to bear years of practical experience on every review.