Switch vendors pass interoperability tests

It was billed as a test of Gigabit Ethernet switch interoperability, and the seven vendors that participated not only scored generally high marks for their Layer 2 and Layer 3 hardware, but they also got into the spirit of the event.

The fierce competitors set aside their rivalries last month to help each other during the test, which was conducted by The Tolly Group and Network World at a hotel in New Jersey. Take Lucent's Jeff Tabor, for example. Even when Lucent's boxes weren't being tested, Tabor could be found moving switches, swapping cabling and helping get Ethernet switch pairs up and running.

And he wasn't the only one pitching in. Engineers from Hewlett-Packard and Foundry Networks were helping out each other. Nortel Networks was collaborating with Cabletron. There was Cisco breaking out its new enhanced spanning tree feature in front of IBM, Lucent and others.

Of course, in interoperability tests, one vendor's performance depends on another's, so cooperation among the engineers was something of a requirement. Still, one doesn't see that sort of camaraderie in a typical switch shootout in which vendors work one-on-one with testers. The need to work together created a cordial atmosphere at the Newark hotel suite where The Tolly Group set up camp and began evaluating the five Layer 2 switches and seven Layer 3 switches. And that cooperative spirit helped shave a day off the five-day testing schedule, which included four mandatory and six optional tests.

In general, the interoperability test results were encouraging. When the seven vendors said they followed a standard, most did - and they didn't deviate from the standards so much that interoperability suffered. For customers, this means you can mix and match vendors depending on your deployment needs. All the switches will negotiate speed and duplex settings to maximize throughput, and the Layer 3 switches will be able to communicate over different protocol networks. Most of the Layer 3 devices showed they can back each other up if one fails - after all, the last thing you want to find out in a crisis is that your switches are only interoperable under optimal working conditions.

The test plan

Test 1: Autonegotiation (mandatory)

Description: Each pair of switches must properly negotiate the highest speed and duplex settings common to both devices.

Degree of difficulty: Low

Participants: All (Cabletron's SmartSwitch 6000 and SmartSwitch Router 8600; Cisco's Catalyst 2948G and Catalyst 8510; Foundry's BigIron 4000; Hewlett-Packard's ProCurve Switch 4000M and ProCurve Routing Switch 9304M; IBM's 8275-416 Ethernet Switch and 8371 Multilayer Ethernet Switch; Lucent's Cajun P120 Workgroup Switch and Cajun P550 Gigabit Switch; and Nortel's Accelar 1200)

Results: All 12 products passed.

To warm up the players, we started with autonegotiation drills - an easy win for any vendor serious about interoperability. After first verifying that each switch was properly configured to autonegotiate, we recorded the setting that each switch reported to have negotiated, compared that to the maximum configuration supported by both devices, and confirmed that throughput remained within an appropriate range as we transmitted a stream of 1,518-byte packets from Netcom Systems' SmartBits to each switch.

We expected all the boxes to pass, and they did. Most consumers today take autonegotiation for granted, and it's good to know these boxes take autonegotiation in stride.

Test 2: Flow control (mandatory)

Description: Each switch pair must exchange flow control information.

Degree of difficulty: Moderate

Participants: All except IBM's 8371 Multilayer Ethernet Switch

Results: Varied

This test caused considerable consternation among the participants because every switch vendor handles congestion differently. We knew that, but we pressed on anyway.

Specifically, we set out to verify the initiation and reception of flow control messages, though the IEEE's 802.3x standard for flow control over Fast Ethernet and Gigabit Ethernet only requires a switch to respond to flow control messages, not initiate them (see story, page 82). We wanted to determine how each switch acts upon receiving a control message - whether a switch passes along the message, throttles back traffic or modifies traffic on a port-by-port basis. Different implementation strategies yield different results, even though they may all be considered standard-compliant.

Working with pairs of switches and a high-speed uplink, we deliberately generated congestion at one switch's output port and watched to see if that switch initiated a flow control message to the other switch. We then queried the receiving switch to see if it received the control message. Using network analyzers - Network Associates' Sniffer Pro High-Speed for the gigabit links and Wavetek Wandel Goltermann's DominoFE for the Fast Ethernet links - we verified whether traffic flow from the switch that got the pause message was reduced until congestion was clear. We performed the tests in both directions with each pair.

Four switches passed with flying colors - Lucent's Cajun P120 Workgroup Switch and Cajun P550 Gigabit Switch, and IBM's 8275-416 Ethernet Switch and HP's ProCurve Switch 4000M. Some products responded to pause messages but didn't initiate them - which is acceptable according to the IEEE standard. These are: Cabletron's SmartSwitch 6000 and SmartSwitch Router 8600; Cisco's Catalyst 2948G and Catalyst 8510; and Nortel's Accelar 1200.

Only Foundry's BigIron 4000 and HP's ProCurve Routing Switch 9304M (a Foundry BigIron 4000 switch that HP resells through an OEM agreement) neither initiated nor responded to pause messages. IBM's 8371 Multilayer Ethernet Switch doesn't support the standard, but the company says flow control will be supported in the product's next release late this year.

Test 3: IP routing (mandatory)

Description: Each switch pair must properly route TCP/IP traffic.

Degree of difficulty: Low

Participants: Layer 3 switches (Cabletron's SmartSwitch Router 8600; Cisco's Catalyst 8510; Foundry's BigIron 4000; HP's ProCurve Routing Switch 9304M; IBM's 8371 Multilayer Ethernet Switch; Lucent's Cajun P550 Gigabit Switch; and Nortel's Accelar 1200)

Results: All participants passed.

To verify IP routing interoperability, the visiting engineers configured each Layer 3 switch with static routes or a default gateway, and we paired each switch with the other participants. We required the devices to communicate with one another over two different IP networks - one "backbone" network and one "private" network - using TCP/IP.

As in our autonegotiation tests, we weren't expecting much trouble from those who claim to support the standards. True to form, the seven Layer 3 switches passed the IP routing tests.

Test 4: IP Routing Information Protocol (RIP) Versions 1.0 (mandatory) and 2.0 (optional)

Description: Each switch pair must exchange IP routing table information in compliance with RIP Version 1.0 and, optionally, Version 2.0.

Degree of difficulty: Low

Participants: Layer 3 switches

Results: All participants passed.

To verify compatibility using IP RIP Version 1.0 and 2.0, the participants didn't configure their switches with static route entries but instead allowed the switches to build dynamic routing tables based on exchanged RIP information. We required each switch pair to update their routing tables correctly with network information from the remote networks and made sure that pairs of TCP/IP end stations running Ganymede Software's Chariot could communicate across the network.

Test 5: IPX routing (optional)

Description: Each switch pair must properly route IPX traffic.

Degree of difficulty: Low

Participants: Layer 3 switches except Foundry's BigIron 4000 and HP's ProCurve Routing Switch 9304M

Results: All participants passed.

As in our IP routing tests, we required each pair of Layer 3 switches, configured with static routes or a default gateway, to communicate over two different networks. This time we set up one backbone IPX network and one private IPX network.

Participation wasn't 100%. Of the seven Layer 3 switches, Foundry's BigIron 4000 and HP's ProCurve Routing Switch 9304M don't support static IPX routing.

Test 6: IPX RIP (optional)

Description: Each switch pair must exchange routing table information in compliance with IPX RIP.

Degree of difficulty: Low

Participants: Layer 3 switches

Results: All participants passed.

To verify IPX RIP interoperability, we required each pair of Layer 3 switches to communicate over a backbone IPX network and a private IPX network. As in our IP RIP tests, the participants did not configure their switches with static route entries but instead let the switches build dynamic routing tables based upon exchanged RIP information. We required each switch pair to update their routing tables correctly with network information from the remote networks and again made sure that pairs of IPX end stations running Ganymede's Chariot could communicate across the network.

Test 7: Link aggregation (optional)

Description: Each switch pair must interoperate over a single aggregated link consisting of two full-duplex, Fast Ethernet links.

Degree of difficulty: Moderate

Participants: All except IBM's 8275-416 Ethernet Switch

Results: All participants passed.

All the vendors tested have developed link aggregation schemes that let you bundle multiple point-to-point links to create a single logical link of greater bandwidth; Cisco's Fast Ether-

Channel, Lucent's OpenTrunk and Cabletron's SmartTRUNK are a few examples. Because implementations vary, we didn't verify compliance with any particular specification, such as the IEEE's pending 802.3ad Link Aggregation Protocol. Nonetheless, we wanted proof that proprietary trunking solutions can work in a multivendor environment.

To test link aggregation, we set up pairs of switches connected by a single aggregated link consisting of two full-duplex Fast Ethernet links. We generated 1,518-byte packets from a pair of SmartBits ports to two nonaggregated "feeder" ports on each switch and checked to see that each switch then forwarded traffic to its mate across an aggregated "trunk" link at a rate that exceeded the bandwidth of a single link. That is, we ensured the system forwarded more than 100M bit/sec in each direction across the trunk.

Only IBM's 8275-416 Ethernet Switch doesn't support link aggregation. The other 11 switches participated, and all passed.

Test 8: Accelerated convergence (optional)

Description: Each switch pair must demonstrate that proprietary reconvergence schemes can interoperate and yield faster recovery times than Spanning Tree Protocol.

Degree of difficulty: High

Participants: Cisco's Catalyst 2948G and Nortel's Accelar 1200

Results: The two participants are interoperable.

Continued interoperability in the event of a hardware or software failure is critical. In the same way vendors have customized trunking procedures, they have also developed proprietary accelerated convergence systems. These systems purport to speed the recovery process in the event a link fails. The systems are typically a replacement for the 802.1d Spanning Tree Protocol in Layer 2 switch environments.

So we challenged vendors to prove that their independently developed reconvergence schemes can interoperate - and actually realize a speed gain. Because typical spanning tree converges in as little as 30 seconds, we were looking for quicker redirection and much faster recovery times - within three seconds.

But it seems vendors aren't as confident in this arena as they are with link aggregation. Only two vendors accepted the challenge. We connected two of Cisco's Catalyst 2948G switches and two of Nortel's Accelar 1200 switches and configured them in a partial mesh so each switch from Cisco connected to both switches from Nortel and vice versa, yielding four links. We then disconnected the active link for each vendor and verified, using ping, the reconvergence time was less than three seconds.

Test 9: Standby routing protocol (optional)

Description: Each router pair must demonstrate that a backup router will take over if a primary device fails, in accordance with Virtual Router Redundancy Protocol (VRRP).

Degree of Difficulty: Moderate

Participants: All Layer 3 devices except Cisco's Catalyst 8510

Results: All participants passed.

Along the same idea as the reconvergence test is our standby routing protocol test. The VRRP designates a backup router that automatically takes over if the primary router fails. The backup router must assume all responsibilities and identities of the failed device and intercept IP traffic sent from clients to the failed router. This process saves the manual step of reconfiguring all clients to a replacement router.

As in the reconvergence tests, we connected two routers from each vendor and deliberately disconnected one router. A passing grade required that the remaining routers recovered and took over communications with the failed device's clients. All six participants passed. Cisco didn't participate because its units support the company's proprietary Hot Standby Routing Protocol rather than VRRP.

Test 10: Gigabit Ethernet uplink (optional)

Description: Each pair of switches must interoperate across a single, full-duplex Gigabit Ethernet (1000Base-SX) uplink.

Degree of difficulty: Low

Participants: All except IBM's 8275-416 Ethernet Switch and 8371 Multilayer Ethernet Switch

Results: All 10 participants passed.

Our last optional test was the Gigabit Ethernet uplink test - another gimme in our book and the one area vendors didn't shy away from. This test was optional only because not all products have gigabit ports: IBM's 8275-416 Ethernet Switch and 8371 Multilayer Ethernet Switch don't. The other 10 switches have gigabit ports, and all successfully interoperated with one another across single, full-duplex Gigabit Ethernet (1000Base-SX) uplinks.

Interoperability report card

Vendors may not talk, but their switches do:

Layer 2 devices

P - Pass F - Fail	NT - Not tested U - Unsupported
* Responds to, but doesn't initiate, flow control messages.

	Auto- nego- tia- tion	Flow con- trol	IP rout- ing	IP RIP v1, v2	IPX rout- ing	IPX RIP	Link agg.	Ac- cel. con- ver- gence	Stand-- by rout- ing pro- to- col	Gig. Eth. up- link
Cable- tron Smart- Switch 6000	P	P*	U	U	U	U	P	NT	NT	P
Cisco Cata- lyst 2984G	P	P*	U	U	U	U	P	P	NT	P
HP Pro- Curve Switch 4000M	P	P	U	U	U	U	P	NT	NT	P
IBM 8275-416	P	P	U	U	U	U	U	NT	NT	U
Lu- cent Cajun P120 Work- group Switch	P	P	U	U	U	U	P	NT	NT	P

Layer 3 devices

	Auto- nego- tia- tion	Flow con- trol	IP rout- ing	IP RIP v1, v2	IPX rout- ing	IPX RIP	Link agg.	Ac- cel. con- ver- gence	Stand-- by rout- ing pro- to- col	Gig. Eth. up- link
Cable- tron Smart- Switch Router 8600	P	P*	P	P	P	P	P	NT	P	P
Cisco Cata- lyst 8510	P	P*	P	P	P	P	P	NT	NT	P
Foun- dry Big- Iron 4000	P	F	P	P	P	U	P	NT	P	P
HP Pro- Curve Rout- ing Switch 9304M	P	F	P	P	P	U	P	NT	P	P
IBM 8371 Multi- layer Ether- net Switch	P	U	P	P	P	P	P	NT	P	U
Lu- cent Cajun P550 Giga- bit Switch	P	P	P	P	P	P	P	NT	P	P
Nor- tel Ac- celar 1200	P	P*	P	P	P	P	P	P	P	P

Interoperability report card

Flow control feedback
How we dealt with flow control, and what vendors had to say about it. Network World, 9/13/99.

SwitchMetric results
See which switch gives the most bang for the buck in our latest round of LAN switch testing.

Feedback
Tell us your thoughts on this article or the issues it raises.