Switch metrics put vendors to the test

By Kevin Tolly
Network World, 09/06/99

The results from two joint Network World/Tolly Group SwitchMetric testing efforts hit the street next week at NetWorld+ Interop '99 Atlanta.

Next week in Network World and on The Tolly Group's Web site, we'll unveil the second set of results for the 1999 Network World/Tolly Group SwitchMetric. The program's aim is to generate independent test results that validate vendor claims with regards to performance capabilities for Layer 2 and Layer 3 switches. SwitchMetric is a new benchmark to help determine the price for each gigabit per second of throughput that a switch provides.

The upshot is that nine months after we kicked off the program, companies such as Cisco, Compaq, Nortel Networks and Xylan still haven't tested their switches. Each company offers a suspect reason, which falls into two camps: a) The product is in such high demand that the company can't seem to locate a unit for testing, or b) The vendor prefers to wait for the next round of testing because it has a next-generation product in the works.

In this latest round of testing, which occurred in July and August, four vendors offered up five switches. Network World readers who contacted me by e-mail maintain that these tests are important. Moreover, network managers use the SwitchMetric results to independently verify vendor performance claims they find in advertising or that sales engineers lob at them. Vendors who refuse to participate lose credibility. Some users have even dropped vendors from consideration, as our report will show.

What makes it more confounding is that Network World and The Tolly Group teamed again this summer to conduct a Switch Interoperability study. Seven vendors demonstrated interoperability among 12 Gigabit Ethernet switches. The vendors were Cisco, Cabletron, Foundry Networks, Hewlett-Packard, IBM, Lucent and Nortel.

Switches were tested for autonegotiation, flow control, IP and IPX routing, IP and IPX Routing Information Protocol (RIP), standby routing protocol, accelerated convergence, link aggregation and Gigabit Ethernet uplinks. What you should note as you view the Switch Interoperability results next week is not who showed up, but what they tested. Universally, a number of vendors tested interoperability of some functions, while bunches of other vendors shied away from other tested functions.

Like the AWOL vendors in the SwitchMetric tests, the missing test data from the Switch Interoperability study should speak volumes about what vendors can and cannot accomplish from an interoperability standpoint.

Why are key industry leaders hesitant to test their products from a cost-metric viewpoint? Why are other vendors that participate in switch interoperability reluctant to test interoperability for IP and IPX routing, and IP and IPX RIP information?

As you roam the cavernous show floor in Atlanta next week, see if vendors will give you good answers or if they'll skirt the issues.