How We Tested 10Gigabit Ethernet switches

We invited 13 vendors to participate in this test. Besides the six that accepted (Arista, Blade, Cisco, Dell, Extreme, HP ProCurve), we also invited 3Com/H3C, Brocade, Enterasys, Force10, Fujitsu, Juniper and Raptor.

3Com/H3C, which was bought by HP during testing, earlier declined due to resource constraints (which, we later learned, had to do with preparation for the acquisition by HP). Brocade, Enterasys, Force10 and Juniper also cited resource constraints. Fujitsu and Raptor did not respond to our invitation.

We assessed switches in 10 areas: features; management and usability; power consumption; MAC address capacity; unicast and multicast throughput; unicast and multicast latency and jitter; link aggregation fairness; multicast group capacity; multicast join/leave delay; and forward pressure.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

We invited 13 vendors to participate in this test. Besides the six that accepted (Arista, Blade, Cisco, Dell, Extreme, HP ProCurve), we also invited 3Com/H3C, Brocade, Enterasys, Force10, Fujitsu, Juniper and Raptor.

3Com/H3C, which was bought by HP during testing, earlier declined due to resource constraints (which, we later learned, had to do with preparation for the acquisition by HP). Brocade, Enterasys, Force10 and Juniper also cited resource constraints. Fujitsu and Raptor did not respond to our invitation.

We assessed switches in 10 areas: features; management and usability; power consumption; MAC address capacity; unicast and multicast throughput; unicast and multicast latency and jitter; link aggregation fairness; multicast group capacity; multicast join/leave delay; and forward pressure.

For all but the first two areas, we used the Spirent TestCenter traffic generator/analyzer equipped with 24 10-gigabit Ethernet Hypermetrics CV modules and 10GBase-SR transceivers.

To assess switch features, we asked vendors to complete a detailed questionnaire. We did not verify every answer to this questionnaire.

The management and usability assessment was based in equal part on our use of the switches during testing and on responses to the features questionnaire (for example, in terms of supported network management methods).

We measured power consumption using Fluke 322 and Fluke 335 clamp meters. This test involved three measurements: AC line voltage; AC amperage when idle; and AC amperage when fully loaded. We fully loaded the switch control and data planes by configuring Spirent TestCenter to offer traffic at line rate to all ports. We derived wattage by multiplying voltage and amperage. We then removed 12 transceivers from each switch and repeated these measurements.

To measure MAC address capacity, we used the RFC 2889 wizard in Spirent TestCenter. This wizard conducts a binary search to find the largest number of MAC addresses a switch can learn without flooding. In all test iterations, Spirent TestCenter's MAC address aging timer is set to twice that of the switch under test.

To measure unicast throughput, latency and jitter, we configured Spirent TestCenter to offer traffic to all ports in a fully meshed pattern. For each test, we conducted separate 60-second runs with 64-, 65-, 108-, 256-, 1,518- and 9,216-byte frames, using a binary search to determine the throughput rate. For each frame length, we measured throughput, average and maximum latency and average and maximum jitter.

The multicast throughput, latency and jitter tests used the same frame lengths as in the unicast tests. Here, we configured a single Spirent TestCenter port to transmit multicast traffic, and the remaining 23 ports to join the same 989 multicast groups.

To assess link aggregation fairness, we configured Spirent TestCenter to act as a link aggregation partner using link aggregation control protocol (LACP), and also to emulate transmit and receive hosts on seven pairs of ports. Initially we brought up a link aggregation group (LAG) consisting of eight ports, and then offered unidirectional traffic to seven ports (in the direction from emulated ports on TestCenter to the switch, then on to LAG members on TestCenter, and then on to emulated hosts on TestCenter). We offered traffic at 10% of line rate to avoid oversubscription of any LAG member. Then we disabled LACP on one TestCenter port and offered the same traffic as in the eight-member LAG tests. In both cases, we recorded packets received on each LAG member, and derived fairness by calculating standard deviation across received-frame counts.