How we tested Juniper's switch

We assessed Juniper's new enterprise switch using the same methodology we previously used to test other vendors' access switches. The one exception, as noted below, was in our use of IGMPv2 instead of IGMPv3 this time around.

This methodology comprises 10 sets of tests covering L2 and L3 unicast performance; IGMP group multicast capacity; L2 and L3 multicast performance; network access control (NAC)/802.1X; storm control; power consumption; switch manageability, security, and usability; and switch features. A detailed, complete methodology is available here.

In the L2 unicast performance tests, we configured each switch with a single virtual LAN (VLAN) encompassing all ports. We attached a Spirent TestCenter generator/analyzer to all 48 gigabit Ethernet and two 10-Gigabit ports on the switch and ran three sets of tests: all ports, gigabit ports only, and 10-Gigabit ports only. We offered traffic to the gigabit ports in a fully meshed pattern and to the 10-Gigabit ports in a meshed pattern. For each test, we conducted separate 60-second runs with 64-, 256- and 1,518-byte frames, and measured throughput, average latency and maximum latency for each frame length.

The L3 unicast performance tests were similar to the L2 unicast tests, except in this case we configured each switch port to use a different VLAN and IP subnet.

In the IGMP group capacity tests, we reverted to an L2 configuration, enabled IGMP snooping, and set the switch to act as an IGMP querier. In this test, 47 of 48 TestCenter gigabit Ethernet ports joined some number of IGMPv2 groups; the 48th TestCenter port acted as a monitor to detect flooding.

After sending group membership (join) messages and waiting at least twice the switch's IGMP query interval, TestCenter's ScriptMaster software then offered multicast traffic to the switch's first 10-Gigabit port, destined for all multicast groups. Per RFC 3918, if all groups received at least one frame, the test iteration was considered a pass. If loss or flooding occurred, the iteration was considered a failure. Using a binary search algorithm, we repeated this procedure to determine multicast group capacity.

In the L2 multicast performance tests, we configured all switch ports to join a single VLAN, to use IGMP snooping, and to act as an IGMP querier. Then TestCenter's 48 gigabit ports joined 500 IGMPv2 groups (or fewer, depending on results from the group capacity test). The Juniper switch did not support IGMPv3 at test time, requiring the use of IGMPv2; this is the one significant departure from earlier tests of access switches.

After waiting at least twice the switch's IGMP query interval, TestCenter's ScriptMaster software then offered multicast traffic to the switch's first 10-gigabit port, destined for all multicast groups. Using a binary search algorithm, TestCenter determined the throughput rate. In a separate test, TestCenter measured average and maximum latency at the throughput rate.

In the L3 multicast throughput and latency tests, we configured each switch port to use a separate VLAN and IP subnet, enabled protocol independent multicast-sparse mode routing on each port, and set the switch to act as a PIM rendezvous point. The test setup and traffic pattern was similar to the L2 multicast test. We again determined the throughput rate and measured average and maximum latency at that rate.

To assess 802.1X/NAC support, we developed six scenarios that describe roles a switch might play as part of the NAC infrastructure. In this case we attached the switch to a Windows 2003 server running Juniper Steel-Belted Radius Enterprise Edition 6.1 (SBR). The SBR configuration used Windows Active Directory credentials to authenticate users.

In the first scenario, the switch places an authenticated client (in all cases, a PC running Windows XP Professional and Juniper Odyssey client software) into a previously configured VLAN. The second case is like the first, but requires authentication of multiple clients attached to a single port. In the third case, the switch dynamically assigns a VLAN after authentication. In the fourth case, the switch dynamically applies an access control list after authentication. In the fifth case, the switch places a client into a guest or restricted VLAN upon authentication failure. Finally, the sixth case determines whether a switch port concurrently supports 802.1X and media access control authentication support.

To assess storm control, we used common attack techniques such as broadcast and TCP SYN flooding as generated by a Mu Dynamics Mu-4000 security analyzer and by Spirent TestCenter. We configured the Juniper switch to limit forwarding rates of attack traffic, and verified these limits using real-time rate counters in Spirent TestCenter.

We measured power consumption using Fluke 322 and Fluke 335 clamp meters. This test involved three measurements: AC line voltage; AC amperage when idle; and AC amperage when fully loaded. We fully loaded the switch control and data planes by configuring Spirent TestCenter to offer traffic at line rate to all ports consisting of IPv4 packets with IP options set. We derived wattage by multiplying voltage and amperage.

Our tests of switch manageability, security and usability had objective and subjective components. In the objective component, we determined which management methods the switch supported over IPv4 and IPv6, as well as its ability to conform to best security practices (for example, by disabling vulnerable services such as telnet and enabling secure service such as SSHv2). We also determined which management methods were enabled by default, and which could be enabled/disabled by users. In addition, we determined whether erasing configuration files would remove all personally identifiable information, a regulatory requirement and security best practice.

The subjective part of our assessment consisted of our judgments on ease of accomplishing these and all other tests described here.

To assess the final area, switch features, we asked vendors to complete a detailed questionnaire. We did not verify every answer to this questionnaire.

< Return to test: Juniper switch proves to be credible choice >

Learn more about this topic

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2008 IDG Communications, Inc.