How we tested Cisco's ASR

We assessed Cisco ASR 1000 performance with tests of unicast and multicast throughput and latency; high availability features, including failover and upgrades; and IPSec tunnel capacity. A more detailed version of the test methodology is available here.

For everything except IPSec, the device under test was one Cisco ASR 1006 chassis equipped with one 10Gigabit Ethernet and 10 1-gigabit Ethernet interfaces; two ESP20 modules; and two RP1 route processor modules, all running Version 12.2(33)XNB2 of IOS XE software. The test instrument in all cases was Spirent TestCenter running Version 2.32 software.

In tests of unicast throughput and latency, we configured the router with 20 subinterfaces on each of 10 1-gigabit interfaces and five subinterfaces on the 10G Ethernet interface. Then we configured Spirent TestCenter to bring up 205 Open Shortest Path First (OSPF) adjacencies and advertise 320,000 routes – 100 apiece to the gigabit subinterfaces, and 60,000 apiece to each 10-gigabit subinterface. We offered external advertisements to the 10G subinterfaces (all in OSPF area 0) and NSSA advertisements (all in area 1) to the gigabit subinterfaces.

Once all the OSPF routing tables were populated, we offer bidirectional streams of 64-, 256- and 1,518-byte Ethernet frames in a backbone pattern, so that all gigabit subinterfaces offered traffic to all 10G subinterfaces and vice-versa. We used a binary search pattern to determine the throughput rate and, per RFC 2544, measured average and maximum latency at the throughput rate. We also determined whether any frames were delivered out of sequence. The duration for each test iteration was 300 seconds.

To assess multicast performance, we configured the Cisco ASR router to run PIM-SM to distribute multicast routes. We also configured Spirent TestCenter's 10G Ethernet port to transmit traffic to 200 multicast groups, each with 50 transmitting hosts per group, for a total of 10,000 multicast routes (mroutes). On the gigabit Ethernet side, Spirent TestCenter emulated one host on each of 20 subinterfaces per port, for a total of 200 emulated hosts. Each host used IGMPv3 to join all 200 multicast groups. Once all joins were complete and the multicast tree was fully populated, we offered 64-, 256- and 1,518-byte frames using a binary search pattern to find the throughput rate, and to measure average and maximum latency at that rate. As with the unicast tests, the test duration was 300 seconds.

We assessed high availability and resiliency with four tests of route processor (RP) and embedded services processor (ESP) failover and in-service software upgrades and downgrades. The ASR router was equipped with redundant RP and ESP modules. To test failover of these components, we offered 64-byte unicast frames from Spirent TestCenter at the throughput rate and asked Cisco's engineer to administratively disable the primary module. In separate tests for the RP and ESP, we noted any packet loss and derived failover time from this figure.

To test in-service software upgrades and downgrades, we moved all system components between version 12.2(33)XNB1 and 12.2(33)XNB2 of the IOS XE software, all while offering 64-byte frames at 50% of the throughput rate (we were unable to complete testing with traffic offered at the throughput rate). Again, we derived failover time by dividing packet loss counts into the offered packet rate.

In the IPSec tunnel capacity tests, we used two ASR 1006 routers, each with a single ESP20 module and redundant RP1 modules, linked via a Cisco Catalyst 7604 switch (required to set up different routes for each tunnel). The IPSec tunnel endpoints were the 10G Ethernet interfaces facing the Catalyst 7600.

We configured Spirent TestCenter to offer a unidirectional stream of 64-, 256- and 1,400-byte frames to determine the throughput rate and measure average and maximum latency. In this case, all traffic went through 2,000 IPSec tunnels with unique security policy identifiers, something we verified with a packet capture. To demonstrate the Cisco routers' ability to handle a mix of encrypted and cleartext traffic, we then repeated these tests offering 64-byte frames to be encrypted at the throughput rate while concurrently offering 1,518-byte frames to travel as cleartext. We offered cleartext traffic in both directions to consume as much bandwidth as possible and again measured throughput and average and maximum latency.

< Return to test: Cisco's new router built for 10-year tenure >

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10