How we tested Cisco's ASR

We assessed Cisco ASR 1000 performance with tests of unicast and multicast throughput and latency; high availability features, including failover and upgrades; and IPSec tunnel capacity. A more detailed version of the test methodology is available here.

For everything except IPSec, the device under test was one Cisco ASR 1006 chassis equipped with one 10Gigabit Ethernet and 10 1-gigabit Ethernet interfaces; two ESP20 modules; and two RP1 route processor modules, all running Version 12.2(33)XNB2 of IOS XE software. The test instrument in all cases was Spirent TestCenter running Version 2.32 software.

In tests of unicast throughput and latency, we configured the router with 20 subinterfaces on each of 10 1-gigabit interfaces and five subinterfaces on the 10G Ethernet interface. Then we configured Spirent TestCenter to bring up 205 Open Shortest Path First (OSPF) adjacencies and advertise 320,000 routes – 100 apiece to the gigabit subinterfaces, and 60,000 apiece to each 10-gigabit subinterface. We offered external advertisements to the 10G subinterfaces (all in OSPF area 0) and NSSA advertisements (all in area 1) to the gigabit subinterfaces.

Once all the OSPF routing tables were populated, we offer bidirectional streams of 64-, 256- and 1,518-byte Ethernet frames in a backbone pattern, so that all gigabit subinterfaces offered traffic to all 10G subinterfaces and vice-versa. We used a binary search pattern to determine the throughput rate and, per RFC 2544, measured average and maximum latency at the throughput rate. We also determined whether any frames were delivered out of sequence. The duration for each test iteration was 300 seconds.

To assess multicast performance, we configured the Cisco ASR router to run PIM-SM to distribute multicast routes. We also configured Spirent TestCenter's 10G Ethernet port to transmit traffic to 200 multicast groups, each with 50 transmitting hosts per group, for a total of 10,000 multicast routes (mroutes). On the gigabit Ethernet side, Spirent TestCenter emulated one host on each of 20 subinterfaces per port, for a total of 200 emulated hosts. Each host used IGMPv3 to join all 200 multicast groups. Once all joins were complete and the multicast tree was fully populated, we offered 64-, 256- and 1,518-byte frames using a binary search pattern to find the throughput rate, and to measure average and maximum latency at that rate. As with the unicast tests, the test duration was 300 seconds.

We assessed high availability and resiliency with four tests of route processor (RP) and embedded services processor (ESP) failover and in-service software upgrades and downgrades. The ASR router was equipped with redundant RP and ESP modules. To test failover of these components, we offered 64-byte unicast frames from Spirent TestCenter at the throughput rate and asked Cisco's engineer to administratively disable the primary module. In separate tests for the RP and ESP, we noted any packet loss and derived failover time from this figure.