How we did it

How we tested router performance when equipped with a filter.

We assessed router performance in four ways: baseline measurements; performance with filters applied; performance with routing and filters applied; and reboot time under load.

For all tests, vendors were asked to supply a matched pair of devices, each equipped with at least two Ethernet and two T-1 interfaces (see test diagram). We also required support for Border Gateway Protocol (BGP) and Open Shortest Path First (OSPF) routing, including the ability to hold a full Internet table (125,000 entries) learned via BGP.

We used the SmartBits traffic generator/analyzer from Spirent Communications for all tests. In the filtering and routing tests, we used the SmartBits 6000B chassis equipped with LAN-3301A cards. For the reboot tests, we used the SmartBits 2000 chassis equipped with ML-7710 cards. We custom-developed test scripts for this project using perl and Spirent’s SmartBits API scripting facility.

In all cases, we attached the SmartBits to two Ethernet interfaces on each router and used WAN crossover cables to connect the T-1 interfaces.

In the baseline tests, we configured the SmartBits to offer traffic in a bidirectional partial-mesh pattern as described in RFC 2889. We measured throughput and average and maximum latency over a 60-second test duration. We repeated this test using 64-, 256- and 1,518-byte Ethernet frames carrying UDP/IP packets.

In the filtering tests, we offered the same traffic as in the baseline event but configured one router under test with various numbers of filters. We repeated this test using eight, 16, 64 and 256 filters in place. We picked the various numbers to see whether routers would be able to check against progressively larger rule sets. We picked the filter conditions to verify routers could enforce commonly used traffic criteria, including source and destination IP addresses, protocol, and TCP and User Datagram Protocol (UDP) port numbers.

In the routing tests, we used the same procedure and configuration as in the filtering test but also enabled BGP and OSPF on one router under test. We tested in two configurations. First, we tested a “small routing” setup in which we advertised 64 routes via each of the two routing protocols. Second, we used “big routing” setup with 4,096 routes advertised via OSPF and 125,000 routes via BGP.

In the reboot tests, we set up the routers with no filters and no dynamic routing enabled. We offered 64-byte frames to all four Ethernet interfaces at T-1 line rate. Then we used an APC Masterswitch remote power console to power down the routers, wait 5 second and then power up the routers. The SmartBits continued to offer traffic continuously. At the end of a 5-minute test duration, we derived reboot time from frame loss.

Back to main review: “Filters on routers: The price of performance”