Link aggregation: Arista, Blade and Cisco fare best

At least until the standards are done for 40G and 100G Ethernet, link aggregation will remain the preferred high-speed uplink method. Link aggregation, described in IEEE standard 802.3ad, binds multiple physical ports into a single, larger logical port – which in turns makes it easier to provision and manage high-speed flows.

Although link aggregation combines ports, it's not necessarily the case that throughput will scale linearly with port count. Fairness, or the uniformity of flow distribution across members of a link aggregation group (LAG), is always a concern.

Typically, switches inspect flow contents and use a hashing algorithm to assign each flow to a given member of a LAG. For example, every flow entering an eight-port LAG can take one of eight possible paths. A simple 3-bit hashing algorithm might inspect frame contents to determine where to assign the frame, because a 3-bit hash has eight possible outcomes. For example, the algorithm could inspect just the last 3 bits of each frame's source and destination MAC addresses to make a hashing decision.

To test the fairness of flow distribution, we configured each switch to support an eight-member LAG, and set up Spirent TestCenter to offer seven ports worth of unidirectional traffic across the LAG. The Spirent TestCenter test instrument emulated a total of 1,050 hosts, each using pseudorandom MAC addresses.

We counted the frames forwarded by each LAG member, and calculated the standard deviation of frame counts across all LAG members. We then administratively shut down one of the LAG members and again offered the same seven ports' worth of traffic across the remaining LAG members. While both the eight- and seven-port test cases measured fairness of distribution across LAG members, this second test also examined how each switch redistributes flows when LAG membership changes.

In both tests, we offered traffic at just 10% of line rate. Why? Because LAGs always introduce some unfairness of flows across LAG members, and the point of these tests is to find out how much. In situations with big disparities in frame counts between LAG members, frame loss can result if the offered load is high enough. Given the low offered load, we were confident that any disparities in frame counts were solely due to hashing across LAG members.

The results showed sizable differences between products. In the eight-port LAG case, Cisco's Nexus 5010 had the lowest standard deviation, meaning the most uniform distribution of frames across LAG members. The Dell and Arista switches were next most efficient.

In the seven-member case, the Cisco switch was about three times less uniform than in the eight-member case. And frame distributions were even more uneven for the Dell and Extreme switches, with standard deviations 11 and six times larger, respectively, than in the eight-member test case.

With the Dell and Extreme switches, if a LAG member goes away, all its flows simply migrate to one other LAG member. That led to a big jump in frame counts on that single member.

The most efficient switches in the seven-member LAG tests by far were Blade's and Arista's. Both actually improved hashing efficiency, something none of the other switches did. Arista's switch showed the least variation between eight- and seven-port cases, while the Blade switch showed the greatest improvement in the seven-port case.

See next part: Multicast group capacity: Extreme comes out on top

Return to main test.