Nexus 5000 Only Supports 16 EtherChannels

Interesting the Things You Learn Later On

An interesting, and most likely uncritical, factoid popped up this week while our operations team was configuring a set of Nexus 5000 switches for 10GbE connections to blade servers in our DC.

By default, our interface configuration standard for blade centers used port-channels since, in the past, we connected blade centers with 1GbE interfaces, not 10GbE. With 10GbE from blade center switches to N5000s, there was only 1 interface in the port-channel, but we still used the configuration for historical consistency. Until we hit the limit this week. Apparently, the Nexus 5000 only supports 16 EtherChannels (PortChannels): I found this surprising for a few reasons. First, that's not a lot portchannels when you have 52 10GbE ports to design with.

Most portchannels in the 10GbE world will be 2 or 3 interfaces, 4 being very high. That would mean you would run out of portchannels before interfaces. A portchannel to interface ratio on the N5000 is 16:52 or 4:13. Other access-layer Cisco switches have much higher portchannel density:

Switch TypeMax PortChannelsRatio
3750-E481:1
C4948644:3
N5000164:13

I suppose Cisco's thinking with a 10GbE switch was EtherChannels would only be used for uplinks, thus requiring a fewer amount as compared to 1GbE switches which may need to create more portchannels for connections to hosts. Finally, if more than one interface is used as a downlink on the N5000 to a Nexus 2000 FEX, those port channels would be included in the limitation. Having more than one 10GbE uplink out of a FEX in a portchannel on the N5000 is not out of the ordinary. All-in-all, I don't see this as a huge issue. 16 portchannels will be more than enough for the time being. Just interesting what you learn later on. I wonder if this is a software limitation or a hardware/ASIC limitation?

More >From the Field blog entries:

A Graphical View of My Idea of WAN Acceleration and More Bandwidth

I Have Been Disillusioned by WAN Acceleration

WAN Carriers Need Better Bandwidth Pricing Models for Enterprises

I Got an iPad

Is Enterprise IT Here to Provide Great Internet Access?

Latency in the Data Center Matters

  Go to Cisco Subnet for more Cisco news, blogs, discussion forums, security alerts, book giveaways, and more.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Related:

Copyright © 2010 IDG Communications, Inc.

SD-WAN buyers guide: Key questions to ask vendors (and yourself)