Even though I professed my dislike for the new name, the Cisco Nexus 7000 does have some very interesting new hardware features.
First, the concept of a unified fabric inside the data center has the ability to finally merge many DC network topologies and technologies. While it's not ready yet, the Nexus series will have FCoE for SAN connectivity, collapsing FC into the Ethernet network. However, I have to hold off judgement to see if this is a true differentiator since FCoE is not ready. It would be nice to have Fiber Channel SAN cards in the Nexus, truly creating a unified fabric (I couldn't find anything saying Cisco was working on that).
Second, we all love bandwidth, and the Nexus 7000 brings it. Now, most enterprise DCs will never need this much bandwidth, but it does set the switch up for, in essence, non-blocking transfers even on oversubscribed cards. You are essentially throwing bandwidth at the data center. Proper port assignment for hosts and uplinks could further protect bandwidth levels. A nice idea for the future would be per-port bandwidth reservations like the MDS-series can do. While today's backplane bandwidth is far less, the marketing hype is for 15 Tbps across the whole chassis in the future. The math works out, best as I can tell, like this:
230 Gbps In per slot
+230 Gbps Out per slot
----
460 Gbps
x 16 slots for cards in an 18-slot chassis
----
7360 Gbps (7.3 Tbps)
x 2 Future speed enhancements
----
~15000 Gbps (15 Tbps)
Third, and what I think is most intriguing, is the use of Virtual Output Queues (VOQs) in an Ethernet network to prevent head-of-line (HOL) blocking. HOL blocking can be a serious detriment to switch performance. VOQs, according to wikipedia, are "an input queuing strategy in which each input port maintains a separate queue for each output port". So, instead of a single queue for traffic entering the backplane destined for an output interface, now there is a separate queue on each input port for every output port (yes, that's a lot of queues on a 256 10Gig system).
HOL blocking is now gone. The centralized fabric arbiter ensures data from the VOQs can only enter the fabric when the receiving module, with the output interface, is ready to actual serialize the data onto the output wire. The fabric arbiter also ensures fabric module bandwidth is used as efficiently as possible.
Finally, there's Ethernet inside the Nexus 7000.
Cisco Nexus 7000 uses a switched Ethernet out-of-band channel (EOBC) for management and control traffic between the supervisors and line cards and between the supervisors and spine cards. On the supervisor modules, Ethernet connectivity is provided using an on-board 24-port Ethernet switch on a chip, with a one 1 Gbps Ethernet link from each supervisor to each line card, each supervisor to each switch fabric card (up to five), and between the two supervisors (Figure 9). Two additional redundant 1 Gbps Ethernet links are used on each supervisor to connect to the local CPU within the supervisor. This design provides a highly redundant switched-Ethernet-based fabric for control and management traffic between the supervisors and all other processors and line cards within the system.
This is not meant to be a perfect score for the Nexus 7000. There are some hardware deficiencies, particular the paucity of line card options. There are only three right now, one of which is the supervisor. If you want SFP gigabit Ethernet you're out of luck. And remember, these aren't cheap.
So, the hardware definitely has potential and is setup for the future. Smoothing some rough edges and adding some more options and this will be a powerhouse in DC networks....in 2009.
Michael Morris is a communications team lead and network architect at a $3 billion high-tech company. His background is in enterprise WANs working with telcos, and developing large-scale routing designs. He has worked on networks at government and corporate organizations, including networks at two Fortune 10 companies. In his current role, he leads large-scale IT networking projects and develops and maintains architectural standards for data networks, storage area networks, IP Telephony, and security. Michael is a CCIE and has 11 years experience in networking and communications, including four years as a paratrooper in the U.S. Army. He has a bachelor's degree in MIS from the University at Buffalo. Recently, he was awarded the Network Professional Association® (NPA) Professional Excellence and Innovation Award for his work on network architecture, templates and enterprise MPLS design.
|
|
Foundry has been doing this for 3+ Years!
I am an engineer for Foundry Networks. I just read your article. Very nice read! The only thing I’d like to respond to is the Virtual Output Queue technology in this product. Foundry has been shipping this feature since 2005 on the BigIron RX platform and it works quite well. Perhaps an article on some Foundry boxes might be beneficial in comparison to Cisco? The fact that Cisco isn’t doing anything really revolutionary in the datacenter is something that needs to be addressed. Also, the fact that they are late to the game is another. However, by Cisco using VOQ, it validates our decision to do so over 3 years ago so I guess it isn’t all that bad :)
Virtual Output Queues not new at Cisco.
Great overview on the Nexus 7000. This switch seems to have a bright future and really shows that Cisco is still capable of creating new cool stuff.
Nice to see that it took Foundry Networks biggest switches almost 8 years to finally copy a technology that Cisco have had for about 10 years.
From the beginning in GSR routers, MDS SAN Swtiches, Catalyst 6500 switches, and down the line all the way to the Catalyst 2950 switches, Virtual Output Queues have been used for over a decade in Cisco switching products to solve the HoL blocking problems.
The point of the article mentioning VOQ and HoLB seems to be that now that Ethernet and FC share a fabric, the use if VoQ ensures the "lossless" requirements of FC once it enters the switch fabric. Although the MDS has had this feature, its not something other SAN Switch vendors needed to necessarily do because of inherent flow control built in to FC.