Ethernet Adapts for Data Center Applications – Part 2

Data Center Bridging – You Want Multi-Pathing With That Network?

Ethernet inherently doesn't provide the ability to multi-path because STP is blocking our redundant links to mitigate loops in the network.  So if you are implementing Fibre Channel over Ethernet and have promised your SAN team that the network won't lose their Fibre Channel frames, the next hurdle will be multi-pathing.  (See previous post that discussed the ways Fibre Channel and Ethernet don't get along, and why Data Center Bridging is the answer.)

How do we cross that chasm? There are two approaches in my mind and they relate to how you plan to implement FCoE in your network. We'll start with the mode available today, single-hop, and then delve into multi-hop. 

Single - hop as its name implies is a topology where the Fibre Channel traffic is encapsulated in Ethernet for a single hop, from the host to the switch. The switch strips the Ethernet frame off of the Fibre Channel frame and transmits it as a native Fibre Channel frame into the SAN. In this topology multi-pathing isn't a real consideration because the best practice would be to connect the FCoE enabled host to different physical switches which are in turned connected to the A fabric and B fabric in the SAN. Multi-pathing would be handled on the host and VLAN/VSAN isolation provides the separation on the Ethernet topology.

In a multi-hop topology, the design becomes more interesting as our Fibre Channel network isn't connected to the first hop switch and we now have multiple Ethernet paths through the network. As we discussed earlier, STP will block links in our network to provide a loop free topology which won't meet Fibre Channel's needs. Enter Cisco Fabric Path and Transparent Interconnections of Lots of Links (TRILL). We'll discuss Cisco Fabric Path and TRILL in more detail in the next post, but know that these two technologies provide the ability to run Ethernet networks without STP by doing MAC based routing.

Cisco FabricPath as its name implies is a Cisco proprietary implementation of MAC routing while TRILL is the IETF project. With either Cisco FabricPath or TRILL we'll be able to provide topologies with mammoth amounts of parallel links and lots of loops that are all forwarding, all the time due to the absence of STP. I refer to this as "full-tilt boogie" networking. Pretty exciting stuff and when coupled with DCB provides a network SAN admins can place their lossless traffic on without worry of dropped frames and all of the benefits of multi-pathing.

                The practical reality I've experienced with customers as they adopt FCoE into their network has been positive. Most customers are not in a hurry to be the first to implement a new technology and as such, put FCoE through some rigorous tests in their lab to better understand the operational changes it bring to the table on both the LAN and SAN side. None of these changes have been monumental or show stoppers by any stretch and we continue to see FCoE adoption increase. The FCoE ecosystem continues to grow and mature as CNA vendors work on releasing a 3^rd generation of their technology and we see FCoE become available in more host and storage platforms.

Ultimately we'll see more adoption of FCoE over time as the technologies mature, operational best practices are established and mature and those lessons learned from the school of hard knocks are identified. We observed the same trend with SNA to IP migrations (RSRB to DLSw and then DLSW+ anyone?) and VoIP. The industry as a whole has the experience and wherewithal to reap the same cost benefits and reduction of equipment in the data center as we have with the other consolidated technologies. Will it be easy, not necessarily, but as we've learned from history we've been able to avoid many of the early challenges we saw with convergence.  QoS as an example is much better known and understood than it was years ago and will become a key technology in the data center.

I look forward to your comments and a deeper analysis of Cisco FabricPath and TRILL in my next post.

• Cisco
• FCoE TRILL FabricPath DCB VSAN