What to look for in network fabrics

Fabric is one of the hottest topics in the data center, and with good reason. Virtualization of data center resources, an explosion of server-to-server traffic, and ever increasing application demands are stressing traditional infrastructures. But there are different ways to approach fabrics, all of which seem to have merit. What are the most important things to look for?

The Experts
Jason Nolet
Jason Nolet

Vice President, Data Center Networking Group, Brocade says you have to look for an approach that will let you grow incrementally, when you need it, with the least amount of pain. View debate

Shehzad Merchant
Shehzad Merchant

Vice President of Technology, Extreme Networks argues that customers need to look for high performance converged Ethernet switches, an open and interoperable network architecture, and a mature, modular and extensible network operating system. View debate

Jason Nolet

Minimal disruption

The Ethernet fabric market has grown rapidly in the past year. Market interest, the range of offerings and customer adoption are all on the rise. If you have been looking into a fabric you’ve likely developed expectations about what you want it to do that your existing network can’t. However, with the variety of offerings on the market, it can be difficult to figure out how to focus the decision process.

Brocade has more than 550 customers that have deployed Brocade’s fabric-enabled VDX Data Center Switches since their release in November 2010. Here is what our customers tell us is most meaningful to them, and likely would be to you as well.

First off, they relish the automation features we’ve built in and the simplicity of building a fabric:
•    Connect the switches together, and trunks form automatically with no configuration required.
•    Switching from “classic” (STP-enabled) mode to fabric mode involves a single command.
•    With all links active, the total number of devices required in service is greatly reduced, simplifying the overall network and reducing both cap-ex and op-ex significantly. Customers have cited savings of hundreds of thousands of dollars in the first year of operations.
•    “Almost perfect load balancing”, in the words of one industry expert, due to unique per-packet load-balancing that draws on techniques that are part of our storage fabric heritage.
•    15% to 20% reduction in time spent on basic network management through automation of common tasks.

“It just works,” one customer said. Brocade fabrics are self-aware and self-healing, automatically redistributing traffic in the event of a link outage to avoid disruptions or performance degradation. Additional links can be added non-disruptively, with rapid fabric reconvergence time. Human error, the most common cause of down time, is reduced with the automation of common functions.

Despite the fact that Ethernet fabrics are a relatively new phenomenon, some fabrics, such as Brocade’s, are based on very mature, reliable technology.

Customers also want minimal disruption. An access layer-based approach, like Brocade’s, directly addresses the pains that drive many customers to look at fabrics to begin with: the latency and unnecessary load on core switches that is incurred for the east-west traffic patterns that are increasingly common. Most other fabrics are based on much more expensive core switches which in many cases do little to simplify existing network architecture or improve east-west traffic performance.

We allow customers to deploy fabrics progressively, workload by workload or pod by pod, without necessitating a redesign of their overall data center network. Some switches, such as Brocade’s VDX series, can be deployed in traditional STP mode first, and then switched to fabric mode when the customer is ready. Traffic is passed seamlessly to and from upstream switches running STP, with the fabric appearing as a single switch to the rest of the network. Fabrics with these capabilities are the best choice for targeted experimentation with fabric technologies with little operational risk and relatively low capital outlay.

Also, the right fabric does not require an entirely new set of skills and training to manage. CLI commands should be familiar, even to professionals whose primary training is on equipment from other vendors. Most find they can set up and operate a fabric easily with little instruction.

Altogether, our customers have found that partnering with Brocade has helped them work through network inflexibility and brittleness, improve application performance, and increase service velocity while significantly reducing capital outlays and ongoing operating costs.

Brocade networking solutions help the world's leading organizations transition smoothly to a world where applications and information reside anywhere.  Interested in learning more?  See What is an Ethernet Fabric?, “Ethernet Fabrics” or this video “SDN As it Relates to Ethernet Fabrics

Shehzad Merchant

No vendor lock-in

A data center Ethernet fabric requires certain attributes such as high performance, low latency and resiliency. However, a key aspect of any fabric is the network operating system (NOS) and the software protocols and management layer required in making the fabric high performance while reducing operational overhead. The evolution to technologies such as VXLAN/NVGRE, as well as the move towards software defined networking (SDN), all point to the need for a high performance fabric that is open and interoperable, and that does not lock the network into a vendor-specific proprietary technology.

A high performance network fabric requires both the right network switches and the right network architecture. Some key attributes for high performance network switches are:
* High density, high fan-out and non-blocking. High density 10G at the server network edge, high density 40G in the network core.
* Cut through forwarding on both chassis and stackables for low latency.
* Dynamic adaptive per port/queue buffer thresholds for good burst absorption capabilities. This is important when dealing with high performance storage along with big data Hadoop and map-reduce type technologies to address incast type scenarios and temporary congestion in the network.
* Single copy egress pipeline replication for multicast traffic to improve multicast performance and reduce multicast latency variance across ports.
* Support for LAN-SAN convergence with DCB capability (PFC, ETS, and DCBX).

And when it comes to network architecture, there are many attributes to consider. You should strive for as few tiers as possible to reduce latency, oversubscription and management overhead. For example, a high availability chassis that has 700+ 10Gigabit Ethernet or 175+ 40G Ethernet non-blocking ports will inherently require fewer network switches than one that has a fewer number of ports when it comes to fan out.

You should also look for Active-Active redundancy. Multi-system Link Aggregation (MLAG) is a technology that builds upon traditional LAG and works in conjunction with LAG to provide active-active redundancy across servers, network, storage as well as other network infrastructure such as firewalls, application delivery controllers, etc.

MLAG does not require any new encapsulation and hence can work with most existing infrastructure. While technologies such as TRILL or SPB can also provide benefits of active-active redundancy, there is little to no support for either TRILL or SPB in servers, hypervisors or in storage devices. In other words, MLAG/LAG can provide full end to end active-active redundancy, today.  And as newer technologies such as VXLAN/NVGRE come into play, MLAG/LAG can continue to work seamlessly in these environments.

Another important factor to consider is that, while Ethernet fabrics today tend to be Layer 2 oriented, in the near future with VXLAN/NVGRE, the fabric can move towards a segmented Layer 3 fabric with Equal Cost Multi Path routing. By avoiding vendor lock-in into a proprietary single vendor Layer 2 fabric technology, the network can evolve and take advantage of these advancements without requiring a “rip and replace” strategy. 

A network fabric discussion would not be complete without mention of storage and convergence. The move to a converged fabric is becoming a reality with 10G/40G Ethernet (and in the near future 100G Ethernet) and Data Center Bridging technology now becoming commonly available both in network switches and Converged Network Adapters. As technology evolves to accommodate Layer 3 fabrics, it will be important to use storage technology that is both routable and easy to virtualize. Technologies such as iSCSI provide an increasingly attractive alternative to legacy Fibre Channel storage for just those reasons.

A high performance Ethernet fabric is only as effective as the network operating system running on it. A single, mature and modular NOS that runs across the network switch infrastructure significantly reduces overhead and simplifies day-to-day management. Furthermore, a NOS that integrates with the virtual machine environment can significantly reduce the operational overhead. For example, the NOS needs to support virtual port based configuration, “follow the VM” type policies, and complete network-based VM life-cycle management, across multiple hypervisor technologies.

Finally, consideration needs to be given to the fact that the data center fabric is evolving rapidly. The fabric needs to support the evolution towards a scalable, segmented Layer 3 network without requiring “rip and replace.” The NOS also needs to support Software Defined Networking (SDN) oriented technologies such as OpenFlow and OpenStack. 

In conclusion, customers need to consider not just high performance Converged Ethernet switches, but also an open and interoperable network architecture, along with a mature, modular and extensible network operating system. By taking a holistic view to network fabrics, a more robust, high performance and cost effective fabric can be deployed to address customer needs as they evolve.

Extreme Networks is a technology leader in high performance Ethernet switching for cloud, data center and mobile networks.  The company has more than 6,000 customers in more than 50 countries.

Want more Tech Debates? Check out our archive page


Copyright © 2012 IDG Communications, Inc.

The 10 most powerful companies in enterprise networking 2022