Traditional approaches to data center networking cannot satisfy the scale, bandwidth, latency and cost points required for evolving data center software architectures.

Fibre Channel and InfiniBand remain costly due to the specialized knowledge needed to implement, tune and administer, while traditional Ethernet switches fail to meet the stringent new demands. The best alternative is Ethernet fabric solutions designed specifically for the modern data center.

An Ethernet fabric provides a low-cost, high-speed, ultra-low-latency, loss-less and scalable network infrastructure. It can offer a full cross-sectional interconnect for 1 Gigabit Ethernet (GE) or 10 GE attached servers, and allows the compute and storage servers to maximize aggregate compute power in a cost effective way. Key attributes are:

• Scalability to hundreds of 10 GE ports and thousands of 1 GE ports.

• Nonblocking (wirespeed) full cross-sectional bandwidth.

• Loss-less packet delivery.

• Ultra-low latency (sub-six-microsecond latency across a multidevice core).

Conventional Ethernet networks consist of a multitier hierarchy, with bandwidth oversubscription in every tier, as shown in Figure 1. These networks are typically designed for Fast Ethernet and 1GB servers and clients, where 10GB is often used for interconnecting the Ethernet switches.

Click to see: Diagram showing overcoming Ethernet limitations in the data center

Spanning tree protocols are used for loop avoidance and active-standby resiliency. While spanning tree accommodates loop-free Layer-2 topology, it reduces the overall bandwidth efficiency of the network because of the lack of readily available alternate paths. The reason is that the time needed to reconstruct the tree, even with the so-called rapid spanning tree protocol, takes many seconds rather than the few milliseconds required. Therefore, conventional Ethernet switches are not optimized for high-density 10GB interfaces.

Ethernet fabrics, on the other hand, are based on a Clos architecture, also known as a fat-tree topology. Unlike traditional multitier Ethernet networks that oversubscribe 1GB or 10GB links to interconnect switches, Ethernet fabrics require every switching tier to be connected to the next tier closer to the root with higher aggregate bandwidth and without any over-subscription, guaranteeing a nonblocking switch fabric.