To arrive at the DCoE – the Data Center over Ethernet – we need to find ways to make Ethernet behave like the non-Ethernet networks we deploy in our data centers: the fiber channel storage-area network and the high-performance compute network, whether InfiniBand, Myrinet or something else. The SAN represents the larger challenge in the sense that SANs are now nearly ubiquitous, in ways HPC fabrics are not.

Of course, there are already methods out there for sending storage traffic over Ethernet. In the open-standards space, there are things like FCIP and iSCSI, using TCP/IP as their base. There are proprietary protocols too, like Zetera’s ZSAN, which eschews TCP but still builds on IP. All have their applications and limitations, their proponents, and growing bases of users.

But perhaps the key to really widespread adoption of Ethernet for storage networking will turn out to be dropping IP for core block-level storage networks and instead falling back one layer, to Ethernet itself. With Ethernet, you discard some overhead from IP, but retain access to a control flow mechanism that allows nodes to do something other than simply drop packets when they have too many. This is the premise underlying the Fiber Channel over Ethernet (FCoE) effort.

As the name implies, FCoE essentially layers FC protocols over Ethernet. The key to making FCoE work is implementing the PAUSE frame type. PAUSE frames let the receiver tell the sender to stop sending (say, because buffers are nearly full). Each PAUSE specifies a pause length, with length 0 meaning “resume sending.” With this in hand, Ethernet is more able to serve as the essentially lossless transport that people rely on Fibre Channel to provide. It does this without the overhead of IP and/or TCP or UDP.

Of course there are hurdles and limitations; foremost among them is that PAUSE is not universally implemented. Sometimes it is even half-implemented (with a card or device knowing how to send or to read a PAUSE but not both). So, use will require a technology refresh in NICS and switches. And, of course, storage devices will have to support the new technology too, and that will take time.

There is also the key fact that, as a Layer 2 protocol, FCoE is not routable, unlike iSCSI and FCIP. And although it partakes of the market and engineering strengths of Ethernet, it cuts itself off from the equally rich ecosystem of TCP and IP, banking among other things on the reduction of processing overhead to improve performance more directly and cheaply than advances in full-stack implementations can.