So following on from my previous post, I threw out the comment that FCoE was nowhere. Lets dig into that a bit and turn over the garden bed of truth. Where else to start but fibrechannel.org
- From http://www.fibrechannel.org/FCoE.html#Q5
Q5: Where is FCoE in the standards process?
A: The project has been approved and meetings have been scheduled to agree upon details of the standard. There is wide agreement on most fundamental aspects of the technology. The wide technical interest assures that the standard is likely to meet its target date of 2H’08.
- Interesting. We have a Cisco Nexus 7000 (1 billion dollars of R&D) claiming a key benefit is to combine FCoE and DCE ( Data Centre Ethernet - which is apparently so different from other ethernet that it gets its own TLA) and FCoE is not out of standards any time soon.
No comparison with 802.11n is implied or intended.
But Cisco is laying down some serious commitment to FCoE. Is this going to drive acceptance ? Are people going to deploy FCoE ? This makes it seem inevitable ?!?
Does Cisco have any anyone else supporting them ?
Here is an interesting fact I wasn't aware of:
Q17: Can FCoE traffic and LAN traffic run on the same link?
A: Yes.
So now I get the point of FCoE. One monster server with two 10GB ethernets, will have support for both storage and data connections, and a big-ass load of virtual machines inside. I thought that was called iSCSI though. You know, the technology that has been around for ten years now.
Experience suggests that iSCSI is being resisted by the storage people. It doesn't have features that they need apparently. (It would if they got onto it, but maybe it is not in their interest to do so ? Adding new features to iSCSI wouldn't take much).
So is FCoE the technology that will act as the catalyst for transition to iSCSI. Possibly.
And since FCoE can't be routed to remote data centers, this is strictly limited. Surely FCIP is better than this, what I can't understand is why iSCSI is not considered.
I still wonder why Cisco is emphasising the storage element of the Cisco Nexus 7000. Obviously decisions about storage take years to happen, while the networking community can integrate new technologies in months. Cisco might be banking on networking people to pick the Nexus as the next thing after the C6509 and working it out without being told. So this might be a way for Cisco to target the SAN folks and get the slow wheels of change happening in the forward planning.
Also of interest is that Intel put FCoE software into Open Source in December http://www.networkworld.com/news/2007/121807-intel-fcoe-specification.html.
Q18: What are the benefits to running both FCoE and LAN traffic on the same link?
A: IO consolidation, e.g. fewer adapters, consistent management, power savings, cable management.
Well thats important. They forgot to point out that you can already do this in iSCSI...... oops sorry, forgot myself there.
Q19: How much credence should I give to Robert Metcalfe when he says “Fibre Channel is dead”?
A: None.
Well that is a little bit snotty. I also think that FibreChannel hasn't got a long term future and its a bit defensive to say it like that. How many technologies are the storage people going to play with before they accept convergence and simplicity ?
FCoE looks distinctly like whipping a losing horse from where I am standing.
And I would offer a differing, and contrarian opinion...
Gotta love open discussions-
I would take a different stance. :)
iSCSI is a 'stateful' protocol. This is based on each end-point's requirement to maintain the session state and buffer for any retransmission. As we all know IP and Ethernet are inherently lossey today. i.e. they drop traffic to signal to TCP that their is congestion so TCP can back off and not overrun the network. Thus iSCSI has to buffer for the entire bandwidth of the end-to-end transport for the round-trip latency of the end-to-end transport. i.e. large big buffers.
FCoE is a 'stateless' protocol. Since the underlying transport is lossless it can forward the traffic without having to buffer for the entire network.
Therefore FCoE is much lower latency - thus there is no tradeoff on any services a customer has come to expect and take for granted from their FibreChannel network: services like Synchronous replication, storage media encryption, data migration, etc.
Many of these services would be invalidated in an iSCSI environment because the latency will be much higher and frankly the cost-points will also be higher especially as you get to 10GbE.
If you want to calculate how much buffering you would need on a 150ms latency end-to-end network with up to 10Gb bandwidths there is a bit of memory required on each iSCSI initiator and target.
Thus, I feel that a network that can support iSCSI is required, but one that can also support lossless FCoE will be even more powerful and lower cost on initiator and target without sacrificing host-CPU cycles for I/O.
dg
FCoE and iSCSI - Horses for Courses
Clearly the writing is on the wall for native Fibre Channel's long term survival. Which technology dethrones the king will depend on many factors.
iSCSI makes sense for SMB and where performance and connectivity to existing FC storage are not high priorities.
FCoE at 10Gb/s will provide high performance, connectivity to existing FC devices via simple gateways and leverage of SAN management tools.
As customers look towards alternatives to fibre channel they can also consider InfiniBand connected storage using SRP or NFSoRDMA. This especially makes sense for customers building InfiniBand connected clusters and when low latency and high bandwidth are very important.
More on topic
I have had more to say in this post.
http://www.networkworld.com/community/node/24715
Hope you take the time to read it.
If your only tool is a hammer, every problem looks like a nail - Abraham Maslow
FCoE and iSCSI - Horses for Courses
Clearly the writing is on the wall for native Fibre Channel's long term survival. Which technology dethrones the king will depend on many factors.
iSCSI makes sense for SMB and where performance and connectivity to existing FC storage are not high priorities.
FCoE at 10Gb/s will provide high performance, connectivity to existing FC devices via simple gateways and leverage of SAN management tools.
As customers look towards alternatives to fibre channel they can also consider InfiniBand connected storage using SRP or NFSoRDMA. This especially makes sense for customers building InfiniBand connected clusters and when low latency and high bandwidth are very important.
RE: And I would offer a differing, and contrarian opinion...
>>> If you want to calculate how much buffering
>>> you would need on a 150ms latency end-to-end
>>> network with up to 10Gb bandwidths
>>> there is a bit of memory required on
>>> each iSCSI initiator and target.
I might be reading this wrong, but why would you run iSCSI over a 150ms latency link? I wouldn't do iSCSI over more than 3-5 ms.
?????
Mike
RE: And I would offer a differing, and contrarian opinion...
Some Fear Uncertainty and Doubt there I think ?
Yes, Ethernet (Data Centre Ethernet that is) would have a delay of less than 3 ms inside a data centre. But you can always sway the inexperienced or the illierate by pushing out pseudo facts.
Who the hell needs another transport protocol for Block transfer ? I have researched the FCoE and cannot work out what the point is ?
VMware works fine with iSCSI drives, why bother with FCoE ?
iSCSI Latency
Here's a question, if I use the Ethernet enhancements that are required for FCoE for iSCSI, won't I eliminate most dropped packets. Combined with an iSCSI accelerator/TCP offload engine (which seems a reasonable comparison since FCoE requires one), won't the latencies and performance be essentially the same over local networks (i.e. not routed)?
FCoE does not require a TOE
It is Fibre-Channel over Ethernet, not Fibre-Channel over TCP/IP.
FCoE traffic over a 10Gb Ethernet link incurs no TCP overhead.
Exactly!
People tend to miss that. In high performance networks, the constraints and overheads of TCP/IP prevent iSCSI from being practical. By going directly over Ethernet, FCoE can leverage a common fabric while taking advantage of FCP.
Load of rubbish
Encoding into IP is not the weakest link. While introducing a small amount of latency, technologies exist to manage this.
The biggest problem remains Windows with its very poor kernel driver structure, and inability to handle high performance NICs.
The second biggest problem is that CPU vendors are still focussed on CPU performance, and bus performance is very slow. That is why encoding into IP can be an issue as every IP requires two or three traversals of the bus.
Again, why choose second rate technology ! Choose to do it right.
If your only tool is a hammer, every problem looks like a nail - Abraham Maslow