Multi-Hop FCoE Is Not Ready For Prime Time (Yet)

Apparently my “bias” against FCoE is showing. I asked a question, Will 16 Gb Fibre Channel Derail FCoE?, and stirred up controversy and a series of responses: Metz, Reams, Onisick and lots of Twitter talk. Although FCoE wasn’t really the topic of that little post, some readers criticized my statement that FCoE isn’t “really ready for prime time at this point.” So let’s talk about that.

Ready For Prime Time?

Now, “ready for prime time” isn’t really a technical term with a defined meaning, and perhaps this is the root of our issue. “Prime time” refers to the weekday night hours that have  traditionally been popular with television viewers and from which network ratings are derived. A program in prime time must have broad appeal and be developed well enough for a good long run if it becomes popular. It doesn’t need to be popular yet, but it must be ready for mass market acceptance.

J Metz goes out of his way to argue that FCoE really is “ready for prime time”, refuting four “statements” attributed to critics (including me). But he appears to be using a different definition of that term, suggesting that Cisco’s product GA is sufficient. This is his opinion, but I don’t share it. I think it takes much more than a few initial products and deployments for any technology to be ready for prime time, especially in storage!

Multi-Hop Standards

My article poses the question, “why use a 10 Gb Ethernet standard that remains in flux when 16 Gb FC is shaping up nicely?” Looking back, I agree that the standards aren’t what’s in flux so much as the interpretation and implementation of them. Although the standards will evolve (and are already evolving), they are fixed and functional today.

The standards for DCB are done, and implementation and interoperability is looking good (with the exception of QCN, which is of questionable value). But multi-hop FCoE needs way more than DCB. FC-BB5 is the real standard for placing Fibre Channel over an Ethernet backbone, and that’s been accepted for a long time. Real scalable FCoE networks will also probably need a datacenter fabric, and Cisco’s TRILL-esque FabricPath technology is closest to some kind of standard for that.

But real, functioning end-to-end multi-hop FCoE networks need more than standards, they need consistent and predictable implementation, and that picture is a lot less clear. For every confident Metz, Miniman, or Onisick there’s a Ferro, Fratto, or Bourke who continue to question the implementation of, and need for, these standards.

Multi-Hop Implementation

I later call Fibre Channel Forwarding and Ethernet fabric technology “decidedly experimental.” Metz counters that Cisco has a functional implementation, and I do not doubt that. But one company’s recent GA status doesn’t make Multi-Hop FCoE “ready for prime time” by my standards. By his own admission, “multihop FCoE for Director-Class systems (the most common for Aggregation and Core deployments) has only been available for two months.”

I trust that Cisco and Metz have a working implementation, but not enough to go out telling enterprise storage administrators to take the plunge on multi-hop FCoE in general or even Cisco’s product in particular. Give it a little more time to mature, and give me a reference customer or two. And it would be nice to have more than one vendor to buy from.

FCoE Interoperability

I also state that Multi-Hop FCoE interoperability “is a serious question” and Metz points out that HP and Cisco have an interoperable solution. But his example is an FEX module for HP blade servers, not an FCoE-capable switch that interacts correctly with Cisco Nexus using standards-based Multi-Hop FCoE technology. It’s not even using FabricPath, let alone TRILL or some other FCoE standard!

This whole interoperability conversation reminds me of the “Fool the Guesser” scene in The Jerk: Your interoperability is right here, between the ashtrays and the thimbles. As long as you want to connect this very specific thing with that very specific thing, ignoring the rest of the world of products, you’re interoperable. And don’t ask for multi-vendor FC forwarding yet.

Metz also makes a non-sequitur suggestion that someone wants the industry to wait for Brocade, but I’ve never said anything of the sort. We don’t need everyone to proceed, but we do need more than one company. Standards only matter if they help us do something productive and positive, and single-vendor standards might as well not exist at all.

The bottom line is that we do not have interoperability of FCoE switches today, and we won’t have it for a long, long time. This is not Cisco’s fault, since they’re closest to standards-compliant, but it’s the truth. Eventually someone (Juniper? HP? Brocade? Dell?) will come out with a standards-compliant FCF/TRILL FCoE switch and they will issue a joint press release with Cisco and the industry will rejoice. But most customers probably won’t mix switch vendors anyway…

FCoE Adoption

Metz’ final point is to refute something I’ve often said: That “no one” is using FCoE today. Truly, I say this more for laughs and to provoke thoughtful questions than as a statement of fact. I know that lots of customers are using edge-only FCoE in critical production environments with Cisco UCS today. In fact, I consider edge-only FCoE to be a sound practice and do recommend it to buyers of high-end enterprise IT gear.

But edge-only FCoE adoption is a double-edged sword (if you pardon the pun) for proponents of convergence. It benefits customers with simplified client connectivity, delivering much of the benefit of convergence in an easy-to-adopt package. And it gets the protocol out there in production, offering a path to an Ethernet-based SAN future. But it might just short-circuit the value proposition for full end-to-end FCoE, blunting its impact and slowing the urgency for exactly the kind of customers who might adopt FCF switches.

Metz and I have often talked about real customer adoption, and he assures me that there are customers of “Full Monty FCoE” out there, but they’re not talking yet. After all, it’s only been available for two months.

Stephen’s Stance

I leave it to the reader (and the buyer) to decide if FCoE is ready for prime time.

• First, they have to define FCoE: Does edge-only count?
• Then they have to decide if they have a use case that some flavor of FCoE fits.
• Then there’s the real question of risk: Are you ready to take the plunge?

I know that a number of FCoE-related standards are settled, and I know that there are products in the market and even some limited multi-vendor compatibility. I even accept that some customers are deploying real “Full Monty FCoE” in production. But I just can’t recommend that technology yet: It’s not prudent, widespread, and low-risk, so I say it’s not ready for prime time.