Today, Brocade announced that it has completed its acquisition of Foundry Networks.  This is just the latest move in the strategic game to control the next generation of Ethernet, and possibly all local connectivity, including storage. Although 1 Gb Ethernet, 4 and 8 Gb Fibre Channel, and InfiniBand are all still going strong, the attention of the industry, the pundits, and the prognosticators (myself included) is firmly fixed on enhanced 10 Gb Ethernet. So Brocade’s move seems especially relevant to the core question of which companies will thrive and which will fail in a 10 Gb world.

Cisco is, as always, the big player everyone is gunning for. They created a subsidiary, Nuova, to develop a next-generation architecture, then absorbed the company in April. The result of Nuova’s work is Cisco’s Nexus line of data center switching products as well as much contribution to enhancements to Ethernet, known as Data Center Bridging (DCB) to the IEEE, Data Center Ethernet (DCE) to Cisco, and Converged Enhanced Ethernet (CEE) to others. The whole Nuova episode worked out quite well for Cisco, and they are well-positioned in the next-generation Ethernet game.

Although the fight to control the next generation of datacenter I/O can sometimes seem like a two-way battle between Cisco and Brocade, there are many players involved:

• Woven Systems launched its second-generation product in October. Aimed squarely at the largest data centers, the Woven line sells at bargain prices and expands incrementally, maintaining performance, according to the company.
• Force10 has also found success selling a complete line of 10 Gb Ethernet products, from the core data center to the edge.
• HP’s ProCurve line is mostly focused on smaller businesses, but the number-two supplier of networking hardware shouldn’t be discounted in this horse race!
• Arista Networks is another dark horse. With ex-Cisco “Data Center 3.0 commander” Jayshree Ullal and ex-Sun Andy Bechtolsheim, Arista has so far played its 10 Gb song to a cloud computing refrain. But the company reportedly has strong technology, and could be a contender in the converged I/O space as well.

This playing field is wide open. I expect Brocade and Cisco to be major players over the next few years, and that HP will acquire one of the other players at some point to add to their ProCurve line. But no matter who wins, the dominance of 10 Gb Ethernet is a safe bet.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Who Will Capture the 10 Gigabit Ethernet Crown?
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

This is part of an ongoing series of longer articles I will be posting every Sunday as part of an experiment in offering more in-depth content.

There has been a lot of discussion in the storage industry about Fibre Channel over Ethernet (FCoE), making it the toast of Storage Networking World, but this technology remains relatively unknown to end users. Like so many storage protocols before it, the $10,000 question is whether FCoE will take off like iSCSI or fizzle as a niche product like FCIP, DAFS, and so many others.

If it does succeed, another critical question is what this means for iSCSI, Fibre Channel, InfiniBand, and to a lesser extent AoE, expanded SAS, and other options for SAN storage. The enterprise data center is poised for a complete change in server connectivity, with 10 Gb Ethernet converged network adapters (CNAs) and new core switches carrying both network and storage traffic, and this holds promise, especially in virtualized environments. But CNAs do not equal FCoE, and iSCSI, conventional Fibre Channel, and other protocols are roaring ahead. What impact will FCoE really have?

Why FCoE Matters

With 8 Gb Fibre Channel and alternative storage solutions leveraging InfiniBand now available (and with 10 Gb iSCSI imminent), many would ask why we need another enterprise block storage medium. In real-world applications, FCoE at 10 Gb will likely deliver roughly the same performance as 8 Gb native FC. But FCoE will be one or two years late and (initially) more expensive. Although active standards participation and “plug fests” mean FCoE will likely be more interoperable than Fibre Channel was in its early years, the lack of support from operating system manufacturers is troubling. Plus, users will soon be able to build a very similar infrastructure by mixing iSCSI and 10 Gb Ethernet, and this will include all the advantages of IP and solid support.

So why pay more for the same performance from an untested protocol? It’s all about the future, and enterprise users will go where the market goes, just as wide availability of VHS tapes buried Betamax. Storage, network, and SAN vendors alike are lining up solidly behind FCoE as the next-generation enterprise interconnect. Although InfiniBand plays Betamax in this home video metaphor, with superior technology and availability, FCoE’s VHS camp has all the market ammunition. To paraphrase the (alleged) words of Bob Metcalfe, no matter what the technology looks like, the future of networking will be called Ethernet.

The biggest storage vendors are behind FCoE simply because they see that converging and leveraging I/O technology makes sense for them. They can swap out the physical and data link layers from Fibre Channel to Ethernet relatively easily, so the FCoE switch is an easier change than iSCSI. It is likely that they will be able to leverage commodity Ethernet hardware to reduce (their, not your) cost and increase profit margins once this switch is made. Plus, FCoE will potentially increase SAN attachment rates (and thus enterprise storage market penetration) thanks to the potential availability of converged network adapters (CNAs) on the server side, and the cost-effectiveness that sharing a CNA between network and storage implies. From the storage side, FCoE is all good.

Also see my posts on FCoE versus iSCSI and Cisco VFrame

The drive is similar on the network side. The era of differentiated SAN and LAN producers is over – all of the major networking and SAN vendors are repositioning themselves as next-generation I/O providers, setting up a battle in the network space to rival the mainframe shakeout of the 1980s and the PC wars of the 1990s. Converged I/O is the business model for connectivity vendors, and most are taking up the “data center Ethernet” (DCE) charge (also called “converged enhanced or enterprise Ethernet” or CEE) which includes FCoE as the storage protocol for virtual I/O. iSCSI is still there in a DCE world, but FCoE will take center stage for the enterprise market.

Counting the Benefits of FCoE

It may seem strange to declare an upstart like FCoE the winner when established options like InfiniBand, conventional Fibre Channel, and iSCSI are already out in the market, but this examination of the vendors indicates that it is indeed the case. Is this a case of the tail (vendors) wagging the dog (consumers)? Perhaps, but they will come along willingly given the strong case presented by converged and virtualized I/O.

Enterprise buyers are ready for a next-generation SAN technology, and some are beginning to look at 8 Gb Fibre Channel. The few that really need performance will certainly buy 8 Gb FC today, but this has little bearing on the overall prospect for FCoE. When an application requires performance and money is available, purchases will be made regardless of future strategy.

Enterprise storage and network architects are beginning to consider the implications of server consolidation and virtualization. As they see footprint shrink thanks to compact or blade servers and server virtualization, they will begin to question the proliferation of interconnects on the back end required to keep up with the I/O demands of these super servers. Already, virtual I/O purveyors like Xsigo are making hay in this market, and, as mentioned above, their SAN and LAN vendors are spreading the message, too. It won’t be long before they are convinced.

Also see my posts on VMware VDC-OS and VDC-OS vStorage

Many people mistakenly assume that DCE means pushing all protocols through a single LAN, but this is not the case. These networks will be engineered like SANs from the start, with redundant connections and transparent failover. Although storage and network connectivity will share the same physical “pipe”, they will certainly be segregated on separate VLANs and protected with quality of service technologies. They have to be separated – FCoE (lacking IP) will require a totally different network topology than LAN connections.

So Who Buys FCoE?

Note that, throughout this discussion, I am referring only to the large-scale enterprise data center storage market. Smaller corporate environments have already embraced iSCSI en masse, expanding the penetration of consolidated storage concepts beyond anything Fibre Channel could ever accomplish. And small office and home networks are beginning to embrace these concepts as well, but are relying on protocols like CIFS and AFP for file servers and may begin to look at ATA over Ethernet (AoE) and proprietary protocols like the one pushed by Zetera/NetGear instead of iSCSI.

This leaves us with a layer cake of appropriate protocols from the smallest to largest networks. But all have one thing in common: They are all converged and they are all carried in Ethernet packets. Bob Metcalfe was right!

Update: Storagebod points out that FCoE might see its first application in inter-switch links and other similar storage network infrastructure connections. And Chris Mellor at The Register points out that SAS is already displacing FC as an internal drive interconnect.

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Reality Check: The FCoE Forecast
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The pistol shot heard this week was the starting gun for FCoE, not the execution of iSCSI

To hear this week’s storage industry news reports, one might think that Wagner’s fat lady came to Storage Networking World (SNW), singing her song as the iSCSI world collapses. Storagebod wonders what iSCSI’s death will look like. Chris Mellor at The Register says “Game Over” as NetApp, QLogic, Emulex and VMware join EMC and Cisco in singing the praises of Fibre Channel over Ethernet (FCoE). Mellor suggests that the protocol will devalue Dell’s EqualLogic investment, as if HP’s acquisition of LeftHand wasn’t enough, even as fellow Register-ite, Bryan Betts disagrees.

But The Register didn’t invent the “FCoE kills iSCSI” meme – it’s just natural to imagine that these two protocols would be in a fight to the death. And if it’s a duel, then this year’s SNW conference would seem to be the first volley, as EMC introduced a FCoE Connectrix switch (based on Cisco), NetApp announced the first native FCoE array, and everyone qualified Emulex and QLogic adapters. However, despite these announcements, it’s way too early to bury iSCSI!

FCoE and iSCSI are similar in concept:

• Both rely on Ethernet physical connectivity
• Both transmit SCSI packets
• Both are aimed at date center users

But there are major differences as well:

• iSCSI is routable in an IP network
• iSCSI can use IP services like IPsec
• Software initiators can give iSCSI connectivity to any server, regardless of hardware
• FCoE will require converged network adapters (CNAs), while iSCSI can run on any Ethernet adapter
• FCoE will start at 10 Gb, while iSCSI can operate at just about any speed

Looking at this list, one might realize that FCoE is really a competitor for faster-than-4 Gb Fibre Channel. It’s not just a data center product, it’s an enterprise (read high-end and expensive) product, and that’s exactly where it will flourish. I have no doubt that Cisco and Brocade will successfully transition their Fibre Channel product lines to FCoE, and that QLogic and Emulex will sell a gazillion CNAs. But what about the rest of the market?

VMware’s adoption has shown that there is a taste for shared, networked storage outside the rarified budgets of the enterprise world. So far, no storage protocol has won the midrange and virtual server market, with Fibre Channel, iSCSI, and NFS duking it out along side internal SAS and SATA and the odd InfiniBand and external SAS solution. Although CNAs and FCoE ought to work fine in the virtual data center, not everyone will have a taste for them. There will always be plenty of folks who just want inexpensive external networked storage arrays, and iSCSI is the best thing they’re likely to see any time soon!

And iSCSI isn’t done evolving. Folks like Mellor, Chuck Hollis, and Storagebod are lauding FCoE at 10 gigabit speeds, but seem to forget that iSCSI can run at that speed, too. It can also run on the same CNAs and enterprise switches. And although wide(r)-area SANs aren’t common, I’m beginning to see some interest in leveraging the routability and other advanced features of IP in the storage world. iSCSI still has some cards to play! And the non-enterprise space isn’t nearly as awful as some make it sound – it is and will remain a bigger, more diverse market than the high end, and there are some serious buyers that will never get into FCoE.

Right now, the SAN world is expanding like it hasn’t done for years. iSCSI deployments are accelerating, growing the whole market. Sure, FCoE will probably completely replace old-school Fibre Channel over the next five years. But it will have to share the market with the now well-established iSCSI. It looks to me like Dell and HP made smart investments.

Update: More coverage on the topic:

• Doug Rainbolt from Alacritech is skeptical of the drivers for FCoE
• David Dale from NetApp feels that FCoE is unlikely to intrude on the iSCSI “sweet spot”

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Is the FCoE Starting Pistol Aimed at iSCSI?
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I am not the typical enterprise storage user. In fact, I am not an enterprise storage user at all – I am a consultant focused for over a decade on assisting enterprises with their storage architecture and strategy, working with businesses of all sizes. My background is both a blessing and a curse – I have seen far more enterprise storage environments in much more detail than most people, but I am unable to truly empathize with my corporate storage compatriots since it’s not really my gear and data that I am working with.

Based on this experience, what does the future hold? Where is enterprise storage heading? Read on for my thoughts.

Fibre Channel

Although the commoditizing storage market would seem ripe for stagnation, the opposite is happening. In fact, the enterprise storage market has continued to diversify, with iSCSI recently expanding the options for storage connectivity to a new market and FC seeing rapid uptake in the virtual server arena. Fibre Channel over Ethernet (FCoE), whether or not part of the datacenter Ethernet push, is joining virtualized I/O technology based on InfiniBand as the next wave in connectivity. Simply put, the market is not standing still.

Focus on the new world of storage area network (SAN) connectivity can take away from the basics, however. Traditional switched Fibre Channel (FC) remains the healthy market leader, and the recent upgrade to 8 Gb speed has attracted customer attention. Although they are unlikely to run out and perform a mass upgrade, 8 Gb FC is as as much of an inevitability as 4 and 2 Gb before it. The switch will be made, and customers will upgrade organically.

One of the big beneficiaries of 8 Gb FC will be those with virtual servers. This new hardware is coming just as N_Port ID virtualization (NPIV) is maturing, and this technology, which enables multiple servers to share a single FC interface, in combination with a doubling of bandwidth will make modern Fibre Channel tech extremely attractive to virtual server shops.

Even those who do not adopt 8 Gb FC connectivity in their SAN will benefit from the upshift in throughput, as native 8 Gb disks appear and enterprise arrays are redesigned to accept them, just as devices like the DMX-4 from EMC benefited from 4 Gb FC back ends.

IP Storage: iSCSI and NAS

As far as iSCSI goes, those of us who saw the promise of this technology can finally declare victory. Every storage device that matters offers iSCSI as a connectivity option, and most buyers are considering adopting it. It is being weighed fairly against FC, and the promise of reduced heterogeneity and cost are proving attractive to many. In fact, it is wrong to continually compare it against FC, since many iSCSI buyers would never adopt an FC SAN due to concerns about cost or learning curve. Indeed, much of the uptake in iSCSI comes from areas where SAN was never adopted, and iSCSI’s growth can be partly attributed to these happy customers spreading the technology wider than originally intended.

A hidden benefit of iSCSI adoption is the technologies and techniques that have come along with it. Clustering of smaller storage systems has become a common option for scalability, and has proven itself against old modular “head/shelf” arrays. Although the rate of adoption for security technologies like CHAP and IPsec in iSCSI remains low, they are far more common than their FC relatives. And Microsoft’s simplified and universal iSCSI drivers, which include multi-path and snapshot technologies, have been much more successful than their proprietary equivalents.

Let us not forget humble old network attached storage (NAS), either. File server consolidation to NAS filers continues to be a healthy (but less flashy) market, and NAS virtualization is on the rise as these devices proliferate. And some in the server virtualization community are beginning to consider NFS for their servers, especially when it comes to VMware. NAS definitely still has life and legs and will benefit from the shift to 10 Gb Ethernet just like iSCSI and FCoE.

The Future

As for the future, it is clear that most storage vendors are lining up behind Fibre Channel over Ethernet (FCoE) . Although true I/O virtualization, as envisioned by datacenter Ethernet (DCE) and InfiniBand, may not gain traction outside the largest data centers, FCoE seems to be the inevitable next generation for massive enterprise storage. Just about every vendor is committed to it, and the customers I have spoken to accept it as the future. Although 8 Gb FC might delay FCoE in some cases, it will almost certainly be the predominant SAN connectivity mechanism for large block storage devices within five years.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | A Consultant’s View Of The Enterprise Storage Market
This post was categorized as Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.