Iomega updated the 12-drive rack mount StorCenter device, introducing the px12-350r today

Iomega surprised exactly no one by announcing an updated 12-drive rack mount storage array today. Featuring “Cloud Edition” software introduced earlier this year, the px12-350r also sports mildly updated hardware specs, though still relies on Intel’s “Core2 Duo” CPUs and Gigabit Ethernet. The new device slots in between the desktop px4/px6 line and parent EMC’s new VNXe storage devices.

What’s In A Name?

According to Iomega’s established naming standard, the px12-350r uses the new “Cloud Edition” OS (“px”), features 12 drives, ought to perform better (“350″) than the existing “300″ devices, and is rack-mountable. Comparing it to the previous-generation ix12-300r reveals what Iomega changed and what they didn’t.

What’s new compared with the ix12-300r:

• Iomega stepped up from 5900 rpm “green” drives to faster 7200 rpm disks for more rotational IOPS
• 3 TB drives are now available, for a maximum of 36 TB raw storage, though many customers will likely choose less-expensive 2 TB drives instead
• Solid-state drives (SSD) are now available as an option like the desktop px line, though Iomega doesn’t get automatic sub-LUN tiered storage like EMC’s VNXe line
• The “Cloud Edition” OS includes Iomega’s nifty “personal cloud” replication features, allowing data to be moved to other Iomega px devices
• This new OS also supports SCSI3 persistent reservations, required to support many clustered server operating systems
• Iomega finally rolled out the Avamar deduplicating backup client I spotted last year at EMC World a few months ago, and it will be a featured component of the px12-350r
• On the hardware side, Iomega doubled the system memory to 4 GB, so it should handle more users than the previous device
• The px12-350r still uses Intel’s older Core2 Duo CPU range, though it’s not clear if this is the same 3 GHz E8400 chip
• The ix12-350r still features quad Gigabit Ethernet ports, not 10 Gigabit Ethernet like some competing products from companies like D-Link, HP, Overland, and Winchester

Impressions of the px12-350r

The software is all-new, but the px12 still relies on a Core2 Duo CPU and Gigabit Ethernet

Overall, I’m pleased but not overly amazed by Iomega’s new top-of-the-line device. This is partly due to the fact that I knew what was coming: This product was telegraphed by the px4-300r introduced previously. In fact, the shipping px12-350r doesn’t even offer the enhancements I expected: I suggested it would include Sandy Bridge CPUs and hoped for 10 Gigabit Ethernet.

I still expect a transition to next-generation Core CPUs in the coming year, and boxes like this will likely use that extra horsepower to run additional applications. I’m dreaming of a Sandy Bridge Xeon box with an integrated copy of VMware ESXi and 10 Gb Ethernet – such a device would be awesome for remote offices or small businesses!

Iomega’s “personal cloud” technology is perhaps improperly named for this product. It allows a small or mid-sized business to replicate and protect data to another Iomega px-series device without any monthly fees. The device can also back up to Mozy or Amazon S3, and the integrated Avamar client will be appreciated by larger customers.

This product, like the desktops and the previous-generation ix12-300r, includes PCI Express slots. I had hoped these might be populated with optional 10 Gigabit Ethernet cards (as many competitors offer) but these haven’t turned up yet. Iomega suggests smaller IT groups are still holding off on 10 GbE, perhaps waiting for 10GBASE-T. But 10 GbE pricing is dropping rapidly, and I’m seeing much interest from end users. I strongly suggest Iomega roll out this connectivity option in short order!

It’s interesting that VMware chose to develop their own software for the vSphere Storage Appliance (VSA) in vSphere 5 rather than use this Iomega/EMC LifeLine OS. LifeLine is possible as a VM: It will be available in the lab at VMworld! But this is not available for production.

Stephen’s Stance

Despite a lack of surprises, the px12-350r is still a solid SMB or remote office product. Retail price is $6k-$10k, slotting in below the EMC VNXe3100, and the device’s features reflect this product stratification as well. The consumer-sounding “personal cloud” data replication is actually much more useful in a small-business environment, and I suspect Iomega resellers will be pushing this (as long as they don’t sell some other replication option).

I expect further product updates from Iomega in the coming year, including a switch to Sandy Bridge (second-generation Intel Core) CPUs and the availability of 10 GbE. I’d also like to see the LifeLine software (and Iomega’s recommendations) abandon RAID5 in favor of some wide-striping “post-RAID” for performance and reliability. And integrated automated storage tiering would be great, though this would really show up the “big brother” VNXe line. Hey, a guy can dream right?

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Iomega Introduces the StorCenter px12-350r
This post was categorized as Enterprise storage, Everything, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

• Things I wrote
  • I kicked off my new Network Computing column with this one: Ethernet Has A Goldilocks Problem (and here’s a response: Network Convergence: More than Just 10 GbE)
  • TSIA! Is PCIe SSD right for you? Deploying PCI Express SSD devices
  • I haven’t written for Silicon Angle in a while, but thought this was best there: From Scale-Out to Big Data to the Cloud
  • Will You Be At The ExecEvent? I will, and it’s just two weeks away!
  • Sometimes you have an idea: The Three Requirements To Overcome Inertia
  • I’m Open-Sourcing My 2011 Storage For Virtual Environments Seminar! So now I Need Numbers!
  • Apple Breaks ICS Calendar Auto-Subscription In iOS 4.2
• Other great links
  • Derek Schauland asks the eternal datacenter question: Where can I plug in?
  • Brandon Carroll documents that Solarwinds Install is Easy as Pie!
  • I love Greg Ferro’s Network ZEN, but this week’s was especially awesome: Network ZEN: A Switch
  • Nigel Poulton puts together some amazing storage content. Case in pout: VMAX vs VSP
  • Seriously, Google? Google stripping support for H.264 video out of Chrome
  • The most-scary WordPress article I’ve read in a long time: Why You Should Never Search For Free WordPress Themes in Google or Anywhere Else
  • Howard Marks takes on a topic we discussed last time I was in New York: More on Advanced Erasure Codes
  • Top Five Storage Technologies for 2011 <- Good guess!
  • The only insanely great thing to come out of CES: Breakthrough device of CES 2011: Motorola Atrix = Phone + PC
  • I love words 20 Awesomely Untranslatable Words from Around the World

Subscribe to my Google Reader feed or follow me on Twitter to see these in real-time.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Back From the Pile: Interesting Links, January 14, 2011
This post was categorized as Enterprise storage, Everything, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Networking Basics

Let’s get the first two components of the network name out of the way:

The first part is the signaling rate in megabits per second. In layman’s terms, this is the speed of the network at hand. You are likely to come across one of the following:

• 10 megabits
• 100 megabits – Fast Ethernet
• 1000 megabits – Gigabit Ethernet, GbE, or 1000BASE-X
• 10,000 megabits – 10 Gigabit Ethernet, 10GbE, or 10GBASE-X
• 40,000 megabits – 40 Gigabit Ethernet, 40GbE, or 40GBASE-X
• 100,000 megabits – 100 Gigabit Ethernet, 100GbE, or 100GBASE-X

It may strike you as odd that the next part is always the word, “BASE”. But there is a reason for this, too. BASE refers “baseband”, meaning that this is an unfiltered line not requiring a digital modulation scheme. Back in the day, there was a 10PASS-TS version of Ethernet that used a signaling scheme similar to a modem, but baseband is dominant today.

So 100BASE refers to a Fast Ethernet connection that uses the unfiltered cable for transmission.

BASE-What?

The third part of an Ethernet network type refers to the cabling used to carry the signals. The earliest forms of Ethernet used coaxial cable, but thin twisted-pair cabling became popular in the mid-1990s. Faster versions of Ethernet also often use fiber optics rather than electrical signals.

There are a bewildering assortment of physical interconnects for Ethernet. But the naming system isn’t as complex as it might appear:

• The first letter tells us which kind of wire we are talking about:
  • “T” means twisted-pair cable (e.g. the common Cat5 in use today)
  • “K” means a copper backplane
  • “C” means balanced copper cable
  • “F” means optical cable
  • “B” uses two wavelengths over a single optical cable
  • “S” means short-range multi-mode optical cable (less than 100 m)
  • “L” means long-range single- or multi-mode optical cable (100 m to 10 km)
  • “E” means extended-range optical cable (10 km to 40 km)
  • “Z” means long-range single-mode cable at a higher wavelength
• Next is the coding scheme for data on the wire
  • “X” means 4B/5B block coding for Fast Ethernet or 8B/10B block coding for Gigabit Ethernet
  • “R” means 64B/66B block coding
• Finally, we have a number representing the number of parallel “lanes” for data
  • “1″ would mean serial (non-parallel) but is omitted instead
  • “4″ or “10″ are available for copper wire
  • Just about any other number could be used for optical lanes or wavelengths

Examples

Now let’s look at some examples:

• Back in the day, 10BASE-T became more common than coaxial 10BASE2. It was a simple 10 megabit baseband signal over common twisted-pair.
• When Fast Ethernet first rolled out, there was some concern that traditional (usually Cat3) cabling couldn’t handle 100 megabits. Some early implementations used four copper pairs (100BASE-T4) or fiber optics (100BASE-FX), but nearly every 100 megabit Ethernet connection today is 100BASE-TX, using plain two pairs on plain Cat5 cable.
• Gigabit Ethernet had a similar history. Many were concerned that two pairs on unshielded Cat5 wiring could not handle 1000 megabits per second, so optical (1000BASE-SX) and balanced shielded wiring (1000BASE-CX) were specified. Although an unshielded 2-pair standard was developed (1000BASE-TX), it never really caught on. Therefore, today’s predominant gigabit LAN connection, 1000BASE-T, uses all four pairs of unshielded twister-pair wiring on a Cat5 cable (see note 1).
• The 10 Gigabit Ethernet world has mostly shifted to the block coding scheme from Fibre Channel, 64B/66B, which is denoted by the letter “R”. This gives us a family of fiber optic cables (10GBASE-SR, LR, ER, etc), and a copper backplane interconnect (10GBASE-KR). The earlier copper wiring standard (10GBASE-CX4) used InfiniBand-like 4-lane cables and 8B/10B signaling, as did 10GBASE-KX4 on the backplane. A backwards-compatible twisted-pair 10GBASE-T has also been developed, but work continues to make it power-efficient enough to be practical (see note 2).
• Looking ahead, we see Higher-Speed Ethernet emerging: 40GBASE-KR4 for backplane use, multi-mode optical 40GBASE-SR4 and 100GBASE-SR10, and long-range single-mode optical 40GBASE-LR4 and 100GBASE-LR10.

As you can see, all this alphabet soup does have some consistency. Common unshielded twisted pair wiring is all “BASE-T”, optics are denoted according to their range (“S”, “L”, “E”), and backplanes use “K” copper. Clear as mud?

Note 1: Lots of people (and even equipment makers) incorrectly refer to common Gigabit Ethernet as “1000BASE-TX”, but this really should be called “1000BASE-T”.

Note 2: We will probably never see a 10GBASE-TX, which would use just 2 pairs of unshielded twisted pair copper wiring.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | 1000Base-What?
This post was categorized as Computer History, Enterprise storage, Gestalt IT, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

What iSCSI Did

Let’s review the test for a moment. In March, Microsoft and Intel demonstrated that the combination of Windows Server 2008 R2 and the Xeon 5500 could saturate a 10 Gb Ethernet link, pushing iSCSI throughput to wire speed. That’s 1,174 MB/s, right around the theoretical maximum of a ten-gigabit link, given a tiny bit of overhead. The pair reunited in January to show that this same combination could deliver an astonishing million I/O operations per second, too.

Both of these results are astonishing. Sure, many high-end Fibre Channel SANs and storage systems blast out gigabytes of data and millions of I/O operations every second, but these tests are much more focused. Benchmarks are perilous, but the folks at Microsoft and Intel devised a fairly clever and focused set. Rather than a “mine’s bigger” contest, the pair only needs to prove that iSCSI can play with the pros.

The side effect is a demonstration of the capabilities of Microsoft and Intel components. Microsoft showed off the capabilities of Windows Server 2008 R2, Hyper-V, and their software iSCSI initiator, while Intel can brag about the Xeon 5500 server platform and X520-2 10 Gb Ethernet Server Adapter with their 82599EB controller. Your mileage may vary, but it is possible to construct a true storage monster on an average server budget.

Intel Inside

Let’s start by looking at the configuration of the local end of the tested configuration. I’m a storage guy so I think of it as the initiator, but you might say it’s the server, the client, or the host. Regardless, the system under test (SUT) is what was put under the microscope. The configuration was a common one: A high-end computer packing an Intel Xeon CPU and 82599-based 10 Gb Ethernet adapter. Most data centers have a machine or two just like this one.

Looking closely, we see that the test in question relied on the following key components:

• Intel’s “Shady Cove” S5520SC workstation-class motherboard
• The Intel Xeon W5580 CPU (4 cores, 8 MB cache, 3.20 GHz)
• 24 GB of DDR3 RAM
• Intel “Niantic” 82599EB 10 Gb Ethernet controller
• Microsoft Windows Server 2008 R2 x64

This combination would set you back about $7,500 – $450 for the motherboard, $1,500 for the CPU, 6 2 GB DDR3 SDRAM modules at $80 each, $1,200 for the Intel X520 NIC, and $4,000 for an Enterprise copy of Windows Server 2008 R2. Not cheap, but not an exotic server either.

Initiate and Optimize

The secret to push the tested system to perform like it did is in the optimizations in the server platform, the NIC, and Windows Server itself.

• The Xeon 5500 processor series includes many enhancements:
  • An integrated memory controller allows for faster RAM access
  • QuickPath interconnect (QPI) replaces the old front-side bus and enhances I/O off the core
  • A new I/O subsystem with PCIe integrated into the CPU
  • MSI-X expands the number of interrupts a PCI device can use
  • New instructions for on-board CRC-32C decoding, speeding up iSCSI digest processing
• The 82599 Ethernet controller also includes enhanced capabilities:
  • VMDq maps I/O queues to multiple cores and virtual machines, reducing I/O bottlenecks
  • Offload of TCP segmentation and receive-side coalescing
  • Interestingly, it does not appear that VMDc/SR-IOV was employed in the test
• Microsoft Windows Server 2008 R2 and Hyper-V are ready to use all of these features and more:
  • R2 uses multi-core CPUs more effectively in general
  • Receive-side scaling (RSS) spreads the I/O workload across all four Xeon cores
  • The iSCSI initiator now allows CRC digest offload (using the new Xeon command set)
  • Numerous “NUMA I/O” optimizations in the initiator
  • TCP/IP Nagle can be disabled in the registry
  • Hyper-V VMQ allows the network packets to be copied directly into the guest virtual machine’s memory

Whew! Put all of these optimizations in a blender and Hyper-V virtual machine iSCSI access will be twice as fast as before. No kidding!

Stay On Target

But we knew all of this back in January. We also saw that a Cisco Nexus 5020 switch was used to fan out to 10 software iSCSI targets. But until now there was no mention of what targets were used exactly.

The final footnotes in Intel’s whitepaper reveals that the storage backing the million IOPS test was none other than StarWind Software‘s iSCSI SAN! It is unclear what led Microsoft and Intel to use this particular iSCSI target (the earlier throughput tests ran on NetApp filers), but it does speak to the quality of this product.

It is not clear how many disk drives were used, but I would guess that SSDs or ramdisks might have been employed to pull a million IOPS. Network optimizations are also not mentioned, though jumbo frames would not be a benefit in an IOPS test.

StarWind’s software runs on Microsoft Windows and creates a full-featured iSCSI target, complete with data mirroring, automatic failover and failback, replication, snapshots, and thin provisioning. The company prices their iSCSI SAN at $6,000 for two nodes and competes with the likes of DataCore and Open-E. But the StarWind solution seems at a glance to be more full-featured than these other offerings.

Try It Yourself!

I imagine many folks like me might be tempted to try to reproduce these results. More valuable would be a set of best practice guidelines for the deployment of software iSCSI in Windows Server 2008 R2 and Hyper-V environments. Given the relatively modest hardware involved, there should be nothing stopping us!

These test results also prompted me to get in touch with StarWind to try their iSCSI target software. I was pleasantly surprised to learn that they are currently offering free non-production licenses to VMware vExperts, VCPs, and VCIs as well as Microsoft MVPs, MCPs, and MCT Professionals. Many of my readers fall into one (or more) of those buckets, and I applaud the company for this offer. If only more companies realized the value in giving away test licenses to influencers and thought leaders!

© sfoskett for Stephen Foskett, Pack Rat, 2010. | How Did Microsoft and Intel Get 1 Million iSCSI IOPS?
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

How fast can iSCSI get?

“Maximizing Hyper-V iSCSI Performance with Microsoft and Intel” might sound like another “blah blah” marketing piece, but a little birdy tells me that this webcast will drop a bombshell about iSCSI performance.

Lots of storage and networking folks don’t give iSCSI and Microsoft the credit they deserve. “iSCSI is cheap and easy,” they say, “but real performance requires Fibre Channel.” Those of us who have an open mind about such things know that this is simply not the case. The fastest SAN I ever saw was based on iSCSI, and Microsoft demonstrated wire-speed iSCSI over 10 Gb Ethernet in March. I never saw a Fibre Channel SAN (even an 8 Gb/s one) push over a gigabyte per second over a single link!

Still, ask the average sysadmin and they will tell you that iSCSI isn’t for high performance applications. That’s why folks should tune in to this webcast, as Microsoft and Intel knock down another iSCSI performance myth. Note that even though Hyper-V is called out in the title and description, this discussion is really about Windows Server 2008 R2 and applies equally regardless of whether or not you use Microsoft’s hypervisor.

Watch this space for a summary of the news immediately following the announcement.

• What: Maximizing Hyper-V iSCSI Performance with Microsoft and Intel webcast
• When: Thursday, January 14, 2010 8:00 AM Pacific Time
• Where: MSEvents.Microsoft.com
• Who: Anyone interested in high-performance storage and server I/O

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Microsoft and Intel Pushing iSCSI Performance Limits
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Sure, prices remain high, but they’re falling fast and many folks are already implementing 10 GbE in their data centers. Intel CNAs have dropped to around $900, Cisco Nexus switches are running around $1,500 per 10 GbE port and Arista has hit $400, and 10 GbE iSCSI storage arrays are available. Intel has declared that 2009 is the year of iSCSI, and server vendors are moving to make that happen, embedding 10 GbE ports on motherboards. If you’re looking to go cheap, check out D-Link’s 10 GbE iSCSI xStack array, which sells for under $13,000 and can be populated with 15 of your favorite drives.

So is 10 GbE worth the upgrade? Absolutely! Microsoft reported wire speed iSCSI to a NetApp array at WinHEC (see my writeup on wire-speed 10 GbE iSCSI on Gestalt IT). How would you like to push over a gigabyte per second with non-crazy-or-high-end hardware? iSCSI over 10 GbE works great, and Microsoft and NetApp, at least, have demonstrated that they can make use of all that bandwidth.

Then there is Fibre Channel over Ethernet. Cisco has been pushing that technology hard as a unification of datacenter interconnectivity, and Microsoft announced that they are putting together a logo test for Fibre Channel over Ethernet FCoE hardware. Storage array makers are hopping onboard, too, and angling for position in this new market. It’s not just the year of 10 GbE; 10-gig is the future of connectivity, and the future is here.

I’m moving lots of my enterprise storage and networking content to Gestalt IT so I can focus on lower-end topics here. If you want to catch my latest work on those topics, it’s worth subscribing!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | 10 GbE, iSCSI, FCoE, Microsoft, and the Future
This post was categorized as Enterprise storage, Gestalt IT. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Microsoft delivered a solid MVP Summit

Note: Every one of these items has previously been discussed by Microsoft. I can’t yet talk about any other storage features I might have seen…

1. iSCSI Quick Connect – No more fumbling through tabs and popups, just enter an IP address and you’re connected! Yeah, baby!
2. Wire-speed 10 Gb iSCSI – The software iSCSI initiator can saturate a 10 Gb Ethernet link to a NetApp FAS3070; 1,174 megabytes per second
3. BranchCache – Accelerated access for workers in remote offices; it can use peer-to-peer sharing of hashed chunks if a Windows server isn’t available to cache data
4. VSS LUN Resync – The awesome VSS API is extended with the ability to resync a LUN from a snapshot; supports both software and native array capability
5. VSS Express Writer – Now there’s no excuse for applications to not be VSS backup-compatible
6. PowerShell – The very-sweet command line and scripting environment works with many server areas, including some parts of the storage world. Please implement it everywhere, Microsoft!
7. Transparent Caching – Administrators can allow Windows 7 clients to transparently cache file content from SMB servers when links are slow
8. Background file sync – Windows 7 will synchronize offline files in the background
9. Native VHD Support – VDS now supports a virtual drive just like any other – create, expand, compact, etc… And Server Backup stores its data in VHDs too!
10. Windows Home Server Drive Extender – WHS has file-level data protection and flexible disk management – it reminded me of DROBO! Not new, but I hadn’t ever heard about it… Thanks to Donavon and Alex for the info!

“Hate Microsoft” is a popular meme, but it’s misplaced. The Windows Server team, and the storage folks in particular, keep moving the ball forward, and the whole world of IT infrastructure benefits. My laptop may have a fruit on the lid, and my focus may be on three-letter storage and server companies, but Microsoft is a key part of our world, too.

The MVP Summit wasn’t just a great learning opportunity – it was a blast! We’ll see how much of that leaks out, but there might be an incriminating YouTube video of yours truly on stage at the Experience Music Project, Donavon tagging Steve Ballmer for WHS, and a lot of MVPs letting their PC love fly for the camera. There was also a Twitter tribute to Frank McCallister an MVP who passed away shortly before the Summit.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | 10 Cool Storage Features From the 2009 Microsoft MVP Summit
This post was categorized as Enterprise storage, Everything, Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Today, Brocade announced that it has completed its acquisition of Foundry Networks.  This is just the latest move in the strategic game to control the next generation of Ethernet, and possibly all local connectivity, including storage. Although 1 Gb Ethernet, 4 and 8 Gb Fibre Channel, and InfiniBand are all still going strong, the attention of the industry, the pundits, and the prognosticators (myself included) is firmly fixed on enhanced 10 Gb Ethernet. So Brocade’s move seems especially relevant to the core question of which companies will thrive and which will fail in a 10 Gb world.

Cisco is, as always, the big player everyone is gunning for. They created a subsidiary, Nuova, to develop a next-generation architecture, then absorbed the company in April. The result of Nuova’s work is Cisco’s Nexus line of data center switching products as well as much contribution to enhancements to Ethernet, known as Data Center Bridging (DCB) to the IEEE, Data Center Ethernet (DCE) to Cisco, and Converged Enhanced Ethernet (CEE) to others. The whole Nuova episode worked out quite well for Cisco, and they are well-positioned in the next-generation Ethernet game.

Although the fight to control the next generation of datacenter I/O can sometimes seem like a two-way battle between Cisco and Brocade, there are many players involved:

• Woven Systems launched its second-generation product in October. Aimed squarely at the largest data centers, the Woven line sells at bargain prices and expands incrementally, maintaining performance, according to the company.
• Force10 has also found success selling a complete line of 10 Gb Ethernet products, from the core data center to the edge.
• HP’s ProCurve line is mostly focused on smaller businesses, but the number-two supplier of networking hardware shouldn’t be discounted in this horse race!
• Arista Networks is another dark horse. With ex-Cisco “Data Center 3.0 commander” Jayshree Ullal and ex-Sun Andy Bechtolsheim, Arista has so far played its 10 Gb song to a cloud computing refrain. But the company reportedly has strong technology, and could be a contender in the converged I/O space as well.

This playing field is wide open. I expect Brocade and Cisco to be major players over the next few years, and that HP will acquire one of the other players at some point to add to their ProCurve line. But no matter who wins, the dominance of 10 Gb Ethernet is a safe bet.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Who Will Capture the 10 Gigabit Ethernet Crown?
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

But when will 10 Gb and FCoE arrive? Dave Raffo and I discussed the topic after Storage Decisions, and he just wrote an article on the topic, 10GigE still seeking killer app. Here are some points that came up in our discussion:

• Shipping and supported 10 Gb HBAs and CNAs are the key requirement, and it looks like the schedules of Microsoft, VMware, Sun, and Linus Torvalds will determine when the floodgates open
• iSCSI people are talking 10 Gb, too, and they might end up adopting it first with software initiators
• Interest in FCoE is focused at the largest enterprise shops, and I’m seeing a distinct line between “iSCSI shops” and “(future) FCoE shops” with very little overlap (as I previously noted, iSCSI and FCoE aren’t mortal enemies, and FCoE will rule in the largest environments)
• No one is talking about 8 Gb Fibre Channel – they have all decided that 10 Gb FCoE or iSCSI is the next step for block storage
• InfiniBand has its believers (and they are rabid fans!), but the users I talk to are, as a rule, heading toward FCoE rather than IB for their future connectivity
• There is a tiny bit of user interest in moving back to SAS-enabled DAS for virtual server environments
• Everyone I talk to is shocked there isn’t a 10 Gb iSCSI array on the market yet, and we all expect to see this before FCoE
• Although some vendors have announced FCoE products, they’re not here yet – and it will be another year still before we see production deployment

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Storage Folks Are Talking 10-Gig and FCoE
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

This is part of an ongoing series of longer articles I will be posting every Sunday as part of an experiment in offering more in-depth content.

There has been a lot of discussion in the storage industry about Fibre Channel over Ethernet (FCoE), making it the toast of Storage Networking World, but this technology remains relatively unknown to end users. Like so many storage protocols before it, the $10,000 question is whether FCoE will take off like iSCSI or fizzle as a niche product like FCIP, DAFS, and so many others.

If it does succeed, another critical question is what this means for iSCSI, Fibre Channel, InfiniBand, and to a lesser extent AoE, expanded SAS, and other options for SAN storage. The enterprise data center is poised for a complete change in server connectivity, with 10 Gb Ethernet converged network adapters (CNAs) and new core switches carrying both network and storage traffic, and this holds promise, especially in virtualized environments. But CNAs do not equal FCoE, and iSCSI, conventional Fibre Channel, and other protocols are roaring ahead. What impact will FCoE really have?

Why FCoE Matters

With 8 Gb Fibre Channel and alternative storage solutions leveraging InfiniBand now available (and with 10 Gb iSCSI imminent), many would ask why we need another enterprise block storage medium. In real-world applications, FCoE at 10 Gb will likely deliver roughly the same performance as 8 Gb native FC. But FCoE will be one or two years late and (initially) more expensive. Although active standards participation and “plug fests” mean FCoE will likely be more interoperable than Fibre Channel was in its early years, the lack of support from operating system manufacturers is troubling. Plus, users will soon be able to build a very similar infrastructure by mixing iSCSI and 10 Gb Ethernet, and this will include all the advantages of IP and solid support.

So why pay more for the same performance from an untested protocol? It’s all about the future, and enterprise users will go where the market goes, just as wide availability of VHS tapes buried Betamax. Storage, network, and SAN vendors alike are lining up solidly behind FCoE as the next-generation enterprise interconnect. Although InfiniBand plays Betamax in this home video metaphor, with superior technology and availability, FCoE’s VHS camp has all the market ammunition. To paraphrase the (alleged) words of Bob Metcalfe, no matter what the technology looks like, the future of networking will be called Ethernet.

The biggest storage vendors are behind FCoE simply because they see that converging and leveraging I/O technology makes sense for them. They can swap out the physical and data link layers from Fibre Channel to Ethernet relatively easily, so the FCoE switch is an easier change than iSCSI. It is likely that they will be able to leverage commodity Ethernet hardware to reduce (their, not your) cost and increase profit margins once this switch is made. Plus, FCoE will potentially increase SAN attachment rates (and thus enterprise storage market penetration) thanks to the potential availability of converged network adapters (CNAs) on the server side, and the cost-effectiveness that sharing a CNA between network and storage implies. From the storage side, FCoE is all good.

Also see my posts on FCoE versus iSCSI and Cisco VFrame

The drive is similar on the network side. The era of differentiated SAN and LAN producers is over – all of the major networking and SAN vendors are repositioning themselves as next-generation I/O providers, setting up a battle in the network space to rival the mainframe shakeout of the 1980s and the PC wars of the 1990s. Converged I/O is the business model for connectivity vendors, and most are taking up the “data center Ethernet” (DCE) charge (also called “converged enhanced or enterprise Ethernet” or CEE) which includes FCoE as the storage protocol for virtual I/O. iSCSI is still there in a DCE world, but FCoE will take center stage for the enterprise market.

Counting the Benefits of FCoE

It may seem strange to declare an upstart like FCoE the winner when established options like InfiniBand, conventional Fibre Channel, and iSCSI are already out in the market, but this examination of the vendors indicates that it is indeed the case. Is this a case of the tail (vendors) wagging the dog (consumers)? Perhaps, but they will come along willingly given the strong case presented by converged and virtualized I/O.

Enterprise buyers are ready for a next-generation SAN technology, and some are beginning to look at 8 Gb Fibre Channel. The few that really need performance will certainly buy 8 Gb FC today, but this has little bearing on the overall prospect for FCoE. When an application requires performance and money is available, purchases will be made regardless of future strategy.

Enterprise storage and network architects are beginning to consider the implications of server consolidation and virtualization. As they see footprint shrink thanks to compact or blade servers and server virtualization, they will begin to question the proliferation of interconnects on the back end required to keep up with the I/O demands of these super servers. Already, virtual I/O purveyors like Xsigo are making hay in this market, and, as mentioned above, their SAN and LAN vendors are spreading the message, too. It won’t be long before they are convinced.

Also see my posts on VMware VDC-OS and VDC-OS vStorage

Many people mistakenly assume that DCE means pushing all protocols through a single LAN, but this is not the case. These networks will be engineered like SANs from the start, with redundant connections and transparent failover. Although storage and network connectivity will share the same physical “pipe”, they will certainly be segregated on separate VLANs and protected with quality of service technologies. They have to be separated – FCoE (lacking IP) will require a totally different network topology than LAN connections.

So Who Buys FCoE?

Note that, throughout this discussion, I am referring only to the large-scale enterprise data center storage market. Smaller corporate environments have already embraced iSCSI en masse, expanding the penetration of consolidated storage concepts beyond anything Fibre Channel could ever accomplish. And small office and home networks are beginning to embrace these concepts as well, but are relying on protocols like CIFS and AFP for file servers and may begin to look at ATA over Ethernet (AoE) and proprietary protocols like the one pushed by Zetera/NetGear instead of iSCSI.

This leaves us with a layer cake of appropriate protocols from the smallest to largest networks. But all have one thing in common: They are all converged and they are all carried in Ethernet packets. Bob Metcalfe was right!

Update: Storagebod points out that FCoE might see its first application in inter-switch links and other similar storage network infrastructure connections. And Chris Mellor at The Register points out that SAS is already displacing FC as an internal drive interconnect.

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Reality Check: The FCoE Forecast
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.