EMC VFCache is a Simple Read Cache

I was not pre-briefed on this product, and I’m not all that thrilled at the prospect of attending a launch webinar, so what you read here is based on my own research and reading of the available information as of this morning.

When EMC announced Project Lightning last year, company insiders expressed surprise to me. It seems that many had never heard of the project, and those that had didn’t think it was far enough along to be announced. I didn’t even bother to write about the Project Lightning announcement at the time. But today EMC unveils the production product that came out of Project Lightning.

VFCache is a filter driver that caches writes

EMC VFCache appears to be a simple and straightforward offering:

1. A PCIe SSD from Micron or LSI sits in the server and acts as a read cache to accelerate performance
2. EMC software also runs on the server, snooping on I/O and filling the cache with relevant data

There’s not a lot more to the product than that. EMC will sell the PCIe SSD and bundled software as VFCache, and will no doubt market the heck out of this product. Perhaps the only novel twist is the so-called “split-card” mode, which allows the card to act as a write cache. But EMC only supports this for transient “throwaway” data with direct attached storage (DAS) as a backend. There’s no way a conservative, enterprise focused company like EMC would risk sanctioning a writeback cache with no redundancy or data protection features.

VFCache uses a filter driver installed in the VM guest

Perhaps the biggest limitation of the initial VFCache offering is its limited applicability to enterprise server virtualization environments. VFCache uses a filter driver installed in each VM guest, and includes no hypervisor drivers though there is a vCenter plug-in. This makes VMware vMotion very tricky, involving scripting to remove and re-add storage. This means VMware SRM will not easily work, and there is no support for clustering, either.

This is no surprise, since VFCache appears to the host as a local storage volume (AKA, a disk drive or LUN) which would disappear if a virtual machine is moved to another server. Virsto solved this problem by virtualizing storage presentation to the hypervisor, and Fusion-io’s ioTurbine software does not interfere with vMotion either. EMC will likely go in this direction in the future, but it’s a big hole in the product for now.

You might also like reading Micron Bursts Into the PCIe SSD Market to learn more about the card EMC is using

The News: EMC Is in the Host-Based Storage Business

The primary use case for this product is server I/O acceleration. This is desperately needed, as applications and servers are rapidly outrunning the capabilities of conventional storage arrays. EMC and other legacy array manufacturers initially tried to address this I/O imbalance with tiered storage and in array caching. Indeed, these technologies are fairly effective at accelerating the performance of conventional disk storage arrays.

But flash manufacturers like Fusion-io (not to mention Micron and LSI) absolutely demolished storage array performance with their in-server offerings. EMC faced the prospect of losing out on the high-performance storage market. EMC simply could not allow their bread-and-butter enterprise customers to look elsewhere for strategic, high-performance storage for high-profile applications.

VFCache gives EMC salespeople a silver bullet when customers demand maximum performance, but this launch may not spell doom for the flash startups. For one thing, it legitimizes host-based flash cards as a viable component of enterprise storage architectures. It also opens the door to comparison between SAN storage and non–SAN alternatives that go well beyond what EMC is currently offering.

Shared Flash Storage Is on Deck: Project Thunder

"Project Thunder" will externalize the PCIe flash cards over a high-performance "Server Area Network"

As part of the VFCache introduction, EMC is also talking about Project Thunder, a shared version of VFCache. At the very least, thunder will allow multiple servers to access a shared pool of flash cache. This should allow VMware vMotion and DRS to function, and could be much more than that.

EMC could build a high-availability, high-performance all-flash storage array that may even use InfiniBand as an interconnect. The new Nimbus Data E-Class storage array matches this description perfectly, and their CEO tells me that performance over InfiniBand is indeed comparable to in-server PCIe flash cards. It seems logical for EMC to enter this market, if only to disrupt the momentum of Fusion-io and the rest of the all-flash storage upstarts.

Read more about the Nimbus E-Class: The First Big, Redundant, All-Flash Enterprise Array

The only fly in the ointment here is the recent consolidation of the InfiniBand market. Mellanox bought Voltaire, and QLogic sold out to Intel, putting that protocol on tenuous grounds. Perhaps 40 or 100 Gb Ethernet will emerge as a viable alternative for high-performance connectivity, or perhaps these products will retrench on shared PCI Express instead. Micron recently purchased Virtensys for just such a product, and Xsigo has been making big waves in the area of converged I/O as well. The market clearly need something better than Fibre Channel for maximum performance storage, even if InfiniBand isn’t it.

Stephen’s Stance

EMC VFCache (née Project Lightning) is a fairly simple offering: A server-based PCIe flash card that acts as a read cache with no integration with storage arrays or hypervisors. But EMC’s entrance into the host-based flash storage market is a powerful demonstration of the wave of disruption caused by flash-based storage and high-performance computing. Although I am not all that impressed with the product itself, I would be distressed if EMC had not introduced it.

VFCache illustrations are copyright EMC Corporation and are used here with permission

More solid, independent VFCache coverage:

• EMC Enters The Market With “Me Too” Flash Products (Chris Evans)
• Cache Splash and Complex is the new Simple (Martin Glassborow)

© sfoskett for Stephen Foskett, Pack Rat, 2012. | EMC VFCache (aka “Project Lightning”) Is One Small Step, But an Important One
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Today’s announcement of the E-Class storage array is an important milestone for Nimbus Data and solid-state storage in the enterprise. Until now, most solid-state storage arrays have been fairly small-scale, focused on point performance rather than enterprise-wide capacity. But the E-Class, which scales to 500 TB and sports a redundant, multi-protocol interface, is the first all-flash array to go toe to toe at the top of the market.

The State of Solid

No one would deny that solid-state storage is making a huge impact on the market. With mind-bending performance and reduced power requirements, flash memory matches up nicely with modern data center requirements. But the one missing element has always been capacity: NAND flash is more expensive than magnetic disks on a gigabyte by gigabyte basis.

Check out my 4 Horsemen series for more on the issues of disk!

The solid-state enterprise storage market started with point products targeted at specific needs within the data center. Companies like Texas Memory Systems and Violin have long supported the most challenging database applications with their external arrays, while Fusion-io, Virident, Micron and others stashed flash within the server. These companies were able to sell expensive storage into performance-hungry niches, but have found it difficult to address the capacity needs of the broader storage market.

Technology has the answer to this challenge, as demonstrated by Pure Storage, Nimbus, SolidFire, and others. Thin provisioning makes up much of the difference in cost, and deduplication or compression can even bring parity on a per-capacity cost basis. And even with these features turned on, solid-state storage arrays absolutely murder spinning disks in terms of random I/O performance.

Accumulating Nimbus

Unlike Whiptail, Pure Storage, Kaminario, SolidFire, and the rest of the startup crowd, Nimbus Data is not a new company. Founded by former TrueSAN wunderkind, Thomas Isakovich, Nimbus began as a disk storage startup before transitioning to an all-flash lineup two years ago. The company has steadily improved its product line over the years, adding NFS and SMB for a unified storage experience as well as InfiniBand for extreme performance.

Unlike most other companies in the space, Nimbus builds their own flash memory modules from raw NAND. This allows the company to avoid some of the tricky engineering required to qualify and adapt to the peculiarities of existing consumer or enterprise SSD modules. It also gives the company greater control and better flexibility to launch new capacity points when they are ready, rather than when their suppliers give the go-ahead.

Nimbus is rolling out a dual-PCB SSD module which doubles performance and capacity

The E-Class includes a new dual-PCB module which stripes data internally for better performance and capacity. This bumps each Nimbus E2000X enclosure to 20 TB, twice the capacity previously achieved, in just two rack units. And each of these enclosures draws as little as 100 W, allowing them to be stacked tall without exceeding the power capacity of typical datacenters.

E Is for Enterprise

The real innovation in the Nimbus E-Class is a brand-new “dual active” redundant controller architecture. Most previous solid-state arrays had a single controller, requiring users to mirror two entire arrays for high-availability. In contrast, most enterprise storage systems feature multiple active controllers with no single point of failure.

Read Multipath: Active/Passive, Dual Active, and Active/Active to understand what I’m on about here!

The E-Class introduces a new controller architecture for Nimbus. Each controller services all access to a LUN or volume until a failure is detected, in which case the alternate controller immediately comes online. But both controllers can have active storage at once, in what I call a “dual active” scenario. Although not truly “active/active”, the E-Class is in a different league from older single controller arrays.

Interestingly, the performance of Nimbus’ solid-state enclosures is high enough that the controllers do not need to mirror internal cache or hash tables. They simply write them out to SSD to be picked up in the event of a failure. This simplifies engineering for a dual controller system, and may lead to additional controllers added in the future.

Straightforward Pricing

Solid-state storage provides much more performance per dollar than spinning disk, but most customers still pay on a raw capacity basis. Rather than rocking the boat with alternative pricing models, Nimbus sticks to a straightforward method: $25,000 per controller, plus $100,000 per 10 TB enclosure. A minimum E-Class configuration includes two controllers and one enclosure for $150,000 with no extra cost for software licensing or features.

This seems fairly expensive for 10 TB of storage, but is actually quite competitive even with disk-based storage systems in the high-end, high-feature enterprise market. Thin provisioning increases the usable capacity of the E-Class, and the all SSD architecture means performance will not suffer. Unlike PCIe solutions, the E-Class is a shared, networked device and can be used by many servers at once.

Features and More Features

Speaking of features, Nimbus includes just about anything you could ask for in a storage array:

• In-line thin provisioning and deduplication
• Snapshots and synchronous or asynchronous replication
• 10 Gb NFS (2, 3, and 4) as well as SMB (CIFS/SMB1 and, SMB2)
• 10 Gb iSCSI
• 8 Gb Fibre Channel
• 40 Gb QDR InfiniBand

Although it has not yet been announced, Nimbus has added VMware VAAI support to the HALO operating system found in the S- and E-Class arrays. The company will support the major block storage components for vSphere 5: Block zeroing, atomic test and set, and full copy. Tom told me that Nimbus found the T10 interfaces fairly straightforward to implement but are still working on the NFS primitives. Although Nimbus does not yet have a vCenter plug-in, I expect that one is in the works.

Tom also tells me Nimbus is a “hypervisor hugger“, in that they intend to support features there rather than try to add them to the array. This is a smart choice for a smaller company, and I am glad to see Nimbus embracing the server virtualization market. I imagine an array like the E-Class would totally demolish any competing disk-based array in a virtual infrastructure deployment!

Stephen’s Stance

Nimbus has always been an interesting company, with a longer history in the storage world than most startups. Their switch to all-flash architecture was perfectly timed with the market shift, and the new E-Class comes just at the right moment. Boasting 500 TB of maximum capacity, a fully redundant “dual active” controller architecture, massive performance (even InfiniBand), and complete feature set (once VAAI is released), Nimbus may have hit on their hands.

© sfoskett for Stephen Foskett, Pack Rat, 2012. | Nimbus E-Class: The First Big, Redundant, All-Flash Enterprise Array
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Startup Pure Storage launches today with an all-flash array for the same price as disk

Flash memory is awesome, but SSD isn’t a drop-in replacement for disk. Flash is totally different from spinning disk, yet most storage arrays still treat it as disk-like block storage. That’s why I was pleased to hear from startup Pure Storage, who are asking the core question, “what should an all-flash storage device look like?”

Another Storage Startup?

Pure Storage is a storage startup led by veterans of Veritas, NetApp, Sun, and Apple. The company has raised $55 million from key investors, including flash giant, Samsung.

Pure is working on what they call “the first all-flash enterprise array”, a slogan that will certainly draw the daggers of Nimbus Data, which has been selling an all-flash enterprise array for over a year. Like the Numbus S-Class, the Pure array relies on inline data reduction and compression to optimize the cost of storage, bringing flash capacity within reach of enterprise customers.

The Pure Storage FlashArray FA-300 array features active/active controllers with 12 CPU cores. These controllers attach to 24-bay drive shelves full of 2.5″ SSDs, and multiple units are clustered with 40 Gb InfiniBand. Pure Storage boasts that this combination allows 300,000 read and 180,000 write IOPS with less than one millisecond of latency. These are impressive performance numbers, but that’s not really the focus of the company.

Most beta customers use Pure Storage for a combination of performance and capacity. They are pleased to not have to worry about storage performance, but are also interested in reducing floor space and power demands of many-spindle storage arrays.

The smallest Pure Storage array (which includes one controller and one shelf of SSDs) provides 5.5 TB of raw capacity, presumably using 24 256 GB SSDs. A high-availability configuration would include two controllers and two shelves of SSDs for 11 TB of raw storage.

You should also read my follow-up piece on Pure’s pricing, When Pricing Gets Squishy Competition Heats Up

Optimized I/O

Pure also relies on MLC flash, a technical choice that will likely be the target of competitors. But the company insists they can overcome the limitations of inexpensive MLC flash (slower writes, reduced longevity) through intelligent software optimized for just this storage medium.

All I/O is thin provisioned and zero-detected, de-duplicated, and compressed in-line before it hits the SSDs. Since flash excels at random reads, data de-duplication does not have the performance impact most folks assume. In fact, de-duplicating data actually improves performance since less writing is required. Pure Storage uses a 512 B chunk rather than the larger chunks used by competitors, and they claim this gives a capacity advantage.

Pure’s array was designed from the ground up around flash, with minuscule latency and no tiering to spinning disk. The Pure Storage array does not use raw NAND flash but still relies on SSD (likely 470-series SATA drives sourced from partner Samsung). The Pure Storage “Purity” software optimizes I/O for these SSDs rather than relying on the in-drive software. They “cook” the data to the optimum chunk size, so the drive never needs to re-arrange I/O internally and performance does not diminish over time. Another optimization is I/O scheduling so drives are never written to and read from at the same time. Pure Storage also moves data over time for wear leveling, though it’s not clear how this interacts with the similar functionality already present in the SSDs.

Since flash memory has unique failure patterns, Pure Storage designed their own “RAID 3D” system to protect data. SSD drives sometimes fail entirely, but unrecoverable read errors (URE) are much more common. And as NAND flash ages (and device generations get finer geometry), error rate increases. Data is collected into a segment before writing, and each segment is written to all available drives. If a drive fails, data is re-protected in the background with at least dual parity. Since flash is so fast, rebuilding parity is much quicker than spinning disk-based systems. This wide striping also makes performance more consistent as drives fail and the system fills.

The Pure Storage Use Case

Pure Storage’s pricing is based on a 5:1 capacity reduction target, though the company claims that they often beat this in production, especially for VMware environments. Real-world data reduction cited by Pure Storage ranges from 4:1 for an Oracle environment to 17:1 for VMware.

The Pure Storage array is in its final beta round and the company expects to ship GA product by the end of the year. Key use cases are VMware and database environments – two high-I/O applications that have traditionally benefitted from flash storage.

Stephen’s Stance

It’s great to see fresh thinking in storage, and Pure Storage comes out of the gate with some impressive credentials: A top-tier team, excellent technical capabilities, and reasonable pricing. But it takes more than a great product to succeed in storage, and building awareness and sales are the next challenge for the company.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Pure Storage All-Flash Storage Array Revealed
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

"FCoE vs. iSCSI" isn't a battle or cage match. Your choice depends on many factors, and is more a reflection of convergence than a religious conviction. (photos by Peter Tsai, @SuperTsai)

My presentation at Interop in Las Vegas on May 11, 2011, focused on the protocols that will underpin converged storage networking in the future. My topic, assigned by network computing editor Mike Fratto, was “FCoE vs. iSCSI – Making the Choice.” Although this sounds like a grand competition between the two protocols, my take on the subject is very far from that idea. Rather than a battle, the rise of FCoE and iSCSI is part of the ascendance of convergence of storage and data networking on Ethernet.

“The notion that Fibre Channel is for data centers and iSCSI is for SMB’s and workgroups is outdated. Increases in LAN speeds and the coming of lossless Ethernet position iSCSI as a good fit for the data center. Whether your organization adopts FC or iSCSI depends on many factors like current product set, future application demands, organizational skill-set and budget. In this session we will discuss the different conditions where FC or IsCSI are the right fit, why you should use one and when to kick either to the curb.”

I began my session by pointing out that I am neither a vendor nor protocol cheerleader and don’t really have a horse in the race in terms of a transition to FCoE, iSCSI, InfiniBand, SAS, or any other protocol.  Frankly, I don’t see this as a race, and I don’t care who wins if it is one as long as IT infrastructure progresses to a more flexible state.

Converging on Convergence

The important aspect of any discussion of FCoE is not the protocol itself but the underlying shift away from specialized storage networks converging on Ethernet. ISCSI began this trend almost a decade ago, and the Ethernet roadmap leaves Fibre Channel in the dust.

I see three key elements converging to bring, if you pardon the pun, convergence of data and storage networking:

1. The wholesale adoption of Intel compatible processing architectures
2. A shift toward open systems (Windows and UNIX) for application processing
3. And the widespread adoption of IP as an internetworking protocol.

None of these “trends” is surprising or even questionable: Intel compatible open systems servers using IP dominate modern data centers.

Given this dominant processing architecture, Ethernet is a logical choice as an interconnect. No other network protocol even comes close to the market share, compatibility, and support for Ethernet.

Then we must consider the factors that drive convergence of networking protocols. After all, we have long seen a variety of different protocols in niches such as storage, voice, video, WAN, clustering, and other areas. But virtualization of servers, the need for consolidation to reduce port count and cabling, and a continuing thirst for better performance makes convergence on a single protocol a logical step for these and other areas of IT infrastructure.

If we converge on Ethernet, much will change both inside and outside the data center. Server managers will see greater flexibility and mobility of virtualized servers and blades, as well as increased performance overall. Storage managers will shift from managing esoteric networking protocols to a focus on data management and array performance. But network managers will bear the brunt of the shift, with a wider sphere of influence and new headaches from workloads that do not behave like conventional LAN applications.

The Performance Picture

Turning back to the face of storage networking, we see that one major driver for convergence is pure performance. Although Fibre Channel has an impressive roadmap, with performance doubling again and again, it can’t hold a candle to Ethernet. With historical leaps of an order of magnitude and performance, Ethernet will soon leave Fibre Channel well behind.

When iSCSI first appeared, it was hitched to fairly unimpressive Gigabit Ethernet even as Fibre Channel networks made a transition from 2 to 4 Gb. But iSCSI made a quantum leap in performance this year, transitioning to 10 Gb Ethernet even as Fibre Channel networks moved to 8 Gb. ISCSI FCoE will continue benefiting from Ethernet performance improvements in the coming years, transitioning to 40 Gb and 100 Gb. This will make 16 Gb and 32 Gb Fibre Channel look slow by comparison.

One area that is often overlooked in terms of performance is latency of I/O operations. Although iSCSI over 10 Gb Ethernet can carry 50% more data than 8 Gb Fibre Channel (thanks to more efficient encoding), it also benefits from drastically lower latency. It can handle 50% more packets than 8 Gb Fibre Channel or 10 times as many as Gigabit Ethernet. In other words, in a shared virtual environment, 10 Gb Ethernet allows more systems to get more work done in the same amount of time.

Ethernet Enhancing Data Centers

But performance is only half the story of converged Ethernet. It also supplies server connectivity, reducing the all too frequent situation where configuration and location of servers is dictated by cable availability rather than application need. This will change the face of the data center, encouraging the use of blade servers, virtualization, and flexible (dare I say “cloud”?) infrastructure. It will encourage mobility of machines, especially virtual ones, and demand new networking protocols like OpenFlow.

Ethernet required a serious upgrade to handle this workload, however. Although iSCSI works fine over just about any network, thanks to TCP/IP, FCoE and similar protocols require flow control and guaranteed lossless data delivery. This led to the development of data center bridging protocols (DCB), including priority flow control, bandwidth management, and congestion management. With the first two of these now widely available and the third following shortly, Ethernet is ready to take center stage.

FCoE vs. iSCSI

With discussion of convergence out of the way, we can finally talk about making the choice between iSCSI and FCoE. There are four main reasons to choose one protocol or the other:

1. Data center strategy
2. Performance needs
3. Desire for compatibility
4. Cost concerns

Each of these is a valid reason to pick FCoE or iSCSI in any given situation, and none is a drop–dead decision-maker. There are cases where FCoE will be cheaper than iSCSI and vice versa, for example.

Regardless of the choice between these two protocols, one element remains the same: SCSI. Nearly every enterprise block storage protocol is based on SCSI, and it is one of the seminal technologies that enabled the development of enterprise storage as an industry. Every enterprise block storage protocol, including FCoE, iSCSI, SAS, and plain old Fibre Channel, is really a transport for SCSI.

This makes the selection of protocol less relevant to operating systems and applications, since all will “see” storage the same way. There are major differences between the three SAN protocol choices, in terms of routability, availability of host and initiator hardware and software, maturity, and the availability and selection of management tools.

iSCSI has a more robust support matrix than Fibre Channel over Ethernet, with hardware and software drivers available for nearly every operating system. It is widely supported with mature storage systems available from nearly every vendor. Green field SAN designs with no existing Fibre Channel infrastructure should look no further: iSCSI is a great choice for new storage networks.

The selection of FCoE, on the other hand, is more about evolution from Fibre Channel in enterprise storage networks. There is a threefold path for Fibre Channel architects: They can continue with end-to-end Fibre Channel, and Ethernet and FCoE at the edge, or attempt to build out an end-to-end FCoE SAN. This last option is only recently possible, and is by far the least popular model for Fibre Channel architecture at the present time but will become dominant eventually.

Making the Choice

There are good reasons and bad to pick one protocol over the other, and none rises to the level of religious conviction one might see perusing blogs and tweets on the subject.

FCoE is an evolutionary transition for organizations that already have a large installed base of Fibre Channel equipment, tools, and skills. These environments can incrementally adopt Ethernet as an edge protocol while they continue to leverage the enterprise Fibre Channel storage arrays they already own. Strategically, FCoE makes perfect sense for users of “blocks” or “stacks” from vendors like Cisco, EMC, HP, and NetApp. But FCoE remains somewhat unproven, and some supporting protocols, like congestion notification and so-called Ethernet fabric technology, are immature at best when it comes to interoperability.

One common refrain when comparing FCoE and iSCSI is the efficiency of the protocols. Packaging SCSI in TCP and IP can’t be efficient, can it? But an analysis of the protocols reveals that absolute bit efficiency is very similar between Fibre Channel, FCoE, and iSCSI. Tests by Dell’s Tech Center and others show that iSCSI is fairly efficient in terms of data throughput and CPU utilization as well.

Stephen’s Stance

iSCSI is an excellent choice in situations where Fibre Channel investment is nonexistent or badly in need of wholesale upgrade, while FCoE is likely to take over in high-end enterprise shops

iSCSI is an excellent choice in situations where Fibre Channel investment is nonexistent or badly in need of wholesale upgrade. It will continue to grow based on ease of use, low cost and high performance, and widespread support, in the transition to 10 Gb Ethernet could not be simpler. FCoE, on the other hand, is likely to take over in high-end enterprise shops. It is relentlessly promoted by major vendors, and it seems that they will force the upgrade eventually. But some areas are still not ready for prime time, and buyers should beware of grandiose promises at this point.

In counterpoint, one may ask the question of why we chose Ethernet at all. It required much work, and unnatural acts like DCB, to prepare Ethernet to become the dominant protocol for convergence. Why not use InfiniBand instead, since it already works, has widespread implementation, excellent performance and scalability, as well as interoperability and hardware availability? Price is one concern, but the major factor is far more basic: No one doubts that Ethernet will eventually ascend and overcome its obstacles. It is a foregone conclusion.

In retrospect, many alternative protocols might have been better suited to convergence, including ATM and even Token Ring. Although the topic of Fibre Channel over Token Ring (FCoTR) brings a smile to the faces of network and storage nerds everywhere, we all expect that fiber Channel over Ethernet (FCoE) and iSCSI will rule the day.

Photos by Peter Tsai

Watch the presentation from Interop (apologies for the poor camera angle and sound!)

© sfoskett for Stephen Foskett, Pack Rat, 2011. | FCoE vs. iSCSI – Making the Choice
This post was categorized as Computer History, Enterprise storage, Everything, Features, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Virtualization decouples physical infrastructure from logical resources

Virtualization of IT systems decouples physical infrastructure from logical resources, hiding complexity and enabling new capabilities. However, not all potential benefits of virtualization have meaningful value outside IT circles: Too many of our discussions revolve around the very complexity that virtualization technology should be hiding! True business value is derived from transformed virtual resources in the next-generation data center, not the incremental capacity gains of virtual servers. But how will we get there, and what will this future look like?

The Problem with Virtual Servers

Implementation of virtualization technology to date has merely delivered condensation of physical resources: 250 physical servers are condensed onto 20 physical servers, but 250 virtual server images remain. True, this does result in the reduction of data center footprint, from rack space to power and cooling, enabling moderate cost savings. But these are not examples of real consolidation, let alone business transformation.

Many have lamented this “virtual server sprawl” and suggested alternative methods of consolidating low-utilization applications into larger, more flexible “resource servers”. For example, numerous SQL servers can be combined on a single central server with more focused management. But these larger resource servers are not normally virtualized since their concentrated I/O demands can overtax current server virtualization platforms. Therefore, consolidation and virtualization remain separate.

This is the problem with conventional server virtualization. It enables us to condense data center demands for some systems, but delivers very little else apart from new backup and management headaches. Certainly we can provision servers more quickly, and we might be able to recover from a disaster more easily, but these are IT-facing benefits that other business entities care little about.

Storage and Network Virtualization

Virtualization of storage and network resources face even higher barriers. Where server virtualization has quickly delivered incremental “green” savings, these benefits are harder to come by in other areas.

Storage virtualization primarily delivers flexibility. SAN or NAS systems can be combined into larger pools, allowing existing resources to be better utilized or provisioned more quickly. But there is only a little cost avoidance to be gleaned from more efficient use of storage capacity. Real cost savings would require reduction of infrastructure, and constant data growth makes this extremely difficult to achieve. Other benefits, like enhanced data migration or heterogeneous replication, ought to be invisible to the business anyway.

Network virtualization lags even further behind. Only a few shops have attempted to use technology like InfiniBand to enable flexible virtual connectivity, though the future Converged Enhanced Ethernet concept is beginning to spark some interest. Here again, financial benefits from network virtualization technology are limited to a moderate reduction in future equipment cost.

Transforming the Data Center

In all three instances (server, storage, and network), the financial benefits are merely the sideshow. The underlying benefit from virtualization of IT infrastructure comes from the extension of IT systems outside the data center, a change on the order of the advent of minicomputers or the spread of open systems.

VMware recently laid out a serious and compelling vision of this future Virtual Data Center as VDC-OS. Their concept is evolutionary and radical at once, with the simple virtual server infrastructure of today augmented with increasingly uniform and flexible storage and network layers. This culminates in a truly virtual data center, where running server images can move from device to device, location to location, and even out to the cloud.

VMware’s brilliance is in leveraging what works today (virtual server images on ESX) to build a foundation for complete virtualization of physical resources. But virtual servers running on VDC-OS remain tied to the present: They run the same operating systems and will likely remain bound to the same “one (virtual) server per application” world view that pervades open systems today. This leads to exactly the same situation of server sprawl that has proven a management nightmare.

Others are extending the web hosting concept to enable custom applications to be run on the scalable, flexible, multi-homed servers that run the world’s biggest Internet applications. Google and Amazon’s visions are decidedly post-data center, with applications, rather than server images, being the primary unit, and database-style storage replacing conventional blocks and files. Use of these web-oriented application platforms has so far been limited to entirely new systems built from scratch to take advantage of them, limiting their appeal to current IT environments.

Where Is the Business Value?

Yet, most discussions of these virtualization strategies (mine included) fails when it comes to demonstrating real business value. We must move away from quickly-forgotten cost savings and focus instead on profoundly transforming how IT serves business goals. Virtualized infrastructure allows flexibility and scalability, changing how everything in IT works.

Whether it uses conventional operating systems and applications or re-engineered web-enabled solutions, virtualized infrastructure fundamentally changed our world. Organizations would be free to physically move their systems, even outsourcing or offshoring the infrastructure component entirely. They could move to an on-demand purchasing model for logical capacity, not just bits and bytes.

In the process, they would render current server platforms, operating systems, and storage devices irrelevant. Undoubtedly, attaining this future remains a while off, but IT professionals should consider its implications. Much of what we do is focused on making the “plumbing” work efficiently rather than serving the needs of the business. Where do we stand once the perennial issues of performance, availability, and scalability are solved?

photo by Roy Smith

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Where Will Virtualization of Data Center Infrastructure Take Us?
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

Why do some data storage solutions perform better than others? What tradeoffs are made for economy and how do they affect the system as a whole? These questions can be puzzling, but there are core truths that are difficult to avoid. Mechanical disk drives can only move a certain amount of data. RAM caching can improve performance, but only until it runs out. I/O channels can be overwhelmed with data. And above all, a system must be smart to maximize the potential of these components. These are the four horsemen of storage system performance, and they cannot be denied.

The Chain of Command

It is tempting to think of storage as a game of hard disk drives, and consider only The Rule of Spindles. But RAM cache can compensate for the mechanical limitations of hard disk drives, and Moore’s Law continues to allow for ever-greater RAM-based storage, including cache, DRAM, and flash. But storage does not exist in a vacuum. All that data must go somewhere, and this is the job of the I/O channel.

To be useful, storage capacity must connect to some sort of endpoint. This could be the CPU in a personal computer or an embedded processor in an industrial device. Indeed, there are endpoints and I/O channels throughout modern systems, with potential bottlenecks, caches, and smarts at each point. “Storage people” like me tend to think too small – imagining that the I/O channel ends at the disk drive, the “front end” of the array, or the storage network. But data must travel further, all the way to its final useful point in the core of the CPU.

Once we consider I/O as a long chain of interconnected endpoints, we begin to see the fact that I/O constraints at any point can strangle overall system performance. This is not merely an academic exercise: Optimizing the I/O channel is a consuming passion for most practitioners of enterprise IT, including architects, engineers, and system developers. And, like a good game of Whack-a-Mole, increasing the speed of one link causes another chokepoint to rear its head.

Parallel and Serial I/O

Imagine you had a warehouse full of boxes to move across the country as fast as possible. There are a few options available to you:

1. A fast truck can zip back and forth with just a few boxes
2. A train is slower, but its many cars can haul a huge quantity

But there are realistic limits to both capacity and speed: The train has to fit on the tracks, and the truck can’t move at the speed of light. Plus, one must consider the time taken to load and unload the chosen vehicle.

We continually shift between parallel and serial I/O paradigms

The same trade-offs are true of computer busses: Serial channels can be optimized to zip individual bits back and forth, or parallel busses can be designed to carry whole bytes (or more) at a time. The simplicity of serial communications is tempting, but designers continue to resort to parallelization for added throughput.

Note: Most serial protocols actually feature two links, making them “full duplex”: One for transmit and another for receive.

Serial storage interconnects are dominant currently, with fraternal twins SAS and SATA coming to dominate the disk interface landscape. SAS and SATA share the same 1.5, 3, and now 6 gigabit per second serial physical interconnect, offering more than enough throughput for conventional hard disk drives and edging out older serial (Fibre Channel, SSA) and parallel (ATA and SCSI) alternatives.

Networks (Ethernet, Fibre Channel, and InfiniBand) are predominately serial as well, as are lower-end interconnects like USB and FireWire. Serial communication also dominates in the system bus world, with serial PCI Express toppling parallel PCI.

But parallel variants are often offered for increased throughput: Multi-lane PCI Express and bonded multi-link InfiniBand make up a fair portion of the installed base, while load balancing MPIO drivers are common in Fibre Channel storage. And let’s not forget that the “X4″ variants of Ethernet use multiple bonded links as well.

The Definition of Bottle Neck

Most English speakers have encountered the French term, “cul de sac”, meaning “bottom of the bag” or dead end. But hard disk drives have plenty of “bottom end”, or storage capacity. When it comes to disks, the issue is usually at the neck of the bag: Data just can’t be pulled out of a hard disk drive fast enough.

Emptying a barrel of wine through a spigot takes hours, but pry the end off and the floor is covered in a moment!

The density of modern hard disk drives (the capacity of our barrel) has been growing much more rapidly than the I/O channels serving them (the spigot). Where once a hard disk drive could be filled or emptied in an hour or two, modern drives take days or weeks!

I once called this “flush time“, but I think the wine metaphor is much more appetizing!

This “bottle neck” has serious implications beyond basic storage performance. Data protection is impacted, since ever-larger storage systems can no longer be backed up by dumping their content; system reliability is reduced, since week-long RAID rebuilds increase the risk of multiple drive failures; and cost containment efforts are also impacted, since adding spindles drives up prices.

Nowhere is this bottleneck more evident than in portable devices. Modern drives (like the 1 TB Seagate USB drive I recently reviewed) have massive capacity and pathetic performance. The USB 2.0 interface just can’t keep up, and this creates a limit to the expansion of capacity. It would take half a day to fill that drive under perfect conditions at 25 MB/s, reducing its value as a massive data movement peripheral. The emerging USB 3.0 standard promises to alleviate this performance issue for now, as illustrated with Iomega’s new external SSD.

Cache and solid state storage can help, but they have their own bottlenecks. Storage arrays typically use Fibre Channel or SAS SSDs, and their front-end interface remains the same. The best-performing SSDs use the PCI Express bus directly rather than emulating hard disk drives over SCSI interfaces. And even PCI Express might not be enough to handle the massive I/O of NAND flash or DRAM. In each case, the bottleneck moves down the chain.

A Chain of Bottlenecks

Let’s follow a typical I/O operation from the disk to the CPU core and count the I/O channels:

1. A read head senses the state of a bit of magnetic material on the surface of a disk
2. The head transmits this signal to a buffer on the disk controller board
3. The data is picked up by the disk controller CPU and transmitted over a SATA or SAS connection
4. The storage array or RAID controller receives the data and moves it over an internal bus to another buffer or cache
5. The data is picked up by another CPU in the array controller and sent out another interface using Fibre Channel or Ethernet
6. The data is buffered and retransmitted by one or more switches in the storage network
7. The host bus adapter (HBA) on the server side receives the data and buffers it again before sending it over a local PCI Express bus to system memory
8. The server memory controller pulls the data out of system memory and sends it via a local bus to the CPU core

There are actually many more steps than this, but the picture should be clear by now. There are many, many I/O channels to consider when it comes to storage, and the drive interface is just one potential bottleneck.

We constantly move bottlenecks around - as one link is improved, another choke-point appears

Optimizing Storage I/O

Tactical steps to improve storage performance typically focus at one link in the chain: Drive vendors move from 1.5 Gb to 3 Gb SATA, or SAN buyers upgrade from 4 Gb to 8 Gb Fibre Channel. But the basic architecture of enterprise storage has remained constant for over a decade, and the reliance on block SCSI commands endures. This is all about to change.

One critical bit of I/O optimization exists at the point of connection between the various chipsets inside the server. AMD pulled the memory controller off of the “northbridge” with their Athlon line. Intel did the same with their Nehalem and is eliminating the northbridge entirely with the Lynnfield/Jasper Forest CPU lines. This gives serious bandwidth to the crucial PCI Express-to-CPU-core link, moving the bottleneck downstream.

We are in the midst of a massive upgrade of the storage network as well. Between 8 Gb Fibre Channel and iSCSI and Fibre Channel over 10 Gb Ethernet, not to mention persistent interest in InfiniBand, storage network throughput is rapidly expanding. As with the internal PC connections, the expansion of network bandwidth has pushed the bottleneck to the storage array interface for the time being.

Microsoft and Intel recently pushed over a gigabyte per second over 10 GbE using iSCSI, but they needed multiple storage targets to feed that connection. It isn’t that modern storage systems couldn’t push that kind of I/O (indeed, arrays are tens to hundreds of times faster internally thanks to their spindles and cache), but that the conventional storage protocols are tightly linked to a single “front-end” interface. The current state of the art for storage array design is moving to distributed models, exemplified by pNFS and scale-out NAS concepts like MaxiScale (now acquired by Overland).

Once the array interfaces can pump out massive I/O, attention will turn once again to the disk interfaces themselves. Although 6 Gb/s SAS and SATA is now a reality, this interface is inappropriate for future high-performance SSDs. Arrays designed around flash or DRAM are likely to switch to PCI Express as their internal connection of choice for performance and to optimize data placement on these new devices. Companies like Nimbus and NetApp are already moving in this direction.

Time To Get Smart

Hard disk drive spindles make up the bulk of storage capacity, but small amounts of cache make them far more effective. But both of these horsemen must operate within the constraints of the I/O channels they pass through. This brings us to the final horseman of performance: Smarts. Clever designers have created clever controlling mechanisms to overcome the limits of spindles, cache, and I/O channels.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | The Four Horsemen of Storage System Performance: I/O As a Chain of Bottlenecks
This post was categorized as Computer History, Enterprise storage, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

But when will 10 Gb and FCoE arrive? Dave Raffo and I discussed the topic after Storage Decisions, and he just wrote an article on the topic, 10GigE still seeking killer app. Here are some points that came up in our discussion:

• Shipping and supported 10 Gb HBAs and CNAs are the key requirement, and it looks like the schedules of Microsoft, VMware, Sun, and Linus Torvalds will determine when the floodgates open
• iSCSI people are talking 10 Gb, too, and they might end up adopting it first with software initiators
• Interest in FCoE is focused at the largest enterprise shops, and I’m seeing a distinct line between “iSCSI shops” and “(future) FCoE shops” with very little overlap (as I previously noted, iSCSI and FCoE aren’t mortal enemies, and FCoE will rule in the largest environments)
• No one is talking about 8 Gb Fibre Channel – they have all decided that 10 Gb FCoE or iSCSI is the next step for block storage
• InfiniBand has its believers (and they are rabid fans!), but the users I talk to are, as a rule, heading toward FCoE rather than IB for their future connectivity
• There is a tiny bit of user interest in moving back to SAS-enabled DAS for virtual server environments
• Everyone I talk to is shocked there isn’t a 10 Gb iSCSI array on the market yet, and we all expect to see this before FCoE
• Although some vendors have announced FCoE products, they’re not here yet – and it will be another year still before we see production deployment

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Storage Folks Are Talking 10-Gig and FCoE
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

This is part of an ongoing series of longer articles I will be posting every Sunday as part of an experiment in offering more in-depth content.

There has been a lot of discussion in the storage industry about Fibre Channel over Ethernet (FCoE), making it the toast of Storage Networking World, but this technology remains relatively unknown to end users. Like so many storage protocols before it, the $10,000 question is whether FCoE will take off like iSCSI or fizzle as a niche product like FCIP, DAFS, and so many others.

If it does succeed, another critical question is what this means for iSCSI, Fibre Channel, InfiniBand, and to a lesser extent AoE, expanded SAS, and other options for SAN storage. The enterprise data center is poised for a complete change in server connectivity, with 10 Gb Ethernet converged network adapters (CNAs) and new core switches carrying both network and storage traffic, and this holds promise, especially in virtualized environments. But CNAs do not equal FCoE, and iSCSI, conventional Fibre Channel, and other protocols are roaring ahead. What impact will FCoE really have?

Why FCoE Matters

With 8 Gb Fibre Channel and alternative storage solutions leveraging InfiniBand now available (and with 10 Gb iSCSI imminent), many would ask why we need another enterprise block storage medium. In real-world applications, FCoE at 10 Gb will likely deliver roughly the same performance as 8 Gb native FC. But FCoE will be one or two years late and (initially) more expensive. Although active standards participation and “plug fests” mean FCoE will likely be more interoperable than Fibre Channel was in its early years, the lack of support from operating system manufacturers is troubling. Plus, users will soon be able to build a very similar infrastructure by mixing iSCSI and 10 Gb Ethernet, and this will include all the advantages of IP and solid support.

So why pay more for the same performance from an untested protocol? It’s all about the future, and enterprise users will go where the market goes, just as wide availability of VHS tapes buried Betamax. Storage, network, and SAN vendors alike are lining up solidly behind FCoE as the next-generation enterprise interconnect. Although InfiniBand plays Betamax in this home video metaphor, with superior technology and availability, FCoE’s VHS camp has all the market ammunition. To paraphrase the (alleged) words of Bob Metcalfe, no matter what the technology looks like, the future of networking will be called Ethernet.

The biggest storage vendors are behind FCoE simply because they see that converging and leveraging I/O technology makes sense for them. They can swap out the physical and data link layers from Fibre Channel to Ethernet relatively easily, so the FCoE switch is an easier change than iSCSI. It is likely that they will be able to leverage commodity Ethernet hardware to reduce (their, not your) cost and increase profit margins once this switch is made. Plus, FCoE will potentially increase SAN attachment rates (and thus enterprise storage market penetration) thanks to the potential availability of converged network adapters (CNAs) on the server side, and the cost-effectiveness that sharing a CNA between network and storage implies. From the storage side, FCoE is all good.

Also see my posts on FCoE versus iSCSI and Cisco VFrame

The drive is similar on the network side. The era of differentiated SAN and LAN producers is over – all of the major networking and SAN vendors are repositioning themselves as next-generation I/O providers, setting up a battle in the network space to rival the mainframe shakeout of the 1980s and the PC wars of the 1990s. Converged I/O is the business model for connectivity vendors, and most are taking up the “data center Ethernet” (DCE) charge (also called “converged enhanced or enterprise Ethernet” or CEE) which includes FCoE as the storage protocol for virtual I/O. iSCSI is still there in a DCE world, but FCoE will take center stage for the enterprise market.

Counting the Benefits of FCoE

It may seem strange to declare an upstart like FCoE the winner when established options like InfiniBand, conventional Fibre Channel, and iSCSI are already out in the market, but this examination of the vendors indicates that it is indeed the case. Is this a case of the tail (vendors) wagging the dog (consumers)? Perhaps, but they will come along willingly given the strong case presented by converged and virtualized I/O.

Enterprise buyers are ready for a next-generation SAN technology, and some are beginning to look at 8 Gb Fibre Channel. The few that really need performance will certainly buy 8 Gb FC today, but this has little bearing on the overall prospect for FCoE. When an application requires performance and money is available, purchases will be made regardless of future strategy.

Enterprise storage and network architects are beginning to consider the implications of server consolidation and virtualization. As they see footprint shrink thanks to compact or blade servers and server virtualization, they will begin to question the proliferation of interconnects on the back end required to keep up with the I/O demands of these super servers. Already, virtual I/O purveyors like Xsigo are making hay in this market, and, as mentioned above, their SAN and LAN vendors are spreading the message, too. It won’t be long before they are convinced.

Also see my posts on VMware VDC-OS and VDC-OS vStorage

Many people mistakenly assume that DCE means pushing all protocols through a single LAN, but this is not the case. These networks will be engineered like SANs from the start, with redundant connections and transparent failover. Although storage and network connectivity will share the same physical “pipe”, they will certainly be segregated on separate VLANs and protected with quality of service technologies. They have to be separated – FCoE (lacking IP) will require a totally different network topology than LAN connections.

So Who Buys FCoE?

Note that, throughout this discussion, I am referring only to the large-scale enterprise data center storage market. Smaller corporate environments have already embraced iSCSI en masse, expanding the penetration of consolidated storage concepts beyond anything Fibre Channel could ever accomplish. And small office and home networks are beginning to embrace these concepts as well, but are relying on protocols like CIFS and AFP for file servers and may begin to look at ATA over Ethernet (AoE) and proprietary protocols like the one pushed by Zetera/NetGear instead of iSCSI.

This leaves us with a layer cake of appropriate protocols from the smallest to largest networks. But all have one thing in common: They are all converged and they are all carried in Ethernet packets. Bob Metcalfe was right!

Update: Storagebod points out that FCoE might see its first application in inter-switch links and other similar storage network infrastructure connections. And Chris Mellor at The Register points out that SAS is already displacing FC as an internal drive interconnect.

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Reality Check: The FCoE Forecast
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I am not the typical enterprise storage user. In fact, I am not an enterprise storage user at all – I am a consultant focused for over a decade on assisting enterprises with their storage architecture and strategy, working with businesses of all sizes. My background is both a blessing and a curse – I have seen far more enterprise storage environments in much more detail than most people, but I am unable to truly empathize with my corporate storage compatriots since it’s not really my gear and data that I am working with.

Based on this experience, what does the future hold? Where is enterprise storage heading? Read on for my thoughts.

Fibre Channel

Although the commoditizing storage market would seem ripe for stagnation, the opposite is happening. In fact, the enterprise storage market has continued to diversify, with iSCSI recently expanding the options for storage connectivity to a new market and FC seeing rapid uptake in the virtual server arena. Fibre Channel over Ethernet (FCoE), whether or not part of the datacenter Ethernet push, is joining virtualized I/O technology based on InfiniBand as the next wave in connectivity. Simply put, the market is not standing still.

Focus on the new world of storage area network (SAN) connectivity can take away from the basics, however. Traditional switched Fibre Channel (FC) remains the healthy market leader, and the recent upgrade to 8 Gb speed has attracted customer attention. Although they are unlikely to run out and perform a mass upgrade, 8 Gb FC is as as much of an inevitability as 4 and 2 Gb before it. The switch will be made, and customers will upgrade organically.

One of the big beneficiaries of 8 Gb FC will be those with virtual servers. This new hardware is coming just as N_Port ID virtualization (NPIV) is maturing, and this technology, which enables multiple servers to share a single FC interface, in combination with a doubling of bandwidth will make modern Fibre Channel tech extremely attractive to virtual server shops.

Even those who do not adopt 8 Gb FC connectivity in their SAN will benefit from the upshift in throughput, as native 8 Gb disks appear and enterprise arrays are redesigned to accept them, just as devices like the DMX-4 from EMC benefited from 4 Gb FC back ends.

IP Storage: iSCSI and NAS

As far as iSCSI goes, those of us who saw the promise of this technology can finally declare victory. Every storage device that matters offers iSCSI as a connectivity option, and most buyers are considering adopting it. It is being weighed fairly against FC, and the promise of reduced heterogeneity and cost are proving attractive to many. In fact, it is wrong to continually compare it against FC, since many iSCSI buyers would never adopt an FC SAN due to concerns about cost or learning curve. Indeed, much of the uptake in iSCSI comes from areas where SAN was never adopted, and iSCSI’s growth can be partly attributed to these happy customers spreading the technology wider than originally intended.

A hidden benefit of iSCSI adoption is the technologies and techniques that have come along with it. Clustering of smaller storage systems has become a common option for scalability, and has proven itself against old modular “head/shelf” arrays. Although the rate of adoption for security technologies like CHAP and IPsec in iSCSI remains low, they are far more common than their FC relatives. And Microsoft’s simplified and universal iSCSI drivers, which include multi-path and snapshot technologies, have been much more successful than their proprietary equivalents.

Let us not forget humble old network attached storage (NAS), either. File server consolidation to NAS filers continues to be a healthy (but less flashy) market, and NAS virtualization is on the rise as these devices proliferate. And some in the server virtualization community are beginning to consider NFS for their servers, especially when it comes to VMware. NAS definitely still has life and legs and will benefit from the shift to 10 Gb Ethernet just like iSCSI and FCoE.

The Future

As for the future, it is clear that most storage vendors are lining up behind Fibre Channel over Ethernet (FCoE) . Although true I/O virtualization, as envisioned by datacenter Ethernet (DCE) and InfiniBand, may not gain traction outside the largest data centers, FCoE seems to be the inevitable next generation for massive enterprise storage. Just about every vendor is committed to it, and the customers I have spoken to accept it as the future. Although 8 Gb FC might delay FCoE in some cases, it will almost certainly be the predominant SAN connectivity mechanism for large block storage devices within five years.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | A Consultant’s View Of The Enterprise Storage Market
This post was categorized as Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Overall, I was impressed with many of these products this last year, but certainly CommVault, HDS, ProStor, Xsigo, and Akorri (winners all) stood out with important products. I too was surprised to see the absence of some heavyweights: Symantec’s strength in archiving and backup didn’t net them an award, and both Emulex and QLogic were skipped for the 8 Gb FC market. EMC was overlooked, too, but I’m certain that will be remedied for 2008 after the flurry of excellent new products announced these last few months.

Backup Software

• Winners: CommVault’s Simpana 7.0 – an impressive product that brings a very wide range of data protection features in an integrated package. They’d be in my top three! Next was the intriguing RecoverGuard from Continuity Software, a product which intrigues me. Finally, Yosemite’s amazingly priced FileKeeper Professional took bronze – I definitely would have put this up there, too!
• Strong contenders: EVault’s solution would have been my pick since it effectively tackles a real need – mobile and SMB data protection! Also, Symantec’s strong NetBackup 6.5 upgrade deserved more attention.

Backup Hardware

• Winners: NEC’s HydraStor HS8 isn’t a platform I’m familiar with – I’ll have to take the judges at their word! But I can’t argue with silver and bronze winners, Copan and ProStor’s cool RDX – both are truly revolutionary products and deserve the spotlight.
• Strong contenders: I would definitely have added Gresham’s Clareti VTL to the list – it’s much more than “just a VTL” and I hope it gets some more attention!

Disk and Disk Subsystems

• Winners: Seagate’s Momentus 5400 FDE.2 deserves considerable attention, bringing built-in hardware encryption to the mobile data market – every mobile disk should have this technology! NetApp’s FAS2000 (silver) is nice enough, but I was much more impressed with the bronze-winning USP V from HDS this year.
• Strong contenders: I would have given the USP V the top prize for 2007, but I can’t argue with the little Seagate disk. The AMCC 3ware 9600 RAID adapter deserves some attention, too.

Storage Management Software

• Winners: Akorri’s BalancePoint (gold) impresses me much like bronze-winning Onaro’s Application Insight does. Both make the big leap from “storage” to “application data”, and both are worthy winners. I’ve never used the silver-winning Finisar NetWisdom product, but it looks like others are impressed with it.
• Strong contenders: This was a crowded field, and Symantec, WysDM, Quantum StorNext, and the rest probably came close. I agree with the Akorri placing, but would have picked one of these instead of the potentially mis-categorized Finisar product.

Networking Equipment

• Winners: Xsigo took the top honors with their InfiniBand-based VP780 platform. This is truly a next-generation product, and it is getting serious attention and traction, and deserved a spot on the list! Riverbed’s excellent Optimization System (silver) also deserved its ranking, but I’m not familiar enough with the Storwize product to know if it’s truly bronze-worthy.
• Strong contenders: I would have tipped either QLogic or Emulex‘s 8 Gb offerings for a spot. Despite my jokes, 8 Gb FC is an important element of the modern SAN and both companies have carved out a compelling product, but apparently neither shipped in volume until this month…

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Reacting to TechTarget’s Storage Products of the Year
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.