EMC VFCache (aka “Project Lightning”) Is One Small Step, But an Important One

EMC will today unveil a new product, and will no doubt attract a great deal of press. The modest technical capabilities of VFCache 1.0 limit its use case, but the announcement is big news, since it marks EMCs first foray into the hot server-attached storage market.

EMC VFCache is a Simple Read Cache

I was not pre-briefed on this product, and I’m not all that thrilled at the prospect of attending a launch webinar, so what you read here is based on my own research and reading of the available information as of this morning.

When EMC announced Project Lightning last year, company insiders expressed surprise to me. It seems that many had never heard of the project, and those that had didn’t think it was far enough along to be announced. I didn’t even bother to write about the Project Lightning announcement at the time. But today EMC unveils the production product that came out of Project Lightning.

EMC VFCache appears to be a simple and straightforward offering:

1. A PCIe SSD from Micron or LSI sits in the server and acts as a read cache to accelerate performance
2. EMC software also runs on the server, snooping on I/O and filling the cache with relevant data

There’s not a lot more to the product than that. EMC will sell the PCIe SSD and bundled software as VFCache, and will no doubt market the heck out of this product. Perhaps the only novel twist is the so-called “split-card” mode, which allows the card to act as a write cache. But EMC only supports this for transient “throwaway” data with direct attached storage (DAS) as a backend. There’s no way a conservative, enterprise focused company like EMC would risk sanctioning a writeback cache with no redundancy or data protection features.

Perhaps the biggest limitation of the initial VFCache offering is its limited applicability to enterprise server virtualization environments. VFCache uses a filter driver installed in each VM guest, and includes no hypervisor drivers though there is a vCenter plug-in. This makes VMware vMotion very tricky, involving scripting to remove and re-add storage. This means VMware SRM will not easily work, and there is no support for clustering, either.

This is no surprise, since VFCache appears to the host as a local storage volume (AKA, a disk drive or LUN) which would disappear if a virtual machine is moved to another server. Virsto solved this problem by virtualizing storage presentation to the hypervisor, and Fusion-io’s ioTurbine software does not interfere with vMotion either. EMC will likely go in this direction in the future, but it’s a big hole in the product for now.

You might also like reading Micron Bursts Into the PCIe SSD Market to learn more about the card EMC is using

The News: EMC Is in the Host-Based Storage Business

The primary use case for this product is server I/O acceleration. This is desperately needed, as applications and servers are rapidly outrunning the capabilities of conventional storage arrays. EMC and other legacy array manufacturers initially tried to address this I/O imbalance with tiered storage and in array caching. Indeed, these technologies are fairly effective at accelerating the performance of conventional disk storage arrays.

But flash manufacturers like Fusion-io (not to mention Micron and LSI) absolutely demolished storage array performance with their in-server offerings. EMC faced the prospect of losing out on the high-performance storage market. EMC simply could not allow their bread-and-butter enterprise customers to look elsewhere for strategic, high-performance storage for high-profile applications.

VFCache gives EMC salespeople a silver bullet when customers demand maximum performance, but this launch may not spell doom for the flash startups. For one thing, it legitimizes host-based flash cards as a viable component of enterprise storage architectures. It also opens the door to comparison between SAN storage and non–SAN alternatives that go well beyond what EMC is currently offering.

Shared Flash Storage Is on Deck: Project Thunder

As part of the VFCache introduction, EMC is also talking about Project Thunder, a shared version of VFCache. At the very least, thunder will allow multiple servers to access a shared pool of flash cache. This should allow VMware vMotion and DRS to function, and could be much more than that.

EMC could build a high-availability, high-performance all-flash storage array that may even use InfiniBand as an interconnect. The new Nimbus Data E-Class storage array matches this description perfectly, and their CEO tells me that performance over InfiniBand is indeed comparable to in-server PCIe flash cards. It seems logical for EMC to enter this market, if only to disrupt the momentum of Fusion-io and the rest of the all-flash storage upstarts.

Read more about the Nimbus E-Class: The First Big, Redundant, All-Flash Enterprise Array

The only fly in the ointment here is the recent consolidation of the InfiniBand market. Mellanox bought Voltaire, and QLogic sold out to Intel, putting that protocol on tenuous grounds. Perhaps 40 or 100 Gb Ethernet will emerge as a viable alternative for high-performance connectivity, or perhaps these products will retrench on shared PCI Express instead. Micron recently purchased Virtensys for just such a product, and Xsigo has been making big waves in the area of converged I/O as well. The market clearly need something better than Fibre Channel for maximum performance storage, even if InfiniBand isn’t it.

Stephen’s Stance

EMC VFCache (née Project Lightning) is a fairly simple offering: A server-based PCIe flash card that acts as a read cache with no integration with storage arrays or hypervisors. But EMC’s entrance into the host-based flash storage market is a powerful demonstration of the wave of disruption caused by flash-based storage and high-performance computing. Although I am not all that impressed with the product itself, I would be distressed if EMC had not introduced it.

VFCache illustrations are copyright EMC Corporation and are used here with permission

More solid, independent VFCache coverage:

• EMC Enters The Market With “Me Too” Flash Products (Chris Evans)
• Cache Splash and Complex is the new Simple (Martin Glassborow)