EMC VFCache is a Simple Read Cache

I was not pre-briefed on this product, and I’m not all that thrilled at the prospect of attending a launch webinar, so what you read here is based on my own research and reading of the available information as of this morning.

When EMC announced Project Lightning last year, company insiders expressed surprise to me. It seems that many had never heard of the project, and those that had didn’t think it was far enough along to be announced. I didn’t even bother to write about the Project Lightning announcement at the time. But today EMC unveils the production product that came out of Project Lightning.

VFCache is a filter driver that caches writes

EMC VFCache appears to be a simple and straightforward offering:

1. A PCIe SSD from Micron or LSI sits in the server and acts as a read cache to accelerate performance
2. EMC software also runs on the server, snooping on I/O and filling the cache with relevant data

There’s not a lot more to the product than that. EMC will sell the PCIe SSD and bundled software as VFCache, and will no doubt market the heck out of this product. Perhaps the only novel twist is the so-called “split-card” mode, which allows the card to act as a write cache. But EMC only supports this for transient “throwaway” data with direct attached storage (DAS) as a backend. There’s no way a conservative, enterprise focused company like EMC would risk sanctioning a writeback cache with no redundancy or data protection features.

VFCache uses a filter driver installed in the VM guest

Perhaps the biggest limitation of the initial VFCache offering is its limited applicability to enterprise server virtualization environments. VFCache uses a filter driver installed in each VM guest, and includes no hypervisor drivers though there is a vCenter plug-in. This makes VMware vMotion very tricky, involving scripting to remove and re-add storage. This means VMware SRM will not easily work, and there is no support for clustering, either.

This is no surprise, since VFCache appears to the host as a local storage volume (AKA, a disk drive or LUN) which would disappear if a virtual machine is moved to another server. Virsto solved this problem by virtualizing storage presentation to the hypervisor, and Fusion-io’s ioTurbine software does not interfere with vMotion either. EMC will likely go in this direction in the future, but it’s a big hole in the product for now.

You might also like reading Micron Bursts Into the PCIe SSD Market to learn more about the card EMC is using

The News: EMC Is in the Host-Based Storage Business

The primary use case for this product is server I/O acceleration. This is desperately needed, as applications and servers are rapidly outrunning the capabilities of conventional storage arrays. EMC and other legacy array manufacturers initially tried to address this I/O imbalance with tiered storage and in array caching. Indeed, these technologies are fairly effective at accelerating the performance of conventional disk storage arrays.

But flash manufacturers like Fusion-io (not to mention Micron and LSI) absolutely demolished storage array performance with their in-server offerings. EMC faced the prospect of losing out on the high-performance storage market. EMC simply could not allow their bread-and-butter enterprise customers to look elsewhere for strategic, high-performance storage for high-profile applications.

VFCache gives EMC salespeople a silver bullet when customers demand maximum performance, but this launch may not spell doom for the flash startups. For one thing, it legitimizes host-based flash cards as a viable component of enterprise storage architectures. It also opens the door to comparison between SAN storage and non–SAN alternatives that go well beyond what EMC is currently offering.

Shared Flash Storage Is on Deck: Project Thunder

"Project Thunder" will externalize the PCIe flash cards over a high-performance "Server Area Network"

As part of the VFCache introduction, EMC is also talking about Project Thunder, a shared version of VFCache. At the very least, thunder will allow multiple servers to access a shared pool of flash cache. This should allow VMware vMotion and DRS to function, and could be much more than that.

EMC could build a high-availability, high-performance all-flash storage array that may even use InfiniBand as an interconnect. The new Nimbus Data E-Class storage array matches this description perfectly, and their CEO tells me that performance over InfiniBand is indeed comparable to in-server PCIe flash cards. It seems logical for EMC to enter this market, if only to disrupt the momentum of Fusion-io and the rest of the all-flash storage upstarts.

Read more about the Nimbus E-Class: The First Big, Redundant, All-Flash Enterprise Array

The only fly in the ointment here is the recent consolidation of the InfiniBand market. Mellanox bought Voltaire, and QLogic sold out to Intel, putting that protocol on tenuous grounds. Perhaps 40 or 100 Gb Ethernet will emerge as a viable alternative for high-performance connectivity, or perhaps these products will retrench on shared PCI Express instead. Micron recently purchased Virtensys for just such a product, and Xsigo has been making big waves in the area of converged I/O as well. The market clearly need something better than Fibre Channel for maximum performance storage, even if InfiniBand isn’t it.

Stephen’s Stance

EMC VFCache (née Project Lightning) is a fairly simple offering: A server-based PCIe flash card that acts as a read cache with no integration with storage arrays or hypervisors. But EMC’s entrance into the host-based flash storage market is a powerful demonstration of the wave of disruption caused by flash-based storage and high-performance computing. Although I am not all that impressed with the product itself, I would be distressed if EMC had not introduced it.

VFCache illustrations are copyright EMC Corporation and are used here with permission

More solid, independent VFCache coverage:

• EMC Enters The Market With “Me Too” Flash Products (Chris Evans)
• Cache Splash and Complex is the new Simple (Martin Glassborow)

© sfoskett for Stephen Foskett, Pack Rat, 2012. | EMC VFCache (aka “Project Lightning”) Is One Small Step, But an Important One
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Today’s announcement of the E-Class storage array is an important milestone for Nimbus Data and solid-state storage in the enterprise. Until now, most solid-state storage arrays have been fairly small-scale, focused on point performance rather than enterprise-wide capacity. But the E-Class, which scales to 500 TB and sports a redundant, multi-protocol interface, is the first all-flash array to go toe to toe at the top of the market.

The State of Solid

No one would deny that solid-state storage is making a huge impact on the market. With mind-bending performance and reduced power requirements, flash memory matches up nicely with modern data center requirements. But the one missing element has always been capacity: NAND flash is more expensive than magnetic disks on a gigabyte by gigabyte basis.

Check out my 4 Horsemen series for more on the issues of disk!

The solid-state enterprise storage market started with point products targeted at specific needs within the data center. Companies like Texas Memory Systems and Violin have long supported the most challenging database applications with their external arrays, while Fusion-io, Virident, Micron and others stashed flash within the server. These companies were able to sell expensive storage into performance-hungry niches, but have found it difficult to address the capacity needs of the broader storage market.

Technology has the answer to this challenge, as demonstrated by Pure Storage, Nimbus, SolidFire, and others. Thin provisioning makes up much of the difference in cost, and deduplication or compression can even bring parity on a per-capacity cost basis. And even with these features turned on, solid-state storage arrays absolutely murder spinning disks in terms of random I/O performance.

Accumulating Nimbus

Unlike Whiptail, Pure Storage, Kaminario, SolidFire, and the rest of the startup crowd, Nimbus Data is not a new company. Founded by former TrueSAN wunderkind, Thomas Isakovich, Nimbus began as a disk storage startup before transitioning to an all-flash lineup two years ago. The company has steadily improved its product line over the years, adding NFS and SMB for a unified storage experience as well as InfiniBand for extreme performance.

Unlike most other companies in the space, Nimbus builds their own flash memory modules from raw NAND. This allows the company to avoid some of the tricky engineering required to qualify and adapt to the peculiarities of existing consumer or enterprise SSD modules. It also gives the company greater control and better flexibility to launch new capacity points when they are ready, rather than when their suppliers give the go-ahead.

Nimbus is rolling out a dual-PCB SSD module which doubles performance and capacity

The E-Class includes a new dual-PCB module which stripes data internally for better performance and capacity. This bumps each Nimbus E2000X enclosure to 20 TB, twice the capacity previously achieved, in just two rack units. And each of these enclosures draws as little as 100 W, allowing them to be stacked tall without exceeding the power capacity of typical datacenters.

E Is for Enterprise

The real innovation in the Nimbus E-Class is a brand-new “dual active” redundant controller architecture. Most previous solid-state arrays had a single controller, requiring users to mirror two entire arrays for high-availability. In contrast, most enterprise storage systems feature multiple active controllers with no single point of failure.

Read Multipath: Active/Passive, Dual Active, and Active/Active to understand what I’m on about here!

The E-Class introduces a new controller architecture for Nimbus. Each controller services all access to a LUN or volume until a failure is detected, in which case the alternate controller immediately comes online. But both controllers can have active storage at once, in what I call a “dual active” scenario. Although not truly “active/active”, the E-Class is in a different league from older single controller arrays.

Interestingly, the performance of Nimbus’ solid-state enclosures is high enough that the controllers do not need to mirror internal cache or hash tables. They simply write them out to SSD to be picked up in the event of a failure. This simplifies engineering for a dual controller system, and may lead to additional controllers added in the future.

Straightforward Pricing

Solid-state storage provides much more performance per dollar than spinning disk, but most customers still pay on a raw capacity basis. Rather than rocking the boat with alternative pricing models, Nimbus sticks to a straightforward method: $25,000 per controller, plus $100,000 per 10 TB enclosure. A minimum E-Class configuration includes two controllers and one enclosure for $150,000 with no extra cost for software licensing or features.

This seems fairly expensive for 10 TB of storage, but is actually quite competitive even with disk-based storage systems in the high-end, high-feature enterprise market. Thin provisioning increases the usable capacity of the E-Class, and the all SSD architecture means performance will not suffer. Unlike PCIe solutions, the E-Class is a shared, networked device and can be used by many servers at once.

Features and More Features

Speaking of features, Nimbus includes just about anything you could ask for in a storage array:

• In-line thin provisioning and deduplication
• Snapshots and synchronous or asynchronous replication
• 10 Gb NFS (2, 3, and 4) as well as SMB (CIFS/SMB1 and, SMB2)
• 10 Gb iSCSI
• 8 Gb Fibre Channel
• 40 Gb QDR InfiniBand

Although it has not yet been announced, Nimbus has added VMware VAAI support to the HALO operating system found in the S- and E-Class arrays. The company will support the major block storage components for vSphere 5: Block zeroing, atomic test and set, and full copy. Tom told me that Nimbus found the T10 interfaces fairly straightforward to implement but are still working on the NFS primitives. Although Nimbus does not yet have a vCenter plug-in, I expect that one is in the works.

Tom also tells me Nimbus is a “hypervisor hugger“, in that they intend to support features there rather than try to add them to the array. This is a smart choice for a smaller company, and I am glad to see Nimbus embracing the server virtualization market. I imagine an array like the E-Class would totally demolish any competing disk-based array in a virtual infrastructure deployment!

Stephen’s Stance

Nimbus has always been an interesting company, with a longer history in the storage world than most startups. Their switch to all-flash architecture was perfectly timed with the market shift, and the new E-Class comes just at the right moment. Boasting 500 TB of maximum capacity, a fully redundant “dual active” controller architecture, massive performance (even InfiniBand), and complete feature set (once VAAI is released), Nimbus may have hit on their hands.

© sfoskett for Stephen Foskett, Pack Rat, 2012. | Nimbus E-Class: The First Big, Redundant, All-Flash Enterprise Array
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Many enterprise SSD makers are familiar to storage folks like myself, including Hitachi Global Storage Technologies, Imation, Intel, Samsung, Seagate, and Toshiba. Readers of my blog may also recognize Fusion-io, LSI, Micron, OCZ Technology, Texas Memory Systems, and Viridant from my recent coverage, and may have heard of Solid Access Systems, Anobit, Pliant (now part of SanDisk), and others. But some names remain unfamiliar, at least for now. And others, notably Nimbus Data and Violin Memory, are puzzlingly missing from the StorageNewsletter list.

So here’s my rundown of the enterprise SSD makers to keep an eye on in the coming year!

For fun, I am grouping these by how much contact I had with them over the previous year. Although obviously not the most scientific measure of their impact on the storage world, perhaps this will help highlight those that are reaching out to independent bloggers like myself.

Well-Known SSD Players

See also Micron Bursts Into the PCIe SSD Market

First up is a group of companies that I’m quite familiar with, having recently been briefed on their SSD plans and products.

1. LSI
2. Micron
3. Nimbus Data
4. Texas Memory Systems

Next is a list of companies that, although I have contacts of one sort or another, I eagerly anticipate future briefings regarding SSD technology.

1. Anobit
2. Intel
3. Fusion-io
4. Hitachi GST
5. Seagate Technology
6. Stec
7. Violin Memory

SSD Contenders

See also Toshiba Offers “Blade” SSDs (Like Apple’s MacBook Air)

I have spotted the following companies at conferences, in industry publications, and on the web and feel that I am somewhat familiar with their SSD plans. But I welcome any opportunity to get to know them better, and encourage them to contact me via e-mail or by telephone.

1. OCZ Technology
2. Imation
3. Samsung
4. SanDisk / Pliant Technology
5. Solid Access
6. Toshiba
7. Viking Modular Solutions
8. Virident Systems

Who?!?

I was surprised to see this set of companies listed in the StorageNewsletter article, and more surprised when I went to their website and found that they were working on genuine and interesting enterprise SSD products. I would love to get in contact with folks at these companies so I can learn more about what they are up to!

1. BiTMICRO Networks
2. Foremay
3. Macrotron Systems
4. pureSilicon
5. RunCore
6. Smart Modular Technologies
7. Super Talent

Stephen’s Stance

In all, I located 25 companies that build solid-state storage devices for the enterprise market. My quick examination of their websites (or previous knowledge of the companies) reveals a wonderful and vibrant culture of innovation around solid-state storage technology. Watch this space over the next year, since I intend to cover the SSD space in detail!

I would like to thank Jean-Jacques Maleval for spurring my research into these enterprise SSD companies. He is really doing a great job over at StorageNewsletter, and I recommend that you subscribe!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | My Incomplete, Subjective List of Enterprise SSD Companies
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Micron RealSSD P320h is more than a new form-factor for the company; it also introduces their RAIN technology

This morning, Micron announced their “RealSSD” P320h SSD, bringing them into the PCIe storage market for the first time. Already a leading supplier of both consumer and enterprise SATA SSDs, as well as the world’s leading supplier of NAND flash in partnership with Intel, this move puts Micron in direct competition with existing suppliers like media darling Fusion IO.

Introducing the RealSSD P320h

The RealSSD P320h SSD can be seen as a PCIe companion to the existing P300 series of enterprise SSDs, but it is much more than that. Micron brands their entire SSD line “RealSSD”, from the consumer grade C300 and C400 (which Crucial sells as the M4) to the enterprise P300. All include Micron’s own SSD controller ASICs and flash memory modules, offering greater levels of integration and profitability for the company.

The P320h seems similar to the recently introduced P300 in terms of componentry, but the PCIe interface puts it into a different realm of performance. Although Micron’s RealSSD line is lauded for its 6 Gb SATA interface, an internal PCI express card like that P320h blows the doors off any disk interface. Micron’s presentations show the P320h delivering over many times the IOPS of their already speedy P300! As demonstrated already by numerous competitors, there is no substitute to the low latency I/O performance of a PCIe card.

The RealSSD P320h will initially be offered in two models, both in the full-height half-length PCIe form factor. The 350 GB model offers slightly lower random write performance than its 700 GB big brother, but both boast massive performance numbers. Both use 34 nm SLC flash chips and are built on a 32-channel design.

Massive Performance

The P320h delivers astonishing performance, at least in Micron’s PowerPoint illustrations. My own C300 can push 250 MB per second of sequential writes, and the enterprise class P300 is maybe a bit faster. But the P320h is rated at 2 GB per second sustained sequential write performance, nearly an order of magnitude faster. It would be difficult find an application, let alone a server, that could sustain this kind of throughput for long.

Micron boasts industry-beating throughput

Even more important than throughput is I/O performance. SSDs like my C300 excel at servicing I/O requests, and are rated at 30,000 to 45,000 random 4K write IOPS. Again, the P300 is able to match this level of performance while providing five-year reliability for enterprise applications. But the P320h delivers nearly 350,000 4K write IOPS, besting the fastest and most expensive enterprise storage arrays in existence.

Micron's IOPS claims are astonishing

Note that all numbers in these charts are from the manufacturers’ own specification sheets: Micron P320h, Fusion I/O ioDRIVE DUO, TMS RamSan-70, LSI WarpDrive SLP-300, OCZ Z-Drive R2 e88, Virident tachIOn. I have included two of Micron’s RealSSD SATA drives for comparison purposes: Micron P300, Micron/Crucial C300.

Benchmarks should always be taken with a grain of salt, and manufacturer spec-sheet claims are doubly dubious. But Micron makes these claims, and it won’t be long before these devices are independently benchmarked.

Digging Deeper Into the Hardware: RAIN Reliability

The P320h is a first for Micron in a number of ways. Along with being their first PCIe card, it also is the first SSD to feature Micron’s so-called RAIN architecture and RealSSD Manager software. The P320h includes optimized drivers for Windows and Linux to further improve performance, but Micron is leaving it to partners to integrate the SSD with operating systems, hypervisors, or applications.

RAIN is Micron’s answer to concerns about SSD reliability in the enterprise. Although already using ultra-reliable SLC chips, the P320h introduces a raid like technology Micron calls Redundant Array of Independent NAND, or RAIN for short. Reminiscent of SandForce’s Redundant Array of Independent Silicon Elements (RAISE), RAIN arranges flash memory chips in a 7+1 arrangement to improve reliability and recoverability in the event of a failure.

Micron claims that this combination of intelligent controllers, RAIN, and SLC memory chips delivers top-notch reliability. The company measures reliability in terms of the number of full drive fills that can be sustained per day for five years. They claim that enterprise customers demand SSDs that can sustain 30 fills per day for five years, or an incredible 54,750 drive fills. That’s 25 PB of data written for 350 gig unit or 50 PB for the big 750 GB P320h!

Putting things another way, if the 700 GB P320h was pounded with sequential data at its rated maximum of 2 GB per second, it would take over nine months to wear out this drive. Just in case the customer expects to hammer on the drive constantly, they may use the RealSSD Manager software to throttle performance and ensure reliability to a given date.

Stephen’s Stance

PCIe SSDs like Micron’s new P320h offer mindbending performance and enterprise class reliability. Although expensive, these devices are in an entirely different league from any other storage option. Micron promises to bring the PCIe P320h to market at nearly $15 per gigabyte, a substantial discount over other PCIe SSD competitors. But the card will still cost more than $5000, making it an expensive add-on for most servers.

The challenge in enterprise storage is not delivering absolute capacity or performance any longer. Today’s challenge is making that capacity and performance available to applications and, ultimately, and users in the form of improved satisfaction or profitability. Micron is moving the ball forward on the hardware front, but my focus remains on software providers that will deliver this performance in a usable form for applications, hypervisors, and operating systems.

And what’s next from Micron? I expect a SAS HDD-form factor SSD shortly, and perhaps a line of PCI Express Mini Cards or “Blade” SSDs might follow.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Micron Bursts Into the PCIe SSD Market
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Get used to it: Commodity hardware like this Super Micro SBB server will dominate enterprise storage

The history of technology moves in fits and starts, but one trend trumps all else: An inevitable shift from fine-tuned specialized gear to general-purpose commodity building blocks. We see it in both hardware and software, and at all levels of the industry, from chips and wafers to operating systems and networking devices. Take a step back and you’ll certainly agree: Commodity hardware always wins (eventually).

Doing the Impossible

Building something new often requires amazing feats of engineering. Steve Wozniak’s hardware wizardry with the original Apple computer is now legend, but similar tales are found everywhere.

It’s frightfully difficult to do the impossible without at least some wizardry. Most new technologies are therefore built on specialized hardware and hand-tuned software. Engineers at the forefront of technology must eke out every ounce of performance from trailing technologies.

This is why we see so many special-purpose processors in high-performance devices, and why companies employing custom ASICs often enjoy a performance advantage. In enterprise storage, we look to companies like HDS and BlueArc who packed their arrays with special-purpose hardware, pointing the way to the future. We also see impressive developments from Fusion IO and SandForce in the SSD space, leading to the next generation of storage.

Can You Keep Ahead Of Intel?

Just about every technology sector progresses from the impossible to the commonplace, and these changes are often very quick. High-performance storage systems were once exotic multi-million dollar devices but millions of IOPS are now available for under $100k from dozens of vendors.

This progression typically includes a move from special-purpose to commodity underpinnings. Exotic real-time operating systems have been pushed aside in favor of Linux, BSD, and even Windows, while ASICs and FPGAs give way to Intel’s CPU juggernaut. Even Apple computers are today almost entirely commodity PCs.

I love how Denton Gentry phrases this in his Monday blog entry about Intel and Achronix:

“Once you commit to a specialized hardware design, the clock starts ticking. There will come a day when a software implementation could meet the requirements, and at that point the FPGA becomes an expensive liability in the BOM cost. You have to make enough profit from the hardware offload product to pay for its own design, plus a redesign in software, or the whole exercise turns out to be a waste of money.”

In other words, specialized software on proprietary hardware will eventually be overtaken by general-purpose software on commodity hardware. The decision must include not just what one can do today but what that baggage will mean in the future. Designing a system around proprietary components might look good now, but the next-generation product will be put at risk by this decision.

Stephen’s Stance

My opinion is right there in the title of this piece: Commodity hardware always wins. No matter how great your ASIC is, industry-standard CPUs will out-perform it sooner or later. No matter how much effort you put into tuning your software, Linux-based systems will eventually do just as well.

The rise of commodity hardware is everywhere: EMC, HDS, IBM, Oracle, and HP have all embraced Intel CPUs and their hardware is looking more and more like Intel’s reference designs, too. Startups are increasingly relying on software rather than hardware for their differentiation, and we’re seeing Supermicro servers shipped with just about everyone’s name on them. The Storage Bridge Bay specification is looking better all the time, too. Commodity hardware is winning in storage, just like it always does.

For more on this topic, see these related posts by others:

• The World Has Changed – Is Hardware Getting Softer?
• Commodity hardware always loses
• Storage is software
• Better storage through hardware

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Commodity Hardware Always Wins
This post was categorized as Computer History, Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Unconventional SSDs

Most conventional SSDs use conventional disk interfaces, normally serial ATA (SATA). These plug right into just about any modern computer using standard internal SATA power and data connectors. But small form factor PCs like the MacBook Air need a more-compact SSD, leading to some clever (and confusing) engineering.

Most modern computers use the PCI Express and USB busses for internal peripheral connections other than storage. Most folks are familiar with the full-size USB ports and PCI Express slots found in desktop machines, but portable computers (and a few others, including my iMac) need something more compact.

Introducing PCI Express Mini Card

This led to the development of the PCI Express Mini Card standard, commonly called Mini PCI-E. Like the similar ExpressCard format, Mini PCI Express includes both USB and a single PCI lane, along with a host of other pinouts for flexible use in portable machines:

• A single PCI Express lane
• USB 2.0
• System Management Bus (SMBus) – a 2-wire lightweight communication bus derived from I2C for power-on/off commands
• Pins reserved for GPS use
• Pins for diagnostic LEDs showing Wi-Fi activity
• Pins to communicate with a GSM SIM card slot located elsewhere
• Reserved pins for an additional PCI Express lane
• 1.5 and 3.3 volt power

This is one full-featured slot, as you can see, packing many useful features into a compact connector.

Nearly every modern laptop has at least one Mini PCI-E slot, and they are frequently used for Wi-Fi and 3G radio devices. Apple often uses them for their “AirPort” Wi-Fi adapters, so machines as diverse as the desktop iMac include Mini PCI-E slots.

Mini PCI-E SSDs

mSATA SSDs like this Toshiba model reuse reserved Mini-PCIe pins for SATA connections

As SSD performance increases, conventional disk interfaces like SATA and SAS are becoming increasingly-saturated. And block storage protocols like ATA and SCSI make little sense when communicating with NAND flash devices like SSDs. So PCI Express SSDs have become increasingly popular lately. Companies like Fusion IO and SandForce are aggressively pushing these devices, and the performance numbers are impressive.

Unsurprisingly, the availability of a compact PCI Express Mini Card slot in portable machines has led to the development of tiny PCI Express SSDs for these applications. A wide variety of options are available but, as with ExpressCards, caution is in order since not all are equal. Manufacturers have developed a variety of Mini PCI-E SSDs using the various pins in this tiny slot:

• Native PCI Express SSDs can use the single included PCI Express lane and are compatible with most computers
• Cheaper USB SSDs (essentially glorified thumb drives) can use the included USB pins and are also compatible with most computers, though much slower
• A de-facto standard for SATA over Mini PCI-E called mSATA is seeing wider adoption but is incompatible with standard Mini PCI-E slots since it uses (reserved) pins 5 and 7 for SATA communication
• Various alternative methods of sending SATA signals have been developed, including the format used in the popular Asus Eee PC, which are also widely incompatible

This proliferation of Mini PCI-E SSDs has caused confusion and frustration among users. Many have purchased Mini PCI-E to SATA adapters designed for the Eee PC, for example, hoping to connect SATA devices to their machines. Finding compatible devices is a trial-and-error process and is played out repeatedly in forums dedicated to machines like the JooJoo and Mac Mini.

Stephen’s Stance

Apple chose another format entirely for the new MacBook Air, designing their own connector rather than selecting a standard like PCI Express or even mSATA. The variety of incompatible Mini PCI-E cards is frustrating to enthusiasts like myself, and I had hoped that the entry of a major company like Apple would bring sanity to this disorder.

As with full-sized PCI Express SSDs, Mini PCI-E SSDs hold great promise for performance, reliability, and power consumption and their compact size is perfect for a variety of applications. I look forward to examining and reviewing these devices in the future!

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Unconventional SSDs: PCI Express Mini Card (Mini PCI-E)
This post was categorized as Apple, Computer History, Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Hard disk drive makers are adding flash storage to their conventional spinning-platter drives to improve performance and are targeting the performance PC market. Wait a second, haven’t we seen this before? As Rocky eventually said to Bullwinkle, “but that trick never works!”

Flash as a Cache

Using flash memory as a disk cache is a pretty good idea. Flash has awesome random read performance and fairly good write speed (compared to a hard disk drive, at least). That’s why more and more enterprise storage vendors are adding flash as a disk cache, not just a plain tier of storage.

EMC is the latest to make the move, announcing “FAST Cache” for their midrange Clariion and Celerra enterprise storage systems last week. They join NetApp, Sun, and others already offering similar capability. Fusion-IO has been the champion PCIe flash provider, but STEC is expected to join them soon.

See my post, Is Flash A Disk Or A Cache?

Flash-as-a-cache hasn’t been as easy to roll out as flash-as-a-disk, but it promises to be more effective. An array that completely integrates flash can take advantage of its positives (fast random read, fast-ish write, low power) without stumbling over its shortcomings (big write blocks, shorter lifespan).

Flash in a Disk

Although EMC is doing the right thing by adding FAST Cache, their implementation uses disk drive form factor flash rather than the PCI cards selected by others. It may prove more-difficult to optimize the system for the characteristics of flash when one is writing through a conventional disk drive interface like Fibre Channel or SAS. Would-be flash-and-platter drives face the same issue: How do you use flash effectively when it’s abstracted from the server and presented as a conventional disk?

The hybrid hard disk drive (H-HDD) method, rolled out back in 2007, added ATA commands allowing a compatible operating system to specify whether data sent to a hybrid drive should be written to flash or disk. These products were paired with Windows Vista’s ReadyBoost and SuperFetch to produce performance gains that never materialized in practice. The so-called “ReadyDrive” has become a footnote in history, along with Intel’s “Robeson” effort to add a flash cache to the motherboard.

It is unclear what the new generation of hybrid hard drives allegedly on the drawing boards at Toshiba and Seagate would look like. It is unlikely that they would use the H-HDD interface, but they will likely be aimed at the same performance laptop and desktop market. Servers have continued migrating towards advanced enterprise storage systems that pack their own cache, reducing the impact of bare hybrid drives.

Ingredients for Success

Rather than repeat the mistakes of the past, these companies could integrate real smarts into the disk controller, allowing it to autonomously move data to the flash cache to improve everyday performance without any special operating system support. This “tiered storage in a can” approach might deliver the goods that old-fashioned H-HDDs never could.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Hybrid SSD/Hard Disk Drives: This Time For Sure!
This post was categorized as Enterprise storage, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Register did a nice job of summing up the (late 2008) flash positioning of the various storage companies, and I recently posted a strategic look at this core issue. Note that some, like HP and Sun (and probably IBM), seem to have an end-to-end strategy, while others are firmly in one camp or the other. In the “not yet” column, apparently, are 3PAR, BlueArc (though they offer TMS RAM), Dell/EqualLogic, HP/LeftHand.

Update: 3PAR has joined the “disk” camp.

It’s a Cache

Fusion-IO has rocketed to the forefront of the cache side with their PCI Express flash boards for servers. Joining them in this position are the following companies:

• HP (blades and servers)
• NetApp (PAM read cache card)
• Pillar (Slammer cache)
• Sun (read and write cache)

It’s a Disk

STEC is the darling of the flash-as-a-disk world, though Intel, Marvell, and Samsung are also playing here. Joining them in the corner are the following:

• EMC (STEC flash drives shipping the DMX now and CLARiiON in the future)
• Compellent (flash drives)
• HDS (flash drives in the USP-V)
• HP (flash drives in the MSA, perhaps, and maybe that Oracle thing)
• IBM (Fusion-IO storage behind SVC and standard flash drives in the DS5000)
• LSI (flash drives in the 7900)
• NetApp (ssd drives)
• Pillar (flash drives)
• Sun (flash drives in Thumper and JBOD)
• Xiotech (flash drives in the next Emprise canisters)
• 3PAR (flash drives in InServe)

So there you have it. It’s both a disk and a cache, depending on whether you sell servers or arrays apparently. If you sell both, it’s both. Simple!

If anyone wants to correct any of this, drop me a line or comment below!

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Is Flash A Disk Or A Cache?
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Fram approaching in front of iceberg upernavik, CC-by-SA copyright Kim Hansen

Solid state (NAND flash) storage is all the rage right now, but there are many lingering questions regarding its true performance, reliability, and cost. But no question is more important in determining its ultimate usefulness than that of location: Where should flash storage be placed to maximize return on investment?

Storage companies have argued that flash disks can be used most effectively in external storage devices, arguing that it’s simpler to just leverage existing storage technologies. Server companies have tended to prefer to place it inside the server, asking why, if flash disks are capable of massive random I/O performance and extremely low latency, one would put them at the other end of a Fibre Channel or iSCSI connection, which introduces latency and tends to combine I/O operations?

The Case For Servers

The argument for placing flash in (or very close to) servers boils down to two key contentions:

1. Distance = latency, so moving quick flash devices away from I/O-hungry CPUs erodes their effectiveness
2. Granularity (or lack thereof) is the core problem facing storage management, so moving flash (and other types of storage) closer to the omniscient application is likely to bring greater effectiveness

The server folks are relying on a technical argument – that placing high-speed cache where it could theoretically do the most good is the right decision. And they are right, in a perfect world: A flash-aware application talking to a low-latency flash device over PCI ought to really fly!

There is some disagreement in the server-side argument as well: Is flash a “fake disk” or a new level of caching between RAM and storage? It seems that the pitch leans toward the latter, even when the SSD appears as a disk drive. This is what Fusion-IO is pitching: They skip old-school disk connections like SATA and SAS altogether, placing their storage on PCI Express and asking hardware and software vendors to integrate it as best they can. Consider Sun’s flash integration for ZFS, for example. Note, by the way, that Intel’s Turbo Memory products also offer PCIe flash, despite what you might be hearing.

Of course, we can just use a flash drive in place of an internal hard drive. Just about everyone makes something like this now, and they work pretty well in some cases. Then there are hybrid drives, which have gone nowhere so far.

But will this work? We need operating systems and applications that can make use of this local flash, and that has been a problem. Intel’s flash-on-the-motherboard idea never caught on, even as Vista included ReadyBoost, because the truth is that operating systems, file systems, or applications must be re-engineered to really make use of flash in a server. That’s happening, but slowly.

The Case for Arrays

Then we turn to the other end of the storage pipe. EMC put flash in the DMX in January, and Compellent is doing it as we speak. IBM went wild with a bunch of Fusion-IO drives and an SVC over the summer, too. All this proves that flash works in storage arrays!

Why? Simply because modern storage arrays are already engineered to make good use of disk drive capabilities. This is a “what works” strategy – even though it doesn’t sound as nice in theory, the array doesn’t need lots of re-engineering to see some benefit from flash. And post-RAID virtualized systems like that Compellent can really make hay with a few super-speed flash drives, since they can move hot blocks to flash dynamically.

Sure, there is latency between the CPU and the flash drive, but storage arrays are really computers in their own right. So they can derive the same benefit from flash that a server could, and they can share that benefit to connected servers rather than leaving it locked up.

Why Not Everywhere?

How about we end the debate. Flash works great in the server, and it works great in the array. Why not just put it anywhere it makes sense in your particular environment? Have an operating system, application, or file system that can make use of server-side flash? Go buy a Fusion-IO card! Have a virtualized enterprise storage array? Get some SSD there, too. And remember that it’s not all about NAND flash – RAM-based solid state storage from companies like Texas Memory Systems, Gear6, and Violin are even faster!

But remember one thing: This stuff is still very very expensive, so you have to really need the performance to make a case for flash.

Image by Kim Hansen, GFDL or CC-BY-SA (source)

© sfoskett for Stephen Foskett, Pack Rat, 2008. | SSD: So Close and Yet So Far
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.