This is the second entry in my Top-Ten in Storage series.

Not every innovative product can succeed in the market, and no matter how good some ideas seem, they can fail to make much of an impact. The truth is, people buy solutions, not technologies.

This list includes products so cool, so ahead of their time, that they just couldn’t fail. But they did.

1988 EMC Orion/Allegro

EMC’s RAM-based storage array predated and presaged its Symmetrix, offering lightning-quick I/O for very small workloads. But the Orion had a few serious drawbacks: It was astonishingly expensive, offered tiny capacity, and was never available for open systems. EMC’s initial offerings in the open systems market, Centriplex and Harmonix, also deserve mention though the company proved its worth with its “best” Symmetrix a few years later.

1985 3COM 3Server

I’ve written about the 3Server before, but let’s sum up: Before RAID, SAN, and NAS, and even before NetWare and NFS became entrenched, 3COM introduced a flexible multi-disk network storage and print server called the 3Server. It was an innovative combination of hardware and software, but it lost out to NetWare on open PC hardware in the market. Although the 3Server lasted just a few years, it made a lasting contribution to the field: Its 3+Share software became 3+Open, then LAN Manager, and was finally absorbed by Microsoft to become SMB/CIFS.

HP AutoRAID (High Availability Disk Array Model 12)

The first post-RAID virtual sprang fully-formed from the big brains at HP to challenge the industry-leading modular offerings from Data General’s CLARiiON and DEC/Compaq’s StorageWorks. It one-upped everyone with automatic relocation of LUNs between different RAID levels based on workload about a decade before Compellent, EqualLogic, and 3PAR had anything of the sort. But the AutoRAID had two key drawbacks: Like mama bear, it wasn’t big enough for big companies but was too big and expensive for little ones, and it was SCSI-only (though a Fibre Channel bridge was a common accessory). Some of the concepts lived on in the HP EVA line, but AutoRAID didn’t last long.

Microsoft Object File System (OFS)/Relational File System (RFS)/WinFS

Microsoft shoulda-coulda-woulda revolutionized file storage with nearly every major operating system release. It looks the same every time: A database/filesystem hybrid that would add structure to the vast unstructured file world. Despite the hype, we haven’t seen it yet. 

MangoSoft Medley

Consider the typical LAN: Lots of clients with lots of hard drive space connected to a server with yet more disk capacity. Wouldn’t it be nice if the space on these clients could be used as a virtual network server? Well that’s just what MangoSoft announced in 1997, gaining much attention in the PC press. Sadly, the idea never really caught on, although MangoSoft continued. But distributed network storage has become the industry’s Brigadoon, trotted out as a new idea year after year.

StorageNetworks

This is my flop – I worked for StorageNetworks from 1999 through 2001.

StorageNetworks was launched in 1998 to provide off-premise storage and backup as an on-demand service for enterprise customers. They sought to take advantage of the emergence of high-speed Fibre Channel connectivity over metro distances to commoditize storage capacity. But only Houston (where I worked) had an acceptable infrastructure for the service, so the company changed focus to hosting centers. Then off-site backup. Then software. Nothing worked apart from vendor-independent services, and the (now public) company was shy to base its revenues on that. StorageNetworks was gone in 2003, but does the service remind anyone of Amazon S3 or Nirvanix? Maybe it was just ahead of its time.

Compaq VersaStor

Announced in 1999, VersaStor would have been a revolution in Fibre Channel SANs, a full out of band virtualization solution leveraging specialized HBAs directing traffic. It was continually pushed back, finally being “merged” (after Compaq) with HP’s StorageApps in-band SANLink to become CASA. But then EMC sued HP over virtualization patents, derailing CASA, and the whole mess was permanently shelved at the end of 2003.

IBM Ice Cube/Collective Intelligent Bricks Hardware

IBM rocked the storage press in 2003 with their announcement of Almaden Research’s Ice Cube concept. Instantly dubbed “Lego brick storage” (trademarks be damned!), the storage units could be stacked in two dimensions, scaling without limit. Coolest of all (literally), the bricks were chilled with water! Although the concept progressed, we still haven’t seen it. But this didn’t stop Seagate from developing a similar concept, ISE, which is now Xiotech’s main offering. IBM’s Storage Tank and VSS could have been nominated, too, but I’m not a sadist.

Revivio CDP

Revivio was the pioneer of continuous data protection (CDP), with great technology and people. But no one (other than Symantec, eventually) bought it.

Pirus

Little Pirus was working on a small but scalable virtualized target when it was acquired by Sun in September of 2002. Their technology was launched as the StorEdge 6920, a mini alternative to the HDS USP in 2004, but (according to insiders) it didn’t exactly light the world on fire. Sun finally pulled the plug on the 6920 in early 2007, with HDS taking over continuing support for anyone who bought the moribund product.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Top Ten Coolest Enterprise Storage Flops
This post was categorized as Computer History, Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

It has been the core technology behind the storage industry since day one, but the sun is setting on traditional RAID technology. After two decades of refinement and fragmentation, we are abandoning the core concepts of disk-centric data protection as storage and servers go virtual. Next-generation storage products will feature refined and integrated capabilities based on pools of storage rather than combinations of disk drives, and we will all benefit from improved reliability and performance.

RAID Classic

Early storage systems were revolutionary, in physically removing storage from the CPU, in enabling sharing of storage between multiple CPUs, and especially in virtualizing disk drives using RAID. When Patterson, Gibson, and Katz proposed the creation of a redundant array of inexpensive disks (RAID) in 1987, they specified five numbered “levels”. Each level had its own features and benefits, but all centered on the idea that a static set of disk drives would be grouped together and presented to higher-level systems as a single drive. Storage devices, as a rule, mapped host data back to these integral disk sets, sometimes sharing a single RAID group among multiple “LUNs”, but never spreading data more broadly. Storage has remained stuck with small sets of drives ever since.

The core insight of the 1980s remains true: More spindles means better performance. Although additional overhead dulls the impact somewhat, the benefit of spreading data across multiple drives can be tremendous. A typical RAID set offers much better performance than the drives alone, and can handle a mechanical failure as a bonus.

Cracks are appearing in the RAID veneer, however. Double drive failures are much more common than one would expect, leading to the development of hot spare drives and dual-parity RAID 6. If four drives perform well, then forty drives perform much better, leading to the common practice of “stacking” one RAID set on others. Caches and specialized processors were introduced to overcome the performance issues related to parity calculation.

But traditional RAID cannot overcome today’s most critical storage issues. As drives have become larger, the tiny chance of an unrecoverable media error compounds, becoming a certainty. Even dual-parity will not be able to guarantee data protection on the massive disks predicted for the near future – statistics cannot be denied. The latest disks contain so much data, without commensurate improvements in throughput, that rebuild times have skyrocketed, resulting in hours or days of reduced data protection.

RAID is also ill-suited to the demands of virtualized systems, where predictable I/O patterns become fragmented. It cannot provide tiered storage or account for changing requirements over time. It cannot take advantage of the latest high-performance solid state storage technology. It cannot be used in cloud architectures, with massive numbers of small devices clustered together. It interferes with power-saving spin-down ideas. Most RAID implementations cannot even grow or shrink with the addition or removal of a disk. In short, traditional RAID cannot do what we now need storage to do.

RAID is Dead

Although most vendors still use the name, nearly every one has abandoned much of the classic RAID technology. EMC’s Symmetrix pioneered the idea of sub-disk RAID, pairing just a portion of each disk with others to reduce the impact of “hot spots”. HP’s AutoRAID added the ability to dynamically move data from one RAID type to another to balance performance. And NetApp paired disk management so closely with their filesystem that they were able to use RAID 4 and the flexibility it brings.

Today, a new generation of devices has even evolved beyond RAID’s concept of coherent disk sets. Compellent, Dell EqualLogic, 3PAR and others focus on blocks of data, moving portions of a LUN between RAID sets, disk drive types, and even inner or outer tracks based on access patterns. With these devices, a single LUN could encompass data on every drive in the storage array. And the latest clustered arrays can spread data across multiple storage nodes to scale performance and protection.

These innovative devices point the way to a future in which virtual storage is serviced and protected very differently than in the past. Perhaps software like Sun’s ZFS serves to illustrate this future best: It unifies storage as a single pool, intelligently protecting it and presenting flexible storage volumes to the operating system. Although Sun calls its data protection scheme “RAID-Z”, it has little in common with its namesake. Like NetApp’s WAFL, the copy-on-write ZFS filesystem is totally integrated with the layout of data on disk, allowing mobility and efficient use of storage. A single pool can include striping, single- or dual-parity, and mirroring, and disks can be added as needed. Importantly, ZFS also checksums all reads, detecting disk errors.

Long Live RAID

The post-RAID future will see these concepts spread across all enterprise storage devices. Disks will be pooled rather than segregated into RAID sets. Tight integration between layout and data protection will allow for much greater flexibility, integrating tiering and differing data protection strategies in a unified whole. Storage virtualization will allow mobility of data within these future storage arrays, and clustering will enable massive scalability.

Two things will likely remain to remind us of Patterson, Gibson, and Katz, however. First, the core principle that multiple drives working as one yields dividends in terms of performance and data protection. And second, that whatever we use should be called RAID, even though the definition of that term has changed beyond recognition in the last two decades.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Turning the Page on RAID
This post was categorized as Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.