I think this sentiment is just as valid today as when I posted it in 2008!

Although many discussions in the storage industry focus on the relative merits of one protocol or another, the conversation occasionally turns to the core issue at hand: We continue to patch together a system based on outdated concepts. Most storage protocols continue to mimic direct attached storage, and most of our so-called networks act as point to point channels. An ultra-modern virtualized storage infrastructure with all the latest bells and whistles still holds the concepts of block and file at its core. Whenever the storage industry has tried to bring about real storage management they have been stymied by a lack of context for data.

No amount of virtualization, and no new protocol, will fix this. Put simply, we need a storage revolution.

Channels, Blocks, and Files

Most innovation in the 1980s and early 1990s focused on moving storage out of the server. SCSI allowed disk to exist in a separate cabinet, RAID allowed multiple physical disks to become a single virtual one, and these were mixed to become the prototype storage array. Although SCSI allowed one-to-many connectivity, it was never a true peer-to-peer network, even once it was mixed with network concepts in the form of Fibre Channel.

Even today, SAN storage is focused on providing faster, more flexible, and feature-packed direct-attached storage. A modern virtual SAN hides a complex arrangement of caching, data protection, tiered storage, replication, and deduplication, masquerading the lot as a simple, lowly disk drive. It is sad but true that all of our work as an industry has been dedicated to recreating what we started with.

Networked file-based storage is no better. Although NAS devices have all the advanced features of their SAN cousins, they must present a simple file tree to the host to retain compatibility. File virtualization merely presents a larger homogenous tree.

Inside the server, too, features and complexity are hidden to retain a familiar file system format. Volume managers can do anything a virtualization device can, but must present their output as a simple (though virtual) disk drive. File systems, too, have added features but still present a familiar tree of mount points, inodes, and files. Even ZFS, possibly the most advanced combination of volume management and file system technology yet, must present a simple tree of storage to applications.

The Metadata Roadblock

This outdated paradigm, of disks and file trees, is ill-suited to today’s storage challenges. Data must be categorized so actions can be taken to preserve or destroy it based on policies. Data must be searchable so users and applications can find what they want. Data must be flexible so it can be used in new ways. Our antiquated notions are not capable of meeting these challenges.

One simple problem is that we lack context for our data. Most file systems merely assign to a file a name, location, owner, and security attributes. The most advanced can contain extended metadata, but this is rarely seen in practice since many applications cannot agree on how to use this data. Microsoft’s Office suite can store and share extended file attributes, for example, but these live inside the file rather than in the file system. The promise of expanded Office attributes is only realized in conjunction with a content management system like SharePoint which lies above the lowly file system.

What if the storage system could keep this data instead? What if it could logically group files according to project or client, mining keywords and authors, and maintaining revisions? These concepts are not new, having been implemented in content management systems for years, and certain elements appeared in file systems, like Apple’s HFS and VMS’ Files-11, for decades.

Cut Down the Tree

File metadata would allow advanced features, but truly taking advantage of them requires a more fundamental shift in the way applications access files. Rather than sticking to a traditional hierarchy of directories in a tree (which was, after all, simply a primitive metadata system), we should remove the tree altogether. Allow files to become data objects, identified by arbitrary attributes and managed according to an overarching policy.

This future vision is decidedly different from our current notion of storage, but is not so far off. Many organizations now rely on central data warehouses based on SQL-language relational databases. As many storage managers have grumbled, databases tend to ignore storage management concepts entirely, managing their own content independently.

But not all applications need a database back-end, so another initiative seeks to provide generic object storage for wider use. Called content-addressable storage or CAS, these devices have traditionally been used only for archival purposes, since that was their first market application. As vendors break free of proprietary interfaces in favor of open ones like XAM, CAS could transform storage itself by eliminating both file and block storage at once.

Similar concepts are already at work in the so-called Web 2.0 world. Non-traditional databases like Google BigTable, Amazon S3, and Hadoop allow massive scalability for object storage. API-sharing initiatives with many Web 2.0 companies can be seen as similar prototypical object storage frameworks. Any of these could be leveraged to provide a new world of data storage, and many are gaining traction even now.

Stephen’s Stance

Although traditional block storage is here to stay for disk drives, and tree-type file systems are likely to remain the foundation of operating system storage, new object-based concepts could change the world in fundamental ways. As applications become “web aware”, they also become object aware, increasing the likelihood of such a storage revolution. For the majority of applications, this new world would be a welcome one indeed.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | We Need a Storage Revolution
This post was categorized as Apple, Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

You may not know Caringo, but you have probably heard of cloud storage, EMC Centera, and Dell DX. Read on to learn the link!

Now that the hype of “cloud everything” is subsiding, organizations are getting down to work deploying cloud storage to do actual useful tasks. The march from CAS to cloud to object storage has seen high-profile high-end flare-ups (think EMC Centera and Atmos) but the bulk of work is done by more pedestrian (think lower-cost) hardware and software. Through it all, Paul Carpentier has been at the forefront. Now his company, Caringo, is back in the news, delivering much-needed storage service features like multi-tenancy, named objects, and dynamic caching.

For essential background, check out my article, From CAS to Cloud: Revolutionary Storage

The Back-Story of Caringo

The “Caringo” company name refers to its three founders, CTO Paul Carpentier, President Jonathan Ring, and CEO Mark Goros. Carpentier is the man behind CAS pioneer FilePool, which EMC acquired and markets as Centera. The three formed Caringo and launched the CAStor product in 2006 as a software alternative to Centera.

Although you may not have heard of Caringo, you may have encountered their product in the form of the Dell DX object storage system. Many were puzzled when Dell, known for its EMC-powered storage offerings, embraced Caringo for object storage, but those familiar with the products weren’t surprised. Caringo’s approach is much more in line with Dell’s image of affordability, simplicity, and commodity products, and their relationship with EMC is increasingly shaky due to their recent acquisition strategy.

Caringo’s CAStor is a software product that transforms commodity servers into a scale-out object repository. It is all-inclusive, with compliance, tiering, spin-down, and replication part of the total package. Like most CAS and cloud storage solutions, CAStor uses a simple HTTP interface for client access, with “gateways” available for NAS along with some native support from applications.

What’s New in CAStor 5?

Caringo has set a course for the service provider market, adding essential features like multi-tenancy and flexible permissions to version 5 of CAStor. Although still pitched as an object store, CAStor 5 is close enough to be thought of as a cloud storage platform.

The ability to support and segregate multiple “tenants” is a holy grail for service provider storage systems and a key ingredient of cloud storage solutions. CAStor 5 can be segmented into multiple domains, each with its own security and authentication and each subdivided into “buckets” for different applications. This would be useful both for a public service provider and an internal-only solution, since segmenting applications is relevant in the enterprise as well.

CAStor 5 no longer clings to system-assigned names for objects, allowing users to assign their own names for public consumption. This is a huge advancement for CAS, and was one of the key differentiators of cloud solutions which often directly serve content to web clients. Another “ripped from the cloud” feature is dynamic caching, allowing high performance access to popular content, again useful for direct client access.

Stephen’s Stance

Caringo seems reluctant to wear the “cloud storage” mantle, but their product has been steadily moving in that direction. CAStor 5, with its multi-tenancy, segmented security and authentication, named objects, and caching, looks an awful lot like Amazon S3 and the rest. But the hype around “cloud storage” is dying away. Businesses looking for functionality rather than marketing labels will find a lot to like in CAStor.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Caringo Bulks Up CAStor For Cloud Services
This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

We need to move beyond fake disks and deploy application-centric storage

Change is not a word normally associated with storage, and revolution is practically unheard of. Today’s modern enterprise storage systems and networks employ massive resources to do one simple thing: Emulate the basic hard disk drives used over three decades ago. But cracks are appearing in our mausoleum of fake disks: Application developers are discovering the value of object storage, and storage systems are appearing to support this need.

I also wrote about this two years ago, proclaiming that We Need a Storage Revolution and forecasting The Techie/Business Schism

The CAS Revolution

Caringo founder and CTO, Paul Carpentier, rose to prominence around 2000 at FilePool, one of the prime movers in the content-addressable storage (CAS) space. I recall a light going off in my head as Paul introduced me to FilePool’s CAS technology back then, imagining the possibilities of the concept. Files would be stored based on “what they were” rather than “where they were” and could be organized according to application needs rather than the conventional “extent of blocks” or tree heirarchy.

CAS discarded decades of filesystem and block storage baggage, introducing a new method for storing and retrieving data that better-matched the burgeoning web and enterprise applications of today. I had seen the failure of the first wave of storage service providers from inside StorageNetworks, and it was this desire for a real storage revolution that led me to dive into cloud storage at Nirvanix almost a decade later. Although I am now on my own, I remain convinced that the future belongs to storage systems that look nothing like today’s SAN and NAS.

Shortly after that 2001 meeting, EMC acquired FilePool and launched it as the Centera product line. But CAS systems quickly ran into a serious roadblock: Conventional applications cannot read and write to unconventional storage systems like Centera. EMC pushed key software vendors (especially in the archiving space) to create special Centera interfaces, and the industry bogged down developing the XAM standard. Other companies, like Seven Ten Storage Software, jumped in to help with the translation from proprietary CAS interfaces, but the transition from legacy files and blocks to object storage has been long and slow.

Cloud Storage: Another Dimension

Meanwhile, in an alternate dimension, web developers realized they had a serious problem. They were developing applications that scaled massively, spanning servers and exhausting conventional filesystems. Conventional systems just wouldn’t cut the mustard.

Since they were soaking in web applications, these developers applied the lessons of web services to storage: Why not just make an HTTP connection and ask for an object by a unique ID rather than walk a filesystem tree? Why not encapsulate the “state” of this request in the request itself rather than make a lasting connection and association between the client and server?

Thus was born cloud storage, and it was bookseller Amazon who opened the floodgates with their 2006 introduction of a “Simple Storage Service” or S3. They allowed anyone to store and retrieve objects from their massive web services infrastructure. S3 and similar services from Rackspace, Nirvanix, and others, are special-purpose web servers, and their simple interfaces are wonderfully attractive to web developers. For example, this WordPress-based blog uses cloud storage to serve images to your browser!

Similarities in CAS and Cloud

Although developed from vastly-differing starting points, CAS and cloud storage are essentially similar: Both reject conventional blocks and files in favor of object storage; both organize data with metadata databases; both multiply and scale out. There is one other major similarity between CAS and cloud storage: Both are attractive to service providers.

Imagine you operate a business that stores data for customers. You would want a flexible infrastructure that would scale with demand and segment each “tenant” from others for security and performance. As we learned at StorageNetworks, conventional SAN and NAS systems just weren’t meant to work in this kind of environment. Whether operating an internal service or a public cloud, service providers require something entirely different.

Cloud storage was designed from the start with service providers in mind, embedding per-object and per-”bucket” security, scalability, and abstraction between hardware and clients. Although quite complex to design, cloud storage is amazingly simple to use, provided an application can interface with it.

CAS wasn’t designed like this. Systems like EMC’s Centera were created for the needs of applications like enterprise archiving, but secure storage of content and extreme scalability are critical here as well. But early CAS systems didn’t need simple web-style interfaces or extreme hardware abstraction. These were enterprise systems, after all.

The CAS/Cloud Colission

CAS wasn’t exactly successful. Although object storage found a niche in enterprise archiving, the enterprise storage world has mostly continued with blocks and files. The major storage vendors all have some kind of object storage, but most are repurposed NAS rather than dedicated CAS like the Centera.

Although much skepticism has been raised about cloud storage in the enterprise, its impact on application development cannot be denied. Indeed, the majority of developers are now focused on programming platforms that abstract both compute and storage from conventional operating systems. The next generation of applications will run in “platform as a service” environments first, and cloud storage is a key component.

Storage vendors are rapidly moving to rework their conventional systems for cloud use. Although block and file systems from 3PAR, NetApp, Isilon, Symantec, HDS, HP, and others are useful in cloud environments, unconventional CAS becomes more valuable here. This is where EMC, Mezeo, and Caringo (with Dell) shine, and why HDS bought Parascale, NetApp bought Bycast, and what Overland could do with MaxiScale. In the mean time, “gateway” products from Nasuni, Cirtas, StorSimple, Twin Strata, and Asigra are awfully interesting.

Stephen’s Stance

The storage revolution is coming, whether we in the industry are ready or not. Developers are voting with their feet, targeting cloud storage and application platforms rather than conventional filesystems. Although the market for cloud storage products is slow to develop, the cloud storage concept will eventually dominate the landscape.

It seems most likely that this revolution will decimate the storage industry as we know it today. Unable to push high-margin storage arrays into the ballooning cloud space, product vendors will see their market share eroded by service providers with no use for these expensive systems. Monolithic file and block will soldier on in the new legacy applications, but the action will inevitably slip away.

The likely winners will be those who can leverage commodity hardware for scale-out cloud storage use. The proliferation of cloud platforms will settle down, with a few gaining traction and the rest discarded. Then we will see companies like HP, Dell, and Oracle rise to lead the storage sales charts with massive volume shipments to service providers.

Disclosure: I used to work for StorageNetworks (which is now defunct) and Nirvanix.

Image credit: Barcelona Graffiti by Aeioux

© sfoskett for Stephen Foskett, Pack Rat, 2010. | From CAS to Cloud: Revolutionary Storage
This post was categorized as Computer History, Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Great! But I thought the new era for IT was a cloud with choice, mobility, and application support, as trumpeted by EMC’s VMware subsidiary! Wasn’t Cloud vServices from VDC-OS supposed to be the prototype cloud strategy for the datacenter?

What we have here is a simple clash of marketing amusingly taking place at (nearly) the same company. VMware figured out how to extend their server virtualization products outside the confines of the data center, and laid that technology out as a strategy with the trendy “cloud” name. Meanwhile, mother EMC is working on next-generation content storage software and decides to roll that out as a strategy and also jumps on the “cloud” meme. What’s an IT manager to do?

Defining Atmos

As predicted, EMC’s Atmos (code-name Maui) is a distributed software layer to handle the storage and management of data objects across geographically-dispersed storage devices. EMC’s Chuck Hollis demonstrates Atmos with a simple, practical example, perhaps making it sound too much like Akamai but generally getting the point across. You have a data object, write it to Atmos through REST/SOAP or CIFS/NFS, assign some metadata, and the software takes care of data placement for you. It’ll add local copies, replicate for availability and performance, compress or deduplicate, manage versions, and all sorts of goodies (if you ask it to).

But EMC already has a capable object storage platform, the Centera. We’ve just got used to the content-addressable storage (CAS) label for object storage (even though this name misses the point of object storage, in my opinion) and now EMC wants us to learn a new label for a somewhat-similar device? Steve Todd, EMC’s object guy extraordinaire, lays it out:

SAN Value = Centralized, secure multi-tenancy for blocks.

NAS Value = Centralized, secure multi-tenancy for files.

CAS Value = Centralized, secure multi-tenancy for objects (content + metadata).

COS Value = Globalized, secure multi-tenancy for content with rich policies.

Ok, so the defining capabilities of Atmos are its global scale and rich policies. And the fact that “objects” has become “content”, presumably since Atmos can handle traditional NAS (CIFS/NFS) chores as well.

Prayers Answered?

It sounds like EMC is answering my prayers for a storage revolution, delivering a highly-capable object storage platform that transcends the old limits of blocks, directories, and files. Steve Todd points out that Atmos handles five policy categories out of the box:

• Replication
• Compression
• Spin-down
• Object de-dup
• Versioning

So we write some data to Atmos, using either traditional NAS or webby dubby protocols like SOAP, and can then apply policies in any of these five categories to that data. One can also extend the Atmos to accept other policies, but the absence (out of the box) of concepts like encryption, secure deletion, retention, and access control are surprising.

I am quite puzzled about how practical these policy capabilities will be in the real world. How exactly would an application say “I want you to compress that file I wrote over NFS just now?” Hitachi’s HCAP platform, for example, also has policy capabilities and a NAS front end, and although archiving applications can communicate their policy needs, I don’t see lots of current general-purpose applications using it.

Strategic Storage?

This brings me to my puzzlement: The default Atmos policies are all general-purpose, production computing ideas, not the special-purpose, archiving and retention needs served by Centera, HCAP, and the rest. So the Atmos is clearly intended to be a production data storage system, not an archiving system to compete with Centera.

Since mainstream business applications currently don’t have any capability to specify policies like these when writing files, and since NAS protocols lack any means to communicate them even if the apps want to, we can conclude that EMC expects that Atmos users will write special applications to take advantage of it.

EMC certainly doesn’t expect that the NAS-capable Atmos will simply replace today’s distributed NAS solutions. NAS is a sideshow for Atmos. The real action will be in the REST/SOAP webby dubby applications that will be written with the platform in mind and will take full advantage of these capabilities.

If this is true, and I and others suspect that it is, then Atmos really isn’t a game-changing platform unless you change your game. If you write new applications to store data with SOAP, Atmos is a nice in-house alternative to Amazon S3 or Nirvanix, and offers a very compelling set of data management capabilities. And if you want to set up shop to compete with those service providers, Atmos is a dream come true with built-in multi-tenancy.

Datacenter Strategy

So EMC alone has two seemingly competitive datacenter strategies. And then there’s Microsoft, which announced its Azure cloud platform recently, and Amazon and the other cloud providers.

So let’s say you’re a CIO for a large corporation. Which of the following strategies is more compelling:

1. Use VMware VDC-OS to add capabilities and Cloud vServices extend your current virtual infrastructure geographically
2. Recompile and tweak your Windows applications to leverage Microsoft Azure
3. Develop new applications to take advantage of the impressive storage capabilities of an in-house EMC Atmos system
4. Point your new applications at a third-party cloud provider like Amazon or Nirvanix

IT people are practical. Although we love new technology, we tend to be cautious. We also hate massive software development efforts, and only sanction them when they’re absolutely necessary. Given these personality traits, I’d say VDC-OS and perhaps Cloud vServices still stands out as the most likely and practical scenario for the majority of applications and businesses.

This is not to say that EMC Atmos will be a flop. I’m impressed by the technology, and expect that Atmos will find buyers, just as Centera did. And Atmos might even replace Centera once EMC adds retention policies to it and scales it down as well as up and out. But Atmos will not redefine the datacenter. We’re stuck with blocks and files, and VMware’s practical strategy is a winner in that world.

Update: Marc Farley compares Atmos to WAFS, with ominous implications, and echos my recent question on what is and is not innovative.

Update 2: Chuck Hollis, Storagezilla, and Len Devanna have all come right out and said that this is only intended for certain customers with massive distributed storage needs, and is not intended as a new datacenter strategy. Even the “cloudfella” says “ciao”:

Update 3: More great information, including a reply regarding VDC-OS and Atmos from the one and only Chad Sakac, more great detail about the inner workings of Atmos from Steve Todd, and even more info from Dave Graham. Finally, although I think that Cloudfellas video is cute, I wouldn’t categorize it as viral. But those Mozy ads are awesome!

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | EMC Atmos Versus VMware VDC-OS: Will The Real Cloud Strategy Please Stand Up?
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Although deduplication of storage is nothing new, with Data Domain and other making hay with the technique for years, it has never been ready for prime time – reduction of active primary storage applications like email and databases. Instead, deduplication has been relegated to second- or third-tier status, deduplicating archives and backup data. But change is in the air, and deduplication vendors are starting to bustle towards the bright lights of primary storage.

Stone Knives and Bear Skins

We have all been here before, of course. Back at the dawn of the personal computer era, data compression was a hot topic of conversation. I recall being so impressed by an article in Byte (1986:5, p99) outlining Huffman coding that I tried cooking up an implementation in Atari BASIC. Lossless compression has a magical pull to the geek in many of us – redundant data just wants to be eliminated!

Stacker dominated the disk compression world - until Microsoft introduced DOS 6.0

Companies soon applied compression to primary storage, especially the limited storage in personal computers. Stacker was a hit after 1990, until Microsoft built a workalike, called DoubleSpace, into DOS 6.0 in 1993, leading to a historical lawsuit. I personally used the ADDSTOR disk compression built into DR-DOS 6.0 to stretch two more years out of the 20 MB MFM hard drive in my AT&T PC6300 at WPI.

But something funny happened in the late 1990s: Compression began to lose its luster. Compressing data always takes quite a bit of CPU power, but this was offset somewhat by the truncated data transfers and more-efficient file system layout afforded in early PCs. But as disks got larger and faster, using precious CPU time to save space seemed less and less compelling. Today, although nearly every operating system includes built-in compression of files, folders, or perhaps disks, these features are rarely used. And compression was never popular in the performance-sensitive enterprise space.

Deduplication Has a Nice Ring

Although traditional fine-grained compression has not been very successful in the enterprise, its lanky cousin, single-instance storage, has long found niche jobs. Applications from databases to email systems to file servers have long had the ability to recognize to requests to store the exact same file or record, and to store just a single instance in this case. Even file systems have the ability to do single instance storage through the use of links, though this is initiated by the user rather than in an automated fashion.

In the late 1990s, FilePool began developing a content-addressable storage device, which was acquired by EMC in 2001. This device, later known as the Centera, was one of a number of storage platforms targeted at the archiving market introduced this decade. At the same time, virtual tape libraries made the jump from the mainframe to open systems. Both devices, being outside the critical path of performance but offering massive capacity, were well-suited to implement advanced capacity optimization technologies that combined the concepts of compression with single-instance storage. Thus was created the modern world of data deduplication.

What we think of as deduplication is neither fish nor fowl: It assesses larger “chunks” of data than compression technologies, delivering greater capacity savings and potentially reducing performance impact, but is more flexible than single-instancing, recognizing the similarities within files or objects.

But it is still maddeningly difficult to scale deduplication while maintaining performance. Rather than fight to maintain reasonable write throughput, most deduplication products have switched to post-processing, deferring their work to quieter times.

It’s Not Just for Breakfast

Regardless of their methods or underlying technology, no deduplication vendor has stood up to support challenging low-latency or high-throughput production applications, however. NetApp was the first to raise the issue of support for production applications, but although they tout the technology for VMware, they haven’t exactly been shouting from the rooftops to get their A-SIS deduplication technology deployed in other high-I/O applications. And I haven’t seen Hifn’s card yet.

Yesterday, I mentioned that greenBytes was adding deduplication to their ZFS-based storage array for primary data. And now Riverbed has fired another shot over the bow, repurposing their (deduplicating) WAN accelerator product for primary (file) storage. They might be able to pull it off, too, since they have a long list of customers who are already enjoying the technology in production. It’s not a stretch to suggest that Riverbed’s appliances can scale to handle production data loads. Although it’s file-only, I can imagine quite a few scenarios where this tech could really yield benefits. Could we come full-circle, with deduplication finally reaching the enterprise storage world?

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Deduplication Coming to Primary Storage
This post was categorized as Computer History, Enterprise storage, Features, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.