I began by introducing the core problem: Storage isn’t getting any cheaper due to storage utilization and provisioning problems. Thin provisioning isn’t all it’s cracked up to be, but it has a place. So what’s wrong with it?

The problem with thin provisioning starts with the telephone game. Did you ever play the telephone game as a kid? Maybe you whisper “I like potatoes” to the first person in a circle and when it comes back to you it’s changed to “Meet Mike on the patio” or something like that. It’s a totally different message.

What happens in the telephone game is that a little bit of information gets lost at each step along the path, and at the end of the chain you’ve basically lost all the information. And this happens all the time in computers, especially in data storage.

We storage guys are stuck at the bottom of a stack that includes many layers. Each of those layers loses something in translation, mostly because it’s pretending to be something that it’s not.

Think about storage today: We’ve got fake file systems pretending to live on fake discs that pretend to be directly connected to your computer. But they’re not.

Everything we do in enterprise storage is basically faking out something else so compatibility is maintained. And at each step (the file system, the database, the host, the network) you’re losing information. By the time you get to the storage system, there’s just no communication whatsoever.

This is the core problem with thin provisioning. The application knows what data is temporary, and that would be very useful for the storage system to act upon. But by the time the data gets there, the message is lost. Maybe the application tells a database. Maybe the database tells the file system. Maybe the file system tells the volume manager. But, that’s about as far is it’s going to go. So, this is really the issue. It’s the telephone game.

Let’s say we have a file system and some storage. We want to write some data. So, the file system says, “Hey, here’s my new block.” And the storage says, “Yeah, I got it.”

This is the classic way of doing thin provisioning. So the storage system is now only using the little blue box. The file system adds some new data, then some more data, and the storage just keeps growing. The rest of the space can be reallocated.

We’re good so far. This is so simple that most products in storage now have something like this. Of course, it took them 10 years to do it, but they finally have it.

So, we’re good. We can allocate storage. But, what about deallocate?

If I delete something, I have to tell the storage, and then it has to shrink the allocated capacity. But we’re not doing that. Most file systems don’t actually send that information on. When you delete a file, most file systems actually write more data instead of actually deleting anything.

Thin reclamation is the core technical challenge to thin provisioning, and the telephone game is the reason. Next we’ll present some solutions that are currently being worked out.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Thin Provisioning: Playing the Telephone Game
This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Back in the old days, vendors used to give away Harley Davidson motorcycles and Mercedes-Benz cars (no kidding!), but even in these days of waning investment, conferences still have some expensive goodies. I decided to take a walk around the show floor at the April 2010 Storage Networking World (SNW) in Orlando, checking out the loot.

So we’ve got some solid prizes here, with a definite Apple bias. The score:

1. Amazon Kindle x4
2. Apple iPad x3
3. Apple iPod Touch x3
4. Cisco Flip x3
5. HP Mini netbook x2
6. Dell Mini 10
7. Beats Headphones
8. Flo TV
9. Apple iPod Nano
10. Spy Robot (!)

Folks were most excited about the iPad giveaways, and there were at least five other iPads given away off the show floor. But NetApp wins my award for creativity: The “spy robot” was a clever choice amid a sea of book readers, cameras, and music players!

Prizes like these really did draw a crowd. The smartest companies had second- and third-prize drawings as well, and some had simple tests on the form to judge knowledge of their products. I still don’t like chotchkies, but these prize drawings pass my sniff test.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Conference Loot: SNW Orlando 2010 Edition
This post was categorized as Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

But cloud services are different. Seriously! Cloud services don’t need standards because:

1. Cloud services are still rapidly evolving – No one knows how they will look in a year, let alone a decade, and a premature standard will be worthless. Similarly, it’s not at all clear what use cases will eventually win out, and usage should drive interfaces, not the other way around.
2. Cloud services are many and varied – It’s incredibly hard to come up with a reasonably-complete standard programming API or management platform when each vendor’s offering is radically different. Standards must follow the 80/20 rule, but today’s cloud offerings are only about 20% similar.
3. (Real) cloud systems are open already – The whole point of the public cloud is to leverage existing open standards for access (IP/HTTP) and any worthwhile service already has a freely-usable REST-like API. Cloud services are engineered to be programmable and open, so the only lock-in is in how you use the cloud.

We can’t even agree on terminology at this point. Is data storage as a service DaaS (as SNIA says) or STaaS (as NetApp says)? How do you define public, private, and hybrid cloud? And what is cloud anyway? Cloud computing is not a war, it’s a fantastically exciting race to deliver value!

Open for Business

I’d like to return for a moment to that last point: The key element I’ve seen in most interesting cloudy products is programmability. Service providers publish API documents outlining the inputs, processing, and outputs for their systems and developers and end users create applications that leverage these. The best of these APIs use the concept of REST, delivering services through extremely simple and self-contained HTTP calls. This barely even rises to the level of software coding (and thus isn’t a true API) and is the hallmark of the cloud.

These systems are wide open: You can explore their interfaces, discovering new ways to use the them that were never intended. The same process accompanies all Internet systems, from RSS and Atom to Yahoo Finance. Just as one can rapidly migrate from Yahoo to Google by substituting a few URLs and parameters, so too can one move between cloud platforms.

Note that certain cloud systems lend themselves more to this kind of mobility. Once cannot move virtual machines from Amazon EC2 to Rackspace or Terremark because the underlying hypervisor technology is different. But even here companies like RightScale are stepping in to enable mobility.

When it comes to cloud storage services, the major players’ interfaces are open enough that migrating data in and out is simply a matter of performance: Read from this one, write to that one, and wait until the process is done. I am not a programmer and yet I was able to port an application from S3 to Nirvanix in just a few hours using only the respective API documentation. Interfaces like CloudLoop can also be leveraged to ease the movement of data.

Standards When?

Cloud services will eventually settle down and be standardized. I expect a workable cross-platform API for RESTful cloud storage within 24 months, for example. And one expects that the management of cloud compute instances will pass through a consistent and stable interface in that same timeframe. But these will develop as a natural part of the evolution of the systems themselves, not through some artificial “build it and they will come” standardization process.

There is nothing wrong with big companies sending their representatives to SNIA and DMTF meetings to talk about standardization. In fact, this is a great way to discuss ideas and begin to orient the industry. But the time for standards has not yet come, and users of cloud services have no need to wait for them. In fact, waiting for a standard will just prolong the maturation of cloud services, since real-world applications are the external pressure that forces evolutionary selection. Amazon would never have created their virtual private cloud (VPC) capability without customer input, and they will never perfect this capability if they rely only on pundits, bloggers, and product marketers.

Even when standards do appear, they will not eliminate per-solution APIs. Cloud service providers will continue to explore new concepts, and these will appear first in “proprietary” interfaces. Perhaps they will use entirely unique calls, or perhaps they will leverage reserved or unassigned sections of the standard, but innovation will continue. Witness the radical changes in HTML versions to date, the additions to CSS, and the wide world of browser plugins.

So we don’t need cloud standards yet. They will come, whether artificially pushed by committees or evolving through use, but only useful standards will survive. Isn’t that just how it should be?

Note: I am employed by Nirvanix, a cloud managed storage service provider, providing independent cloud strategy advice as Director of Consulting. Although this article was not created for my employer and is not intended to reflect their views, my perceptions are obviously colored by my daily work.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | We Don’t Need Cloud Standards (Yet)
This post was categorized as Computer History, Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.