The Dumb Disk Fallacy

I am spending a few weeks examining the truths and fictions that bind our industry together. Let’s start with one of my favorite old canards: That enterprise storage must be overpriced because bare disk drives are so cheap.

I have seen this straw man argument set up by so many throughout my career that it has become laughably predictable. Every time a new high or low point is set for enterprise storage cost, someone is there pointing out that a bunch of disk drives is vastly cheaper. Let’s call this the Dumb Disk Fallacy: Only a fool would claim that dumb disks are comparable to enterprise storage.

Are we supposed to be surprised that raw materials make up such a small percent of the cost of an integrated system? One can prepare a delicious meal at home using supermarket-bought ingredients for a quarter the cost of a restaurant outing. A single glass of wine at a bar often costs as much as a whole bottle at the store, yet the restaurant industry is holding on even through a recession.

Lack of expertise only one factor: Even one who can not whip up a soufflé can certainly boil some spaghetti, warm up some sauce, and pour a glass of Chianti! Convenience is another driver, since many lack the time required to shop and cook. But there are other benefits as well: Dining out is a social activity and restaurant meals allow us to sample unfamiliar cuisine. Clearly, there are many reasons for one to pay far more for a finished product than for the ingredients it contains.

The same is true of enterprise storage. Although they make up a significant proportion of the bulk of a given storage array, hard disk drives are often a small element in the overall cost. One does not price a salad by its lettuce or a lasagne by its pasta, after all. Modern enterprise storage systems are defined their capabilities, ranging from performance to reliability to flexibility. Indeed, core components like disk drives and processors have long since been commoditized, and many vendors are leveraging much larger-scale commodity subsystems these days. The day will soon come when major vendors will differentiate entire product lines based solely on the software they run.

Now, it is possible to build very cheap storage systems, and the price of these can even approach the raw disk cost. But finished storage systems will never be as cheap as just a bunch of disks (JBOD, for newbies), because of the following:

1. Disk arrays include lots of hardware components beyond disk drives – chassis, power supplies, controllers (complete with CPUs, RAM, etc), and cabling and connectors
2. Data must be protected using extra disk capacity for parity, mirrors, snapshots, and spares
3. Arrays include software to orchestrate the whole operation; even free software requires development, testing, and integration
4. Support and maintenance contracts are required for any production system
5. Supporting software is often a requirement, too, for configuration, operation, management, and integration with servers and applications
6. The companies selling the array deserve a bit of profit or they’ll go out of business and your support contract won’t be worth much
7. Don’t forget utilization: A half-empty disk costs twice as much as a full one, and it is awfully hard to make use of 100% of available capacity without a storage network of some kind

So how close to “dumb disk” can a production storage system get? Google’s is probably the closest: They use custom server boards, custom chassis, custom power supplies, custom racks, and low-end disk drives to keep hardware costs at a minimum; the entire thing is composed of commodity components, too; the array software is all written and supported in-house for very low overhead; the capacity is very highly utilized due to the inherent flexibility of the applications they support. So Google has attacked every one of these areas in an effort to drive out costs. But would it really be practical for folks other than Google to invent, construct, and run their own storage system?

Instead of dreaming about building our own storage solution, let’s look at the spectrum of storage available to the modern enterprise IT shop:

1. Bare disk drives are widely available with a handful of companies producing just a few different models. The idiosyncracies of these are interesting to some, but a hard disk drive is useless alone
2. Direct-attached storage offerings range from single-drive USB-connected external disks to multi-drive racks, yet these too are useless without a server to drive them. These normally lack any sort of intelligence or advanced features and quite a bit of effort is required to ensure acceptable levels of performance or reliability. But so-called JBOD or DAS storage can be incredibly cheap to purchase.
3. True storage arrays range from the most basic home NAS boxes to the most advanced enterprise systems. All are purchased primarily for their features, with price tags merely differentiating between competitive offerings. For this reason, storage developers focus the vast majority of their engineering, sales, and marketing efforts on advanced capabilities rather than per-GB cost.
4. Storage as a service is a different realm entirely. Although advances in manageability and utilization and commoditization of hardware allow today’s cloud storage offerings to be offered at very attractive per-GB price points, low cost is merely a welcome side benefit. Engineering and support resources focus almost entirely on enhancing user experience, while hardware is small component of the cost of delivering enterprise-class storage as a managed service.

I’m not an apologist for overpriced enterprise storage, but I recognize that arrays are more than just a collection of disks. I look forward to the day when we finally dispense with the dumb disk fallacy and focus instead on the real value added by enterprise storage innovation. But the realist in me knows that this straw man will continue raising his head for some time to come. I am sure that developments like EMC’s unified CLARiiON and Symmetrix hardware platform, the spread of software appliances on commodity server hardware, and excellent free storage software like Nexenta and FreeNAS will only add fuel to the fire. Therefore I entreat you, dear reader: Do not succumb and pitch disk that is dumb!