September 2, 2014

4 Horsemen

Why do some data storage solutions perform better than others? What tradeoffs are made for economy and how do they affect the system as a whole? These questions can be puzzling, but there are core truths that are difficult to avoid. Mechanical disk drives can only move a certain amount of data. RAM caching can improve performance, but only until it runs out. I/O channels can be overwhelmed with data. And above all, a system must be smart to maximize the potential of these components. These are the four horsemen of storage system performance, and they cannot be denied.

The Four Horsemen of Storage System Performance: The Rule of Spindles

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

Why do some data storage solutions perform better than others? Mechanical performance, RAM caching, I/O capacity, and the intelligence of the system all have a part to play. Today we examine the rule of spindles: Adding more disk spindles is generally more effective than using faster spindles.

The Four Horsemen of Storage System Performance: Never Enough Cache

Write-Through Cache

Perhaps the previous discussion of spindles left you exhausted, imagining a spindly-legged centipede of a storage system, trying and failing to run on stilts. The Rule of Spindles would be the end of the story were it not for the second horseman: Cache. He stands in front of the spindles, quickly dispatching requests using solid state memory rather than spinning disks. Cache also acts as a buffer, allowing writes to queue up without forcing the requesters to wait in line.

The Four Horsemen of Storage System Performance: I/O As a Chain of Bottlenecks

We continually shift between parallel and serial I/O paradigms

It is tempting to think of storage as a game of hard disk drives, and consider only The Rule of Spindles. But RAM cache can compensate for the mechanical limitations of hard disk drives, and Moore’s Law continues to allow for ever-greater RAM-based storage, including cache, DRAM, and flash. But storage does not exist in a vacuum. All that data must go somewhere, and this is the job of the I/O channel.

The Four Horsemen of Storage System Performance: Get Smart

Your disk doesn't contain anything resembling your files

The Four Horsemen of storage system performance cannot be denied, but they do offer a clear path forward. Storage systems must improve in many different areas, from spindles and drives to caching and I/O bottlenecks. But above all else, storage systems must become smarter in order to become faster, and this requires greater insight into the true nature of the data stream being stored. All storage performance developments, from the laptop to the enterprise, boiled down to adaptations to the demands of the Four Horsemen.