• Skip to main content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / The Four Horsemen of Storage System Performance Are Coming To Life!

The Four Horsemen of Storage System Performance Are Coming To Life!

October 30, 2015 By Stephen Leave a Comment

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.
The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

A few years back, I wrote an immensely-popular series of blog posts outlining the four things that were holding storage system performance back, and the ways to fix them. At the time, I created some presentation content to go along with these posts, and even considered pulling them into a white paper, but nothing came of that. Now, however, I’m pleased to announce that my Four Horsemen are accompanying me to the stage November 10, 2015 at the DeltaWare Data Solutions Emerging Technology Summit in Edina, Minnesota.

Here’s the original “Four Horsemen” series:

  1. The Rule of Spindles
  2. Never Enough Cache
  3. I/O As a Chain of Bottlenecks
  4. Get Smart

When I wrote those four pieces, back in 2010, the industry was on the verge of a major shift: Hard disk drives had always been the critical inhibitor of storage system performance (see “The Rule of Spindles“), but widespread use of flash memory meant this was about to change. Many people don’t realize that RAID was originally designed primarily to deliver increased IOPS, not to protect data in the event of a drive failure. Yet even today, hard disk drives remain a critical gating factor to storage system performance. The advent of sequentialization, hybrid flash cache, and smarter storage systems has mitigated the problem, but most data still lands on a spinning disk eventually. Which means that we’re still stuck optimizing them.

NAND flash has also dented the second “horseman” (see “Never Enough Cache“), since it’s much cheaper on a per-capacity basis than DRAM. This is why virtually every storage system today has a large (by RAM standards) pool of NAND flash to serve as a temporary landing zone. And to stretch that metaphor, most systems also use NAND flash as a “pre-departure waiting area”, storing data that is likely to be read soon. And now that storage I/O has become quicker (see below), we’re starting to see optimization above NAND, with SLC caching MLC and DRAM above both.

Regardless of the performance of the storage capacity layer, however, there will always be bottlenecks in the I/O channel (see “I/O As a Chain of Bottlenecks“). Faster storage (thanks largely to flash) has disrupted the industry, with the “big SAN” model giving way to top-of-rack flash, converged infrastructure, and even re-internalized storage. Next-generation solid state storage is so fast, in fact, that it requires some extreme re-engineering to take advantage of all these IOPS. Otherwise we’re left harnessing a racehorse to a wagon.

The most exciting area of enterprise storage today is the emergence of intelligence and integration (see “Get Smart“). No matter what a storage device can do, we can’t maximize return on investment without optimizing data access. This is why today’s trend toward data-aware storage, integrated caches, server/storage integration protocols, and software-defined storage is exciting. If the array’s capabilities (and limitations) are exposed to an intelligent data management layer, we can extract maximum performance from the whole system. Even those pesky hard disk drives.

If you’re in Minnesota in two weeks, come on down to the DeltaWare Data Solutions Emerging Technology Summit! I’ll also try to film my presentation and post it here. And if you’re organizing a conference yourself, might I suggest that you reach out to bring me in as a speaker?

You might also want to read these other posts...

  • GPS Time Rollover Failures Keep Happening (But…
  • What You See and What You Get When You Follow Me
  • Ranting and Raving About the 2018 iPad Pro
  • Electric Car Over the Internet: My Experience Buying…
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

Filed Under: Computer History, Enterprise storage, Events, Personal Tagged With: 4 horsemen, cache, DeltaWare Data Solutions, flash, keynote, presentations, RAID, seminar, spindles, top of rack flash

Primary Sidebar

Imagine if every Thursday your shoes exploded if you tied them the usual way. This happens to us all the time with computers, and nobody thinks of complaining.

Jef Raskin

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

May 29, 2020

What You See and What You Get When You Follow Me

May 28, 2019

GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

April 6, 2019

Symbolic Links

    Featured Posts

    How Will Cisco Recover From The Consumer Strategy Blunder?

    January 2, 2013

    How To Keep Your Family Activities In Sync With A Shared Google Calendar

    April 18, 2010

    10 Mysteries The Lost Finale Definitively Settled

    May 25, 2010

    My 2012 Project: Improving Energy Efficiency

    January 3, 2012

    New England Takes On Seattle To Determine Who’s Number 2 … In Tech!

    January 19, 2015

    The Best Mac OS X Terminal Font: Glass TTY VT220

    October 6, 2015

    Why Are PCIe SSDs So Fast?

    June 12, 2013

    How Smart Is the Mondaine Helvetica Smart Watch?

    December 30, 2015

    Why Buy a NEX-7? Why Sony NEX At All?

    October 17, 2011

    Defining Failure: What Is MTTR, MTTF, and MTBF?

    July 6, 2011

    Copyright © 2021 · Log in