• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / The Four Horsemen of Storage System Performance Are Coming To Life!

The Four Horsemen of Storage System Performance Are Coming To Life!

October 30, 2015 By Stephen Leave a Comment

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.
The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

A few years back, I wrote an immensely-popular series of blog posts outlining the four things that were holding storage system performance back, and the ways to fix them. At the time, I created some presentation content to go along with these posts, and even considered pulling them into a white paper, but nothing came of that. Now, however, I’m pleased to announce that my Four Horsemen are accompanying me to the stage November 10, 2015 at the DeltaWare Data Solutions Emerging Technology Summit in Edina, Minnesota.

Here’s the original “Four Horsemen” series:

  1. The Rule of Spindles
  2. Never Enough Cache
  3. I/O As a Chain of Bottlenecks
  4. Get Smart

When I wrote those four pieces, back in 2010, the industry was on the verge of a major shift: Hard disk drives had always been the critical inhibitor of storage system performance (see “The Rule of Spindles“), but widespread use of flash memory meant this was about to change. Many people don’t realize that RAID was originally designed primarily to deliver increased IOPS, not to protect data in the event of a drive failure. Yet even today, hard disk drives remain a critical gating factor to storage system performance. The advent of sequentialization, hybrid flash cache, and smarter storage systems has mitigated the problem, but most data still lands on a spinning disk eventually. Which means that we’re still stuck optimizing them.

NAND flash has also dented the second “horseman” (see “Never Enough Cache“), since it’s much cheaper on a per-capacity basis than DRAM. This is why virtually every storage system today has a large (by RAM standards) pool of NAND flash to serve as a temporary landing zone. And to stretch that metaphor, most systems also use NAND flash as a “pre-departure waiting area”, storing data that is likely to be read soon. And now that storage I/O has become quicker (see below), we’re starting to see optimization above NAND, with SLC caching MLC and DRAM above both.

Regardless of the performance of the storage capacity layer, however, there will always be bottlenecks in the I/O channel (see “I/O As a Chain of Bottlenecks“). Faster storage (thanks largely to flash) has disrupted the industry, with the “big SAN” model giving way to top-of-rack flash, converged infrastructure, and even re-internalized storage. Next-generation solid state storage is so fast, in fact, that it requires some extreme re-engineering to take advantage of all these IOPS. Otherwise we’re left harnessing a racehorse to a wagon.

The most exciting area of enterprise storage today is the emergence of intelligence and integration (see “Get Smart“). No matter what a storage device can do, we can’t maximize return on investment without optimizing data access. This is why today’s trend toward data-aware storage, integrated caches, server/storage integration protocols, and software-defined storage is exciting. If the array’s capabilities (and limitations) are exposed to an intelligent data management layer, we can extract maximum performance from the whole system. Even those pesky hard disk drives.

If you’re in Minnesota in two weeks, come on down to the DeltaWare Data Solutions Emerging Technology Summit! I’ll also try to film my presentation and post it here. And if you’re organizing a conference yourself, might I suggest that you reach out to bring me in as a speaker?

You might also want to read these other posts...

  • What You See and What You Get When You Follow Me
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Electric Car Over the Internet: My Experience Buying From…
  • Tortoise or Hare? Nvidia Jetson TK1
  • Running Rabbits: More About My Cloud NUCs

Filed Under: Computer History, Enterprise storage, Events, Personal Tagged With: 4 horsemen, cache, DeltaWare Data Solutions, flash, keynote, presentations, RAID, seminar, spindles, top of rack flash

Primary Sidebar

An unlimited-length file name is a file. The content of a file is its own best name.

Jef Raskin

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    Regarding My Symbolic Links and Good Reads

    April 16, 2015

    Top VMware Blogs 2014: How I Voted

    February 25, 2014

    Instapaper for iPad and iPhone Enhances My Web World

    June 1, 2010

    What More Could Alan Turing Have Accomplished?

    October 7, 2012

    Why Big Disk Drives Require Data Integrity Checking

    December 19, 2014

    New England Takes On Seattle To Determine Who’s Number 2 … In Tech!

    January 19, 2015

    Virtual Machine Mobility: Of What, and to Where and in What State?

    January 16, 2012

    Here’s Something Your Raspberry Pi Can’t Do: Gigabit Ethernet and SATA in the Olimex A20-OLinuXIno-LIME2

    May 25, 2016

    My 2012 Project: Improving Energy Efficiency

    January 3, 2012

    Infographic: Real-World Port Throughput Relative To Thunderbolt (Formerly Light Peak)

    February 21, 2011

    Footer

    Legalese

    Copyright © 2022 · Log in