• Skip to main content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / Waves of Storage Innovation

Waves of Storage Innovation

July 8, 2015 By Stephen Leave a Comment

Before you read this, check out Today’s Storage: Same As It Ever Was for a bit of background…

For the past few decades, some of the best minds in computer science have been drawn to the question of storage. How can data be better protected, shared, and exploited? Given the price and performance profiles of various storage devices, how can we maximize benefit for a given application? What can we do to transcend inertia and make data available everywhere at once?

Innovation comes in waves, undermining the status quo
Innovation comes in waves, undermining the status quo

These fundamental questions have been tackled by a continuing stream of companies promoting waves of new technology. The limitations of block storage gave us CAS, object storage, and the cloud. RAID was invented because hard disk drives can not physically drive sufficient I/O operations, but wide striping, data dispersal, tiering, and caching are table stakes today. Data compression was pioneered as a transmission technology but has moved to array controllers and static data sets.

In each area, a predictable pattern emerged:

  1. New algorithms and architectures are developed
  2. Start-up companies are funded and launched, creating standalone products centered on them
  3. These technologies become features as they are leveraged broadly across the industry, with the start-ups acquired or withering
  4. Mainstream adoption is predictably slow and insider attention turns to the next wave

This last bit is pretty remarkable, and something I’ll be talking about in a follow-on post, but let’s focus on the early phases for now.

How many product, and even company, categories have become merely an obscure feature? I’ve watched this pattern repeat with SRM, object storage, data deduplication, thin provisioning, iSCSI, hybrid storage, and all-flash architectures (just off the top of my head). And now we’ve got RAM and flash caching, storage virtualization, and data-aware storage.

It is interesting too that each wave tends to build on those before it. Deduplication was once a marquee product but now is mainly a way to optimize the use of flash for hybrid or all-flash storage systems. SRM was the “next big thing” but is now an integrated feature of data-aware storage systems. Scale-out and virtual storage is easiest to do with an object store under the covers. And so on.

Each technology wave is accompanied by the same archetypal companies, too:

  • Academics and technologists who can’t articulate a use case but usually have the best technology
  • Re-packagers of outdated products trying to stay relevant by claiming they’re part of the current wave
  • Marketing-first companies making noise with little or no IP of their own but capable of making lots of noise
  • Super-funded leaders who hire and expand rapidly, are flush with cash, and become the standard-bearers for the wave
  • Actual companies selling useful products (too few of these!) who have solid products and rely on real sales and use cases

It is rare that any of these companies lasts more than a few years. But their products tend to live on, occasionally in standalone companies (usually the last two) but more typically as a product line or feature inside one of the giant incumbents.

And so it goes. Waves of innovation and waves of companies, crash on the storage market, but the same incumbent leaders and product lines survive for decades. It’s actually pretty depressing. Are things changing? It’s hard to see sometimes, but real progress has been made. And perhaps all those waves have finally chipped away at the outdated storage paradigms I complained about last time to send it all toppling down.

You might also want to read these other posts...

  • GPS Time Rollover Failures Keep Happening (But…
  • Electric Car Over the Internet: My Experience Buying…
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • What You See and What You Get When You Follow Me
  • Tortoise or Hare? Nvidia Jetson TK1

Filed Under: Computer History, Enterprise storage Tagged With: all-flash array, caching, CAS, cloud storage, companies, compression, data dispersal, data-aware storage, enterprise storage, history, hybrid storage, iSCSI, object storage, RAID, SRM, startups, thin provisioning, tiering

Primary Sidebar

The best way to predict the future is to invent it.

Alan Kay

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

May 29, 2020

What You See and What You Get When You Follow Me

May 28, 2019

GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

April 6, 2019

Symbolic Links

    Featured Posts

    The Rack Endgame: Open Compute Project

    September 17, 2014

    MacBook Users: Encrypt Your Drive with OS X FileVault! It’s Easy and Free!

    December 20, 2012

    Why I Am Biased Against FCoE

    October 21, 2011

    Donate Your Swag to School Kids In Need

    July 28, 2010

    EMC Redefine Possible (TL;DR Edition)

    July 9, 2014

    ZFS Is the Best Filesystem (For Now…)

    July 10, 2017

    VMware’s Hardware Partner Strategy: Heads I Win, Tales You Lose

    February 11, 2013

    On the Death of Innovation, or “These Kids These Days!”

    May 21, 2012

    Why You Should Never Again Utter The Word, “CIFS”

    February 16, 2012

    Cisco’s Trojan Horse

    September 15, 2014

    Copyright © 2021 · Log in