• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / Waves of Storage Innovation

Waves of Storage Innovation

July 8, 2015 By Stephen Leave a Comment

Before you read this, check out Today’s Storage: Same As It Ever Was for a bit of background…

For the past few decades, some of the best minds in computer science have been drawn to the question of storage. How can data be better protected, shared, and exploited? Given the price and performance profiles of various storage devices, how can we maximize benefit for a given application? What can we do to transcend inertia and make data available everywhere at once?

Innovation comes in waves, undermining the status quo
Innovation comes in waves, undermining the status quo

These fundamental questions have been tackled by a continuing stream of companies promoting waves of new technology. The limitations of block storage gave us CAS, object storage, and the cloud. RAID was invented because hard disk drives can not physically drive sufficient I/O operations, but wide striping, data dispersal, tiering, and caching are table stakes today. Data compression was pioneered as a transmission technology but has moved to array controllers and static data sets.

In each area, a predictable pattern emerged:

  1. New algorithms and architectures are developed
  2. Start-up companies are funded and launched, creating standalone products centered on them
  3. These technologies become features as they are leveraged broadly across the industry, with the start-ups acquired or withering
  4. Mainstream adoption is predictably slow and insider attention turns to the next wave

This last bit is pretty remarkable, and something I’ll be talking about in a follow-on post, but let’s focus on the early phases for now.

How many product, and even company, categories have become merely an obscure feature? I’ve watched this pattern repeat with SRM, object storage, data deduplication, thin provisioning, iSCSI, hybrid storage, and all-flash architectures (just off the top of my head). And now we’ve got RAM and flash caching, storage virtualization, and data-aware storage.

It is interesting too that each wave tends to build on those before it. Deduplication was once a marquee product but now is mainly a way to optimize the use of flash for hybrid or all-flash storage systems. SRM was the “next big thing” but is now an integrated feature of data-aware storage systems. Scale-out and virtual storage is easiest to do with an object store under the covers. And so on.

Each technology wave is accompanied by the same archetypal companies, too:

  • Academics and technologists who can’t articulate a use case but usually have the best technology
  • Re-packagers of outdated products trying to stay relevant by claiming they’re part of the current wave
  • Marketing-first companies making noise with little or no IP of their own but capable of making lots of noise
  • Super-funded leaders who hire and expand rapidly, are flush with cash, and become the standard-bearers for the wave
  • Actual companies selling useful products (too few of these!) who have solid products and rely on real sales and use cases

It is rare that any of these companies lasts more than a few years. But their products tend to live on, occasionally in standalone companies (usually the last two) but more typically as a product line or feature inside one of the giant incumbents.

And so it goes. Waves of innovation and waves of companies, crash on the storage market, but the same incumbent leaders and product lines survive for decades. It’s actually pretty depressing. Are things changing? It’s hard to see sometimes, but real progress has been made. And perhaps all those waves have finally chipped away at the outdated storage paradigms I complained about last time to send it all toppling down.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • How To Connect Everything From Everywhere with ZeroTier
  • How To Install ZeroTier on TrueNAS 12
  • Tortoise or Hare? Nvidia Jetson TK1

Filed Under: Computer History, Enterprise storage Tagged With: all-flash array, caching, CAS, cloud storage, companies, compression, data dispersal, data-aware storage, enterprise storage, history, hybrid storage, iSCSI, object storage, RAID, SRM, startups, thin provisioning, tiering

Primary Sidebar

I’m of the opinion that science fiction writers suck at predicting the future. We mostly go around describing the present in futuristic clothes

Cory Doctorow

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    A Fairy Tale of Two Storage Protocols

    September 23, 2014

    How Will Cisco Recover From The Consumer Strategy Blunder?

    January 2, 2013

    My 2012 Project: Improving Energy Efficiency

    January 3, 2012

    How Fast Is It? A Storage Infographic

    October 29, 2010

    Defining Failure: What Is MTTR, MTTF, and MTBF?

    July 6, 2011

    It’s Time To Move Beyond Passwords (Especially On Web Sites)

    January 8, 2016

    How To Connect Everything From Everywhere with ZeroTier

    January 14, 2022

    Cisco’s Trojan Horse

    September 15, 2014

    Rocking Out With the Topping VX1 Desktop/Bookshelf Amplifier

    October 6, 2015

    Free as in Coffee – Thoughts on the State of OpenStack

    May 2, 2016

    Footer

    Legalese

    Copyright © 2022 · Log in