• Skip to main content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Virtualized and Distributed Storage: This Time For Sure!

Virtualized and Distributed Storage: This Time For Sure!

September 2, 2014 By Stephen 3 Comments

I’ve posted before about the spot where most successful enterprise storage arrays sit: The “fat middle” between performance and capacity. But software is driving a wedge into the SAN, and I see radical architectural change on the horizon. There are so many storage virtualization technologies sneaking into server architecture that it’s impossible to think one won’t take hold. And virtualized storage doesn’t want a “fat middle” array!

The principles of data physics suggest that data should be located closer to compute
The principles of data physics suggest that data should be located closer to compute

Here’s the Rack Endgame series:

  1. The Fat Middle: Today’s Enterprise Storage Array
  2. Virtualized and Distributed Storage: This Time For Sure!
  3. The Rack Endgame: A New Storage Architecture For the Data Center

No Demand For Storage Virtualization

Storage virtualization has been around longer than me in this industry, but it has never taken hold. As far back as I can remember, companies have been developing and trying to sell a universal platform that would allow storage to be more dynamic, moving from place to place according to application needs. Companies have also been working on distributed storage platforms for decades, applying the principle of data gravity which says that storage should be close to compute.

Yet, as I discussed in my post, “The Fat Middle: Today’s Enterprise Storage Array“, today’s storage architecture is remarkably simple: Centralized storage systems form the core of the datacenter, a “Jack of all trades” approach with no consistent virtualization outside the array. Why has storage virtualization or distribution never taken hold?

Perhaps the main reason for today’s lack of storage virtualization is the heterogeneity of data center operating systems and applications. Simply put, there is no single platform that can be targeted by a storage virtualization layer, and there has been no demand for an in-network virtualization platform either. The safest approach has been to let the storage array itself provide what little virtualization there is, along with high availability and data services.

VMware Provides an Opening

But VMware challenges this paradigm. Today’s most widely-used “storage array” is actually not an array at all: It’s VMFS, the storage layer of VMware vSphere. It provides all of the functions of a traditional storage array within the hypervisor itself. Yet VMware is in the process of radically transforming this storage layer, and the implications are very exciting! VMware Virtual SAN (colloquially called “VSAN”) brings storage distribution and virtualization to mainstream data centers, reducing the pressure on centralized storage arrays to provide advanced data services.

With VSAN, a vSphere data center no longer needs centralized storage! VSAN supports all of the best features of vSphere (HA, DRS) without SAN or NAS and brings data distribution to the party as well, localizing data close to compute. It even supports automated tiering (caching really, but that’s a semantic battle for a different day). As demonstrated by EVO:Rail, VSAN gives customers a mainstream SAN-less alternative.

But it’s not just VMware that’s challenging the status quo. Microsoft is promoting an all-Windows Server data center. Nutanix has made waves with their converged data center solution, and it too boasts a “No SAN” approach. SimpliVity, Atlantis Computing, and Scale Computing are also charging in. Then there’s the dark horses of storage virtualization: Avere, PernixData, and Infinio are virtualizing storage today, though they’ve been pretty quiet about the implications of this. It’s a short jump from a caching solution to a real storage hypervisor!

Endgame: Distributed Storage

The goal of all this is accommodation of data physics, specifically the “gravity” of data proposed by Dave McCrory. Because data should not be centralized, storage shouldn’t follow a central/shared architecture either. Storage should be moved to the compute nodes to reduce latency and bring about real scalability of the entire system.

Architecturally, this is what Nutanix, VSAN, and the rest are all about. They move data closer to compute, eliminating the need for an artificial, external shared storage infrastructure yet preserving the enterprise-class data services we’ve all come to rely on. They identify and move data by placing storage intelligence closer to applications (on the same physical hardware if not in the same operating system).

The benefits are too compelling to ignore and have profound implications for the shape of enterprise storage in the future. If we live in a “no SAN” world, we no longer need to build “Jack of all trades” storage systems or rely on “do it all” interconnects like Fibre Channel. The distributed storage world wants specialist storage devices: Ultra-performance flash on PCIe or memory channel and ultra-scale storage on a slower and cheaper bus.

Stephen’s Stance

We were never able to achieve storage virtualization in mainstream enterprise IT because we lacked the ability to identify and move data non-disruptively. This has been solved by caching and distributed storage solutions, and it’s only a matter of time before the legacy need for centralized storage falls away. As the wise man said, maybe not today and maybe not tomorrow, but soon we’ll live in a different world.

Note: These are topics I discuss in my public speaking engagements, including my Truth in Storage seminar with Truth in IT. Check out the schedule for Truth in Storage and come listen to the whole story!

Disclaimer: I work with VMware, PernixData, Infinio, Avere, Nutanix, Atlantis Computing, Scale Computing, SimpliVity, and most other companies in enterprise storage with Foskett Services and Tech Field Day. I don’t think I’m biased, but you can draw your own conclusions.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying…
  • GPS Time Rollover Failures Keep Happening (But…
  • What You See and What You Get When You Follow Me
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Introducing Rabbit: I Bought a Cloud!

Filed Under: Enterprise storage, Everything, Features, Virtual Storage Tagged With: Atlantis Computing, Avere, Data Gravity, Dave McCrory, distributed storage, Infinio, Nutanix, PernixData, Scale Computing, SimpliVity, storage virtualization, VMFS, VMware, vSAN

Primary Sidebar

It is often easier to ask for forgiveness than to ask for permission.

Grace Hopper

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

May 29, 2020

What You See and What You Get When You Follow Me

May 28, 2019

GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

April 6, 2019

Symbolic Links

    Featured Posts

    My Core i7 Macintosh SE

    May 25, 2017

    Co-Processors, GPGPU, and Heterogeneous Computing

    June 26, 2017

    We Live in the Future: Robotic Cat Litter Boxes!

    May 8, 2010

    Go Get a ProtonMail Account and Protect Your Online Life!

    July 19, 2017

    What You See and What You Get When You Follow Me

    May 28, 2019

    MacBook Users: Encrypt Your Drive with OS X FileVault! It’s Easy and Free!

    December 20, 2012

    Fasting to Mitigate Jet Lag: Surprise! It Works!

    February 11, 2013

    ZFS Is the Best Filesystem (For Now…)

    July 10, 2017

    From Kipling’s Dirigibles to the Jet Age

    May 13, 2012

    How Smart Is the Mondaine Helvetica Smart Watch?

    December 30, 2015

    Copyright © 2021 · Log in