• Skip to main content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / The I/O Blender Part 2: What Does Virtualization Do?

The I/O Blender Part 2: What Does Virtualization Do?

May 24, 2012 By Stephen 7 Comments

Virtualization is a disruptive technology in every sense of the word. By abstracting and simplifying physical resources, virtualization enables dynamic utilization. But this “translation” from physical to virtual disrupts the assumptions that enable performance and flexibility of physical devices such as storage arrays.

Virtualization disrupts the I/O path, invalidating the core assumptions that make storage arrays work

Your Assumption Is Invalid

See Storage Arrays Do A Few Things Very Well to learn what exactly is disrupted by virtualization

Traditionally, storage arrays relied on a static and predictable mapping between server, HBA, controller, LUN, and RAID set. This enabled the array to deliver the performance, data protection, and data movement features that would become the key selling point for enterprise storage companies. This was the topic of part 1 of this series.

But server virtualization changes all that. By presenting storage at the hypervisor level, server virtualization environments deprive the storage array of the information it needs to function effectively.

No longer can the array prefetch cache content, since a single front end port might have the interleaved I/O of a dozen servers. The hypervisor maintains its own virtual LUNs (vDisks) on top of a few large LUNs mounted from the array. And it mixes I/O in real time from multiple virtual machines without indicating which “flow” is which. Plus, most virtualization environments include clustered datastores, so I/O for the same one can come from multiple HBA’s at once.

The storage array can’t effectively move or copy a LUN, either, since each likely contains the data of a number of different servers. Increasingly, “hypervisor huggers” are relying on VMware for data protection features and storage presentation. This takes the array out of the picture entirely. And even if they want to use array-based copying or replication, the large LUNs they use are not granular, singular, or consistent with respect to servers.

Do Not Want!

If server virtualization disrupts the fundamental assumptions that enabled storage arrays to be of compelling value, what does this mean for the future of the enterprise storage industry? After all, server virtualization has its own compelling value even without traditional storage arrays in the mix. Many are of the opinion that server virtualization even trumps the need for the expensive, dedicated storage devices that are the hallmark of this industry.

Conventional storage arrays performed well because they could predict the I/O stream; now they perform poorly because prefetching doesn’t work very well and hard disk drives are terrible at random I/O. Buyers loved snapshot and replication technology, but these are no longer consistent because the LUN is not a valid unit of storage. Virtualization requires shared storage, but whole-LUN SCSI reservations limit scalability and shareability.

Old-school RAID arrays just don’t work well in a virtual world. Needless to say, this is a major problem for the incumbent storage array vendors.

VAAI Is Half of the Solution

VMware recognized the fact that storage arrays were no longer able effectively to function in a virtual world. The company responded by introducing an API for array integration (VAAI) in vSphere 4.1.

It is particularly telling that the three initial “primitives” of VAAI directly address the challenges raised by the “I/O blender”:

  • Atomic Test and Set (ATS) introduces sub-LUN locking, dramatically reducing contention for shared LUNs
  • Cloning Blocks (also called full copy or extended copy) allows storage arrays to make copies of data independent of large, static LUNs
  • Zeroing File Blocks addresses the communication barrier facing thin provisioned storage arrays, allowing them to free up unused capacity

Thanks to these primitives, VAAI recovers some of the value lost when enterprise storage arrays are used to support server virtualization workloads. Microsoft is introducing similar functionality with ODX in Windows Server 2012 and Hyper-V 3 this Summer.

We need a communications channel that bypasses the "I/O blender"

But VAAI cannot alleviate the “random I/O” issue facing block storage protocols. Once the “fruit” (virtual disks) is chopped up in the “blender”, there is no way for the storage array to reconstruct it. This dramatically reduces the impact of traditional caching and forces the use of flash for DRAM as a tier of storage instead. But solid-state storage remains much more expensive than spinning disks, and not all arrays are capable effectively of managing it.

File Servers Don’t Go Far Enough Either

Because only a few storage arrays supported VAAI, and because the quality of the support varied dramatically, many customers turned to NFS for virtual machine storage. Don’t get me wrong: NFS helps quite a bit, and is much more friendly to “non-storage people”, but it’s still not quite good enough.

NFS didn’t need many of the mitigations provided by VAAI to block storage. NFS never had the LUN locking issue, since, as a file protocol, each virtual machine disk had its own connection through the network. Similarly, NFS natively passes thin provisioning information to the storage array, so it does not need block zeroing assistance.

In fact, because each virtual machine disk is a file on the NFS server, it retains much of the information that is lost in the I/O blender on the block side. A specialized NFS server could act on this information intelligently to provide data protection and replication of individual virtual machine disk files. This is precisely what Tintri does, as a matter of fact.

But most legacy NFS servers are poorly suited to operations on individual files. Additionally, many NFS servers are simply incapable of handling the volume of I/O load generated by server virtualization in production. As is the case with block storage, the vast majority of NFS servers are not up to the job when it comes to supporting virtual machine workloads. On the whole, however, NFS is the superior protocol today.

Stephen’s Stance

Server virtualization is a big problem for conventional enterprise storage arrays. It reduces or eliminates the value of the very features customers seek when selecting an array. Although VAAI helps, it cannot completely eliminate the impact of the I/O blender.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying…
  • GPS Time Rollover Failures Keep Happening (But…
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • What You See and What You Get When You Follow Me
  • Ranting and Raving About the 2018 iPad Pro

Filed Under: Computer History, Enterprise storage, Virtual Storage Tagged With: Atomic Test and Set, ATS, block zeroing, full copy, Hypervisor Hugger, I/O blender, LUN, NFS, ODX, RAID, Tintri, VAAI, VMware

Primary Sidebar

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine.

Alan Turing

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

May 29, 2020

What You See and What You Get When You Follow Me

May 28, 2019

GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

April 6, 2019

Symbolic Links

    Featured Posts

    Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)

    August 4, 2011

    Deduplication Coming to Primary Storage

    September 16, 2008

    Review: American Standard’s Champion 4 Toilet Flushes Almost Anything

    July 31, 2012

    Why Big Disk Drives Require Data Integrity Checking

    December 19, 2014

    The Rack Endgame: Open Compute Project

    September 17, 2014

    What’s the Deal with Containers?

    October 21, 2016

    New England Takes On Seattle To Determine Who’s Number 2 … In Tech!

    January 19, 2015

    Scaling Storage Is Hard To Do

    June 4, 2013

    Electric Car Over the Internet: My Experience Buying From Vroom

    November 28, 2020

    Scaling Storage In Conventional Arrays

    November 19, 2013

    Copyright © 2021 · Log in