The great battle of enterprise storage is on!

The time has come to take sides on the core question of storage for virtual servers: Do you want storage intelligence to live in the hypervisor or the array? Most administrators are already lining up on one side or the other, unintentionally casting their vote while the rest flounder. But the storage industry must wake up and embrace the divide.

Hypervisor Huggers Unite!

Do you "heart" virtualization?

VMware’s vSphere dominates the world of enterprise server virtualization and has become the core element of the modern open systems datacenter. Microsoft recognizes this but has been unable to field a competitive hypervisor ecosystem for the virtual datacenter. Today, vSphere is the state of the art and nowhere is this more apparent than in storage.

In just a few years, VMware has delivered and updated a host of advanced storage functionality, from provisioning to migration and load balancing to backup and data protection. vSphere 5 includes an advanced and scalable storage virtualization layer, delivering everything a datacenter needs. VMFS sculpts basic block storage into a shared resource for virtual machines, with snapshots, policy-based layout and movement, and flexible allocation and thin provisioning.

Most VMware administrators are “server guys” and relish these features. They have never experienced an automated “storage service” like this, and the enterprise storage world has never been able to construct anything remotely as flexible, user-friendly, and functional. And Hypervisor Huggers don’t need complex enterprise storage arrays to do it: They can use basic iSCSI or Fibre Channel devices to provide performance and capacity and let VMware do the rest!

Storage DRS is exemplary of the new virtual datacenter world. Introduced in vSphere 5 (and restricted to the pricey Enterprise Plus license), Storage DRS uses the core technology of Storage vMotion to dynamically balance I/O and capacity across a diverse pool of storage. Storage DRS even uses Policy-Driven Storage and VASA to enforce tiered storage and data placement strategy. This kind of virtualization has been a “holy grail” quest for the enterprise storage industry, but they’ve never delivered on their promises.

Cheers for Storage Stalwarts!

Do you wish all this virtualization nonsense would just go away?

But not every IT environment wants be 100% vSphere focused, and many aren’t convinced that dumb storage is the smartest place for their data. These Storage Stalwarts want smarter and better-integrated storage arrays, and VMware is innovating here as well.

VMware’s Storage APIs for Array Integration (VAAI) technology is squarely aimed at this market. VAAI allows vSphere to hand off heavy storage operations to the high-end storage arrays from the major players. It works transparently, too, coordinating cloning without the kind of scripting and hair-pulling that used to require. VAAI in vSphere 4.1 also includes block zeroing support and something called “atomic test and set” which we’ll get to in a moment. Microsoft announced their own cloning integration, ODX, but it won’t ship until Windows Server 8 appears sometime next year.

But cloning is just the tip of the iceberg when it comes to Hypervisor-to-array integration. The rising army of NAS users have seen the glory of more-complete array integration for a while, and they’re not quiet about it. They love that VMware’s NFS protocol support makes storage “disappear” in vCenter, becoming just another resource with integrated thin provisioning and flexible allocation and movement.

VMware is moving aggressively to please their Storage Stalwarts, adding more VAAI support for block and file in vSphere 5. But, as the company laid out at VMworld 2011, neither access method is ideal for virtual servers. So VMware has been pushing the enterprise array vendors for ever-greater integration. They see a future where a VAAI-based protocol enables arrays to de-multiplex I/O streams from the hypervisor and intelligently handle per-VM data.

Stephen’s Stance

You can spot a Hypervisor Hugger by their big LUNs: They would rather treat storage as a bulk commodity, and array vendors should be lining up to get their business. Storage Stalwarts will jump on each new VMware innovation, finally making use of the capabilities they have spent over a decade paying for but not utilizing. The only untenable stance is trying to keep a foot in both worlds: It’s foolish to buy an enterprise array and use it as bulk storage!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Are You a Hypervisor Hugger or a Storage Stalwart?
This post was categorized as Computer History, Enterprise storage, Features, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

VASA allows a "provider" application to tag vSphere storage with a "capabilities" string

The list of VMware storage integration features in vSphere is growing at a rapid pace. From backup to enterprise array offload, VMware is staking their claim. But information about one new storage feature in vSphere 5 has been scarce: The true nature of the Storage API for Storage Awareness (VASA) is only just beginning to be revealed.

VASA: Born of Necessity

vSphere has some amazing automated storage mobility features. The ease of moving data between LUNs and even arrays with Storage vMotion is a revelation to those of us unlucky enough to have used manual migration methods in the past. And VMware has automated this with the new Distributed Resource Scheduler (DRS) capability, allowing vSphere to make its own decisions about data placement.

But this kind of dynamic movement can hurt as well as help. What if an administrator moves a VMDK from high-performance to low-performance storage? Server responsiveness would suffer, perhaps even resulting in an application outage. And how could Storage DRS avoid doing the same thing?

VMware realized they needed some mechanism that could “tag” a storage resource with its capabilities. This is the real reason for their creation of Profile-Driven Storage and VASA.

VASA Does One Thing

In vSphere 5, VASA is incredible simple: It is a basic protocol for vSphere to request a “capabilities” string regarding a LUN or NFS share from a “provider”. The content of this response, and indeed the form that this provider will take, is left up to the individual storage vendors.

VASA Providers can take many forms. Dell is reportedly developing a vCenter plugin to report capabilities. EMC and NetApp will use a software application that runs on a virtual or physical server. In all cases, the software uses a proprietary call to the storage array, in effect relaying and translating the VMware request.

The VASA “Capabilities” string is undefined as well. Most vendors use a list of technical attributes as their response string, and each has adopted their own strategy about how to present information:

• Dell’s EqualLogic returns a comma-separated list like “RAID, SSD, REPLICATED”
• NetApp’s provider uses semicolons, as in “Dedupe; Replication”
• HP’s is very detailed, with a schema specifying Drive Type, RAID Type, Provisioning Type, VV Type, and Remotecopy
• EMC appears to have a standardized set of much-shorter tags, like “Performance”, “Multi-Tier”, “Capacity” and such

These are passed through to vCenter, where an administrator can decide how to interpret them.

VASA Today and Tomorrow

In its simplest form, VASA capabilities tags enable both manual and automatic storage placement features to respect tiered storage policies. This is important to maintain proper system performance and availability. In the future, I expect more advanced VASA providers, perhaps even integration of the providers into vCenter plugins. I also look forward to a standard capabilities tagging schema and smarter handling of returned tags. Right now, for example, EMC’s Clariion and Symmetrix lines both tag LUNs as “Performance”, but these are obviously not equivalent.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | What is VMware VASA? Not Much (Yet)
This post was categorized as Enterprise storage, Features, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Like Oedipus, VMware has a tragic destiny

Later this week, I’ll be presenting a session at Storage Expo NL which I imagine will be quite controversial: “VMware as Oedipus: How Server Virtualization will Change Storage Forever“. That’s right, I’m suggesting that VMware is the biggest threat to traditional enterprise storage, and that the company might just bump off its own parent (EMC) in the process. What the heck am I thinking? Read on!

Let’s Talk About Oedipus’ Fate

Most people have only a passing knowledge of the tragedy of Oedipus: he killed his father and slept with his mother, or something like that. But the classical plays about Oedipus revolve on a theme with much broader relevance: How do free will and fate interact?

Both Oedipus and his father, Laius, set up their shared tragedy in response to related prophecies. In trying to avoid their fate, both bring it about. This is a common theme in Greek tragedy and in life more generally.

Before Oedipus is born, his father, King Laius of Thebes, is told by an Oracle that he is doomed to perish by the hand of his own son. In response, he sends the infant away to be killed. Unable to do the deed, Oedipus’ mother, Jocasta, send him away with a servant who leaves him on a mountaintop. A shepherd rescues him and takes him to the childless King of Corinth, where he is raised.

On hearing rumors that he is not the biological son of the king, Oedipus goes to the Oracle at Delphi for advice. But the Oracle tells him that he is destined to murder his father and mate with his own mother, a repulsive fate indeed. To avoid this, Oedipus leaves Corinth and heads to Thebes.

On the road to Thebes, Oedipus meets an unknown man and the to quarrel over who has the right-of-way. Oedipus kills the man and continues on, solving the riddle of the Sphinx and freeing the kingdom from her curse. His reward is kingship and marriage to the Queen. The prophecy is fulfilled: The man on the road is King Laius, and Oedipus’ new wife is his own mother, Queen Jocasta.

The plays of Sophocles, Aeschylus, and Euripides explore the implications of the story and the impact that the revelation has on the characters involved. The most important question raised is that of fate versus free will: Could Laius have made his own fate worse by sending away his infant son? And could Oedipus have avoided the prophecy by choosing thought over action? Sophocles decided that fate could not be avoided, no matter what the motivations of the individuals involved.

The Fate of VMware In Enterprise IT

This is the prophecy that I bring to the audience at Storage Expo NL: As VMware adds both internal capabilities and external integration with storage devices, they are destined to come into conflict with the storage industry in general and their corporate parent, EMC, in particular.

Storage is one of the most critical gating factors to the success of server virtualization, so it comes as no surprise that VMware is rapidly innovating in this area. Integrating and developing snapshot, replication, thin provisioning, and other features in VMFS enables everyone to have advanced storage functionality, regardless of which storage device they use. In this way, VMware is already causing many users to forego an enterprise storage array purchase.

But VMware is owned by the King of Storage, EMC, and surely recognizes the benefits that enterprise storage arrays can bring to the hypervisor. Therefore, the VMware engineers are simultaneously attempting to add tighter integration to SAN and NAS storage systems. Truly, VMware does not want to destroy the enterprise storage industry anymore than Oedipus wanted to marry his own mother!

Now comes the tragedy. As VMware innovates on storage, they run up against the inevitable conclusion that current protocols and arrays are poorly suited to the sort of “multiplexed” random I/O that characterizes virtual server workloads. They’ve already noticed this, and responded with vCenter plugins, VAAI, VASA, and a proposal to add demultiplexers to enterprise storage arrays. Storage array vendors are scrambling to keep up with the rapid pace of innovation triggered by VMware’s integration features.

The irony of all this is that this ever tighter integration serves to anonymize and homogenize enterprise storage devices. If all array management is done within vCenter, and only VAAI-supported storage features are used, brand loyalty is no longer required. After all, one storage device is just as good as the next if all offer similar levels of capability and performance. The likely beneficiaries of the nomination of enterprise storage are the vendors of complete solutions: HP, Dell, IBM, and (yes) the EMC/Cisco combination.

But EMC and Cisco are not low cost providers. Their current success is based on the combination of innovation, integration, and support. But the big three server vendors are hot on their tail in all three areas and are better equipped to add value pricing as a sweetener. Again, if an HP server and storage combination works just as well a vBlock, with identical features and management in the VM environment, why not buy it?

Stephen’s Stance

VMware doesn’t want to hurt its parent, EMC, any more than Oedipus desired his own parents’ fate. Indeed, VMware spends an incredible amount of time and effort innovating both internal and external integration features for storage. They do this to meet their own I/O demands, not out of bloodlust or hubris. But like the tragic hero in a Greek play, VMware is destined to anonymize and homogenized enterprise storage, and this will drastically affect the future of EMC and other pure storage vendors.

Some Responses:

• Fate, Free Will, Virtualization, and Storage
• Is VMware forcing the homogenization of enterprise storage?

© sfoskett for Stephen Foskett, Pack Rat, 2011. | VMware as Oedipus: How Server Virtualization will Change Storage Forever
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

For more information on earlier updates, see my articles:

• Storage Fixes in VMware ESX Server 3.5 Update 2
• Storage Changes in VMware ESX 3.5 Update 3
• Storage Changes in VMware ESX 3.5 Update 4
  

One first step is VMware’s whitepaper, “What’s New in VMware vSphere 4: Storage“.

Licensing and Availability of Features

The most important change at a glance is in the licensing model for VMware.

• A new low-end “standard” level includes all of the components a small business might need, including thin provisioning of storage, at a very attractive price.
• The “advanced” level includes advances in the area of data protection.
• At the “enterprise” level, live migration of storage is enabled.
• Top of the heap is “enterprise plus”, which enables the intriguing plug-in third-party multipathing support previously rumored.
• Enterprise customers with current support will get vStorage thin provisioning and data protection when they upgrade.

Each step up the licensing ladder enables important new storage features (source: VMware)

Thin Provisioning

One of the most widespread storage features is native thin provisioning. VMware ESX 4 will allocate storage in 1 MB chunks as capacity is used. This isn’t really completely new – similar support was enabled by default for virtual disks on NFS in VI 3, and thin provisioning could be enabled on the command line for block-based storage as well. It was also present in VMware’s desktop products, including my own copy of Fusion. And ESX allows thick-to-thin conversion during Storage VMotion.

The difference with vSphere 4 is that thin provisioning is fully supported and integrated into every version of ESX. Although many storage arrays now also offer thin storage, the addition of native, integrated thin provisioning right in ESX is huge. This alone will be a major capacity (and thus, cost) savings feature! VMware claims 50% storage savings in their lab tests.

Dynamic Expansion of VMFS Volumes

VMFS volumes can now grow (and, in some cases, shrink) online without resorting to spanning to a new LUN. Under vSphere 4, VMFS volumes can grow to take advantage of expanded LUNs (up to 2 TB per LUN). The old method still works as well, and multi-LUN spanned VMFS volumes can grow when any of their LUNs is expanded.

Enhanced Storage VMotion

Like thin provisioning, Storage VMotion has been elevated to first-class status, supported just about everywhere you’d want it. It’s in all the likely spots within vCenter.

Storage VMotion gives serious storage flexibility now, enabling (almost) any-to-any migration of VMFS volumes: Pick up a Fibre Channel, iSCSI, or NFS disk image and move it to another datastore running any of those protocols to convert live. And you can do thick-to-thin provisioning at the same time.

Under the hood, the whole infrastructure has been revised. Storage VMotion leverages VMware’s change block tracking instead of disk snapshots now, speeding up the migration process and reducing the (formerly excessive) memory and CPU requirements of Storage VMotion in 3.5. This is the same technology leveraged by vSphere’s High Availability features, by the way.

Pluggable Storage Architecture (PSA)

Only “Enterprise Plus” licensees will get to use it, but the vSphere family also sports a new pluggable storage architecture (PSA) which will initially be leveraged to deliver vendor-specific multipath support. Note that the native multipath support in vSphere continues to be a basic round-robin or fail-over system – it will not dynamically load balance I/O across multiple paths or make more intelligent decisions about which paths to use.

vSphere 4's Pluggable Storage Architecture allows third-party developers to replace ESX's storage I/O stack (source: VMware)

As you may gather from this VMware illustration (but would probably miss since it’s not all that comprehensible), there are two classes of third-party plug-ins:

1. Basic path-selection plugins (PSPs) will merely optimize the choice of which path to use, ideal for active/passive type arrays
2. Full storage array type plugins (SATPs) will allow load balancing across multiple paths in addition to path selection for active/active arrays

EMC also announced PowerPath/VE for vSphere, integrating their popular multi-platform path management software directly into ESX. It’s not clear at this point whether PowerPath will require an Enterprise Plus license (or if it will come with one) or if it will work with all editions, but I’m sure that will be clarified soon. My EMC contacts do tell me that PowerPath/VE is licensed on a per-socket basis (like VMware of yore) and that EMC sales reps have some room to get creative on licensing.

iSCSI Software Initiator Enhancements

It’s harder to claim it as a new feature, but the iSCSI software initiator has also been tweaked and tuned to use less CPU time and deliver better throughput. The iSCSI configuration process has also been smoothed out so one no longer needs to have a live Service Console connection in order to communicate with an iSCSI target. And changes made in the general tab are now global, so they’ll propagate down to each target. Bi-directional CHAP is also added, so the target can now be authenticated in addition to the initiator.

vSphere also includes a paravirtualized iSCSI driver (PVSCSI) which works like vmxnet to present a higher-performance iSCSI adapter within certain supported guest OSes.

More information on multipathing iSCSI can be found at Yellow Bricks and in Chad Sakac’s mega-post on the topic.

High-Performance I/O

vSphere really pours on the I/O power. They claim a 3x increase, to “over 300,000 I/O operations per second”, but mentioned in the launch that they’ve gotten to 400,000 in some workloads. This is really amazing, and if it’s true in real-world use means that VMware ESX can now host just about any application you can think of.

It’s funny, but just a few weeks before the announcement VMware’s Chief Data Center architect, Scott Davis, told me at Storage Networking World that vSphere would double I/O performance. I thought this was a bold statement at the time, but VMware went further in their official documents, claiming 3x improvement. Now we have an on-stage admission that it could be up to 4x! This is serious stuff, folks: Can VMware really do 400,000 IOPS?

One question is whether these IOPS improvements require the new VMDirectPath I/O for Storage, which binds a physical Fibre Channel HBA to a single guest OS, or if they’re generalized across all systems.

vCenter Improvements

vCenter now includes more storage information:

• Each VM and ESX in the inventory has a tab showing storage information and allowing users to set alarms on storage use – setting capacity alarms becomes extremely important when thin provisioning is used!
• vCenter inventory has a view showing datastore details
• There’s also a nice storage topology map

Data Protection APIs (Updated)

VMware also enhanced the APIs for consolidated backup (VCB) into “vStorage Data Protection”. Available only with Advanced or Enterprise licenses, these APIs allow enterprise backup vendors to develop specific integration with VMware without VCB in the middle.  All versions of ESX include this vStorage API and boast VCB support. Standard and Essentials licenses do not include vCenter Data Recovery, a standalone disk-to-disk backup capability for virtual machines.

Conclusion

vSphere is loaded with storage improvements, though many seem like incremental updates rather than new features. Admins will welcome thin provisioning support, and the full integration of Storage VMotion is certainly welcome, but I think the changes to vCenter will be most noticeable. I’ll be looking into EMC’s PowerPath/VE and the amazing I/O improvements as we move forward and learn more about the next ESX!

Duncan Epping posted much more vSphere Linkage on his Yellow Bricks blog if you’re interested in non-storage features. I also suggest reading Chris Evans’ outline of vSphere storage changes.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Storage Changes in the VMware vSphere 4 Family
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.