For more information on earlier updates, see my articles:

• Storage Fixes in VMware ESX Server 3.5 Update 2
• Storage Changes in VMware ESX 3.5 Update 3
• Storage Changes in VMware ESX 3.5 Update 4
  
• Storage Changes in the VMware vSphere 4 Family

One first step is VMware’s whitepaper, “What’s New in VMware vSphere 5.0 – Storage“.

Licensing and Availability of Features

VMware has once again changed the licensing and pricing model, throwing the Internet into a tizzy:

• “Advanced” has been eliminated, moving up to “Enterprise”
• Pooled vRAM entitlements work across the entire vCenter environment
• New features like Policy-Driven Storage and Storage DRS (along with SIOC) are exclusive to “Enterprise Plus” licenses
• VAAI, PSP multipathing, and Storage vMotion are only found in “Enterprise”
• Thin Provisioning and VADP are available regardless of edition

VMware mucked with vSphere licensing again...

Major New vSphere 5 Storage Features

Storage DRS

Storage DRS is the world’s worst-kept secret, with everyone and his brother talking about it for over a year. Like the existing VM DRS capability, Storage DRS creates resource clusters and automatically moves VMs between them. Storage DRS uses utilization and performance metrics to make the call, and has three modes of operation. It sounds awesome, but it’s an Enterprise Plus-only feature.

Storage APIs – Storage Awareness (VASA)

There’s not much information presently, but a VAAI companion is introduced in vSphere 5: The vSphere Storage APIs for Storage Awareness (VASA) is a communication mechanism for vCenter to detect array capabilities like RAID Level, Thin Provisioning State, Replication State, etc. This will come in handy for all the other features in vSphere 5, especially policy-driven storage!

Policy-Driven Storage

Another new Enterprise Plus feature is Policy-Driven Storage. This allows storage tiers to be defined in vCenter based on SLA, performance, and other metrics which are used during provisioning, cloning, Storage vMotion, and Storage DRS. It leverages VASA for metrics and characterization and supports all arrays in the HCL, regardless of whether they’re NFS, iSCSI, or FC. It includes easy compliance status reporting in vCenter as well.

FCoE Software Initiator

Those of us “in the know” about storage expected VMware to add software FCoE support, so it’s no surprise that they did. This dramatically expands the potential FCoE footprint from just the few CNAs already supported in vSphere 4. It appears to be based on Intel’s OpenFCoE, since it shows up as “Intel Corporation FCoE Adapter” in the GUI!

vSphere Storage Appliance (VSA)

VMware enters the virtual storage appliance (VSA) market with their own offering, the vSphere Storage Appliance (also called VSA). Aimed primarily at the SMB market, it’s actually fairly clever, replicating storage between two or three nodes in a cluster for high availability and using NFS for access rather than iSCSI. And unlike the Celerra UBER that so many techies loved, the VMware VSA is ready for production use!

Existing Storage Features Enhanced in vSphere 5

VMFS 5

VMFS has been improved for scalability and efficiency, but the 2 TB limit on VMDKs remains (except for physical RDM). Only storage geeks like me need to worry about the specifics, but suffice to say that VMFS 5 requires less tuning and worrying and ought to scale and perform better thanks to increased maximums and leveraging the Atomic Test and Set (ATS) technology also used in VAAI. For newly-created volumes, there’s no more block size tuning, and alignment issues are addressed.

Storage APIs – Array Integration (VAAI 2)

See also VMware VAAI Storage Array Support in Plain English

VAAI has been revved, bringing back Thin Provisioning Stun (the AWOL “fourth primitive”) and adding NFS support.

There are now five block primitives for VAAI, depending on if you count thin space reclaim. This is really more of a bug fix than anything, since most folks assumed that the existing thin support already reclaimed deleted VMs and vMotioned VMDKs. I’m more interested in the addition of SCSI UNMAP in addition to WRITE_SAME! There’s also some additional T10 support, though I’m not clear on what it is or where it works.

We’ve also got VAAI for NFS environments now. NAS had sweet thin provisioning support even before block datastores, but the new Reserve Space command adds thick provisioning if that’s what you like. We’ve also got Full File Clone, which is like Full Copy for NFS but doesn’t work with Storage vMotion. And there’s some Extended Stats API to bring in more detail on file status. I also hear there’s an API for Native Snapshot Support, but it’s not widely discussed. Finally, note that NFS plugins come from vendors, not VMware as is the case for block VAAI.

Storage I/O Control

SIOC is enhanced for use in Storage DRS environments, becoming aware of the new datastore clusters. It also gets NFS support, and presumably uses VASA for metrics. But it’s still only available with Enterprise Plus licenses.

iSCSI Initiator GUI

The solid vSphere iSCSI initiator gets a friendly GUI for configuration. I like friendly iSCSI GUIs – just ask Microsoft!

Storage vMotion

The mechanism behind Storage vMotion has changed for a third time in as many releases, this time using “Mirror Mode” to mirror writes to in-progress vMotions. It also now supports migration of vSphere snapshots and Linked Clones. This can be offloaded for VAAI block, but not NFS.

vSphere Replication

New in vCenter Site Recovery Manager 5.0 is software-based replication. Although not technically a vSphere 5 feature, this is a major new storage feature in the VMware world. It allows any-to-any software-based storage replication for disaster recovery.

Stephen’s Stance

Once again, VMware added a ton of new storage enhancements to vSphere. With storage rapidly becoming the limiting factor in scalability and performance of virtual machine environments, this is no surprise. Also not surprising is the fact that major features like Policy-Driven Storage and Storage DRS (along with SIOC) are exclusive to “Enterprise Plus” licenses. I can’t blame VMware for making a buck, but it would be nice if more capabilities were available to the small shops!

I’ll be writing on all these features in detail shortly. Watch this space!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Storage Changes in VMware vSphere 5
This post was categorized as Enterprise storage, Features, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Apple

• MacBook Users: Turn off This Bluetooth Default Setting Now – Now I know what turned on my MacBook Pro in the bag: My BlueTooth mouse!

Enterprise Computing

• Windows Storage Server-Based Systems Step Into 2008 – My Gestalt IT coverage of the features of WSS08.
• EMC Symmetrix V-Max: When Does It Get FAST and Virtual? – Another Gestalt IT piece, pointing out that the V-Max isn’t going to be fully realized for a very long time
• Microsoft Unveils Final Windows Storage Server – Paul Shread runs with the “Windows Storage Server 2008″ theme, quoting yours truly extensively.
• What is a best practice and Are Best Practices Just Shared Opinions? – W. Curtis Preston and I pickup an old conversation about the definition of “best practice”.
• Infosmack Episode 2 – EMC, David Donatelli and Non-Compete Agreements – Tony Asaro and I join Marc Farley and Greg Knieriemen on the Storage Monkeys podcast!
• The US Federal Government defines Cloud Computing - Ruven Cohen covers the US Federal Government’s cloud computing meetings and gives us a sneak peek at the new definition of cloud computing from the world’s number one IT user.
• Support and Q&A for Solid-State Drives – Lots of great info on how Windows 7 (and probably Server 2008 R2) deals with solid state drives.
• VMware Slaps Enterprise and Cisco In Face, Opens Door For Competitors – More on the VMware vSphere 4 licensing changes, and what it might mean.

Misc

• Non-competes are evil – Robin Harris lays it out. I happen to agree.
• Five Technologies Our Kids Won’t Even Recognize – Wired’s Gadget Lab makes us all feel old!
• Chrysler HQ Designed To Convert Into Shopping Mall - Wait Wait Don’t Tell Me says it’s true, so I believe it!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Back From the Pile: Interesting Content From the Week of May 9, 2009
This post was categorized as Everything. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Enterprise IT

• Reduce, Reuse, and Recycle Your Reliance On Backup Tapes – What’s wrong with backup tapes? They’re inaccessible, making them unsuitable for most applications. My latest post for my Enterprise Storage Strategies blog.
• Is Licensing Turning vSphere Into Vista? – A group post for Gestalt IT suggesting that VMware’s tiered licensing strategy for the vSphere 4 family might cause anguish for customers
• Infosmack Episode 2 – EMC, David Donatelli and Non-Compete Agreements – I joined the StorageMonkeys Infosmack podcast to discuss EMC, Dave Donatelli, and con-compete agreements with Greg Knieriemen, Marc Farley of 3Par and StorageRap, and Tony Asaro of the INI Group
• IT utilities, the biggest game in town – Chris Mellor of The Register suggests that the biggest IT vendors are positioning to lock up the IT market.
• The Non-Compete Clause Debate….. - Steve Duplessie muses on the pros and cons of non-compete agreements and what they mean for the IT industry
• Who’s the pot & who’s the kettle? - W. Curtis Preston reacts to the skirmishes between Tony Asaro and the EMC bloggers
• The Only VMware-certified NAS in its Category: Why the Iomega StorCenter ix4-200r Will Resonate with Small Businesses - Jerome Wendt expands on the Iomega StorCenter Pro ix4-200r
• Don’t focus too much on costs, cloud computing is about business agility - CapGemini reminds us that cloud computing isn’t just about reducing costs
• What we talk about when we talk about cloud computing - The Google Enterprise blog lays down some of the hard facts about building a cloud on your own
• Review: Sun Storage 7000 Unified Storage System - Chris Evans provides one of the best reviews yet of Sun’s Amber Road open source-based storage systems
• Brocade Pokes Cisco in the Eye, Switches for IBM - A great headline and solid reporting by Om Malik on one of the first reactions to Cisco UCS
• TheInfoPro: Analysis or Marketing? - StorageMonkeys continues their probing of the storage analyst community
• A “Tweeterview” with Stephen Foskett, Nirvanix Consulting Director - Sunshine interviews me about Nirvanix, storage, and my new role

Apple

• Achieving Email Bliss with IMAP, Gmail, and Apple Mail – A deep, long post on GMail, IMAP, Mac OS X Mail, and how to get all three to play nicely together.
• Monitor Your Mac Remotely With iStat for iPhone - I love iStat for Mac OS X and the iPhone and had been meaning to write a review. This AppleBlog post is just about what I might have written!

Life

• There Goes Chrysler – NPR’s awesome Planet Money blog reported the breaking news of Chrysler’s bankruptcy
• The Venture Capital Math Problem and The Venture Capital Math Problem (continued) – Fred Wilson points out that there can be no such thing as unlimited venture capital
• Is it time to cut the Ethernet access cable? – Has pervasive wireless Ethernet made wired ports unnecessary?
• Interview with Micah Baldwin, Father of FollowFriday – Micah calls me out as someone who is doing interesting things on Twitter!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Back From the Pile: Interesting Content From the Week of May 2, 2009
This post was categorized as Everything. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

For more information on earlier updates, see my articles:

• Storage Fixes in VMware ESX Server 3.5 Update 2
• Storage Changes in VMware ESX 3.5 Update 3
• Storage Changes in VMware ESX 3.5 Update 4
  

One first step is VMware’s whitepaper, “What’s New in VMware vSphere 4: Storage“.

Licensing and Availability of Features

The most important change at a glance is in the licensing model for VMware.

• A new low-end “standard” level includes all of the components a small business might need, including thin provisioning of storage, at a very attractive price.
• The “advanced” level includes advances in the area of data protection.
• At the “enterprise” level, live migration of storage is enabled.
• Top of the heap is “enterprise plus”, which enables the intriguing plug-in third-party multipathing support previously rumored.
• Enterprise customers with current support will get vStorage thin provisioning and data protection when they upgrade.

Each step up the licensing ladder enables important new storage features (source: VMware)

Thin Provisioning

One of the most widespread storage features is native thin provisioning. VMware ESX 4 will allocate storage in 1 MB chunks as capacity is used. This isn’t really completely new – similar support was enabled by default for virtual disks on NFS in VI 3, and thin provisioning could be enabled on the command line for block-based storage as well. It was also present in VMware’s desktop products, including my own copy of Fusion. And ESX allows thick-to-thin conversion during Storage VMotion.

The difference with vSphere 4 is that thin provisioning is fully supported and integrated into every version of ESX. Although many storage arrays now also offer thin storage, the addition of native, integrated thin provisioning right in ESX is huge. This alone will be a major capacity (and thus, cost) savings feature! VMware claims 50% storage savings in their lab tests.

Dynamic Expansion of VMFS Volumes

VMFS volumes can now grow (and, in some cases, shrink) online without resorting to spanning to a new LUN. Under vSphere 4, VMFS volumes can grow to take advantage of expanded LUNs (up to 2 TB per LUN). The old method still works as well, and multi-LUN spanned VMFS volumes can grow when any of their LUNs is expanded.

Enhanced Storage VMotion

Like thin provisioning, Storage VMotion has been elevated to first-class status, supported just about everywhere you’d want it. It’s in all the likely spots within vCenter.

Storage VMotion gives serious storage flexibility now, enabling (almost) any-to-any migration of VMFS volumes: Pick up a Fibre Channel, iSCSI, or NFS disk image and move it to another datastore running any of those protocols to convert live. And you can do thick-to-thin provisioning at the same time.

Under the hood, the whole infrastructure has been revised. Storage VMotion leverages VMware’s change block tracking instead of disk snapshots now, speeding up the migration process and reducing the (formerly excessive) memory and CPU requirements of Storage VMotion in 3.5. This is the same technology leveraged by vSphere’s High Availability features, by the way.

Pluggable Storage Architecture (PSA)

Only “Enterprise Plus” licensees will get to use it, but the vSphere family also sports a new pluggable storage architecture (PSA) which will initially be leveraged to deliver vendor-specific multipath support. Note that the native multipath support in vSphere continues to be a basic round-robin or fail-over system – it will not dynamically load balance I/O across multiple paths or make more intelligent decisions about which paths to use.

vSphere 4's Pluggable Storage Architecture allows third-party developers to replace ESX's storage I/O stack (source: VMware)

As you may gather from this VMware illustration (but would probably miss since it’s not all that comprehensible), there are two classes of third-party plug-ins:

1. Basic path-selection plugins (PSPs) will merely optimize the choice of which path to use, ideal for active/passive type arrays
2. Full storage array type plugins (SATPs) will allow load balancing across multiple paths in addition to path selection for active/active arrays

EMC also announced PowerPath/VE for vSphere, integrating their popular multi-platform path management software directly into ESX. It’s not clear at this point whether PowerPath will require an Enterprise Plus license (or if it will come with one) or if it will work with all editions, but I’m sure that will be clarified soon. My EMC contacts do tell me that PowerPath/VE is licensed on a per-socket basis (like VMware of yore) and that EMC sales reps have some room to get creative on licensing.

iSCSI Software Initiator Enhancements

It’s harder to claim it as a new feature, but the iSCSI software initiator has also been tweaked and tuned to use less CPU time and deliver better throughput. The iSCSI configuration process has also been smoothed out so one no longer needs to have a live Service Console connection in order to communicate with an iSCSI target. And changes made in the general tab are now global, so they’ll propagate down to each target. Bi-directional CHAP is also added, so the target can now be authenticated in addition to the initiator.

vSphere also includes a paravirtualized iSCSI driver (PVSCSI) which works like vmxnet to present a higher-performance iSCSI adapter within certain supported guest OSes.

More information on multipathing iSCSI can be found at Yellow Bricks and in Chad Sakac’s mega-post on the topic.

High-Performance I/O

vSphere really pours on the I/O power. They claim a 3x increase, to “over 300,000 I/O operations per second”, but mentioned in the launch that they’ve gotten to 400,000 in some workloads. This is really amazing, and if it’s true in real-world use means that VMware ESX can now host just about any application you can think of.

It’s funny, but just a few weeks before the announcement VMware’s Chief Data Center architect, Scott Davis, told me at Storage Networking World that vSphere would double I/O performance. I thought this was a bold statement at the time, but VMware went further in their official documents, claiming 3x improvement. Now we have an on-stage admission that it could be up to 4x! This is serious stuff, folks: Can VMware really do 400,000 IOPS?

One question is whether these IOPS improvements require the new VMDirectPath I/O for Storage, which binds a physical Fibre Channel HBA to a single guest OS, or if they’re generalized across all systems.

vCenter Improvements

vCenter now includes more storage information:

• Each VM and ESX in the inventory has a tab showing storage information and allowing users to set alarms on storage use – setting capacity alarms becomes extremely important when thin provisioning is used!
• vCenter inventory has a view showing datastore details
• There’s also a nice storage topology map

Data Protection APIs (Updated)

VMware also enhanced the APIs for consolidated backup (VCB) into “vStorage Data Protection”. Available only with Advanced or Enterprise licenses, these APIs allow enterprise backup vendors to develop specific integration with VMware without VCB in the middle.  All versions of ESX include this vStorage API and boast VCB support. Standard and Essentials licenses do not include vCenter Data Recovery, a standalone disk-to-disk backup capability for virtual machines.

Conclusion

vSphere is loaded with storage improvements, though many seem like incremental updates rather than new features. Admins will welcome thin provisioning support, and the full integration of Storage VMotion is certainly welcome, but I think the changes to vCenter will be most noticeable. I’ll be looking into EMC’s PowerPath/VE and the amazing I/O improvements as we move forward and learn more about the next ESX!

Duncan Epping posted much more vSphere Linkage on his Yellow Bricks blog if you’re interested in non-storage features. I also suggest reading Chris Evans’ outline of vSphere storage changes.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Storage Changes in the VMware vSphere 4 Family
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.