• My writing:
  • How Will Thunderbolt Affect Enterprise Storage?
  • Introducing Storage for Virtual Environments (From My Seminar)
• Other great stuff:
  • 1394, DMA, and BitLocker
  • Open-FCoE: Will Software Initiators Win Again?
  • NetApp’s Purchase of Engenio
  • VMware View Client for iPad Released
  • VMware’s Consolidated Backup (VCB) & User Requirements
  • The Ultimate In-N-Out Secret Menu (and Super Secret Menu!) Survival Guide
  • Why I Am Dropping Android Phones And Moving To iPhone
  • HP Networking – Part 2(More vision…)
  • Content-Type: joke; genre=bar/packet
  • Gestalt IT Tech Field Day – The First Ever Wireless #TechFieldDay
  • Site Survey Rigs

Subscribe to my Google Reader feed or follow me on Twitter to see these in real-time.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Back From the Pile: Interesting Links, March 11, 2011
This post was categorized as Everything. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Now I have been asked to write an article for TechTarget on the very same subject: How to select a virtual server backup product. I’d like to request input for this piece, and hope we can work together to produce a useful list of recommendations. Note that this isn’t a buyer’s guide like the DCIG effort: There will be no exhaustive lists of functions and features here. Instead, I’m writing about the options available in a more general sense.

I came away from the Tech Field Day presentations by Veeam, Symantec, and others impressed by the creativity of virtual server backup approaches. Clearly, these aren’t the same old backup products! W. Curtis Preston quipped that a hypervisor sometimes seems like a little datacenter in a box, and that changes everything. Every action taken impacts the operation of the whole system, but the virtualization layer also adds new flexibility.

Virtual Server Backup Features

What would you look for in a virtual server backup product? Here’s my initial concept:

• Supported environments - What dependencies do the products have?
  • Do they require integration features like VCP or VADP? We should definitely talk more about the integration features offered by the hypervisor!
  • Some products support only virtual environments, while others work on traditional machines as well. And what about non-VMware support?
• Advantages and disadvantages – Applications designed for virtual environments certainly offer many advantages, but what about the drawbacks?
  • How important is it to use a single unified product across both virtual and non-virtual environments, as many traditional backup vendors suggest?
  • Is the whole “full-machine vs. file-level” argument worth having?
• Application integration – To what extent to they extend above the OS and hypervisor?
  • I’d like to talk about integration with common virtualized applications, but which are most prevalent?
  • We should talk about crash-consistent backup, too, especially when it comes to virtual machines.
  • Then there’s management integration: What options are there for automated deployment and setup? And what metrics and reports are important?
• Advanced features – These new products have some pretty impressive features!
  • Is deduplication a critical need in the backup app?
  • We should also talk about non-scheduled approaches like continuous data protection (CDP)
  • There are also alternative backup targets, like snapshots, disk/VTL, and the cloud
  • I’m also curious about throttling/QoS features
• Other areas
  • What about supportability of virtual machine backups – are there issues with companies not allowing advanced backup features to be used?
  • Do the “new guard” of virtual server backup companies offer any unique support and integration options?

I’d like to identify a few key features to look for in particular. What are the most-important elements for a virtual server backup product, as opposed to a traditional offering?

Finally, what about non-VMware environments. I know a bit about Microsoft DPM, having been at the introduction of that product, but what offerings are there for Hyper-V or Xen environments? How do they differ from VMware-focused products?

Stephen’s Stance

I welcome the input of end-users, analysts and consultants, and the product vendors themselves. I want to come up with a good guide to selecting these products, and I think an open discussion is a good way to make that happen. Who knows, it might even be a worthwhile companion to the information in the DCIG guide!

I’m not sure how the final article will be released, but I believe it will be offered as a free download on BitPipe.com. Regardless, the discussion and commentary here will remain free, open, and available. Thank you!

Updates

I’ve already had some excellent feedback from Twitter. Thanks!

• @Stu suggested I read What’s Next in VMware Backup
• @ChrisDearden says “what about talking about how to get the data out of the backup system – ie to offsite / archive etc” and “I’m not sure if trap is the word I would use. no more than all the tape based providers use the same format (“
• @WGoderis says “what about scalability , what if you need to back up hundreds or thousands of vms”

© sfoskett for Stephen Foskett, Pack Rat, 2011. | How To Select a Virtual Server Backup Product?
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

For more information on earlier updates, see my articles:

• Storage Fixes in VMware ESX Server 3.5 Update 2
• Storage Changes in VMware ESX 3.5 Update 3
• Storage Changes in VMware ESX 3.5 Update 4
  

One first step is VMware’s whitepaper, “What’s New in VMware vSphere 4: Storage“.

Licensing and Availability of Features

The most important change at a glance is in the licensing model for VMware.

• A new low-end “standard” level includes all of the components a small business might need, including thin provisioning of storage, at a very attractive price.
• The “advanced” level includes advances in the area of data protection.
• At the “enterprise” level, live migration of storage is enabled.
• Top of the heap is “enterprise plus”, which enables the intriguing plug-in third-party multipathing support previously rumored.
• Enterprise customers with current support will get vStorage thin provisioning and data protection when they upgrade.

Each step up the licensing ladder enables important new storage features (source: VMware)

Thin Provisioning

One of the most widespread storage features is native thin provisioning. VMware ESX 4 will allocate storage in 1 MB chunks as capacity is used. This isn’t really completely new – similar support was enabled by default for virtual disks on NFS in VI 3, and thin provisioning could be enabled on the command line for block-based storage as well. It was also present in VMware’s desktop products, including my own copy of Fusion. And ESX allows thick-to-thin conversion during Storage VMotion.

The difference with vSphere 4 is that thin provisioning is fully supported and integrated into every version of ESX. Although many storage arrays now also offer thin storage, the addition of native, integrated thin provisioning right in ESX is huge. This alone will be a major capacity (and thus, cost) savings feature! VMware claims 50% storage savings in their lab tests.

Dynamic Expansion of VMFS Volumes

VMFS volumes can now grow (and, in some cases, shrink) online without resorting to spanning to a new LUN. Under vSphere 4, VMFS volumes can grow to take advantage of expanded LUNs (up to 2 TB per LUN). The old method still works as well, and multi-LUN spanned VMFS volumes can grow when any of their LUNs is expanded.

Enhanced Storage VMotion

Like thin provisioning, Storage VMotion has been elevated to first-class status, supported just about everywhere you’d want it. It’s in all the likely spots within vCenter.

Storage VMotion gives serious storage flexibility now, enabling (almost) any-to-any migration of VMFS volumes: Pick up a Fibre Channel, iSCSI, or NFS disk image and move it to another datastore running any of those protocols to convert live. And you can do thick-to-thin provisioning at the same time.

Under the hood, the whole infrastructure has been revised. Storage VMotion leverages VMware’s change block tracking instead of disk snapshots now, speeding up the migration process and reducing the (formerly excessive) memory and CPU requirements of Storage VMotion in 3.5. This is the same technology leveraged by vSphere’s High Availability features, by the way.

Pluggable Storage Architecture (PSA)

Only “Enterprise Plus” licensees will get to use it, but the vSphere family also sports a new pluggable storage architecture (PSA) which will initially be leveraged to deliver vendor-specific multipath support. Note that the native multipath support in vSphere continues to be a basic round-robin or fail-over system – it will not dynamically load balance I/O across multiple paths or make more intelligent decisions about which paths to use.

vSphere 4's Pluggable Storage Architecture allows third-party developers to replace ESX's storage I/O stack (source: VMware)

As you may gather from this VMware illustration (but would probably miss since it’s not all that comprehensible), there are two classes of third-party plug-ins:

1. Basic path-selection plugins (PSPs) will merely optimize the choice of which path to use, ideal for active/passive type arrays
2. Full storage array type plugins (SATPs) will allow load balancing across multiple paths in addition to path selection for active/active arrays

EMC also announced PowerPath/VE for vSphere, integrating their popular multi-platform path management software directly into ESX. It’s not clear at this point whether PowerPath will require an Enterprise Plus license (or if it will come with one) or if it will work with all editions, but I’m sure that will be clarified soon. My EMC contacts do tell me that PowerPath/VE is licensed on a per-socket basis (like VMware of yore) and that EMC sales reps have some room to get creative on licensing.

iSCSI Software Initiator Enhancements

It’s harder to claim it as a new feature, but the iSCSI software initiator has also been tweaked and tuned to use less CPU time and deliver better throughput. The iSCSI configuration process has also been smoothed out so one no longer needs to have a live Service Console connection in order to communicate with an iSCSI target. And changes made in the general tab are now global, so they’ll propagate down to each target. Bi-directional CHAP is also added, so the target can now be authenticated in addition to the initiator.

vSphere also includes a paravirtualized iSCSI driver (PVSCSI) which works like vmxnet to present a higher-performance iSCSI adapter within certain supported guest OSes.

More information on multipathing iSCSI can be found at Yellow Bricks and in Chad Sakac’s mega-post on the topic.

High-Performance I/O

vSphere really pours on the I/O power. They claim a 3x increase, to “over 300,000 I/O operations per second”, but mentioned in the launch that they’ve gotten to 400,000 in some workloads. This is really amazing, and if it’s true in real-world use means that VMware ESX can now host just about any application you can think of.

It’s funny, but just a few weeks before the announcement VMware’s Chief Data Center architect, Scott Davis, told me at Storage Networking World that vSphere would double I/O performance. I thought this was a bold statement at the time, but VMware went further in their official documents, claiming 3x improvement. Now we have an on-stage admission that it could be up to 4x! This is serious stuff, folks: Can VMware really do 400,000 IOPS?

One question is whether these IOPS improvements require the new VMDirectPath I/O for Storage, which binds a physical Fibre Channel HBA to a single guest OS, or if they’re generalized across all systems.

vCenter Improvements

vCenter now includes more storage information:

• Each VM and ESX in the inventory has a tab showing storage information and allowing users to set alarms on storage use – setting capacity alarms becomes extremely important when thin provisioning is used!
• vCenter inventory has a view showing datastore details
• There’s also a nice storage topology map

Data Protection APIs (Updated)

VMware also enhanced the APIs for consolidated backup (VCB) into “vStorage Data Protection”. Available only with Advanced or Enterprise licenses, these APIs allow enterprise backup vendors to develop specific integration with VMware without VCB in the middle.  All versions of ESX include this vStorage API and boast VCB support. Standard and Essentials licenses do not include vCenter Data Recovery, a standalone disk-to-disk backup capability for virtual machines.

Conclusion

vSphere is loaded with storage improvements, though many seem like incremental updates rather than new features. Admins will welcome thin provisioning support, and the full integration of Storage VMotion is certainly welcome, but I think the changes to vCenter will be most noticeable. I’ll be looking into EMC’s PowerPath/VE and the amazing I/O improvements as we move forward and learn more about the next ESX!

Duncan Epping posted much more vSphere Linkage on his Yellow Bricks blog if you’re interested in non-storage features. I also suggest reading Chris Evans’ outline of vSphere storage changes.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Storage Changes in the VMware vSphere 4 Family
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Hopefully, VMware's three pillars (vCloud, vClient, and VDC-OS) are a little more solid (not to mention closer) than the Three Pillars of Creation in the Eagle Nebula

VMware packed a lot into their 2008 VMworld conference, including an outline of their “three pillars” strategy, focused on vClient, vCloud, and something they are calling the Virtual Datacenter Operating System. While it is debatable if this last item really is an operating system, it’s certainly a major strategic change in messaging.

VDC-OS is divided into four “vServices” (Management, Cloud, Application, and Infrastructure), and one core Infrastructure vService is vStorage. Since my focus is enterprise storage, I thought I would take a moment to examine the current and future status of vStorage.

EMC’s Chad Sakac has taken up the challenge of communicating vStorage to the world, in a post to his blog, a pair of YouTube videos (multipathing, I/O dedupe), and (apparently) a session or two at VMworld. But I’m not sure it’s really entirely clear just what vStorage is and what this means to existing and future VMware storage developments. So let’s dive in and take a look.

vStorage = Storage vService for VMware Infrastructure

Let’s get one thing out of the way immediately: VDC-OS is not a new product – it’s a framework to organize VMware’s existing and future datacenter-targeted products like ESX. The company is sweeping away the current hodgepodge of server product elements and placing them all into a more-organized structure called VDC-OS.

As such, the whole thing is not fully baked at present – there are lots of missing elements, and these point to future products from VMware and others. I applaud the organization, and feel that this is the first really viable next-generation datacenter vision that I have seen. It accurately reflects the current reality of the corporate data center and includes a migration path to take it to a compelling new level.

VMware has a realistic approach to bringing current datacenter elements into the future

As we can see in VMware’s diagram (above), the Virtual Datacenter OS is made up of VMware Infrastructure, with application vServices resting on infrastructure vServices or the cloud. This last bit is the fundamental breakthrough that makes VDC-OS more than just slideware – if it works, applications (virtual appliances) can transparently move from local infrastructure to cloud infrastructure owned and operated by others. It’s transformative, and if they pull it off, this could usher in a new datacenter world.

vStorage is one of the core infrastructure services, along with vCompute and vNetwork. Although it’s not entirely clear at this point, vStorage presumably includes all existing VMware storage technologies as well as the new APIs for multipathing and I/O deduplication highlighted at the show. I envision a future in which vStorage is a set of APIs for everything from I/O to device management, and VMware even includes virtual storage appliances in this umbrella.

vStorage: What’s In It?

At the very least, vStorage includes the following:

• VMFS for shared storage
• Thin Provisioning support in VMware Infrastructure is apparently coming in 2009, but is also apparently incompatible with fault tolerance.
• Linked Clones is another desktop technology coming to VI 2009.
• The Site Recovery Manager API has also become part of vStorage, though this isn’t getting much press.
• VMware Consolidated Backup (VCB) will also apparently be a vStorage API.
• The existing snapshot API will also presumably included.
• The new API for Multipathing covers the old generic native multipathing (NMP) from ESX as well as EMC’s most excellent PowerPath. The latter is presumably an extra-cost option that enables lots of niftiness, as demonstrated by Chad in this video.
• Another new API claims to deduplicate I/O, reducing traffic for certain supported disk operations, as demonstrated in Chad’s second video.

What is in and what is out is still kind of up in the air. But you can see that vStorage includes most of our old favorites (VMFS, SRM, VCB), some desktop elements (thin volumes, linked clones), and some new surprises (PowerPath, I/O dedupe). There is enough meat here to separate this from traditional slideware: VDC-OS has legs!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | What is VMware VDC-OS vStorage?
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

1. Support for Volume Shadow Copy Service (VSS) for filesystems Windows Server 2003 and Windows Server 2008 guests.  This means VMware services like VCB and SRM can now signal Windows Server to quiesce filesystems before creating a clone or snapshot and is a major addition!
2. Windows Server 2003 guests also get application quiescing, where supported.
3. You can now extend a live, running VMFS volume as storage is added, just like Windows Vista and 2008 guests could already do with raw device mode (RDM).  Note that this only works for flat disks with no persistent snapshots open.

A few items are now officially supported, too:

1. Support for Storage VMotion between and among Fibre Channel and iSCSI datastores is here, and in Update 1.
2. iSCSI over 10 gigabit Ethernet (10 GbE) is now supported allegedly, though the I/O Compatibility Guide still says “Gigabit only”.  Of course, it already worked, but it’s nice to have official support just the same…  Bring on the datacenter Ethernet!  They’ll support this in Update 1 too.
3. And NFS over 10 GbE is supported, too!

1. They supposedly added support for 8 gigabit Fibre Channel (8 Gb FC) HBAs, but I don’t see it in the I/O Compatibility Guide. The Emulex LP12000/LP12002 are noted as 4 Gb only, the QLogic QLE2560/QLE2562 aren’t listed, the HP 81/82 series isn’t there either.  Update:  QLogic and Emulex HBAs are supported after all!
2. Support for more Serial-Attached SCSI (SAS), including Dell SAS 5/E, LSI 1078, Sun SG‐XPCIESAS‐R‐EXT‐Z, and HP SC08GE is listed in the Stotage Compatibility Guide.
3. CIM system health monitoring of some QLogic and Emulex HBAs is new too.

[ad#banner]

See also:

• Scott Lowe’s (as always) excellent overview
• VM /ETC’s colorful post
• Excellent coverage at Yellow Bricks

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Storage Fixes in VMware ESX Server 3.5 Update 2
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Seriously, you gotta cheer when a “big gorilla” app like NetBackup adds this kind of technology, though. CDP and deduplication were great ideas but needed n application to focus them, and data backup is an excellent place to apply them.

And although the press release doesn’t highlight it, the application-specific recovery enhancements look especially tasty to me. NetBackup leverages VCB in VMware but can do file-level restore, which is awesome. And it can also do document-level or full-database restore in SharePoint from the same image. Over in Exchange land, it claims to be able to restore from snapshots instead of the backup image, speeding (all too frequent) recoveries.

© sfoskett for Stephen Foskett, Pack Rat, 2007. | NetBackup 6.5 Spreads the Love Around
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.