VASA allows a "provider" application to tag vSphere storage with a "capabilities" string

The list of VMware storage integration features in vSphere is growing at a rapid pace. From backup to enterprise array offload, VMware is staking their claim. But information about one new storage feature in vSphere 5 has been scarce: The true nature of the Storage API for Storage Awareness (VASA) is only just beginning to be revealed.

VASA: Born of Necessity

vSphere has some amazing automated storage mobility features. The ease of moving data between LUNs and even arrays with Storage vMotion is a revelation to those of us unlucky enough to have used manual migration methods in the past. And VMware has automated this with the new Distributed Resource Scheduler (DRS) capability, allowing vSphere to make its own decisions about data placement.

But this kind of dynamic movement can hurt as well as help. What if an administrator moves a VMDK from high-performance to low-performance storage? Server responsiveness would suffer, perhaps even resulting in an application outage. And how could Storage DRS avoid doing the same thing?

VMware realized they needed some mechanism that could “tag” a storage resource with its capabilities. This is the real reason for their creation of Profile-Driven Storage and VASA.

VASA Does One Thing

In vSphere 5, VASA is incredible simple: It is a basic protocol for vSphere to request a “capabilities” string regarding a LUN or NFS share from a “provider”. The content of this response, and indeed the form that this provider will take, is left up to the individual storage vendors.

VASA Providers can take many forms. Dell is reportedly developing a vCenter plugin to report capabilities. EMC and NetApp will use a software application that runs on a virtual or physical server. In all cases, the software uses a proprietary call to the storage array, in effect relaying and translating the VMware request.

The VASA “Capabilities” string is undefined as well. Most vendors use a list of technical attributes as their response string, and each has adopted their own strategy about how to present information:

• Dell’s EqualLogic returns a comma-separated list like “RAID, SSD, REPLICATED”
• NetApp’s provider uses semicolons, as in “Dedupe; Replication”
• HP’s is very detailed, with a schema specifying Drive Type, RAID Type, Provisioning Type, VV Type, and Remotecopy
• EMC appears to have a standardized set of much-shorter tags, like “Performance”, “Multi-Tier”, “Capacity” and such

These are passed through to vCenter, where an administrator can decide how to interpret them.

VASA Today and Tomorrow

In its simplest form, VASA capabilities tags enable both manual and automatic storage placement features to respect tiered storage policies. This is important to maintain proper system performance and availability. In the future, I expect more advanced VASA providers, perhaps even integration of the providers into vCenter plugins. I also look forward to a standard capabilities tagging schema and smarter handling of returned tags. Right now, for example, EMC’s Clariion and Symmetrix lines both tag LUNs as “Performance”, but these are obviously not equivalent.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | What is VMware VASA? Not Much (Yet)
This post was categorized as Enterprise storage, Features, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

VMware's PSA is awash in abbreviations and options

I am often questioned during my Storage for Virtual Environments seminar presentations about VMware’s Pluggable Storage Architecture (PSA). This system is fairly straightforward and concept: VMware provides native multipathing support for a variety of storage arrays, and allows third parties to substitute their own plug-ins at various points in the stack. But the profusion of acronyms and third-party options makes it difficult for end-users to figure out what is going on. In an effort to help, I present here another entry in my “VMware storage features in plain English” series.

Note: I am more of a storage guy than a virtualization expert. I consider myself one of those end-users who have had trouble figuring out what’s going on with PSA specifically, in VMware storage features in general. I welcome comments and suggestions for corrections or improvements to this and all of my articles. Thanks for your help!

Introducing Pluggable Storage Architecture (PSA)

Pluggable storage architecture was one of the major enhancements introduced in vSphere 4. Functionally similar to Microsoft’s MPIO stack for Windows, PSA includes native multipathing support and allows vendors to plug in their own advanced features.

I find the VMware diagram confusing. Is mine more or less accurate and readable?

The ESX kernel (VMkernel) walks down through three layers when communicating with storage:

1. In the top layer, VMware native NMP or third-party MPP software decides which SATP to use, or whether to use the native interface. MASK_PATH also operates at this layer.
2. The SATP layer includes native generic path selection (active/active, active/passive), standard ALUA, as well as allowing third-party plugins (SATP) to override its behavior. The SATP monitors these paths, reports changes, and initiates fail-over on the array as needed.
3. At the PSP layer, software decides which physical channel to use for I/O requests.

There are three types of PSA plugins for vSphere 4:

1. Storage Array Type Plug-In (SATP)
2. Path Selection Plug-in (PSP)
3. A complete third-party multipathing software stack (MPP)

As is the case with VAAI, VMware includes a number of third-party plug-ins in the ESXi install. Users can simply activate many of these according to their needs, though some require additional fees and licensing.

Storage Array Type Plug-in (SATP) List

Storage Array Type Plug-Ins (SATPs) to the VMware Pluggable Storage Architecture multipathing solution for the specific characteristics of the storage array. This is very important, since each storage array design differs substantially in detail and support, especially when it comes to load-balancing and failover between controllers, ports, and paths. So it is critical for VMware to have developed a standard interface to communicate with arrays.

SATPs allow load balancing across multiple paths, intelligent path selection, and over troubled conditions such as “chatter”, when passed rapidly fail back and forth between controllers.

The SATP has critical tasks to perform in the PSA stack:

1. Decide which method of communication to use with the storage (PSA or native)
2. Monitor the health of the physical I/O channels or paths
3. Report any changes in the state of the paths up the stack
4. Perform actions required to fail over storage between controllers on the array

VMware vSphere includes a variety of generic plugins for storage arrays. I’ve identified the following:

• VMW_SATP_LOCAL – Local SATP for direct-attached devices
• VMW_SATP_DEFAULT_AA – Generic for active/active arrays
• VMW_SATP_DEFAULT_AP – Generic for active/passive arrays
• VMW_SATP_ALUA – Asymmetric Logical Unit Access-compliant arrays

Although I have sometimes seen other SATP plug-ins mentioned, the following plug-ins are all that are listed in the VMware ESX Hardware Compatibility List.

• VMW_SATP_LSI – LSI/NetApp arrays from Dell, HDS, IBM, Oracle, SGI
• VMW_SATP_SVC – IBM SVC-based systems (SVC, V7000, Actifio)
• VMW_SATP_CX – EMC/Dell CLARiiON  and Celerra (also VMW_SATP_ALUA_CX)
• VMW_SATP_SYMM – EMC Symmetrix DMX-3/DMX-4/VMAX, Invista
• VMW_SATP_INV – EMC Invista and VPLEX
• VMW_SATP_EQL – Dell EqualLogic systems

EMC PowerPath and HDS HDLM also support a variety of storage arrays, but I would classify these as full MPP replacements for PSA, rather than SATP plug-ins.

You can see which SATP plug-ins are available using the following esxcli command:

esxcli nmp satp list

Path selection plug-in (PSP) List

In contrast to the diversity of VAAI and SATP plug-ins, the universe of path selection plug-ins is fairly small. Most storage arrays are supported with either Most Recently Used (MRU) or Fixed path selection approaches. Many also support Round Robin (RR) path selection. The only vendor with a specific PSP that is not also part of a full MPP (like EMC PowerPath or HDS HDLM) is Dell, which offers a special routed path selection plug-in for the EqualLogic iSCSI arrays.

• VMW_PSP_MRU – Most-Recently Used (MRU) – Supports hundreds of storage arrays
• VMW_PSP_FIXED – Fixed – Supports hundreds of storage arrays
• VMW_PSP_RR – Round-Robin – Supports dozens of storage arrays
• DELL_PSP_EQL_ROUTED – Dell EqualLogic iSCSI arrays

As mentioned, EMC PowerPath also offers path selection as a plug-in in addition to the full MPP stack. Many other vendors offer unique path selection plug-ins, over 100 in total, but these are not specifically called out in the VMware HCL apart from their existence. I would love to learn more about them, however.

You can see which SATP plug-ins are available using the following esxcli command:

esxcli nmp psp list

Tell Me More About PSA!

As mentioned above, I am by no means an expert in VMware Pluggable Storage Architecture. Rather, I am interested in learning more and passing on this knowledge to others. I welcome your comments and feedback, and especially your corrections to the information presented here. I will try to keep this page updated as new versions of vSphere are introduced and as I learn more about this technology. Thank you for your help and understanding!

Further Reading:

• What’s New in VMware vSphere 4: Storage
• Next-Generation ESX Storage: A Pluggable Core Storage Architecture
• Storage Changes in the VMware vSphere 4 Family
• Pluggable Storage Architecture, exploring the next version of ESX/vCenter
• Vmware vSphere 4.1 PSA (Pluggable Storage Architecture) Understanding
• Explain the Pluggable Storage Architecture (PSA) Layout

© sfoskett for Stephen Foskett, Pack Rat, 2011. | VMware PSP and SATP in Plain English
This post was categorized as Enterprise storage, Everything, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Note: This information was based on the “VMware Storage/SAN Compatibility Guide” and is being regularly updated. Please add comments here and I will add products and change and update listings as soon as they appear in the guide!

Where, Why, and What is VAAI?

I’ve previously discussed the fact that VMware’s excellent ESX hardware compatibility list (HCL) is so comprehensive than obscures basic facts about supported products. This is especially true for VAAI, since compatibility is only noted as a footnote in individual storage array listings. It does not help matters that not all VAAI plugins support all three capabilities.

Like my previous posts regarding FCoE CNA’s, SATA and PATA chipsets, and home/lab network cards, I’ll attempt to boil down the VMware ESX HCL into plain English. This data will also be part of my Storage for Virtual Servers seminar presentation, the first of which will be given on March 10 in Philadelphia.

I’m not going to spend too much time on “what is VAAI” in this post. Instead, I suggest you read the following blog posts and VMware’s excellent guide, “What’s New in VMware vSphereTM 4.1 — Storage“?

• What is VAAI, and how does it add spice to my life as a VMware admin?
• vStorage APIs for Array Integration aka VAAI
• If You Ever Needed Convincing About VAAI…
• VMware VAAI pros and cons and the hidden fourth primitive

The Three VAAI Primitives

You ought to read the updated Complete List of VMware VAAI Primitives since it’s much more thorough and informative!

In ESX 4.1, that vStorage API for Array Integration includes three basic capabilities or primitives:

1. Blocks zeroing is a communication method for thin provisioning capable storage arrays, allowing them to quickly zero out storage capacity for later reclamation.
2. Full copy commands the storage array to make a mirror or snapshot of data without any I/O on the server hardware.
3. Hardware assisted locking enables more granular control of shared storage resources in ESX clusters

In order to support VAAI, a storage array requires two things:

1. Hardware capable of supporting one or more of the three primitives listed above
2. A software plug in for ESX enabling communication and integration

VAAI Plug In Support

Creating a VAAI plug in is not a trivial task, and not all storage arrays are yet supported. I have heard grumbling from storage vendors that EMC (the storage vendor that owns VMware) has been given early access to VAAI information, allowing them to support this feature set before their competitors. However, this has not stopped a diverse set of other unrelated storage vendors from quickly producing and releasing effective and complete VAAI plugins.

As of this writing, there are 11 array-specific plugins and one general-purpose plug in available for ESX 4.1. EMC, NetApp, 3PAR (HP), HDS, FalconStor, Fujitsu, IBM, Dell (EqualLogic), and HP (LeftHand, P9000, P2000) have produced VAAI plugins supporting all three primitives. Additionally, a cloud in supporting the T10 blocks zeroing methods is available, enabling other arrays to support this one primitive. Note that the T10 primitive should support nearly any capable array, but not all have been tested and qualified for use with it.

VAAI Support Matrix

Note that similar OEM versions (for example, Fujitsu’s FibreCAT CLARiiONs, and the Gateway/Lenovo/Acer AMS line) are also supported the same as the manufacturer’s offerings. I’ve simplified and eliminated similar models (the Dell EqualLogic PS6000E, PS6000S, PS6000X, PS6000XV, and PS6000XVS are all listed simply as PS6000).

Updates:

• IBM recently added full VAAI for the XIV, SVC, and similar Storwize V7000. I’m sure they’re also working on complete VAAI plugins for the big DS8000 systems!
• EMC certified the new VNX line for VAAI (FC only for now) and the V-Max just gained iSCSI VAAI support.
• FalconStor added VAAI for NSS, enabling any storage array to be used.
• HP created VAAI plugins for the P9500 and MSA P2000 lines as well.

Stephen’s Stance

VAAI is an exciting new capability for VMware ESX, and demonstrates the enterprise readiness of vSphere 4.1. Although not all storage arrays are yet supported, the diverse assortment listed above should cover the majority of enterprise storage environments. I fully expect that the obvious holes will be filled in soon, and I look forward to updating this list when I hear news of those product releases. I also look forward to learning of additional capabilities added as VAAI primitives in the future!

The Exhaustive List

I am attempting to keep this list up to date. My authoritative source of information is the VMware Storage Compatibility Guide. This is the only source of information I will use, since only official and supported implementations belong in production. But I welcome pointers, suggestions, and referrals for updates!

This list is complete as of February 21, 2011

Full VAAI (All 3 primitives)

• Dell
  • EqualLogic
    • iSCSI (vmw_vaaip_eql)
      • Dell EqualLogic PS4000E (Dell EqualLogic PS4000X; Dell EqualLogic PS4000XV)
      • Dell EqualLogic PS5000E (Dell EqualLogic PS5000X; Dell EqualLogic PS5000XV)
      • Dell EqualLogic PS5500E
      • Dell EqualLogic PS6000E (Dell EqualLogic PS6000S; Dell EqualLogic PS6000V; Dell EqualLogic PS6000XV; Dell EqualLogic PS6000XVS)
      • Dell EqualLogic PS6010E (Dell EqualLogic PS6010S; Dell EqualLogic PS6010X; Dell EqualLogic PS6010XV; Dell EqualLogic PS6010XVS)
      • Dell EqualLogic PS6500E (Dell EqualLogic PS6500X)
      • Dell EqualLogic PS6510E (Dell EqualLogic PS6510X)
      • EqualLogic PS100E
      • EqualLogic PS200E
      • EqualLogic PS300E
      • EqualLogic PS3600X
      • EqualLogic PS3700X
      • EqualLogic PS3800XV
        • EqualLogic PS3900XV
      • EqualLogic PS400E
      • EqualLogic PS50E
      • EqualLogic PS70E
• EMC
  • FC (vmw_vaaip_cx)
    • EMC CLARiiON CX4-120 (EMC CLARiiON CX4-120 C8/C8X; Dell/EMC CLARiiON CX4-120; Fujitsu FibreCAT CX4-120)
    • EMC CLARiiON CX4-240 C8/C8X
    • EMC CLARiiON CX4-480 (EMC CLARiiON CX4-480 C8/C8X; Dell/EMC CLARiiON CX4-480)
    • EMC CLARiiON CX4-960 (EMC CLARiiON CX4-960 C8/C8X; Dell/EMC CLARiiON CX4-960)
    • EMC Celerra NS-120
    • EMC Celerra NS-480
    • EMC Celerra NS-960
    • EMC VNX5100
    • EMC VNX5300
    • EMC VNX5500
    • EMC VNX5700
    • EMC VNX7500
  • FC/iSCSI/FCoE (VMW_VAAI_SYMM)
    • EMC Symmetrix VMAX
  • iSCSI (vmw_vaaip_cx)
    • EMC Celerra CNS
    • EMC Celerra NS-120
    • EMC Celerra NS-480
    • EMC Celerra NS-960
    • EMC CLARiiON CX4-120 C8
    • EMC CLARiiON CX4-240 C8
    • EMC CLARiiON CX4-480 (EMC CLARiiON CX4-480 C8)
    • EMC CLARiiON CX4-960 (EMC CLARiiON CX4-960 C8)
• FalconStor
• FC (unknown)
• CDP Gateway
• CDP SA
• CDP VS
• CDPx Gateway
• IPStor Enterprise
• NSS Gateway
• NSS SA
• NSS VS
• Fujitsu
  • FC (fjt_vaaip_module)
    • Fujitsu Eternus DX410
    • Fujitsu Eternus DX440
    • Fujitsu Eternus DX8100
    • Fujitsu Eternus DX8400
    • Fujitsu Eternus DX8700
  • iSCSI (fjt_vaaip_module)
    • Fujitsu Eternus DX410
    • Fujitsu Eternus DX440
    • Fujitsu Eternus DX8100
    • Fujitsu Eternus DX8400
    • Fujitsu Eternus DX8700
• Hitachi
  • FC (vmw_vaaip_hds)
    • Hitachi AMS 2100 (Acer AMS2100; Gateway AMS2100; HDS AMS 2100; Lenovo-HDS AMS2100)
    • Hitachi AMS 2300 (Acer AMS2300; Gateway AMS2300; HDS AMS 2300; Lenovo-HDS AMS2300)
    • Hitachi AMS 2500 (HDS AMS 2500)
    • Acer AS2040; Gateway GS2040
    • Hitachi AMS 2010
    • Hitachi BR1600/BR1600E/BR1600S
    • Hitachi Virtual Storage Platform (Hitachi VP9500; HP StorageWorks P9500)
    • Nihon-Unisys Sanarena 1910
    • Nihon-Unisys Sanarena 1930
    • Nihon-Unisys Sanarena 1970
    • Nihon-Unisys Sanarena 1990
  • iSCSI (vmw_vaaip_hds)
    • Hitachi AMS 2010
    • Hitachi AMS 2100 (HDS AMS 2100; Lenovo-HDS AMS2100)
    • Hitachi AMS 2300 (HDS AMS 2300; Lenovo-HDS AMS2300)
    • Hitachi AMS 2500 (HDS AMS 2500)
    • Hitachi BR1600E
    • Nihon-Unisys Sanarena 1930
    • Nihon-Unisys Sanarena 1970
    • Nihon-Unisys Sanarena 1990
    • Hitachi Virtual Storage Platform
• HP
  • 3PAR
    • FC (3PAR_vaaip_InServ)
      • InServ E200
      • InServ F-Class
      • InServ S400
      • InServ S800
      • T-Class
    • iSCSI (3PAR_vaaip_InServ)
      • InServ E200
      • InServ F-Class
      • InServ S400
      • InServ S800
      • T-Class
  • P9500
  • FC (hp-vaaip-p9000)
  • HP P9500
  • P2000
  • FC/iSCSI (hp-vaaip-p2000)
  • HP MSA P2000
  • LeftHand
    • iSCSI (vmw_vaaip_lhn)
      • HP LeftHand P4500
      • HP LeftHand P4000 VSA
      • HP LeftHand P4000sb
      • HP LeftHand P4300 (HP LeftHand P4300 G2)
      • HP LeftHand P4500 (HP LeftHand P4500 G2)
      • HP LeftHand P4800
      • HP ProLiant DL380
      • Dell 2950
      • IBM x3650
      • LeftHand NSM 160
      • LeftHand NSM 185
      • LeftHand NSM 2060 (LeftHand NSM 2060 G2)
      • LeftHand NSM 2120 (LeftHand NSM 2120 G2)
      • LeftHand NSM 260
      • LeftHand NSM 320
      • LeftHand NSM 326
      • LeftHand NSM 3650
      • LeftHand NSM 380
      • LeftHand NSM 4150
      • LeftHand VSA
• IBM
  • FC/iSCSI (IBM_VAAIP_MODULE)
    • IBM XIV
    • IBM SVC
    • IBM V7000
    • Fujitsu VS850
    • Actifio
• NetApp
  • FC (VMW_VAAIP_NETAPP)
    • NetApp N3000 Series
    • NetApp N5000 Series
    • NetApp N6000 Series
    • NetApp N7000 Series
    • NetApp FAS2000 Series
    • NetApp FAS3000 Series
    • NetApp FAS3100 Series
    • NetApp FAS3200 Series
    • NetApp FAS6000 Series
    • NetApp FAS6200 Series
  • FCoE (VMW_VAAIP_NETAPP)
    • NetApp FAS3000 Series
    • NetApp FAS3100 Series
    • NetApp FAS3200 Series
    • NetApp FAS6000 Series
    • NetApp FAS6200 Series
  • iSCSI (VMW_VAAIP_NETAPP)
    • NetApp N3000 Series
    • NetApp N5000 Series
    • NetApp N6000 Series
    • NetApp N7000 Series
    • NetApp FAS2000 Series (Fujitsu Eternus NR1000F Series Model F2040)
    • NetApp FAS3000 Series
    • NetApp FAS3100 Series (Fujitsu Eternus NR1000F Series Model F3160)
    • NetApp FAS3200 Series
    • NetApp FAS6000 Series
    • NetApp FAS6200 Series

T10 Block Zero Only

• Bull
  • FC (vmw_vaaip_t10)
    • Bull Optima2000
  • iSCSI (vmw_vaaip_t10)
    • Bull Optima2000c
    • Bull Optima2000i
• NEC
  • FC (vmw_vaaip_t10)
    • NEC iStorage D3-30
    • NEC iStorage D4-30
  • iSCSI (vmw_vaaip_t10)
    • NEC iStorage D3-30/D3-30i
    • NEC iStorage D4-30/D4-30i
• Fujitsu
  • iSCSI (vmw_vaaip_t10)
    • Fujitsu Eternus VS850
• IBM
  • iSCSI (vmw_vaaip_t10)
    • IBM Storwize V7000
    • IBM SVC

© sfoskett for Stephen Foskett, Pack Rat, 2011. | VMware VAAI Storage Array Support in Plain English
This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Everyone has a target on their backs, but they all aim in different directions

One of the amusing aspects of being self-employed is watching all the giants battle it out. Every company is gunning for someone, but the amazing thing is that they rarely point their sights directly at each other: NetApp is gunning for EMC who’s more focused on HP who wants to knock off Oracle who’s fixated on IBM. It sounds very “high school romance” but this is deadly-serious business.

The Enterprise IT Ladder: Dell, HP, Oracle, and IBM

Dell -> HP

No question about it: Dell wants to beat HP. It was bad enough back when PCs were king, with Dell constantly undercutting HP on price and supply chain efficiency. Dell went to retail, elbowing HP aside on the shelves at Staples, and even introduced a line of printers. But the enterprise data center has taken things to a whole new level. Dell’s plays in blade servers and storage are drawn from the same playbook, only there’s more in-house IP and R&D involved this time. Dell arguably got the iSCSI prize in EqualLogic but couldn’t seal the deal for 3PAR, settling on Compellent instead. They’re working hard in the professional services market; Could networking be next?

HP -> Oracle

For the longest time, HP wanted to be IBM. Then Oracle stomped on their servers with Sun and started flaunting their software-heavy profit margins. Appointing Leo Apothiker as CEO sends an unambiguous signal: HP’s gunning for Oracle. The printers, PC’s, servers, storage and networking gear isn’t going anywhere (HP’s got fights picked in each area), but software and services are the only way to make shares of HPQ perk up. Watch for some startling acquisitions and more bizarre rear-guard antics from Ellison and company.

Oracle -> IBM

Although Oracle is clearly enjoying poking sharp sticks in HP’s direction, they’re focused forward. Larry Ellison has seen the future, and it looks an awful lot like IBM: Massive services revenue, a “we do it all” executive sales pitch, and “hate to love us” handcuffs on the C-level executives at the largest global corporations. It’s worked so far: ORCL shares have risen steadily since the end of the last recession.

IBM -> IBM

IBM has seemed aimless for a decade. Armonk has won every enterprise IT war it’s fought, leaving it nowhere to turn. Spinning out the printer and PC businesses sent a signal that IBM was a different kind of company, and Wall Street is singing the chorus. In many ways, the middle-aged IBM of the 1980′s is the enemy of the new, with the company dreaming of a return to the “Mad Men” Big Blue of yore. Although this is much exactly what HP and Oracle are aiming for as well, IBM is the aloof valedictorian who just needs to keep his grades up until graduation.

Tag Team: EMC and Cisco -> HP

Cisco is like a mini IBM: All of their historical rivals are dead and buried, forcing them to look outside their traditional market for growth. Cisco seems loathe to climb the HP/Oracle/IBM ladder, seeking instead to take enough of their market to maintain solid revenue growth and profitability. Their blade servers smacked HP, which responded with a challenge in the networking space. Realizing they needed help, Cisco looked around for an “enemy of my enemy” to mount a serious IT infrastructure challenge.

What happens when you combine the market leaders in enterprise IT verticals like SAN storage, encryption and authentication, server virtualization, backup, and records management? You get EMC Corporation, the biggest company most “regular people” have never heard of. Something about “information.” Wall Street seems to have a hard time making sense of this company, too.

EMC seems overly concerned about smaller competitors (NetApp, Symantec, cloud providers), but the combination of EMC and Cisco is formidable indeed. The two and their joint venture, known as VCE, have the sales muscle to go head-to-head with Dell, HP, and IBM in the enterprise data center, and their control of key components make them a hard team to ignore.

The Wolves: NetApp and Juniper

NetApp -> EMC

Back in the 1980′s, Honda decimated the American carmakers with just a few models; They sold millions of Accords and Civics while GM watch whole brands disappear. That’s NetApp in the enterprise storage space. They pull in top-tier revenue quarter after quarter with essentially a single product line, taking on dozens of storage devices from HP, HDS, IBM, and EMC. Especially EMC. Talk to NetApp insiders and you’ll hear those three letters frequently, with precious little attention paid to anyone else. Data Domain might have made them a broader play, but NTAP shares have risen steadily since losing that takeover battle with EMC. Now NetApp is stripped down and running hard to close the revenue gap, too.

Juniper -> Cisco

Cisco is vulnerable in their core networking markets, and Juniper is ready to take them on. Since their $4 Billion acquisition of NetScreen in 2004, Juniper has been cherry-picking up-and-coming technologies in every market Cisco dominates, from the WAN to wireless. Though JNPR shares have taken their lumps, this is every bit a “Dell” to Cisco’s “HP”. The various networking sub-markets include many agile competitors, but Juniper is like NetApp: Insiders have a single-minded focus on the market leader.

Stephen’s Stance

The enterprise IT game is getting serious. Emerging from their historical strongholds, Dell, HP, and Oracle are each undermining the other, and everyone is chipping away at IBM. They each realize that they must focus upward for real growth rather than fighting a rear-guard against smaller and newer competitors. The challenge for HP and Oracle will be to expand rapidly enough to keep Wall Street from noticing the erosion from below.

Meanwhile, Cisco and EMC have joined forces out of necessity to grown both revenue and share price, while their own bases are chipped away by Juniper and NetApp. The upstarts can flourish within the verticals of networking and storage, but the VCE team is seems tenuous and uncertain. The joint venture can challenge the full-line players on a customer-by-customer basis, but history shows that only an integrated vendor can rock the enterprise IT world.

Although the information economy is growing, it will not be enough for everyone to survive. These competitors are too cut-throat to allow a smaller challenger to live, and each faces a real threat himself. The largest have a strong base to draw on, while the wolves will always find something to eat. It is those in the middle that face the most serious threat.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Every Company Is Gunning For Someone Else
This post was categorized as Computer History, Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The players are lining up for the biggest acquisition game enterprise IT has witnessed in a while

Today is the (a?) day of reckoning in the 3Par saga, with Dell widely expected to make a counter-offer higher than HP’s bid. But this mega deal, like the Data Domain war before it, sends a strong signal to the enterprise IT world: It’s open season on data storage companies! But the rising superpowers are also likely looking at networking as an area of expansion. The game is afoot!

The Competitors

The simple fact is, enterprise IT prefers to buy from large, established names like HP and IBM rather than smaller and less-familiar companies with names like Ocarina, EqualLogic, or even 3Par. The acceleration of sales by enterprise providers is what makes these big acquisitions so successful and why others involving less-powerful players often fail to deliver the same results.

Full-line “superpowers” like IBM, HP and now Dell and the new Oracle can influence purchasing decisions across a broad spectrum of hardware and software. Many large organizations are tightly coupled to one of these vendors, and will give their products stronger consideration even when they are new or unfamiliar. HP is already flexing their muscle selling their broadened network lineup, and Dell found that it could do this in data storage.

Software vendors like Microsoft, Citrix, and Oracle adamantly maintained a neutral stance toward hardware, but Ellison and company seem serious about changing this. Oracle’s success in selling Sun hardware will likely dictate further software acquisitions for Dell and HP, though IBM already has strength there. Then there is Intel, the wild card who just got wilder with their acquisition of McAfee.

Companies with narrower focus like Cisco, EMC, NetApp, Juniper, and Brocade have the same power within their sphere of influence but cannot pull in wholly-distinct products. Cisco is in the midst of this fight with their technically-excellent UCS blade servers: Although they were certainly a strategic CIO-level vendor in the largest organizations, “blades from a networking company” isn’t the as easy a decision as “networking gear from a full-line company.”

The Game

Then one must consider the market in contention. The enterprise IT space is not expanding, especially in the United States. This is very close to a zero-sum game, with Oracle’s or Dell’s wins being HP’s or IBM’s losses and vice-versa. There is massive money in play, and flexibility enough for it to swing between the competitors, but growth is not unlimited.

Enterprise storage and data center and campus networking are two areas where smaller companies retain enviably-large slices of the pie, explaining the interest in these spaces among the superpowers. These verticals still have room for sales to grow without displacing a fierce full-line foe, and the superpowers have lately been weak here. Storage and networking are enticing opportunities, but each slice is similarly dominated by “vertical superpowers” and partners EMC and Cisco.

So this is the game: Four full-line enterprise superpowers battling each other for datacenter dominance and coveting the extra profits of a few verticals. HP clearly believes they can chip away at EMC and Cisco in storage and networking; Dell and IBM have so far focused mainly on storage; and Oracle hasn’t made a move in either direction, instead challenging the other three in the core server and software space.

Pawns or Knights?

So where does this leave the smaller players? Are they merely pawns in the game, waiting to be sacrificed, or are they knights who can wield power across the field? The largest (Cisco and EMC especially) appear to have ambitions of their own as well as the financial and technical strength to shake up the game. They are unlikely to be acquired by the superpowers. Brocade, too, has broad strength in storage and networking but maintains relationships across the board that make an acquisition difficult.

Strong vertical players like Juniper, NetApp, Riverbed, and Compellent are ripe for acquisition, as were Foundry, 3Com, Data Domain, and 3Par. IBM, Dell, and Oracle are all likely buyers of the networking players, though HP may consider filling in where 3Com was found lacking. All four will likely take a strong look at the remaining storage players as well, with the loser in the battle for 3Par likely to be hungry indeed.

One should also consider the potential impact of smaller acquisitions. Although they would not immediately “move the needle” for a massive superpower, there are many excellent technology companies that could be bought low and pushed strongly. The enterprise-class technology at Sepaton, Pillar Data, Xiotech, BlueArc, Extreme Networks, Force10, Blue Coat, Isilon, CommVault, FalconStor, and many others should not be overlooked. If a superpower can drive a larger acquisition to succeed, imagine what they can do with strong but inexpensive technology from one of these!

Stephen’s Stance

This game is nowhere near finished. The 3Par acquisition will not only generate revenue, it will shake up the ranking of data storage array dominance. This is very likely to kick off additional acquisition in the data storage space, spurring either Dell or HP to pick up additional technology and perhaps causing IBM or Oracle to engage as well. With no easy alternatives to 3Par, I expect Compellent, Xiotech, and Pillar to get closer looks, but Sepaton and BlueArc are just as ripe. NetApp may be too expensive at this point, but would be a nice match for Oracle’s strategy.

HP’s acquisition of 3Com could also signal a race to integrate datacenter and campus networking technology into the stack. Many are suggesting a Brocade acquisition, and it would be much cheaper than Juniper, but OEM ties make it a difficult purchase for any of the superpowers. Extreme and Force10 would be excellent and less-expensive alternatives.

We should also keep our eyes outside the superpower space. Intel showed that they can make big moves, and Microsoft might consider a diversification into hardware as well. One should also look to the East, where Huawei could try to enter the Western market with a merger or joint-venture to cast off the China stigma. Although I would love to see a rebel alliance rise (imagine Juniper, NetApp, and Symantec joining forces!) this is not a likely scenario.

Image credit: Chess Board by mosdave

© sfoskett for Stephen Foskett, Pack Rat, 2010. | The Enterprise IT Acquisition Game
This post was categorized as Computer History, Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Who will power the enterprise? The smart money is betting on a few superpowers taking over.

After years spent focusing on personal technology, businesses are increasingly turning back to the enterprise. There are many reasons for this, but the biggest one is the poor economy. Individuals simply have less free cash to spend on gadgets and software, and the meagre profits are increasingly going into the pockets of a single company: Apple.

The corporate IT market is much more dynamic and competitive, with a few very large “superpower” companies discovering their power to drive purchasing decisions. If a supplier can create an integrated “stack” of hardware and software, they can push product purchases that might otherwise be overlooked or postponed. This is the main reason that enterprise IT acquisitions work so well: Where a small company must fight to sell their product, a large one can hitch it to a much more strategic sale and have it pulled along.

The old IBM model is the prototype, with that company once selling everything from office equipment to datacenter gear as well as the consulting and integration services to make it all work.

HP has spent almost two decades bulking up to become the new IBM, buying their way into open systems laptops, desktops, and servers (Compaq), networking (3Com), services (EDS), and storage (Compaq, LeftHand, Ibrix, and perhaps 3Par). HP has been remarkably proficient at executing on this enterprise plan: In talking to enterprise IT folks, I often hear IBM-esque sentiments regarding the new HP. They tell me they’re willing to give HP the benefit of the doubt when it comes to new technologies and products, buying on basis of the company’s reputation and ability to make everything work. This bodes well for the company’s post-Hurd future, and HP has the most-complete “enterprise stack” in the business.

But HP has a target on its back, pinned there by Dell. The folks from Round Rock believe they can be more efficient (and thus profitable) than HP in the same markets, and have been making moves to fortify their enterprise offerings. Dell was always more of a manufacturing than R&D business, but they have shown a desire to broaden their focus. Intrigued by the high-margin mid-enterprise storage business built from their EqualLogic acquisition and their success selling EMC storage, Dell is moving into the enterprise. They matched HP/EDS by purchasing Perot and have made smaller buys in storage (Ocarina, Exanet) as well as the big move for 3Par.

The next big emerging stack player is Oracle. The acquisition of Sun gave Oracle a strong hardware base to complement their command of enterprise software, and many expect further acquisitions. But Oracle is playing a different game than HP and Dell, focusing on the high-margin enterprise space and ignoring more competitive outlying areas. Many suspect the company might make a play in the network space (Brocade, Juniper, and F5 have been mentioned) but storage is possible as well. CEO Larry Ellison is a major investor in Pillar Data Systems, so many expect a spin-in here. But Oracle has the appetite for something much bigger, even EMC or NetApp.

Then there is Cisco, who have attempted to parlay their data center networking strength into a broader position. But Cisco’s halting moves into storage (Fibre Channel switching and SAN extension) did not displace the market leaders, and their server products (UCS) have not made much of a dent on HP, IBM, and Dell either. A solid partnership with EMC has delayed further forays into the enterprise storage market, and Cisco seems puzzlingly interested in low-margin access businesses (Linksys, Flip) and their Cius tablet.

There are other players in the enterprise space as well. EMC has diversified under CEO Joe Tucci, taking a dominant position in server virtualization (VMware) and making a strong enterprise security acquisition (RSA). But the many faces of enterprise storage remains EMC’s strength, and they seem content to partner with Cisco for a stack sale. Hitachi, NEC, and Fujitsu also offer varying enterprise hardware and software stacks, but their comparatively small sales presence in the US market limits their ability to execute. In the final analysis, only IBM, HP, Dell, and perhaps Oracle can claim to be enterprise IT superpowers at this point.

Image credit: Steam Engine by Stuck in Customs

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Meet the Enterprise IT Superpowers
This post was categorized as Computer History, Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I’ve long wished for a new enterprise storage superpower. Competition is good for everyone, and the enterprise storage space has always been highly competitive. Traditional SAN storage powers (EMC, HDS, HP, and IBM) have been under continual attack from tech-heavy upstarts like EqualLogic, LeftHand, Compellent, Xiotech, and 3Par. The smaller (revenue-wise) NAS market has been more serial, with NetApp knocking off Auspex, then challenged by EMC. Yet innovators have been thick there as well, from Exanet to Ibrix, Isilon to Onstor.

Through it all, one thing has been clear: The major companies, though perhaps lagging in technology, were usually able to withstand the attack of the upstarts through sheer strength of salesforce. Storage is a strategic investment, and selection of a storage platform is much more far-reaching than many IT product decisions. The inertia of an installed storage environment makes it a real challenge to switch vendors, giving the established players massive leverage.

It became clear to me and many others that the best way for upstart companies (and, by extension, technologies) was to be part of an established vendor’s sales process. OEM relationships were a big part of this (witness the success of BlueArc and even NetApp and HDS) but acquisition was a much stronger proposition. If customers were warmer to OEM products than independent sales, they are much hotter when it comes to acquired technology. HP, Dell, IBM, and EMC have all demonstrated the power that comes when an established company buys a startup and puts the power of their sales force behind these new products.

This explains Dell’s fantastic success with EqualLogic. They took a product that was emerging as dominant in its niche (midrange iSCSI SAN) and blasted it into the market, while at the same time optimizing manufacturing and deployment. EMC did the same with Clariion and DataDomain, and HP is showing strong signs of health with LeftHand and Ibrix. Then there is IBM, who took XIV out of Israel and made it a source of irritation to the rest of the industry.

Many industry watchers have long wondered what would happen if the smaller guys got together, forming a new superpower of their own. Would 3Par, BlueArc, and Sepaton be a real challenger? What about Xiotech or Compellent and Isilon or FalconStor? Is mixing and matching some smaller companies a recipe for success? The answer was often a counter-question: What if someone like Dell, who knows how to manufacture and sell, picked them up instead? This seemed much more like a sure-thing, since the established management and financials stave off potential integration issues.

It appears that this is the future. Established players will pick up smaller companies, fortifying their offerings and accelerating sales in a way the little guys weren’t capable of. Dell’s billion-dollar acquisition of 3Par reportedly headed off a similar offer from HP, and will likely spark another acquisition. I imagine the management teams at Compellent and Xiotech just got a lot busier…

Clearly, Dell and HP are playing this game. IBM and EMC are in it, too. But what about Cisco and Oracle? Could they be planning storage acquisitions of their own, to the detriment of partners like EMC and Hitachi? What about the strong contingent from Japan, NEC and Hitachi? And who gets picked up next? We shall see!

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Dell + EqualLogic, Exanet, Ocarina, 3Par = What?
This post was categorized as Computer History, Enterprise storage, Gestalt IT. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Snow Leopard is coming - will iSCSI finally tag along?

ZFS wasn’t the only AWOL storage technology in Apple’s OS X 10.5 – early builds of Leopard included a built-in iSCSI initiator. When the operating system was finally released in October of 2007, both ZFS and iSCSI were quietly dropped, making room for 300 other features Apple felt were more prime-time-ready.

With the next major OS X release, 10.6 “Snow Leopard”, just around the corner, many have wondered if iSCSI will join the reanimated ZFS support. Sadly, there has been no word from Apple on the fate of their iSCSI initiator. But silence does not necessarily tell a tale at the always-circumspect company, so hope springs eternal for the iSCSI faithful. If the rousing success of Microsoft’s iSCSI initiator is any indication, Apple is seriously missing out!

In the mean time, those seeking iSCSI connectivity within Mac OS X do have third-party options to consider:

• Studio Network Solutions offers a free (as in beer) iSCSI initiator for OS X Tiger and Leopard. The globalSAN iSCSI Initiator is the client-side portion of their SANmp storage sharing solution, and purports to support most of the required protocols. In practice, however, the company cannot support other targets than their own. globalSAN failed to mount any LUNs from my Windows Storage Server iSCSI test rig, and reports claim mixed results from EqualLogic and EMC SANs besides.
• ATTO Technology sells a commercial iSCSI initiator for OS X, Xtend SAN, and reports here are very favorable. Although the $195 price puts it out of reach for home users, corporations wishing to connect Leopard systems to an iSCSI SAN should look no further.
• Another commercial option is SmallTree Software’s abcSAN. Listed at $150, abcSAN lists EqualLogic among its functional targets, but the standalone initiator is nowhere to be found on SmallTree’s web site. They also have an ATA-over-Ethernet (AoE) solution for the true nerds in the audience!
• Ardis may also sell an iSCSI initiator for OS X.

All in all, iSCSI on OS X is in a sorry state of affairs. If only Apple had released the initiator seen in early Leopard builds! Let’s all hope Snow Leopard brings iSCSI to the party at last!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Will Snow Leopard Finally Bring iSCSI To The Mac?
This post was categorized as Apple. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

This is how we used to avoid hotspots in 1998: Carefully planning every detail of the storage layout.

Each disk drive consists of a spindle of spinning platters with read/write heads move back and forth. Each time you access a piece of data that’s not in cache, the drive moves its arm over the platter to access the correct piece of data. Since each drive can only access one piece of data at once, and since caches can only hold so much data, tuning a system to minimize the number of requests per drive is essential.

Manual storage array layout was an art, but we never fooled ourselves into thinking our designs were optimal. There were just too many intractable problems, so we had to compromise at every turn:

• We usually had no performance data to base our layout decisions on, so we had to rely on guesses and rules of thumb
• Workloads tend to change over time and manual layouts are painful to modify
• The smallest unit of allocation was an entire LUN or drive, so even the best disk layout mixed hot and rarely-accessed data everywhere
• Much of the allocated space was unused, so we used expensive disks to store nothing

One might think that, 10 years later, advances in technology would have solved these basic issues. But for many people using many of the so-called modern mainstream enterprise storage systems, these problems remain.

Like all good systems administrators, I’m a natural control freak. I am uncomfortable letting the system manage itself, having been burned too many times by computers (well, software really) making stupid decisions. It’s analogous to the backlash against anti-lock brakes, traction control, and automated transmissions among racing enthusiasts.

Do we allow technology to help us get better performance, or do we try to micro-manage everything? Photo by ClearInnerVision

But the time has come to let go. We don’t have to micro-manage storage anymore, and we have much to gain by letting the array do the work:

• Just as traction control can manage each wheel independently, something a driver could never do, modern virtualized storage systems can allocate small “chunks” to the optimal drive type, creating a better layout than anyone could manage with LUNs
• Dynamic optimization technology can move these chunks around, adapting as loads change
• Thin provisioning can go a step further, not wasting drive capacity for unused space
• Wide striping and post-RAID storage systems have a higher threshold before performance suffers due to spindle hotspots
• Widespread availability of tiered storage, including advanced caches, solid state drives, high-performance SAS and FC, and cheap bulk disks, gives us many more options

As I mentioned, not all systems have these capabilities, and not all implementations are created equal. I’m concerned about misuse of thin provisioning, for example, but it’s hard to argue with its effectiveness in many circumstances. Find out how granular your system’s allocation is – some remain LUN-only, while others are much more effective, using tiny chunks.

These new storage automation technologies really become essential once high-dollar flash storage is added to the mix. If you’re paying 30 times more for a flash drive, you want to make sure you’re making the best use of it that you can! Look at IBM’s recently-announced SAN Volume Controller (SVC) and solid state drive (SSD) combination, for example: It will almost certainly have fine-grained thin provisioning of SSDs, and should be able to dynamically move data between flash and disk storage and even between different storage arrays, but I still have questions on how granular this capability will be. HDS can do similar things with their USP-V. NetApp’s V-Series NAS systems will do dynamic allocation, thin provisioning, and data deduplication to enable a better return on the flash drive investment. I’d love to see 3PAR, Compellent, Dell/EqualLogic, and HP/LeftHand apply their solid dynamic allocation tech to solid state storage as well!

Then there’s the 800 lb gorilla: EMC. More enterprise SSD has probably been shipped out of Hopkinton than every other vendor combined, and both the CX and DMX support (optional/expensive) “virtual provisioning” (aka, thin provisioning) of flash storage. But EMC’s Optimizer is not widely used, and only migrates entire LUNs based on user input – hardly the kind of dynamic and granular technology needed to optimally use all of that flash storage. I’m sure the company is working on addressing this issue, though. Perhaps it will appear in the DMX-5 announcement we are all expecting this year?

This article can also be found on Gestalt IT: How I Learned to Stop Worrying and Love Storage Automation

© sfoskett for Stephen Foskett, Pack Rat, 2009. | How I Learned to Stop Worrying and Love Storage Automation
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Update: Check out the latest multi-vendor iSCSI post!

I usually don’t write about other peoples’ articles on this blog, preferring to stick to my own independent work. But this time I’m making an exception.

If you use or are interested in VMware ESX 3.x and iSCSI, you simply must go read Chad Sakac’s post on the topic. Co-written with just about everyone in the industry (including EMC, VMware, NetApp, Dell/EqualLogic, and HP/Lefthand), Chad has put together a “cheat sheet” on the ins and outs of iSCSI connectivity and performance in ESX 3.x.

Top takeaways (and I’ve been preaching about these for a while myself, too!)

• Ethernet link aggregation doesn’t buy you anything in iSCSI environments
• iSCSI HBA’s don’t buy you much other than boot-from-SAN in ESX, either
• The most common configuration (ESX software iSCSI) is limited to about 160 MB/s per iSCSI target over one-gigabit Ethernet, but that’s probably fine for most applications
• Adding multiple iSCSI targets adds performance across the board, but configurations vary by array
• Maximum per-target performance comes from guest-side software iSCSI, which can make use of multiple Ethernet links to push each array as fast as it can go

Thanks for putting together such a great article, guys!

This post can also be found on Gestalt IT: Essential Reading for VMware ESX iSCSI Users!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Essential Reading for VMware ESX iSCSI Users!
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.