It's not going to be this easy to bridge Fibre Channel and Ethernet!

Before the holidays, I posed a question on Google+ that generated quite a bit of interest and feedback. Now that it has settled down a bit I’d like to summarize the unresolved elements to make FCoE truly a world-class storage interconnect.

Setting the Stage

FCoE has been a controversial topic in both storage and networking, and for good reason. No one would deny that Ethernet is not an ideal transport mechanism for block storage I/O. “Porting” Fibre Channel to run on Ethernet networks has been a supreme technical challenge, and many companies and individuals have labored long and hard to make FCoE a reality.

Now that FCoE is specified in the standard and has been deployed in production environments, the question turns to its future. Will it take off and seize the mantle of dominance currently held by what I like retroactively to call “Fibre Channel over Fibre Channel?” Will they coexist for the next decade, with FCoE mainly deployed in “block” environments such as Cisco UCS? Or will FCoE ultimately fail to catch on, displaced by some other storage protocol like plain FC, iSCSI, NFS, or something entirely different?

The data center needs a flexible new protocol to meet the needs of virtual environments, and convergence of storage and data networking makes a great deal of sense in these environments. This was the root of my question, and I ask it in all earnestness.

My question: What elements remain unresolved to make FCoE truly world-class? What should the vendors be prioritizing? Here are the answers I received.

Technical Considerations

Link Aggregation on CNA’s

Converged network adapters (CNA’s) allow multiple protocols to access a single Ethernet connection, but some also include multiple ports that can be aggregated. In traditional Ethernet networks, link aggregation is a respectable approach for performance and availability. But storage networks have traditionally relied on host-based MPIO software, and these features are mutually exclusive. The zeitgeist seems to be a recommendation to avoid link aggregation on CNA’s that are used for storage networks.

How Do You Handle Virtual Machine Mobility?

As I described recently, virtual machine mobility is a major technical challenge for existing networks. The VMware proposal, the VXLAN, seems to be gaining traction right now. But this is only a solution for data networking. How will FCoE SANs handle virtual machine mobility? This remains unresolved as far as I can tell, though Ethernet switch vendors have come up with their own answers. Brocade demonstrated just such a solution at Networking Field Day 2, and I know that others have answers as well. But will there be an interoperable industry solution?

How Should FCoE Be Implemented Over Longer Distances?

Fibre Channel has traditionally relied on routers and other protocols (FCIP and iFCP) to span distances, but FCoE raises the possibility of native traversal. While it is certainly possible to span distances with FCoE, this is definitely not a recommended or supported idea. Without TCP/IP, or any routing mechanism, it’s just a bad idea. But I imagine that it won’t be long before vendors decide to give it a go anyway.

Implementation Considerations

Is TRILL Required for FCoE Networks?

This has been one of my own questions since the very beginning. Clearly, edge only FCoE works just fine without TRILL. But as networks become more complicated, and virtual machines move, it seems an awfully good idea to have some protocol to alleviate East-West routing concerns. I feel much better with TRILL (or some similar Ethernet fabric technology) in a complicated FCoE network.

Should All Switches Be Full FC Forwarders?

There are number of ways to implement FCoE on Ethernet network, and not all involve building a full Fibre Channel stack in each switch. While many (including myself) assumed that FCoE implied Fibre Channel forwarding in all switches, this is clearly not the direction taken by vendors, at least initially. Perhaps the current “Ethernet forwarding” approach is only a stepping stone, or perhaps it will emerge as the dominant FCoE standard.

How Will OpenFCoE and LoM Be Used?

OpenFCoE is a software solution allowing FCoE to be run without a CNA. If this became popular, it wouldn’t be long before data center architects began looking at LAN on Motherboard (LoM) and even 10GBASE-T as a potential SAN alternative. Will this be used in the long run? It could happen, but it’s certainly not something that’s here at the moment. But OpenFCoE is a real player, especially with Intel’s backing.

How Will Technologies like Zoning Interoperate?

Many networkers are just now beginning to see the true complexity of Fibre Channel SANs. Although interoperability of higher-level Fibre Channel functions between vendors has never been a priority in “FC over FC” SANs, Ethernet could change things. I would not be at all surprised to see a groundswell of customer support demanding greater levels of interoperability from FCoE than from FC, and zoning and VSAN is the likely first beachhead.

The Big Question: When Will We See the “Killer App” For FCoE

Just about everyone agreed that the real challenge for FCoE is market acceptance. Customers aren’t yet demanding FCoE, and vendors are finding it hard to articulate a compelling case to move from “tried-and-true” FC. Convergence, cost savings, and performance have all been put forth, but customers aren’t biting. Perhaps they just need a little time and a little more proof.

This post relies extensively on feedback from a number of people, including Ivan Pepelnjak, Tony Bourke, J Metz, Dmitri Kalintsev, Derick Winkworth, David Hardaker, Juan Lage, and Corey Hines.

Read Scott Lowe’s response: What Does FCoE Have To Do With VM Mobility?

© sfoskett for Stephen Foskett, Pack Rat, 2012. | Eight Unresolved Questions About FCoE
This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

VMware's PSA is awash in abbreviations and options

I am often questioned during my Storage for Virtual Environments seminar presentations about VMware’s Pluggable Storage Architecture (PSA). This system is fairly straightforward and concept: VMware provides native multipathing support for a variety of storage arrays, and allows third parties to substitute their own plug-ins at various points in the stack. But the profusion of acronyms and third-party options makes it difficult for end-users to figure out what is going on. In an effort to help, I present here another entry in my “VMware storage features in plain English” series.

Note: I am more of a storage guy than a virtualization expert. I consider myself one of those end-users who have had trouble figuring out what’s going on with PSA specifically, in VMware storage features in general. I welcome comments and suggestions for corrections or improvements to this and all of my articles. Thanks for your help!

Introducing Pluggable Storage Architecture (PSA)

Pluggable storage architecture was one of the major enhancements introduced in vSphere 4. Functionally similar to Microsoft’s MPIO stack for Windows, PSA includes native multipathing support and allows vendors to plug in their own advanced features.

I find the VMware diagram confusing. Is mine more or less accurate and readable?

The ESX kernel (VMkernel) walks down through three layers when communicating with storage:

1. In the top layer, VMware native NMP or third-party MPP software decides which SATP to use, or whether to use the native interface. MASK_PATH also operates at this layer.
2. The SATP layer includes native generic path selection (active/active, active/passive), standard ALUA, as well as allowing third-party plugins (SATP) to override its behavior. The SATP monitors these paths, reports changes, and initiates fail-over on the array as needed.
3. At the PSP layer, software decides which physical channel to use for I/O requests.

There are three types of PSA plugins for vSphere 4:

1. Storage Array Type Plug-In (SATP)
2. Path Selection Plug-in (PSP)
3. A complete third-party multipathing software stack (MPP)

As is the case with VAAI, VMware includes a number of third-party plug-ins in the ESXi install. Users can simply activate many of these according to their needs, though some require additional fees and licensing.

Storage Array Type Plug-in (SATP) List

Storage Array Type Plug-Ins (SATPs) to the VMware Pluggable Storage Architecture multipathing solution for the specific characteristics of the storage array. This is very important, since each storage array design differs substantially in detail and support, especially when it comes to load-balancing and failover between controllers, ports, and paths. So it is critical for VMware to have developed a standard interface to communicate with arrays.

SATPs allow load balancing across multiple paths, intelligent path selection, and over troubled conditions such as “chatter”, when passed rapidly fail back and forth between controllers.

The SATP has critical tasks to perform in the PSA stack:

1. Decide which method of communication to use with the storage (PSA or native)
2. Monitor the health of the physical I/O channels or paths
3. Report any changes in the state of the paths up the stack
4. Perform actions required to fail over storage between controllers on the array

VMware vSphere includes a variety of generic plugins for storage arrays. I’ve identified the following:

• VMW_SATP_LOCAL – Local SATP for direct-attached devices
• VMW_SATP_DEFAULT_AA – Generic for active/active arrays
• VMW_SATP_DEFAULT_AP – Generic for active/passive arrays
• VMW_SATP_ALUA – Asymmetric Logical Unit Access-compliant arrays

Although I have sometimes seen other SATP plug-ins mentioned, the following plug-ins are all that are listed in the VMware ESX Hardware Compatibility List.

• VMW_SATP_LSI – LSI/NetApp arrays from Dell, HDS, IBM, Oracle, SGI
• VMW_SATP_SVC – IBM SVC-based systems (SVC, V7000, Actifio)
• VMW_SATP_CX – EMC/Dell CLARiiON  and Celerra (also VMW_SATP_ALUA_CX)
• VMW_SATP_SYMM – EMC Symmetrix DMX-3/DMX-4/VMAX, Invista
• VMW_SATP_INV – EMC Invista and VPLEX
• VMW_SATP_EQL – Dell EqualLogic systems

EMC PowerPath and HDS HDLM also support a variety of storage arrays, but I would classify these as full MPP replacements for PSA, rather than SATP plug-ins.

You can see which SATP plug-ins are available using the following esxcli command:

esxcli nmp satp list

Path selection plug-in (PSP) List

In contrast to the diversity of VAAI and SATP plug-ins, the universe of path selection plug-ins is fairly small. Most storage arrays are supported with either Most Recently Used (MRU) or Fixed path selection approaches. Many also support Round Robin (RR) path selection. The only vendor with a specific PSP that is not also part of a full MPP (like EMC PowerPath or HDS HDLM) is Dell, which offers a special routed path selection plug-in for the EqualLogic iSCSI arrays.

• VMW_PSP_MRU – Most-Recently Used (MRU) – Supports hundreds of storage arrays
• VMW_PSP_FIXED – Fixed – Supports hundreds of storage arrays
• VMW_PSP_RR – Round-Robin – Supports dozens of storage arrays
• DELL_PSP_EQL_ROUTED – Dell EqualLogic iSCSI arrays

As mentioned, EMC PowerPath also offers path selection as a plug-in in addition to the full MPP stack. Many other vendors offer unique path selection plug-ins, over 100 in total, but these are not specifically called out in the VMware HCL apart from their existence. I would love to learn more about them, however.

You can see which SATP plug-ins are available using the following esxcli command:

esxcli nmp psp list

Tell Me More About PSA!

As mentioned above, I am by no means an expert in VMware Pluggable Storage Architecture. Rather, I am interested in learning more and passing on this knowledge to others. I welcome your comments and feedback, and especially your corrections to the information presented here. I will try to keep this page updated as new versions of vSphere are introduced and as I learn more about this technology. Thank you for your help and understanding!

Further Reading:

• What’s New in VMware vSphere 4: Storage
• Next-Generation ESX Storage: A Pluggable Core Storage Architecture
• Storage Changes in the VMware vSphere 4 Family
• Pluggable Storage Architecture, exploring the next version of ESX/vCenter
• Vmware vSphere 4.1 PSA (Pluggable Storage Architecture) Understanding
• Explain the Pluggable Storage Architecture (PSA) Layout

© sfoskett for Stephen Foskett, Pack Rat, 2011. | VMware PSP and SATP in Plain English
This post was categorized as Enterprise storage, Everything, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

Why do some data storage solutions perform better than others? What tradeoffs are made for economy and how do they affect the system as a whole? These questions can be puzzling, but there are core truths that are difficult to avoid. Mechanical disk drives can only move a certain amount of data. RAM caching can improve performance, but only until it runs out. I/O channels can be overwhelmed with data. And above all, a system must be smart to maximize the potential of these components. These are the four horsemen of storage system performance, and they cannot be denied.

The Chain of Command

It is tempting to think of storage as a game of hard disk drives, and consider only The Rule of Spindles. But RAM cache can compensate for the mechanical limitations of hard disk drives, and Moore’s Law continues to allow for ever-greater RAM-based storage, including cache, DRAM, and flash. But storage does not exist in a vacuum. All that data must go somewhere, and this is the job of the I/O channel.

To be useful, storage capacity must connect to some sort of endpoint. This could be the CPU in a personal computer or an embedded processor in an industrial device. Indeed, there are endpoints and I/O channels throughout modern systems, with potential bottlenecks, caches, and smarts at each point. “Storage people” like me tend to think too small – imagining that the I/O channel ends at the disk drive, the “front end” of the array, or the storage network. But data must travel further, all the way to its final useful point in the core of the CPU.

Once we consider I/O as a long chain of interconnected endpoints, we begin to see the fact that I/O constraints at any point can strangle overall system performance. This is not merely an academic exercise: Optimizing the I/O channel is a consuming passion for most practitioners of enterprise IT, including architects, engineers, and system developers. And, like a good game of Whack-a-Mole, increasing the speed of one link causes another chokepoint to rear its head.

Parallel and Serial I/O

Imagine you had a warehouse full of boxes to move across the country as fast as possible. There are a few options available to you:

1. A fast truck can zip back and forth with just a few boxes
2. A train is slower, but its many cars can haul a huge quantity

But there are realistic limits to both capacity and speed: The train has to fit on the tracks, and the truck can’t move at the speed of light. Plus, one must consider the time taken to load and unload the chosen vehicle.

We continually shift between parallel and serial I/O paradigms

The same trade-offs are true of computer busses: Serial channels can be optimized to zip individual bits back and forth, or parallel busses can be designed to carry whole bytes (or more) at a time. The simplicity of serial communications is tempting, but designers continue to resort to parallelization for added throughput.

Note: Most serial protocols actually feature two links, making them “full duplex”: One for transmit and another for receive.

Serial storage interconnects are dominant currently, with fraternal twins SAS and SATA coming to dominate the disk interface landscape. SAS and SATA share the same 1.5, 3, and now 6 gigabit per second serial physical interconnect, offering more than enough throughput for conventional hard disk drives and edging out older serial (Fibre Channel, SSA) and parallel (ATA and SCSI) alternatives.

Networks (Ethernet, Fibre Channel, and InfiniBand) are predominately serial as well, as are lower-end interconnects like USB and FireWire. Serial communication also dominates in the system bus world, with serial PCI Express toppling parallel PCI.

But parallel variants are often offered for increased throughput: Multi-lane PCI Express and bonded multi-link InfiniBand make up a fair portion of the installed base, while load balancing MPIO drivers are common in Fibre Channel storage. And let’s not forget that the “X4″ variants of Ethernet use multiple bonded links as well.

The Definition of Bottle Neck

Most English speakers have encountered the French term, “cul de sac”, meaning “bottom of the bag” or dead end. But hard disk drives have plenty of “bottom end”, or storage capacity. When it comes to disks, the issue is usually at the neck of the bag: Data just can’t be pulled out of a hard disk drive fast enough.

Emptying a barrel of wine through a spigot takes hours, but pry the end off and the floor is covered in a moment!

The density of modern hard disk drives (the capacity of our barrel) has been growing much more rapidly than the I/O channels serving them (the spigot). Where once a hard disk drive could be filled or emptied in an hour or two, modern drives take days or weeks!

I once called this “flush time“, but I think the wine metaphor is much more appetizing!

This “bottle neck” has serious implications beyond basic storage performance. Data protection is impacted, since ever-larger storage systems can no longer be backed up by dumping their content; system reliability is reduced, since week-long RAID rebuilds increase the risk of multiple drive failures; and cost containment efforts are also impacted, since adding spindles drives up prices.

Nowhere is this bottleneck more evident than in portable devices. Modern drives (like the 1 TB Seagate USB drive I recently reviewed) have massive capacity and pathetic performance. The USB 2.0 interface just can’t keep up, and this creates a limit to the expansion of capacity. It would take half a day to fill that drive under perfect conditions at 25 MB/s, reducing its value as a massive data movement peripheral. The emerging USB 3.0 standard promises to alleviate this performance issue for now, as illustrated with Iomega’s new external SSD.

Cache and solid state storage can help, but they have their own bottlenecks. Storage arrays typically use Fibre Channel or SAS SSDs, and their front-end interface remains the same. The best-performing SSDs use the PCI Express bus directly rather than emulating hard disk drives over SCSI interfaces. And even PCI Express might not be enough to handle the massive I/O of NAND flash or DRAM. In each case, the bottleneck moves down the chain.

A Chain of Bottlenecks

Let’s follow a typical I/O operation from the disk to the CPU core and count the I/O channels:

1. A read head senses the state of a bit of magnetic material on the surface of a disk
2. The head transmits this signal to a buffer on the disk controller board
3. The data is picked up by the disk controller CPU and transmitted over a SATA or SAS connection
4. The storage array or RAID controller receives the data and moves it over an internal bus to another buffer or cache
5. The data is picked up by another CPU in the array controller and sent out another interface using Fibre Channel or Ethernet
6. The data is buffered and retransmitted by one or more switches in the storage network
7. The host bus adapter (HBA) on the server side receives the data and buffers it again before sending it over a local PCI Express bus to system memory
8. The server memory controller pulls the data out of system memory and sends it via a local bus to the CPU core

There are actually many more steps than this, but the picture should be clear by now. There are many, many I/O channels to consider when it comes to storage, and the drive interface is just one potential bottleneck.

We constantly move bottlenecks around - as one link is improved, another choke-point appears

Optimizing Storage I/O

Tactical steps to improve storage performance typically focus at one link in the chain: Drive vendors move from 1.5 Gb to 3 Gb SATA, or SAN buyers upgrade from 4 Gb to 8 Gb Fibre Channel. But the basic architecture of enterprise storage has remained constant for over a decade, and the reliance on block SCSI commands endures. This is all about to change.

One critical bit of I/O optimization exists at the point of connection between the various chipsets inside the server. AMD pulled the memory controller off of the “northbridge” with their Athlon line. Intel did the same with their Nehalem and is eliminating the northbridge entirely with the Lynnfield/Jasper Forest CPU lines. This gives serious bandwidth to the crucial PCI Express-to-CPU-core link, moving the bottleneck downstream.

We are in the midst of a massive upgrade of the storage network as well. Between 8 Gb Fibre Channel and iSCSI and Fibre Channel over 10 Gb Ethernet, not to mention persistent interest in InfiniBand, storage network throughput is rapidly expanding. As with the internal PC connections, the expansion of network bandwidth has pushed the bottleneck to the storage array interface for the time being.

Microsoft and Intel recently pushed over a gigabyte per second over 10 GbE using iSCSI, but they needed multiple storage targets to feed that connection. It isn’t that modern storage systems couldn’t push that kind of I/O (indeed, arrays are tens to hundreds of times faster internally thanks to their spindles and cache), but that the conventional storage protocols are tightly linked to a single “front-end” interface. The current state of the art for storage array design is moving to distributed models, exemplified by pNFS and scale-out NAS concepts like MaxiScale (now acquired by Overland).

Once the array interfaces can pump out massive I/O, attention will turn once again to the disk interfaces themselves. Although 6 Gb/s SAS and SATA is now a reality, this interface is inappropriate for future high-performance SSDs. Arrays designed around flash or DRAM are likely to switch to PCI Express as their internal connection of choice for performance and to optimize data placement on these new devices. Companies like Nimbus and NetApp are already moving in this direction.

Time To Get Smart

Hard disk drive spindles make up the bulk of storage capacity, but small amounts of cache make them far more effective. But both of these horsemen must operate within the constraints of the I/O channels they pass through. This brings us to the final horseman of performance: Smarts. Clever designers have created clever controlling mechanisms to overcome the limits of spindles, cache, and I/O channels.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | The Four Horsemen of Storage System Performance: I/O As a Chain of Bottlenecks
This post was categorized as Computer History, Enterprise storage, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Driving Massive I/O

Chuck Hollis treated us to a discussion of vSphere as an I/O Engine on his blog this morning with some background on multipath IO (MPIO for short), but I’m not sure he did the topic justice. In my opinion, server virtualization is the greatest I/O driver ever brought into the data center, and it messes with all of our preconceived notions about I/O at the same time.

What’s so special about server virtualization?

1. Hypervisors concentrate I/O, shifting loads that were formerly distributed to a large number of I/O channels into a far fewer channels. Picture 10 servers doing what they do. Now put all 10 in a single physical box. All of their storage access must now share a bus, a host adapter, a cable, and perhaps a LUN on the storage system. It’s the difference between lemonade and lemon juice!
2. Hypervisors randomize I/O, chunking everything up and mixing it together. Forget about the carefully-designed read-ahead algorithms and caching used in enterprise storage – VMware, Hyper-V and the rest throw those expectations out the window! Virtualization is a blender – it grinds up your lemons, skin, seeds, and all!
3. Hypervisors demand low I/O latency, forcing infrastructure to get quicker, not just faster. This is one reason that caching, solid state disks, and 10 GbE are going to be huge in virtual environments – all reduce latency by orders of magnitude! As any car guy will tell you, quick and fast are two very different things!

The upshot of all of this is that virtual servers are very very hard to satisfy when it comes to I/O. And the “back end” has always been a bit of a bottleneck for virtualization software. Now we have VMware claiming that vSphere 4 can push over 300,000 I/O operations per second (IOPS) without resorting to VMDirectPath and similar “cheater” measures. Of course not all IOPS are equal, and I doubt that that 300k number would hold up with a real-world workload, but it’s impressive nonetheless!

A Brief History of MPIO

Let’s turn back to multipath I/O. PowerPath/VE is just the latest in a long line of path managers, not all of which have been well-loved. Back in my HP-UX days I learned to make the most of PVlinks, the native path management on that operating system. It wasn’t always easy to get it to work well, but it sure was nice to have a path manager built into the operating system! Veritas also offered a multi-platform path manager, DMP, which worked with a variety of array types. Back in the day, both were limited to simple failover and lacked the “intelligence” to deal with the peculiarities of the weird storage arrays we learned to not hate.

Array-specific path managers from storage vendors were much more successful. CLARiiONs used ATF, Hitachi arrays used HDLM, IBM had SDD, and of course EMC had PowerPath. EMC introduced PowerPath in 1997, the software reportedly having been developed by Conley Corporation, which EMC acquired the next year and turned into its Cambridge (MA) development center. After acquiring Data General, EMC adapted PowerPath to support CLARiiON, pushing ATF off stage right. Then they kept right on developing the software, adding support for IBM, HDS, and HP arrays and data migration.

Meanwhile, Microsoft decided that HP and Veritas were on to something when they developed standard path management software, so they began working on a standard multi-path IO (MPIO) driver for Windows. But Microsoft learned a thing or two from the mediocre device support in those old solutions, so they decided to allow vendors to plug their own smarts into the standard Windows Server 2000/2003 MPIO framework. Microsoft provided basic failover capability and third parties, including EMC, wrote their own device-specific modules (DSMs). This MPIO support evolved and spread, standard on Microsoft’s iSCSI initiator and Hyper-V virtualization platform. PowerPath 5.2.1 for Windows already supported Hyper-V thanks to this.

PowerPath and VMware PSA

VMware also learned a thing or two from HP and Microsoft. Although basic path failover support has been included in ESX for years, vSphere 4 takes it to a new level with pluggable storage architecture (PSA). Every version of ESX 4 includes native multipathing (NMP), but Enterprise Plus licensees can use vendor-supplied plugins to enable more advanced path management. As I noted on Tuesday, there are two three different levels of path selection: Basic path-selection plugins (PSPs), more advanced storage array type plugins (SATPs), and complete multi-path plugins (MPPs).

This is what EMC has introduced: An MPP for vSphere 4 called PowerPath/VE. Like the DSM for Windows MPIO, PowerPath/VE for vSphere slots right into an existing MPIO framework and enables advanced path selection and load balancing without mucking with the internals of the hypervisor. PowerPath/VE has all sorts of smarts in it. It has eight different predictive load balancing policies, proactive disconnect, bus testing, and HBA monitoring.

Super VMware guy Chad Sakac described PowerPath/VE as part of the launch. He notes that EMC is first out of the gate with a multipathing plugin for vSphere, but I suspect that just about every vendor will release similar functionality pretty quickly. In particular I expect support to come from NetApp and 3PAR, since they’re so interested in VMware support.

Licensing Questions

One thing really stuck out in the vSphere launch: PSA is only included in the top-of-the-line Enterprise Plus license. Presumably, this means that, in addition to paying for a PowerPath/VE license, users will have to spring for maximum ESX, too. This is a dumb move, if you ask me. Microsoft made MPIO successful by giving it away with every copy of Windows. They even included it in the free iSCSI initiator download. VMware, in contrast, seems to be actively limiting PSA’s usefulness to the top tier of users. If it was up to me, I would set the VMware MPIO free!

I’m working with EMC and VMware to determine the extent of the NMP/PSA/PowerPath licensing mess. I’ll update this post as I find out the answers!

1. Does every edition of ESX 4 include the basic VMware native multipathing (NMP)?
2. Can one use a vendor-supplied PSA plugin like PowerPath/VE without an enterprise plus license?
3. Does it matter (to licensing) if the plugin is a PSP or an SATP?
4. If “no” to 2 or 3, can PSA be added separately without the plus license if someone wants to use something like PowerPath/VE?

Update: I received a nice email from an EMC engineer correcting me about the plugin types. This kind of open communication is why the web is so great! It turns out that PowerPath/VE is a sort of super plugin called an MPP, not “just” an SATP or PSP. I’ve updated the section above!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | PowerPath To The Virtual People
This post was categorized as Computer History, Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Each year, Microsoft invites all current MVPs to a big shindig in Redmond and Seattle, the MVP Global Summit. It’s a pay-your-own-way kinda thing, though, so I wasn’t sure I could attend. In fact, given the economic climate and pressures from work, I was sure I could not attend.

But things have a way of working out. Through a curious set of circumstances, I will be able to attend two days of the Summit after all!

So next week I will be in Washington State meeting with the gurus behind Windows Server, the most excellent iSCSI initiator and MPIO system, and the rest of the actually quite good stuff to come out of Redmond.

If you’re in Seattle too, drop me a line. Otherwise, let me know if there’s anything I can ask them, or if you want a kewl mouse pad from the company store! And yes, I’m bringing my Mac!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Attending Microsoft’s MVP Global Summit
This post was categorized as Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

So let me sum up:

1. Server Message Block (SMB) 2.0 is probably the biggest news.  Microsoft re-worked this protocol for Vista and 2008 to reduce chattiness, combine multiple commands in a single packet, and allow larger packets.  This should improve performance for just about everyone, but is only in those two operating systems right now.
2. The excellent multi-path I/O (MPIO) driver from Microsoft is also included right out of the box, including device-specific modules for both iSCSI and Fibre Channel.  This is the first version of Windows to come with MPIO, though it’s available with the iSCSI initiator and OEM drivers for other versions.
3. As in Vista, Server 2008 allows you to shrink NTFS filesystems on the fly.  It’s not quite as flexible or forgiving as some third-party tools, but it’s certainly easy to use!  Just go into the disk administrator snap-in and try it for yourself.
4. Windows finally has the option to leave new LUNs alone instead of trying to mount them.  This was one of those things that gave Windows a rep as a bad neighbor in SAN environments.
5. NTBackup is gone, replaced by a new Server Backup MMC snap-in.  I haven’t tried it, but I hope it’s better than the one that comes in Vista!  I ended up keeping NTBackup around on my laptop…
6. A new MMC snap-in called Storage Explorer lets you manage WMI-compliant SAN devices.
7. The Volume Shadow Copy Service (VSS) and DFS interfaces are improved and are now scriptable.
8. Server 2008 now automatically aligns filesystem boundaries with storage, which was one of those dark and secret skills us storage guys used to share amongst ourselves.  This can increase performance in high-I/O environments.
9. NTFS (in both Server 2008 and Vista) now has symbolic link support, just like UNIX and Mac OS X.  Note that this got mangled in the TechTarget article.
10. NTFS was also tuned and tweaked a bit for better stability and crash recovery.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Windows Server 2008 Changes Storage
This post was categorized as Enterprise storage, Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

It wasn’t always like this – be thankful if you don’t remember FTEDIT! But ever since Windows 2000, Redmond has made improvement after improvement, remaking “bad (SAN) citizen” Windows into Martha Stewart. But unlike the latter, Microsoft hasn’t called much attention to its skills, and this is a shame…

Take iSCSI – Microsoft was an early supporter of the protocol, releasing an excellent software iSCSI driver as a free download. They also bundle a limited (but continuously-improving) volume manager with all modern versions of Windows. Then there’s VSS, which is the first hardware-independent (even hardware-free!) snapshot API I know of. I’ve written articles (1, 2) on Windows storage technologies which go into these in more detail.

But one question that came up in my virtualization seminar made me realize that I forgot one key piece of Gates-tech: MPIO. See, Microsoft has also been bundling a free hardware-independent multipath I/O driver in server editions of Windows since 2003, but lots of folks haven’t gotten the memo. It’s good stuff: A generic driver with device-specific modules (DSMs) for different storage array and network types. MPIO handles transparent multi-path failover (for availability) and load balancing (for performance).

If you have iSCSI, you simply must try MPIO since Microsoft’s own free DSM supports about everything you need, and compatibility is required for logo support. And if you’re on Fibre Channel, you’re probably in luck, too, since most major vendors provide DSMs for their arrays (but some might not be free, I’m told).

So there you have it. Another excellent (and free!) Microsoft product that you (probably) never heard of, cutting out proprietary solutions (at least for Windows Server…)

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Microsoft: The A-Rod of Storage
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.