What is Multipath? And Why?

The good old days: One device, one path

In the beginning, computers connected to peripherals and other computers through a single bus or channel and life was easy. Although one might mistake the names of the dominant printer connection (parallel) for some kind of multipath system when compared to the modem connection (serial), this was not the case. Only the bits traveled in parallel – the logical connection was a simple single path.

Early-90's servers might confuse admins with two SCSI connections to a single device

Then things got complicated. The SCSI protocol allowed for multiple devices in a chain, and even for two different “initiators” (computers or controllers) to interact with these “targets”. Some folks even dual-attached devices to a single computer with multiple controllers.

Why would one device and one computer need more than one connection? It boils down to two factors:

1. Performance – I/O channels have typically been slower than the computer could handle, so multiple channels might be used to increase the amount of data that can flow in and out.
2. Reliability – If one connection failed, the other might still be usable, reducing the risk of an outage.

Late-90's enterprise systems might have four or more paths to a single storage array

Pretty soon, enterprise computer architecture had gotten incredibly complex. I remember connecting a massive HP V-class server to an EMC Symmetrix with eight separate Fibre Channel cables. Each disk “LUN” showed up twice, and we had hundreds of them. We managed all of these virtual storage paths using HP’s PVLinks dynamic multipathing software. We used Veritas DMP and EMC PowerPath to do pretty much the same thing on Solaris and other UNIX systems.

Active/Passive to Active/Active

The earliest path management software provided two incredibly important functions: It figured out which of the SCSI targets it saw were actually different names for the same one, and it allowed the operating system to choose one and fail over to the other in case of an interruption. These were Active/Passive links – no matter how many paths were presented (and Fibre Channel switches sometimes presented eight or more), only one was active at any one time.

Modern systems have abstracted and virtual I/O channels, making path management much more important

But the EMC Symmetrix and similar high-end storage systems changed all this. Symmetrix storage was fully virtualized – the presentation of LUNs to servers was entirely disconnected from the actual disks and RAID sets in the array. This meant the Symmetrix could handle I/O requests across different paths and controllers for the same LUN. EMC and the rest responded with Active/Active path management software, allowing I/O to travel in parallel for the first time.

How is Dual Active Different?

Not everything called Active/Active is created equal. In fact, many supposed Active/Active setups really shouldn’t be called that since they don’t use both paths for all data. Instead, I like to call these Dual Active – both paths are active but with different data.

Consider the differences between the following two solutions:

A true active/active setup uses all paths for all data all the time

A "dual active" setup uses both paths, but each target is directed to one or the other

See the difference? Although the paths are active in both cases, they are not the same. Both approaches have merit, and neither is inherently superior, but they should have different names applied. Even active/passive has its place, since simplicity is often a virtue.

Dual Active Outside Storage

These same concepts apply outside the field of storage and I/O. Many server clustering systems use the same terminology, right down to the misapplication of “active/active” when “dual active” is more appropriate. It’s easy to miss the significance of this difference, but it can make more of an impact in clustering since CPU workloads are harder to balance.

Let me know what you think. If there is interest, I might dive into path management strategies like round robin!

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Multipath: Active/Passive, Dual Active, and Active/Active
This post was categorized as Computer History, Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

One of the great things about being a Microsoft MVP is the access I get to Microsoft software and staff. As I mentioned in my post about the 10 cool storage features from the 2009 Microsoft MVP Summit, I was able to preview a lot of what Microsoft is doing with their Server software and storage features. And the best part is that the Microsoft product teams are keenly interested in our feedback and suggestions. I’m told, for example, that the awesome iSCSI Quick Connect feature in the new Windows iSCSI initiator software was developed based on my feedback!

As I note on my Windows Storage Server 2008 preview on Gestalt IT, Microsoft has always kept WSS close to the vest. It’s only available to OEMs, not retail customers, and has never even been shared with TechNet or MSDN subscribers in the past. So I was really pleased when Microsoft gave the File System Storage MVPs access to a beta version of WSS 2008 so we could get a feel for all of the new features. I’ve also had some great conversations this week with the Microsoft product managers responsible for it.

What’s exciting about Windows Storage Server 2008?

1. It includes all of the storage enhancements in Windows Server 2008, including SMB 2.0 for much much faster file servicing over higher-latency links, SMfS, FSRM, enhanced VDS, and failover clustering.
2. WSS is the only way to get access to Microsoft’s iSCSI target software. It’s been improved in many ways from the prior releases, but its support for what Microsoft calls dual-active clustering is probably its most notable feature: You can’t share the same active LUN between cluster members, but each can have its own active LUNs and the can all fail over in the event that one member goes down.
3. The included single-instance storage (SIS) file-based deduplication has been much improved, scaling to 128 volumes per server and millions of files. It’s still not as effective capacity-wise as block-level deduplication (which I’d love to see, hint hint), but the performance is solid enough to use it for primary storage with production applications.
4. Probably the coolest feature exclusive to Windows Storage Server 2008 is its new browser-based remote administration capability. Just point your browser to the Storage Server machine (for example, “http://wss/desktop”) and you’ll get a full ActiveX version of RDP. Don’t use Internet Exploder? Firefox and Linux users will get a Java-based RDP instead! I will cover this feature more in the future, but let’s just say that every operating system should offer this!

Want to try Windows Storage Server for yourself? Breaking from the past, Microsoft will soon (like next week!) allow TechNet subscribers to download the full install. OEMs have a sekrit back-door site to try it out, too.

One more thing… This will be the last release of Windows Storage Server. There won’t even be a special Storage Server version of Server 2008 Service Pack 2! Starting now, Storage Server is just an optional feature of Windows Server. Purchasing and production use will still be limited to storage OEMs, but Microsoft has finally reconciled Storage Server with the rest of the Windows Server world. I imagine that most OEMs will release Service Pack 2 updates for their Storage Server customers shortly, and that future versions of the product will come closer to the base Server versions than WSS 2008. Although I can’t share what I know, I will say that Microsoft is continuing active development on their iSCSI target, single-instance storage, and other Storage Server features. I imagine that Windows Server 2008 R2 will support storage systems in the very near future!

On a personal note, reading A Brief History of Windows Storage Server Releases from the Microsoft Storage Server blog reminded me of the original unveiling of Windows Storage Server at Storage Decisions Chicago in June, 2003. The company loaded us all on one of those lake cruise boats with some celebrity impersonators, chocolate “Oscar”-style statues, an open bar, and a band. Good times ensued!

For more details, check out my Gestalt IT piece, Windows Storage Server-Based Systems Step Into 2008

Microsoft is detailing the new version of Windows Storage Server 2008 in a webcast Thursday at 8 AM Pacific. You should also check out the Microsoft Storage Server blog.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | I Can Finally Talk About Windows Storage Server 2008!
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

One of the main questions I always get is which protocol one should use for VMware storage. My recommendation remains that the answer is an organizational one more than a technical one.  There are certainly performance, CPU utilization, and support differences between Fibre Channel, SCSI, iSCSI, and NFS on VMware, all of these can work fine in many situations.  Although this is addressed in my presentation, I thought it wise to point out some of my sources and (concurring) opinions.

First, I point you to the official VMware VI Team blog, where they reiterate that VMware is protocol-agnostic.  They commit to support all storage protocols equally, and promise to add missing support as soon as possible.  See especially their table of support, which shows that iSCSI currently can’t be used for clustering (!), among other insights.

I’d also like to point out three sources for my seminar slides:

• VMware’s Comparison of Storage Protocol Performance paper, which pits Fibre Channel against hardware iSCSI, software iSCSI, and NFS.
• VMware’s Performance Characterization of VMFS and RDM Using a SAN, which shows that there is a negligible performance difference between shared storage and the two varieties of RDM.
• NetApp’s NetApp and VMware Virtual Infrastructure 3 Storage Best Practices (recently updated), which is a wealth of information on shared storage with NFS, even if you’re not a NetApp customer.

The only real gotchas at this point are the lack of clustering support for iSCSI, the inability to boot a VM from software iSCSI, and the learning curve for Fibre Channel.  Make your choice based on what you have and what you know – that’s the best choice to make!

For more information, check out this post from vmguy.com!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Which Storage Protocol For VMware?
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.