I'll be presenting at two private events and one public webinar this week!

I’ve been quite busy lately recording webinars, writing articles, and setting up speaking events. This week, I’m in Boston and New York for private speaking engagements. I’ll also be recording a webinar with an exciting new storage/virtualization company, Nutanix, and I urge you to tune in to that one!

Nutanix is Truly Innovative

Watch Nutanix present at Tech Field Day 8

It’s rare to come across a truly innovative idea in enterprise IT, but that’s exactly what Nutanix is. They’ve taken the scale-out Google Filesystem and ran with it, applying it to the core issues of sever and desktop virtualization.

My webinar with Nutanix is little bit different from the typical “give them an excuse to go through their slide deck” event. We’ve organized it as a discussion between me, Tiffany To of Nutanix, and a customer of the company. Tiffany will have a set of slides as backup, but we won’t walk through them one by one.

I’m glad to be able to shake things up a little bit, since the old webinar formula is getting pretty stale. We were able to do this thanks to the third party involved in this production: Truth in IT. They have put together an excellent webinar platform and are eager to try new ideas.

If you would like to view the recording, it is now available. You can register to view this webinar at the TruthinIT website.

Speaking Engagements in the Northeast

I’m also speaking on a couple of interesting topics this week. Today, I’m giving an overview of enterprise storage industry trends to an investment banking firm. I often get this sort of request, and am happy to develop a customized presentation. In this case, the event was initiated by an attendee at my Storage for Virtual Environments seminar.

On Thursday, I’ll be speaking at a private event in New York City sponsored by Dell and Symantec. This time, the topic is the divergent paths of performance and capacity in enterprise storage. This is another topic I’ve spoken and written about in the past, but developed some new content just for this CIO audience. I will probably be delivering similar evening speeches and events next year.

Sadly, my Storage for Virtual Environments seminar in San Diego next week has just been canceled by the organizer. I was really looking forward to this, my final seminar of the year. But I’ll be in San Diego regardless, and look forward to meeting up with some of my friends who are planning to attend. If you are around next week, drop me a line.

Oracle ZFS Storage Appliance

I’ve also recorded a webinar with Oracle last week. We talk about my favorite Oracle product: The Sun ZFS Storage Appliance. This hasn’t been posted yet, but I’ll link to it here when it appears!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Two Online Events You Should Attend (And Two More That You Can’t)
This post was categorized as Enterprise storage, Everything, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

You probably also want to read Key Storage Features in Mac OS X 10.7 “Lion” and Three Key Storage Features Missing in Mac OS X “Lion”

Apple’s Storage Story: A Mixed Bag

Apple’s file system, HFS+, is quite long in the tooth, dating back even before Mac OS X! Introduced in 1998′s Mac OS 8.1, HFS+ was extension of the original hierarchical file system (HFS) used by classic Mac operating systems back to System 2.1. But HFS+ has been extended and updated dramatically since those days, including the introduction of journaling in Mac OS 10.2.2, Unicode in 10.3, access control list-based file security in 10.4, and compression in 10.6.

The storage technology that underlies HFS+ has also improved dramatically over the years. Apple adopted EFI firmware and the GUID partition table (GPT) much more rapidly than the PC world. This allows Mac OS to scale to larger storage capacities than Windows systems, and has made the adoption of 4K sectors (so-called “advanced format”) much easier.  In fact, Apple is well ahead of the Windows world in these areas even as it lags with an outdated filesystem and no volume manager.

Mac OS never had much in terms of storage flexibility. Sure, AppleRAID brought some flexibility and redundancy to the server world, but the average Mac user never delved much into this technology. While Apple was asleep at the wheel, every other operating system vendor introduced a flexible volume manager. Even Microsoft, who is often criticized for slow innovation, is way ahead of Apple when it comes to data storage.

What is a volume manager? Volume managers add flexibility to operating system allocation of storage, acting as a virtualization layer between disks and file systems. This is important, since modern computer systems need to maintain a consistent filesystem image even as storage devices change. Volume managers like Symantec’s Veritas Volume Manager and the OSF LVM allow many operating systems this sort of flexible storage management. But Apple never had a volume manager of their own until the introduction of CoreStorage in Mac OS X “Lion”. Read more at Volume Management: Virtualizing Host Storage.

Introducing CoreStorage

All this has changed in Mac OS X 10.7 (“Lion”), however. Although Mac OS never did get ZFS, as was planned in 10.5, the storage enhancements found in 10.7 are getting awfully close in terms of features. Along with integrated file versions and full disk encryption comes a new technology called CoreStorage, including a full logical volume manager.

CoreStorage became necessary due to the inclusion of full volume encryption in Lion. Previously, Apple’s FileVault encryption technology relied on regular files to contain an encrypted filesystem. This worked fine until the need became apparent to encrypt an entire volume, whether it is the operating system disk or a removable drive. Although file-based technology can work in these scenarios, it is much less than ideal and also less than what competing operating systems offer.

But enabling full volume encryption requires an abstraction layer that Mac OS X simply did not have. Apple appears to have taken the long route, engineering a complete volume manager rather than hacking together some sort of kludgy mechanism. Although it is only used for FileVault 2 at the moment, CoreStorage appears to be a full volume manager that could, in the future, boast advanced features like mirroring, snapshots, and online storage migration.

CoreStorage Data Structures

CoreStorage implements a full complement of data structures but uses them in a very simplistic manner currently.

Apple did not simply copy existing volume manager semantics, and it does not appear that CoreStorage is an OEM version of someone else’s product. Rather, there are some intriguing new innovations present in CoreStorage that point to exciting possibilities in the future.

Like most volume managers, the basic building block of CoreStorage is the “logical volume group” (LVG). Illustrated in blue and purple above, each LVG takes storage from a “physical volume” (PV) and presents it to the operating system as a “logical volume” (LV). In this way, storage from a physical disk is abstracted before used by the operating system, allowing a great deal of flexibility. This is somewhat analogous to a hypervisor presenting a virtual machine interface.

Apple goes a step further than most volume managers, however, with the introduction of a new concept, the “logical volume family” (LVF). The LVF specifies properties that will be inherited by logical volumes that it contains. Currently, the only property specified by an LVF is FileVault encryption, but one can imagine that performance characteristics or redundancy could also be specified in this manner.

CoreStorage Today and Tomorrow

Today, when full volume FileVault encryption is used, Mac OS X automatically converts the desired volume into a CoreStorage volume, and encapsulating the partition as a PV, importing it into an LVG, and setting up an LVF and LV to contain the new filesystem. Only one of each element is created in this way, and all capacity is allocated at creation time. So there isn’t much use for the entire stack right now.

Using diskutil from the command line, one can create multiple logical volume families and logical volumes.

The Disk Utility GUI is fairly limited as well. Volumes can be converted to use FileVault 2, but all data is lost in this process. Only boot volumes can be corrected nondestructively through the GUI, and this happens in the Security & Privacy preference pane rather than Disk Utility. When boot volumes are encrypted, Apple gives users the option to “escrow” the decryption key online using their MobileMe or iCloud account, but this is not allowed for other volumes.

Currently, whenever a volume is converted to using FileVault, it is also automatically converted to CoreStorage behind the scenes. And if encryption is turned off in the GUI, it is converted back (“reverted”) to whatever it was before. But it’s possible to convert a volume on the command line without encryption, and to remove encryption without reverting. It’s also possible to manually manipulate CoreStorage in other ways.

In the future, it should be possible to manipulate storage much more fully.

The fact that Apple included a full volume manager of their own making indicates a bright future for CoreStorage. Already, it is possible to use the command line utility, diskutil to nondestructively convert and encrypt volumes. It is also possible to manipulate volume groups, logical volume families (indirectly), and logical volumes to a limited extent using diskutil.  One hopes that future versions of Mac OS X will include additional functionality, including the ability to manipulate physical volumes.

Stephen’s Stance

Either Apple has big plans for CoreStorage, or some Apple software engineer got way ahead of himself and designed a complete logical volume manager just to house FileVault data! I think the former is likely, since we know that Apple wanted to include ZFS, a flexible combination volume manager and file system, in Mac OS X 10.5 but was unable to due to licensing concerns. Now two versions later, Lion includes all of the basic technology needed to effectively manage storage volumes. It is likely that the GUI simply lags behind this core technology, and we will see additional functionality added in later operating system revisions.

Regardless of intent, Mac OS X now includes a fairly full storage stack, from EFI and GPT through CoreStorage and HFS+. No longer is Apple behind the times in terms of core storage (if you pardon the pun) features. Now if only we could get them to implement iSCSI!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)
This post was categorized as Apple, Computer History, Enterprise storage, Everything, Features, Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Mac OS X 10.7 "Lion" lacks many of the storage features we've long hoped for, including ZFS, iSCSI, and USB 3.0

Last week, at WWDC, Apple introduced many of the features found in their next operating system, OS X “Lion”. At that time, I posted an article about the storage features found in this new release, including integrated revision control, a major update to the FileVault encryption package, and additional enterprise storage protocol support. But, like Snow Leopard, Lion still lacks many storage related features, and it doesn’t look like Apple will get around to adding these anytime soon.

A Better Filesystem (ZFS, Please)

See also ZFS: Super File System! and Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)

During the development of Mac OS 10.5, “Leopard”, Apple spent a great deal of time working to replace their legacy HFS+ filesystem with ZFS, a much more advanced option. For starters, ZFS would have given Mac OS better reliability and flexibility, and it has been extended to include advanced features for security and capacity optimization.

But the battle between Sun and NetApp over patents related to the development of ZFS cast a shadow over the long needed replacement of HFS+. With Oracle buying Sun and focusing away from infrastructure products like ZFS, Apple seems to have lost interest in replacing their crufty old filesystem.

Instead of adding an advanced filesystem like ZFS, Mac OS X Lion extends HFS+ with versioning and enhanced security. The new Core Data incremental storage technology in Lion would probably have been easier to implement on ZFS, but Apple was able to add it to HFS+, and it will be a lifesaver in the guise of autosave, versions, and resume. The same goes for encryption, with FileVault 2 boasting background full disk encryption, remote wipe, and external drive support.

Those hoping for the integration of ZFS with Mac OS X appear to be out of luck. All components were removed from Snow Leopard, and Lion is moving forward without it. Sadly, this means that Mac OS X still lacks a flexible volume manager, something even Microsoft Windows boasts.

Perhaps the next version of Mac OS X will include friendly volume management features, but it is more likely that Apple will focus away from the filesystem and direct application developers toward the iCloud Storage API. And cloud truly is next-generation storage, making this a leapfrog approach and leaving ZFS in the dust.

Update: Lion does indeed include a full logical volume manager! See Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)

USB 3.0: Still AWOL

See also Will OS X 10.7 “Lion” Bring USB 3.0 To The Mac? and my Thunderbolt series

Although Apple may have simply overlooked or neglected to mention it, USB 3.0 apparently made no appearance at WWDC. Thunderbolt is an impressive technology to be sure, and I am bullish on its future application and performance. But “SuperSpeed” USB 3.0 seems poised to seize the baton and become the ubiquitous next-generation interconnect for every day peripherals.

Thunderbolt is a strategic protocol for Apple, and I expect it to rapidly spread across the entire Mac product range. Rather than simply a high-speed interconnect, Thunderbolt will soon enable advanced docking features, as envisioned in my recent post about the iMac as a Thunderbolt peripheral. It will also enable changes to the physical size and shape of laptop and desktop computers and servers, with many suggesting that the next-generation MacBook Air will become the standard Apple laptop.

In contrast, USB 3.0 is simply a performance bump for USB. It is likely that Apple will support USB 3.0 sooner or later, and third-party vendors are already rolling out Mac OS support. CalDigit recently shipped their third USB 3.0 controller for the Mac, and LaCie sells their own “walled garden” card and peripherals as well. I heard rumors that a few vendors are working on Thunderbolt to USB 3.0 bridges and breakout boxes as well.

USB 3.0 will come to the Mac sooner or later, but Thunderbolt is here to stay.

Enterprise iSCSI Support

See also Will Snow Leopard Finally Bring iSCSI To The Mac?

Another technology that Apple has flirted with in Leopard but never delivered is a software initiator for iSCSI, the block storage protocol that runs over Ethernet. We have not heard anything further about iSCSI since 2007, and there was no mention in the Lion introduction either.

Interestingly, Apple did rollout other new enterprise storage protocol options, including NFSv4, DFS, and even integration of Xsan, the Fibre Channel filesystem. Xsan also added ALUA compatible multipathing, a real surprise for storage geeks like me. But iSCSI was nowhere to be found.

This is a real shame, since iSCSI is becoming increasingly common in enterprise storage circles. Convergence on Ethernet is a hot topic right now, and iSCSI for Mac would give exceptional flexibility and interoperability and fit right into the “prosumer” Mac market niche.

Instead, end-users are stuck working with third-party iSCSI initiators, Fibre Channel and Xsan, or NFS. Although I am a fan of their free globalSAN product, Studio Network Solutions does not offer enterprise support for third-party arrays. The other major option for Mac iSCSI is ATTO’s Xtend, which is supported but somewhat expensive. Drobo also offers an iSCSI client for use with their storage arrays, but it is severely limited. An integrated Apple solution would be a welcome addition, both for consumers and enterprise systems administrators.

Stephen’s Stance

Apple is not in enterprise storage company to be sure, and news from WWDC dashed any hopes we had for ZFS and iSCSI support. USB 3.0 seems a foregone conclusion, but Apple seems intent on ignoring it as long as possible. Although I welcome the new storage features included in Lion, it is disappointing that these were left out.

Note that TRIM support was also not mentioned at WWDC, but it is likely included.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Three Key Storage Features Missing in Mac OS X “Lion”
This post was categorized as Apple, Enterprise storage, Everything, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I think this sentiment is just as valid today as when I posted it in 2008!

Although many discussions in the storage industry focus on the relative merits of one protocol or another, the conversation occasionally turns to the core issue at hand: We continue to patch together a system based on outdated concepts. Most storage protocols continue to mimic direct attached storage, and most of our so-called networks act as point to point channels. An ultra-modern virtualized storage infrastructure with all the latest bells and whistles still holds the concepts of block and file at its core. Whenever the storage industry has tried to bring about real storage management they have been stymied by a lack of context for data.

No amount of virtualization, and no new protocol, will fix this. Put simply, we need a storage revolution.

Channels, Blocks, and Files

Most innovation in the 1980s and early 1990s focused on moving storage out of the server. SCSI allowed disk to exist in a separate cabinet, RAID allowed multiple physical disks to become a single virtual one, and these were mixed to become the prototype storage array. Although SCSI allowed one-to-many connectivity, it was never a true peer-to-peer network, even once it was mixed with network concepts in the form of Fibre Channel.

Even today, SAN storage is focused on providing faster, more flexible, and feature-packed direct-attached storage. A modern virtual SAN hides a complex arrangement of caching, data protection, tiered storage, replication, and deduplication, masquerading the lot as a simple, lowly disk drive. It is sad but true that all of our work as an industry has been dedicated to recreating what we started with.

Networked file-based storage is no better. Although NAS devices have all the advanced features of their SAN cousins, they must present a simple file tree to the host to retain compatibility. File virtualization merely presents a larger homogenous tree.

Inside the server, too, features and complexity are hidden to retain a familiar file system format. Volume managers can do anything a virtualization device can, but must present their output as a simple (though virtual) disk drive. File systems, too, have added features but still present a familiar tree of mount points, inodes, and files. Even ZFS, possibly the most advanced combination of volume management and file system technology yet, must present a simple tree of storage to applications.

The Metadata Roadblock

This outdated paradigm, of disks and file trees, is ill-suited to today’s storage challenges. Data must be categorized so actions can be taken to preserve or destroy it based on policies. Data must be searchable so users and applications can find what they want. Data must be flexible so it can be used in new ways. Our antiquated notions are not capable of meeting these challenges.

One simple problem is that we lack context for our data. Most file systems merely assign to a file a name, location, owner, and security attributes. The most advanced can contain extended metadata, but this is rarely seen in practice since many applications cannot agree on how to use this data. Microsoft’s Office suite can store and share extended file attributes, for example, but these live inside the file rather than in the file system. The promise of expanded Office attributes is only realized in conjunction with a content management system like SharePoint which lies above the lowly file system.

What if the storage system could keep this data instead? What if it could logically group files according to project or client, mining keywords and authors, and maintaining revisions? These concepts are not new, having been implemented in content management systems for years, and certain elements appeared in file systems, like Apple’s HFS and VMS’ Files-11, for decades.

Cut Down the Tree

File metadata would allow advanced features, but truly taking advantage of them requires a more fundamental shift in the way applications access files. Rather than sticking to a traditional hierarchy of directories in a tree (which was, after all, simply a primitive metadata system), we should remove the tree altogether. Allow files to become data objects, identified by arbitrary attributes and managed according to an overarching policy.

This future vision is decidedly different from our current notion of storage, but is not so far off. Many organizations now rely on central data warehouses based on SQL-language relational databases. As many storage managers have grumbled, databases tend to ignore storage management concepts entirely, managing their own content independently.

But not all applications need a database back-end, so another initiative seeks to provide generic object storage for wider use. Called content-addressable storage or CAS, these devices have traditionally been used only for archival purposes, since that was their first market application. As vendors break free of proprietary interfaces in favor of open ones like XAM, CAS could transform storage itself by eliminating both file and block storage at once.

Similar concepts are already at work in the so-called Web 2.0 world. Non-traditional databases like Google BigTable, Amazon S3, and Hadoop allow massive scalability for object storage. API-sharing initiatives with many Web 2.0 companies can be seen as similar prototypical object storage frameworks. Any of these could be leveraged to provide a new world of data storage, and many are gaining traction even now.

Stephen’s Stance

Although traditional block storage is here to stay for disk drives, and tree-type file systems are likely to remain the foundation of operating system storage, new object-based concepts could change the world in fundamental ways. As applications become “web aware”, they also become object aware, increasing the likelihood of such a storage revolution. For the majority of applications, this new world would be a welcome one indeed.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | We Need a Storage Revolution
This post was categorized as Apple, Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

It’s that time again, when everyone who thinks they’re a pundit (that would be everyone with a blog or Twitter account) has to make predictions for the coming year. But predictions are perilous: Get it right and you look like a mere trend-watcher; get it wrong and you look like a fool. It’s such a hassle! So I’m doing something different this year: I’m going to make predictions for 2009 now that it’s over, and reflect on just how smart I am (not) to have made them. Or something.

What I Would Have Gotten Right

I definitely could have predicted a lot of what happened in 2009. I mean, these were slam dunks!

1. Twitter rocks the world – I wasn’t early to Twitter, but I spent the early part of 2009 evangelizing its benefits to companies and co-workers alike. Considering how common Twitter is today, it’s hard to believe how roundly criticized and misunderstood it was this time last year. Yet here we are, on the verge of 2010, and Twitter has seeped onto our business cards, presentation templates, and web sites. I might not have predicted how stable (!) Twitter got by the end of the year, though.
2. Apple’s Macs and iPhones rule – I switched to the iPhone and the Mac in 2007 and 2008, respectively, but it looks like I wasn’t much of an iconoclast after all: By November, half of the Tech Field Day delegates were using MacBooks, and the Windows and Blackberry holdouts have started vocally defending their operating system choice. Pretty much like Mac folks used to do way back in 2008.
3. The recession is a serious pain – Companies put the brakes on spending and hiring, many even shifting both into reverse in 2009. This came as no surprise to humans capable of thought. The impact on enterprise IT companies was similarly predictable: Although most were able to survive, the impact of 2009 will continue to be felt for years. I might have predicted it would be worse, though I’m glad to say I would have been wrong.
4. EMC, NetApp, HDS, HP, and IBM continue to quibble – Surprise: Big company bloggers spend way too much time criticizing the products and actions of each other and way to little time talking about the true value of their own products.

Non-IT slam-dunk predictions: Obama was reviled by the right; the war in Afghanistan continues; people do stupid stuff in the name of reality shows.

What I Probably Could Have Predicted

Although some details would likely have been missed, I think I would have seen these coming.

1. Cloud compute and storage hits the enterprise – I was a believer in the cloud this time last year, and I bet my future on it by taking a position at enterprise cloud storage provider, Nirvanix, in March. I would have predicted that enterprise buyers would be putting serious thought to buying cloud products, but the scope has surprised me. We’re talking enough petabytes that the non-cloud players felt compelled to strike back with the private cloud pitch. Awesome!
2. Sun and Data Domain were acquired – My money would have been on Dell, IBM, or HP as buyers for this pair, but EMC wouldn’t have been outside my guesses. Still, Oracle buying Sun and vocally committing to keep it going, SPARC and all, would never have come to mind. But I wouldn’t have guessed against it either, so I’ll give myself a point here!
3. Cisco and EMC buddy up – I’ve long thought an outright merger of these two was in the cards, but even the recession couldn’t make the financials work. A partnership would have been on the list, and Acadia came as no surprise to anyone.
4. Cloud outages and data loss – I definitely could have predicted that high-profile cloud services would fall over throughout the year, and that some would lose data. Not all are enterprise-grade, after all. But the outages at Google, Rackspace, and Amazon, and Microsoft’s Danger data loss, surprised me. Don’t those guys have their acts together?
5. IT conferences falter – I spoke at Interop in 2009, but it lacked the 20,000-strong crowd it once had. Storage Decisions and Storage Networking World managed to fill their halls, but the old-school IT conference has lost its luster. Although VMworld remains strong, attendance was definitely off.
6. FCoE and SSD are still starting – I’ve been lukewarm on Fibre Channel over Ethernet and Solid State Drives, but I’m a bit surprised that storage vendors didn’t push them harder in 2009. I might have guessed there would have been more customer uptake to match the buzz.
7. SMB storage is hot – There’s a hole in the storage market between $1,000 and $20,000, and companies like Drobo and Iomega are rushing in to fill it. Now that ESX has solid iSCSI support, I expect a world of innovation here. (Oops, that sounds kind of like a 2010 prediction!)

Also in the predictable category: Goldman Sachs and Bank of America thrived while others fell; Ford is the strongest of the remaining US automakers; Boeing finally got the 787 off the ground.

What I Never Would Have Guessed

I’m not perfect, even in retrospect. Some of the Tech news from 2009 was just completely off the wall.

1. Microsoft Bing: This time for sure! – Seriously, Microsoft should stick to in-house thinking instead of trying to copy its rivals. Yet somehow, miraculously, Bing appeared and did not suck. In fact, I’m hearing regular (non-techie) folks around town talking about using the search engine. I’ve even used it! Could they actually have a winner?
2. Windows 7 rocks – Really? Seriously? Could Microsoft have come up with a solid replacement for Windows XP?
3. Ship it! – It’s not even 2010, and enterprise storage buyers can go out and purchase NetApp’s OnTap 8, EMC’s FAST, EMC Atmos Compute, and unicorn tears. Well, maybe not unicorn tears.
4. Still no GDrive – Seemingly every company has a cloud storage platform, from Amazon to Rackspace, Nirvanix to EMC, so why not Google? Could GDrive join Duke Nukem Forever as the most famous vaporware of the decade?
5. The executive shuffle – Dave Donatelli was supposed to lead EMC, not HP. Alan Atkinson was supposed to launch another startup, not take over Xiotech. At least NetApp was gentle.
6. Mac OS X (still) lacks iSCSI and ZFS – Come on, Cupertino, what’s wrong with you guys? I’ve been hyping ZFS for years, and iSCSI is commonplace. Yet Snow Leopard is stingy with both. Makes me want to hiss like one of those blue folks in Avatar.
7. Gestalt IT is a success – On a personal note, Gestalt IT didn’t even exist this time last year, and now we have a successful IT infrastructure blog and social media event. Amazing!

Other total shockers: Everyone loves Michael Jackson again; digital Beatles tunes are available everywhere but iTunes; Obama’s Nobel Peace Prize arrives 10 years early.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | My 2009 IT Industry Predictions
This post was categorized as Apple, Computer History, Enterprise storage, Everything, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Snow Leopard will be here in September, but ZFS and iSCSI are AWOL

Apple wowed its fans and impressed its critics with a successful worldwide developer conference keynote yesterday. Along with much obvious focus on iPhone OS 3.0 and the new speedier iPhone 3GS, the company turned the spotlight on new Mac hardware and the next version of OS X, Snow Leopard. This is a lower-profile OS release than the last two (Tiger and Leopard), as indicated by the lower price ($29) and derivative name. But although there are many few features under the hood, Snow Leopard does little to move the ball forward from a storage perspective. ZFS and iSCSI, two long-rumored features, are notably absent.

Apple first teased both ZFS and iSCSI support back in the Tiger days, and these features were rumored for Leopard but absent on launch. Leopard Server had included read-only ZFS support, but even this level is support is reportedly absent in Snow Leopard. An iSCSI initiator was present in early Leopard builds but never showed up again, even in Snow Leopard. Still, the spread of the iSCSI protocol in the SMB market that has taken to Apple’s server products as well as Microsoft’s successful iSCSI initiator software suggested that an iSCSI initiator might become a Snow Leopard feature, at least in the server version.

Apple’s reduced focus on ZFS is understandable. Although it offers some compelling advantages, especially in server environments, it would bring little to the laptop/desktop market. In fact, ZFS is not really all that useful on the typical end-user Mac, with a single hard disk drive and a raft of removables attached with USB or FireWire. On the other hand, iSCSI support would have been increasingly welcome even in the consumer world as products like Drobo and Iomega add iSCSI as an option. The lack of a built-in initiator marginalizes the Mac in the emerging Ethernet SAN world, leaving vendors like Drobo to go it alone.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Snow Leopard Is Stingy With The Storage Love
This post was categorized as Apple, Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Snow Leopard is coming - will iSCSI finally tag along?

ZFS wasn’t the only AWOL storage technology in Apple’s OS X 10.5 – early builds of Leopard included a built-in iSCSI initiator. When the operating system was finally released in October of 2007, both ZFS and iSCSI were quietly dropped, making room for 300 other features Apple felt were more prime-time-ready.

With the next major OS X release, 10.6 “Snow Leopard”, just around the corner, many have wondered if iSCSI will join the reanimated ZFS support. Sadly, there has been no word from Apple on the fate of their iSCSI initiator. But silence does not necessarily tell a tale at the always-circumspect company, so hope springs eternal for the iSCSI faithful. If the rousing success of Microsoft’s iSCSI initiator is any indication, Apple is seriously missing out!

In the mean time, those seeking iSCSI connectivity within Mac OS X do have third-party options to consider:

• Studio Network Solutions offers a free (as in beer) iSCSI initiator for OS X Tiger and Leopard. The globalSAN iSCSI Initiator is the client-side portion of their SANmp storage sharing solution, and purports to support most of the required protocols. In practice, however, the company cannot support other targets than their own. globalSAN failed to mount any LUNs from my Windows Storage Server iSCSI test rig, and reports claim mixed results from EqualLogic and EMC SANs besides.
• ATTO Technology sells a commercial iSCSI initiator for OS X, Xtend SAN, and reports here are very favorable. Although the $195 price puts it out of reach for home users, corporations wishing to connect Leopard systems to an iSCSI SAN should look no further.
• Another commercial option is SmallTree Software’s abcSAN. Listed at $150, abcSAN lists EqualLogic among its functional targets, but the standalone initiator is nowhere to be found on SmallTree’s web site. They also have an ATA-over-Ethernet (AoE) solution for the true nerds in the audience!
• Ardis may also sell an iSCSI initiator for OS X.

All in all, iSCSI on OS X is in a sorry state of affairs. If only Apple had released the initiator seen in early Leopard builds! Let’s all hope Snow Leopard brings iSCSI to the party at last!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Will Snow Leopard Finally Bring iSCSI To The Mac?
This post was categorized as Apple. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Fram approaching in front of iceberg upernavik, CC-by-SA copyright Kim Hansen

Solid state (NAND flash) storage is all the rage right now, but there are many lingering questions regarding its true performance, reliability, and cost. But no question is more important in determining its ultimate usefulness than that of location: Where should flash storage be placed to maximize return on investment?

Storage companies have argued that flash disks can be used most effectively in external storage devices, arguing that it’s simpler to just leverage existing storage technologies. Server companies have tended to prefer to place it inside the server, asking why, if flash disks are capable of massive random I/O performance and extremely low latency, one would put them at the other end of a Fibre Channel or iSCSI connection, which introduces latency and tends to combine I/O operations?

The Case For Servers

The argument for placing flash in (or very close to) servers boils down to two key contentions:

1. Distance = latency, so moving quick flash devices away from I/O-hungry CPUs erodes their effectiveness
2. Granularity (or lack thereof) is the core problem facing storage management, so moving flash (and other types of storage) closer to the omniscient application is likely to bring greater effectiveness

The server folks are relying on a technical argument – that placing high-speed cache where it could theoretically do the most good is the right decision. And they are right, in a perfect world: A flash-aware application talking to a low-latency flash device over PCI ought to really fly!

There is some disagreement in the server-side argument as well: Is flash a “fake disk” or a new level of caching between RAM and storage? It seems that the pitch leans toward the latter, even when the SSD appears as a disk drive. This is what Fusion-IO is pitching: They skip old-school disk connections like SATA and SAS altogether, placing their storage on PCI Express and asking hardware and software vendors to integrate it as best they can. Consider Sun’s flash integration for ZFS, for example. Note, by the way, that Intel’s Turbo Memory products also offer PCIe flash, despite what you might be hearing.

Of course, we can just use a flash drive in place of an internal hard drive. Just about everyone makes something like this now, and they work pretty well in some cases. Then there are hybrid drives, which have gone nowhere so far.

But will this work? We need operating systems and applications that can make use of this local flash, and that has been a problem. Intel’s flash-on-the-motherboard idea never caught on, even as Vista included ReadyBoost, because the truth is that operating systems, file systems, or applications must be re-engineered to really make use of flash in a server. That’s happening, but slowly.

The Case for Arrays

Then we turn to the other end of the storage pipe. EMC put flash in the DMX in January, and Compellent is doing it as we speak. IBM went wild with a bunch of Fusion-IO drives and an SVC over the summer, too. All this proves that flash works in storage arrays!

Why? Simply because modern storage arrays are already engineered to make good use of disk drive capabilities. This is a “what works” strategy – even though it doesn’t sound as nice in theory, the array doesn’t need lots of re-engineering to see some benefit from flash. And post-RAID virtualized systems like that Compellent can really make hay with a few super-speed flash drives, since they can move hot blocks to flash dynamically.

Sure, there is latency between the CPU and the flash drive, but storage arrays are really computers in their own right. So they can derive the same benefit from flash that a server could, and they can share that benefit to connected servers rather than leaving it locked up.

Why Not Everywhere?

How about we end the debate. Flash works great in the server, and it works great in the array. Why not just put it anywhere it makes sense in your particular environment? Have an operating system, application, or file system that can make use of server-side flash? Go buy a Fusion-IO card! Have a virtualized enterprise storage array? Get some SSD there, too. And remember that it’s not all about NAND flash – RAM-based solid state storage from companies like Texas Memory Systems, Gear6, and Violin are even faster!

But remember one thing: This stuff is still very very expensive, so you have to really need the performance to make a case for flash.

Image by Kim Hansen, GFDL or CC-BY-SA (source)

© sfoskett for Stephen Foskett, Pack Rat, 2008. | SSD: So Close and Yet So Far
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Let’s take a second to review: EMC became the first modern storage vendor to include NAND flash-based solid state drives as a standard tier of storage in an enterprise storage product when they announced flash in the Symmetrix DMX in January. Although every other vendor has made “me too” comments since then, enterprise flash remains pretty rare. Could Compellent really be the second major vendor to actually do something, coming along 10 months later?

More than a year ago, I rhetorically asked where the enterprise solid state drives were. In that post, and others that followed, I suggested that SSD wouldn’t really “work” as a mainstream tier unless a storage array was smart enough to dynamically allocate content to this “tier-0″ in a granular fashion. In other words, adding a big lump of flash to a static storage array and trying to manually allocate it on a LUN-by-LUN basis to hot applications is not likely to meet anyone’s cost/benefit sniff test!

But if a post-RAID storage system was smart, it could really make use of the technology, and that’s what makes Compellent’s announcement so interesting. They dynamically move blocks (rather than the much-bigger LUNs) around, and could thus make a smaller amount of flash go a lot further. Add a few flash drives and let the system tune itself! This is a big differentiator, folks!

Of course, this is not just Compellent’s advantage. Any fully-virtualized system could do the same, and we’ve heard such talk from folks as diverse as HP (I’d love to see it in both EVA and LeftHand), IBM (for real in SVC, not the science experiment), Sun (combined with ZFS), Dell/EqualLogic, and I’d love to hear it from 3PAR. Bring it on, folks! Listen to Greg!  Let’s get this technology integrated, tested, released, and in the field!

Update: Compellent probably won’t ship their SSDs in volume ’till Q1. But Chris Evans seems to agree with me 100%, and Dell is talking SSD (but no promises yet).

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Compellent Does Enterprise SSD Right
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Spun Down

Although ZFS’ universal storage pool with non-RAID is a great concept, it stands in the way of at least one (sometimes) desirable storage technique: disk spin-down. Put simply, since every disk contains metadata, all disks must always be spinning. This issue is by no means a ZFS-only problem, though – certain vendors tout the (laughable) greenness of their storage systems, while hoping that the average user won’t notice the truth: That a disk simply cannot spin down while any part of it is in use. This means that tacking spin-down onto a regular storage array is like painting it a different color: There is no benefit whatsoever to the average user. Sure, a few non-provisioned drives might spin down, but what are you doing buying a lot of non-provisioned drives anyway?

The solution has always been right in front of everyone: Develop a new type of non-RAID with enough intelligence to allow drives to spin down when not used. This is what COPAN Systems did with their MAID technology: Invent an entirely new storage array, with integrated data protection and management techniques that allow alive but not active drives to spin down. Spin-down is not MAID any more than a bicycle is a Ducati.

Let’s make one thing clear: It’s really hard to reduce the power demands of storage devices. Disks guzzle watts like few other data center devices, and enterprise storage uses lots of disks. Lots of vendors are looking to hop onto the green storage bandwagon, and they all seem to realize that bringing some intelligence to power management by enabling spin-down is an open door. But it’s awfully hard to maintain performance and data protection when disks are spinning up and down all the time.

One element of the greenByte story is the way in which they have tweaked ZFS to allow disks to spin down. They limit the metadata updates to just a few disks, so the others can be idled when no access to them is made. The company suggests scheduling this for off hours to minimize latency as drives are brought back online, an approach that is less than optimal from an energy perspective but demonstrates that they understand just how difficult this problem is to crack. The core is there, however: They have integrated the data protection and storage management elements to enable spin-down to be practical.

Compressed

Another major storage industry theme of the last few years is deduplication of data. An advanced (or devolved, depending on your perspective) form of compression, deduplication allows a storage array to store duplicate data more efficiently, reducing the amount of capacity required for some applications. Data Domain is top-of-mind in this space, but just about everyone now offers some form of deduplication technology.

One major roadblock on the way to deduplication (or compression) nirvana is performance. Simply put, it’s really really hard to process data on the fly without affecting performance, especially as data scales up to the multi-terabyte range or as systems scale out to include multiple devices. One approach to tackling this issue is post-processing dedupe, which accepts incoming data in the normal way but goes back and processes it later to remove duplicates. This is the method NetApp uses, and they have leveraged it to become the first vendor to support deduplication of production applications.

Predictably, deduplication is another technology integrated into greenBytes’ “ZFS+” technology. They claim that they can handle inline compression at wire speed, and also claim deduplication inline. It’s not yet clear exactly what the difference between compression and deduplication is to the company, or just what kind of performance their inline technology will yield, but it’s certainly nice to see this tech integrated with ZFS!

Thin is In (the House!)

greenBytes gets closer to enterprise storage bingo by adding thin provisioning to the mix. Actually, as the company’s CTO was quick to point out, they had to offer virtual or thin provisioning to enable the rest of the system to function. When your storage is sliced and diced by their Cypress array, the only way to present storage is with a wink and a promise of capacity to spare. Thankfully this is not the core of their pitch, however.

The company also promises snapshots and CDP replication, all leveraging ZFS at the core. All they need to add is tier-0 solid state storage to get five chips in a row without even using the free space! Although greenBytes is using Sun’s Thumper chassis currently for their Cypress array, their core technology is the ZFS+ software, and I expect we might see this mixed quite differently in the future. This is a software company, not an array vendor.

All considered, greenBytes has thoroughly broken the link between physical and logical storage, and I applaud them for it. This is exactly the kind of storage revolution the industry needs right now.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | greenBytes Embraces and Extends ZFS
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.