You probably also want to read Key Storage Features in Mac OS X 10.7 “Lion” and Three Key Storage Features Missing in Mac OS X “Lion”

Apple’s Storage Story: A Mixed Bag

Apple’s file system, HFS+, is quite long in the tooth, dating back even before Mac OS X! Introduced in 1998′s Mac OS 8.1, HFS+ was extension of the original hierarchical file system (HFS) used by classic Mac operating systems back to System 2.1. But HFS+ has been extended and updated dramatically since those days, including the introduction of journaling in Mac OS 10.2.2, Unicode in 10.3, access control list-based file security in 10.4, and compression in 10.6.

The storage technology that underlies HFS+ has also improved dramatically over the years. Apple adopted EFI firmware and the GUID partition table (GPT) much more rapidly than the PC world. This allows Mac OS to scale to larger storage capacities than Windows systems, and has made the adoption of 4K sectors (so-called “advanced format”) much easier.  In fact, Apple is well ahead of the Windows world in these areas even as it lags with an outdated filesystem and no volume manager.

Mac OS never had much in terms of storage flexibility. Sure, AppleRAID brought some flexibility and redundancy to the server world, but the average Mac user never delved much into this technology. While Apple was asleep at the wheel, every other operating system vendor introduced a flexible volume manager. Even Microsoft, who is often criticized for slow innovation, is way ahead of Apple when it comes to data storage.

What is a volume manager? Volume managers add flexibility to operating system allocation of storage, acting as a virtualization layer between disks and file systems. This is important, since modern computer systems need to maintain a consistent filesystem image even as storage devices change. Volume managers like Symantec’s Veritas Volume Manager and the OSF LVM allow many operating systems this sort of flexible storage management. But Apple never had a volume manager of their own until the introduction of CoreStorage in Mac OS X “Lion”. Read more at Volume Management: Virtualizing Host Storage.

Introducing CoreStorage

All this has changed in Mac OS X 10.7 (“Lion”), however. Although Mac OS never did get ZFS, as was planned in 10.5, the storage enhancements found in 10.7 are getting awfully close in terms of features. Along with integrated file versions and full disk encryption comes a new technology called CoreStorage, including a full logical volume manager.

CoreStorage became necessary due to the inclusion of full volume encryption in Lion. Previously, Apple’s FileVault encryption technology relied on regular files to contain an encrypted filesystem. This worked fine until the need became apparent to encrypt an entire volume, whether it is the operating system disk or a removable drive. Although file-based technology can work in these scenarios, it is much less than ideal and also less than what competing operating systems offer.

But enabling full volume encryption requires an abstraction layer that Mac OS X simply did not have. Apple appears to have taken the long route, engineering a complete volume manager rather than hacking together some sort of kludgy mechanism. Although it is only used for FileVault 2 at the moment, CoreStorage appears to be a full volume manager that could, in the future, boast advanced features like mirroring, snapshots, and online storage migration.

CoreStorage Data Structures

CoreStorage implements a full complement of data structures but uses them in a very simplistic manner currently.

Apple did not simply copy existing volume manager semantics, and it does not appear that CoreStorage is an OEM version of someone else’s product. Rather, there are some intriguing new innovations present in CoreStorage that point to exciting possibilities in the future.

Like most volume managers, the basic building block of CoreStorage is the “logical volume group” (LVG). Illustrated in blue and purple above, each LVG takes storage from a “physical volume” (PV) and presents it to the operating system as a “logical volume” (LV). In this way, storage from a physical disk is abstracted before used by the operating system, allowing a great deal of flexibility. This is somewhat analogous to a hypervisor presenting a virtual machine interface.

Apple goes a step further than most volume managers, however, with the introduction of a new concept, the “logical volume family” (LVF). The LVF specifies properties that will be inherited by logical volumes that it contains. Currently, the only property specified by an LVF is FileVault encryption, but one can imagine that performance characteristics or redundancy could also be specified in this manner.

CoreStorage Today and Tomorrow

Today, when full volume FileVault encryption is used, Mac OS X automatically converts the desired volume into a CoreStorage volume, and encapsulating the partition as a PV, importing it into an LVG, and setting up an LVF and LV to contain the new filesystem. Only one of each element is created in this way, and all capacity is allocated at creation time. So there isn’t much use for the entire stack right now.

Using diskutil from the command line, one can create multiple logical volume families and logical volumes.

The Disk Utility GUI is fairly limited as well. Volumes can be converted to use FileVault 2, but all data is lost in this process. Only boot volumes can be corrected nondestructively through the GUI, and this happens in the Security & Privacy preference pane rather than Disk Utility. When boot volumes are encrypted, Apple gives users the option to “escrow” the decryption key online using their MobileMe or iCloud account, but this is not allowed for other volumes.

Currently, whenever a volume is converted to using FileVault, it is also automatically converted to CoreStorage behind the scenes. And if encryption is turned off in the GUI, it is converted back (“reverted”) to whatever it was before. But it’s possible to convert a volume on the command line without encryption, and to remove encryption without reverting. It’s also possible to manually manipulate CoreStorage in other ways.

In the future, it should be possible to manipulate storage much more fully.

The fact that Apple included a full volume manager of their own making indicates a bright future for CoreStorage. Already, it is possible to use the command line utility, diskutil to nondestructively convert and encrypt volumes. It is also possible to manipulate volume groups, logical volume families (indirectly), and logical volumes to a limited extent using diskutil.  One hopes that future versions of Mac OS X will include additional functionality, including the ability to manipulate physical volumes.

Stephen’s Stance

Either Apple has big plans for CoreStorage, or some Apple software engineer got way ahead of himself and designed a complete logical volume manager just to house FileVault data! I think the former is likely, since we know that Apple wanted to include ZFS, a flexible combination volume manager and file system, in Mac OS X 10.5 but was unable to due to licensing concerns. Now two versions later, Lion includes all of the basic technology needed to effectively manage storage volumes. It is likely that the GUI simply lags behind this core technology, and we will see additional functionality added in later operating system revisions.

Regardless of intent, Mac OS X now includes a fairly full storage stack, from EFI and GPT through CoreStorage and HFS+. No longer is Apple behind the times in terms of core storage (if you pardon the pun) features. Now if only we could get them to implement iSCSI!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)
This post was categorized as Apple, Computer History, Enterprise storage, Everything, Features, Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I’m kicking off with one of my favorite topics, logical volume management on servers. This is the longest-standing and most-successful face of storage virtualization, and the one I was first exposed to. Put simply, volume managers abstract block storage (LUNs, disks, partitions, what have you) into virtual “volumes” for use by the server. They look the same to the OS, and still need a filesystem, but are much more flexible, as we’ll see.

Volume managers are very common today – all modern OSes (except for that one from Apple!) have volume managers built in. Windows has the Logical Disk Manager, which I’m told was co-engineered (or something) with Veritas way back when and which I’ve covered in my Storage Magazine columns. Linux has an implementation of LVM, which I wrote something about way back when, and which has now not been supplanted by EVMS as had once been supposed. AIX it’s own twist on the original LVM, as does HP-UX. Solaris has the variously-named Solstice DiskSuite/Volume Manager which has evolved substantially in the 15 or so years I’ve been using it. And everyone has Symantec’s Veritas Volume Manager/Foundation Suite, which we in “the biz” view with considerable admiration and some skepticism, as is the case with all good front runners!

Folks mostly use volume managers for flexibility. It’s really quite amazing what you can do when your servers run them, enough that you often wonder how you got along without them!

• You can resize volumes (aka file systems or drives) on the fly (if your file system supports this as most modern ones do)
• You can protect data with RAID, even if your storage doesn’t support it (think bare disk drives)
• You can add storage capacity on the fly by concatenating new to old or (maybe) expanding existing stripes and RAID sets
• You can mirror volumes, create snapshots, and even replicate data to remote locations (this functionality varies by product, of course)
• One of the most powerful things (to me) was the ability to migrate live volumes from one storage device to another when making infrastructure changes

In short, volume management = server-based storage virtualization! So even if you were skeptical about the claims about storage virtualization, you might already be using it! Amazingly, a good volume manager can do anything a storage virtualization appliance or enterprise storage array can do. In fact, some virtualization appliances have more than a little volume management source code in them…

And the only cost for all this great stuff is the impact on your server’s CPU, memory, bus, access control, etc…

The chart below compares the major volume managers, and includes a little easter egg at the bottom… But we’ll cover that on another day.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Volume Management: Virtualizing Host Storage
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.