• Skip to main content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Apple / Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)

Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)

August 4, 2011 By Stephen 15 Comments

Apple fan boys won’t like to hear this, but Mac OS X was majorly deficient in an area near and dear to my heart. You see, unlike every other modern operating system, Mac OS X lacked a volume manager. This wouldn’t seem like a big deal to the average user, but it held back the operating system in many ways. A volume manager brings storage virtualization to an operating system, allowing storage capacity efficiently to be managed and manipulated. But all this has changed in Mac OS X 10.7 “Lion” with CoreStorage.

You probably also want to read Key Storage Features in Mac OS X 10.7 “Lion” and Three Key Storage Features Missing in Mac OS X “Lion”

Apple’s Storage Story: A Mixed Bag

Apple’s file system, HFS+, is quite long in the tooth, dating back even before Mac OS X! Introduced in 1998’s Mac OS 8.1, HFS+ was extension of the original hierarchical file system (HFS) used by classic Mac operating systems back to System 2.1. But HFS+ has been extended and updated dramatically since those days, including the introduction of journaling in Mac OS 10.2.2, Unicode in 10.3, access control list-based file security in 10.4, and compression in 10.6.

The storage technology that underlies HFS+ has also improved dramatically over the years. Apple adopted EFI firmware and the GUID partition table (GPT) much more rapidly than the PC world. This allows Mac OS to scale to larger storage capacities than Windows systems, and has made the adoption of 4K sectors (so-called “advanced format”) much easier.  In fact, Apple is well ahead of the Windows world in these areas even as it lags with an outdated filesystem and no volume manager.

Mac OS never had much in terms of storage flexibility. Sure, AppleRAID brought some flexibility and redundancy to the server world, but the average Mac user never delved much into this technology. While Apple was asleep at the wheel, every other operating system vendor introduced a flexible volume manager. Even Microsoft, who is often criticized for slow innovation, is way ahead of Apple when it comes to data storage.

What is a volume manager? Volume managers add flexibility to operating system allocation of storage, acting as a virtualization layer between disks and file systems. This is important, since modern computer systems need to maintain a consistent filesystem image even as storage devices change. Volume managers like Symantec’s Veritas Volume Manager and the OSF LVM allow many operating systems this sort of flexible storage management. But Apple never had a volume manager of their own until the introduction of CoreStorage in Mac OS X “Lion”. Read more at Volume Management: Virtualizing Host Storage.

Introducing CoreStorage

All this has changed in Mac OS X 10.7 (“Lion”), however. Although Mac OS never did get ZFS, as was planned in 10.5, the storage enhancements found in 10.7 are getting awfully close in terms of features. Along with integrated file versions and full disk encryption comes a new technology called CoreStorage, including a full logical volume manager.

CoreStorage became necessary due to the inclusion of full volume encryption in Lion. Previously, Apple’s FileVault encryption technology relied on regular files to contain an encrypted filesystem. This worked fine until the need became apparent to encrypt an entire volume, whether it is the operating system disk or a removable drive. Although file-based technology can work in these scenarios, it is much less than ideal and also less than what competing operating systems offer.

But enabling full volume encryption requires an abstraction layer that Mac OS X simply did not have. Apple appears to have taken the long route, engineering a complete volume manager rather than hacking together some sort of kludgy mechanism. Although it is only used for FileVault 2 at the moment, CoreStorage appears to be a full volume manager that could, in the future, boast advanced features like mirroring, snapshots, and online storage migration.

CoreStorage Data Structures

CoreStorage implements a full complement of data structures but uses them in a very simplistic manner currently.

Apple did not simply copy existing volume manager semantics, and it does not appear that CoreStorage is an OEM version of someone else’s product. Rather, there are some intriguing new innovations present in CoreStorage that point to exciting possibilities in the future.

Like most volume managers, the basic building block of CoreStorage is the “logical volume group” (LVG). Illustrated in blue and purple above, each LVG takes storage from a “physical volume” (PV) and presents it to the operating system as a “logical volume” (LV). In this way, storage from a physical disk is abstracted before used by the operating system, allowing a great deal of flexibility. This is somewhat analogous to a hypervisor presenting a virtual machine interface.

Apple goes a step further than most volume managers, however, with the introduction of a new concept, the “logical volume family” (LVF). The LVF specifies properties that will be inherited by logical volumes that it contains. Currently, the only property specified by an LVF is FileVault encryption, but one can imagine that performance characteristics or redundancy could also be specified in this manner.

CoreStorage Today and Tomorrow

Today, when full volume FileVault encryption is used, Mac OS X automatically converts the desired volume into a CoreStorage volume, and encapsulating the partition as a PV, importing it into an LVG, and setting up an LVF and LV to contain the new filesystem. Only one of each element is created in this way, and all capacity is allocated at creation time. So there isn’t much use for the entire stack right now.

Using diskutil from the command line, one can create multiple logical volume families and logical volumes.

The Disk Utility GUI is fairly limited as well. Volumes can be converted to use FileVault 2, but all data is lost in this process. Only boot volumes can be corrected nondestructively through the GUI, and this happens in the Security & Privacy preference pane rather than Disk Utility. When boot volumes are encrypted, Apple gives users the option to “escrow” the decryption key online using their MobileMe or iCloud account, but this is not allowed for other volumes.

Currently, whenever a volume is converted to using FileVault, it is also automatically converted to CoreStorage behind the scenes. And if encryption is turned off in the GUI, it is converted back (“reverted”) to whatever it was before. But it’s possible to convert a volume on the command line without encryption, and to remove encryption without reverting. It’s also possible to manually manipulate CoreStorage in other ways.

In the future, it should be possible to manipulate storage much more fully.

The fact that Apple included a full volume manager of their own making indicates a bright future for CoreStorage. Already, it is possible to use the command line utility, diskutil to nondestructively convert and encrypt volumes. It is also possible to manipulate volume groups, logical volume families (indirectly), and logical volumes to a limited extent using diskutil.  One hopes that future versions of Mac OS X will include additional functionality, including the ability to manipulate physical volumes.

Stephen’s Stance

Either Apple has big plans for CoreStorage, or some Apple software engineer got way ahead of himself and designed a complete logical volume manager just to house FileVault data! I think the former is likely, since we know that Apple wanted to include ZFS, a flexible combination volume manager and file system, in Mac OS X 10.5 but was unable to due to licensing concerns. Now two versions later, Lion includes all of the basic technology needed to effectively manage storage volumes. It is likely that the GUI simply lags behind this core technology, and we will see additional functionality added in later operating system revisions.

Regardless of intent, Mac OS X now includes a fairly full storage stack, from EFI and GPT through CoreStorage and HFS+. No longer is Apple behind the times in terms of core storage (if you pardon the pun) features. Now if only we could get them to implement iSCSI!

You might also want to read these other posts...

  • Powering Rabbits: The Mean Well LRS-350-12 Power Supply
  • Electric Car Over the Internet: My Experience Buying…
  • Ranting and Raving About the 2018 iPad Pro
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Tortoise or Hare? Nvidia Jetson TK1

Filed Under: Apple, Computer History, Enterprise storage, Everything, Features, Personal, Terabyte home Tagged With: 10.7, Apple, CoreStorage, Lion, LVM, Mac OS X, Veritas Volume Manager, volume manager, ZFS

Primary Sidebar

If you can keep your head when all about you are losing theirs…
If you can wait and not be tired by waiting…
If you can think – and not make thoughts your aim…
If you can trust yourself when all men doubt you…
Yours is the Earth and everything that’s in it.

Rudyard Kipling

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

May 29, 2020

What You See and What You Get When You Follow Me

May 28, 2019

GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

April 6, 2019

Symbolic Links

    Featured Posts

    What More Could Alan Turing Have Accomplished?

    October 7, 2012

    Aerobie AeroPress Review: The Hacker Coffee Maker

    February 7, 2011

    Thoughts on the Modern Miracle of 3D Printing

    July 28, 2015

    Electric Car Over the Internet: My Experience Buying From Vroom

    November 28, 2020

    Scaling Storage At The Client

    November 25, 2013

    Introducing Rabbit: I Bought a Cloud!

    September 10, 2020

    FCoE vs. iSCSI – Making the Choice

    May 20, 2011

    Storage Changes in VMware vSphere 5

    July 16, 2011

    What is VMware VASA? Not Much (Yet)

    November 11, 2011

    The Best Mac OS X Terminal Font: Glass TTY VT220

    October 6, 2015

    Copyright © 2021 · Log in