March 21, 2014

Undocumented CoreStorage Commands

CoreStorage Series

The commands are there to make CoreStorage do some cool things. But they don't quite work and aren't quite public...

Yesterday I noted that Apple included a full logical volume manager in Mac OS X 10.7 “Lion” without so much as a word. Today I am pleased to say that CoreStorage is much more functional than I had guessed, including a number of undocumented but seemingly functional commands for on-the-fly resizing of logical volumes as well as manipulation of physical volumes. Read on for the details, but please proceed at your own risk with these new commands!

You should probably read Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!) first!

CoreStorage Command Overview

The diskutil command is the core command line interface for CoreStorage, along with regular disk partitioning, AppleRAID, and other disk activities.

All CoreStorage functions use the “coreStorage” adverb, which can be abbreviated “cs”. For example, to show status of all CoreStorage volumes, one may type either:

diskutil coreStorage list

or

diskutil cs list

Typing just “diskutil cs” will show a list of nine supported “verbs”, but there are six other undocumented commands as well. Perhaps these are not fully functional (though all worked fine in my testing), or perhaps Apple simply wasn’t ready to expose them for end users to use.

I have grouped all 15 CoreStorage verbs by functional area, and note here which are officially listed and which are undocumented.

Informational Commands

  • list – Show all CoreStorage volumes
  • info[rmation] – Get detailed information about a physical volume

Conversion Commands

  • convert – Convert a volume into a CoreStorage volume
  • revert – Revert a CoreStorage volume to its native type

Logical Volume Group Commands

  • create – Create a new CoreStorage logical volume group
  • delete – Delete a CoreStorage logical volume group

Physical Volume (Disk) Commands

  • resizeDisk (undocumented) – Resize a physical volume
  • removeDisk (undocumented) – Remove a physical volume from a logical volume group
  • addDisk (undocumented) - Add a new physical volume to a logical volume group

Logical Volume Commands

  • createVolume – Create a new logical volume
  • unlockVolume – Mount an existing encrypted logical volume
  • changeVolumePassphrase – Change the encryption password for a logical volume
  • deleteVolume (undocumented) – Delete a logical volume and all of its contents
  • resizeVolume (undocumented) – Grow or shrink a logical volume (non-destructive)
  • resizeStack (undocumented) – Grow or shrink a logical volume as well as the logical volume group and physical volume that supports it

CoreStorage Informational Commands

Two commands are for information about volumes and disks. Note that these only return information about CoreStorage volumes: Use the bare “diskutil list” and “diskutil info” commands for non-encapsulated storage.

list

Usage:  diskutil coreStorage list
        diskutil coreStorage list -plist
        diskutil coreStorage list UUID
List all current CoreStorage objects in a tree-like view.

info[rmation]

The verbs “info” or “information” get CoreStorage information by UUID.

CoreStorage Conversion Commands

These commands are used to convert a drive or volume to or from CoreStorage.

convert

Usage:  diskutil coreStorage convert
        MountPoint|DiskIdentifier|DeviceNode
        [-stdinpassphrase | -passphrase [passphrase]]
Convert a regular JHFS+ partition into a CoreStorage logical volume.
The file system must be mounted and resizable (i.e. Journaled HFS+).
Ownership of the affected disk is required.

Note that CoreStorage only functions on drives that meet the following criteria:

  1. Partition type is GPT
  2. Filesystem type is Journaled HFS+ (JHFS+)

Conversion from the command line is non-destructive (unlike using Disk Utility) and is the one shot you have to encrypt a volume. If you convert it without the passphrase, it will be placed in a LVF with no encryption and there is no command to encrypt it later!

revert

Usage:  diskutil coreStorage revert
        MountPoint|DiskIdentifier|DeviceNode|lvUUID
        [-stdinpassphrase | -passphrase [passphrase] | -recoverykeychain file]
Convert a CoreStorage logical volume back to its native type.
The file system must be mounted and resizable (i.e. Journaled HFS+).
Ownership of the affected disk and a passphrase (if encrypted) is required.

Reverting a CoreStorage volume completely returns it to its original state, including restoring the partition type and removing encryption. And it’s non-destructive. Slick!

CoreStorage Logical Volume Group Commands

These commands manipulate logical volume groups (LVGs) that have previously been created. Most use the “LVG Name”, which you can discover using the “diskutil cs list” command.

create

Usage:  diskutil coreStorage create lvgName
        MountPoint|DiskIdentifier|DeviceNode ...
Create a CoreStorage logical volume group from one more more disks.
The specified disks will become the initial set of PVs.
All existing data on the drive will be lost.
Ownership of the affected disk is required.
Example: diskutil coreStorage create MyLVG disk1

Note that this command is destructive. Say goodbye to your data or use the “cs convert” command! It automatically creates a PV for you.

delete

Usage:  diskutil coreStorage delete lvgUUID
Delete a CoreStorage logical volume group. All logical volumes will be removed.
Ownership of the affected disk is required.

This is destructive as well. Your volume will be returned to a usable state, but your data will be lost. Use “convert” and “revert” instead if possible!

CoreStorage Physical Volume (Disk) Commands

All physical volume commands are undocumented in Lion 10.7. Proceed with caution! But these are pretty cool, since they allow a LVG to span multiple physical disks!
PV commands use the word, “Disk”, which tells you a lot about what they are and what they do.
Note that the “pvUUID” and “lvgUUID” can be discovered in “diskutil cs list” and refer to the PV and LVG, respectively.

resizeDisk (undocumented)

Usage:  diskutil coreStorage resizeDisk pvUUID size
        [part1Format part1Name part1Size part2Format part2Name part2Size
         part3Format part3Name part3Size ...]
Resize a physical volume, which is one of one or more disks that provide storage
to a logical volume group. The logical volume group will have less or more
available space after this operation, if it was a shrink or grow, respectively.
If this is a shrink operation, you can optionally request that new partitions
be created in the newly-formed free space gap.
Example: diskutil coreStorage resizeDisk
         11111111-2222-3333-4444-555555555555 10g JHFS+ New 1g

This crazy-complicated command allows you to resize (both grow and shrink) active partitions non-destructively.

removeDisk (undocumented)

Usage:  diskutil coreStorage removeDisk pvUUID
Remove a physical volume from its CoreStorage logical volume group.
Ownership of the affected disks is required.
Example: diskutil coreStorage removeDisk 11111111-2222-3333-4444-555555555555

It is unknown what will happen if a used PV is removed from an LVG. Using a combination of addDisk and removeDisk should non-destructively move data from one physical disk to another, but I couldn’t get any of this to work.

addDisk (undocumented)

Usage:  diskutil coreStorage addDisk lvgUUID NewMemberDeviceName
Add a new physical volume to a CoreStorage logical volume group.
Ownership of the affected disks is required.
Example: diskutil coreStorage addDisk
         11111111-2222-3333-4444-555555555555 disk4

This should add additional physical capacity to an existing LVG, but I couldn’t get it to work. It always came back with “Error adding disk to CoreStorage Logical Volume Group: Invalid request (-69886)”

CoreStorage Logical Volume Commands

The logical volume commands are much more functional and friendly. Right now, the only way to really use these is to create a larger-than-needed LV and then shrink it and use the resulting space for new volumes, since you cannot yet fully create a LVG from scratch.

createVolume

This creates a new LV (and LVF) within an existing LVG, using space cleared by a previous resizeVolume or deleteVolume command. It appears to always create a new LVF rather than placing an LV within an existing one. And there are no LVF manipulation commands right now.

Usage:  diskutil coreStorage createVolume lvgUUID type name size
        [-stdinpassphrase | -passphrase [passphrase]]
Add a new logical volume to a CoreStorage logical volume group.

Type is the file system to initialize on the new logical volume. Valid types
are Journaled HFS+ or Case-sensitive Journaled HFS+ or their aliases.

Size is the amount of space to allocate from the parent logical volume group.
Valid sizes are floating-point numbers with a suffix of B(ytes), S(512-byte-
blocks), K(ilobytes), M(egabytes), G(igabytes), T(erabytes), P(etabytes),
or (%) a percentage of the current size of the logical volume group.

Example: diskutil coreStorage createVolume
         11111111-2222-3333-4444-555555555555 jhfs+ myLV 10g

unlockVolume

One of the seemingly-handier commands that isn’t really all that useful. It’s rare that you’ll want to do this alone. Use hdiutil to mount a volume instead.

Usage:  diskutil coreStorage unlockVolume lvUUID
        [-stdinpassphrase | -passphrase passphrase | -recoverykeychain file]
Unlock a logical volume that is encrypted and currently locked. You must
specify the logical volume by its CoreStorage UUID, because if it is locked
it is not online. A passphrase is mandatory: you must either supply it
interactively or with one of the above parameters.
Example: diskutil coreStorage unlockVolume 11111111-2222-3333-4444-555555555555

changeVolumePassphrase

A very handy command allowing you to change the passphrase of an existing VileVault 2 encrypted volume. This is probably the one verb that will be somewhat frequently used by the average user!

Usage:  diskutil coreStorage changeVolumePassphrase|passwd lvUUID
        [-recoverykeychain file] | [-oldpassphrase old]
        [-newpassphrase new] | [-stdinpassphrase]
Change an encrypted logical volume's password. Beyond the CoreStorage UUID,
you will be prompted interactively for parameters that you do not specify.
Parameters must be given in the above order.
Example: diskutil coreStorage changeVolumePassphrase
         11111111-2222-3333-4444-555555555555

resizeVolume (undocumented)

This appears to work fine. You can resize a volume (given its UUID) using this command, and you won’t even lose your data!

Usage:  diskutil coreStorage resizeVolume lvUUID size
Resize a logical volume, which is one of one or more disks that consume storage
out of a logical volume group. The logical volume group will have more or less
available space after this operation, if it was a shrink or grow, respectively.
Example: diskutil coreStorage resizeVolume
         11111111-2222-3333-4444-555555555555 10g

deleteVolume (undocumented)

This also works fine. Create a volume and you can delete it, as long as you get the right UUID from “diskutil cs list”.

Usage:  diskutil coreStorage deleteVolume lvUUID
Delete a logical volume.
Example: diskutil coreStorage deleteVolume 11111111-2222-3333-4444-555555555555

resizeStack (undocumented)

Probably the coolest undocumented command, resizeStack takes a complete 1:1:1:1 CoreStorage stack (PV:LVG:LVF:LV) and resizes everything non-destructively. You’re left with a fully-operational but smaller volume and partition. Nifty!

Usage:  diskutil coreStorage resizeStack lvUUID size
        [part1Format part1Name part1Size part2Format part2Name part2Size
         part3Format part3Name part3Size ...]
Resize both a logical volume and its underlying physical volume in a single
operation. The setup must be simple: Exactly one logical volume and one
related physical volume can, and must, exist.
If this is a shrink operation, you can optionally request that new partitions
be created in the newly-formed free space gap.
Example: diskutil coreStorage resizeStack
         11111111-2222-3333-4444-555555555555 10g JHFS+ New 1g

What’s Missing

These undocumented CoreStorage commands are eye-opening, since it shows that Apple really is working on a complete full-featured volume manager. But lots of functionality is lacking still:

  1. You can’t create a logical volume group (LVG) by hand, adding multiple disks. It seems like you could, but not enough works yet.
  2. There are no commands for manipulating logical volume families (LVFs), and they can’t even be specified when creating new logical volumes (LVs).
  3. The physical volume (PV or disk) commands don’t seem to work. I guess that’s why they’re undocumented!
  4. There are no advanced data protection features (mirror, snapshot, RAID, replicate)
  5. There is no ability to specify where or how an LV is created or to move an LV from one LVF, PV, or LVG to another.
All in all, this is a wonderful start for Apple, showing solid core technology that isn’t yet fleshed out enough to be useful.

Stephen’s Stance

Clearly, CoreStorage is much more than a simple stack to support FileVault 2 encryption. Apple no only built an entire volume manager but also outfitted it with a good set of commands for configuration. Too bad it still lacks so many key features, and that so many of these are unofficial and hidden.

I feel confident that Mac OS X CoreStorage will become much more functional in the future, with complete volume manipulation and migration tools built into the GUI. For now, though, I must remind readers that this is all unofficial and you must proceed with caution. Back up your data, and do not assume that just because cool commands like resizeStack exist they are 100% ready for use!

Note: The pre-formatted text above comes straight from the diskutil command and its “help” responses. I discovered these undocumented verbs through trial and error and the judicial use of the “strings” command!

  • Pingback: Create new partition on encrypted volume in OS X Lion

  • MacKai

    Am getting deep into OS X to solve the problem formatting a 4TB drive with encrypted HFS+ partition. MediaKit gives an error about Blocksize not dividable by 512 (but assumed 4K is dividable by 512!).

    I dont know hat LV, PV, LVG and myLV or myLVG mean. I tried to create a volume (“diskutil cs create volumename disk3″)and media kit gave same error back as apple’s disk utility. I had a

    Some examples of standard volume creation commands would be great.

    Total Size: 4.0 TB (4000443052032 Bytes) (exactly 7813365336 512-Byte-Blocks) Volume Free Space: 0 B (0 Bytes) (exactly 0 512-Byte-Blocks) Device Block Size: 4096 Bytes

  • Valery

    Just played around with CoreStorage, in Mountain Lion, trying to mimic Apple’s Fusion Drive with – for a start – two partitions on one physical drive. First converted one of the two partitions. Then tried to add the second partition to the LVG. No chance : CoreStorage replied with an error, claiming the LVG doesn’t support the addition of more drives. I guess the LVG must somehow tagged with a special property for it to effectively support the addition of various storage resources. I’d love to know how…

  • robwalch

    I just setup a Logical Volume Group with a Logical Volume backed by two Physical Volumes.

    First create a new LVG. This will reformat both disks. Make sure both disks are not converted to CoreStorage already. I couldn’t get “add” to work or “stack” disks converted to CoreStore first:

    diskutil coreStorage create CoreStore disk1 disk2

    Now you just need to create a volume and it will mount. I specified 100% to use as much space as was available. The new volume turned out to be about the same size as the larger of my two disks (assuming this is how Fusion drives work – you get the size of the largest drive, with the occasional access bonus of the faster drive).

    diskutil cs createVolume *lvgUUID* jhfs+ “CoreStore” 100%

    I haven’t used it long enough or with an SSD/HDD combo to say how well it works, but it looks promising so far!

  • Steve

    Cool stuff. I tried the addDisk command in Mountain Lion 10.8.2, and get a slightly different error: Error adding disk to CoreStorage Logical Volume Group: This Core Storage logical volume group does not allow adding Core Storage physical volumes (-69699).

    My guess is Apple are working on it…!

  • http://www.facebook.com/vze4rtzt David Schwartz

    addDisk and removeDisk work in 10.8.3. Be warned that removeDisk seems to make the LVG inoperable currently. addDisk just adds the storage to the pool. You have to manually fill the added storage with resizeVolume or createVolume.

  • André Somers

    Thank you! The undocumented resizeStack was a lifesaver for me!

  • NiklasL

    Is there a way via the corestorage manager disable the auto function of sending files to and from the Fusion Drive all the time? My USB audio interface on my new imac gets crackling noise when the Fusion Drive sends data to anf from the SSD. That could help lots of people. Have been talking to Apples senior advisors but they dont really accept the problem.

  • res1233

    You should probably get that computer fixed. That shouldn’t happen under normal circumstances.