Yesterday I noted that Apple included a full logical volume manager in Mac OS X 10.7 “Lion” without so much as a word. Today I am pleased to say that CoreStorage is much more functional than I had guessed, including a number of undocumented but seemingly functional commands for on-the-fly resizing of logical volumes as well as manipulation of physical volumes. Read on for the details, but please proceed at your own risk with these new commands!
You should probably read Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!) first!
CoreStorage Command Overview
The diskutil command is the core command line interface for CoreStorage, along with regular disk partitioning, AppleRAID, and other disk activities.
All CoreStorage functions use the “coreStorage” adverb, which can be abbreviated “cs”. For example, to show status of all CoreStorage volumes, one may type either:
diskutil coreStorage list
or
diskutil cs list
Typing just “diskutil cs” will show a list of nine supported “verbs”, but there are six other undocumented commands as well. Perhaps these are not fully functional (though all worked fine in my testing), or perhaps Apple simply wasn’t ready to expose them for end users to use.
I have grouped all 15 CoreStorage verbs by functional area, and note here which are officially listed and which are undocumented.
Informational Commands
- list – Show all CoreStorage volumes
- info[rmation] – Get detailed information about a physical volume
Conversion Commands
- convert – Convert a volume into a CoreStorage volume
- revert – Revert a CoreStorage volume to its native type
Logical Volume Group Commands
- create – Create a new CoreStorage logical volume group
- delete – Delete a CoreStorage logical volume group
Physical Volume (Disk) Commands
- resizeDisk (undocumented) – Resize a physical volume
- removeDisk (undocumented) – Remove a physical volume from a logical volume group
- addDisk (undocumented) – Add a new physical volume to a logical volume group
Logical Volume Commands
- createVolume – Create a new logical volume
- unlockVolume – Mount an existing encrypted logical volume
- changeVolumePassphrase – Change the encryption password for a logical volume
- deleteVolume (undocumented) – Delete a logical volume and all of its contents
- resizeVolume (undocumented) – Grow or shrink a logical volume (non-destructive)
- resizeStack (undocumented) – Grow or shrink a logical volume as well as the logical volume group and physical volume that supports it
CoreStorage Informational Commands
Two commands are for information about volumes and disks. Note that these only return information about CoreStorage volumes: Use the bare “diskutil list” and “diskutil info” commands for non-encapsulated storage.
list
Usage: diskutil coreStorage list
diskutil coreStorage list -plist
diskutil coreStorage list UUID
List all current CoreStorage objects in a tree-like view.
info[rmation]
The verbs “info” or “information” get CoreStorage information by UUID.
CoreStorage Conversion Commands
These commands are used to convert a drive or volume to or from CoreStorage.
convert
Usage: diskutil coreStorage convert
MountPoint|DiskIdentifier|DeviceNode
[-stdinpassphrase | -passphrase [passphrase]]
Convert a regular JHFS+ partition into a CoreStorage logical volume.
The file system must be mounted and resizable (i.e. Journaled HFS+).
Ownership of the affected disk is required.
Note that CoreStorage only functions on drives that meet the following criteria:
- Partition type is GPT
- Filesystem type is Journaled HFS+ (JHFS+)
Conversion from the command line is non-destructive (unlike using Disk Utility) and is the one shot you have to encrypt a volume. If you convert it without the passphrase, it will be placed in a LVF with no encryption and there is no command to encrypt it later!
revert
Usage: diskutil coreStorage revert
MountPoint|DiskIdentifier|DeviceNode|lvUUID
[-stdinpassphrase | -passphrase [passphrase] | -recoverykeychain file]
Convert a CoreStorage logical volume back to its native type.
The file system must be mounted and resizable (i.e. Journaled HFS+).
Ownership of the affected disk and a passphrase (if encrypted) is required.
Reverting a CoreStorage volume completely returns it to its original state, including restoring the partition type and removing encryption. And it’s non-destructive. Slick!
CoreStorage Logical Volume Group Commands
These commands manipulate logical volume groups (LVGs) that have previously been created. Most use the “LVG Name”, which you can discover using the “diskutil cs list” command.
create
Usage: diskutil coreStorage create lvgName
MountPoint|DiskIdentifier|DeviceNode ...
Create a CoreStorage logical volume group from one more more disks.
The specified disks will become the initial set of PVs.
All existing data on the drive will be lost.
Ownership of the affected disk is required.
Example: diskutil coreStorage create MyLVG disk1
Note that this command is destructive. Say goodbye to your data or use the “cs convert” command! It automatically creates a PV for you.
delete
Usage: diskutil coreStorage delete lvgUUID Delete a CoreStorage logical volume group. All logical volumes will be removed. Ownership of the affected disk is required.
This is destructive as well. Your volume will be returned to a usable state, but your data will be lost. Use “convert” and “revert” instead if possible!
CoreStorage Physical Volume (Disk) Commands
All physical volume commands are undocumented in Lion 10.7. Proceed with caution! But these are pretty cool, since they allow a LVG to span multiple physical disks!
PV commands use the word, “Disk”, which tells you a lot about what they are and what they do.
Note that the “pvUUID” and “lvgUUID” can be discovered in “diskutil cs list” and refer to the PV and LVG, respectively.
resizeDisk (undocumented)
Usage: diskutil coreStorage resizeDisk pvUUID size
[part1Format part1Name part1Size part2Format part2Name part2Size
part3Format part3Name part3Size ...]
Resize a physical volume, which is one of one or more disks that provide storage
to a logical volume group. The logical volume group will have less or more
available space after this operation, if it was a shrink or grow, respectively.
If this is a shrink operation, you can optionally request that new partitions
be created in the newly-formed free space gap.
Example: diskutil coreStorage resizeDisk
11111111-2222-3333-4444-555555555555 10g JHFS+ New 1g
This crazy-complicated command allows you to resize (both grow and shrink) active partitions non-destructively.
removeDisk (undocumented)
Usage: diskutil coreStorage removeDisk pvUUID Remove a physical volume from its CoreStorage logical volume group. Ownership of the affected disks is required. Example: diskutil coreStorage removeDisk 11111111-2222-3333-4444-555555555555
It is unknown what will happen if a used PV is removed from an LVG. Using a combination of addDisk and removeDisk should non-destructively move data from one physical disk to another, but I couldn’t get any of this to work.
addDisk (undocumented)
Usage: diskutil coreStorage addDisk lvgUUID NewMemberDeviceName
Add a new physical volume to a CoreStorage logical volume group.
Ownership of the affected disks is required.
Example: diskutil coreStorage addDisk
11111111-2222-3333-4444-555555555555 disk4
This should add additional physical capacity to an existing LVG, but I couldn’t get it to work. It always came back with “Error adding disk to CoreStorage Logical Volume Group: Invalid request (-69886)”
CoreStorage Logical Volume Commands
The logical volume commands are much more functional and friendly. Right now, the only way to really use these is to create a larger-than-needed LV and then shrink it and use the resulting space for new volumes, since you cannot yet fully create a LVG from scratch.
createVolume
This creates a new LV (and LVF) within an existing LVG, using space cleared by a previous resizeVolume or deleteVolume command. It appears to always create a new LVF rather than placing an LV within an existing one. And there are no LVF manipulation commands right now.
Usage: diskutil coreStorage createVolume lvgUUID type name size
[-stdinpassphrase | -passphrase [passphrase]]
Add a new logical volume to a CoreStorage logical volume group.
Type is the file system to initialize on the new logical volume. Valid types
are Journaled HFS+ or Case-sensitive Journaled HFS+ or their aliases.
Size is the amount of space to allocate from the parent logical volume group.
Valid sizes are floating-point numbers with a suffix of B(ytes), S(512-byte-
blocks), K(ilobytes), M(egabytes), G(igabytes), T(erabytes), P(etabytes),
or (%) a percentage of the current size of the logical volume group.
Example: diskutil coreStorage createVolume
11111111-2222-3333-4444-555555555555 jhfs+ myLV 10g
unlockVolume
One of the seemingly-handier commands that isn’t really all that useful. It’s rare that you’ll want to do this alone. Use hdiutil to mount a volume instead.
Usage: diskutil coreStorage unlockVolume lvUUID
[-stdinpassphrase | -passphrase passphrase | -recoverykeychain file]
Unlock a logical volume that is encrypted and currently locked. You must
specify the logical volume by its CoreStorage UUID, because if it is locked
it is not online. A passphrase is mandatory: you must either supply it
interactively or with one of the above parameters.
Example: diskutil coreStorage unlockVolume 11111111-2222-3333-4444-555555555555
changeVolumePassphrase
A very handy command allowing you to change the passphrase of an existing VileVault 2 encrypted volume. This is probably the one verb that will be somewhat frequently used by the average user!
Usage: diskutil coreStorage changeVolumePassphrase|passwd lvUUID
[-recoverykeychain file] | [-oldpassphrase old]
[-newpassphrase new] | [-stdinpassphrase]
Change an encrypted logical volume's password. Beyond the CoreStorage UUID,
you will be prompted interactively for parameters that you do not specify.
Parameters must be given in the above order.
Example: diskutil coreStorage changeVolumePassphrase
11111111-2222-3333-4444-555555555555
resizeVolume (undocumented)
This appears to work fine. You can resize a volume (given its UUID) using this command, and you won’t even lose your data!
Usage: diskutil coreStorage resizeVolume lvUUID size
Resize a logical volume, which is one of one or more disks that consume storage
out of a logical volume group. The logical volume group will have more or less
available space after this operation, if it was a shrink or grow, respectively.
Example: diskutil coreStorage resizeVolume
11111111-2222-3333-4444-555555555555 10g
deleteVolume (undocumented)
This also works fine. Create a volume and you can delete it, as long as you get the right UUID from “diskutil cs list”.
Usage: diskutil coreStorage deleteVolume lvUUID Delete a logical volume. Example: diskutil coreStorage deleteVolume 11111111-2222-3333-4444-555555555555
resizeStack (undocumented)
Probably the coolest undocumented command, resizeStack takes a complete 1:1:1:1 CoreStorage stack (PV:LVG:LVF:LV) and resizes everything non-destructively. You’re left with a fully-operational but smaller volume and partition. Nifty!
Usage: diskutil coreStorage resizeStack lvUUID size
[part1Format part1Name part1Size part2Format part2Name part2Size
part3Format part3Name part3Size ...]
Resize both a logical volume and its underlying physical volume in a single
operation. The setup must be simple: Exactly one logical volume and one
related physical volume can, and must, exist.
If this is a shrink operation, you can optionally request that new partitions
be created in the newly-formed free space gap.
Example: diskutil coreStorage resizeStack
11111111-2222-3333-4444-555555555555 10g JHFS+ New 1g
What’s Missing
These undocumented CoreStorage commands are eye-opening, since it shows that Apple really is working on a complete full-featured volume manager. But lots of functionality is lacking still:
- You can’t create a logical volume group (LVG) by hand, adding multiple disks. It seems like you could, but not enough works yet.
- There are no commands for manipulating logical volume families (LVFs), and they can’t even be specified when creating new logical volumes (LVs).
- The physical volume (PV or disk) commands don’t seem to work. I guess that’s why they’re undocumented!
- There are no advanced data protection features (mirror, snapshot, RAID, replicate)
- There is no ability to specify where or how an LV is created or to move an LV from one LVF, PV, or LVG to another.
Stephen’s Stance
Clearly, CoreStorage is much more than a simple stack to support FileVault 2 encryption. Apple no only built an entire volume manager but also outfitted it with a good set of commands for configuration. Too bad it still lacks so many key features, and that so many of these are unofficial and hidden.
I feel confident that Mac OS X CoreStorage will become much more functional in the future, with complete volume manipulation and migration tools built into the GUI. For now, though, I must remind readers that this is all unofficial and you must proceed with caution. Back up your data, and do not assume that just because cool commands like resizeStack exist they are 100% ready for use!
Note: The pre-formatted text above comes straight from the diskutil command and its “help” responses. I discovered these undocumented verbs through trial and error and the judicial use of the “strings” command!
Am getting deep into OS X to solve the problem formatting a 4TB drive with encrypted HFS+ partition. MediaKit gives an error about Blocksize not dividable by 512 (but assumed 4K is dividable by 512!).
I dont know hat LV, PV, LVG and myLV or myLVG mean. I tried to create a volume (“diskutil cs create volumename disk3”)and media kit gave same error back as apple’s disk utility. I had a
Some examples of standard volume creation commands would be great.
Total Size: 4.0 TB (4000443052032 Bytes) (exactly 7813365336 512-Byte-Blocks) Volume Free Space: 0 B (0 Bytes) (exactly 0 512-Byte-Blocks) Device Block Size: 4096 Bytes
Just played around with CoreStorage, in Mountain Lion, trying to mimic Apple’s Fusion Drive with – for a start – two partitions on one physical drive. First converted one of the two partitions. Then tried to add the second partition to the LVG. No chance : CoreStorage replied with an error, claiming the LVG doesn’t support the addition of more drives. I guess the LVG must somehow tagged with a special property for it to effectively support the addition of various storage resources. I’d love to know how…
I just setup a Logical Volume Group with a Logical Volume backed by two Physical Volumes.
First create a new LVG. This will reformat both disks. Make sure both disks are not converted to CoreStorage already. I couldn’t get “add” to work or “stack” disks converted to CoreStore first:
diskutil coreStorage create CoreStore disk1 disk2
Now you just need to create a volume and it will mount. I specified 100% to use as much space as was available. The new volume turned out to be about the same size as the larger of my two disks (assuming this is how Fusion drives work – you get the size of the largest drive, with the occasional access bonus of the faster drive).
diskutil cs createVolume *lvgUUID* jhfs+ “CoreStore” 100%
I haven’t used it long enough or with an SSD/HDD combo to say how well it works, but it looks promising so far!
Cool stuff. I tried the addDisk command in Mountain Lion 10.8.2, and get a slightly different error: Error adding disk to CoreStorage Logical Volume Group: This Core Storage logical volume group does not allow adding Core Storage physical volumes (-69699).
My guess is Apple are working on it…!
addDisk and removeDisk work in 10.8.3. Be warned that removeDisk seems to make the LVG inoperable currently. addDisk just adds the storage to the pool. You have to manually fill the added storage with resizeVolume or createVolume.
Thank you! The undocumented resizeStack was a lifesaver for me!
Is there a way via the corestorage manager disable the auto function of sending files to and from the Fusion Drive all the time? My USB audio interface on my new imac gets crackling noise when the Fusion Drive sends data to anf from the SSD. That could help lots of people. Have been talking to Apples senior advisors but they dont really accept the problem.
You should probably get that computer fixed. That shouldn’t happen under normal circumstances.
In your description of the convert command you write: “If you convert it without the passphrase, it will be placed in a LVF with no encryption and there is no command to encrypt it later!”
Was that really true back at the time of writing? It certainly is not true now.
This is my own step-by-step process of converting a regular disk to a corestorage volume, then starting the conversion process as the final step:
0. Back up everything with Time Machine!
1. (Find the disk identifier for your OS partition, it will be the one with the correct NAME label and the correct type such as Apple_HFS; look in the right-hand IDENTIFIER column to see the exact identifier such as “disk3s2”): diskutil list
2. (Convert the regular disk partition into a corestorage volume; this resizes the volume and inserts a CoreStorage header, be sure to insert your own exact disk partition identifier): diskutil cs convert disk3s2
3. If you are converting your OS disk, the above command will say that the conversion is deferred since the disk could not be unmounted. In that case, do a reboot. The system will start up, the screen will go black (before login), it will do the conversion (hidden), and it will then restart itself again and the OS will come back up. You are now running the OS on CoreStorage.
4. (When the OS is back up and working, we can now encrypt the LV that the OS is installed on. First, we must find the unique identifier of the Logical Volume. When you type this command, look for a Logical Volume whose “Volume Name” matches the partition you want to convert, and then copy its identifier (it’s in the titlebar of the section, such as “Logical Volume 4ABEB465-B144-44EC-8347-99B8072552EC”): diskutil cs list
5a. (Alternative 1 (safest): As root, tell the OS to queue up an encryption-conversion process for your exact Logical Volume identifier, and feed it the password interactively (it will automatically ask you twice for a password and confirmation)): sudo diskutil cs encryptvolume 4ABEB465-B144-44EC-8347-99B8072552EC
5b. (Alternative 2 (unsafe, because the password is stored in your Terminal log and because you aren’t asked to enter the password twice to confirm it): As root, do the same thing, but give the password directly on the command line): sudo diskutil cs encryptvolume 4ABEB465-B144-44EC-8347-99B8072552EC -passphrase [yourdesiredpassword]
6. Just wait… you can view the progress with “diskutil cs list”. It will say “Conversion Status: Pending” for a long time as it builds a volume map, and then it will eventually start. You can see the “Conversion Progress: 48%” type indicator inside the Logical Volume’s description. Large drives such as my 2TB drive took 12 hours to convert.
7. Note: This method of encrypting the drive is by far the safest. It creates what’s known as a “Disk-based Password” which means that when your system starts up, it asks for ONE password: Your disk-based password (the one you gave to CoreStorage). This is in contrast to FileVault, which creates a login-screen-like GUI where you see a list of system users and can type in one of THEIR regular passwords to unlock the drive. The Pros and cons of each are as follows:
FileVault:
+ Easy to set up (in the Security preferences)
+ Has a “rescue key” system where you type down a long string of numbers which can be used to unlock a drive even if you forget all passwords; this can also be stored with Apple for extra security (but be warned that the key is stored securely on their end and that they cannot even decrypt the rescue key for you unless you provide them the *exact* answers to 3 security questions you gave them during setup).
+ Every user can use their own password to unlock the drive, meaning it’s easy for them to get in
– EVERY user can use their own password to unlock the drive, meaning it’s easy for a bad guy to guess simple passwords and unlock your whole drive
– If you want security, you must change your actual OS user password to something long and complex, which causes it to become a chore to type in the OS itself on all your usual password prompts when installing software etc.
Disk-based Passwords:
+ Moderately easy to set up (for anyone familiar with the Terminal)
+ Extremely secure; there is a single password to use for the entire drive, and you can make it extremely strong and complex
+ You do not expose any lists of users or passwords at boot-time since the computer only says “Disk Password: ?” when you boot it up
+ Allows you to get the best of both worlds: Your system users can use any simple passwords they want, to make day-to-day tasks easy; for instance, you can have a super complex DISK Password, and a super simple USER password. During bootup you always have to enter the Disk Password to unlock the drive, and then when you’re in OS-land you can just live happily as a user with your easy-to-type password for all admin tasks. A thief would always power down your computer and would immediately be faced with your ultra-strong Disk Password and would never even get as far as the OS login screen itself, so even your super-simple OS password is actually secure as hell using this setup!
– There is NO recovery phrase if you forget the disk password. When your system starts up, you have to enter the correct disk password. If you don’t remember it anymore, you are completely screwed. For this reason, I suggest writing it down somewhere safe until you have memorized it. Nothing worse than “Let’s encrypt my drive with a strong password! #$h0rs3#*7777, I’ll remember that!” and then the next day “Okay, was it… $#h0rs3777?” – I suggest using the GRC Password Haystacks method (Google it) to generate a secure AND memorable disk password using your known-only-to-you padding system. Then throw away the password note after a month or so when you’ve rebooted enough times to have completely memorized it and internalized it.
One more important note: If you’re using Time Machine, be sure that encryption is enabled, otherwise your secure OS data is being completely mirrored on an unencrypted drive, defeating the whole purpose of encryption! Click on “Select Disk” in the Time Machine system preferences, then select the volume you’re using and see if “Encrypt backups” is checked. If it isn’t, you will have to “Remove disk” and add the disk again and this time check the encryption checkbox. You will be asked to make a secure passphrase (perhaps using the same one as the strong password you use for your OS disk, although I personally use a different one). From then on, your Time Machine backups are completely safe too.
By having an encrypted OS partition and an encrypted backup drive stored elsewhere in the house, your data is safe in case of burglary. But I also suggest one last thing: Buy some online backup storage from a trusted, reliable service such as Carbonite and tell it to sync all of your most treasured files and folders to the cloud (not the whole OS itself; just stuff like your projects, personal photos, valuable databases, etc).
If your house burns down or is robbed clean, you will definitely want some off-site way to get the most important data back! That is where the cloud comes in.
Disk-based passwords, encrypted time machine, and cloud backups all work together to make your data ultra-safe and recoverable regardless of what disaster may strike. 🙂
It is actually easier to simply format the disk which you want to use for Time Machine as an Encrypted one via Disk Utility. I did this with my current 3TB external drive and had the added benefit of setting all of this up before even enabling Time Machine! And yes, encrypting backups is a must for maximum security.
I differ with the Disk password vs. user password; FileVault2 allows you to only enable certain users to have their user lock/unlock your encrypted boot volume, so you can theoretically just have your own user as the “boot” user, another generic user with a strong password and have everyone use the generic user for booting up the Mac.
Hats off. I just made a standard HDD “transplant”, something I do all the time when swapping over HDDs for a larger one. This is the first time I do it with OSX and with a GPT-partitioned HDD so it was kind of experimental. I did succeed, but I couldn’t resize my partition with the usual tools. resizeStack saved my life! 🙂
If you’re here because of Yosemite upgrade: diskutil cs revert [volumeID] worked non destructively for me: http://awesometoast.com/yosemite-core-storage-and-partition-woes/
Hi Stephen,
Thank you very much for this article. It’s been very exciting to see these commands. They work with Yosemite as well.
Might I ask two more questions?
– Is there a way to show unused space on the disk that is not shown in DiskUtil?
– And is there a way to re-use such space?
I’m asking because I deleted a volume on a disk and I’m not able to see it anywhere nor to reclaim it.
Thanks again!
It’s unbelievable!
I used a Windows PC with Paragon Partition Manager 2013 free and was able to recover all the lost unused space from there.
Why doesn’t Apple have a utility that works like this?
You should be able to use the recently liberated space with resizeStack. I’ve noticed that Mavericks and Yosemite seem to create coreStorage partitions by default these days.
Another tip: if you deleted another partition in your disk using Disk Utility (the graphical one) there’s a good chance that Disk Utility automatically did a resizeStack to fill up the free space for you. Check out diskutil list and diskutil cs list and see if there is any remaining free space.
Tread carefully with encrypting existing Time Machine backups. I had a year of unencrypyted backups on a 3TB USB drive, and after switching to FileVault on the Mac, I also turned on encryption in Time Machine.
The Time Machine disk is now stuck in mid-encryption, diskutil cs list shows the LV as “Conversion Progress: Paused”.
diskutil cs unlockVolume tells me it’s already unlocked and attached.
diskutil cs revert gives Error: -69854: A disk with a mount point is required
diskutil cs decryptVolume thinks about it for a few seconds and then comes back with:
Started CoreStorage operation on disk2
Scheduling decryption of Core Storage Logical Volume
Error: -69693: Unable to decrypt the Core Storage logical volume
Underlying error: -536870212: Unknown error: -536870212
Pretty sure I’m going to have to reformat the drive and lose those TM backups. Not the end of the world, but a pain.
Oh, and +1 for offsite backups – I used BackBlaze, USD5 per month for unlimited backups of one Mac (plus any attached external storage).
Great reference, thank you √