macOS Sierra Includes a New Apple File System, APFS

Although not discussed in today’s keynote, Apple is adding a new “universal” filesystem to iOS and macOS. Apple File System (APFS) will likely replace HFS+ as the default filesystem for Macintosh computers, iPads, and iPhones and brings a wealth of modern features. But judging from the initial developer documentation, that’s not going to happen for a few more years. And there’s still much confusion about how APFS and CoreStorage, introduced in Mac OS X 10.7, will interact.

CoreStorage, HFS+, and What Might Have Been

Back in 2011, Apple quietly added CoreStorage, a logical volume manager, to Mac OS X “Lion” 10.7. CoreStorage brought much-needed storage management capability to Macintosh computers, but most people never even noticed it. However, there were two important public-facing features based CoreStorage:

1. FileVault 2 brought native full-volume encryption
2. FusionDrive enabled hybrid SSD/HDD volumes to accelerate performance

What CoreStorage didn’t do was improve the basic file system. Even now, Macintosh computers rely on HFS+, which dates back 18 years to MacOS 8.1. Although Apple has done an admirable job adding features to HFS+, modern computers need something better. Many critical storage functions like extended file attributes are tacked on, core elements like inode numbers and timestamp are severely limited, and desirable elements like sparse files, per-file encryption, cloning, and snapshots are missing entirely.

There have been a few moments in the history of Mac OS when Apple almost updated the filesystem. Mac OS X was based on NeXTStep, which used the UNIX File System, UFS. But Apple retrofitted HFS+ for Mac OS X for compatibility with Mac OS 8 and 9. And Apple came very close to implementing the respected ZFS in Mac OS X “Lion” 10.5 before pulling it in the last minute, reportedly due to licensing concerns.

Introducing Apple File System

Now Apple is finally moving things forward, implementing a new filesystem in the new macOS 10.12 “Sierra”. Apple File System (APFS) appears to be a complete rewrite, unique to Apple, and brings many modern capabilities. Details are scarce, but here’s what we know right now based on Apple’s documentation.

Apple bills APFS as a “Next-Generation File System”, with “64-bit inode numbers, 1 nanosecond timestamp granularity, an expansive block allocator, support for sparse files, and a crash protection scheme.” New features include “optimization for Flash/SSD storage, copy-on-write metadata, space sharing, cloning of files and directories, snapshots, fast directory sizing, and atomic safe-save primitives.”

Now let me explain what all this means.

First, APFS takes the 1990’s out of Mac storage. Filesystems use unique identifiers called “inodes” for files and directories. HFS+ used 32-bit inodes, meaning a filesystem could only contain 2³²-1 objects (about 4.3 billion). This sounds like a lot, but storage capacity keeps increasing and many modern features magnify the numbers of inodes needed. Moving to 64-bit inodes helps future-proof the filesystem.

APFS also integrates extended file attributes with the filesystem instead of having these “live” in a separate file, and allows nanosecond-granularity timestamps, quite an improvement over the 1-second granularity in HFS+. Another improvement is “lazy” block allocation, allowing quick formatting of large drives and making it easy to resize a filesystem.

Next, APFS moves to a copy-on-write system, improving crash protection, enabling cloning of files and directories and snapshots of whole volumes, and enabling optimizations for flash storage. With copy-on-write, data on disk is copied only when a modification is made, building up a tree of “data history” and reducing I/O. APFS won’t need a filesystem journal anymore, individual filesystem elements can be cloned (imagine a per-file Time Machine), and whole point-in-time snapshots are possible. Small I/O operations can also be coalesced to reduce wear on SSDs. ¹

APFS also has some interesting and novel new features. APFS Containers are pools of storage similar to a logical volume group, and these contain volumes that generally map to “namespaces” or file systems. In other words, there is an additional layer of abstraction between the filesystem and the disk, and multiple filesystems can be created on the same disk.

But all volumes and namespaces share the same pool of container storage, and this is explicitly displayed to the user: If you create more than one volume in a container, they all share the same capacity. In the picture above, FS1 and FS2 would both show the same total free and used space. This could be very confusing to users, but I doubt macOS users will ever see it. Instead, I imagine it will be exposed as part of a future “Time Machine 2” that uses snapshots and clones on the same disk.²

The Road to APFS

APFS will not be magically implemented when macOS 10.12 “Sierra” is installed. In fact, it probably won’t be used at all until the next version of macOS is released a year from now. Apple suggests that APFS volumes will be readable on OS X El Capitan 10.11, but this will also probably not be implemented in practice. And even in Sierra, APFS can’t be used on startup disks or Time Machine targets.³

One question is the extent to which APFS will coexist with CoreStorage. APFS seems to duplicate some of the functionality of CoreStorage today, overlapping the existing block and volume allocation and encryption functionality. The fact that APFS will not work on Fusion Drives or in conjunction with FileVault 2 suggests that APFS is at the very least incompatible with CoreStorage today. Certainly one can imagine an APFS container being placed on top of a CoreStorage Logical Volume or implemented as a CoreStorage Logical Volume Family. But will it?

It could be that APFS is another attempt at a volume manager five years after CoreStorage and will relegate it to the dust heap. This would be a great disappointment, since APFS seems very “small system” oriented, while CoreStorage is more of an enterprise approach. For example, APFS does not have any multi-disk RAID-like technology, where CoreStorage has very advanced multi-disk support (hello, Fusion Drive!)

Stephen’s Stance

Apple desperately needs a new filesystem, and it’s wonderful to see the company doing just that. Like Microsoft, which is easing into ReFS, Apple will slowly roll out APFS over a few operating system releases. And like ReFS, Apple has created a modern, flash-native filesystem with flexibility and scalability in mind. I applaud this move, but am concerned about how APFS will work with CoreStorage.

I will be experimenting with APFS as the macOS Sierra betas are released and will report my findings here.