An Introduction To exFAT

Microsoft already gave the world FAT and NTFS, and both have become common in the non-Windows world thanks to flash drives, SD cards, and portable disks. But the folks from Redmond are now introducing a new filesystem, exFAT. Do we really need a new filesystem?

The Rise and Fall of FAT

Although computing is a volatile world, standards tend to stick around way past their prime. Nowhere is this more apparent than the ubiquitous FAT filesystem. Although it has been modified and updated, the basic FAT structure pre-dates DOS: It was designed for Microsoft’s first real product, Standalone Basic, and bears the fingerprints of the company’s first employee, Marc McDonald, as well as some other programmer called Bill Gates.

Appealing in its simplicity, FAT was adopted by Tim Paterson at Seattle Computer Products for his QDOS operating system, which found its way back to Microsoft as IBM’s PC-DOS. FAT was improved with 16-bit cluster addresses for the 1984 IBM PC-AT’s hard disk drive, and enhanced again to match growing disk sizes as FAT32 in Windows 95. Along the way, VFAT added 32-bit file access and long file names.

Because it was simple in concept, FAT was reverse-engineered and adopted as a standard filesystem for data interchange between DOS/Windows PCs and other operating systems, from Apple to Atari to Amiga. Today, FAT16 is the dominant format for SD cards, and most thumb drives and SDHC cards use FAT32. But FAT is not an open standard: Microsoft successfully patented VFAT and FAT32 and specifically requires digital camera and flash drive companies to obtain a license for its use.

Although widely dismissed as archaic, many of the limits of FAT (especially FAT32) are due to factors outside the filesystem, particularly the MBR partition scheme and implementation-related issues. FAT32 is capable on disks up to 2 terabytes, which matches the maximum size of the next-generation SDXC flash card specification. And, despite Microsoft’s patents, FAT is widely implemented and stable on a variety of platforms and embedded devices.

Why exFAT?

Hard disk drives long since switched to NTFS or other operating system-specific formats, so FAT is mainly used on flash media today. Sadly, FAT isn’t all that well-suited for applications like these. Thumb drives don’t really push the envelope for FAT since they tend to hold reasonable numbers of smaller files, but flash media for digital cameras is a different story.

SD cards have taken over the digital camera world: Even holdouts like Sony now support SD and its successors, SDHC and SDXC. And just about every SD and SDHC card sold today uses FAT16 and FAT32, respectively. It works reasonably well for still images, but video poses real issues. FAT32 is limited to 4 GB per file, and HD camcorders and digital cameras can easily push past this size limit. Most models today split files or limit recording time to avoid this limitation, but that’s not optimal.

exFAT enables file sizes up to 16 EB and also adds nifty features like a free space bitmap allowing better use of flash memory and eliminating the read/erase/write issue of NAND. It also allows more-granular timestamps (10 ms, rather than 2 seconds on FAT) which better-match high-speed shooting capabilities of today’s digital cameras.

exFAT Limitations

But not everything is rosy with exFAT. Although it seems poised to take off thanks to strong support from the SDcard Association, exFAT remains encumbered by patents and licensing fees from Microsoft. Memory Stick XC requires the use of exFAT, but SDXC merely suggests it and there is no guarantee of third-party support. Digital camera makers may decide to stick with FAT32 rather than switch, though Microsoft makes it more attractive for embedded uses with a flat $300k fee per manufacturer.

There is also one odd technical limitation to exFAT. Because removable media is often physically ejected at inopportune moments, the FAT filesystem was designed with alternating file allocation tables. Should writing of the table be interrupted, FAT can always fall back on the previous iteration. exFAT curiously uses just one allocation table and free space bitmap, making it more likely to become corrupt in precisely the portable devices it was intended for. A follow-on enhancement, Transactional exFAT or TexFAT, adds redundant tables and bitmaps but may not be implemented.

Unfortunately, the old, inflexible MBR partition scheme remains coupled to exFAT. It would have been nice to ditch MBR in favor of the open GPT format, but the static nature of flash media made that less of a priority. Still, MBR is limited to 2 TB, and this will become a roadblock for flash media at some point.

You might also want to read Introducing SDXC and exFAT in Apple Mac OS X

Stephen’s Stance

Adoption of exFAT seems likely, especially for the SDXC and MSXC flash cards used in digital cameras. Because it is not backwards-compatible with devices, media, operating systems, and card readers, however, consumer confusion is likely to result. Even systems with an SDXC card reader might not be capable of reading the exFAT filesystem contained on the card, and vice versa.

Microsoft, being the proud parent of exFAT, supports the new filesystem in its modern Windows 7 and Server 2008 operating systems and has moved quickly to add it to Windows XP, Server 2003, and Vista. A few exFAT drivers exist for Linux and Android, and Apple is likely to add support for exFAT in Snow Leopard 10.6.5, since the SDXC-capable mid-July iMac and Mac Mini computers already have it in 10.6.4.

So it looks like we’ve got a new filesystem on our hands, at least for camera cards. But exFAT will not likely replace FAT32 on thumb drives any time soon. Their portability requires broad compatibility, and FAT32 has the widest support today. Although exFAT will likely be used in embedded applications, it there is no compelling reason for it to displace NTFS and other operating system filesystems like (Apple) HFS+ and (Linux) ext3/ext4 any time soon.

Image credit: Scale Face by Playingwithbrushes