Drobo, XP Users: Beware of 4K “Advanced Format” Drives!

Personal computer hard disk drive access methods have repeatedly been forced to adapt to ever-expanding capacity. Physical (CHS) addressing and 28-bit LBA have been abandoned, but the disk drives themselves retain the tiny 512-byte sectors they have had since IBM introduced fixed-block addressing in the 1970’s. But time marches on, and Western Digital is leading the change to larger 4 kilobyte hard disk blocks. Although this new “Advanced Format” includes mechanisms for backwards compatibility, buyers should be wary of these new drives for the time being.

A Little History

A long, long time ago, before there were personal computers, IBM set the standard for data storage with their CKD (“count key device”) format. Variable-sized records were stored on a raw disk with gaps between them. Each included a sequence number (“count”), an optional key, and the data itself.

This flexible format proved impractical as multi-process systems evolved in the 1970’s, so IBM introduced a new format: Fixed-block architecture (FBA). FBA disk drives were divided up into equal-sized blocks and addressed by their physical location on a specific cylinder accessed by a specific head. Each cylinder/head pairing had its own series of sectors, starting at the outer edge of the physical disk platter and moving inward.

This system of addressing hard disk capacity with a triple number identifying the cylinder, head, and sector (“CHS”) continued in the PC space in the 1980’s. Since early hard disk drives were tiny by modern standards, the PC industry chose the smallest sector size specified by IBM, 512 bytes. The earliest MS-DOS/BIOS PCs were limited to just 504 MB, thanks to incompatible defaults for the CHS address space. This forced a transition, initially raising the addressable capacity to 7.8 GB.

But the writing was on the wall: CHS just didn’t make sense. Storage protocols, including ATA, were being pressed into service for devices that didn’t have cylinders or heads, and the 128 GB limit of ATA-1 wouldn’t cut it forever. So the industry transitioned again, abandoning physical references in favor of a simple logical block addressing (LBA) scheme. By the time ATA-6 was introduced in 2002, all systems used LBA, and the new 48-bit address system allowed a maximum of 128 PB of capacity per device.

Though hard disk capacity had ballooned from the megabyte to the terabyte range (a million-times increase in 30 years), the 512-byte disk sector remained, each with its own error-correcting code (ECC) and gap.

Up To Date

Processors and file systems have long since moved on from 512-byte blocks of data, with 4 kilobytes being the most common size used by the x86 CPU and NTFS, ext3/4, and HFS+ filesystems in Windows, Linux, and Mac OS X. With just about every storage access triggering the reading or writing of eight disk sectors, an increase in hard disk sector size to 4 KB seems obvious. Stepping up to 4K enables better ECC checksums, making disks more reliable (in theory) and freeing up capacity.

Modern operating systems, including Windows Vista, Windows 2008, Windows 7, Mac OS X 10.4-10.6, VMware ESX 3.x-4.x, and most recent versions of Linux, will have no problem with 4 KB sectors in most cases. Installing a 4K drive, like Western Digital’s new “R” line, and partitioning and formatting it fresh should present no issues for most users. In fact, these operating systems will not even “know” they are talking to any special kind of drive, since Western Digital presents its new 4 KB sectors as plain old 512 byte sectors for compatibility reasons.

What’s Your Alignment?

Other systems present a challenge, however. MS-DOS traditionally started filesystems at sector number 63, and most other PC operating systems (including old versions of Linux and VMware) followed this convention. This presented no issue at all as long as disks used 512-byte sectors: Requesting 8 sectors starting at number 63 was the same as requesting eight starting at sector 64. But things get mucked up when the disk drive uses 4 KB sectors.

See the misalignment here? A starting offset of 63 moves the filesystem cluster across the 4K sector boundary. This misalignment means that every filesystem cluster access results in two disk sector accesses. This is a Very Bad Thing, especially for to write performance. Layers upon layers of convention, standards, and compatibility have left us with a problem.

Western Digital solves this problem in one of two ways:

1. Jumper pins 7 and 8 on the drive and it will internally offset all of its logical 512-byte sectors by one
2. Correctly align all partitions written to these new drives

The jumper method, though simple, is not a great idea. If the drive was ever moved or reformatted to a system that aligns its volumes differently (as do all modern operating systems), it would be puzzlingly slow. Instead, it’s best to simply realign any filesystems written to the drives, and Western Digital is providing a utility do just that.

USB, Drobo, and TiVo, Oh My!

Most external drive enclosures will simply pass read and write requests to these new drives as they get them. This means that an external USB enclosure with an Advanced Format drive will have exactly the same issues as an internal drive if it is partitioned under Windows XP or another old operating system. Not only that, but portable drives often get moved from system to system, so an XP-formatted USB disk will have this misalignment issue even when attached to a Windows 7 machine! As these 4K drives begin hitting the market, users should be careful to note if they are correctly aligned.

But what about other disk-using devices like Drobos and TiVos? The jury is still out here. TiVo appears to use misaligned partitions, so I don’t recommend installing 4K drives in a TiVo at this point. I contacted a number of other manufacturers, most of whom are “looking into” the matter.

Data Robotics did respond, and even posted a tech note about 4K drives on their support site. Their short answer is no, that 4K drives should not be used in Drobo storage devices. However, it is unclear if they are affected or just erring on the side of caution until they can test these new hard disks. They also promise an update allowing the use of 4K drives in the future.

I would be cautious about using 4K drives in anything other than a newish PC or Mac at this point. I will continue following the situation and will post updates on my blog as news comes out.

More Info

The following articles also offer great insight into these new 4K drives:

• Western Digital: Advanced Format Technology White Paper
• StorageMojo: Why we need 4k drives
• AnandTech: Western Digital’s Advanced Format: The 4K Sector Transition Begins