• Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Apple / Drobo, XP Users: Beware of 4K “Advanced Format” Drives!

Drobo, XP Users: Beware of 4K “Advanced Format” Drives!

December 23, 2009 By Stephen 4 Comments

Western Digital is first to market with "Advanced Format" 4K-sector drives
Western Digital is first to bring "Advanced Format" 4k-sector drives to market, but buyers should beware of installing them in old systems!

Personal computer hard disk drive access methods have repeatedly been forced to adapt to ever-expanding capacity. Physical (CHS) addressing and 28-bit LBA have been abandoned, but the disk drives themselves retain the tiny 512-byte sectors they have had since IBM introduced fixed-block addressing in the 1970’s. But time marches on, and Western Digital is leading the change to larger 4 kilobyte hard disk blocks. Although this new “Advanced Format” includes mechanisms for backwards compatibility, buyers should be wary of these new drives for the time being.

A Little History

A long, long time ago, before there were personal computers, IBM set the standard for data storage with their CKD (“count key device”) format. Variable-sized records were stored on a raw disk with gaps between them. Each included a sequence number (“count”), an optional key, and the data itself.

This flexible format proved impractical as multi-process systems evolved in the 1970’s, so IBM introduced a new format: Fixed-block architecture (FBA). FBA disk drives were divided up into equal-sized blocks and addressed by their physical location on a specific cylinder accessed by a specific head. Each cylinder/head pairing had its own series of sectors, starting at the outer edge of the physical disk platter and moving inward.

This system of addressing hard disk capacity with a triple number identifying the cylinder, head, and sector (“CHS”) continued in the PC space in the 1980’s. Since early hard disk drives were tiny by modern standards, the PC industry chose the smallest sector size specified by IBM, 512 bytes. The earliest MS-DOS/BIOS PCs were limited to just 504 MB, thanks to incompatible defaults for the CHS address space. This forced a transition, initially raising the addressable capacity to 7.8 GB.

But the writing was on the wall: CHS just didn’t make sense. Storage protocols, including ATA, were being pressed into service for devices that didn’t have cylinders or heads, and the 128 GB limit of ATA-1 wouldn’t cut it forever. So the industry transitioned again, abandoning physical references in favor of a simple logical block addressing (LBA) scheme. By the time ATA-6 was introduced in 2002, all systems used LBA, and the new 48-bit address system allowed a maximum of 128 PB of capacity per device.

Though hard disk capacity had ballooned from the megabyte to the terabyte range (a million-times increase in 30 years), the 512-byte disk sector remained, each with its own error-correcting code (ECC) and gap.

Up To Date

Processors and file systems have long since moved on from 512-byte blocks of data, with 4 kilobytes being the most common size used by the x86 CPU and NTFS, ext3/4, and HFS+ filesystems in Windows, Linux, and Mac OS X. With just about every storage access triggering the reading or writing of eight disk sectors, an increase in hard disk sector size to 4 KB seems obvious. Stepping up to 4K enables better ECC checksums, making disks more reliable (in theory) and freeing up capacity.

LBA 512 Sector
… 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 …
Actual 4K Sector
6 7 8 9

Modern operating systems, including Windows Vista, Windows 2008, Windows 7, Mac OS X 10.4-10.6, VMware ESX 3.x-4.x, and most recent versions of Linux, will have no problem with 4 KB sectors in most cases. Installing a 4K drive, like Western Digital’s new “R” line, and partitioning and formatting it fresh should present no issues for most users. In fact, these operating systems will not even “know” they are talking to any special kind of drive, since Western Digital presents its new 4 KB sectors as plain old 512 byte sectors for compatibility reasons.

What’s Your Alignment?

Other systems present a challenge, however. MS-DOS traditionally started filesystems at sector number 63, and most other PC operating systems (including old versions of Linux and VMware) followed this convention. This presented no issue at all as long as disks used 512-byte sectors: Requesting 8 sectors starting at number 63 was the same as requesting eight starting at sector 64. But things get mucked up when the disk drive uses 4 KB sectors.

DOS Filesystem
MBR 0 1
LBA 512 Sector
… 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 …
Actual 4K Sector
6 7 8 9

See the misalignment here? A starting offset of 63 moves the filesystem cluster across the 4K sector boundary. This misalignment means that every filesystem cluster access results in two disk sector accesses. This is a Very Bad Thing, especially for to write performance. Layers upon layers of convention, standards, and compatibility have left us with a problem.

Western Digital solves this problem in one of two ways:

  1. Jumper pins 7 and 8 on the drive and it will internally offset all of its logical 512-byte sectors by one
  2. Correctly align all partitions written to these new drives

The jumper method, though simple, is not a great idea. If the drive was ever moved or reformatted to a system that aligns its volumes differently (as do all modern operating systems), it would be puzzlingly slow. Instead, it’s best to simply realign any filesystems written to the drives, and Western Digital is providing a utility do just that.

USB, Drobo, and TiVo, Oh My!

Most external drive enclosures will simply pass read and write requests to these new drives as they get them. This means that an external USB enclosure with an Advanced Format drive will have exactly the same issues as an internal drive if it is partitioned under Windows XP or another old operating system. Not only that, but portable drives often get moved from system to system, so an XP-formatted USB disk will have this misalignment issue even when attached to a Windows 7 machine! As these 4K drives begin hitting the market, users should be careful to note if they are correctly aligned.

But what about other disk-using devices like Drobos and TiVos? The jury is still out here. TiVo appears to use misaligned partitions, so I don’t recommend installing 4K drives in a TiVo at this point. I contacted a number of other manufacturers, most of whom are “looking into” the matter.

Data Robotics did respond, and even posted a tech note about 4K drives on their support site. Their short answer is no, that 4K drives should not be used in Drobo storage devices. However, it is unclear if they are affected or just erring on the side of caution until they can test these new hard disks. They also promise an update allowing the use of 4K drives in the future.

I would be cautious about using 4K drives in anything other than a newish PC or Mac at this point. I will continue following the situation and will post updates on my blog as news comes out.

More Info

The following articles also offer great insight into these new 4K drives:

  • Western Digital: Advanced Format Technology White Paper
  • StorageMojo: Why we need 4k drives
  • AnandTech: Western Digital’s Advanced Format: The 4K Sector Transition Begins

You might also want to read these other posts...

  • Turn Off Error Recovery in RAID Drives: TLER, ERC, and CCTL
  • Today’s Storage: Same As It Ever Was
  • How to Use Mac OS X Sparse Bundle Disk Images
  • Plexistor Integrates NVDIMM for Every Application
  • The Four Horsemen of Storage System Performance Are…

Filed Under: Apple, Computer History, Enterprise storage, Everything, Terabyte home Tagged With: Advanced Format, alignment, CHS, CKD, Drobo, FBA, featured, LBA, Linux, MS-DOS, OS X, TiVo, VMware ESX, Western Digital, Windows 7, Windows Server 2008, Windows Vista

The same thing can be identified by many different terms, and the same term may mean many different things.

Douglas John Foskett

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Add a Mirror to an Existing ZFS Drive

December 11, 2017

How To Remove Raw Images From Apple Photos and iCloud

How To Remove Raw Images From Apple Photos and iCloud

July 24, 2017

Recalling An HP MediaSmart Server To Active Duty

Recalling An HP MediaSmart Server To Active Duty

July 21, 2017

Go Get a ProtonMail Account and Protect Your Online Life!

Go Get a ProtonMail Account and Protect Your Online Life!

July 19, 2017

ZFS Is the Best Filesystem (For Now…)

ZFS Is the Best Filesystem (For Now…)

July 10, 2017

Co-Processors, GPGPU, and Heterogeneous Computing

Co-Processors, GPGPU, and Heterogeneous Computing

June 26, 2017

What is OCuLink?

What is OCuLink?

June 22, 2017

Storage is Getting Cloudier!

Storage is Getting Cloudier!

June 21, 2017

Where Have All The GPU’s Gone? Cryptocurrency Mining!

Where Have All The GPU’s Gone? Cryptocurrency Mining!

June 11, 2017

Turn Off Error Recovery in RAID Drives: TLER, ERC, and CCTL

Turn Off Error Recovery in RAID Drives: TLER, ERC, and CCTL

May 30, 2017

Symbolic Links

  • Keeping Yourself on Track and Avoiding Burnout as a Programmer

  • IRL Analogies Explaining Digital Concepts are Terrible

  • Hass.io 2018

  • Describing Network Automation: Automate the Coffee

  • Cisco’s Latest AP is Mind-Blowing (and a quick history lesson)

Featured Posts

Making a Case For (and Against) Software-Defined Storage

January 9, 2014

How To Sync Your iPad With Your Exchange Server

April 3, 2010

Deduplication Coming to Primary Storage

September 16, 2008

Why Big Disk Drives Require Data Integrity Checking

December 19, 2014

The Rack Endgame: Converged Infrastructure and Disaggregation

September 19, 2014

Sony NEX-7 Hands-On Review Part 1: The New Super-Camera

March 26, 2012

What’s (Still) Wrong With Dropbox For Business

April 17, 2013

VMware’s Hardware Partner Strategy: Heads I Win, Tales You Lose

February 11, 2013

My Advice For New Business Travelers: Get The Credit Cards!

March 20, 2014

United Boeing 787 Dreamliner: Butt-In-Seat Economy Plus Review!

November 13, 2012

Copyright © 2018 · Log in