Taming Monster Disk Drives: 3 TB and Beyond!

Fast-forward 6 months into the future! You just bought a new hard disk drive from Seagate or Western Digital. Rather than sticking to the basic 1 TB model, you opted for a brand-new 3 TB whopper. You take it home, unbox it (on camera for YouTube, of course) and slap it into your home-brewed PC. Watch out! A combination of factors is conspiring to wreck your newfound storage happiness. In order to take advantage of new hard disk drives over 2 TB, you must have a compatible operating system, BIOS, partition table, and file system.

How Big?

Hard disk drive capacity limits are familiar to PC enthusiasts. I remember worrying about the 504 MB barrier but pushing through with a BIOS update. The first greater than 2 GB disk I used required updates to both the BIOS and filesystem. Then there was the 32 GB limit imposed by Windows 95. Each of these limits was caused by two factors: Underestimation of the amount of storage capacity available to future PCs and technical “bitness” limitations.

The evolution of PC hard disk drive interfaces is well documented, and I commented on it in some depth when covering Western Digital’s 4K “Advanced Format” drives. In that piece, I mentioned that most PCs today use the “future-proof” 48-bit LBA addressing method, allowing a maximum of 128 PB of capacity. This wasn’t the whole story, however. Many operating systems still cannot use the full 48-bit address space, and we must also consider partitioning schemes and filesystems.

See also Drobo, XP Users: Beware of 4K “Advanced Format” Drives! and Drobo Adding 4K Drive Support — What About Everyone Else?

When 48 Bits is 32 Bits

LBA now allows 48 bits of block address space, and this should allow modern PCs to address a full 128 PB of capacity. This number is easy to calculate: 48 bits of address space allows us to count to 281,474,976,710,656 in (unsigned) integers. Multiply this by 512 bytes and we get 140,737,488,355,328 KB of storage capacity. Funny base-10 to base-2 disk math translates this to 128 PB of capacity.

But using 48 bits of address space requires a CPU and operating system that can work with 48 bit numbers, and many computer systems remain limited to 32 bits thanks to hardware, software, or both. These systems shrink the address space to 4,294,967,296 times 512 or 2,147,483,648 KB of capacity. This works out to 2048 GB or 2.15 TB in base-10 disk capacity speak.

The 2.1 TB Ceiling

This 2.1 TB number became institutionalized in both software and hardware over the last decade. The old-standby MBR partition method was never extended beyond this number, with manufacturers hoping for a wholesale transition to GUID partition table (GPT) format. GPT was bundled with Intel’s EFI as a replacement for legacy PC BIOS, but few PC manufacturers adopted it. Indeed, Apple is the only company to have standardized on EFI/GPT for all of their home computer systems.

Since so few PCs could ever surpass this capacity limit, many peripherals were designed with it in mind. Hard drive controllers and RAID cards may not be able to handle 3 TB drives, either. Although NTFS can expand very, very large (256 TB or more), most Windows PCs still use MBR and BIOS so they remain limited to 2 TB.

What happens if we put a 3 TB disk in a system that can only handle 32 bit LBA addresses, uses MBR, or has a limited piece of hardware? The disk might show up much smaller than expected (with addresses truncated), it might behave oddly (with truncated addresses “wrapping around”), or it might fail to work at all. Simply put, one should not use a hard disk drive larger than 2 TB in any system without checking with the various component manufacturers and operating system vendor first.

Making It Work

Western Digital attacked this problem by increasing the sector size to a more-modern 4 KB in their “Advanced Format” drives. These began appearing in December 2009 and are somewhat supported in modern computers and external disk units. Seagate does not appear to be adopting 4K and is instead pushing forward past the 32 bit address mark with their forthcoming 3 TB hard disk drives. It remains to be seen which course other drive manufacturers will choose.

Generally speaking, you should be in fairly good shape if your computer meets these tests:

1. Intel-powered Apple Macintosh computers (from 2006 onward) with OS X 10.4 “Tiger” or better should be fine
2. Non-Apple PCs require a fairly recent operating system – Windows Server 2003 or 2008, Windows Vista, Windows 7 or better
3. Non-Apple PCs must also have one of the following:
   1. A regular MBR boot disk and the big new drive mounted as a non-boot drive with GUID Partition Table (GPT)
   2. EFI or UEFI firmware rather than legacy BIOS (e.g. some HP, IBM, and MSI systems)
4. Most recent OSes will support these large disks in non-booting external drive units connected with USB 2.0, USB 3.0, FireWire, or eSATA

In other words, current PCs will never boot from these new giant drives, but most other uses should be fine as long as a modern OS is used.

We should also consider external storage systems, from the smaller Drobo, Iomega, or Synology units to the largest enterprise storage arrays from companies like EMC, HDS, HP, NetApp, and IBM. We expect that many of these systems already support the full 48-bit LBA address space, but manufacturers will certainly want to qualify these new drives before officially blessing them. We recommend holding off on trying these large drives in external storage systems until the manufacturers have officially added them to their compatibility lists.