Two portable RAID systems arrived at my door. Which would you be more interested in?

RAID is becoming more familiar in the SOHO market, but portable devices using 2.5″ drives remain a novelty. I recently tested two such units, and came away with mixed impressions. Although the ToughTech Duo from CRU-DataPort/Wiebetech sports attractive design, it didn’t perform much better than the homely Akitio Taurus Mini Super-S. And neither is all that portable.

A Tale of Two Drives

CRU-Dataport/Wiebetech ToughTech Duo and Akitio Taurus Mini Super-S

I’ve been intrigued by the industry-wide shift from 3.5″ to 2.5″ hard disk drives, and was curious when this trend would come to consumer devices. CRU-DataPort, also (confusingly) known as Wiebetech, was first to offer up a two-drive portable RAID unit for me to test. Their ToughTech Duo is a rounded elongated brick with a brushed aluminum finish, LCD screen, and big rubbery buttons.

The ToughTech Duo arrived with a pair of 750 GB Seagate drives installed, but it supports many others as well. I swapped in a pair of 500 GB Toshiba drives (sent separately by IDEMA for my use in tests) and was immediately impressed by the ToughTech’s metal drive carriers. They hold the drive firmly and latch in place securely yet require no screws. The overall build quality of the ToughTech Duo is solid, and the interface is easy to use – perhaps too much so, as we will shortly see!

I really appreciated the CRU-Dataport/Wiebetech ToughTech Duo drive carrier

A short time later, Akitio offered me their 2-drive “Taurus Mini Super-S” for testing. A sharp square brick with a rough black finish, the Taurus Mini’s LCD was harder to read and the two tiny control buttons were nearly impossible to press. The drives reside behind a metal door operated with thumbscrews but attach to rough trays with standard Phillips screws. I suppose the drives are secure, but the Taurus isn’t reassuring to work with.

Both the ToughTech Duo and Taurus Mini Super-S include USB 2.0 and eSATA as well as two FireWire 800 ports for daisy-chaining. The omission of USB 3.0 is curious considering how widespread that interface has become on storage devices, but eSATA and FireWire allow these devices to reach their (limited) performance potential.

The CRU-Dataport/Wiebetech packaging is much friendlier

Both devices ship in gaudy boxes that would be right at home on the shelves of MicroCenter or Fry’s, but the Akitio is excessively verbose and nerdy. The packaging, like the overly-long product names, would put off casual computer users, especially Apple buyers looking for simplicity and ease of use. This is disappointing, since any device with FireWire but no USB 3.0 must be aimed at Apple users!

Configuring RAID

Both drives allow the user to switch between RAID 1 (mirroring) and RAID 0 (striping, with no data protection) using the panel buttons. But since this operation is destructive to the data stored, this might not be all that useful. Most users will likely select one or the other and leave it that way, a blessing given the microscopic buttons on the Akitio.

An average user is likely to leave the drive plugged in when performing RAID changes, leading to disastrous results with the Wiebetech drive. The Akitio resets itself when RAID levels are changed, but the Wiebetech (curiously) does not. I was able to continue writing and reading data after switching to RAID 1, but a reboot destroyed the format and the data. I made the company aware of this issue and they assured me it would be corrected. But I haven’t seen a fix yet, and it’s been a few months. The fact that the documentation instructs users to unplug the drive on RAID changes isn’t reassuring: Few buyers will actually read and follow these instructions.

Interestingly, both devices use the same on-disk format for both RAID 0 and 1. I was able to pull the drives from one and use them in the other with no apparent issues. Wiebetech assured me that their RAID 1 format simply writes all data to both drives. I was able to confirm that this was the case, with a removed RAID 1 drive remaining readable using a direct SATA connection.

Usability Concerns

There's a lot of stuff in the Akitio Taurus Mini Super-S box!

Although both devices are admirably small, they are not really all that compact. Carrying either drive (along with its power brick) in my backpack was definitely noticeable both in bulk and weight. And these are surprisingly heavy devices – each weighs more than my 2011 MacBook Pro!

Both devices get fairly hot under heavy use as well, and neither sports a fan. The Akitio has what looks like a heatsink along the bottom, but I didn’t notice that the rest of the body stayed cooler. And the Wiebetech got downright hot, even setting off its internal temperature alarm during benchmarking! This was an isolated occurrence, however, and I was unable to cause it to overheat again.

It is way too easy to erase all data on the ToughTech Duo by changing RAID levels with the big, friendly buttons

The ease of switching RAID modes is disturbing to me, since this destroys all data on the drives even without considering the Wiebetech’s data loss-inducing bug. But the LCD and buttons are fairly useless apart from this rare operation.

Performance and Compatibility

I tested both devices with the same Toshiba 500 GB drives to gauge their controller performance, but the Seagate drives shipped with the CRU-Dataport/Wiebetech device are much quicker overall.

Sequential read performance with RAID 0 pushes the limits of FireWire at about 83 MB/s, with RAID 1 only slightly behind

Sequential write performance isn't as good, with both units lagging when using RAID 1. Although equal in performance with RAID 0, the Wiebetech lagged severely in RAID 1 performance

Random read performance is nicely homogenous, with both drives topping out around 45 MB/s

Random writes were a different story, with the Akitio and Wiebetech close together for both RAID 0 and 1, but each demonstrating superiority in a different area. The Akitio just kept accelerating as I/O's got larger with RAID 1, while the Wiebetech pulled ahead with RAID 0. Oddly, RAID 0 writes lagged behind RAID 1 on the whole.

Wiebetech's bundled Seagate drives easily outperformed the Toshibas is random benchmarks, nearly doubling random write throughput

But the Toshiba drives were faster in sequential operations, with the Seagates lagging well behind

Stephen’s Stance

I often receive storage devices for review, but it’s unusual that two such similar ones arrive at once. After giving each a fair amount of testing and use, I come away unimpressed. The Akitio looks and feels like the sort of no-name OEM device often overlooked by buyers, while the slickly-designed Wiebetech exhibited disconcerting bugs. Both have confusing names and packaging as well. If I had to pick one, it would be the CRU-Dataport/Wiebetech ToughTech Duo, thanks to its more compact size and user-friendly drive sleds. Note that the Akitio appears to be cheaper at retail, but this is due to the cost of the hard disk drives in the Wiebetech/CRU-Dataport device.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Two Dual-Drive Portable RAIDs Reviewed: Akitio and Wiebetech
This post was categorized as Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I think this sentiment is just as valid today as when I posted it in 2008!

Although many discussions in the storage industry focus on the relative merits of one protocol or another, the conversation occasionally turns to the core issue at hand: We continue to patch together a system based on outdated concepts. Most storage protocols continue to mimic direct attached storage, and most of our so-called networks act as point to point channels. An ultra-modern virtualized storage infrastructure with all the latest bells and whistles still holds the concepts of block and file at its core. Whenever the storage industry has tried to bring about real storage management they have been stymied by a lack of context for data.

No amount of virtualization, and no new protocol, will fix this. Put simply, we need a storage revolution.

Channels, Blocks, and Files

Most innovation in the 1980s and early 1990s focused on moving storage out of the server. SCSI allowed disk to exist in a separate cabinet, RAID allowed multiple physical disks to become a single virtual one, and these were mixed to become the prototype storage array. Although SCSI allowed one-to-many connectivity, it was never a true peer-to-peer network, even once it was mixed with network concepts in the form of Fibre Channel.

Even today, SAN storage is focused on providing faster, more flexible, and feature-packed direct-attached storage. A modern virtual SAN hides a complex arrangement of caching, data protection, tiered storage, replication, and deduplication, masquerading the lot as a simple, lowly disk drive. It is sad but true that all of our work as an industry has been dedicated to recreating what we started with.

Networked file-based storage is no better. Although NAS devices have all the advanced features of their SAN cousins, they must present a simple file tree to the host to retain compatibility. File virtualization merely presents a larger homogenous tree.

Inside the server, too, features and complexity are hidden to retain a familiar file system format. Volume managers can do anything a virtualization device can, but must present their output as a simple (though virtual) disk drive. File systems, too, have added features but still present a familiar tree of mount points, inodes, and files. Even ZFS, possibly the most advanced combination of volume management and file system technology yet, must present a simple tree of storage to applications.

The Metadata Roadblock

This outdated paradigm, of disks and file trees, is ill-suited to today’s storage challenges. Data must be categorized so actions can be taken to preserve or destroy it based on policies. Data must be searchable so users and applications can find what they want. Data must be flexible so it can be used in new ways. Our antiquated notions are not capable of meeting these challenges.

One simple problem is that we lack context for our data. Most file systems merely assign to a file a name, location, owner, and security attributes. The most advanced can contain extended metadata, but this is rarely seen in practice since many applications cannot agree on how to use this data. Microsoft’s Office suite can store and share extended file attributes, for example, but these live inside the file rather than in the file system. The promise of expanded Office attributes is only realized in conjunction with a content management system like SharePoint which lies above the lowly file system.

What if the storage system could keep this data instead? What if it could logically group files according to project or client, mining keywords and authors, and maintaining revisions? These concepts are not new, having been implemented in content management systems for years, and certain elements appeared in file systems, like Apple’s HFS and VMS’ Files-11, for decades.

Cut Down the Tree

File metadata would allow advanced features, but truly taking advantage of them requires a more fundamental shift in the way applications access files. Rather than sticking to a traditional hierarchy of directories in a tree (which was, after all, simply a primitive metadata system), we should remove the tree altogether. Allow files to become data objects, identified by arbitrary attributes and managed according to an overarching policy.

This future vision is decidedly different from our current notion of storage, but is not so far off. Many organizations now rely on central data warehouses based on SQL-language relational databases. As many storage managers have grumbled, databases tend to ignore storage management concepts entirely, managing their own content independently.

But not all applications need a database back-end, so another initiative seeks to provide generic object storage for wider use. Called content-addressable storage or CAS, these devices have traditionally been used only for archival purposes, since that was their first market application. As vendors break free of proprietary interfaces in favor of open ones like XAM, CAS could transform storage itself by eliminating both file and block storage at once.

Similar concepts are already at work in the so-called Web 2.0 world. Non-traditional databases like Google BigTable, Amazon S3, and Hadoop allow massive scalability for object storage. API-sharing initiatives with many Web 2.0 companies can be seen as similar prototypical object storage frameworks. Any of these could be leveraged to provide a new world of data storage, and many are gaining traction even now.

Stephen’s Stance

Although traditional block storage is here to stay for disk drives, and tree-type file systems are likely to remain the foundation of operating system storage, new object-based concepts could change the world in fundamental ways. As applications become “web aware”, they also become object aware, increasing the likelihood of such a storage revolution. For the majority of applications, this new world would be a welcome one indeed.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | We Need a Storage Revolution
This post was categorized as Apple, Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Promise Pegasus R4 is a desktop RAID system featuring Thunderbolt connectivity

When Apple announced the new MacBook Pro at the end of February, there were just two Thunderbolt peripherals featured: The LaCie Little Big Disk and the Promise Pegasus. Both of these storage devices were on display at the NAB Show in Las Vegas last week, and each appeals to a different market segment. The 2-drive Little Big Disk is a portable matched up with the MacBook Pro, while the Promise Pegasus is a 4- or 6-drive desktop RAID system. Promise expects to deliver the Pegasus to the market sometime after the summer.

Introducing Pegasus

Promise Technology is a storage company that should be familiar to many Mac users, since Apple sells their rackmount RAID units as a replacement for their canceled Xserve RAID. Most of Promise’s storage products are larger devices intended for business use, but the company does make a line of smaller desktop RAID systems called the SmartStor.

The new Pegasus storage systems from Promise are Thunderbolt versions of the company’s latest 4- and 6-drive SmartStor lineup. Unlike LaCie, Promise’s SmartStor drives are available in a “bring your own drive” model, allowing consumers to use whatever hard disk drive they desire. This reduces the initial cost of purchase but also raises compatibility and support headaches, since customers may use unsupported or mismatched hard disk drives.

Although some SmartStor devices offer NAS connectivity, the Pegasus is a direct attached storage (DAS) device with no Ethernet connectivity possible. In fact, the Pegasus appears very simple in the flesh, with nothing but two Thunderbolt ports and a power outlet around the back. Promise will sell two models of the Pegasus Thunderbolt RAID storage system: the 4-bay Pegasus R4, and the 6-bay Pegasus R6.

The Thunderbolt cables appear tiny, even on a compact desktop device like the Pegasus R4. Those are quiet variable-speed fans.

Inside, the Pegasus features seven different RAID modes, including high-performance striping (RAID 0), drive mirroring (RAID1), single or dual drive parity raid (RAID 5 and 6, respectively), and stacked raid 10, 50, and 60.

Promise suggests 800 MB per second throughput, though only a stripe set of six drives is likely to deliver this kind of speed. RAID-protected data will likely come in below 200 MB per second, even for the 6-drive unit. This is still much faster than any USB or FireWire drive, and likely beats LaCie’s 2-drive Little Big Disk as well. But it is unfortunate that such lofty performance numbers are promised, if you pardon the pun.

Stephen’s Stance

The Promise Pegasus was featured in many displays at NAB, and appears nearly ready for production. We look forward to experimenting with the final product and putting it through its paces, since it will likely be the fastest Thunderbolt product on the market for a while. Although the Pegasus is luggable, it is not exactly a portable device like the Little Big Disk. This is not a criticism: It is good that two distinct devices will come to market rather than a raft of similar products. Like LaCie, Promise is setting the standard for the products that will follow.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Promise Pegasus Thunderbolt Preview
This post was categorized as Apple, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

LaCie will be first to market with a Thunderbolt storage device

LaCie looks to be the first out of the gate with a Thunderbolt storage system. They promise to deliver their Little Big Disk portable RAID storage device sometime this summer, and the polished look of the devices on display at the NAB show suggests that they will meet this target.

Hello, Little Big Disk

The Little Big Disk is a portable RAID storage device featuring two hard disk drives or flash solid-state disks (SSD). On display at NAB was a two-drive 1 TB hard disk drive RAID featuring 7200 rpm drives. This will be the initial product to market, with the SSD devices following later.

Each Little Big Disk features two Thunderbolt ports, and LaCie demonstrated daisy-chaining four of the devices at NAB

LaCie’s demonstration at NAB was impressive, but not really a real-world scenario. They had four Little big disk devices daisychained off a single Thunderbolt port, each configured as RAID 0 for maximum performance. These eight hard disk drives were able to deliver over 800 MB per second to Apple’s Final Cut Pro.

But real-world users will likely configure the drives as RAID 1 mirrors for data protection, and the Thunderbolt port does not have enough power to spin up this many drives without external power bricks. Therefore, most Little Big Disk buyers will experience less-mindbending performance – around 100 MB per second. Although not quite as fast as some might hope, this still leaves USB and FireWire in the dust, making this new product the fastest official external storage device available for a MacBook Pro. It’s just too bad Apple doesn’t support USB 3, however, since some USB 3 devices already push past 200 MB per second.

Stephen’s Stance

Since Thunderbolt was introduced on a portable device, the Apple MacBook Pro, first, it makes sense that a portable storage product should be the first peripheral to market. The LaCie Little Big Disk looks like an excellent combination of data protection, portability, and performance, and will likely sell like hot cakes once it is released. LaCie should be applauded for targeting the right niche to start out, as well as for their early access to the Thunderbolt technology from Apple and Intel. I look forward to seeing what products they follow on with.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | LaCie Little Big Disk Thunderbolt Preview
This post was categorized as Apple, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

How granular is your storage? This question is just as relevant today as when I first published this article in 2008!

Many storage challenges focus on correlating high-level uses of data (such as applications) with the nuts and bolts of storage infrastructure. These discussions often revolve around the conflict between data management, which demands an ever-smaller unit of management, and storage management, which benefits most from consolidation. Developing data management capability that is both granular enough for applications and scalable enough for storage is one key to the future of storage.

Storage Management: Scaling Up

As I discussed in a previous Sunday Series piece, Turning the Page on RAID, the data storage industry has traditionally focused on reducing granularity. Disk capacity has expanded, and RAID technology has multiplied this by combining multiple physical drive mechanisms into a single virtual one. Storage virtualization technologies, from the SAN to the server, have also often been touted primarily as a mechanism to reduce heterogeneity. From a technical perspective, therefore, granularity has been an obstacle to overcome.

The core organizational best practice for storage management is the reduction of complexity and the enforcement of standardization. Consolidation of storage arrays and file servers is a common goal, as IT seeks to benefit from economies of scale. The goal of both initiatives is the creation of a storage utility or managed storage service. This mirrors efforts on the server and network sides to consolidate and virtualize hardware.

Although both technological and organizational factors have traditionally driven granularity out of storage, this does not have to be the case. Virtual pools of storage are ideal for providing storage on demand, as disk-focused RAID groups give way to more flexible sub-disk storage arrangements. And an operational focus on standardized storage service offerings has the potential to enable scalable management of these smaller units.

Filing Service

File-based protocols would seem to have more potential for granular storage management, but they have been undermined by the hierarchical nature of modern file storage. Whether the connection to a file server uses NFS, CIFS, or AFP, the key unit of management is actually the shared directory, not the file. All files in the share \\firefly\backups would be located on the same server and would be managed as a unit.

NAS virtualization can change this somewhat, as can more specialized NAS servers. Although Microsoft DFS enables consolidation and virtualization of NAS shares, it does not allow subdivision of shares below the directory level – all files in a directory must be placed on the same server. Tricks like stubbing and links allow for some movement, but these do not solve the core issue. Specialized virtual NAS devices from F5 (the ARX, nee Acopia), NetApp, BlueArc, Symantec, and others have the ability to move files individually, providing as much a virtualized storage environment as any block-focused enterprise array. Avere is also beginning to talk about granular file management.

But even an ideal virtualized file server lacks the kind of granularity demanded by users. They care about data, not files, and most applications consolidate their data storage into a few files. Consider a database, for example, where users want each record treated uniquely but storage devices see just a few much larger files. We need a storage revolution, where someone creates an ideal storage platform in which each individual record or object includes custom metadata and is managed independently. This would truly be a massive change, however, and it is not clear that all applications will follow the object storage model of Google and Amazon.

Small is Beautiful

Barring a revolution in data management, our best hope is to allow greater granularity in storage management. As mentioned above, virtualization technology has the potential to enable management and protection of any unit of storage, right down to the individual block or record. But the reality of storage virtualization has not matched its promise.

What is needed is greater integration. Each layer of virtualization (file system, volume manager, hypervisor, network, array, and RAID) also hides necessary details from lower layers. Consider the case of a virtual server snapshot: The application and filesystem must be in a quiesced state to allow a snapshot to be taken at the storage level, but the storage array has no intrinsic information about how its capacity is used. A given LUN might contain dozens of servers on a shared VMFS volume, so all must be snapped together.

Integration can be enabled by sharing more information through APIs. VMware leverages Microsoft Volume Shadow Copy Service (VSS) integration for shared storage so a VMFS snapshot can call the operating system and even applications (Windows Server 2003 only, for now) to prepare the data. Similarly, VSS can communicate directly with supported iSCSI and Fibre Channel arrays, calling a snapshot at the right moment. And Microsoft is, no doubt, enhancing VSS as we speak.

As virtualization technology matures, expect this type of integration to improve. We hope to see more APIs exposed by VMware and Microsoft, allowing communication up and down the stack to break through the information barrier. Imagine a future where a standard API like VSS can pass a message through VMware, Xen, and Hyper-V to the underlying storage array to initiate a snap. I predict that this kind of integration-enabled granularity is not too far off.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Granularity: The Hidden Challenge of Storage Management
This post was categorized as Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

• My writing
  • First up, a battery charger? La Crosse BC-700 Review: A Battery Charger That Does Not Suck
  • I urge you to See W. Curtis Preston’s Backup Central Live!
  • I was looking for The Best Network Card For VMware ESX Home Lab Machines
  • From my Network Computing blog: Unified Storage: What Is It Good For?
  • I was amazed to see the result of my HP Photosmart printer series: The Power of Negative Publicity
• Other great links
  • Chris Evans wrote a solid piece for Datamation: Virtualization and Storage: Overview, Vendor Solutions
  • Matt Simmons talks about dealing with storage: I’m here to shard data and chew bubblegum…
  • Alex Popescu’s 5 Approaches to Scalable Storage Solutions led to Jeff Darcy’s Introduction to Distributed Filesystems
  • Xangati talks Field Day: Running Comprehensive and Deep at Tech Field Day #5
  • And then there’s Wireless Field Day, courtesy of Andrew von Nagy: Shamrockin’ the Wireless Industry
  • Greg Ferro continues spreading the word on FCoTR: Presenting Silvano Gai With FCoTR Button
  • Howard Marks takes up the topic of specialized hard disk drives: Yes, Virginia, RAID Drives Are Different
  • Howard was spurred by Simon Gallagher, among others: Home Labbers beware of using Western Digital SATA HDDs with a RAID Controller
  • The continuing saga of Google Chrome and H.264 takes a weird turn: Microsoft adds H.264 support to Google Chrome
  • Simon Long fights the good fight, presenting solid technical info: VMware View Desktops: IDE or SCSI? BusLogic, LSI Logic or PVSCSI?
  • A key topic in storage is the balance between performance and capacity, as noted by Xiotech: Performance and Capacity Tradeoffs and the Rise of a New Class of Storage
  • An amusing infographic: The United States of Shame (CHART)
  • Kevin Houston takes on Cisco UCS: What Cisco Has to Do to Win the Blade Server Market
  • Finally, Jeremy Gaddis is back with Bypassing the Internet Kill Switch

Subscribe to my Google Reader feed or follow me on Twitter to see these in real-time.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Back From the Pile: Interesting Links, February 4, 2011
This post was categorized as Enterprise storage, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

2011 will look pretty much like 2010 apart from the differences...

Prognostication is a perilous business, but pundits are drawn to the topic in the month of December. The fact that most predictions fall on their faces demonstrates the intoxicating mix of hope, dreams, and irrationality that mark both geniuses and fools. I am neither, so I like to make predictions after the fact! But this year I’ve been asked to look to the future, so I’ll stick with the safe road and pick current trends rather than guessing what I hope will come.

Five Trends For Everyone

1 – Ubiquitous Connectivity

I named mobility as the mega-trend of the last decade, noting that it’s hard to spot a trend from the middle and harder still from the start. But I feel vindicated on that 2009 call, and will take it one further: 2011 will see ubiquitous connectivity become mainstream. With “MiFi” entering the vernacular, a proliferation of “4G” networks and integrated wireless data, and free WiFi having already become passe, I’d say “online everywhere” is here. Although incredibly challenging from both a technical and business perspective, I expect everyone and everything to be online-capable.

2 – The iPad and Foes

Steve Jobs’ bizarre claim that the overgrown iPhone tablet was the most important thing he ever did is starting to look prescient. A new generation is coming of age without windowing GUIs, mice, and keyboards thanks to gaming consoles, smartphones, iPods, and (finally) tablets. 2011 will finally see serious Android, WebOS, and Windows tablets, but the iPad (and forthcoming iPad 2) are the platform to beat, and Apple is firing on all cylinders. This war will be all-consuming next year.

3 – Wave “Hi” to Kinect!

Microsoft has a serious hit on their hands with the Kinect add-on to the Xbox 360. The gaming system is the best thing to come out of Redmond in a long time, and it continues the “no controller” concept of the iPad, finally kicking the Wii to the curb. The massive success of the Xbox will lead Microsoft shareholders to abandon their calls for Ballmer’s head, instead asking for a spin-out or IPO of the gaming division. Expect PCs to include Kinect-like features in the coming years as well.

4 – Facebook Is the Internet

Now boasting a quarter of all web pageviews. Facebook is looking increasingly like AOL for the rest of us. 2011 will see Facebook’s gravity pull in content from everywhere, and its satellites sprout all over the Internet. It will become the single sign-on, the central “like”, the address book, and the meeting place. But fear not, Facebook-phobes: Nothing is permanent, and this too shall pass.

5 – The Internet Changes and No One Notices

The IP address space is exhausted, BGP and DNS security lapses rise, and net neutrality falls by the wayside but Farmville still works so no one cares. The Internet is changing, and controversies over key components are coming coming to a head. I imagine the network engineers will be busy keeping ahead of catastrophe, but they’ll manage somehow. I’m not sure if IPv6 will finally take off or if the carrier gambit will succeed, but I’m confident we’ll still have an Internet at the heart of the technology world!

Five Trends For the Datacenter

6 – Clouds Gather Quietly

“Cloud” was the buzzword of the last two years, but now it’s getting down to work. Traditional IT staff still won’t see much of it outside of blogs and conferences, but non-traditional systems are all heading that way. Look for major uptake of cloud platforms and services from the home to enterprise applications and everywhere in between. Ironically, the “c-word” itself will soon be dropped from these successful services just as it gains acceptance in IT shops.

7 – Virtual Everything

The impact of server virtualization hasn’t been as great as supporters claim, but widespread acceptance of hypervisor-centric data centers is here. There’s really no reason not to deploy every datacenter server as a virtual machine and lots of resulting benefits. Expect to see mission-critical apps finally move to VMware ESX and Microsoft Hyper-V in 2011. And expect to see the resulting infrastructure called “cloud”!

8 – Farewell, Fast Hard Drives

Storage I/O performance is now the realm of solid state, not spinning disk. SSDs have reached the level of performance, capacity, availability, and sophistication that we no longer need 15k rpm enterprise hard disk drives. You’ll use SSD if you want IOPS, but you’ll still need spinning platters for capacity and maximum throughput for a long while. Don’t expect hard disk drives to disappear, but the fastest will exit at the end of the year.

9 – Not-So-Converged I/O (Yet)

DCB ain’t ready, folks. Neither is FCoE. Although Ethernet will eventually sideline InfiniBand and Fibre Channel, that’s not a 2011 topic. I expect to hear a lot of noise about converged network and storage I/O, including high-profile customer adoption stories, but we’re still a few years short of actual impact and serious market share movement. Practical application starts in 2011, though, and it’ll get major coverage and big-money action in the vendor space.

10 – RAID is (Finally) Dead!

There won’t be much ink spilled in memoriam outside storage blogs like this one, but conventional mirroring and parity has finally met its maker. Today’s hard disk drives are too big to rebuild singly, and alternatives like wide striping, erasure coding, and touch-me-not disk packs are taking over.

And One More For Me

11 – The Internet is Shiva

Every business will be permanently changed as Internet-enabled platforms destroy profitable monopolies and build new opportunities. Google conquered advertising and destroyed traditional publishing but enabled a flowering of democratic dialog. PayPal and Square will do the same to banking in 2011, but their own come-uppance might come sooner than they like. No matter your business, someone has their sights set on you and the Internet is their tool. The trick is to keep dancing, keep innovating, and love the paradox.
The text to appear after expiration date.
Image Credit: Eleven by Wetsun

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Eleven Tech Trends To Watch In 2011
This post was categorized as Apple, Computer History, Enterprise storage, Everything, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

• My items:
  • How To Get People To Share Your Tweets, Blog Posts, and Updates
  • The Four Horsemen of Storage System Performance: I/O As a Chain of Bottlenecks
  • From CAS to Cloud: Revolutionary Storage
  • Niketown588: Coffee With Thomas Episode 12 – The Founder of Gestalt IT
• Enterprise IT:
  • The end of RAID
  • Flexible IT and the Path to the Services Future
  • The State of Statelessness: Cisco UCS vs. HP Virtual Connect
  • Microsoft FY11Q1 Results
  • Toward A Faster Web: Increase the Speed of Light
  • What version of SMB2 am I using on my Windows file server?
• Consumer tech:
  • Can I Get an ‘Hallelujah’ for Auto-Completion With the Esc and F5 Keys?

Subscribe to my Google Reader feed or follow me on Twitter to see these in real-time.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Back From the Pile: Interesting Links, October 29, 2010
This post was categorized as Enterprise storage, Everything. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

Why do some data storage solutions perform better than others? What tradeoffs are made for economy and how do they affect the system as a whole? These questions can be puzzling, but there are core truths that are difficult to avoid. Mechanical disk drives can only move a certain amount of data. RAM caching can improve performance, but only until it runs out. I/O channels can be overwhelmed with data. And above all, a system must be smart to maximize the potential of these components. These are the four horsemen of storage system performance, and they cannot be denied.

Overcoming the Limits of Spindles

Perhaps the previous discussion of spindles left you exhausted, imagining a spindly-legged centipede of a storage system, trying and failing to run on stilts. The Rule of Spindles would be the end of the story were it not for the second horseman: Cache. He stands in front of the spindles, quickly dispatching requests using solid state memory rather than spinning disks. Cache also acts as a buffer, allowing writes to queue up without forcing the requesters to wait in line.

Cache may be quick, but practical concerns limit its effectiveness. Solid state memory is available in many types, but all are far more expensive per gigabyte than magnetic hard disk media. DRAM has historically cost 400 times as much as disk capacity, and even NAND flash (the current darling of the industry) is more than 40 times as expensive. Practically speaking, this means that disk devices, from the drives themselves to large enterprise storage arrays, usually include a very small amount of cache relative to their total capacity.

When specifying a storage system, the mathematics of cache and spindles adhere to a simple rule: More is better for performance but worse for the budget. This leads to a trade-off, where a point of diminishing return tells us to stop adding both spindles and cache and accepting the storage system as it is.

A History of Cache

Cache was not always as common as it is today. When even a small amount of DRAM cost hundreds of dollars, adding a single RAM chip to a hard disk drive would have broken the bank. So many drives had no cache at all well into the mid 1990′s. Operating systems of the time used expensive system memory as a buffer for storage operations rather than expecting cache in the disk controller or drive – remember setting the Buffers command in config.sys?

This was not as bad as it seems, at least in theory. Operating systems stand a fighting chance of “knowing” what data will be requested next, and could therefore request it ahead of time. They also might get a hint about data that will never be used again and can thus flush that from the so-called buffer cache. Although MS-DOS wasn’t very good at this, modern systems have greatly advanced in this respect using a technology called demand paging.

Caching at the array was the key differentiator for early enterprise RAID systems, overcoming the punishing slowdowns caused by parity calculations when data was written. EMC adapted their DRAM-based solid-state storage systems to become a cache in front of hard disk drives and the Symmetrix was born. The Data General (now EMC) CLARiiON was notable as well, bringing a large intelligent write cache to the vast market of midrange systems that could never justify the high price of a Symmetrix. Today, all vendors, from IBM to HP to NetApp to HDS, have vast and clever caches.

The importance of cache on enterprise storage performance can not be over-stated. Mix together rotational latency, seek time, and RAID penalty and you get seriously-compromised I/O response time. But cache can eliminate this penalty entirely, provided there is capacity, by confirming the write and queueing it for later (a concept known as write-back caching). Busy shared storage systems would be simply unusable without cache.

Five Uses for Disk Buffers

Hard disk drives today normally contain a small amount of RAM to use as a buffer for I/O requests. This serves the following needs, though not all are found on all drives:

1. A read cache, allowing frequently-requested data to be read from memory rather than involving mechanical disk operations
2. An I/O-matching mechanism, allowing slower disks and faster interfaces to work together
3. A read-around (ahead or behind) pre-fetch cache, saving a few blocks around any requested read on the assumption that they will also be requested soon
4. A read-after-write cache, saving recently-written data to serve later read requests
5. A command queue, allowing write commands to be reordered, avoiding the “elevator seeking” common to early hard disk drives

Disk buffer size has expanded rapidly in recent years, with some devices including 64 MB or more or DRAM. Seagate’s Momentus XT drive even includes 4 GB of NAND flash as a massive read cache!

Write-Through and Write-Back Cache

There are two basic methods of caching data:
The earliest systems used read-only or write-through caches. All I/O requests pass through the cache, which usually saves the most recent and serves them up when a read is requested. They don’t buffer write requests at all, simply passing them through to the storage system to process. They are safe, since the storage device always has a consistent set of committed writes, but they do nothing to offset the RAID penalty.	Most modern storage systems use a write-back (also called “write-behind”) cache, which acknowledges writes before they are committed to disk. They use non-volatile RAM, battery-backed DRAM, or NAND flash to ensure that data is not lost in the event of a power outage. Though far more effective, this type of memory is also far more costly.

Just about every modern storage array uses caching, and most employ the write-back method to accelerate writes as well as reads. Some have very smart controllers that perform other tricks, but Smart is another Horseman for another day. As mentioned before, RAID systems would be nearly unusable without write-back cache allowing the disks to catch up with random writes.

Onward: I/O, and Smarts

The horseman of spindles is harsh, but he does not rule the day. There are many ways to overcome his limits and his three brothers often come into play. These are cache, which bypasses the spindle altogether; I/O, which can constrain even the fastest combination of disk and cache; and the intelligence of the whole system, which limits or accelerates all the rest. We will examine these horsemen in the future!

I’ve been meaning to write this up for a long time. Thanks for listening and commenting!

Note: Some of these links include affiliate codes that help pay for this blog. For example, buying an Amazon Kindle with this link sends a few bucks my way! But I don't write this blog to make money, and am happy to link to sites and stores that don't pay anything. I like Amazon and buy tons from them, but you're free to buy whatever and wherever you want.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | The Four Horsemen of Storage System Performance: Never Enough Cache
This post was categorized as Computer History, Enterprise storage, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The Four Horsemen of Storage System Performance: These four ugly gentlemen stand between you and your data.

Why do some data storage solutions perform better than others? What tradeoffs are made for economy and how do they affect the system as a whole? These questions can be puzzling, but there are core truths that are difficult to avoid. Mechanical disk drives can only move a certain amount of data. RAM caching can improve performance, but only until it runs out. I/O channels can be overwhelmed with data. And above all, a system must be smart to maximize the potential of these components. These are the four horsemen of storage system performance, and they cannot be denied.

The Nature of Disks

Hard disk drives are getting faster all the time, but they are mechanical objects subject to the laws of physics. They spin, their heads move to seek data, they heat up and are sensitive to shock. Storage industry insiders recognize the physicality of hard disk drives in the name we apply to them: Spindles. And there is no way to escape the bounds of a spindle.

The performance of a hard disk drive is constrained by both its physical limitations and how we use it. Physically, a hard disk drive must spin its platters under a moving arm with a read/write head at the tip. This arm slides across the media, creating a two-dimensional map of data across the disk. Hard disk drives spin at a constant speed, so data at the edge passes under the head quicker than data at the center, creating a distinctive curve of performance.

Although they are random-access devices, hard disk drives cannot access multiple locations at once. Although modern command queueing and processing allows the drive controller to optimize access, I/O operations are serialized before the drive can act on them. It takes a moment for the head to move (seek time) and the disk to spin (rotational latency) before data can be accessed, so sequential operations are much faster than random ones.

Most operating systems lay data out sequentially, beginning at the edge of the disk and moving inward. Although modern file systems try to keep individual files contiguous and optimize placement to keep similar data together, seeking is inevitable. This is the nature of physical hard disk drives.

Accelerating Disks

Disks can be made faster mechanically in one of three ways:

1. Data can be packed closer together, allowing more to be read in a given amount of time. This is a natural outgrowth of storage density improvements, and explains why today’s slowest hard disks have much better sequential access performance than the fastest enterprise drives of just a few years past.
2. Spindle speeds can be accelerated. This has a dual benefit of passing more data under the heads in the same amount of time and reducing the time it takes for the proper spot on the disk to pass the heads. Maximum spindle speed has remained constant at 15,000 rpm for decades, but the slowest drives have accelerated in recent years, with 5,400 now the minimum and 7,200 rpm now common even for portable devices.
3. Disk media can be made smaller, reducing the area the heads must travel across. Increasing media density allows for the viability of smaller disks outside volume-sensitive areas like laptops. Many high-speed enterprise drives use small-diameter media even where the outer shell remains at the standard 3.5″ form factor.

Each of these methods are limited by practicality, however. Doomsayers have been declaring that density is reaching the limits of technology for decades, and while the industry has always surpassed these expectations, it can only adapt so far. Spindle speeds beyond 15,000 rpm are impractical as well, especially since this limits the density of storage used. And geometry limits how small hard disk media can get: The spindle itself requires some space, leaving less area for data storage. In fact, it appears that today’s 15,000 rpm 2.5″ hard disk drives are as quick and small as practicality allows. Only increasing density will improve their native performance further.

Combining Spindles

Although they are quick, the mechanical limitations of hard disk drives makes them the first suspect in cases of poor storage performance. A single modern hard disk drive can easily read and write over 100 MB per second, with the fastest drives pushing twice that much data. But most applications do not make this sort of demand. Instead, they ask the drive to seek a certain piece of data, introducing latency and reducing average performance by orders of magnitude.

Then there is the I/O blender of multitasking operating systems and virtualization. Just as each application requests data spread across a disk, multitasking operating systems allow multiple applications and process threads to request their own data at once. File system development has lagged behind the advent of multi-core and multi-thread CPUs, leading to frustrating slowdowns while the operating system waits for the hard disk drive. Virtualization magnifies this, allowing multiple operating systems running multiple applications with multiple threads to access storage all at once.

The key innovation in enterprise storage, redundant arrays of independent disks or RAID, was designed to overcome the limits of disk spindles. In their seminal paper on RAID, Patterson, Gibson, and Katz focus on “the I/O crisis” caused by accelerating CPU and memory performance. They suggest five methods of combining spindles (now called RAID levels) to accelerate I/O performance to meet this challenge. Many of today’s storage system developments are outgrowths of this insight, allowing many more spindles to share the I/O load or optimizing it between different drive types.

The Rule of Spindles

This is the rule of spindles: Adding more disk spindles is generally more effective than using faster spindles. Today’s storage systems often spread I/O across dozens of hard disk drives using concepts of stacked RAID, large sets, subdisk RAID, and wide striping.

Faster spindles can certainly help performance, and this is evident when one examines the varying performance of midrange storage systems. Those that rely on large, slow drives are much slower than the same systems packed with smaller, quicker drives. But the rule of spindles cannot be ignored. Systems that spread data across more spindles, regardless of the capabilities of each individual disk, are bound to be quicker than those that use fewer drives.

Onward: Cache, I/O, and Smarts

The horseman of spindles is harsh, but he does not rule the day. There are many ways to overcome his limits and his three brothers often come into play. These are cache, which bypasses the spindle altogether; I/O, which can constrain even the fastest combination of disk and cache; and the intelligence of the whole system, which limits or accelerates all the rest. We will examine these horsemen in the future!

I’ve been meaning to write this up for a long time. Thanks for listening and commenting!

© sfoskett for Stephen Foskett, Pack Rat, 2010. | The Four Horsemen of Storage System Performance: The Rule of Spindles
This post was categorized as Computer History, Enterprise storage, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.