Stephen Foskett, Pack Rat

Based on Floral Matryoshka by BrokenSphere/Wikimedia Commons

My February 2003 column for Storage magazine focused on the surprising difficulty of measuring storage utilization. I wrote:
 

“A true measurement of utilization would reflect every layer of usage metrics - from raw disk in a shared array to used storage within files. Raw storage for each new frame of reference is contained within the used storage measured above it, so low utilization is compounded as we move deeper into the stack.”

In that column, I suggested that utilization of any resource was based on just three metrics:

1. Raw
2. Usable
3. Used

But this is confounded by the frame of reference being measured. It’s trivially simple to determine the raw, usable, and used capacity for a storage array, server, or database. But what happens when one tries to measure storage utilization all the way through the stack? Continue Reading »

In an article for Enterprise Storage Forum, Paul Shread comments on the positive reviews that various deduplication technologies got at Storage Decisions from analysts and end users. He suggests that less than 10% of attendees were using deduplication already, but that others were inspired by their experience and would be using it soon.

Paul goes on to quote me, saying I “didn’t think primary data de-duplication technology was ready for prime time just yet.” I absolutely did say these words, but I am not sure if my point came across.

I’ve recently expounded about the benefits of deduplication technology, but have warned that it might not be all it’s cracked up to be in primary storage environments. By “primary” I mean those storage environments serving mission-critical applications. Although dedupe works great for backup and archiving, the random I/O, low latency, and high throughput of primary storage (and especially virtualized servers) might be too much for current systems. And as of now, only NetApp, Riverbed (soon), and startups greenBytes (see my story) and Ocarina (more on them another time) were willing to go on record with me as supporting deduplication of primary storage.

So what I meant was that deduplication is not yet ready for prime time in primary storage applications. No one should hesitate to use the technology for backup or archiving at this point, but make sure you do a thorough evaluation of the specific product you are selecting to make sure it delivers the performance you require!

This is a follow-up to my story, De-Duplication Goes Mainstream

Although deduplication of storage is nothing new, with Data Domain and other making hay with the technique for years, it has never been ready for prime time - reduction of active primary storage applications like email and databases. Instead, deduplication has been relegated to second- or third-tier status, deduplicating archives and backup data. But change is in the air, and deduplication vendors are starting to bustle towards the bright lights of primary storage.

Continue Reading »

I’ve long hollered that ZFS is a real storage revolution in the making, but recognized that it still had a way to go before replacing UFS, HFS+, and most volume managers. Well, a little Rhode Island company called greenBytes comes out of stealth today to announce that they’re doing just that - taking the solid ZFS core and adding some serious enterprise storage features to it. And they’re rolling the lot into a multi-protocol storage array using commodity (Sun Thumper) hardware. These guys have cooked up a seriously interesting entrant in the storage market, though I can’t say much for the decapitated camel-case spelling of their (already in use) name!

Continue Reading »

Watching the announcements coming out of EMC World today, one bit of jargon stuck out at me:  The EMC bloggers are starting to refer to “data deduplication” as “3D”.  I had never heard this terminology before yesterday, but the EMCers are all using it, so it must be a popular term inside that company.  So I’m just giving my readers a heads-up: 3D is deduplication, at least at EMC.

Lotsa people use and love data de-duplication (to hyphen or not - that is the question) technology for backup and archiving, but it looks like this tech is about to bust out for primary, live, real, mainstream, what-you-are-using-to-store-your-stuff storage. Big news!

First up was the company now officially called NetApp, which officially blessed the use of their advanced single instance storage (A-SIS) de-dupe tech for primary storage use as of now. They added block-based de-dupe way back when, and even said people could use it for primary storage, but it doesn’t look like that happened much. So the new release of OnTap 7G includes performance tweaks and more powerful blessing from the company, making this the first play for primary storage de-dupe that I know of. Some have been (predictably) skeptical of Network Appliance NetApp’s A-SIS technology, but others appreciate the results…

Next is the smarties over at Hifn, who make super (de)duper storage controller hardware that often finds its way into OEM products. On Monday, the company announced availability of a card that can handle de-duplication, compression, and encryption, though not all three at once and only at 250 megabyte per second speed. And it looks like a partnership with Freescale Semiconductor will give a big performance boost later in the year, enabling gigabyte per second throughput and all three functions at once. This could really be something, with upstart OEMs launching de-duping storage arrays for little bucks before Storage Decisions comes to San Francisco!

Image by Gila Brand, used according to Creative Commons 2.5 license

Symantec announced availability of the latest NetBackup revision today, version 6.5. This release is nearly complete in its buzzword-compliance, with enhanced support for VTL and backup to disk, data deduplication, CDP, LAN-free backup, SharePoint and Exchange, and even VMware! What’s the matter, Symantec, was Thin Provisioning not ready for release? How about green computing? Holographic storage? Yes, I jest…

Seriously, you gotta cheer when a “big gorilla” app like NetBackup adds this kind of technology, though. CDP and deduplication were great ideas but needed n application to focus them, and data backup is an excellent place to apply them.

And although the press release doesn’t highlight it, the application-specific recovery enhancements look especially tasty to me. NetBackup leverages VCB in VMware but can do file-level restore, which is awesome. And it can also do document-level or full-database restore in SharePoint from the same image. Over in Exchange land, it claims to be able to restore from snapshots instead of the backup image, speeding (all too frequent) recoveries.