Data Reduction can be hazardous to your health!

I’m not a big fan of data reduction technology, yet I found myself talking compression and de-duplication all week. Between Storage Decisions and my recent posts over at SearchStorage and The Storage Community, I’ve had quite a bit to say on the subject. Funny enough, I’m not really a fan of data reduction technology for primary storage. Too often, data reduction is more expensive and difficult than just storing raw data.

You should also read Deduplication Coming to Primary Storage and Compression, Encryption, Deduplication, and Replication: Strange Bedfellows

Storage Decisions

My Storage Decisions presentation on data reduction was hilarious, if I do say so myself, even though turnout was poor at 8:30 AM on Tuesday morning. Maybe it was this “intimate” group, but I found myself really getting into the discussion. And the nods and hollers from the audience helped, too!

My basic thesis at Storage Decisions was the same as always: Don’t throw good money at technology that will have little ROI. Considering that disk capacity is incredibly cheap, and dropping all the time, data reduction doesn’t look like a great fit except in certain situations. Why spend money to reduce utilization? Why put in the effort when most primary storage data reduction technologies don’t do anything to address the “multiplier effect” of archiving, DR, and backup storage?

This is not to say that all data reduction technology is worthless. In fact, the free compression and de-duplication built into many SSDs and even some enterprise storage devices make perfect sense. I just don’t understand spending a bunch of money to address storage capacity when most applications are starved for storage performance.

You might like reading my two other posts on the subject from last week:

• Interest in data reduction methods needs to keep pace with data growth (SearchStorage.com)
• Has the Time Finally Come for Data Reduction? (The Storage Community, sponsored by IBM)

You’re Losing Me

On the other hand, I do see quite a bit of value in something many people would overlook out of hand: Lossy compression of office files. Every systems administrator knows that end-users do “stupid stuff” like embedding massive photos and videos in PowerPoint presentations and Word documents. But not everyone knows that there are technological means to address this “PEBKAC” issue.

Some office applications already automatically reduce the size of embedded content, and operating systems can do the same. One of my more popular blog posts, in fact, is a technique to create a filter to reduce the size of PDF files in Mac OS X Preview. And the Microsoft “X” Office file formats include lossless compression as well.

An application that recently caught my eye is the FILEminimizer Suite by Balesio. This inexpensive application reduces the size of Office and media files while leaving them in their native format. It re-compresses image files, reducing them to optimum size for use in presentations, documents, or printouts. A companion product, FILEminimizer Server, can be used on enterprise file servers to perform the same magic across the whole range of users.

Stephen’s Stance

Native Format Optimization (NFO) makes a lot of sense, since it addresses a common user error in a practical way, and allows capacity savings to “trickle-down” to backups, e-mail systems, and archives. But wholesale compression and the duplication of primary storage may not be worth much, especially since the cost of disk keeps dropping dramatically.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Data Reduction: the Condensed Version
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

• My State of the Art Thin Provisioning series
  • Storage is Not Getting Cheaper
  • Thin Provisioning: Attacking Storage Utilization
  • De-Allocating is the Core Issue for Thin Provisioning
  • Thin Provisioning: Playing the Telephone Game
  • Monitoring Filesystem Metadata For Thin Provisioning
  • Zero Page Reclaim: Savior of Thin Provisioning?
  • What is WRITE_SAME? Green Eggs and Ham!
  • The Bridge: Veritas Thin (Provisioning) API
  • Is TRIM Useful For Thin Provisioning?
• The Raising the Floor podcast series
  • Raising the Floor 1: Cloud Storage Gateways
  • Talking Cloud Storage Gateways With Nasuni and Cirtas
  • Raising the Floor: How the Cloud Impacts Enterprise IT
• Tech Field Day links
  • Data Robotics Is First Three-Time Tech Field Day Presenter
  • The Nerd’s Going to Have a Field Day! | The Networking Nerd
• And more links!
  • I wrote this piece for SearchStorage: Is PCIe SSD right for you? Deploying PCI Express SSD devices
  • I was interviewed for another thin story: Thin provisioning driven by technology advances, best practices
  • My friend Erin Collopy from HP is starting a blog: A new year, a fresh slate « Social Swimmer
  • Bob Plankers always impresses with his technical writing: How To Create and Measure NTFS & ext3 Disk Fragmentation
  • I loved this idea from Freedom to Tinker: Monitoring all the electrical and hydraulic appliances in your house
  • Chris Mellor spills the beans on EMC’s big announcement next week: Storage Virtualization, Unified and the VNX
  • Chris also posted a great piece about InPhase: How I watched a holographic storage company implode
  • Then there’s this Winextra bit on Microsoft’s Biggest Mistake of 2010
  • A new EMCer blog, Storage Savvy, has a nice pNFS piece: Why pNFS can be a big deal even if NFS4.1 isn’t…
  • Martin Glassborow is always worth reading: Reading the Ruins
  • Floyd Strimling makes a go at prognostication: Fun Alert: Insane 2011 Predictions That May Come True
  • Another great Bill Hill post on cloud: Cloud Crossroads
  • And Ian Foster isn’t happy with Amazon: Amazon – Masters of marketing or perfecting the art of customer dissatisfaction?

Subscribe to my Google Reader feed or follow me on Twitter to see these in real-time.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Back From the Pile: Interesting Links, January 7, 2011
This post was categorized as Computer History, Enterprise storage, Everything, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Dave called me for this piece, and I was pleased with the question.  Truth be told, there really are compelling benefits from virtualization, but most folks have been waiting for a real “must have” killer application for the technology.  In order for this tech to make the impact it should, we in the industry have to change some of our thinking:

• Storage virtualization means more than just Fibre Channel block aggregation.  There are great applications inside servers and arrays and in the NAS world, too.
• Speaking of NAS, Microsoft DFS is probably the most-implemented storage virtualization product, and just about every NAS array has cool aggregation and migration features.
• Virtualization is a feature, not a product.  HDS has seen the amazing potential for block virtualization in migration and storage flexibility, and this is just the tip of the iceberg.
• Storage and server virtualization go well together – so well, in fact, that ESG reports that 24% of those who have implemented the latter are also using the former!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Storage Virtualization: What Is It Good For?
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.