• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Data Reduction: the Condensed Version

Data Reduction: the Condensed Version

September 22, 2011 By Stephen 1 Comment

Data Reduction can be hazardous to your health!

I’m not a big fan of data reduction technology, yet I found myself talking compression and de-duplication all week. Between Storage Decisions and my recent posts over at SearchStorage and The Storage Community, I’ve had quite a bit to say on the subject. Funny enough, I’m not really a fan of data reduction technology for primary storage. Too often, data reduction is more expensive and difficult than just storing raw data.

You should also read Deduplication Coming to Primary Storage and Compression, Encryption, Deduplication, and Replication: Strange Bedfellows

Storage Decisions

My Storage Decisions presentation on data reduction was hilarious, if I do say so myself, even though turnout was poor at 8:30 AM on Tuesday morning. Maybe it was this “intimate” group, but I found myself really getting into the discussion. And the nods and hollers from the audience helped, too!

My basic thesis at Storage Decisions was the same as always: Don’t throw good money at technology that will have little ROI. Considering that disk capacity is incredibly cheap, and dropping all the time, data reduction doesn’t look like a great fit except in certain situations. Why spend money to reduce utilization? Why put in the effort when most primary storage data reduction technologies don’t do anything to address the “multiplier effect” of archiving, DR, and backup storage?

This is not to say that all data reduction technology is worthless. In fact, the free compression and de-duplication built into many SSDs and even some enterprise storage devices make perfect sense. I just don’t understand spending a bunch of money to address storage capacity when most applications are starved for storage performance.

You might like reading my two other posts on the subject from last week:

  • Interest in data reduction methods needs to keep pace with data growth (SearchStorage.com)
  • Has the Time Finally Come for Data Reduction? (The Storage Community, sponsored by IBM)

You’re Losing Me

On the other hand, I do see quite a bit of value in something many people would overlook out of hand: Lossy compression of office files. Every systems administrator knows that end-users do “stupid stuff” like embedding massive photos and videos in PowerPoint presentations and Word documents. But not everyone knows that there are technological means to address this “PEBKAC” issue.

Some office applications already automatically reduce the size of embedded content, and operating systems can do the same. One of my more popular blog posts, in fact, is a technique to create a filter to reduce the size of PDF files in Mac OS X Preview. And the Microsoft “X” Office file formats include lossless compression as well.

An application that recently caught my eye is the FILEminimizer Suite by Balesio. This inexpensive application reduces the size of Office and media files while leaving them in their native format. It re-compresses image files, reducing them to optimum size for use in presentations, documents, or printouts. A companion product, FILEminimizer Server, can be used on enterprise file servers to perform the same magic across the whole range of users.

Stephen’s Stance

Native Format Optimization (NFO) makes a lot of sense, since it addresses a common user error in a practical way, and allows capacity savings to “trickle-down” to backups, e-mail systems, and archives. But wholesale compression and the duplication of primary storage may not be worth much, especially since the cost of disk keeps dropping dramatically.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • How To Connect Everything From Everywhere with ZeroTier
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Powering Rabbits: The Mean Well LRS-350-12 Power Supply
  • Introducing Rabbit: I Bought a Cloud!

Filed Under: Enterprise storage, Personal, Virtual Storage Tagged With: Balesio, compression, data reduction, deduplication, FILEminimizer, IBM, SearchStorage, Storage Decisions, The Storage Community

Primary Sidebar

The less the prudence with which others conduct their affairs, the greater the prudence with which we must conduct our own

Warren Buffett

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    Datacenter History: Through the Ages in Lego

    October 22, 2013

    Frequent Flier Kung Fu for Novices

    March 12, 2012

    A High-Tech Water Heater? Yep! Introducing the A. O. Smith Vertex

    November 15, 2012

    Electric Car Over the Internet: My Experience Buying From Vroom

    November 28, 2020

    Replacing Google Reader With Feedbin and Reeder

    May 5, 2013

    The Rack Endgame: Converged Infrastructure and Disaggregation

    September 19, 2014

    The Prime Directive of Storage: Do Not Lose Data

    December 12, 2014

    Ten Terrible Apple Products

    June 14, 2012

    GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

    April 6, 2019

    The Best Mac OS X Terminal Font: Glass TTY VT220

    October 6, 2015

    Footer

    Legalese

    Copyright © 2022 · Log in