• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Sailing the Titanic (Why We Need ILM and Then Some!)

Sailing the Titanic (Why We Need ILM and Then Some!)

July 24, 2007 By Stephen 2 Comments

Without getting into the debate on blogketing (I’ll save that for another post), I was pretty impressed by Chuck Hollis’ recent post on ILM. I think he’s made a good discussion of the wherefores of ILM, and maybe counteracted a bit of the prevailing anti-ILM argument.

I’ve been in the trenches on storage content (aka data) for a long time. I, too, have often reverted to the old “gigs of MP3s and porn” argument from time to time. But I’ve done enough filesystem assessments at real companies to realize that that’s not really the norm. In fact, I’ve rarely found much porn, music, video, or jokes on full-up corporate file servers. And I’ve analyzed enough storage environments to know that, while file servers are big, they’re not normally the majority user of storage in large data centers.

On the contrary, most enterprise storage is taken up by business applications, though not necessarily critical data. Email, backup, and certainly user file servers are big space users. But give me a few Oracle instances, source code repositories, or image processing servers, and watch those applications shrink in significance.

No matter what the application, though, the real issue with storage growth (and ILM) is the (in)ability of IT managers to do anything about it. Let’s say we had permission to delete really inappropriate data, which is not a sure thing. Would we IT folks even be able to recognize it? How would we locate it? Can we even view user files without violating user trust, company privacy policies, or even laws? Many countries (yes, not all data is in the USA), regulate access to data even inside a company.

Now let’s move into grayer areas of “unnecessary” corporate data. Many storage administrators can’t even name the applications that take up all that space, let alone understand the intricacies of the data under management.  To make a timely (and tired) Harry Potter analogy, IT are the house-elves of the business – powerful but subservient, with little input into what happens above and around them.  I’ve talked to business people who don’t want IT to have any input, relegating them to order takers and laborers.

This is a dangerous slide, however.  Lots of people have the capability to take IT orders and keep the lights on,  a realization that leads to outsourcing.  IT pros must prove their worth to the business in order to remain relevant and irreplaceable!

ILM is one way to do that.  To get back to Chuck’s post, we need to take the reins and try to understand data better.  We need to pick certain applications that lend themselves to automated data classification and tiered storage and try to get them under control.  Email is a great candidate, and that’s why email archiving applications have taken off recently.  File servers are coming along, too, especially with file virtualization in the ascendancy.

I’m particularly excited about what a smart IT manager I know called the “second wave” of SRM tools.  Rather than just collecting stock metadata (age, name, owner, etc), the latest filesystem scanning tools look inside a file, trying to better classify them.  Let’s say 1/4 of your file server is made up of Microsoft Word, Excel, and PowerPoint documents.  What can you do about that unless you can identify which are critical and which are not?  Each business will have its own criteria, and you need a flexible tool to scan them all and report back to you before you can “ILM” them.  That’s what lots of software vendors are currently working on, and though we’re at an early stage still, the results are promising.

Sadly, though, we in IT may soon find that we just can’t delete anything.  Even totally banned content like porn could be critical to a legal case against an employee,  and it won’t be long before we are expected to keep everything that shows up on our servers for a very long time.  Most companies have policies for hardcopy document retention, and many are currenyly diving into the world of data policy as well.  The default policy may be “keep until we decide what to do with it”, and this could cause the current trend of storage growth to accelerate!

If we can’t delete data, we will be forced to sail the Titanic rather than sink it.  Small companies can benefit most from the falling price of storage, since the entire storage footprint for a little shop is often under a terabyte.  But larger organizations will find that they need to start tiering their storage, and quickly in order to keep prices under control.

And then there’s green storage.   Again, Mr. Toigo makes the very valid point that the problem is in the business, not in the hardware we use.  But if we can’t do anything about data growth for the time being, we had better start tackling the technical challenges we face.  I’ve talked to many IT folks who are very worried about data center space, as well as the terrifying trio of heat, power, and cooling.  For them, green technologies are no laughing matter!  If you can’t get any more power, you have to lower your per-GB requirement and quickly.

It’s easy to say “understand your data and delete some”, but hard for IT pros to  actually do it.  Until we can tackle the strategic issue of data growth, we’ll have to continue fighting the tactical problems of storage.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • How To Connect Everything From Everywhere with ZeroTier
  • Tortoise or Hare? Nvidia Jetson TK1
  • Scam Alert: Fake DMCA Takedown for Link Insertion

Filed Under: Enterprise storage Tagged With: blogketing, data classification, data growth, green storage, ILM, SRM, tiered storage

Primary Sidebar

It is often easier to ask for forgiveness than to ask for permission.

Grace Hopper

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)

    August 4, 2011

    Microsoft: Kill the Craptops Before They Destroy Windows!

    January 7, 2013

    Defining Failure: What Is MTTR, MTTF, and MTBF?

    July 6, 2011

    MacBook Users: Encrypt Your Drive with OS X FileVault! It’s Easy and Free!

    December 20, 2012

    SMB 3 is Going to be Huge, in both Scope and Impact

    May 6, 2012

    Put that camera away and enjoy the view!

    April 11, 2012

    We Live in the Future: Robotic Cat Litter Boxes!

    May 8, 2010

    Faster Ethernet Gets Weird

    June 19, 2015

    It’s Time To Speak Out Against Sexism In IT Recruiting

    May 6, 2013

    Replacing Google Reader With Feedbin and Reeder

    May 5, 2013

    Footer

    Legalese

    Copyright © 2022 · Log in