Deletion of data is not a high priority for most IT shops, but this ought to change

When was the last time you deleted data? Even at home, where we have autonomy and authority over our own data, many of us are digital pack rats. But at work? Never! No one ever deletes anything! Let’s talk about why this is.

Retention vs. Deletion

Just about everything we do in IT infrastructure is focused on retention. We back up our data and implement other data protection tools like snapshots and mirrors. We might also archive data so that the General Counsel can place legal hold on it, as well as perform data discovery during litigation. And then there’s the whole field of data security, focused on locking people out of data, keeping it intact and un-viewed.

But what about deletion? Almost no effort is put towards removing data, though the rapid growth of storage might lead one to think this is a key area for IT. We certainly could put some effort on revision control, and especially deleting drafts and outdated data. We could easily expire content that was no longer needed, if only we had some way to know that. And we’ve talked a lot about secure deletion, even though we hardly ever actually perform that task except when moving to new physical storage hardware.

The greatest challenge for deletion is a simple question: What should we delete and when?

IT can not answer these questions. They must be put to the business people who really own the data. Without permission and buy-in, IT is in serious legal peril when it comes to deleting data: Any deletion must be in accordance with policy and must be legal, that is there is no legal or regulatory hold on it. And there is no way most IT staff feel empowered to do that!

Some Data Should Be Deleted

Certainly, not all data should be saved. There is “low-hanging fruit” in every storage estate that can and should be deleted:

• Ephemeral copies – Drafts, temporary data, working copies
• Time-limited projects – Third-party or client data, test and development
• Expired data – Retention policies that are expired and no legal hold remains
• Legally required – Data that isn’t yours, or that legal demands deleted

Tackling these data sets is much easier to tackle than cleaning out primary data stores, since it doesn’t require as much sifting and sorting: These data sets can often be identified programmatically! If you have data sets like these, this is the ideal place to start a deletion effort.

Delete on Demand

Regardless of the type, however, IT should not delete data without direction. It is perilous in today’s legal environment to destroy data without a policy directing that action. So we should continue to focus on retention for most data, while we work with legal to determine which data can be deleted and come up with a process for approval.

But it’s important to start offer a deletion-friendly environment for certain data types. Such a storage system would reduce the difficulties associated with data deletion. Really, only an integrated solution can truly delete data:

• It must maintain custody of data from start to end and not allow it to leak all over the organization
• It must be accessible since any restrictions tempt users to create “working copies”, thus thwarting deletion
• It must be secure – Data must always be encrypted to avoid remnants on media
• It must be protected so data will not spread to external systems and sites

Stephen’s Stance

Data deletion is a real problem for most IT shops. I’m just getting my head around the ramifications, and continue to look for an ideal deletion-friendly storage solution.

If you’re interested in the topic of data deletion, I recommend joining me for a webinar on the topic on Wednesday, April 13. Sponsored by Nasuni, I will discuss the dilemma of deletion and CEO Andres Rodriguez will weigh in about the capabilities of his cloud storage solution. Register now!

Note: Nasuni is sponsoring this webinar, but the content was created by me. This blog post is intended to engage my audience in discussion of the subject, and is not a paid promotion or advertisement.

Image credit: “Delete” by blmurch

© sfoskett for Stephen Foskett, Pack Rat, 2011. | The Deletion Dilemma
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Webcast: Automating Policy With Email Archiving Technology
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Although my email archiving session always attracts a smaller crowd, they are all a dedicated bunch. One pertinent suggestion from an attendee was to ingest PST files into a special separate archive in order to ensure that messages recovered from it are treated with the proper skepticism. Questions after the session focused on the trick of engaging legal and business people in the decisions around email policy, truly a challenge. I suggested that an on-site mini-seminar for the relevant folks might help to break the logjam and illustrate the issues, something that I would be happy to arrange!

My storage virtualization session was once again placed in the main room, and a much larger group attended it. I was interested to hear just how great the impact of VMware’s VDC-OS had been. In just a week, a dozen or more folks in the audience had heard, comprehended, and strategized about the concept. It’s really that big! Others were very interested in the topic of green metrics for data center usage. How does one monitor and report the real “green” savings (power, carbon, cooling, space) for a virtualized environment? Although storage greenness is debatable, the savings from a virtualized server environment are real, and these often bundle in some of the storage numbers, too.

These topics are top of mind to me as well, and I will continue to investigate (and speculate) about them in the coming year. If you missed the show (or the handouts), I will be posting them here soon! Get my email address or head to LinkedIn by clicking the links in the sidebar (at top left).

Watch this space, and consider coming to my virtualization seminar in Charlotte on October 21 or to the Storage Decisions show in San Francisco, held November 17 to 19.

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Storage Decisions New York 2008 Feedback
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

In the mean time, I must say how completely impressed I am by Chuck Hollis’ blog postings.  Employing guys like him is one of the reasons why EMC continues to dominate the storage market even as challenger after challenger takes them on.  Sure, they’re not always right.  And sure, Chuck’s as guilty of anyone of a little blogketing.  But who isn’t?

Anyway, let’s take these two recent posts:

“As we look at our portfolio of services going forward, where are we going to need help getting good?  And who can help us get good without keeping us dependent?

And, finally, who are you going to trust to do this for you?”

So asked Chuck yesterday in his piece about outsourcing and services.  The emphasis is his, and I left it because I think it’s very relevant.  As I’ve often said, people hire consultants for two reasons: Either they lack time and focus or they lack experience and knowledge.  It makes sense to hire a consultant from EMC (or HDS or HP or whoever made it) to help you through the thickets and learn to really utilize a piece of equipment.  Anyone can use a piece of hardware, but utilizing it (putting it to a practical purpose) takes knowledge, and an insider or expert is best.

Now for Chuck’s piece from today on information governance:

“Bottom line: these questions of “how do we manage information at a corporate level?” are all over the place  if you look around, and more are coming every day.

And, rather than try and address them individually in an ad-hoc manner — with limited participation, measurement and evolution, the idea is to create a role of information governance function within the organization.”

This is the kind of work I do every day at Contoural.  It’s difficult – but that’s why we get hired.  Once again, companies lack the knowledge and focus to set up good data management policies.  But of course, Chuck’s right that the primary effort and input comes from inside – I’m a coach, not a player when it comes to governance work.

We in IT like to try to pretend we know everything because we would never want the rest of the business to see the emperors clothes as it were.  Everyone comes to us as the end-all knowledge guys, and we hate to disappoint them.  But maybe it’s time we admitted that we don’t always have all the answers and that we could use some help.

So there we are – admitting that we don’t know the answer and that we need help.  It’s hard for IT to do it, and harder for consultants.  But we’re all people, and if we’re brave enough to admit our limitations we usually find out that the rest of the world respects that.

© sfoskett for Stephen Foskett, Pack Rat, 2007. | Chuck Hollis Gets It!
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.