Enterprise Storage

• HDS’ HAM-Fisted Announcement did not impress, with many wondering (So Long And Thanks For All The Fish) what exactly the company had announced. But by week’s end (USP-V and Hitachi High Availability Manager) we had figured out How It Works and even had some time for jokes.
• What should be in the cloud? How about some profiling questions nobody’s asking re: cloud applications
• Want your own iSCSI array? StarWind has responded to your comments, 2TB at no cost!
• What Is a Backup? Some great discussion (Backup to the Future) and a solution: When Is A Copy A Backup?
• Howard Marks is also doing a great job with backup and archiving: See Of Backups and Archives, Storing Archival Data – Part Deux, and All Archive Data is Not Alike
• I joined Chris Evans, Marc Farley, and Greg Knierieman for Infosmack Episode #6 – HDS USP-V announcement, EMC non-competes and changes at VMworld
• ‘Nuff said: And people wonder why I hate HP, Google Could Learn a Thing or Two from EMC, Virtualization: Why Microsoft cares

Misc

• Tim O’Reilly took a look at a technology I just didn’t get, but changed my mind with Google Wave: What Might Email Look Like If It Were Invented Today?
• Want a crazy car to get your mind off of GM and Chrysler? How about the 500-hp Ariel Atom 500 V8
• A wonderfully long and detailed article: Lessons from the Vaccine–Autism Wars
• Finally, take a look at this amazing video: Louis CK “Everything’s amazing, nobody’s happy”

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Back From The Pile: May 30, 2009
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

IT infrastructure is following consumer technology out of the data center glass house and into the wide world

Dave Hitz over at NetApp poses a very interesting question: What is the ten-year trend in information technology that we are currently building to? He supplies these historical examples:

• 1982-1992: A computer on every (business) desk
• 1990s: Networking all those computers

He then goes on to suggest three ten-year trends that we might currently be living through:

1. Cloud/Outsourced Computing
2. Server Virtualization
3. Flash Memory

Although I agree on the importance of these three to enterprise IT, I don’t think they’ll be seen as the megatrends of this decade in hindsight. I suggest that, more than anything, we are witnessing a wholesale shift from information tied to place/device to information mobility. Cloud computing, server virtualization, and even flash memory are all contributors to this massive trend, along with the user-side trends of the post-PDA mobile phone, 3G data, social web services, and connected home.

What Is Mobility?

The meaning of mobility, to me, is expansive. It doesn’t just refer to taking a copy of your data with you, ubiquitous connectivity, or portable devices. Mobility is a new paradigm of computing.

• Your data no longer “sits” in one place – your data lives out there in the network!
• Your applications no longer “live” on this device or that – your applications live out there in the network!
• Your productivity environment no longer requires a particular piece of hardware – you expect to be productive everywhere on every device!

This doesn’t sound strange to the modern Internet user. We have completely accepted the role of Google, Facebook, Yahoo, Wikipedia and the rest in our personal lives. Just as they did in the early days of the PC, business people have transitioned these concepts into the professional world – witness Salesforce and LinkedIn! In all cases, we have endorsed the idea that certain types of information want to live in the cloud because it makes them better!

Once you’ve used these services, old-fashioned email, contact management, encyclopedias, maps, and the rest seem incredibly limiting. A GPS system that can’t update its maps seems antiquated, and we want it to have real-time traffic data, too. An iPod that needs to be physically connected to a PC to add music or applications is simply unacceptable. Time- and place-shifting technologies like TiVo To Go, over-the-air podcast downloads, and Slingboxes reset our expectations about availability and choice of entertainment, but they are mere symptoms of our changing perceptions. We want mobility of data, applications, and platforms, and we are getting it.

Consider two truly revolutionary platforms: the iPhone and the netbook. In both cases, we knowingly accept limitations in the name of portability, knowing that the cloud will give us what we can’t hold in our hands. These devices are limited in ways that would seem inconceivable just a few years ago: Apple has locked their platform up tighter than any in history, and netbooks are too small, underpowered, and cheap in all senses of the word. But we love them because they get us where we want to go, which is up and out!

Mobility and Enterprise IT

The concept of mobile data, applications, and devices is just as applicable to enterprise IT infrastructure as it is to personal technology. Some enterprise data must be kept close to the vest, especially where privacy laws and litigation concerns are applicable. But there is certainly a vast pool of corporate data that wants to be out working in the field! Setting this data free is the enterprise equivalent of the mobility megatrend!

Cloud computing is hype. Server virtualization is hype. Flash storage is hype. XaaS is hype. Web 2.0 is hype. But once the cloud of hype passes, we will be left with solid technologies to enable mobility and transform corporate computing. Why should corporate email have to punch through your firewall? Why should the intranet be limited to internal or VPN users? Why can’t customers interact with a (limited/controlled) set of your corporate records? Salesforce showed us that roaming users (sales teams) need greater access than most IT staff were ready to build. What if we applied the same ideas to other data types?

Many companies are already doing this. Microsoft offers a variety of internal/external services for their customers through Live (see Connect, for example). Many companies are using mail and productivity applications in the cloud from Google, MessageOne, and Zimbra. Backup and archiving as a service to mobile users is widespread (see Iron Mountain Connected and Mozy). And more and more corporate PR relies on blogs, twitter, and social networking sites. Corporate security and legal types are worried about data “escaping” from the eggshell of control they exert, but this cat is out of the bag. Enterprise IT will never be the same!

It comes down to a single core question that IT folks ought to have been asking themselves all along: What should be held internally and what should be let loose? We already “outsource” many non-core corporate functions. Sometimes we do this for cost reasons. But the most effective outsourcing decision is when a third party will do a better job, offering levels of expertise or service that an internal group could never realistically reach. We already buy enterprise software to leverage outside development (remember, this was not always the case!), so why not also buy enterprise services? Corporate-grade outsourced email, groupware, sales automation, and the like is not only more robust and less expensive than internal systems, they enable a disconnected, mobile workforce.

Today, I Was Angry

I bought a new album from Amazon, but I forgot to sync my iPhone with my laptop, so it was sitting at home when I wanted to listen to it in the car. Then I couldn’t find a colleague’s phone number because he moved to a new company and my address book didn’t automatically update. And I couldn’t review a presentation because I needed a special account to access a corporate document system behind a firewall.

These little accomplishments would have seemed like miracles just a few years ago: I remember the joy I felt ten years ago when I could read a web page offline on my Palm Pilot using AvantGo; I was amazed when I first fired up 802.11a wireless networking and could work anywhere in the office; I was gleeful to be able to take 5 GB of music with me on the train. But all this is past. Today, I want to access my portable data and work anywhere. We are in the midst of a revolution in the mobility and ubiquity of computing.

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Ten-Year Trend: Mobility
This post was categorized as Apple, Computer History, Enterprise storage, Everything, Personal, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Storage Magazine has returned with an online edition

Although the dead-tree version of TechTarget’s excellent Storage Magazine is no more, the company today released the premiere issue of its online counterpart! Available as both a web-based magazine and a PDF download, Storage Magazine Online continues with many of the same editors and writers, including Rich Castagna at the helm.

You can still subscribe to the online edition too, so you needn’t miss an issue! Existing qualified subscribers should receive an email today outlining the shift from paper to bits.

One article I’d personally like to highlight is my own piece on consolidated archiving. I put a great deal of effort into the article, and I welcome your comments!

I wish the TechTarget all the best with this transition, and urge you all to check out their work. Now if they would only add all of the back issues as downloadable PDFs as well!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Introducing Storage Magazine Online!
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

1. What are vital records versus ordinary records? I like this suggestion from another attendee: “Vital Record” is, by definition, a record without which the organization could not continue to function.
2. Would you consider Accounts Payable and Accounts Receivable receipts and related backup documentations vital records? They may be vital to some businesses and not vital to others. I suspect that these would be much more vital in the short term and much less important after the year is closed out. But I can imagine scenarios where they would be required for decades.
3. It was mentioned that it was risky to scan paper into a electronic format without a written, compact retention policy. My question is this: If you are in the process of setting policy and adding to your policy and procedures manual is it still ok to scan? My boss has pushed for me to begin the conversion process even though I haven’t managed to complete the new policy and procedures. I think this is his way of showing his superiors that we are making progress on a project that they weren’t neccesarily behind in the first place. I would worry that you’ll waste effort scanning documents that shouldn’t be saved. But it’s better to duplicate and waste some effort than to not save at all. So scan away! But get that policy done too!
4. What about TIFF storage? Is that still viable? and For long term storage, why isn’t tiff G4 an option? TIFF is certainly a widely-used format for images, and G3 compressed TIFF will probably be readable a century from now (assuming we’re still here!) Plus, TIFF is (usually) lossless. However, it’s very flexible, and a TIFF file can contain lots of different data – you could easily create a TIFF that won’t be readable next week! Also, there have been some patent/royalty arguments about TIFF and related technologies. Finally, TIFF files tend to be huge (since they’re lossless) compared to other (lossy) formats. I like open standards like PNG better since they’re fully documented and portable, but TIFF is probably a decent choice.
5. When storing paper does the newer printing technology – inkjet, laser etc. match the archival quality of older paper and inks? and What do we need to know about the kind of paper that should be used for long term archiving? I’m concerned that newer technologies, including printer tech, won’t last like old ones. I have impact-printed pages from the 1980s that look like new, and laser-printed pages from the 1990s that are clear and clean. But my old inkjet pages are very faded. Now, this could have to do with the paper I chose, and it could be that old inkjet isn’t a predictor of new inkjet, but I’m skeptical. HP claims that their Vivera inkjet ink will last “108 years”, which seems oddly precise. I guess they took a Photosmart printer back to 1900 and printed out some pages? In all seriousness, be concerned about any claims like this that cannot possibly be tested. And consider handling as well – proper temperature, humidity, and storage will make documents last much, much longer!
6. Dry caskets of nuclear waste are dangerously radioactive for over 10,000 years. Consider the “pancake” drive: titanium disks written with an ion beam in both analog and digital. Even without ASCII, it can be read with an electron microscope. Now that is longterm storage. Do you really trust current formats for 50-100 years? Think of how much change there has been in the last 50 years. and If There was not a question of Blu-ray discs lasting for decades , what is your opinion on the technology as a viable archive solution? I’m skeptical of all unprovable longevity claims, since media has never lived up to them in the past! If I had to guess, I’d say that mechanical devices like disks and tapes will have more problems than plain discs, and that optical will last longer than magnetic. So I bet Blu Ray media will outlive most other current mainstream media.
7. If it isn’t vital you should not convert. On another project (our archives) I am trying to save and prevent any further damage to the records because they are currently stored in a basement with water leaks, rats, and no climate control. Is this instance how do you weigh the vital and non-vital? Specifically, I was asked to bring in outside companies to see what they had to say and get quotes after my warnings and price estimations were rejected. Any suggestions on how to get across the importance of not allowing these records to just rot? Rats don’t make great records managers! Seriously, though, I’d say that something is seriously out of whack when records are stored like this. Maybe you should come in to work one day and claim that all of the records are destroyed and see what reaction you get? But be sure to have another job lined up before you pull this stunt…
8. I just want to point out there is a whole profession that deals with this every day – the Archivist. Information can be found at www.archivists.org – they even have a group of Business Archivists that you can talk to. Thank you for that link – I’ll look into it!
9. Can you provide more information on what Open Standard Format is? I like standards that are well documented in public places – ASCII charts are freely available and widely distributed, for example. These are a message to future generations – when you encounter this file, decode it with this chart! But proprietary formats concern me, especially when their documentation isn’t freely available and open.
10. When will the presentation slides be available? The webinar and slides should be posted at AIIM’s site within two weeks.

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Thoughts on Long-Term Archiving
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

First, consider whether to store records as atoms or bits. You can convert paper to data or vice versa, and there are pros and cons to both:

• Properly handled physical (paper or film) records should last for hundreds of years and can remain readable without software or devices. But they’re hard to search (you need an index), and paper is bulky, heavy, and difficult to work with.
• Digital records can either be stored offline or kept “alive,” but questions remain about their long-term reliability and readability. Living records can be easy to search and use, and digital storage can be very space-efficient, but data tends to pile up “out of sight.”

Long-term storage of records on physical media is proven – think about papyrus, tablets, gold or nickel discs, film, and paper. But will digital media fare as well? Data tapes and disks can degrade over time, and manufacturer reliability specs are based on accelerated testing, not actual experience. Regardless of media type, careful handling can extend media life.

But will you still be able to read it? Tapes and optical disks require additional hardware to read, while disk drives are paired with their read heads. Software applications are needed to read and interpret data (backup, archiving, compression, encryption, deduplication, database) as well. What about content format? Should you use ASCII, XML, PDF/A?

I’ll be presenting a webinar on this topic tomorrow, Wednesday, December 3, at 2:00 PM Eastern time. Register on-line at the AIIM web site and join me for the discussion!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Long-Term Versus Longer-Term Archiving
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Although my email archiving session always attracts a smaller crowd, they are all a dedicated bunch. One pertinent suggestion from an attendee was to ingest PST files into a special separate archive in order to ensure that messages recovered from it are treated with the proper skepticism. Questions after the session focused on the trick of engaging legal and business people in the decisions around email policy, truly a challenge. I suggested that an on-site mini-seminar for the relevant folks might help to break the logjam and illustrate the issues, something that I would be happy to arrange!

My storage virtualization session was once again placed in the main room, and a much larger group attended it. I was interested to hear just how great the impact of VMware’s VDC-OS had been. In just a week, a dozen or more folks in the audience had heard, comprehended, and strategized about the concept. It’s really that big! Others were very interested in the topic of green metrics for data center usage. How does one monitor and report the real “green” savings (power, carbon, cooling, space) for a virtualized environment? Although storage greenness is debatable, the savings from a virtualized server environment are real, and these often bundle in some of the storage numbers, too.

These topics are top of mind to me as well, and I will continue to investigate (and speculate) about them in the coming year. If you missed the show (or the handouts), I will be posting them here soon! Get my email address or head to LinkedIn by clicking the links in the sidebar (at top left).

Watch this space, and consider coming to my virtualization seminar in Charlotte on October 21 or to the Storage Decisions show in San Francisco, held November 17 to 19.

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Storage Decisions New York 2008 Feedback
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

• How to improve the receptivity of the e-mail archieve message to management to overcome the biases you mentioned? Consider what management is interested in most: They don’t necessarily care about email archiving, but they will respond to well-presented solutions to business objectives like cost avoidance for litigation and e-discovery or future reduced spending on IT infrastructure (storage).
• How do you get things moving when Legal is sponsor and can demonstrate that one lawsuit discovery failure will be the same cost as the project? This would be an ideal scenereo. Legal probably already understands the high cost of e-discovery and potential risk of adverse judgement. Now demonstrate the ways that email archiving can help. But be realistic – an email archiving app will help with e-discovery, but is definitely not a complete solution. You will still need policy and process for hold and discovery, and may even need additional e-discovery tools.
• Have you seen standard or default retention periods for email, and how are they determined? I would not want to suggest standard or “rule of thumb” retention periods. In my experience with many organizations, I have seen such a wide variety of carefully considered retention schedules that I have come to believe that there is no such thing as a standard. However, I would like to suggest not making a too-short maximum (say, less than three months) or else you will drive end users to underground archiving to get around the policy. Such a short retention period can work for certain classes of messages (company picnic, lunch invites, mass mailings), but I can’t imagine a business where no one would need to keep messages longer. Try to be realistic in setting your retention, balancing productivity with compliance. Setting retention periods is an exercise in consensus building, with representatives from IT, Legal, HR, Compliance, and others each contributing their own needs. It’s doable, though – I do it for a living!
• How many years back do people hold on to mail typically? Most organizations lack standards or policies for retention, so their typical retention varies based on employee actions. When they do have retention policies for email, they tend to classify messages (by organization or user, keywords, or through user action) into a few “buckets” with appropriate retention for each. Most have short retention (3 to 9 months) for some content, and longer retention (on the order of a few years) for other types. Most also have a “keep forever” category for special record types.
• Do you have an opinion on whether journaling is better or not for meeting legal requirements? Most legal groups desire the most complete set of messages possible. Both journaling and log shipping should meet this need.
• Have you seen much use for email archiving for supporting Audit to prove that an event was reviewed and approved? Many organizations use email as part of their workflow. In these cases, maintaining a complete set of messages is critical for audit and compliance. Many certainly do use email archives for this purpose, while others prefer to route such approvals to other content management systems to maintain these records.
• Is there a legal argument to be made that getting rid of pst files and having a central archive will help with privacy law issues? Decentralized personal archives like PST files are a privacy nightmare waiting to happen – they are highly mobile, can be copied and viewed easily, and tend to “live” on theft-prone devices like laptops and portable drives. Replacing PST files with a centralized archive has many many benefits, and the reduced risk of privacy breaches is certainly among these. However, most mail clients still maintain offline caches of mail server (and archive) data, so simply replacing PST with an archive will not elimate the privacy risk.
• You might want to mention that accelerating volumes of stored email are not necessarily bad because it is electronic and therefore searchable. Offline stuff dumped into file servers with no meta data is actually more of a nightmare because no one knows who created it or why it is needed (or not). Adding structure to electronic stored information is always valuable, and email archives certainly do make that data much more organized and searchable, along with enabling retention policies to be applied (assuming they exist!) Unstructured data on file servers is indeed a nightmare, and can pose a serious risk. There are other tools to tackle this type of data, however, and they might do a better job than relying on an email archive as a primary repository of critical records.
• Can you elaborate on user training needs? I was under the impression that changes to the end user were minimal. The amount of impact an email archiving system has on end users can vary from minimal to significant, based mostly on how much user interaction is desired for classification. However, even the most unobtrusive system is likely to change certain aspects of user behavior, from PST files to searches to stubbing and deletion of messages. Therefore, training is probably needed with every email archiving implementation.
• Litigation holds can last for YEARS !! Tape is DEAD especially to try to rebuild mail boxes. Tape remains non-dead for certain applications like backup, despite years of prognostication. However, you are correct that e-discovery of data on tape is incredibly difficult and time-consuming for most systems. There are tools to help, but an online archive is vastly more flexible and speedy than recovering backup data from tape. I’ve seen backup-sourced e-discovery costs rise into the millions of dollars, too!
• How important is classification in an archive solution? The importance of classification depends on the business objectives that an archive is meant to serve. However, most implementations will require at least some message classification mechanism in order to determine how long different messages should be retained, since a one-size-fits-all retention schedule is unlikely to meet anyone’s needs. Therefore, it is important to begin developing an overall record retention policy for email when implementing an email archive, and to consider how messages will be classified. Classification of email messages is a complicated topic – worthy of its own webinar at least!
• We are a globally dispersed organization which increases the complexity and cost. Do you feel implementation is an all or nothing scenario or does it make sense to focus on domestic and large campuses and then in a second phase chase funding for remote and international sites? Dispersed organizations face many unique technical challenges, especially when implementing email archiving. Certainly it is better to have some archiving than none at all, but unless your retention requirements are also segmented by geography, focusing on only a single location can be risky. I’d definitely suggest starting with the biggest fish, but you will probably have to get to everyone fairly soon afterward.
• Are there solution that do both email and file data (shared folders) archiving? Yes, some email archiving products also do file archiving. However, some choose to implement these functions separately to keep one from standing in the way of the other – it’s more important to have some archiving than to wait and wait for a complete solution.
• Not to mention, extremely short retention periods for email messages will NOT allow an enterprise to comply with litigation hold needs… Once a litigation hold is ordered, all retention schedules for covered data must be suspended. Therefore, a short retention schedule does not necessarily preclude compliance with a legal hold order. However, unreasonably short retention will not look good in court!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Answering Your Email Archiving Questions
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The answer to this question can vary greatly based on conditions, but it is always a solvable problem.  Email is a unique application, and is especially interesting when it comes to litigation and e-discovery, so creating a corporate policy for retention, and implementing it, is critical.  The quandry often boils down to a simple question: Is the email system a repository of business records or a temporal system for communication?  You would never implement a policy for retaining cardboard boxes or manilla folders, because you never know whether it contains an important memo (or a MacBook Air!)  But email could be considered the same way – it’s a mechanism not a data type!

Another key point is that IT often feels that it cannot unilaterally implement a retention policy without outside involvement.  This is true, to a point, but IT had better start the discussion, or they’ll be asked to implement an unreasonable (or technologically unfeasible) policy sooner or later.  Strange as it may sound, a policy that reflects the functionality of email archiving systems might be a good place to start, since this is all you can really implement anyway!

This topic is discussed in far more detail in my new whitepaper (sponsored by Symantec), How Long Should Email Be Saved? Download a copy from their web site to see more information about what a retention policy looks like, the impact of various laws and regulations, and how to get a retention project off the ground!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | How Long Should Companies Retain Email?
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.