Join me in New York for Storage Decisions, September 19 & 20

Tomorrow, I will be in San Francisco for TechTarget’s Storage Decisions conference. This show does a good job on the editorial side, suggesting timely topics and bringing in folks like Dennis Martin, Mark Staimer, and Jon Toigo. I will have two presentations on data reduction and storage virtualization in the main conference track – both are updated from my New York sessions. Registration is free for qualified end-users, and I urge you to attend.

Reclaim Capacity with Data Reduction for Primary Storage

I have updated the session with additional information on thin provisioning and compression, as well as expanding the slides to reflect many of the questions and comments I received in New York. The end result remains the same: I’m not sold on data reduction for primary storage as a product and recommend tackling data growth if at all possible. If it’s completely impractical to delete data, there are a few products that work well.

Depending on which industry study you read, most companies are wasting anywhere from 30% to 50% of their installed disk capacity, which translates into thousands of dollars spent with no effective return on investment. Storage vendors are beginning to provide tools that can help storage managers make the most of the disk they have installed. For example, data reduction for primary storage borrows data deduplication technology developed for backup and classic compression algorithms to help squeeze the air out of nearline and primary data and reduce its footprint. This session’s topics will include an overview of data reduction technologies and where they will have the greatest impact, what key storage vendors are offering in data reduction and an update on the major players, and the consequences of using primary data dedupe along with dedupe for backups. We’ll also look at the potential for vendor lock-in and consider why we’re reducing data in the first place.

Topics include:

• Introducing data reduction technologies
  • Compression: How it works and where it’s found
  • Deduplication: From single-instancing to variable block
  • Application-specific: Cracking open files
• Overview of data reduction products
• Where to use them
  • The capacity conundrum: Store less and reduce utilization
  • Ideal applications: Justifying the cost of data reduction
  • Side effects: Considering the impact on backup, replication, I/O workload and vendor lock-in

Storage Virtualization: Who’s Doing It and Why

My storage virtualization session has been massively tweaked, including updated information on standalone virtualization products as well as a more in-depth discussion of successful and failed use cases. This session presents the conundrum of why server virtualization has been so successful while storage virtualization has failed for over a decade. I believe this is due to the problems these products try to solve. Consolidation of resources and reduction of administrator effort are noble goals, but not really compelling in the long term. Unless some real is this value can be extracted, storage virtualization will continue to be a failed product.

Storage virtualization has been around for decades and, although research indicates that 70% of companies have already virtualized at least some of their installed block or file storage, most remain unaware of this technology. Grandiose schemes for comprehensive virtual SANs have given way to more practical host- and array-based virtualization technologies, and server virtualization has created a new opportunity to create a pool of storage. This session will look at the current state of storage virtualization, how to quantify its benefits and describe which approaches are best for particular environments, and also cover how storage virtualization compares to private storage clouds.

Topics include:

• Defining storage virtualization: What it is and where to find it
  • Abstraction of storage resources
  • Tiered storage
  • Flexibility
• Popular approaches to storage virtualization
  • SAN controllers
  • File virtualization
  • Volume managers
• The pool, the hypervisor and the cloud
  • The impact of server virtualization
  • Is this a private cloud?

Registration

You can see the future from here!

To register for Storage Decisions San Francisco, just go to the TechTarget registration page.

Disclosure: TechTarget pays my expenses to attend and present at Storage Decisions, and has for many years. But they don’t pay me to present and I own the copyright on my session content. Happily, I license it all CC-by-NC-SA so I can give it out freely!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Storage Decisions San Francisco 2011: Optimization and Virtualization
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Join me in New York for Storage Decisions, September 19 & 20

Later this month, I will be heading to New York for TechTarget’s Storage Decisions conference. This show does a good job on the editorial side, suggesting timely topics and bringing in independent voices like Howard Marks. I will have two presentations on data reduction and storage virtualization in the main conference track. Registration is free for qualified end-users, and I urge you to attend on September 19 and 20, 2011.

Reclaim Capacity with Data Reduction for Primary Storage

Depending on which industry study you read, most companies are wasting anywhere from 30% to 50% of their installed disk capacity, which translates into thousands of dollars spent with no effective return on investment. Storage vendors are beginning to provide tools that can help storage managers make the most of the disk they have installed. For example, data reduction for primary storage borrows data deduplication technology developed for backup and classic compression algorithms to help squeeze the air out of nearline and primary data and reduce its footprint. This session’s topics will include an overview of data reduction technologies and where they will have the greatest impact, what key storage vendors are offering in data reduction and an update on the major players, and the consequences of using primary data dedupe along with dedupe for backups. We’ll also look at the potential for vendor lock-in and consider why we’re reducing data in the first place.

Topics include:

• Introducing data reduction technologies
  • Compression: How it works and where it’s found
  • Deduplication: From single-instancing to variable block
  • Application-specific: Cracking open files
• Overview of data reduction products
• Where to use them
  • The capacity conundrum: Store less and reduce utilization
  • Ideal applications: Justifying the cost of data reduction
  • Side effects: Considering the impact on backup, replication, I/O workload and vendor lock-in

Storage Virtualization: Who’s Doing It and Why

Storage virtualization has been around for decades and, although research indicates that 70% of companies have already virtualized at least some of their installed block or file storage, most remain unaware of this technology. Grandiose schemes for comprehensive virtual SANs have given way to more practical host- and array-based virtualization technologies, and server virtualization has created a new opportunity to create a pool of storage. This session will look at the current state of storage virtualization, how to quantify its benefits and describe which approaches are best for particular environments, and also cover how storage virtualization compares to private storage clouds.

Topics include:

• Defining storage virtualization: What it is and where to find it
  • Abstraction of storage resources
  • Tiered storage
  • Flexibility
• Popular approaches to storage virtualization
  • SAN controllers
  • File virtualization
  • Volume managers
• The pool, the hypervisor and the cloud
  • The impact of server virtualization
  • Is this a private cloud?

Registration

You can see the future from here!

To register for Storage Decisions New York, just go to the TechTarget registration page.

Disclosure: TechTarget pays my expenses to attend and present at Storage Decisions, and has for many years. But they don’t pay me to present and I own the copyright on my session content. Happily, I license it all CC-by-NC-SA so I can give it out freely!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Storage Decisions New York: Capacity Optimization
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Join me in Chicago for Storage Decisions, June 21

Next month, I will be heading to Chicago for TechTarget’s Storage Decisions conference. This show does a good job on the editorial side, suggesting timely topics and bringing in independent voices like Howard Marks. I will have three presentations to give: Sessions on data reduction and storage virtualization in the main conference track, as well as a dinner discussion focusing on controlling the growth of data. Registration is free for qualified end-users, and I urge you to attend on June 21, 2011.

Reclaim Capacity with Data Reduction for Primary Storage

Depending on which industry study you read, most companies are wasting anywhere from 30% to 50% of their installed disk capacity, which translates into thousands of dollars spent with no effective return on investment. Storage vendors are beginning to provide tools that can help storage managers make the most of the disk they have installed. For example, data reduction for primary storage borrows data deduplication technology developed for backup and classic compression algorithms to help squeeze the air out of nearline and primary data and reduce its footprint. This session’s topics will include an overview of data reduction technologies and where they will have the greatest impact, what key storage vendors are offering in data reduction and an update on the major players, and the consequences of using primary data dedupe along with dedupe for backups. We’ll also look at the potential for vendor lock-in and consider why we’re reducing data in the first place.

Topics include:

• Introducing data reduction technologies
  • Compression: How it works and where it’s found
  • Deduplication: From single-instancing to variable block
  • Application-specific: Cracking open files
• Overview of data reduction products
• Where to use them
  • The capacity conundrum: Store less and reduce utilization
  • Ideal applications: Justifying the cost of data reduction
  • Side effects: Considering the impact on backup, replication, I/O workload and vendor lock-in

Storage Virtualization: Who’s Doing It and Why

Storage virtualization has been around for decades and, although research indicates that 70% of companies have already virtualized at least some of their installed block or file storage, most remain unaware of this technology. Grandiose schemes for comprehensive virtual SANs have given way to more practical host- and array-based virtualization technologies, and server virtualization has created a new opportunity to create a pool of storage. This session will look at the current state of storage virtualization, how to quantify its benefits and describe which approaches are best for particular environments, and also cover how storage virtualization compares to private storage clouds.

Topics include:

• Defining storage virtualization: What it is and where to find it
  • Abstraction of storage resources
  • Tiered storage
  • Flexibility
• Popular approaches to storage virtualization
  • SAN controllers
  • File virtualization
  • Volume managers
• The pool, the hypervisor and the cloud
  • The impact of server virtualization
  • Is this a private cloud?

Cutting Off Data Growth at the Disk

In this special dinner presentation, Stephen Foskett will discuss how to apply key data management technologies to arrest the growth of data. You’ll learn how capacity optimization technologies such as data deduplication and compression can reduce the trajectory of data growth as well as how tiering can reduce the cost of storage. Finally, Stephen will explore why the time may have finally come for active archiving and will leave you with practical ways to help your corporation better manage its data.

Note that space is limited for the dinner, which is sponsored by my friends at Dell.

Registration

To register for Storage Decisions Chicago, just go to the TechTarget registration page. Dinner guests will apparently be selected from that same pool of attendees.

Disclosure: TechTarget pays my expenses to attend and present at Storage Decisions, and has for many years. I also get a speaker fee for the dinner session.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Storage Decisions Chicago: All About Capacity Optimization
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Virtualization decouples physical infrastructure from logical resources

Virtualization of IT systems decouples physical infrastructure from logical resources, hiding complexity and enabling new capabilities. However, not all potential benefits of virtualization have meaningful value outside IT circles: Too many of our discussions revolve around the very complexity that virtualization technology should be hiding! True business value is derived from transformed virtual resources in the next-generation data center, not the incremental capacity gains of virtual servers. But how will we get there, and what will this future look like?

The Problem with Virtual Servers

Implementation of virtualization technology to date has merely delivered condensation of physical resources: 250 physical servers are condensed onto 20 physical servers, but 250 virtual server images remain. True, this does result in the reduction of data center footprint, from rack space to power and cooling, enabling moderate cost savings. But these are not examples of real consolidation, let alone business transformation.

Many have lamented this “virtual server sprawl” and suggested alternative methods of consolidating low-utilization applications into larger, more flexible “resource servers”. For example, numerous SQL servers can be combined on a single central server with more focused management. But these larger resource servers are not normally virtualized since their concentrated I/O demands can overtax current server virtualization platforms. Therefore, consolidation and virtualization remain separate.

This is the problem with conventional server virtualization. It enables us to condense data center demands for some systems, but delivers very little else apart from new backup and management headaches. Certainly we can provision servers more quickly, and we might be able to recover from a disaster more easily, but these are IT-facing benefits that other business entities care little about.

Storage and Network Virtualization

Virtualization of storage and network resources face even higher barriers. Where server virtualization has quickly delivered incremental “green” savings, these benefits are harder to come by in other areas.

Storage virtualization primarily delivers flexibility. SAN or NAS systems can be combined into larger pools, allowing existing resources to be better utilized or provisioned more quickly. But there is only a little cost avoidance to be gleaned from more efficient use of storage capacity. Real cost savings would require reduction of infrastructure, and constant data growth makes this extremely difficult to achieve. Other benefits, like enhanced data migration or heterogeneous replication, ought to be invisible to the business anyway.

Network virtualization lags even further behind. Only a few shops have attempted to use technology like InfiniBand to enable flexible virtual connectivity, though the future Converged Enhanced Ethernet concept is beginning to spark some interest. Here again, financial benefits from network virtualization technology are limited to a moderate reduction in future equipment cost.

Transforming the Data Center

In all three instances (server, storage, and network), the financial benefits are merely the sideshow. The underlying benefit from virtualization of IT infrastructure comes from the extension of IT systems outside the data center, a change on the order of the advent of minicomputers or the spread of open systems.

VMware recently laid out a serious and compelling vision of this future Virtual Data Center as VDC-OS. Their concept is evolutionary and radical at once, with the simple virtual server infrastructure of today augmented with increasingly uniform and flexible storage and network layers. This culminates in a truly virtual data center, where running server images can move from device to device, location to location, and even out to the cloud.

VMware’s brilliance is in leveraging what works today (virtual server images on ESX) to build a foundation for complete virtualization of physical resources. But virtual servers running on VDC-OS remain tied to the present: They run the same operating systems and will likely remain bound to the same “one (virtual) server per application” world view that pervades open systems today. This leads to exactly the same situation of server sprawl that has proven a management nightmare.

Others are extending the web hosting concept to enable custom applications to be run on the scalable, flexible, multi-homed servers that run the world’s biggest Internet applications. Google and Amazon’s visions are decidedly post-data center, with applications, rather than server images, being the primary unit, and database-style storage replacing conventional blocks and files. Use of these web-oriented application platforms has so far been limited to entirely new systems built from scratch to take advantage of them, limiting their appeal to current IT environments.

Where Is the Business Value?

Yet, most discussions of these virtualization strategies (mine included) fails when it comes to demonstrating real business value. We must move away from quickly-forgotten cost savings and focus instead on profoundly transforming how IT serves business goals. Virtualized infrastructure allows flexibility and scalability, changing how everything in IT works.

Whether it uses conventional operating systems and applications or re-engineered web-enabled solutions, virtualized infrastructure fundamentally changed our world. Organizations would be free to physically move their systems, even outsourcing or offshoring the infrastructure component entirely. They could move to an on-demand purchasing model for logical capacity, not just bits and bytes.

In the process, they would render current server platforms, operating systems, and storage devices irrelevant. Undoubtedly, attaining this future remains a while off, but IT professionals should consider its implications. Much of what we do is focused on making the “plumbing” work efficiently rather than serving the needs of the business. Where do we stand once the perennial issues of performance, availability, and scalability are solved?

photo by Roy Smith

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Where Will Virtualization of Data Center Infrastructure Take Us?
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

How granular is your storage? This question is just as relevant today as when I first published this article in 2008!

Many storage challenges focus on correlating high-level uses of data (such as applications) with the nuts and bolts of storage infrastructure. These discussions often revolve around the conflict between data management, which demands an ever-smaller unit of management, and storage management, which benefits most from consolidation. Developing data management capability that is both granular enough for applications and scalable enough for storage is one key to the future of storage.

Storage Management: Scaling Up

As I discussed in a previous Sunday Series piece, Turning the Page on RAID, the data storage industry has traditionally focused on reducing granularity. Disk capacity has expanded, and RAID technology has multiplied this by combining multiple physical drive mechanisms into a single virtual one. Storage virtualization technologies, from the SAN to the server, have also often been touted primarily as a mechanism to reduce heterogeneity. From a technical perspective, therefore, granularity has been an obstacle to overcome.

The core organizational best practice for storage management is the reduction of complexity and the enforcement of standardization. Consolidation of storage arrays and file servers is a common goal, as IT seeks to benefit from economies of scale. The goal of both initiatives is the creation of a storage utility or managed storage service. This mirrors efforts on the server and network sides to consolidate and virtualize hardware.

Although both technological and organizational factors have traditionally driven granularity out of storage, this does not have to be the case. Virtual pools of storage are ideal for providing storage on demand, as disk-focused RAID groups give way to more flexible sub-disk storage arrangements. And an operational focus on standardized storage service offerings has the potential to enable scalable management of these smaller units.

Filing Service

File-based protocols would seem to have more potential for granular storage management, but they have been undermined by the hierarchical nature of modern file storage. Whether the connection to a file server uses NFS, CIFS, or AFP, the key unit of management is actually the shared directory, not the file. All files in the share \\firefly\backups would be located on the same server and would be managed as a unit.

NAS virtualization can change this somewhat, as can more specialized NAS servers. Although Microsoft DFS enables consolidation and virtualization of NAS shares, it does not allow subdivision of shares below the directory level – all files in a directory must be placed on the same server. Tricks like stubbing and links allow for some movement, but these do not solve the core issue. Specialized virtual NAS devices from F5 (the ARX, nee Acopia), NetApp, BlueArc, Symantec, and others have the ability to move files individually, providing as much a virtualized storage environment as any block-focused enterprise array. Avere is also beginning to talk about granular file management.

But even an ideal virtualized file server lacks the kind of granularity demanded by users. They care about data, not files, and most applications consolidate their data storage into a few files. Consider a database, for example, where users want each record treated uniquely but storage devices see just a few much larger files. We need a storage revolution, where someone creates an ideal storage platform in which each individual record or object includes custom metadata and is managed independently. This would truly be a massive change, however, and it is not clear that all applications will follow the object storage model of Google and Amazon.

Small is Beautiful

Barring a revolution in data management, our best hope is to allow greater granularity in storage management. As mentioned above, virtualization technology has the potential to enable management and protection of any unit of storage, right down to the individual block or record. But the reality of storage virtualization has not matched its promise.

What is needed is greater integration. Each layer of virtualization (file system, volume manager, hypervisor, network, array, and RAID) also hides necessary details from lower layers. Consider the case of a virtual server snapshot: The application and filesystem must be in a quiesced state to allow a snapshot to be taken at the storage level, but the storage array has no intrinsic information about how its capacity is used. A given LUN might contain dozens of servers on a shared VMFS volume, so all must be snapped together.

Integration can be enabled by sharing more information through APIs. VMware leverages Microsoft Volume Shadow Copy Service (VSS) integration for shared storage so a VMFS snapshot can call the operating system and even applications (Windows Server 2003 only, for now) to prepare the data. Similarly, VSS can communicate directly with supported iSCSI and Fibre Channel arrays, calling a snapshot at the right moment. And Microsoft is, no doubt, enhancing VSS as we speak.

As virtualization technology matures, expect this type of integration to improve. We hope to see more APIs exposed by VMware and Microsoft, allowing communication up and down the stack to break through the information barrier. Imagine a future where a standard API like VSS can pass a message through VMware, Xen, and Hyper-V to the underlying storage array to initiate a snap. I predict that this kind of integration-enabled granularity is not too far off.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Granularity: The Hidden Challenge of Storage Management
This post was categorized as Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

I began by introducing the core problem: Storage isn’t getting any cheaper due to storage utilization and provisioning problems. Thin provisioning isn’t all it’s cracked up to be, but it has a place. So what’s wrong with it?

The problem with thin provisioning starts with the telephone game. Did you ever play the telephone game as a kid? Maybe you whisper “I like potatoes” to the first person in a circle and when it comes back to you it’s changed to “Meet Mike on the patio” or something like that. It’s a totally different message.

What happens in the telephone game is that a little bit of information gets lost at each step along the path, and at the end of the chain you’ve basically lost all the information. And this happens all the time in computers, especially in data storage.

We storage guys are stuck at the bottom of a stack that includes many layers. Each of those layers loses something in translation, mostly because it’s pretending to be something that it’s not.

Think about storage today: We’ve got fake file systems pretending to live on fake discs that pretend to be directly connected to your computer. But they’re not.

Everything we do in enterprise storage is basically faking out something else so compatibility is maintained. And at each step (the file system, the database, the host, the network) you’re losing information. By the time you get to the storage system, there’s just no communication whatsoever.

This is the core problem with thin provisioning. The application knows what data is temporary, and that would be very useful for the storage system to act upon. But by the time the data gets there, the message is lost. Maybe the application tells a database. Maybe the database tells the file system. Maybe the file system tells the volume manager. But, that’s about as far is it’s going to go. So, this is really the issue. It’s the telephone game.

Let’s say we have a file system and some storage. We want to write some data. So, the file system says, “Hey, here’s my new block.” And the storage says, “Yeah, I got it.”

This is the classic way of doing thin provisioning. So the storage system is now only using the little blue box. The file system adds some new data, then some more data, and the storage just keeps growing. The rest of the space can be reallocated.

We’re good so far. This is so simple that most products in storage now have something like this. Of course, it took them 10 years to do it, but they finally have it.

So, we’re good. We can allocate storage. But, what about deallocate?

If I delete something, I have to tell the storage, and then it has to shrink the allocated capacity. But we’re not doing that. Most file systems don’t actually send that information on. When you delete a file, most file systems actually write more data instead of actually deleting anything.

Thin reclamation is the core technical challenge to thin provisioning, and the telephone game is the reason. Next we’ll present some solutions that are currently being worked out.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Thin Provisioning: Playing the Telephone Game
This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

HP invited bloggers to Colorado to show off their storage offerings at Tech Day 2009

Well that was interesting! I spent Monday and Tuesday with a dozen bloggers in Colorado Springs learning about HP’s key StorageWorks products from the executives and engineers of that storied company. Throughout the event, we listened, asked tough questions, and tweeted incessantly. HP Tech Day generated an avalanche of publicity for the company, including press articles and cries of FUD and misdirection.

Here is the truth: HP and Ivy did a darn fine job of putting together a set of sessions to tell us what they have. They presented folks who really knew their stuff, warts and all. They invited a variety of independent voices and let us ask and say anything we wanted with no expectations, let alone an NDA. This was a stellar event, and every other IT company should be asking why they didn’t do it first.

Won’t Be Fooled

But what was HP Tech Day really all about? Did they brainwash us into thinking the EVA was exciting? Did HP hoodwink and misdirect us from noticing the gaps and overlaps in their product line? Were we dazzled enough to no longer question their storage strategy? Of course not! Robin Harris has seen it all before. Rich Brambley can dissect a presentation with the best of them. Nigel Poulton questions everything he sees. This was not a love-fest, and there were some seriously uncomfortable moments for the HP crew.

We were all savvy enough to know what the score was: HP (and especially their StorageWorks product line) has never received much press or blogger attention. They invited us in to spread the word about their products and get blog exposure. And it worked! The storage Twitter-sphere was dominated with #HPTechDay postings for days, and attendees did indeed blog about the stuff they saw. It was a success from HP’s perspective, and now that we know more about their products we will likely cover them in the future.

I personally never blogged much about HP products. I compared them to Ford, called one product an all-time cool flop, and covered their acquisition of LeftHand and Ibrix, but that’s about it. I didn’t even notice that they had started selling LSI’s StoreAge-based virtualization platform, and didn’t know enough about their other product releases over the last two years to mention them. If it hadn’t been for Calvin Zito and the HP Tech Day, I probably wouldn’t mention them in the future, either. They just weren’t on my radar.

We Were Educated

This has changed as a result of Tech Day. I now know that EVA is still pretty much what I thought it was but that it is fairly simple to configure. I now know that HP has two lines of deduplication appliances, and that one is homegrown. I now know that HP develops and sells a FC SAN virtualization platform based on the LSI/StoreAge product. This was great product exposure for HP: Even though I still don’t think I’ll focus on these offerings, at least I now know that they exist.

We also saw some more interesting content: HP plans to leverage their excellent ProLiant server and blade technology to underpin a line of commodity-based storage and consolidated virtualization offerings. This is exciting stuff, and builds on HP’s base and their recent acquisitions. The most interesting concept I saw was a combination of a blade chassis and ultra-dense storage system with VMware ESX, LeftHand, Ibrix, and HP’s management software. They definitely plan to challenge Cisco/EMC and IBM in this market.

But the best part of the event was the people of HP. The company was smart enough to bring in the techies rather than executive marketing droids. They gave us straight and honest answers about their product capabilities and their place within the company, sometimes to the chagrin of others in the room. We saw conflicting definitions, product line overlaps, internal competition, and got a feel for the realities of this massive company. Every company is like this, but most would never admit what we already know. By not showering us with slick FUD, HP won our respect in a way that I would not have thought possible.

Colorado Take-Away

The event was great. Every other IT company should wise up and do the same. But I bet they won’t have the nerve to do it as openly, and I expect HP will be more careful next time too. As for HP’s products, let me lay out my honest opinion:

1. LeftHand was a bargain. HP could replace their entire sub-XP block storage product line with LeftHand variants based on commodity Intel hardware. I think they should.
2. Ibrix is too new to judge, but will likely take a seat next to LeftHand in a unified commodity-based scale-out SAN/NAS platform.
3. I hope HP hasn’t lost focus on their excellent Hitachi-based XP line, since it’s the only challenger they have to EMC Symmetrix at the high end of the market.
4. HP’s excellent server and blade hardware should be leveraged throughout the company and oddball hardware should be curtailed.
5. HP has an interesting lineup of Windows Storage Server and Windows Home Server hardware aimed at the low-end SOHO and SMB market. I’m not a Windows hater, but wouldn’t it be cool to have a super-low-end LeftHand variant there too?
6. I wonder if HP’s home-grown in-line deduplication (the D2D line) could replace the Sepaton-sourced post-process VLS line or vice-versa if an acquisition of that company happens.
7. I don’t “get” SVSP, the LSI/StoreAge SAN virtualization platform. It’s nice and it seems to work, but why introduce a FC SAN virtualization platform at this point? Maybe it sells when integrated with EVA, but not as a standalone product.
8. Where are all the storage arrays that use 2.5″ disk drives that HP talked about over the last few years? And where are the flash solid state drives?
9. They were like deer in the headlights with no story when I asked about sub-LUN automated tiered storage since even full-LUN automation has not yet been released. Might EMC get there first?
10. As for the absence of DCB and FCoE, HP seems to think that Virtual Connect and Flex-10 are good enough for now. HP will OEM a CNA soon and might possibly consider the Cisco Nexus 4000. Maybe. They had better have a more-convincing story when this stuff takes off next year!
11. Most importantly, where is the cloud strategy? I applaud HP for not overusing the current buzzword, but it almost seems like they are avoiding the topic entirely.

I am left thinking that HP is like a boxed puzzle. All of the pieces are there, but they haven’t been put into place yet. Let’s hope Dave Donatelli and his crew can help them work it out when he takes his post as czar of servers, networking, and storage next year. For now, I’ve gained a lot of respect for the people of HP and a pile of knowledge about their storage products. And HP has gained my attention.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | The Truth About HP’s Tech Day
This post was categorized as Enterprise storage, Everything, Gestalt IT, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

The good news is that this is no longer a technical problem: Modern virtualized and networked servers ought to have decent utilization of storage capacity, and technology is improving all the time. Consider the compounded impact of modern technology on storage capacity utilization:

• Shared storage (SAN and NAS) allows different servers to share a common pool of storage, reducing the likelihood that excess capacity will be stranded in isolated “puddles”. Pervasive use of NAS technology, and the rise of simple and inexpensive iSCSI SANs, means that every system in the modern data center can use shared storage.
• Organizational and architectural optimization allows storage to be provisioned from a common pool rather than building “stovepipe systems” with their own resources. Quicker provisioning also helps reduce over-provisioning.
• Network connectivity allows servers to share resources, including storage, on a peer-to-peer or client-server basis, ultimately resulting in things like cloud computing.
• Managed and utility services reduce the impact of low utilization, potentially focusing on efficiency or perhaps passing the buck to a service provider.
• Thin provisioning might help certain systems to keep less storage in reserve.

So why don’t things get better? It’s hard to be sure why people don’t use these pervasive tools to improve storage utilization, but I do have some ideas…

• Storage utilization might not be a priority. Utilization isn’t often in the critical path of performance or availability, so overtaxed IT departments aren’t going to focus on it.
• Incentives can be lacking. With the cost of storage constantly falling, the effort required to improve the efficiency of already-allocated storage can be just as easily spent migrating to a newer, cheaper storage platform.
• Virtualization has perversely harmed the efficiency of allocation. One might think that the ease and flexibility of virtual disks would improve things, but it hasn’t. Server and storage virtualization just adds another place to hide unused storage.
• Metrics remain a problem, since everyone gets all balled up trying even to talk about capacity utilization.

I think this last point is something we in the industry really ought to do something about. We say “utilization” but what do we mean? Chris Evans has proposed a set of metrics for the “storage waterfall“, and I mentioned back in October that this all boils down to three key metrics: Raw, usable, and used. The key question is where to apply them!

Way back before the 2001 bubble-burst, I managed professional services for a company called StorageNetworks. At that time, I was quite aggressive in pushing this same idea, even co-writing a whitepaper on the topic titled Measuring and Improving Storage Utilization. My co-author (Jonathan Lunt) and I recently reminisced about that paper, and we both agreed that everything in it still stands today, apart from the high dollar cost per gigabyte.

Each ratio in the storage waterfall can be diagnosed and improved

I suggest that the following key storage utilization ratios (taken directly from this paper) make just as much sense today as they did then:

• Array Overhead is the percentage of installed storage capacity that is not usable. Dividing Array Usable by Array Raw and subtracting that number from 100% yields the percent of overhead. Overhead here is usually due to the desired level of data protection (e.g. RAID, mirroring) rather than to poor management.
• Array Utilization is the percentage of usable array capacity that is allocated to hosts. It indicates the efficiency of storage deployment operations.
• Allocation Efficiency reflects the ratio of storage presented or allocated to hosts to the amount actually seen by them. In many mature environments this ratio is near 100% (i.e. all the storage allocated is being seen), but this ratio can be extremely difficult to determine. It relies on accurate measurements of both Array Used storage and Host Raw.
• Host Overhead reflects the amount of storage configured for use versus the amount the host can see. Since the Host Raw metric is a function of the storage administration team and the Host Usable a function of the systems administration team, this metric is a useful measurement of how well the two functions are cooperating. Data for this classification is collected from the host.
• File System Utilization is the amount of available file system space that actually contains data. File system utilization is familiar to most systems administrators. This metric is often shown in simple system commands like “df” on UNIX or “dir” on Windows. Data for this classification is collected from the host.
• Total Storage Utilization summarizes how well a company manages its storage assets across the entire business. This ratio is the default storage utilization metric used in publications and reflects the actual value an enterprise is deriving from its storage asset. Care is required in calculating this ratio to ensure that it accurately indicates utilization of the storage environment. Since the result of this ratio is often used in business cases and receives wide attention, it must be both logical and defendable.

To these, I would add another intermediate and optional set of virtualization metrics and ratios for environments with storage or server virtualization. One could also presumably add a higher-level set of application efficiency ratios as well.

In the paper, Jon and I also proposed three best practices to improve storage utilization:

1. Drive Array Utilization (Array Usable to Array Used) to greater than 90% (a storage administration responsibility)
2. Drive Allocation Efficiency: Bring Host Usable to be as close to Array Used as possible (a joint responsibility)
3. Drive Filesystem Utilization (“Host Usable to Host Used”) above 80% (a systems administration responsibility)

Go read the paper and let me know what you think. Are we still stuck in 2001?

This post can also be found on Gestalt IT: Storage Utilization Remains at 2001 Levels: Low!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Storage Utilization Remains at 2001 Levels: Low!
This post was categorized as Computer History, Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Although I doubt Canada is much different from the US in terms of adoption of particular technologies, I am sure that the different governmental and litigation drivers will affect certain infrastructure and organizational practices. Here in lawsuit-happy USA, for example, it is easy to justify the cost of advanced archiving and e-discovery infrastructure to reduce the risk and impact of legal action. But Canada has a very different corporate environment, and information privacy is much more the focus.

I’ll report back on what I find after the trip. In the mean time, if you can get to Toronto tomorrow, there are still spots for qualified attendees! We’ll be at the Westin Harbour Castle, a beautiful location right on the water. I’ll also be in Louisville, Kentucky this week, and would be happy to meet any fellow Pack Rats in the land of bluegrass, bourbon, and thoroughbreds!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Toronto Seminar: Do Canadians Virtualize?
This post was categorized as Enterprise storage, Personal, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

• Want a direct flight? That’ll be $1,080 round trip on an Air Canada Dash-8. But it only takes 1:15 to get there! For comparison, I could fly to London (UK) from Cleveland for $650 round trip.
• Ok, let’s connect then! That’ll run at least $650 on American (or over a grand on United) and take 4 hours, connecting through Chicago. Or you can connect through Washington DC on multiple airlines.
• Hmmm… Amtrak? It’s cheap at $120 round trip, but the journey there takes 13 hours, and it’ll take a whopping 19 hours to get home! London (UK) is only 12 hours away (with a connection!)
• Ok, Amtrak goes straight to Buffalo in 3 hours. Then it’s another hour and a half to Toronto by car. And there’s only one train per day. And it arrives at 3:30 in the freaking morning.
• How about driving? That’s a 300 mile, 5-hour journey. Just long enough to be too painful.
• Might as well walk. Google says it’ll take just 4 days!

Any suggestions?

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Can’t Get There From Here
This post was categorized as Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.