I think this sentiment is just as valid today as when I posted it in 2008!

Although many discussions in the storage industry focus on the relative merits of one protocol or another, the conversation occasionally turns to the core issue at hand: We continue to patch together a system based on outdated concepts. Most storage protocols continue to mimic direct attached storage, and most of our so-called networks act as point to point channels. An ultra-modern virtualized storage infrastructure with all the latest bells and whistles still holds the concepts of block and file at its core. Whenever the storage industry has tried to bring about real storage management they have been stymied by a lack of context for data.

No amount of virtualization, and no new protocol, will fix this. Put simply, we need a storage revolution.

Channels, Blocks, and Files

Most innovation in the 1980s and early 1990s focused on moving storage out of the server. SCSI allowed disk to exist in a separate cabinet, RAID allowed multiple physical disks to become a single virtual one, and these were mixed to become the prototype storage array. Although SCSI allowed one-to-many connectivity, it was never a true peer-to-peer network, even once it was mixed with network concepts in the form of Fibre Channel.

Even today, SAN storage is focused on providing faster, more flexible, and feature-packed direct-attached storage. A modern virtual SAN hides a complex arrangement of caching, data protection, tiered storage, replication, and deduplication, masquerading the lot as a simple, lowly disk drive. It is sad but true that all of our work as an industry has been dedicated to recreating what we started with.

Networked file-based storage is no better. Although NAS devices have all the advanced features of their SAN cousins, they must present a simple file tree to the host to retain compatibility. File virtualization merely presents a larger homogenous tree.

Inside the server, too, features and complexity are hidden to retain a familiar file system format. Volume managers can do anything a virtualization device can, but must present their output as a simple (though virtual) disk drive. File systems, too, have added features but still present a familiar tree of mount points, inodes, and files. Even ZFS, possibly the most advanced combination of volume management and file system technology yet, must present a simple tree of storage to applications.

The Metadata Roadblock

This outdated paradigm, of disks and file trees, is ill-suited to today’s storage challenges. Data must be categorized so actions can be taken to preserve or destroy it based on policies. Data must be searchable so users and applications can find what they want. Data must be flexible so it can be used in new ways. Our antiquated notions are not capable of meeting these challenges.

One simple problem is that we lack context for our data. Most file systems merely assign to a file a name, location, owner, and security attributes. The most advanced can contain extended metadata, but this is rarely seen in practice since many applications cannot agree on how to use this data. Microsoft’s Office suite can store and share extended file attributes, for example, but these live inside the file rather than in the file system. The promise of expanded Office attributes is only realized in conjunction with a content management system like SharePoint which lies above the lowly file system.

What if the storage system could keep this data instead? What if it could logically group files according to project or client, mining keywords and authors, and maintaining revisions? These concepts are not new, having been implemented in content management systems for years, and certain elements appeared in file systems, like Apple’s HFS and VMS’ Files-11, for decades.

Cut Down the Tree

File metadata would allow advanced features, but truly taking advantage of them requires a more fundamental shift in the way applications access files. Rather than sticking to a traditional hierarchy of directories in a tree (which was, after all, simply a primitive metadata system), we should remove the tree altogether. Allow files to become data objects, identified by arbitrary attributes and managed according to an overarching policy.

This future vision is decidedly different from our current notion of storage, but is not so far off. Many organizations now rely on central data warehouses based on SQL-language relational databases. As many storage managers have grumbled, databases tend to ignore storage management concepts entirely, managing their own content independently.

But not all applications need a database back-end, so another initiative seeks to provide generic object storage for wider use. Called content-addressable storage or CAS, these devices have traditionally been used only for archival purposes, since that was their first market application. As vendors break free of proprietary interfaces in favor of open ones like XAM, CAS could transform storage itself by eliminating both file and block storage at once.

Similar concepts are already at work in the so-called Web 2.0 world. Non-traditional databases like Google BigTable, Amazon S3, and Hadoop allow massive scalability for object storage. API-sharing initiatives with many Web 2.0 companies can be seen as similar prototypical object storage frameworks. Any of these could be leveraged to provide a new world of data storage, and many are gaining traction even now.

Stephen’s Stance

Although traditional block storage is here to stay for disk drives, and tree-type file systems are likely to remain the foundation of operating system storage, new object-based concepts could change the world in fundamental ways. As applications become “web aware”, they also become object aware, increasing the likelihood of such a storage revolution. For the majority of applications, this new world would be a welcome one indeed.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | We Need a Storage Revolution
This post was categorized as Apple, Computer History, Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

If WRITE_SAME can be a semaphore for thin un-provisioning, what about TRIM? It sounds like a perfect fit, and has wider implementation to boot! Let’s take a deeper look.

First, I’d like to remind the reader that I’m not some specialist storage engineer. I am regular guy who wanted to know about this TRIM thing. I welcome corrections and feedback on this piece!

We should note that TRIM was invented for SSDs, not thin provisioning. TRIM lets an operating system tell a storage device that certain blocks of data are no longer used. Although this sounds like a thin provisioning tool, it’s not. TRIM was never intended to be used this way.

Even though Microsoft supports TRIM in the file system, and even though, hopefully, Apple will introduce TRIM support in OS X Lion, this doesn’t do what we need it to do for thin provisioning. In fact, TRIM is not anything for enterprise storage folks to be getting all that excited about.

One reason for is is that TRIM is an ATA command, and most enterprise systems are not connected to their storage using any kind of ATA-based protocol. Maybe there are SATA disks somewhere down the line, but, whether they use iSCSI, FC, SAS, or FCoE, the servers speak SCSI.

Even though there are SCSI analogs of of TRIM, these commands were really not designed for thin provisioning. They are designed for SSDs, which have a big performance problem: An SSD can delete data and write data just fine, but in order to update data, they actually have to read, delete, and write a big page of data. What happen is that, as an SSD gets full, it starts slowing down. And that’s a problem because people buy SSDs for speed. So TRIM is a way to tell the SSD, “Sometime in the future, you can go ahead and delete this if you feel like it,” and the SSD will take care of that later.

There’s great OS support for this, but unfortunately, this is not something that enterprise storage vendors are looking at. The only vendor (that I know of) that’s working on this, NetApp, did propose to T10 a mechanism to do thin provisioning based on TRIM. I don’t think it got anywhere. They say that it did, but I just don’t see it. HDS and EMC, seem to like UNMAP and SCSI, which are peripherally related, but I don’t know much about what they’re doing either. Perhaps they’ll let me know in comments on this post.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Is TRIM Useful For Thin Provisioning?
This post was categorized as Apple, Enterprise storage, Everything, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

One of the sticky wickets that holds back thin provisioning is the need to communicate when capacity is no longer needed. Enterprise storage arrays can reclaim zeroed pages, but writing all those zeros can really fill up an I/O queue. This is where WRITE_SAME comes into the picture.

This is a really terrible name. It’s all-capital letters and has an underscore in the middle of it. We sound like engineers.

But WRITE_SAME is an interesting idea: Imagine you wanted to delete a terabyte of data using a storage system with zero page reclaim? You’d have to write a terabyte of zeroes. Well, that’s a lot of IO. You’re basically pouring zeroes across your PCI bus, HBA,network, and array.

Instead, imagine we could just say, “You know that page of zeroes that I just wrote? Can you please write that a million more times for me? Hey, thanks a lot.”

You could do it in one command. That’s what WRITE_SAME is. It’s a SCSI command that says, “That last thing that I just wrote, can you please write it again, and again, and again? Can you please write it a thousand times? Can you please write it over here, over there?” I sound like Dr. Seuss: You can write it in a car. You can write it at the bar. You can write it on a bike. You can write it with a pike.

This conserves IO, and is a really good thing. WRITE_SAME makes zero page reclaim that much more effective. Now if only we had a system that would actually use this command!

It’s popular with array vendors, because all they have to do is say, “Hey look, I already support zero page reclaim. It’s up to you guys up there in the stack to implement the rest of this problem. It’s not our problem. It’s your problem.”

As an aside, consider that, if you’re an array vendor, any problem that reduces the use of disk capacity is your problem. So, they may not all be that eager to have this work, I think, but I’m sure they’ll come around.

But imagine if you did this to an un-thin array. Imagine if the array didn’t support zero page reclaim on ingest and instead was post-processing. You could end up writing a terabyte of zeros on the back end of your storage system, or 10 terabytes or 100 terabytes of data, only to reclaim it later that day, or later in the week or later in the month. And what if your system didn’t support it at all? Suddenly, you’re flooded with IO requests on the storage-array side. So, basically, you’re conserving IO across the host and the network, but you’re potentially generating massive IO on the storage side – which is kind of a problem.

So, there are some issues here with this as well. But, we’re getting there.

© sfoskett for Stephen Foskett, Pack Rat, 2011. | What is WRITE_SAME? Green Eggs and Ham!
This post was categorized as Computer History, Everything, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Fear

Back when I used to teach the “Storage 101″ session at Storage Decisions, I was consistently amazed to find little awareness of enterprise NAS systems. People complained about LUNs and Fibre Channel but when I suggested using NFS or SMB their heads almost exploded. “We would never use that for application storage,” they shouted. “File servers are for home directories, not data center stuff!” Clearly, NAS faces an uphill battle.

In a recent piece I wrote, I referred to what I consider to be the prime best practice: Use the right tool for the job. It’s a simple statement, and one that resonates beyond IT and the technology world. But it can be devilishly difficult to see what the right tool is sometimes. Why not use NAS for virtual machine storage? NetApp has been beating that drum for years, yet NAS has a very small footprint in VMware. How about databases on NFS? Exchange over SMB? Block storage has a massive lead over NAS in all of these areas.

Rear this “best practices” piece, Use Process Solutions For Process Problems, Technical Solutions For Technical Ones

IT folks seem downright fearful of file-level storage protocols. Has NFS really burned them that badly over the decades? Can SMB/CIFS really be as bad as they think?

Loathing

I wonder if this terror has more to do with the products people have used than the fundamental concept of file services. Many NAS servers (and clients) are barely functional. Sadly, NFS and SMB are easy to get 80% right, but the 20% corner case interaction takes decades to overcome. My daily storage consulting work exposes me to a myriad of NAS configurations, and few of the multi-platform combinations end well.

Note: Although it has long been known by a variety of names, the Windows NAS protocol is currently called Server Message Block or SMB. Common Internet File System (CIFS) was a failed mid-1990′s attempt by Microsoft to make this protocol standard on the Internet.

Consider the Mac. Apple added an SMB client to OS X in 2001 but, despite many updates, it is far from reliable. Mac users in general loathe connecting to Windows file servers, and business users have located numerous bugs in the handling of Mac-specific file types. It’s bad enough that one company, GroupLogic, created an entire AFP server for Windows just to solve these tricky issues.

This situation often happens in reverse, too. Windows admins are justifiably cautious when deploying non-Windows SMB servers, whether software (Samba, Novell, etc) or system (NetApp, Celerra, BlueArc, etc). As a very early NetApp user, I watched their CIFS/SMB server evolve over a decade and a half into a fairly robust solution, but the early years were downright painful.

Lest you throw rocks at Redmond, know that SMB is not alone with functionality problems. The interoperability of NFS servers and clients is a bit better thanks to open(ish) standards and open source implementations, but its reputation is just as bad. And Apple’s proprietary AFP protocol is downright notorious.

I’ve been there myself many times. I tried to set up a home server based on open source software (Linux, FreeBSD, Samba, Netatalk, etc) but rejected it outright after many frustrating years. Today I use a Mac Mini for file sharing in OS X and serving iTunes music and movies (goodbye, Firefly!) And years of fighting with Samba in enterprise environments taught me two things: It’s possible to get it running well with Windows clients but it’s astonishingly easy to get it wrong.

Enterprise NAS?

We all know that interoperability is devilishly difficult. I don’t envy the NetApp and EMC engineers that have to tweak and tune their server for every possible client, bugs and all. And I am impressed that, after probably millions of man-hours of work, they were able to come up with something stable for a subset of use cases. But this just makes me even more cautious about third-party NAS servers.

I talk to storage vendors all the time, and many of their new products support NFS and SMB. But my internal alarms start going off when I hear about these products. There are two simple reasons for this:

1. As mentioned above, NAS is rare in primary data center applications. It may be common for user files (euphemistically called “unstructured data”) and certain distributed applications (simulation, rendering, etc), but most use cases still call for block SCSI (FC/iSCSI) storage.
2. As further mentioned, getting NAS right takes a massive amount of effort. New and small vendors tend to slap Samba on their (Linux-based) box and call it a day. This is very, very far from being sufficient for enterprise use.

This is why I usually ignore NAS functionality in storage systems except for long-tenured and deep-pocketed vendors. Although the world is turning to “Unified Storage” and multi-protocol support, I’m focusing primarily on block (SCSI) and cloud (REST) capability because the former has proven somewhat easier than NAS to get working and the latter is both simple and “green field” with no legacy concerns.

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Why Do I Ignore NAS?
This post was categorized as Apple, Computer History, Enterprise storage, Gestalt IT, Terabyte home, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

What is Multipath? And Why?

The good old days: One device, one path

In the beginning, computers connected to peripherals and other computers through a single bus or channel and life was easy. Although one might mistake the names of the dominant printer connection (parallel) for some kind of multipath system when compared to the modem connection (serial), this was not the case. Only the bits traveled in parallel – the logical connection was a simple single path.

Early-90's servers might confuse admins with two SCSI connections to a single device

Then things got complicated. The SCSI protocol allowed for multiple devices in a chain, and even for two different “initiators” (computers or controllers) to interact with these “targets”. Some folks even dual-attached devices to a single computer with multiple controllers.

Why would one device and one computer need more than one connection? It boils down to two factors:

1. Performance – I/O channels have typically been slower than the computer could handle, so multiple channels might be used to increase the amount of data that can flow in and out.
2. Reliability – If one connection failed, the other might still be usable, reducing the risk of an outage.

Late-90's enterprise systems might have four or more paths to a single storage array

Pretty soon, enterprise computer architecture had gotten incredibly complex. I remember connecting a massive HP V-class server to an EMC Symmetrix with eight separate Fibre Channel cables. Each disk “LUN” showed up twice, and we had hundreds of them. We managed all of these virtual storage paths using HP’s PVLinks dynamic multipathing software. We used Veritas DMP and EMC PowerPath to do pretty much the same thing on Solaris and other UNIX systems.

Active/Passive to Active/Active

The earliest path management software provided two incredibly important functions: It figured out which of the SCSI targets it saw were actually different names for the same one, and it allowed the operating system to choose one and fail over to the other in case of an interruption. These were Active/Passive links – no matter how many paths were presented (and Fibre Channel switches sometimes presented eight or more), only one was active at any one time.

Modern systems have abstracted and virtual I/O channels, making path management much more important

But the EMC Symmetrix and similar high-end storage systems changed all this. Symmetrix storage was fully virtualized – the presentation of LUNs to servers was entirely disconnected from the actual disks and RAID sets in the array. This meant the Symmetrix could handle I/O requests across different paths and controllers for the same LUN. EMC and the rest responded with Active/Active path management software, allowing I/O to travel in parallel for the first time.

How is Dual Active Different?

Not everything called Active/Active is created equal. In fact, many supposed Active/Active setups really shouldn’t be called that since they don’t use both paths for all data. Instead, I like to call these Dual Active – both paths are active but with different data.

Consider the differences between the following two solutions:

A true active/active setup uses all paths for all data all the time

A "dual active" setup uses both paths, but each target is directed to one or the other

See the difference? Although the paths are active in both cases, they are not the same. Both approaches have merit, and neither is inherently superior, but they should have different names applied. Even active/passive has its place, since simplicity is often a virtue.

Dual Active Outside Storage

These same concepts apply outside the field of storage and I/O. Many server clustering systems use the same terminology, right down to the misapplication of “active/active” when “dual active” is more appropriate. It’s easy to miss the significance of this difference, but it can make more of an impact in clustering since CPU workloads are harder to balance.

Let me know what you think. If there is interest, I might dive into path management strategies like round robin!

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Multipath: Active/Passive, Dual Active, and Active/Active
This post was categorized as Computer History, Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

For more information on earlier updates, see my articles:

• Storage Fixes in VMware ESX Server 3.5 Update 2
• Storage Changes in VMware ESX 3.5 Update 3

Expanded Support for Enhanced vmxnet Adapter

Not specifically a storage change, but the enhanced vmxnet adapter introduced back in the original release of ESX 3.5 now works with most versions of Windows Server 2003 and XP Pro. Look for improved performance when using guest-side SMB and NFS as well as the guest iSCSI initiator. Note that you cannot select this driver when configuring non-Enterprise Edition machines; you have to select Windows Server 2003 Enterprise Edition (64-bit) regardless of which version of Server 2003 you are using.

Expanded SAS and SATA Controller Support

If you’d like to install ESX on a server equipped with a PMC 8011, Intel ICH9 or ICH10, CERC 6/I SATA/SAS Integrated RAID Controller, or HP Smart Array P700m Controller, you’ll find happiness in Update 4.

The Intel controllers are especially important, as we’re seeing them used more and more and this driver is more full-featured than the earlier Broadcom HT 1000 and Intel ICH7 drivers. The Intel ICH9/ICH10 is a dual-mode (IDE/ATA and AHCI/SATA) driver, supports SATA hard drives, SSDs, and optical drives, and now enables VMFS support when in AHCI/SATA mode. It’s not clear whether VMware actually supports VMFS datastores on ICH9/10 SATA, but it says it works. Anyone want to try it out? One thing is certain: You can’t use SATA drives in a shared/clustered environment because SATA does not include reservations. See this tech note and especially this question:

Earlier, it was mentioned that we can create VMFS if we use AHCI/SATA mode. If so, why did VMware not claim VMFS support when using SATA controller running in AHCI/SATA mode?

VMware might decide to add support in the near future. There is no strong need to have VMFS support on a SATA drive, because native SATA protocol does not support reserve/release. Reserve/release is needed if VMFS is used as clustered file system in a shared disk environment.

PXE Boot Support

Rich at VM/ETC points out that Update 4 includes experimental PXE boot support for ESX and ESXi. As he notes, this has major implications for cloud computing platforms, since it means that ESX servers can boot guests without local storage at all. Very interesting! Let’s bet that Update 5 (expected in June or July) will include this as a supported option.

Updated QLogic, Emulex, and LSI Drivers

Like most ESX updates, this one included updated Fibre Channel drivers.

• The QLogic Fibre Channel Adapter driver and firmware (versions 7.08-vm66 and 4.04.06, respectively) include bug fixes and enhanced NPIV support.
• On the Emulex side, driver version 7.4.0.40 supports the company’s HBAnyware 4.0 management software.
• Users of SAS and SCSI LSI MegaRAIDs will find driver version 3.19vmw (megaraid_sas) and 2.6.48.18 vmw (mptscsi) which improves performance and enhances event handling capabilities.

Expanded Sun Storage Array Support

All you StorageTek loyalists out there will be happy to see support for Sun’s low-end StorageTek 2530 SAS array as well as the modular 6580 and 6780 Fibre Channel arrays. It looks like just about every model in Sun’s current storage lineup is now supported in ESX.

Expanded Network Card Support

Support for Gigabit cards is greatly expanded, including HP’s quad-port NC375i and dual-port NC362i and NC360m, Intel’s Gigabit CT and 82574L, and NetXtreme’s BCM5722, BCM5755, BCM5755M, and BCM5756. Intel’s widely-used 10-gig 82598EB cards are now supported as well.

Tweaks and Fixes

Looking through the release notes, a few storage-related tweaks and fixes stand out:

1. WMware can optionally automatically throttle back the queue depth when congestion is encountered. See Controlling LUN queue depth throttling in VMware ESX for 3PAR Storage Arrays for more information.
2. VMklinux module heap size can now be adjusted as LUN queue-depth values are increased. Since tuning LUN queue depths is one common trick of the storage trade to improve performance, especially in queue-stingy systems like ESX, this is welcome news. But call VMware support before you monkey with it!
3. An RDM-related issue where SCSI inquiry data over 36 bytes was truncated or corrupted (for example when using Microsoft VSS and NetApp SnapDrive) has been resolved.

Well, that’s all folks. I suggest you all read the release notes for yourself, and please leave a comment if you see an error in what I wrote here or have something to add!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Storage Changes in VMware ESX 3.5 Update 4
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Update: Check out the latest multi-vendor iSCSI post!

I usually don’t write about other peoples’ articles on this blog, preferring to stick to my own independent work. But this time I’m making an exception.

If you use or are interested in VMware ESX 3.x and iSCSI, you simply must go read Chad Sakac’s post on the topic. Co-written with just about everyone in the industry (including EMC, VMware, NetApp, Dell/EqualLogic, and HP/Lefthand), Chad has put together a “cheat sheet” on the ins and outs of iSCSI connectivity and performance in ESX 3.x.

Top takeaways (and I’ve been preaching about these for a while myself, too!)

• Ethernet link aggregation doesn’t buy you anything in iSCSI environments
• iSCSI HBA’s don’t buy you much other than boot-from-SAN in ESX, either
• The most common configuration (ESX software iSCSI) is limited to about 160 MB/s per iSCSI target over one-gigabit Ethernet, but that’s probably fine for most applications
• Adding multiple iSCSI targets adds performance across the board, but configurations vary by array
• Maximum per-target performance comes from guest-side software iSCSI, which can make use of multiple Ethernet links to push each array as fast as it can go

Thanks for putting together such a great article, guys!

This post can also be found on Gestalt IT: Essential Reading for VMware ESX iSCSI Users!

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Essential Reading for VMware ESX iSCSI Users!
This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

On the contrary, I suggest that the enterprise storage world we know was shaped by singular innovative products of the past. Without these, the IT world might look very different.

While you’re at it, check out my list of the coolest enterprise storage flops!

So let’s take a walk through history, identifying the ten most innovative and important enterprise storage hardware products. But let me note first that this list could be 100 items long, and we all have our favorites. Lots of the storage blogging world contributed their ideas, too!  So I’m setting some arbitrary rules to keep myself on track:

1. The products have to be reflected in the shape of modern enterprise storage for open systems. If their day came and went, they’re not listed here.
2. Hardware only – I’m working on another list for software!
3. Listed products have to have been successful in the market (or they’d probably fail rule number 1 anyway), so I’ll put together a list of cool flops later.
4. For product families, I tried to pick the most influential and innovative 2member.
5. This is an enterprise storage list – items have to be used in big companies, not little PCs.
6. Only one product per company (sorry, IBM and EMC!)

So without further ado, on with the list!

1964 IBM 2311

Let’s get one thing straight: IBM is responsible for much of modern enterprise computing, so it’s no surprise that when it comes to storage, they developed just about everything, including disk drives, floppies, tapes, arrays, and libraries. So why pick this particular piece of kit? Prior to 1964, computer components were developed as a set (the 350 storage system went with the 305 RAMAC, for example), but the 2311 changed that. It was a generic storage device, plug-compatible with a range of computers. If you claimed that the 2311 was the common ancestor of all modern enterprise storage, I would certainly agree with you!

Other notable IBM products worth mentioning:

• The original 1956 RAMAC 350 disk drive unit
• The 1961 301, which used a separate arm and head for each platter (like every disk drive produced since)
• 9-track tape, which dominated from 1964 until the 1/2-inch tape revolution in the 1990s
• The 1970 3330, which added error correction and remained in production for 13 years
• The 1971 introduction of the 23FD floppy disk drive
• The 1973 3430, whose “30/30″ code name caused people to refer to hard drives as “Winchesters”
• 1974′s 3830 “MSS” tape library
• The 1980 3380, which introduced film-head technology
• 2003′s SAN Volume Controller, the first successful SAN virtualization product (after many others failed!)

1976 Shugart SA-400 minifloppy

Alan Shugart’s floppy drive was a massive hit in the home computing market, but why include it in a list of enterprise storage technologies? Because its storage interface set the standard for plug- and protocol-compatible storage in the nascent microcomputer world.

Designing a general product for use in a multitude of systems was truly innovative, and many later computers were literally designed around both the concept and physical form factor of Shugart’s drive. Simply put, this fat floppy drive inspired computer designers to create computers that could accept standard peripherals, the very definition of open systems. The definition of “peripheral” would soon grow to include standard I/O devices like storage, terminals, printers, communications gear, and everything else we know today.

1980 Seagate ST-506

Shugart left his company in 1979, founding Seagate Technology. That company’s first product was the ST-506, a 5 MB hard disk drive that shared its physical shape with the SA-400 and used a similar interface.

Like the floppy, Shugart’s hard drive set the standard for microcomputers, eventually finding its way into enterprise systems, and catapulting Seagate to its current position of disk drive leadership. Higher-capacity derivatives of the ST-506 were fitted with interfaces using Larry Boucher of Adaptec’s SCSI protocol, which continues in use even today. Although the ST-506 wasn’t a “smart” drive, the ecosystem that developed around it was critical.

1984 DEC TK50

Digital Equipment’s introduction of the MicroVAX II in 1985 was accompanied by a new half-inch backup tape drive and cartridge called the TK50. Open-reel tapes had dominated non-disk storage before this, but cartridges quickly replaced reels, leading to the development of tape robotics and (more) reliable off-site storage.

If you picked up a TK50 tape today, you could be forgiven for mistaking it for a modern SDLT or LTO, because these use a similar cartridge and drive form factor, the same linear tape technology, and the same half-inch tape size. The TK50, DLT, and SDLT were the mainstays of open backup for decades.

1987 Auspex

This terrible photo is an actual Auspex press image!

What was the first open systems enterprise storage array? Leveraging Sun’s NFS protocol, and hiding a Sun workstation inside, Wizard of Oz style, the Auspex set the standard for everything we think of as “enterprise” in the open systems world. The company’s storage systems were truly ahead of their time, with ranks of redundant disks years before RAID became common.

Auspex was founded by Larry Boucher, father of SCSI and founder of Adaptec, and raked in sales while others struggled to figure out how to sell in the enterprise.  But their refusal to produce a smaller device would be their undoing. Today’s monolithic arrays owe as much to Auspex as they do to IBM, but the companies producing them could learn a lesson from the company’s demise.

1987 StorageTek 9310 PowderHorn

StorageTeks versatile and scalable 9310 PowderHorn defined backup

StorageTek was instrumental in many enterprise storage developments (see McData, for example), but one product literally transformed the datacenter: the 9310 “PowderHorn”. Consider the “glass house” datacenter tours that the largest companies would use to impress visitors: They would show off their mainframe, their Cray, or their PowderHorn.

This versatile system would accommodate every major tape cartridge format and system type and could scale to truly massive proportions. When disks were expressed in megabytes, PowderHorns held terabytes. 

1995 EMC Symmetrix 3000

EMC's Symmetrix 3000 family brought mainframe block storage technology to open systems

The 1990 introduction of the Symmetrix was the key turning point for the (now) giant of Hopkinton, but the first two generations were mainframe-only. In 1994, the company delivered data replication capability in the form of SRDF, moving key enterprise functionality to the storage array.

But it was the 1995 introduction of the third-generation Symmetrix 3000 that really changed the storage world. For the first time, open systems could connect to mainframe-class storage over the standard SCSI protocol and leverage features like SRDF and (in 1997′s Symm 4) TimeFinder.

One key ingredient often overlooked in the Symmetrix was in-box virtualization the likes of which hadn’t been seen before. It also featured RAID-like sub-disk data protection, though the Symmetrix line never did implement true RAID.

The Symmetrix was redesigned entirely in 2003 to become the DMX. Although it was as different from its predecessor as the New Beetle was from Volkswagen’s original, the DMX line continued many of the philosophical underpinnings set in 1990.

EMC also deserves credit for their original Celerra enterprise NAS system, which picked up where Auspex left off. The company followed these in 2002 with the Centera CAS system, which abandoned many traditional concepts of enterprise storage, like blocks- and filesystem-access and modular and monolithic architecture. The Centera is certainly innovative, but we have yet to see the impact it will have.

1994 Data General CLARiiON

Data General paired the CLARiiON (left) with their AViiON server

It’s easy to forget the world before RAID. But Data General was one of the first to market with a cached RAID system with their 1991 introduction of the HADA. Although this early system was tied to DG’s servers, it donated much of its architecture to a system that became more valuable than the rest of the company: CLARiiON.

Massively successful, and mighty impressive (PC Magazine called the first CLARiiON “amazing”), this modular block storage array set the template for over a decade. There were dozens of copycat arrays on the market within a few years of the introduction of the HADA. The CLARiiON gained Fibre Channel support, was sold to EMC, and remains a mainstay of corporate data centers, albeit with updates to every component. I have administered every generation of CLARiiON array, and can attest to their capability (when properly configured!)

1996 NetApp Multiprotocol Filer

NetApp's F330 Multiprotocol Filer was a huge hit in mid-size businesses and a major upgrade

NetApp (nee Network Appliance) was formed by a group of ex-Auspex engineers who wanted to create a more modular NAS server based on industry-standard hardware. They released their first product, the NFS-only FASServer, in 1995, but it was their August, 1996 introduction of the Windows-compatible Multiprotocol Filer software that really put them on the map.

Combined with their solid F220, F330, and F540 hardware, NetApp now had a serious challenger to Auspex, and their NAS systems blew away dedicated server-based solutions in terms of flexibility and manageability. NetApp’s unified NFS, CIFS/SMB, and HTTP access to content on their unique WAFL file system was impressive at the time, as was the quick setup and administration and the ease of adding drives to their RAID-4 sets. Plus, they brought a new level of friendliness to the data center with their bright colors and silly “toaster” nomenclature.

1999 McData Fibre Channel Director

McData has an interesting history, beginning with Storage Technology (StorageTek), intersecting with IBM, bought and spun off by EMC, and lately acquired by Brocade. Although the company spent a decade developing various peripherals for IBM mainframe systems, their late-1994 introduction of a switching director for ESCON traffic would change the storage world. We take large-scale SANs and LANs for granted today, but McData’s director was astonishing when it was introduced: There were no words to describe it or its function at the time, and period press articles are puzzling!

EMC scooped the company up a year later, but McData was spun out in 1997, adding Fibre Channel support that same year. In the ensuing years, McData became the enterprise SAN “arms dealer”, supplying IBM and EMC with the ED-5000 and ED-6064 directors. I recall commenting at the time that Fibre Channel connectivity built around the McData director was the first Storage Area Network worthy of the name. McData went IPO in 2000 and was purchased by rival Brocade in 2006. McData’s director architecture survives and thrives today against fierce competition from Brocade, Cisco, and others.

Others To Consider

What would you have included in this list? Here are some honorable mentions that I wish I had room for:

1. StorageTek “Aegis” L700 library
2. Brocade “LOOM” 2xxx switches
3. Adaptec SCSI HBAs
4. Copan MAID
5. EqualLogic iSCSI arrays
6. EMC Centera
7. HP EVA
8. HDS USP
9. Compellent Storage Center

Responses

Here’s a list of responses that others have posted:

• Marc Farley’s Top 10 List
• Alex MacDonald’s Bottom 10 List

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Top Ten Innovative Enterprise Storage Hardware Products
This post was categorized as Computer History, Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

PVSCSI gives virtual machines direct access to SCSI and FC HBAs, and plays nicely with NPIV (Xensummit diagram by Fujitsu)

Functionally similar to VMware’s Physical Compatibility Mode for Raw Device Mode (RDM) volumes, PVSCSI enables certain applications that require direct SCSI communication to function in a virtual environment. Examples include Oracle RMAN, backup applications, and potentially SAN management software.

PVSCSI plays nicely with N_Port ID Virtualization (NPIV), too, so you don’t need to assign a physical HBA port to each guest – they can all share a port or two, and each would have his own N_Port on the Fibre Channel fabric.

In their Xensummit presentation about PVSCSI, Fujitsu showed impressive performance numbers, demonstrating that the technology doesn’t cause much of a performance hit even though it is substantially more complicated than the alternative approaches. I do wonder how PVSCSI managed to outperform Dom0 with 128k writes, but let’s chalk that up to insignificant variations in timing…

Now if only Xen would update the (3.2-era) readme files on their download page!

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Xen 3.3 Update Brings Paravirtualized SCSI
This post was categorized as Enterprise storage, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.