One of the great things about being a Microsoft MVP is the access I get to Microsoft software and staff. As I mentioned in my post about the 10 cool storage features from the 2009 Microsoft MVP Summit, I was able to preview a lot of what Microsoft is doing with their Server software and storage features. And the best part is that the Microsoft product teams are keenly interested in our feedback and suggestions. I’m told, for example, that the awesome iSCSI Quick Connect feature in the new Windows iSCSI initiator software was developed based on my feedback!

As I note on my Windows Storage Server 2008 preview on Gestalt IT, Microsoft has always kept WSS close to the vest. It’s only available to OEMs, not retail customers, and has never even been shared with TechNet or MSDN subscribers in the past. So I was really pleased when Microsoft gave the File System Storage MVPs access to a beta version of WSS 2008 so we could get a feel for all of the new features. I’ve also had some great conversations this week with the Microsoft product managers responsible for it.

What’s exciting about Windows Storage Server 2008?

1. It includes all of the storage enhancements in Windows Server 2008, including SMB 2.0 for much much faster file servicing over higher-latency links, SMfS, FSRM, enhanced VDS, and failover clustering.
2. WSS is the only way to get access to Microsoft’s iSCSI target software. It’s been improved in many ways from the prior releases, but its support for what Microsoft calls dual-active clustering is probably its most notable feature: You can’t share the same active LUN between cluster members, but each can have its own active LUNs and the can all fail over in the event that one member goes down.
3. The included single-instance storage (SIS) file-based deduplication has been much improved, scaling to 128 volumes per server and millions of files. It’s still not as effective capacity-wise as block-level deduplication (which I’d love to see, hint hint), but the performance is solid enough to use it for primary storage with production applications.
4. Probably the coolest feature exclusive to Windows Storage Server 2008 is its new browser-based remote administration capability. Just point your browser to the Storage Server machine (for example, “http://wss/desktop”) and you’ll get a full ActiveX version of RDP. Don’t use Internet Exploder? Firefox and Linux users will get a Java-based RDP instead! I will cover this feature more in the future, but let’s just say that every operating system should offer this!

Want to try Windows Storage Server for yourself? Breaking from the past, Microsoft will soon (like next week!) allow TechNet subscribers to download the full install. OEMs have a sekrit back-door site to try it out, too.

One more thing… This will be the last release of Windows Storage Server. There won’t even be a special Storage Server version of Server 2008 Service Pack 2! Starting now, Storage Server is just an optional feature of Windows Server. Purchasing and production use will still be limited to storage OEMs, but Microsoft has finally reconciled Storage Server with the rest of the Windows Server world. I imagine that most OEMs will release Service Pack 2 updates for their Storage Server customers shortly, and that future versions of the product will come closer to the base Server versions than WSS 2008. Although I can’t share what I know, I will say that Microsoft is continuing active development on their iSCSI target, single-instance storage, and other Storage Server features. I imagine that Windows Server 2008 R2 will support storage systems in the very near future!

On a personal note, reading A Brief History of Windows Storage Server Releases from the Microsoft Storage Server blog reminded me of the original unveiling of Windows Storage Server at Storage Decisions Chicago in June, 2003. The company loaded us all on one of those lake cruise boats with some celebrity impersonators, chocolate “Oscar”-style statues, an open bar, and a band. Good times ensued!

For more details, check out my Gestalt IT piece, Windows Storage Server-Based Systems Step Into 2008

Microsoft is detailing the new version of Windows Storage Server 2008 in a webcast Thursday at 8 AM Pacific. You should also check out the Microsoft Storage Server blog.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | I Can Finally Talk About Windows Storage Server 2008!
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Although deduplication of storage is nothing new, with Data Domain and other making hay with the technique for years, it has never been ready for prime time – reduction of active primary storage applications like email and databases. Instead, deduplication has been relegated to second- or third-tier status, deduplicating archives and backup data. But change is in the air, and deduplication vendors are starting to bustle towards the bright lights of primary storage.

Stone Knives and Bear Skins

We have all been here before, of course. Back at the dawn of the personal computer era, data compression was a hot topic of conversation. I recall being so impressed by an article in Byte (1986:5, p99) outlining Huffman coding that I tried cooking up an implementation in Atari BASIC. Lossless compression has a magical pull to the geek in many of us – redundant data just wants to be eliminated!

Stacker dominated the disk compression world - until Microsoft introduced DOS 6.0

Companies soon applied compression to primary storage, especially the limited storage in personal computers. Stacker was a hit after 1990, until Microsoft built a workalike, called DoubleSpace, into DOS 6.0 in 1993, leading to a historical lawsuit. I personally used the ADDSTOR disk compression built into DR-DOS 6.0 to stretch two more years out of the 20 MB MFM hard drive in my AT&T PC6300 at WPI.

But something funny happened in the late 1990s: Compression began to lose its luster. Compressing data always takes quite a bit of CPU power, but this was offset somewhat by the truncated data transfers and more-efficient file system layout afforded in early PCs. But as disks got larger and faster, using precious CPU time to save space seemed less and less compelling. Today, although nearly every operating system includes built-in compression of files, folders, or perhaps disks, these features are rarely used. And compression was never popular in the performance-sensitive enterprise space.

Deduplication Has a Nice Ring

Although traditional fine-grained compression has not been very successful in the enterprise, its lanky cousin, single-instance storage, has long found niche jobs. Applications from databases to email systems to file servers have long had the ability to recognize to requests to store the exact same file or record, and to store just a single instance in this case. Even file systems have the ability to do single instance storage through the use of links, though this is initiated by the user rather than in an automated fashion.

In the late 1990s, FilePool began developing a content-addressable storage device, which was acquired by EMC in 2001. This device, later known as the Centera, was one of a number of storage platforms targeted at the archiving market introduced this decade. At the same time, virtual tape libraries made the jump from the mainframe to open systems. Both devices, being outside the critical path of performance but offering massive capacity, were well-suited to implement advanced capacity optimization technologies that combined the concepts of compression with single-instance storage. Thus was created the modern world of data deduplication.

What we think of as deduplication is neither fish nor fowl: It assesses larger “chunks” of data than compression technologies, delivering greater capacity savings and potentially reducing performance impact, but is more flexible than single-instancing, recognizing the similarities within files or objects.

But it is still maddeningly difficult to scale deduplication while maintaining performance. Rather than fight to maintain reasonable write throughput, most deduplication products have switched to post-processing, deferring their work to quieter times.

It’s Not Just for Breakfast

Regardless of their methods or underlying technology, no deduplication vendor has stood up to support challenging low-latency or high-throughput production applications, however. NetApp was the first to raise the issue of support for production applications, but although they tout the technology for VMware, they haven’t exactly been shouting from the rooftops to get their A-SIS deduplication technology deployed in other high-I/O applications. And I haven’t seen Hifn’s card yet.

Yesterday, I mentioned that greenBytes was adding deduplication to their ZFS-based storage array for primary data. And now Riverbed has fired another shot over the bow, repurposing their (deduplicating) WAN accelerator product for primary (file) storage. They might be able to pull it off, too, since they have a long list of customers who are already enjoying the technology in production. It’s not a stretch to suggest that Riverbed’s appliances can scale to handle production data loads. Although it’s file-only, I can imagine quite a few scenarios where this tech could really yield benefits. Could we come full-circle, with deduplication finally reaching the enterprise storage world?

© sfoskett for Stephen Foskett, Pack Rat, 2008. | Deduplication Coming to Primary Storage
This post was categorized as Computer History, Enterprise storage, Features, Virtual Storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.