The Future of Home Storage

Posted on Oct 12, 2008 by Stephen

Homes now need data storage as well as closets...

This is part of an ongoing series of longer articles I will be posting every Sunday as part of an experiment in offering more in-depth content.

Along with my professional focus on enterprise storage systems, I’m enamored of home networking, and recently passed the three terabyte mark at home! This got me thinking about where home storage is heading.

As you can see in the photo, my office closet is overflowing with computer equipment (and one sweet guitar), but my data storage is much better organized. I have a hacked Linksys NSLU2 with 500 GB as a file server, a 500 GB PC backup disk, a 160 GB Time Machine disk, 1 TB of TiVo storage, and the rest. But wouldn’t it be nice if this could all be combined into some kind of super home server?

Past Failures: Home Servers

Home storage appliances and servers have come and gone over the year, with none seeming to make much of a mark. The market remains littered with UPNP media servers and home NAS boxes dashed on the shoals of an unappreciative public. Nearly every home network device company has produced one or two home storage servers, none of which have succeeded. Although I use a Linksys NSLU2 at home, I had to hack its Linux software and completely replace Linksys’ features to create a useful device! The un-hacked NAS devices of Buffalo, Western Digital, Netgear, and the rest have generally failed to find buyers as well. So far, consumers seem content with simple USB and FireWire external drives.

The most adventurous home storage servers came from Zetera and Ximeta, both of whom relied on proprietary IP SAN protocols. Note that these were SAN products, sharing block storage over Ethernet, rather than conventional NAS solutions. Both required drivers, limiting client support. The one Zetera buyer I know was pleased by the performance but never used the device as anything but a large hard drive for one PC.

Then there is Microsoft. Recall that the latest Windows Home Server is only their latest attempt to enter this market, and yet I know of no one who has adopted the device. The same can be said of the various media center servers from Microsoft and others. At this point, it seems likely that the future of home storage servers will not come from Microsoft, though their two XBox generations have great potential as clients.

Even EMC has entered the market with their nifty (but largely unnoticed) LifeLine product and Iomega acquisition. Supporting file services and backup for computers as well as audio and video for media players, EMC positions LifeLine much like their Retrospect backup product, but goes further in offering a complete software solution for hardware OEMs wanting to offer a non-Windows home server. Although an impressive offering, it is too early to tell if EMC will have much success with this product.

The Sleek, Shiny Elephant in the Living Room

Of course, there is one company that sells media players and servers by the bushel, complete with sleek, shiny interfaces. Apple’s tremendous success with the iPod has led to their iTunes software becoming the dominant media organization platform, complete with its own proprietary discovery and sharing protocol. Now, with the Apple TV and video iPods, the company is broadening into more media categories. Surely their dominance here puts them in a special position when it comes to setting the stage for a home server or storage revolution.

They also have a strong position in the world of dedicated home storage. Their Airport products are among the only routers to be widely implemented with shared storage. Although many other companies offer similar products, low customer understanding means that these functions are not widely used. And the new Time Capsule device is surely already the most widely-used home NAS product.

But Apple has not yet shown any home server strategy. Administering multiple iTunes servers can be frustrating for users, with no inter-iTunes synchronization or centralization capability. Although the Mac Mini, Apple TV, or Time Capsule could certainly be seen as a home server, the company does not position them as such in the market. Indeed, some iTunes users like myself rely on compatible third party media servers like Firefly and TwonkyVision rather than using iTunes itself. Still, rumors of an Apple home server persist.

One issue for Apple is their reliance on proprietary protocols. Although the Bonjour discovery protocol is certainly simpler than UPnP in practice, Apple stands alone in relying on it. They also steadfastly stick to AFP for NAS and DAAP for remote media streaming. This limits the number of third-party clients and servers that can be used with their hardware and software.

The Future is Friendly

Although Apple has not yet tipped a home storage strategy beyond Time Capsule and Airport Extreme, they are best positioned to deliver a real home storage solution. A simple step would be to create an iTunes media server integrated with Time Capsule and add client/server media synchronization. The company already has OS X backup and file services integrated, and this move would further centralize the digital home around Apple products. But the company’s reliance on closed protocols like DAAP is worrisome, since it locks consumers into nearly all-Apple solutions.

Microsoft’s Media Center and Home Server combination, based around UPnP, shows great promise, with many compatible third-party clients and servers already available. But my own experience with the solution has not been at all positive (I still can’t get my Roku SoundBridge, Vista Ultimate laptop, and Media Center PC to see each other!), leading me to question the viability of this option.

Although Apple or Microsoft could come to dominate, I suspect the future of home storage is out of both companies hands. A number of others are working on improved home server experiences, including EMC’s LifeLine and the expanding use of Debian Linux and open source tools. But all could be sidelined by improved Internet-based services. Google, Microsoft, and Apple continue to expand their online consumer suites with greater storage, synchronization, multimedia integration, and all have the potential to reduce or eliminate the need for in-home storage.

Although I cannot yet tell which service will win, one thing is certain: Consumers demand friendly, flexible solutions. They don’t want to fuss with their media, and they don’t want simple shared storage. They want integration with multiple devices and flexibility to access their content on any device. The first company to offer a simple, flexible storage server for the home will surely be on the right track!

You might also want to read these other posts...

Tags: AFP, Apple, Apple Airport, Apple TV, Bonjour, Buffalo, DAAP, EMC, Firefly, FireWire, IP storage, iTunes, LifeLine, Linksys, Linux, MAC address, Mac Mini, Microsoft, NAS, nas storage, NetGear, network attached storage, network storage, NSLU2, OS X, Retrospect, Roku, SAN, SoundBridge, storage area network, Sunday series, Time Capsule, TwokyVision, Untitled, UPNP, USB, Western Digital, Windows Home Server, Windows Vista, XBox, Ximeta, Zetera

http://cdplayer.livejournal.com/ http://cdplayer.livejournal.co

I can reflect on your home storage organization musings, as I am going through the similar experience. For about 5 years I relied on a ASUS barebone PC running Linux with couple HDDs shoved into it as my file server, running Samba and then atalkd. I chose barebone because I wanted thing to be as quiet as possible, as it sits in my living room. This was quite successful, but had a few drawbacks: not really silent, as fans would periodically come on in short bursts, and drives do get hot in there (especially old ones, especially when there was more than one). Also mt-daapd kind of sucked, especially when I got my AppleTV, which refused to see it.

What I wanted is a centralized iTunes library plus iTunes itself (especally after Apple Remote app came out for iPhone/iPod) – I have Airport Express with iPod Hi-Fi in my bedroom and wanted to be able to play music on it from my main library, but don’t want to keep my or my wife’s laptop running, streaming to Airtunes. To do that, I needed a computer with iTunes running. I tried to get iTunes on Linux running (with wine), which didn’t work, so I abandoned it, not wanting to spend too much time on it. Necessary note – our home now is 100% apple environment, running Windows only in Vmware, when needed (which is almost never).

To cut long story short, about a month ago I decommissioned my Linux box and replaced it with a Mac Mini with two external HDDs – one on Firewire, another on USB. First disk is the main storage, while the second serves as Time Machine backup disk for built-in system disk and external media storage disk. I chose to use two enclosures rather than one hosting two disks to avoid the risk of a common element failure, such as SATA-USB bridge or power supply. I can’t quite rationally explain my choice to use different interface (USB+firewire), but both enclosures have all three interface types (USB+FW+eSATA), and it “felt right” to do it. The external disk arrangement gives very good options to expand storage further, when needed – firewire can be chained, and there are two more USB ports available (and a hub can be used as the last resort).

I used WD AV hard drives, which are meant to be able to work 24×7 at highter than usual operating temperatures, and do have intelligent power management (meaning they change their rotation speed, depending on demand). All this is now lives directly inside my TV cabinet, as its footprint is very small and it makes very little noise. I am running iTunes on it (auto-login + start iTunes as login item), and can play music to my remote speakers, driving it from iPhone or iPod touch. I am also sharing the whole storage HDD for use from all other computers in the house for files, and mount it from my jailbroken AppleTV for playing movies. Oh, and I have my APC SmartUPS wired up to Mac mini via USB extension vable, so the thing can shut itself down if there’s power outage (yes, mac mini and both drives get power from that UPS, too).

So far this arrangement worked for me beautifully. Apple Remote Desktop is perfect for remote management, it even shows you the post-logout screen when you’re installing OSX patches/updates (it logs you out before running patch installation). File system performance I can’t really complain about – with both Mac Mini and my desktop being on a GigE network, I routinely get throughput of about 40 Mb/s (which I think is limited by Firewire/USB speeds anyway). I find this pretty plenty for what we use it for (shared storage).
http://cdplayer.livejournal.com/ http://cdplayer.livejournal.com/

I can reflect on your home storage organization musings, as I am going through the similar experience. For about 5 years I relied on a ASUS barebone PC running Linux with couple HDDs shoved into it as my file server, running Samba and then atalkd. I chose barebone because I wanted thing to be as quiet as possible, as it sits in my living room. This was quite successful, but had a few drawbacks: not really silent, as fans would periodically come on in short bursts, and drives do get hot in there (especially old ones, especially when there was more than one). Also mt-daapd kind of sucked, especially when I got my AppleTV, which refused to see it.

What I wanted is a centralized iTunes library plus iTunes itself (especally after Apple Remote app came out for iPhone/iPod) – I have Airport Express with iPod Hi-Fi in my bedroom and wanted to be able to play music on it from my main library, but don’t want to keep my or my wife’s laptop running, streaming to Airtunes. To do that, I needed a computer with iTunes running. I tried to get iTunes on Linux running (with wine), which didn’t work, so I abandoned it, not wanting to spend too much time on it. Necessary note – our home now is 100% apple environment, running Windows only in Vmware, when needed (which is almost never).

To cut long story short, about a month ago I decommissioned my Linux box and replaced it with a Mac Mini with two external HDDs – one on Firewire, another on USB. First disk is the main storage, while the second serves as Time Machine backup disk for built-in system disk and external media storage disk. I chose to use two enclosures rather than one hosting two disks to avoid the risk of a common element failure, such as SATA-USB bridge or power supply. I can’t quite rationally explain my choice to use different interface (USB+firewire), but both enclosures have all three interface types (USB+FW+eSATA), and it “felt right” to do it. The external disk arrangement gives very good options to expand storage further, when needed – firewire can be chained, and there are two more USB ports available (and a hub can be used as the last resort).

I used WD AV hard drives, which are meant to be able to work 24×7 at highter than usual operating temperatures, and do have intelligent power management (meaning they change their rotation speed, depending on demand). All this is now lives directly inside my TV cabinet, as its footprint is very small and it makes very little noise. I am running iTunes on it (auto-login + start iTunes as login item), and can play music to my remote speakers, driving it from iPhone or iPod touch. I am also sharing the whole storage HDD for use from all other computers in the house for files, and mount it from my jailbroken AppleTV for playing movies. Oh, and I have my APC SmartUPS wired up to Mac mini via USB extension vable, so the thing can shut itself down if there’s power outage (yes, mac mini and both drives get power from that UPS, too).

So far this arrangement worked for me beautifully. Apple Remote Desktop is perfect for remote management, it even shows you the post-logout screen when you’re installing OSX patches/updates (it logs you out before running patch installation). File system performance I can’t really complain about – with both Mac Mini and my desktop being on a GigE network, I routinely get throughput of about 40 Mb/s (which I think is limited by Firewire/USB speeds anyway). I find this pretty plenty for what we use it for (shared storage).
http://blog.fosketts.net sfoskett

This sounds awesome! It’s exactly the setup I’ve been dreaming of – and I think I might pull the trigger!

I’m getting about 5 MB/s to my NSLU2 setup – not great.
http://stephen.fosketts.net Stephen

This sounds awesome! It’s exactly the setup I’ve been dreaming of – and I think I might pull the trigger!

I’m getting about 5 MB/s to my NSLU2 setup – not great.
KendallK

You mention the fact that Apple's reliance on protocols like DAAP is worrisome, since it locks consumers into nearly all-Apple solutions. That's exactly why Apple does it. They want people to buy Apple and Apple only. Unlike Microsoft, the OS company that has a stranglehold on the business market, Apple is a Hardware company that makes it's own OS. They do this so that they can capture a segment of the population that appreciates easy-to-use solutions and slick design. Apple has taken a complicated beast of home media storage and made it relatively simple. I think you're right… the home server option may be on the horizon, but only when they get it dialed in to the point where you can teach your Grandma how to use it.

By the way… I love your blog. I'm an IT manager at a large non-denominational church. I love Macs (used them for media production for 15 years), and your info on updating mac-minis is giving me the leverage I need to install easy-to-use Mac computers in all of our meeting rooms, giving us a shared music library (hosted on our Xserve) and a dvd / presentation / youtube / whatever you need solution for presentations.
http://blog.fosketts.net sfoskett

Thanks for posting your thoughts! I'm glad to help you!

Yes, Apple loves proprietary protocols for their core products. They lock us in and give Apple the ability to do whatever development they want without waiting for anyone. Smart!

I'm currently converting my home network to all-Apple, with iTunes as a server, AirPort Express as a player, and iPhone as a remote. Lock-in works! Hahaha!
http://blog.fosketts.net sfoskett

Thanks for posting your thoughts! I'm glad to help you!

Yes, Apple loves proprietary protocols for their core products. They lock us in and give Apple the ability to do whatever development they want without waiting for anyone. Smart!

I'm currently converting my home network to all-Apple, with iTunes as a server, AirPort Express as a player, and iPhone as a remote. Lock-in works! Hahaha!
http://innovativestorageideas.com/ Storage Suggestions

Looking for extra storage space? Just look up. The ceiling is the most overlooked storage area in the garage. There are store bought kits that can turn wasted space into useful garage storage, or a few sheets of plywood laid across the ceiling beams will work just as well.
http://innovativestorageideas.com/ Storage Suggestions

Looking for extra storage space? Just look up. The ceiling is the most overlooked storage area in the garage. There are store bought kits that can turn wasted space into useful garage storage, or a few sheets of plywood laid across the ceiling beams will work just as well.
http://innovativestorageideas.com/ More Storage

Looking for extra storage space? Just look up. The ceiling is the most overlooked storage area in the garage. There are store bought kits that can turn wasted space into useful garage storage, or a few sheets of plywood laid across the ceiling beams will work just as well.

blog comments powered by Disqus