Disk Utility in Mac OS X can create an encrypted "SparseBundle" compatible with Dropbox

Security is always possible but has rarely been easy. Storing unencrypted files and folders is much simpler and more compatible than going through the effort to use encryption. But the recent security flub at Dropbox set me looking for a secure way to store data there. Here’s a quick and easy way to create a secure, Dropbox-compatible disk image for Mac OS X.

Dropbox-Friendly

You might also want to read Keep Multiple Macs in Sync with Dropbox and Compression, Encryption, Deduplication, and Replication: Strange Bedfellows

Yesterday, “Storagemistress” suggested a method of securely hiding … “stuff” … using the excellent open-source software, TrueCrypt. This has the advantage of cross-platform compatibility and obfuscation (she suggests using a movie title as “cover”) but isn’t all that Dropbox friendly. Change one bit in that file, and the whole thing will have to be re-synced.

Mac users have a great alternative method: The SparseBundle. This has a few advantages over monolithic disk images:

1. SparseBundles are thin provisioned, growing as data is added and only taking up as much space as is actually used (to the nearest 1 MB), while conventional disk images are entirely provisioned when they are created.
2. SparseBundles store data in 1 MB “bands” that can be independently synchronized with rsync or Dropbox, as opposed to conventional images that are stored as a single file.

These two elements make SparseBundles excellent for securing valuable data while protecting it off-site. Dropbox loves the 1 MB bands, and I feel better knowing my data is encrypted in case the service has another “oops” moment.

Create Your SparseBundle

Creating a SparseBundle Disk Image is fairly straightforward. Note that this only works in Mac OS X 10.5 “Leopard” and newer releases.

1. Launch Disk Utility – I like to just type that into Spotlight, but it’s in Applications/Utilities
2. Click “New Image” as seen in the image at the top of this post
3. Give your SparseBundle a file name in “Save As” and locate it in a folder (e.g., your Dropbox folder!)


Disk Utility is all you need to create a secure storage location

4. Enter a friendly name for the disk image – this can be the same or different from your file name
5. We’ll leave “Format” and “Partitions” untouched



6. Click the dropdown menu for “Size” and select “Custom” to enter a custom maximum file size



7. Click “Encryption” to enable 128- or 256-bit encryption



8. Click “Image Format” and select “sparse bundle disk image”
9. Now click “Create” to start the image process



10. Since we selected encryption, a new dialog box will appear asking for a password – this is what you will enter every time you use this image, and Apple helpfully rates your password strength for you
11. Click “Ok” and you’re done – the bundle will mount as a new drive

As you add and remove files from this disk image, they will be encrypted and stored in 2 MB “slices” (actually files in a directory). These work great with Dropbox, Rsync, and many other utilities.

This image can be mounted on any Mac, provided the password is known. But don’t try to mount it on more than one machine simultaneously!

Note: If you want to obscure these files, you can rename the SparseBundle. It won’t be usable from Finder, but you can still mount it from the command line using hdiutil. But it’ll be a directory of equal-sized files, so you won’t fool anyone who knows what they’re doing.

Stephen’s Stance

Encryption is an important tool for individuals regardless of what they’re storing. Given the recent security failings of Dropbox, I highly recommend using methods like this to secure your important data before using the service!

© sfoskett for Stephen Foskett, Pack Rat, 2011. | Mac Users, Secure Your Stuff in Dropbox
This post was categorized as Apple, Everything, Personal, Terabyte home. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Do we really want to escape our cages?

Things are getting awfully complicated, aren’t they? The custom parallel CPUs, proprietary communications networks, and encrypted data ports require extensive training, special tools, and a computerized reference library to comprehend, much less debug. And the manufacturers, who derive much of their money and differentiation from warranties and authorized repair centers, are loathe to see independent shops get a piece of the action.

Yes, today’s automobile market sure is complex. Did you think I was talking about computer systems? The current state of the retail auto industry has a lot to teach us in the computers and communication devices! There are many parallels between cars and computers, after all.

Selling an Ecosystem

The Lincoln Highway gave early motorists a reason to want a car

Early automobiles were a lot like early computers: Many were hand-made and none was all that useful. Sure, Bertha Benz managed to drive to Pforzheim in 1888 to visit her mother, but she had to repair her husband’s motorwagen multiple times during the 60 mile trip. While automobile technology has certainly improved since then, the real innovation that drove the world to drive cars was the ecosystem that surrounds them.

Carl Fisher may not be familiar to most, but he deserves as much credit as Henry Ford, Billy Durant, Ferdinand Porsche, Eiji Toyoda, and so many other engineers and industrialists. Fisher realized that, in addition to being technically sound and affordable, the automobile must be embraced by the public to succeed. He switched from supplying headlights to convincing the public that car ownership was a desirable thing, creating a multi-brand dealership, founding the Indianapolis 500, and redirecting the Good Roads Movement to support highways suitable for long-distance automobile trips.

Fisher’s insight was similar to that of Steve Jobs’ at the first incarnation of Apple: People want a place in an ecosystem not just a piece of technology. Fisher’s Lincoln and Dixie Highway projects gave the public a reason to buy a car. Similarly, Jobs’ Apple II and Macintosh projects focused as much on what people would do with the computers as what they were made of. Much of Apple’s current success is due to the way customers experience the company, from the upscale yet inviting Apple Stores to the curated iOS App Store.

Apple’s “walled garden” approach to computers and mobile devices may draw derision from techies but the public loves it. They don’t want a Macintosh, iPhone, or iPad; they want a thing that lets them do what they do. The ability to easily install and use applications and accessories easily outweighs the arbitrary and capricious limitations placed on them by Apple and AT&T. Regular people don’t want multiple windows on screen with multitasking applications; they want instantly to turn on their “pad”, perform a task, and set it aside. The Apple ecosystem empowers this.

Total Customer Experience

Lexus set the benchmark for modern automobile ownership after their 1989 debut in the United States. The company created a total customer experience, from inquiry to sales to ownership and support, recognizing that upscale buyers would pay a premium to be coddled by their car company, not just their car. In fact, if Apple is copying anyone with their glass Stores and Geniuses and AppleCare it is Lexus.

Every premium automobile marque labors to offer a Lexus-like customer experience today. Basic mechanical warranties became bumper-to-bumper service, and dealer service bays transformed into destinations with free car washes and cappuccino bars. Like Apple, today’s carmakers want to coddle their customers and shield them from the cruel world of discount stores and independent repairmen.

Right to Repair and DMCA Exemptions

Proprietary systems and "Total Customer Experience" threaten to marginalize independent shops

But Apple and the automakers might have stepped a bit too far, turning from walled garden to gilded cage. Worried that openness would corrupt both their pristine equipment and customer experience, manufacturers began to lock out unauthorized parts and modifications. But in so doing they also imposed restrictions that threaten to turn owners into mere users at the whim of the companies.

My Mazda MX-5 Miata is far from a walled-garden “experience” luxury car. I love it, in fact, because it’s a wonderfully basic, visceral, and mechanical car. But the Bose audio system that came with it is as complex and locked-down as anything in a Lexus. Adding a basic auxiliary input required decoding a proprietary bus and creating a custom chip to emulate the tape deck module, an admirable piece of work indeed!

With even a simple car’s audio system gaining this kind of complexity, it’s no wonder that independent repair shops are feeling squeezed out. This is why the Massachusetts “Right to Repair” law is getting such widespread attention. Independent repair shops (and yes, aftermarket equipment makers) are attempting to force automakers to allow them access to the proprietary technology in their cars. Yet even if they break open the gilded cage, the walls of “total customer experience” may prove harder to crack.

The exemptions to the DMCA handed down by the copyright office this week are remarkably similar. Although the Office did not go as far as many would have liked, they firmly stated that companies shouldn’t use copyright law to restrict the use, preservation, and repair of certain products. Of particular interest to both DMCA opponents and Right to Repair fans is the long-term implication of walled-off technology. What happens when a “dongle” or key is no longer manufactured or supported? Will anyone be able to repair a Mercedes-Benz, Audi, Lexus, or Apple in 20 years?

Stephen’s Stance

Consomers like living in a gilded cage

Unquestionably, the world is worse off if manufacturers completely restrict basic access to their products. The short-term impact on buyers, though concerning, are trumped by the long-term implications. The copyright office is wise to realize that DRM, encryption, hardware lock-in, and “dongles” threaten to effectively delete audio, video, print, and entertainment properties once their manufacturer decides they are obsolete. And Right to Repair is urgently needed to keep historically-important automobiles from becoming useless hunks once their manufacturers stop supporting them.

But the rosy picture painted by the EFF, iPhone jailbreakers, independent mechanics, and other freedom-loving folks aren’t entirely justified. As Carl Fisher and Steve Jobs proved, consumers want an ecosystem, and enjoy luxuriating in a gilded cage. The success of the iTunes store demonstrates that buyers value convenience over freedom when it comes to entertainment, making Apple’s move to remove DRM from their audio content gratifying indeed. And Apple and Lexus buyers remain loyal to the companies despite widely-publicized product defects. Perhaps Right to Repair and the DMCA exemptions are about protecting consumers from themselves rather than from predatory manufacturers.

Image credits: “Lemur Cage 2″ by Stephen Foskett, “Lincoln Highway Bridge, Tama, IA” by cwwycoff1, ”Mechanic Shop” by Tomas, “The View Through The Furry Handcuffs” by Annie in Beziers

© sfoskett for Stephen Foskett, Pack Rat, 2010. | Walled Gardens, Gilded Cages, and Right to Repair
This post was categorized as Apple, Computer History, Everything, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

This blog post is probably out of date. If you want to set up Exchange ActiveSync, you should instead consult one my guides:
	iPhone Exchange ActiveSync Setup		iPad Exchange ActiveSync Setup
	iPhone ActiveSync Troubleshooting		iPad ActiveSync Troubleshooting

Is your non-3GS iPhone locked out of your Exchange 2007 ActiveSync server after upgrading to iPhone OS 3.1? It’s a feature, not a bug! Here’s how to get older iPhones up and running with Exchange Server 2007 SP1!

What’s The Problem?

Microsoft Exchange Server 2007 SP1 added a feature to require mobile ActiveSync devices to encrypt data, enhancing security. Before iPhone OS 3.1, all iPhones incorrectly told the Exchange server that they supported on-device encryption. This allowed all iPhone hardware to function with Exchange 2007 SP1 servers that required device encryption. But original and 3G iPhones do not support device encryption, undermining corporate security policies.

The iPhone 3GS hardware actually does support device encryption, and iPhone OS 3.1 correctly reports this capability. But iPhone OS 3.1 also (correctly) reports that earlier hardware (the original iPhone and the iPhone 3G) does not support device encryption, so some Exchange 2007 SP1 servers refuse to allow them to connect. Oops!

What’s The Solution?

There are four possible solutions, three of which require IT assistance. Your Exchange administrator can research the meaning and implications of these options:

1. Disabling device encryption allows all iPhones to connect, but does not force any Exchange ActiveSync device to encrypt data. This is not a great solution from a security perspective, so don’t bother trying to convince IT to implement it!
2. Allowing non-provisionable devices enables all iPhones to connect but weakens security in general, allowing each device to enforce or ignore policies. This is a slightly better solution, since encrypting devices like the iPhone 3GS will encrypt, but it’s still not a great idea.
3. Creating a special policy for “old” iPhones and applying it selectively is probably more acceptable. Administrators can allow certain users to ignore the device encryption policy but still apply it to all others. This commandlet (from Krypted) will create such a policy:
   New-ActiveSyncMailboxPolicy -Name iPhone -AllowNonProvisionableDevices $true
4. Upgrading to an iPhone 3GS is probably the best answer. IT doesn’t need to get involved (as long as you know how to configure Exchange ActiveSync) and no security policies need to be weakened to make it work.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | How iPhone OS 3.1 Locked Some Out Of Exchange, And How To Fix It
This post was categorized as Apple. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Does data encryption throw storage efficiency out the window? Not always!

One of the great ironies of storage technology is the inverse relationship between efficiency and security: Adding performance or reducing storage requirements almost always results in reducing the confidentiality, integrity, or availability of a system.

Many of the advances in capacity utilization put into production over the last few years rely on deduplication of data. This key technology has moved from basic compression tools to take on challenges in the fields of replication and archiving, and is even moving into primary storage. At the same time, interconnectedness and the digital revolution has made security a greater challenge, with focus and attention turning to encryption and authentication to prevent identity theft or worse crimes. The only problem is, most encryption schemes are incompatible with compression or deduplication of data!

Incompatibility of Encryption and Compression

Consider a basic lossless compression algorithm: We take an input file consisting of binary data and replace all repeating patterns with a unique code. If a file contained the sequence, “101110″ eight hundred times in a row, we could replace the whole 4800-bit sequence with a much smaller sequence that says “repeat this eight hundred times”. In fact, this is exactly what I did (using English) in the previous sentence! This basic concept, called run-length encoding, illustrates how most modern compression technology functions.

Replace the sequence of identical bits with a larger block of data or an entire file and you have deduplication and single-instance storage! In fact, as the compression technology gains access to the underlying data, it can become more and more efficient. The software from Ocarina, for example, actually decompresses jpg and pdf files before recompressing them, resulting in astonishing capacity gains!

Now let’s look at compression’s secretive cousin, encryption. It’s only a small intellectual leap to use similar ideas to hide the contents of a file, rather than just squashing it. But encryption algorithms are constantly under attack, so some very smart minds have come up with some incredibly clever methods to hide data. One of the most important advances was public-key cryptography, where two different keys are used: A public key used for writing, and a private key to read data. This same technique can be used to authenticate identity, since only the designated reader would (in theory) have the key required.

Cryptography has become exceedingly complicated lately in response to repeated attacks. Most compression and encryption algorithms are deterministic, meaning that identical input always yields the same output. This is unacceptable for strong encryption, since a known plaintext attack can be used with the public key to reveal the contents. Much work has focused on eliminating residues of the original data from the encrypted version, as illustrated brilliantly on Wikipedia with the classic Linux “tux” image. The goal is to make the encrypted data indistinguishable from random “noise”.

What happens when we mix these powerful technologies? Deduplication and encryption defeat each other! Deduplication must have access to repeating, deterministic data, and encryption must not allow this to happen. The most common solution (apart from skipping the encryption) is to place the deduplication technology first, allowing it access to the raw data before sending it on to be encrypted. But this leaves the data unprotected longer, and limits the possible locations where encryption technology can be applied. For example, an archive platform would have to encrypt data internally, since many now include deduplication as an integral component.

Why do we prefer compression to encryption? Simply because that’s where the money is! If we can cut down on storage space or WAN bandwidth, we see cost avoidance or even real cost savings! But if we “waste” space by encrypting data, we only save money in the case of a security breach.

A Glimmer of Hope

I had long thought this was an intractable problem, but a glimmer of hope recently presented itself. My hosting provider allows users to back up their files to a special repository using the rsync protocol. This is pretty handy, as you can imagine, but I was concerned about the security of this service. What happens if someone gains access to all of my data by hacking their servers?

At first, I only stored non-sensitive data on the backup site, but this limited its appeal. So I went looking for something that would allow me to encrypt my data before uploading it, and I discovered two interesting concepts: rsyncrypto and gzip-rsyncable.

rsync is a solid protocol, reducing network demands by only sending the changed blocks of a file. But, as noted, compression and encryption tools change the whole file even if only a tiny bit has been altered. A few years back, the folks behind rsync (who also happen to be the minds behind the Samba CIFS server) developed a patch for gzip which causes it to compress files in chunks rather than in their entirety. This patch, called gzip-rsyncable, hasn’t been added to the main source even after a dozen years, but yields amazing results in accelerating rsync performance.

The same technique was then applied to RSA and AES cryptography to create rsyncrypto. This open source encryption tool makes a simple tweak to the standard CBC encryption schema (reusing the initialization vector) to allow encrypted files to be sent more efficiently over rsync. In fact, it relies on gzip-rsyncable to work its magic. Of course, the resulting file is somewhat less secure, but it is probably more than enough to keep a casual snooper at bay.

Both of these tools are similar to modern deduplication techniques in that they chop files up into smaller, variable-sized blocks before working their magic. And the result is awesome: I modified a single word in a large word document that I had previously encrypted and stored at the backup site and was able to transfer just a single block of the new file in an instant rather than a few minutes. My only real issue is the lack of integration of all of these tools: I had to write a bash script to encrypt  my files to a temporary directory before rsyncing them. I wish they could be integrated with the main gzip and rsync sources!

If you are interested in trying out these tools for yourself, and if you use a Mac, you are in luck: Macports offers both tools as simple downloads! Just install macports, type “sudo port install gzip +rsyncable” to install gzip with the –rsyncable flag, then type “sudo port install rsyncrypto” and you’re done! I’ll post more details here if there is interest.

© sfoskett for Stephen Foskett, Pack Rat, 2009. | Compression, Encryption, Deduplication, and Replication: Strange Bedfellows
This post was categorized as Enterprise storage, Personal. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.

Now Hifn has purchased Siafu, which will only increase acceptance of this much-needed element of the storage puzzle.  John Matze and the rest have done a terrific job of pushing encryption in the small business segment, and this should get their stuff accepted more broadly.  Plus, it might just spread John’s non-security iSCSI expertise more broadly, and that’s always a good thing.

© sfoskett for Stephen Foskett, Pack Rat, 2007. | Hifn Buys Siafu
This post was categorized as Enterprise storage. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.