<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:series="http://unfoldingneurons.com/"
	>

<channel>
	<title>Stephen Foskett, Pack Rat &#187; Mac OS X Archives  &#8211; Stephen Foskett, Pack Rat</title>
	<atom:link href="http://blog.fosketts.net/tag/mac-os-x/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.fosketts.net</link>
	<description>Understanding the accumulation of data</description>
	<lastBuildDate>Fri, 10 Feb 2012 17:40:43 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=</generator>
<atom:link rel="hub" href="http://pubsubhubbub.appspot.com" />
	<atom:link rel="hub" href="http://superfeedr.com/hubbub" />
			<item>
		<title>The Case of the Missing Letters: Another Obnoxious Bug in Dragon Dictate</title>
		<link>http://blog.fosketts.net/2012/01/16/case-missing-letters-obnoxious-bug-dragon-dictate/</link>
		<comments>http://blog.fosketts.net/2012/01/16/case-missing-letters-obnoxious-bug-dragon-dictate/#comments</comments>
		<pubDate>Mon, 16 Jan 2012 20:48:26 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[10.7]]></category>
		<category><![CDATA[dictation]]></category>
		<category><![CDATA[Dragon]]></category>
		<category><![CDATA[Dragon Dictate]]></category>
		<category><![CDATA[Lion]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[spellcheck]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=6696</guid>
		<description><![CDATA[Any time OS X corrects your spelling or (more likely) inserts appropriate accent marks, Dictate loses its mind and can no longer correctly enter some random letter. There are two ways to fix this problem (apart from just restarting Dictate all the time).]]></description>
			<content:encoded><![CDATA[<p>What&#8217;s a love-hate relationship without the love? Whatever that is, that&#8217;s what I&#8217;ve got going with my copy of Dragon Dictate. I&#8217;ve long been a fan of dictation software, but the bugs and usability flubs in this product continually annoy me. Now it appears that the automatic spell checker in Mac OS X Lion causes Dragon to lose its mind and continually drop a random character after a correction.</p>
<h3>All Your Lttrs Ar Blong To Us!</h3>
<p>Here&#8217;s the problem: You&#8217;re dictating happily into Mail or some other supported application (even Word) and <strong>suddenly one letter starts disappearing</strong> from everything entered. Maybe it&#8217;s the E, or the T, but some random letter just goes away.</p>
<pre>nd ll of your text strts looking like this.</pre>
<p>Dictation still works fine, it&#8217;s just the text entry that&#8217;s broken. And of course, you can still dictate into Note Pad in Dictate without an error. <strong>The only solution at this point is to exit Dictate and restart</strong>.</p>
<p>Apparently, the new in-line spelling correction engine in Mac OS X Lion is <a href="http://www.macspeech.com/extensions/forums/topic.php?id=1637" >causing this new issue</a> with Dragon Dictate. <strong>Any time OS X corrects your spelling or (more likely) inserts appropriate accent marks, Dictate loses its mind and can no longer correctly enter some random letter.</strong></p>
<p>There are two ways to fix this problem (apart from just restarting Dictate all the time):</p>
<ol>
<li><strong>Disable OS X spell correction</strong> in the Language &amp; Text System Preferences panel under the Text tab. Then restart Dictate and it will no longer trip over accent characters and auto corrections. Of course, you will lose automatic spell correction as well.</li>
<li>Strangely, it appears that if you disable spell correction before launching Dragon Dictate and then <strong>enable it afterwards</strong> you get the best of both worlds: In-line spell correction and no more bug bites from Dictate. I haven&#8217;t tested this long, but it hasn&#8217;t failed me yet.</li>
</ol>
<div id="attachment_6697" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2012/01/Screen-Shot-2012-01-16-at-3.03.12-PM.png" ><img class="size-medium wp-image-6697" title="Screen Shot 2012-01-16 at 3.03.12 PM" src="http://static.fosketts.net/wp-content/uploads/2012/01/Screen-Shot-2012-01-16-at-3.03.12-PM-300x266.png" alt="" width="300" height="266" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Sriously? I hav to disabl spll corrction to kp Dragon Dictat from dropping lttrs all th tim?</p></div>
<p>Contrary to rumors and suggestions from Dragon tech support, TextExpander is not the culprit in this particular bug. I found no difference when enabling, disabling, or closing TextExpander. It was only when I disabled OS X spell correction that I was able to eliminate it.</p>
<h3> Why Keep Using Dragon Dictate?</h3>
<p>I do wish that Dragon would correct these of noxious usability gaffes, since the recognition engine is actually pretty strong. I just wonder if anyone at Dragon actually uses this application, or if they just put out there hoping it will work.</p>
<p>A fair question is why I keep using Dragon Dictate even though I despise using it. Truthfully, I get a huge amount of value from dictation software generally, and have been using it since my Windows days. Dragon is the only game in town for Mac dictation, so people like me who like dictation software have no other choice.</p>
<p>I intend to continue to use Dragon Dictate despite its flaws, in hopes that the company will improve the product. But I&#8217;m frankly not that hopeful this will happen, considering the long history of unresolved bugs and frustrating usability issues.</p>
<p>I only hope that Apple will integrate dictation into Mac OS X more thoroughly, since this would eliminate the majority of issues I have Using Dragon Dictate.</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/05/27/review-dragon-dictate-mac/"  rel="bookmark" class="crp_title">Dragon Dictate for Mac: Utterly Frustrating</a></li><li><a href="http://blog.fosketts.net/2011/02/23/dragon-dictate-2-mac-129-today/"  rel="bookmark" class="crp_title">Get Dragon Dictate 2 for Mac, Just $129 Today Only!</a></li><li><a href="http://blog.fosketts.net/2011/06/03/nuance-responds-dragon-dictate-concerns/"  rel="bookmark" class="crp_title">Nuance Responds to My Dragon Dictate Concerns</a></li><li><a href="http://blog.fosketts.net/2011/05/27/pile-interesting-links-27-2011/"  rel="bookmark" class="crp_title">Back From the Pile: Interesting Links, May 27, 2011</a></li><li><a href="http://blog.fosketts.net/2010/05/13/iphone-ipad-auto-correction-tip/"  rel="bookmark" class="crp_title">Quick Tip: iPhone/iPad Auto-Correction &#8220;im/IM/I&#8217;m&#8221;</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2012/01/16/case-missing-letters-obnoxious-bug-dragon-dictate/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2012. |
<a href="http://blog.fosketts.net/2012/01/16/case-missing-letters-obnoxious-bug-dragon-dictate/">The Case of the Missing Letters: Another Obnoxious Bug in Dragon Dictate</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2012/01/16/case-missing-letters-obnoxious-bug-dragon-dictate/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Undocumented CoreStorage Commands</title>
		<link>http://blog.fosketts.net/2011/08/05/undocumented-corestorage-commands/</link>
		<comments>http://blog.fosketts.net/2011/08/05/undocumented-corestorage-commands/#comments</comments>
		<pubDate>Fri, 05 Aug 2011 16:00:57 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Enterprise storage]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[Terabyte home]]></category>
		<category><![CDATA[10.7]]></category>
		<category><![CDATA[CoreStorage]]></category>
		<category><![CDATA[diskutil]]></category>
		<category><![CDATA[Lion]]></category>
		<category><![CDATA[logical volume]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[volume manager]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5994</guid>
		<description><![CDATA[I am pleased to note that CoreStorage, the volume manager in Mac OS X Lion, is much more functional than I had guessed, including a number of undocumented but seemingly functional commands for on-the-fly resizing of logical volumes as well as manipulation of physical volumes.
]]></description>
			<content:encoded><![CDATA[<div id="attachment_5977" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/08/What-CoreStorage-Could-Do.jpg" ><img class="size-medium wp-image-5977" title="What CoreStorage Could Do" src="http://static.fosketts.net/wp-content/uploads/2011/08/What-CoreStorage-Could-Do-300x225.jpg" alt="" width="300" height="225" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">The commands are there to make CoreStorage do some cool things. But they don&#39;t quite work and aren&#39;t quite public...</p></div>
<p>Yesterday I noted that <a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/" >Apple included a full logical volume manager</a> in Mac OS X 10.7 &#8220;Lion&#8221; without so much as a word. Today I am pleased to say that CoreStorage is much more functional than I had guessed, including a number of undocumented but seemingly functional commands for on-the-fly resizing of logical volumes as well as manipulation of physical volumes. Read on for the details, but please proceed at your own risk with these new commands!</p>
<blockquote><p>You should probably read <a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/" >Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a> first!</p></blockquote>
<h3>CoreStorage Command Overview</h3>
<p>The <strong>diskutil</strong> command is the core command line interface for CoreStorage, along with regular disk partitioning, AppleRAID, and other disk activities.</p>
<p>All CoreStorage functions use the &#8220;coreStorage&#8221; adverb, which can be abbreviated &#8220;cs&#8221;. For example, to show status of all CoreStorage volumes, one may type either:</p>
<pre>diskutil coreStorage list</pre>
<p>or</p>
<pre>diskutil cs list</pre>
<p>Typing just &#8220;diskutil cs&#8221; will show a list of nine supported &#8220;verbs&#8221;, but there are six other undocumented commands as well. Perhaps these are not fully functional (though all worked fine in my testing), or perhaps Apple simply wasn&#8217;t ready to expose them for end users to use.</p>
<p>I have grouped all 15 CoreStorage verbs by functional area, and note here which are officially listed and which are undocumented.</p>
<p><strong>Informational Commands</strong></p>
<ul>
<li>list &#8211; Show all CoreStorage volumes</li>
<li>info[rmation] &#8211; Get detailed information about a physical volume</li>
</ul>
<p><strong>Conversion Commands</strong></p>
<ul>
<li>convert &#8211; Convert a volume into a CoreStorage volume</li>
<li>revert &#8211; Revert a CoreStorage volume to its native type</li>
</ul>
<p><strong>Logical Volume Group Commands</strong></p>
<ul>
<li>create &#8211; Create a new CoreStorage logical volume group</li>
<li>delete &#8211; Delete a CoreStorage logical volume group</li>
</ul>
<p><strong>Physical Volume (Disk) Commands</strong></p>
<ul>
<li>resizeDisk (undocumented) &#8211; Resize a physical volume</li>
<li>removeDisk (undocumented) &#8211; Remove a physical volume from a logical volume group</li>
<li>addDisk (undocumented) - Add a new physical volume to a logical volume group</li>
</ul>
<p><strong>Logical Volume Commands</strong></p>
<ul>
<li>createVolume &#8211; Create a new logical volume</li>
<li>unlockVolume &#8211; Mount an existing encrypted logical volume</li>
<li>changeVolumePassphrase &#8211; Change the encryption password for a logical volume</li>
<li>deleteVolume (undocumented) &#8211; Delete a logical volume and all of its contents</li>
<li>resizeVolume (undocumented) &#8211; Grow or shrink a logical volume (non-destructive)</li>
<li>resizeStack (undocumented) &#8211; Grow or shrink a logical volume as well as the logical volume group and physical volume that supports it</li>
</ul>
<h3>CoreStorage Informational Commands</h3>
<p>Two commands are for information about volumes and disks. Note that these only return information about CoreStorage volumes: Use the bare &#8220;diskutil list&#8221; and &#8220;diskutil info&#8221; commands for non-encapsulated storage.</p>
<h4>list</h4>
<pre>Usage:  diskutil coreStorage list
        diskutil coreStorage list -plist
        diskutil coreStorage list UUID
List all current CoreStorage objects in a tree-like view.</pre>
<h4>info[rmation]</h4>
<p>The verbs &#8220;info&#8221; or &#8220;information&#8221; get CoreStorage information by UUID.</p>
<h3>CoreStorage Conversion Commands</h3>
<p>These commands are used to convert a drive or volume to or from CoreStorage.</p>
<h4>convert</h4>
<pre>Usage:  diskutil coreStorage convert
        MountPoint|DiskIdentifier|DeviceNode
        [-stdinpassphrase | -passphrase [passphrase]]
Convert a regular JHFS+ partition into a CoreStorage logical volume.
The file system must be mounted and resizable (i.e. Journaled HFS+).
Ownership of the affected disk is required.</pre>
<p>Note that CoreStorage only functions on drives that meet the following criteria:</p>
<ol>
<li>Partition type is GPT</li>
<li>Filesystem type is Journaled HFS+ (JHFS+)</li>
</ol>
<p>Conversion from the command line is non-destructive (unlike using Disk Utility) and is the one shot you have to encrypt a volume. If you convert it without the passphrase, it will be placed in a LVF with no encryption and there is no command to encrypt it later!</p>
<h4>revert</h4>
<pre>Usage:  diskutil coreStorage revert
        MountPoint|DiskIdentifier|DeviceNode|lvUUID
        [-stdinpassphrase | -passphrase [passphrase] | -recoverykeychain file]
Convert a CoreStorage logical volume back to its native type.
The file system must be mounted and resizable (i.e. Journaled HFS+).
Ownership of the affected disk and a passphrase (if encrypted) is required.</pre>
<p>Reverting a CoreStorage volume completely returns it to its original state, including restoring the partition type and removing encryption. And it&#8217;s non-destructive. Slick!</p>
<h3>CoreStorage Logical Volume Group Commands</h3>
<p>These commands manipulate logical volume groups (LVGs) that have previously been created. Most use the &#8220;LVG Name&#8221;, which you can discover using the &#8220;diskutil cs list&#8221; command.</p>
<h4>create</h4>
<pre>Usage:  diskutil coreStorage create lvgName
        MountPoint|DiskIdentifier|DeviceNode ...
Create a CoreStorage logical volume group from one more more disks.
The specified disks will become the initial set of PVs.
All existing data on the drive will be lost.
Ownership of the affected disk is required.
Example: diskutil coreStorage create MyLVG disk1</pre>
<p>Note that this command is destructive. Say goodbye to your data or use the &#8220;cs convert&#8221; command! It automatically creates a PV for you.</p>
<h4>delete</h4>
<pre>Usage:  diskutil coreStorage delete lvgUUID
Delete a CoreStorage logical volume group. All logical volumes will be removed.
Ownership of the affected disk is required.</pre>
<p>This is destructive as well. Your volume will be returned to a usable state, but your data will be lost. Use &#8220;convert&#8221; and &#8220;revert&#8221; instead if possible!</p>
<h3>CoreStorage Physical Volume (Disk) Commands</h3>
<p>All physical volume commands are undocumented in Lion 10.7. Proceed with caution! But these are pretty cool, since they allow a LVG to span multiple physical disks!<br />
PV commands use the word, &#8220;Disk&#8221;, which tells you a lot about what they are and what they do.<br />
Note that the &#8220;pvUUID&#8221; and &#8220;lvgUUID&#8221; can be discovered in &#8220;diskutil cs list&#8221; and refer to the PV and LVG, respectively.</p>
<h4>resizeDisk (undocumented)</h4>
<pre>Usage:  diskutil coreStorage resizeDisk pvUUID size
        [part1Format part1Name part1Size part2Format part2Name part2Size
         part3Format part3Name part3Size ...]
Resize a physical volume, which is one of one or more disks that provide storage
to a logical volume group. The logical volume group will have less or more
available space after this operation, if it was a shrink or grow, respectively.
If this is a shrink operation, you can optionally request that new partitions
be created in the newly-formed free space gap.
Example: diskutil coreStorage resizeDisk
         11111111-2222-3333-4444-555555555555 10g JHFS+ New 1g</pre>
<p>This crazy-complicated command allows you to resize (both grow and shrink) active partitions non-destructively.</p>
<h4>removeDisk (undocumented)</h4>
<pre>Usage:  diskutil coreStorage removeDisk pvUUID
Remove a physical volume from its CoreStorage logical volume group.
Ownership of the affected disks is required.
Example: diskutil coreStorage removeDisk 11111111-2222-3333-4444-555555555555</pre>
<p>It is unknown what will happen if a used PV is removed from an LVG. Using a combination of addDisk and removeDisk should non-destructively move data from one physical disk to another, but I couldn&#8217;t get any of this to work.</p>
<h4>addDisk (undocumented)</h4>
<pre>Usage:  diskutil coreStorage addDisk lvgUUID NewMemberDeviceName
Add a new physical volume to a CoreStorage logical volume group.
Ownership of the affected disks is required.
Example: diskutil coreStorage addDisk
         11111111-2222-3333-4444-555555555555 disk4</pre>
<p>This should add additional physical capacity to an existing LVG, but I couldn&#8217;t get it to work. It always came back with &#8220;Error adding disk to CoreStorage Logical Volume Group: Invalid request (-69886)&#8221;</p>
<h4>CoreStorage Logical Volume Commands</h4>
<p>The logical volume commands are much more functional and friendly. Right now, the only way to really use these is to create a larger-than-needed LV and then shrink it and use the resulting space for new volumes, since you cannot yet fully create a LVG from scratch.</p>
<h4>createVolume</h4>
<p>This creates a new LV (and LVF) within an existing LVG, using space cleared by a previous resizeVolume or deleteVolume command. It appears to always create a new LVF rather than placing an LV within an existing one. And there are no LVF manipulation commands right now.</p>
<pre>Usage:  diskutil coreStorage createVolume lvgUUID type name size
        [-stdinpassphrase | -passphrase [passphrase]]
Add a new logical volume to a CoreStorage logical volume group.

Type is the file system to initialize on the new logical volume. Valid types
are Journaled HFS+ or Case-sensitive Journaled HFS+ or their aliases.

Size is the amount of space to allocate from the parent logical volume group.
Valid sizes are floating-point numbers with a suffix of B(ytes), S(512-byte-
blocks), K(ilobytes), M(egabytes), G(igabytes), T(erabytes), P(etabytes),
or (%) a percentage of the current size of the logical volume group.

Example: diskutil coreStorage createVolume
         11111111-2222-3333-4444-555555555555 jhfs+ myLV 10g</pre>
<h4>unlockVolume</h4>
<p>One of the seemingly-handier commands that isn&#8217;t really all that useful. It&#8217;s rare that you&#8217;ll want to do this alone. Use hdiutil to mount a volume instead.</p>
<pre>Usage:  diskutil coreStorage unlockVolume lvUUID
        [-stdinpassphrase | -passphrase passphrase | -recoverykeychain file]
Unlock a logical volume that is encrypted and currently locked. You must
specify the logical volume by its CoreStorage UUID, because if it is locked
it is not online. A passphrase is mandatory: you must either supply it
interactively or with one of the above parameters.
Example: diskutil coreStorage unlockVolume 11111111-2222-3333-4444-555555555555</pre>
<h4>changeVolumePassphrase</h4>
<p>A very handy command allowing you to change the passphrase of an existing VileVault 2 encrypted volume. This is probably the one verb that will be somewhat frequently used by the average user!</p>
<pre>Usage:  diskutil coreStorage changeVolumePassphrase|passwd lvUUID
        [-recoverykeychain file] | [-oldpassphrase old]
        [-newpassphrase new] | [-stdinpassphrase]
Change an encrypted logical volume's password. Beyond the CoreStorage UUID,
you will be prompted interactively for parameters that you do not specify.
Parameters must be given in the above order.
Example: diskutil coreStorage changeVolumePassphrase
         11111111-2222-3333-4444-555555555555</pre>
<h4>resizeVolume (undocumented)</h4>
<p>This appears to work fine. You can resize a volume (given its UUID) using this command, and you won&#8217;t even lose your data!</p>
<pre>Usage:  diskutil coreStorage resizeVolume lvUUID size
Resize a logical volume, which is one of one or more disks that consume storage
out of a logical volume group. The logical volume group will have more or less
available space after this operation, if it was a shrink or grow, respectively.
Example: diskutil coreStorage resizeVolume
         11111111-2222-3333-4444-555555555555 10g</pre>
<h4>deleteVolume (undocumented)</h4>
<p>This also works fine. Create a volume and you can delete it, as long as you get the right UUID from &#8220;diskutil cs list&#8221;.</p>
<pre>Usage:  diskutil coreStorage deleteVolume lvUUID
Delete a logical volume.
Example: diskutil coreStorage deleteVolume 11111111-2222-3333-4444-555555555555</pre>
<h4>resizeStack (undocumented)</h4>
<p>Probably the coolest undocumented command, resizeStack takes a complete 1:1:1:1 CoreStorage stack (PV:LVG:LVF:LV) and resizes everything non-destructively. You&#8217;re left with a fully-operational but smaller volume and partition. Nifty!</p>
<pre>Usage:  diskutil coreStorage resizeStack lvUUID size
        [part1Format part1Name part1Size part2Format part2Name part2Size
         part3Format part3Name part3Size ...]
Resize both a logical volume and its underlying physical volume in a single
operation. The setup must be simple: Exactly one logical volume and one
related physical volume can, and must, exist.
If this is a shrink operation, you can optionally request that new partitions
be created in the newly-formed free space gap.
Example: diskutil coreStorage resizeStack
         11111111-2222-3333-4444-555555555555 10g JHFS+ New 1g</pre>
<h3>What&#8217;s Missing</h3>
<p>These undocumented CoreStorage commands are eye-opening, since it shows that Apple really is working on a complete full-featured volume manager. But lots of functionality is lacking still:</p>
<ol>
<li>You can&#8217;t create a logical volume group (LVG) by hand, adding multiple disks. It seems like you could, but not enough works yet.</li>
<li>There are no commands for manipulating logical volume families (LVFs), and they can&#8217;t even be specified when creating new logical volumes (LVs).</li>
<li>The physical volume (PV or disk) commands don&#8217;t seem to work. I guess that&#8217;s why they&#8217;re undocumented!</li>
<li>There are no advanced data protection features (mirror, snapshot, RAID, replicate)</li>
<li>There is no ability to specify where or how an LV is created or to move an LV from one LVF, PV, or LVG to another.</li>
</ol>
<div>All in all, this is a wonderful start for Apple, showing solid core technology that isn&#8217;t yet fleshed out enough to be useful.</div>
<h3>Stephen&#8217;s Stance</h3>
<p>Clearly, CoreStorage is much more than a simple stack to support FileVault 2 encryption. Apple no only built an entire volume manager but also outfitted it with a good set of commands for configuration. Too bad it still lacks so many key features, and that so many of these are unofficial and hidden.</p>
<p>I feel confident that Mac OS X CoreStorage will become much more functional in the future, with complete volume manipulation and migration tools built into the GUI. For now, though, I must remind readers that this is all unofficial and you must proceed with caution. Back up your data, and do not assume that just because cool commands like resizeStack exist they are 100% ready for use!</p>
<blockquote><p>Note: The pre-formatted text above comes straight from the diskutil command and its &#8220;help&#8221; responses. I discovered these undocumented verbs through trial and error and the judicial use of the &#8220;strings&#8221; command!</p></blockquote>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/"  rel="bookmark" class="crp_title">Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></li><li><a href="http://blog.fosketts.net/guides/walkthrough-logical-volume-manager-linux/"  rel="bookmark" class="crp_title">Linux Logical Volume Manager Walkthrough</a></li><li><a href="http://blog.fosketts.net/2011/04/26/5308/"  rel="bookmark" class="crp_title"></a></li><li><a href="http://blog.fosketts.net/2008/02/21/volume-management-virtualizing-host-storage/"  rel="bookmark" class="crp_title">Volume Management: Virtualizing Host Storage</a></li><li><a href="http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/"  rel="bookmark" class="crp_title">Three Key Storage Features Missing in Mac OS X &#8220;Lion&#8221;</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/08/05/undocumented-corestorage-commands/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/08/05/undocumented-corestorage-commands/">Undocumented CoreStorage Commands</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/enterprisestorage/" title="View all posts in Enterprise storage" rel="category tag">Enterprise storage</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>, <a href="http://blog.fosketts.net/category/everything/terabytehome/" title="View all posts in Terabyte home" rel="category tag">Terabyte home</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/08/05/undocumented-corestorage-commands/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<series:name><![CDATA[CoreStorage]]></series:name>
	</item>
		<item>
		<title>Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</title>
		<link>http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/</link>
		<comments>http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/#comments</comments>
		<pubDate>Thu, 04 Aug 2011 19:22:32 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Computer History]]></category>
		<category><![CDATA[Enterprise storage]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Features]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[Terabyte home]]></category>
		<category><![CDATA[10.7]]></category>
		<category><![CDATA[CoreStorage]]></category>
		<category><![CDATA[Lion]]></category>
		<category><![CDATA[LVM]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[Veritas Volume Manager]]></category>
		<category><![CDATA[volume manager]]></category>
		<category><![CDATA[ZFS]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5973</guid>
		<description><![CDATA[Mac OS X was majorly deficient in that it lacked a volume manager. This wouldn't seem like a big deal to the average user, but held back the operating system in so many ways. A volume manager brings storage virtualization to an operating system, allowing storage capacity efficiently to be managed and manipulated. But all this has changed in Mac OS X 10.7 “Lion” with CoreStorage.]]></description>
			<content:encoded><![CDATA[<p>Apple fan boys won&#8217;t like to hear this, but Mac OS X was majorly deficient in an area near and dear to my heart. You see, unlike every other modern operating system, Mac OS X lacked a volume manager. This wouldn&#8217;t seem like a big deal to the average user, but it held back the operating system in many ways. A <a href="http://blog.fosketts.net/2008/02/21/volume-management-virtualizing-host-storage/" >volume manager</a> brings storage virtualization to an operating system, allowing storage capacity efficiently to be managed and manipulated. But all this has changed in Mac OS X 10.7 “Lion” with CoreStorage.</p>
<blockquote><p>You probably also want to read <a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/" >Key Storage Features in Mac OS X 10.7 “Lion”</a> and <a href="http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/" >Three Key Storage Features Missing in Mac OS X “Lion”</a></p></blockquote>
<h3>Apple&#8217;s Storage Story: A Mixed Bag</h3>
<p>Apple&#8217;s file system, HFS+, is quite long in the tooth, dating back even before Mac OS X! Introduced in 1998&#8242;s Mac OS 8.1, HFS+ was extension of the original hierarchical file system (HFS) used by classic Mac operating systems back to System 2.1. But HFS+ has been extended and updated dramatically since those days, including the introduction of journaling in Mac OS 10.2.2, Unicode in 10.3, access control list-based file security in 10.4, and compression in 10.6.</p>
<p>The storage technology that underlies HFS+ has also improved dramatically over the years. Apple adopted EFI firmware and the GUID partition table (GPT) much more rapidly than the PC world. This allows Mac OS to scale to larger storage capacities than Windows systems, and has made the adoption of 4K sectors (so-called “advanced format”) much easier.  In fact, Apple is well ahead of the Windows world in these areas even as it lags with an outdated filesystem and no volume manager.</p>
<p>Mac OS never had much in terms of storage flexibility. Sure, AppleRAID brought some flexibility and redundancy to the server world, but the average Mac user never delved much into this technology. While Apple was asleep at the wheel, every other operating system vendor introduced a flexible volume manager. Even Microsoft, who is often criticized for slow innovation, is way ahead of Apple when it comes to data storage.</p>
<blockquote><p><strong>What is a volume manager? </strong>Volume managers add flexibility to operating system allocation of storage, acting as a virtualization layer between disks and file systems. This is important, since modern computer systems need to maintain a consistent filesystem image even as storage devices change. Volume managers like Symantec&#8217;s Veritas Volume Manager and the OSF LVM allow many operating systems this sort of flexible storage management. But Apple never had a volume manager of their own until the introduction of CoreStorage in Mac OS X “Lion”. Read more at <a href="http://blog.fosketts.net/2008/02/21/volume-management-virtualizing-host-storage/" >Volume Management: Virtualizing Host Storage</a>.</p></blockquote>
<h3>Introducing CoreStorage</h3>
<p>All this has changed in Mac OS X 10.7 (“Lion”), however. Although Mac OS never did get ZFS, as was planned in 10.5, the storage enhancements found in 10.7 are getting awfully close in terms of features. Along with integrated file versions and full disk encryption comes a new technology called CoreStorage, including a full logical volume manager.</p>
<p>CoreStorage became necessary due to the inclusion of full volume encryption in Lion. Previously, Apple&#8217;s FileVault encryption technology relied on regular files to contain an encrypted filesystem. This worked fine until the need became apparent to encrypt an entire volume, whether it is the operating system disk or a removable drive. Although file-based technology can work in these scenarios, it is much less than ideal and also less than what competing operating systems offer.</p>
<p>But enabling full volume encryption requires an abstraction layer that Mac OS X simply did not have. Apple appears to have taken the long route, engineering a complete volume manager rather than hacking together some sort of kludgy mechanism. Although it is only used for FileVault 2 at the moment, CoreStorage appears to be a full volume manager that could, in the future, boast advanced features like mirroring, snapshots, and online storage migration.</p>
<h3>CoreStorage Data Structures</h3>
<div id="attachment_5978" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/08/What-CoreStorage-Does-Today.jpg" ><img class="size-medium wp-image-5978" title="What CoreStorage Does Today" src="http://static.fosketts.net/wp-content/uploads/2011/08/What-CoreStorage-Does-Today-300x225.jpg" alt="" width="300" height="225" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">CoreStorage implements a full complement of data structures but uses them in a very simplistic manner currently.</p></div>
<p>Apple did not simply copy existing volume manager semantics, and it does not appear that CoreStorage is an OEM version of someone else&#8217;s product. Rather, there are some intriguing new innovations present in CoreStorage that point to exciting possibilities in the future.</p>
<p>Like most volume managers, the basic building block of CoreStorage is the “logical volume group” (LVG). Illustrated in blue and purple above, each LVG takes storage from a “physical volume” (PV) and presents it to the operating system as a “logical volume” (LV). In this way, storage from a physical disk is abstracted before used by the operating system, allowing a great deal of flexibility. This is somewhat analogous to a hypervisor presenting a virtual machine interface.</p>
<p>Apple goes a step further than most volume managers, however, with the introduction of a new concept, the “logical volume family” (LVF). The LVF specifies properties that will be inherited by logical volumes that it contains. Currently, the only property specified by an LVF is FileVault encryption, but one can imagine that performance characteristics or redundancy could also be specified in this manner.</p>
<h3>CoreStorage Today and Tomorrow</h3>
<p>Today, when full volume FileVault encryption is used, Mac OS X automatically converts the desired volume into a CoreStorage volume, and encapsulating the partition as a PV, importing it into an LVG, and setting up an LVF and LV to contain the new filesystem. Only one of each element is created in this way, and all capacity is allocated at creation time. So there isn&#8217;t much use for the entire stack right now.</p>
<div id="attachment_5979" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/08/What-You-Can-Make-CoreStorage-Do.jpg" ><img class="size-medium wp-image-5979" title="What You Can Make CoreStorage Do" src="http://static.fosketts.net/wp-content/uploads/2011/08/What-You-Can-Make-CoreStorage-Do-300x225.jpg" alt="" width="300" height="225" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Using diskutil from the command line, one can create multiple logical volume families and logical volumes.</p></div>
<p>The Disk Utility GUI is fairly limited as well. Volumes can be converted to use FileVault 2, but all data is lost in this process. Only boot volumes can be corrected nondestructively through the GUI, and this happens in the Security &amp; Privacy preference pane rather than Disk Utility. When boot volumes are encrypted, Apple gives users the option to “escrow” the decryption key online using their MobileMe or iCloud account, but this is not allowed for other volumes.</p>
<p>Currently, whenever a volume is converted to using FileVault, it is also automatically converted to CoreStorage behind the scenes. And if encryption is turned off in the GUI, it is converted back (&#8220;reverted&#8221;) to whatever it was before. But it&#8217;s possible to convert a volume on the command line without encryption, and to remove encryption without reverting. It&#8217;s also possible to manually manipulate CoreStorage in other ways.</p>
<div id="attachment_5977" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/08/What-CoreStorage-Could-Do.jpg" ><img class="size-medium wp-image-5977" title="What CoreStorage Could Do" src="http://static.fosketts.net/wp-content/uploads/2011/08/What-CoreStorage-Could-Do-300x225.jpg" alt="" width="300" height="225" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">In the future, it should be possible to manipulate storage much more fully.</p></div>
<p>The fact that Apple included a full volume manager of their own making indicates a bright future for CoreStorage. Already, it is possible to use the command line utility, diskutil to nondestructively convert and encrypt volumes. It is also possible to manipulate volume groups, logical volume families (indirectly), and logical volumes to a limited extent using diskutil.  One hopes that future versions of Mac OS X will include additional functionality, including the ability to manipulate physical volumes.</p>
<h3>Stephen&#8217;s Stance</h3>
<p>Either Apple has big plans for CoreStorage, or some Apple software engineer got way ahead of himself and designed a complete logical volume manager just to house FileVault data! I think the former is likely, since we know that Apple wanted to include ZFS, a flexible combination volume manager and file system, in Mac OS X 10.5 but was unable to due to licensing concerns. Now two versions later, Lion includes all of the basic technology needed to effectively manage storage volumes. It is likely that the GUI simply lags behind this core technology, and we will see additional functionality added in later operating system revisions.</p>
<p>Regardless of intent, Mac OS X now includes a fairly full storage stack, from EFI and GPT through CoreStorage and HFS+. No longer is Apple behind the times in terms of core storage (if you pardon the pun) features. Now if only we could get them to implement iSCSI!</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/08/05/undocumented-corestorage-commands/"  rel="bookmark" class="crp_title">Undocumented CoreStorage Commands</a></li><li><a href="http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/"  rel="bookmark" class="crp_title">Three Key Storage Features Missing in Mac OS X &#8220;Lion&#8221;</a></li><li><a href="http://blog.fosketts.net/2008/02/21/volume-management-virtualizing-host-storage/"  rel="bookmark" class="crp_title">Volume Management: Virtualizing Host Storage</a></li><li><a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/"  rel="bookmark" class="crp_title">Key Storage Features in Mac OS X 10.7 &#8220;Lion&#8221;</a></li><li><a href="http://blog.fosketts.net/guides/walkthrough-logical-volume-manager-linux/"  rel="bookmark" class="crp_title">Linux Logical Volume Manager Walkthrough</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/">Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/computerhistory/" title="View all posts in Computer History" rel="category tag">Computer History</a>, <a href="http://blog.fosketts.net/category/everything/enterprisestorage/" title="View all posts in Enterprise storage" rel="category tag">Enterprise storage</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/features/" title="View all posts in Features" rel="category tag">Features</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>, <a href="http://blog.fosketts.net/category/everything/terabytehome/" title="View all posts in Terabyte home" rel="category tag">Terabyte home</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
	
		<series:name><![CDATA[CoreStorage]]></series:name>
	</item>
		<item>
		<title>How Does Dropbox Store Data?</title>
		<link>http://blog.fosketts.net/2011/07/11/dropbox-data-format-deduplication/</link>
		<comments>http://blog.fosketts.net/2011/07/11/dropbox-data-format-deduplication/#comments</comments>
		<pubDate>Mon, 11 Jul 2011 16:30:09 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Enterprise storage]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[Terabyte home]]></category>
		<category><![CDATA[cloud storage]]></category>
		<category><![CDATA[data deduplication]]></category>
		<category><![CDATA[deduplication]]></category>
		<category><![CDATA[Dropbox]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[MD5]]></category>
		<category><![CDATA[SHA-1]]></category>
		<category><![CDATA[TrueCrypt]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5863</guid>
		<description><![CDATA[Dropbox recently clarified (via their blog and privacy policy) that they "de-duplicate" user files. This has been known for quite a while, and is obvious to anyone who's had a large file "upload" instantly. But how exactly does Dropbox store files? Are they really de-duplicated or just single-instanced? I set out to discover the answer.]]></description>
			<content:encoded><![CDATA[<p>Dropbox recently clarified (via their <a href="http://blog.dropbox.com/?p=846" >blog</a> and <a href="https://www.dropbox.com/terms#privacy" >privacy policy</a>) that they &#8220;de-duplicate&#8221; user files. This has been known for quite a while, and is obvious to anyone who&#8217;s had a large file &#8220;upload&#8221; instantly. But how exactly does Dropbox store files? Are they really de-duplicated or just single-instanced? I set out to discover the answer.</p>
<h3>Single Instance Storage</h3>
<p>It&#8217;s fairly simple for a system to eliminate duplicate data by storing only a single instance of multiple identical files. In other words, if you and I both upload &#8220;Presentation.pptx&#8221; and it&#8217;s bit-for-bit identical, it would be a simple matter to store just one copy.</p>
<p>Dropbox definitely does this. I proved it with a simple experiment:</p>
<ol>
<li>Create a new 10 MB encrypted disk image in TrueCrypt (so it&#8217;ll be 100% unique, random data)</li>
<li>Move it to the Dropbox folder and wait a few minutes as it uploads</li>
<li>Copy the file with a new name to the folder and notice that it &#8220;uploads&#8221; instantly</li>
</ol>
<p>Dropbox is at least single-instancing storage. This helps users, since it speeds uploads and reduces bandwidth usage. It helps Dropbox in the same way, but goes further since they still &#8220;charge&#8221; files against your account whether they&#8217;re single-instanced or not.</p>
<p>Note that this single-instancing works across users and geographies. I gave a file to a friend to upload to a different Dropbox account, and saw the same &#8220;acceleration effect.&#8221; This would be quite useful to users and the company for files like iTunes songs which are identical and widespread.</p>
<h3>Clashing MD5 Hashes?</h3>
<div id="attachment_5866" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/07/HashClash.png" ><img class="size-medium wp-image-5866" title="HashClash" src="http://static.fosketts.net/wp-content/uploads/2011/07/HashClash-300x64.png" alt="" width="300" height="64" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Three files with identical sizes and MD5 hashes but different names? Creepy!</p></div>
<p>A global single-instance storage system sounds great, but it opens the door to hash collision issues. Imagine if you and I both uploaded identical files. Both would have the same &#8220;fingerprint&#8221; and Dropbox would only store it once. Now imagine instead that, out of coincidence or malice, I uploaded a file with the same fingerprint as yours but different contents. This is <a href="http://www.schneier.com/blog/archives/2005/02/cryptanalysis_o.html" >not so far-fetched as it seems</a>, and could lead to all sorts of security nightmares.</p>
<p>A common and compromised file checksum method is MD5, so I decided to test how Dropbox handles files of identical size, name, and MD5 hash using the &#8220;<a href="http://www.win.tue.nl/hashclash/Nostradamus/" >Nostradamus Attack</a>&#8221; PDFs generated by Marc Stevens. My tests show that Dropbox correctly handled the files I tried, and no combination of uploading and naming could force it to incorrectly store the right file. So Dropbox either doesn&#8217;t use MD5 or uses a combination of hashing and other mechanisms. Testing other schemes is left as an exercise to the reader.</p>
<p>One more thought: The fact that de-duplication is mentioned in the &#8220;privacy&#8221; section of the Dropbox policies raises my eyebrows, since it indicates that they see this hash collisions as a matter of privacy rather than data corruption. This indicates that Dropbox is both aware of and susceptible to hash collision attacks generally, though obviously not as simply as creating a bogus MD5 match.</p>
<blockquote><p>Note: Dropbox is well aware of this issue, having <a href="http://razorfast.com/2011/04/25/dropbox-attempts-to-kill-open-source-project/" >recently squashed</a> an open-source exploit called <a href="http://forwardfeed.pl/index.php/2011/04/24/dropship-successor-to-torrents-eng/" >Dropship</a>!</p></blockquote>
<h3>Sub-File De-Duplication</h3>
<p>Data de-duplication is like single-instancing, but it applies to some subset of data. Some enterprise storage systems de-duplicate at multi-megabyte levels, while others are far more granular.</p>
<p>To test whether Dropbox de-duplicates data, I devised a simple experiment:</p>
<ol>
<li>Create a new local copy of my existing random TrueCrypt file</li>
<li>Add a single byte to the end using the &#8220;cat&#8221; command</li>
<li>Copy the resulting file to Dropbox</li>
<li>Watch as Dropbox takes just a few seconds to upload the new file</li>
</ol>
<p>This test proves that Dropbox does indeed de-duplicate at the sub-file level. Since it took a bit longer to upload that would be expected for a single byte, we can see that Dropbox &#8220;chunks&#8221; files for hashing and uploading.</p>
<h3>De-Duplication Granularity</h3>
<p>The next question is just what size chunks or blocks Dropbox uses to de-duplicate data. To test this, I created various blocks of random data using TrueCrypt and experimented to see where the &#8220;stair-steps&#8221; were in terms of upload time.</p>
<p>My tests used four basic building blocks of 512 KB, 1024 KB, 2048 KB, and 4096 KB in size. Guessing that Dropbox used one of these sizes for their chunking system, I assumed these would quickly demonstrate the answer.</p>
<div id="attachment_5870" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/07/Comparison-of-Dropbox-Transfer-Time-for-Various-Concatenated-Object-Sizes.jpg" ><img class="size-medium wp-image-5870" title="Comparison of Dropbox Transfer Time for Various Concatenated Object Sizes" src="http://static.fosketts.net/wp-content/uploads/2011/07/Comparison-of-Dropbox-Transfer-Time-for-Various-Concatenated-Object-Sizes-300x202.jpg" alt="" width="300" height="202" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">On my Mac, Dropbox clearly uses a 4 MB &quot;chunk&quot; size for deduplication</p></div>
<p>First, I uploaded each file individually and watched as Dropbox took about 30 seconds per MB. This will vary greatly, of course, but the absolute performance doesn&#8217;t matter. Only relative performance matters for demonstrating chunking.</p>
<p>Next, I concatenated each file with itself to create a new file twice as large. This would be ideally &#8220;chunkable&#8221; since it consists of exactly identical data with a nice, clean, evenly-divisible &#8220;border&#8221;. I uploaded each of these and noticed that the &#8220;4096 KB x 2&#8243; file uploaded nearly instantly, while all others took the expected amount of time.</p>
<p>I repeated this with &#8220;x 3&#8243;, &#8220;x 4&#8243;, and &#8220;x 8&#8243; files and noticed that the 4096 KB (4 MB) &#8220;barrier&#8221; was very obvious. Whenever a file contained 4096 KB or less of data Dropbox had seen before, it single-instanced it. Any time it saw a unique &#8220;block&#8221; smaller than this, it uploaded it fresh.</p>
<p>This proves, at least in the case of my own Mac OS X install of Dropbox, that a 4 MB chunk size is used for de-duplication.</p>
<h3> Stephen&#8217;s Stance</h3>
<p>Dropbox is a very useful service, and I appreciate the technology they use to make it work. By single-instancing storage, the company is able to keep costs and transfer time in check and offer a basic service for free for many users. Despite the recent security issue, I continue to use Dropbox myself and would not hesitate to recommend it. But I do suggest using your own encryption for any sensitive data, as demonstrated in my recent post, <a href="http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/" >Mac Users, Secure Your Stuff in Dropbox</a>.</p>
<p>I remain somewhat concerned about the privacy and security implications of global de-duplication of shared random data. If they use SHA-1 hashes alone, which I suspect, there is a chance that an object will not be stored correctly once 2^80 (or perhaps <a href="http://www.schneier.com/blog/archives/2005/02/sha1_broken.html" >2^69</a> or even <a rel="nofollow" href="http://lukenotricks.blogspot.com/2009/05/cost-of-sha-1-collisions-reduced-to-252.html" >2^52</a>) objects are stored. This would lead to issues of data corruption or inadvertent disclosure. This is a very remote chance indeed, but &#8220;<a rel="nofollow" href="http://en.wikipedia.org/wiki/Birthday_problem" >birthday problems</a>&#8221; like this work against hashing systems. I would love to hear from Dropbox regarding how they prevent this from happening, including disclosure of their methods of hashing data. It&#8217;s nice to see the company taking responsibility by disclosing this in their privacy policy, though!</p>
<blockquote><p>Update: Dropbox apparently does indeed use raw SHA256 hashes to &#8220;uniquely&#8221; identify data, and <a href="http://news.ycombinator.com/item?id=2478567" >this can be exploited in a number of ways</a>.</p></blockquote>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/"  rel="bookmark" class="crp_title">Mac Users, Secure Your Stuff in Dropbox</a></li><li><a href="http://blog.fosketts.net/2011/03/03/multiple-macs-sync-dropbox/"  rel="bookmark" class="crp_title">Keep Multiple Macs in Sync with Dropbox</a></li><li><a href="http://blog.fosketts.net/2011/03/01/google-dropbox-revolutionized-laptop-migration/"  rel="bookmark" class="crp_title">How Google and Dropbox Revolutionized My Laptop Migration</a></li><li><a href="http://blog.fosketts.net/2011/03/05/pile-interesting-links-march-4-2011/"  rel="bookmark" class="crp_title">Back From the Pile: Interesting Links, March 4, 2011</a></li><li><a href="http://blog.fosketts.net/2011/11/17/itunes-match-vbr-mp3-files-heres-fix/"  rel="bookmark" class="crp_title">iTunes Match Does Not Like VBR MP3 Files: Here&#8217;s How to Fix It</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/07/11/dropbox-data-format-deduplication/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/07/11/dropbox-data-format-deduplication/">How Does Dropbox Store Data?</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/enterprisestorage/" title="View all posts in Enterprise storage" rel="category tag">Enterprise storage</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>, <a href="http://blog.fosketts.net/category/everything/terabytehome/" title="View all posts in Terabyte home" rel="category tag">Terabyte home</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/07/11/dropbox-data-format-deduplication/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
		<item>
		<title>Mac Users, Secure Your Stuff in Dropbox</title>
		<link>http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/</link>
		<comments>http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/#comments</comments>
		<pubDate>Tue, 05 Jul 2011 15:30:57 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[Terabyte home]]></category>
		<category><![CDATA[Disk Utility]]></category>
		<category><![CDATA[Dropbox]]></category>
		<category><![CDATA[encryption]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[SparseBundle]]></category>
		<category><![CDATA[Storagemistress]]></category>
		<category><![CDATA[TrueCrypt]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5849</guid>
		<description><![CDATA[Encryption is an important tool for individuals regardless of what they're storing. Given the recent security failings of Dropbox, I highly recommend using methods like this to secure your important data before using the service!]]></description>
			<content:encoded><![CDATA[<div id="attachment_5852" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-New-Image.png" ><img class="size-medium wp-image-5852" title="Disk Utility New Image" src="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-New-Image-300x154.png" alt="" width="300" height="154" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Disk Utility in Mac OS X can create an encrypted &quot;SparseBundle&quot; compatible with Dropbox</p></div>
<p>Security is always possible but has rarely been easy. Storing unencrypted files and folders is much simpler and more compatible than going through the effort to use encryption. But <a href="http://blog.dropbox.com/?p=821" >the recent security flub at Dropbox</a> set me looking for a secure way to store data there. Here&#8217;s a quick and easy way to create a secure, Dropbox-compatible disk image for Mac OS X.</p>
<h3>Dropbox-Friendly</h3>
<blockquote><p>You might also want to read <a href="http://blog.fosketts.net/2011/03/03/multiple-macs-sync-dropbox/" >Keep Multiple Macs in Sync with Dropbox</a> and <a href="http://blog.fosketts.net/2009/02/05/compression-encryption-deduplication-replication/" >Compression, Encryption, Deduplication, and Replication: Strange Bedfellows</a></p></blockquote>
<p><a href="http://blog.fosketts.net/2011/03/03/multiple-macs-sync-dropbox/" ></a>Yesterday, <a rel="nofollow" href="http://storagemistress.wordpress.com/2011/07/04/ladies-hide-your-porn/" >&#8220;Storagemistress&#8221; suggested a method of securely hiding &#8230; &#8220;stuff&#8221; &#8230;</a> using the excellent open-source software, TrueCrypt. This has the advantage of cross-platform compatibility and obfuscation (she suggests using a movie title as &#8220;cover&#8221;) but isn&#8217;t all that Dropbox friendly. Change one bit in that file, and the whole thing will have to be re-synced.</p>
<p>Mac users have a great alternative method: The SparseBundle. This has a few advantages over monolithic disk images:</p>
<ol>
<li>SparseBundles are thin provisioned, growing as data is added and only taking up as much space as is actually used (to the nearest 1 MB), while conventional disk images are entirely provisioned when they are created.</li>
<li>SparseBundles store data in 1 MB &#8220;bands&#8221; that can be independently synchronized with rsync or Dropbox, as opposed to conventional images that are stored as a single file.</li>
</ol>
<p>These two elements make SparseBundles excellent for securing valuable data while protecting it off-site. Dropbox loves the 1 MB bands, and I feel better knowing my data is encrypted in case the service has another &#8220;oops&#8221; moment.</p>
<h3>Create Your SparseBundle</h3>
<p>Creating a SparseBundle Disk Image is fairly straightforward. Note that this only works in Mac OS X 10.5 &#8220;Leopard&#8221; and newer releases.</p>
<ol>
<li>Launch Disk Utility &#8211; I like to just type that into Spotlight, but it&#8217;s in Applications/Utilities</li>
<li>Click &#8220;New Image&#8221; as seen in the image at the top of this post</li>
<li>Give your SparseBundle a file name in &#8220;Save As&#8221; and locate it in a folder (e.g., your Dropbox folder!)</li>
<div id="attachment_5853" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Filled-In.png" ><img class="size-medium wp-image-5853" title="Disk Utility Filled In" src="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Filled-In-300x260.png" alt="" width="300" height="260" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Disk Utility is all you need to create a secure storage location</p></div>
<li>Enter a friendly name for the disk image &#8211; this can be the same or different from your file name</li>
<li>We&#8217;ll leave &#8220;Format&#8221; and &#8220;Partitions&#8221; untouched</li>
<p><img style=' display: block; margin-right: auto; margin-left: auto;'  class="aligncenter size-full wp-image-5854" title="Disk Utility Custom Size" src="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Custom-Size.png" alt="" width="307" height="131" /></p>
<li>Click the dropdown menu for &#8220;Size&#8221; and select &#8220;Custom&#8221; to enter a custom maximum file size</li>
<p><a href="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Encryption.png" ><img style=' display: block; margin-right: auto; margin-left: auto;'  class="aligncenter size-full wp-image-5855" title="Disk Utility Encryption" src="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Encryption.png" alt="" width="349" height="102" /></a></p>
<li>Click &#8220;Encryption&#8221; to enable 128- or 256-bit encryption</li>
<p><a href="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Sparse-Bundle.png" ><img style=' display: block; margin-right: auto; margin-left: auto;'  class="aligncenter size-full wp-image-5856" title="Disk Utility Sparse Bundle" src="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Sparse-Bundle.png" alt="" width="349" height="130" /></a></p>
<li>Click &#8220;Image Format&#8221; and select &#8220;sparse bundle disk image&#8221;</li>
<li>Now click &#8220;Create&#8221; to start the image process</li>
<p><a href="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Password.png" ><img style=' display: block; margin-right: auto; margin-left: auto;'  class="aligncenter size-medium wp-image-5858" title="Disk Utility Password" src="http://static.fosketts.net/wp-content/uploads/2011/07/Disk-Utility-Password-300x240.png" alt="" width="300" height="240" /></a></p>
<li>Since we selected encryption, a new dialog box will appear asking for a password &#8211; this is what you will enter every time you use this image, and Apple helpfully rates your password strength for you</li>
<li>Click &#8220;Ok&#8221; and you&#8217;re done &#8211; the bundle will mount as a new drive</li>
</ol>
<p>As you add and remove files from this disk image, they will be encrypted and stored in 2 MB &#8220;slices&#8221; (actually files in a directory). These work great with Dropbox, Rsync, and many other utilities.</p>
<p>This image can be mounted on any Mac, provided the password is known. But don&#8217;t try to mount it on more than one machine simultaneously!</p>
<p><em>Note:</em> If you want to obscure these files, you can rename the SparseBundle. It won&#8217;t be usable from Finder, but you can still mount it from the command line using hdiutil. But it&#8217;ll be a directory of equal-sized files, so you won&#8217;t fool anyone who knows what they&#8217;re doing.</p>
<h3>Stephen&#8217;s Stance</h3>
<p>Encryption is an important tool for individuals regardless of what they&#8217;re storing. Given the recent security failings of Dropbox, I highly recommend using methods like this to secure your important data before using the service!</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/03/03/multiple-macs-sync-dropbox/"  rel="bookmark" class="crp_title">Keep Multiple Macs in Sync with Dropbox</a></li><li><a href="http://blog.fosketts.net/2011/07/11/dropbox-data-format-deduplication/"  rel="bookmark" class="crp_title">How Does Dropbox Store Data?</a></li><li><a href="http://blog.fosketts.net/2011/03/01/google-dropbox-revolutionized-laptop-migration/"  rel="bookmark" class="crp_title">How Google and Dropbox Revolutionized My Laptop Migration</a></li><li><a href="http://blog.fosketts.net/2010/08/11/time-machine-completed-verification-backups-improve-reliability-time-machine-create-backup/"  rel="bookmark" class="crp_title">&#8220;Time Machine completed a verification of your backups. To improve reliability, Time Machine must create a new backup for you.&#8221;</a></li><li><a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/"  rel="bookmark" class="crp_title">Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/">Mac Users, Secure Your Stuff in Dropbox</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>, <a href="http://blog.fosketts.net/category/everything/terabytehome/" title="View all posts in Terabyte home" rel="category tag">Terabyte home</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/07/05/mac-dropbox-encrypted-volume/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>Three Key Storage Features Missing in Mac OS X &#8220;Lion&#8221;</title>
		<link>http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/</link>
		<comments>http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/#comments</comments>
		<pubDate>Mon, 13 Jun 2011 16:42:43 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Enterprise storage]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[10.7]]></category>
		<category><![CDATA[ATTO]]></category>
		<category><![CDATA[CalDigit]]></category>
		<category><![CDATA[Core Data incremental storage]]></category>
		<category><![CDATA[Drobo]]></category>
		<category><![CDATA[globalSAN]]></category>
		<category><![CDATA[HFS]]></category>
		<category><![CDATA[iCloud]]></category>
		<category><![CDATA[iCloud Storage API]]></category>
		<category><![CDATA[iSCSI]]></category>
		<category><![CDATA[LaCie]]></category>
		<category><![CDATA[Lion]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[Studio Network Solutions]]></category>
		<category><![CDATA[Thunderbolt]]></category>
		<category><![CDATA[USB 3.0]]></category>
		<category><![CDATA[Xsan]]></category>
		<category><![CDATA[Xtend]]></category>
		<category><![CDATA[ZFS]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5654</guid>
		<description><![CDATA[Apple is not in enterprise storage company to be sure, and news from WWDC dashes any hopes we had for ZFS and iSCSI support. USB 3.0 seems a foregone conclusion, but Apple seems intent on ignoring it as long as possible. Although I welcome the new storage features included in Lion, it is disappointing that these were left out.]]></description>
			<content:encoded><![CDATA[<div id="attachment_5658" class="wp-caption aligncenter" style="width: 112px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/06/overview_callout_osx.png" ><img class="size-full wp-image-5658" title="overview_callout_osx" src="http://static.fosketts.net/wp-content/uploads/2011/06/overview_callout_osx.png" alt="" width="102" height="116" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Mac OS X 10.7 &quot;Lion&quot; lacks many of the storage features we&#39;ve long hoped for, including ZFS, iSCSI, and USB 3.0</p></div>
<p>Last week, at WWDC, Apple introduced many of the features found in their next operating system, OS X “Lion”. At that time, I posted an article about the <a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/" >storage features found in this new release</a>, including integrated revision control, a major update to the FileVault encryption package, and additional enterprise storage protocol support. But, <a href="http://blog.fosketts.net/2009/06/09/snow-leopard-storage/" >like Snow Leopard</a>, Lion still lacks many storage related features, and it doesn&#8217;t look like Apple will get around to adding these anytime soon.</p>
<h3>A Better Filesystem (ZFS, Please)</h3>
<blockquote><p>See also <a href="http://blog.fosketts.net/2008/02/27/zfs-super-file-system/" >ZFS: Super File System!</a> and <a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/" >Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></p></blockquote>
<p>During the development of Mac OS 10.5, “Leopard”, Apple spent a great deal of time working to replace their legacy HFS+ filesystem with ZFS, a much more advanced option. For starters, ZFS would have given Mac OS better reliability and flexibility, and it has been extended to include advanced features for security and capacity optimization.</p>
<p>But the battle between Sun and NetApp over patents related to the development of ZFS cast a shadow over the long needed replacement of HFS+. With Oracle buying Sun and focusing away from infrastructure products like ZFS, Apple seems to have lost interest in replacing their crufty old filesystem.</p>
<p>Instead of adding an advanced filesystem like ZFS, Mac OS X Lion extends HFS+ with versioning and enhanced security. The new Core Data incremental storage technology in Lion would probably have been easier to implement on ZFS, but Apple was able to add it to HFS+, and it will be a lifesaver in the guise of autosave, versions, and resume. The same goes for encryption, with FileVault 2 boasting background full disk encryption, remote wipe, and external drive support.</p>
<p>Those hoping for the integration of ZFS with Mac OS X appear to be out of luck. All components were removed from Snow Leopard, and Lion is moving forward without it. Sadly, this means that Mac OS X still lacks a flexible volume manager, something even Microsoft Windows boasts.</p>
<p>Perhaps the next version of Mac OS X will include friendly volume management features, but it is more likely that Apple will focus away from the filesystem and direct application developers toward the iCloud Storage API. And cloud truly is next-generation storage, making this a leapfrog approach and leaving ZFS in the dust.</p>
<blockquote><p>Update: Lion does indeed include a full logical volume manager! See <a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/" >Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></p></blockquote>
<h3>USB 3.0: Still AWOL</h3>
<blockquote><p>See also <a href="http://blog.fosketts.net/2010/10/18/os-107-lion-bring-usb-30-mac/" >Will OS X 10.7 “Lion” Bring USB 3.0 To The Mac?</a> and <a href="http://blog.fosketts.net/series/thunderbolt/" >my Thunderbolt series</a></p></blockquote>
<p>Although Apple may have simply overlooked or neglected to mention it, USB 3.0 apparently made no appearance at WWDC. <a href="http://blog.fosketts.net/series/thunderbolt/" >Thunderbolt is an impressive technology</a> to be sure, and I am bullish on its future application and performance. But “SuperSpeed” USB 3.0 seems poised to seize the baton and become the ubiquitous next-generation interconnect for every day peripherals.</p>
<p>Thunderbolt is a strategic protocol for Apple, and I expect it to rapidly spread across the entire Mac product range. Rather than simply a high-speed interconnect, Thunderbolt will soon enable advanced docking features, as envisioned in my recent post about <a href="http://blog.fosketts.net/2011/04/20/thunderbolt-imac-peripheral-macbook-pro/" >the iMac as a Thunderbolt peripheral</a>. It will also enable changes to the physical size and shape of laptop and desktop computers and servers, with many suggesting that the next-generation MacBook Air will become the standard Apple laptop.</p>
<p>In contrast, USB 3.0 is simply a performance bump for USB. It is likely that Apple will support USB 3.0 sooner or later, and third-party vendors are already rolling out Mac OS support. CalDigit recently shipped <a href="http://blog.fosketts.net/2011/05/02/caldigit-fasta-6gu3-esata-usb-3-mac-pro/" >their third USB 3.0 controller</a> for the Mac, and LaCie sells their own “<a href="http://blog.fosketts.net/2010/11/05/lacie-usb-30-driver-mac-osx-troubleshooting/" >walled garden</a>” card and peripherals as well. I heard rumors that a few vendors are working on Thunderbolt to USB 3.0 bridges and breakout boxes as well.</p>
<p>USB 3.0 will come to the Mac sooner or later, but Thunderbolt is here to stay.</p>
<h3>Enterprise iSCSI Support</h3>
<blockquote><p>See also <a href="http://blog.fosketts.net/2009/03/18/snow-leopard-iscsi/" >Will Snow Leopard Finally Bring iSCSI To The Mac?</a></p></blockquote>
<p>Another technology that Apple has <a href="http://www.zdnet.com/blog/apple/waiting-for-leopards-iscsi-support/1097" >flirted with</a> in Leopard but never delivered is a software initiator for iSCSI, the block storage protocol that runs over Ethernet. We have not heard anything further about iSCSI since 2007, and there was no mention in the Lion introduction either.</p>
<p>Interestingly, Apple did rollout other new enterprise storage protocol options, including NFSv4, DFS, and even integration of Xsan, the Fibre Channel filesystem. Xsan also added ALUA compatible multipathing, a real surprise for storage geeks like me. But iSCSI was nowhere to be found.</p>
<p>This is a real shame, since iSCSI is becoming increasingly common in enterprise storage circles. Convergence on Ethernet is a hot topic right now, and iSCSI for Mac would give exceptional flexibility and interoperability and fit right into the &#8220;prosumer&#8221; Mac market niche.</p>
<p>Instead, end-users are stuck working with third-party iSCSI initiators, Fibre Channel and Xsan, or NFS. Although I am a fan of their <a href="http://www.studionetworksolutions.com/products/product_detail.php?pi=11" >free globalSAN product</a>, Studio Network Solutions does not offer enterprise support for third-party arrays. The other major option for Mac iSCSI is <a href="http://www.attotech.com/products/product.php?scat=17&amp;sku=INIT-MAC0-001" >ATTO&#8217;s Xtend</a>, which is supported but somewhat expensive. <a href="http://www.drobo.com/resources/iscsi.php" >Drobo also offers an iSCSI client</a> for use with their storage arrays, but it is severely limited. An integrated Apple solution would be a welcome addition, both for consumers and enterprise systems administrators.</p>
<h3>Stephen&#8217;s Stance</h3>
<p>Apple is not in enterprise storage company to be sure, and news from WWDC dashed any hopes we had for ZFS and iSCSI support. USB 3.0 seems a foregone conclusion, but Apple seems intent on ignoring it as long as possible. Although I welcome <a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/" >the new storage features included in Lion</a>, it is disappointing that these were left out.</p>
<p>Note that TRIM support was also not mentioned at WWDC, but it is likely included.</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/"  rel="bookmark" class="crp_title">Key Storage Features in Mac OS X 10.7 &#8220;Lion&#8221;</a></li><li><a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/"  rel="bookmark" class="crp_title">Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></li><li><a href="http://blog.fosketts.net/2009/06/09/snow-leopard-storage/"  rel="bookmark" class="crp_title">Snow Leopard Is Stingy With The Storage Love</a></li><li><a href="http://blog.fosketts.net/2009/03/18/snow-leopard-iscsi/"  rel="bookmark" class="crp_title">Will Snow Leopard Finally Bring iSCSI To The Mac?</a></li><li><a href="http://blog.fosketts.net/2011/05/02/caldigit-fasta-6gu3-esata-usb-3-mac-pro/"  rel="bookmark" class="crp_title">CalDigit Brings Both eSATA and USB 3 to the Mac Pro</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/">Three Key Storage Features Missing in Mac OS X &#8220;Lion&#8221;</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/enterprisestorage/" title="View all posts in Enterprise storage" rel="category tag">Enterprise storage</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
	
		<series:name><![CDATA[Mac OS X Lion]]></series:name>
	</item>
		<item>
		<title>Key Storage Features in Mac OS X 10.7 &#8220;Lion&#8221;</title>
		<link>http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/</link>
		<comments>http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/#comments</comments>
		<pubDate>Mon, 06 Jun 2011 21:30:19 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Enterprise storage]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Terabyte home]]></category>
		<category><![CDATA[10.7]]></category>
		<category><![CDATA[AirDrop]]></category>
		<category><![CDATA[Core Data incremental storage]]></category>
		<category><![CDATA[DFS]]></category>
		<category><![CDATA[FileVault]]></category>
		<category><![CDATA[FileVault 2]]></category>
		<category><![CDATA[HFS]]></category>
		<category><![CDATA[iCloud Storage API]]></category>
		<category><![CDATA[Lion]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[NFSv4]]></category>
		<category><![CDATA[Quantum]]></category>
		<category><![CDATA[SMB]]></category>
		<category><![CDATA[Steve Jobs]]></category>
		<category><![CDATA[StorNext]]></category>
		<category><![CDATA[Studio Network Solutions]]></category>
		<category><![CDATA[Time Machine]]></category>
		<category><![CDATA[TRIM]]></category>
		<category><![CDATA[Versions]]></category>
		<category><![CDATA[WWDC]]></category>
		<category><![CDATA[Xsan]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5627</guid>
		<description><![CDATA[Apple's not an enterprise company or a storage company, but Apple does have enterprise storage features in their operating systems. And Mac OS X 10.7 "Lion" is a great case in point. From Versions to Time Machine Local Snapshots to AirDrop, Lion brings some storage love, and NFS, SMB, and Xsan are there, too. Let's look at what's new and key in terms of storage in the latest version of Mac OS X.]]></description>
			<content:encoded><![CDATA[<div id="attachment_5658" class="wp-caption aligncenter" style="width: 112px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/06/overview_callout_osx.png" ><img class="size-full wp-image-5658" title="overview_callout_osx" src="http://static.fosketts.net/wp-content/uploads/2011/06/overview_callout_osx.png" alt="" width="102" height="116" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Mac OS X 10.7 &quot;Lion&quot; is coming, and it&#39;s bringing a few storage features to the table</p></div>
<p>Apple&#8217;s not an enterprise company or a storage company. In fact, they&#8217;re rapidly jettisoning both &#8211; consider the sad fate of the <a href="http://blog.fosketts.net/tag/xserve-raid/" >Xserve RAID</a>. But Apple does have enterprise features and storage features in their operating systems. And Mac OS X 10.7 &#8220;Lion&#8221; is a great case in point. From Versions to Time Machine Local Snapshots to AirDrop, Lion brings some storage love, and iCloud&#8217;s Storage API could be game-changing. Let&#8217;s look at what&#8217;s new and key in terms of storage in the latest version of Mac OS X.</p>
<h3>New and Updated Storage Features in Lion</h3>
<p>Lion is strong on consumer-oriented features, of course. And Apple is leading the industry in pushing user-friendly storage features for data protection and sharing. OS features like Core Data incremental storage enable Auto Save and Versions, Time Machine gets local snapshots, and FileVault is updated into a whole-disk encryption (WDE) tool.</p>
<blockquote><p>Update: Probably the most important storage feature, CoreStorage, went un-covered! Read more at <a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/" >Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></p></blockquote>
<h4>Auto Save, Versions, and Resume</h4>
<div id="attachment_5659" class="wp-caption aligncenter" style="width: 238px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/06/autosave_browse-e1307395294832.jpg" ><img class="size-full wp-image-5659" title="autosave_browse" src="http://static.fosketts.net/wp-content/uploads/2011/06/autosave_browse-e1307395294832.jpg" alt="" width="228" height="130" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">AutoSave and Versions leverage advances in HFS+ and a new Core API</p></div>
<p>Like iOS, Lion enhances the &#8220;back where you were&#8221; nature of computing with Auto Save, Versions, and system Resume. These completely change the end-user computing experience: Applications don&#8217;t have temporal &#8220;use once&#8221; interfaces but have lasting, historical state. And the ability to move through time (à la Time Machine)</p>
<p>Of course, lots of applications have had auto-save in the past. But Lion adds OS-level interfaces and APIs to enable applications to save data in a standard way. And these will be integrated with Resume (see below) for a very iOS-like experience.</p>
<p>Lion enables all this by enhancing the old, familiar HFS+ filesystem with <strong>Core Data incremental storage</strong>, a snapshot-like interface to save and recover multiple point-in-time instances of a single document. This is a delta differencing system, probably on a block level, in the filesystem.</p>
<p>Versions are accessed through the title bar document name, as well as a Time Machine-like interface in some applications. Applications can open up multiple versions of the same document at once, and you can cut and paste between them.</p>
<p>Resume is very cool. Applications using the new Lion APIs can save their state, even through reboots! This is what computers should have always done, but no one ever implemented it. Why should a reboot wipe out where you were last? Although not really a storage feature, resume relies on Auto Save and new APIs to store application state.</p>
<h4>Time Machine Local Snapshots</h4>
<div id="attachment_5431" class="wp-caption alignleft" style="width: 160px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; float: left;"><a href="http://blog.fosketts.net/2011/05/23/mac-osx-lion-time-machine-local-snapshots/" ><img class="size-thumbnail wp-image-5431 " title="Lion Time Machine Local Snapshots" src="http://blog.fosketts.net/wp-content/uploads/2011/05/Lion-Time-Machine-Local-Snapshots-150x110.png" alt="" width="150" height="110" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Time Machine in Mac OS X &quot;Lion&quot; includes local snapshots as well as storage of backups on external disks</p></div>
<blockquote><p>See also <a href="http://blog.fosketts.net/2011/05/23/mac-osx-lion-time-machine-local-snapshots/" >Local Snapshots in Mac OS X Lion Time Machine: Is It A Good Idea?</a></p></blockquote>
<p>And speaking of Time Machine, it&#8217;s been enhanced with Local Snapshots, a mechanism for storing data on the local drive in addition to an external Time Machine drive. The Time Machine interface combines local snapshots and the backup drive or Time Capsule into a single timeline when browsing.</p>
<p>This appears to use Core Data incremental storage as well, though it&#8217;s not definite. And it&#8217;s unclear how local snapshots will interact with Versions.</p>
<h4>AirDrop</h4>
<p><a href="http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/" ><img style=' float: right; padding: 4px; margin: 0 0 2px 7px;'  class="alignright size-full wp-image-5660" title="whatsnew_icon_airdrop" src="http://static.fosketts.net/wp-content/uploads/2011/06/whatsnew_icon_airdrop.png" alt="" width="60" height="60" /></a></p>
<blockquote><p>See also <a href="http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/" >Snooping on AirDrop in Apple’s Mac OS X 10.7 “Lion”</a></p></blockquote>
<p>AirDrop is a new wireless file sharing protocol. Although it smacks of Wi-Fi Direct or Bluetooth, it&#8217;s neither. Instead, AirDrop is a newly-developed proprietary mechanism of sharing data between two Macs over a Wi-Fi link. It&#8217;ll be speedy and simple but incompatible with the vast majority of systems out there.</p>
<h4>FileVault 2</h4>
<p><a href="http://static.fosketts.net/wp-content/uploads/2011/06/features_filevault2_icon.png" ><img style=' float: right; padding: 4px; margin: 0 0 2px 7px;'  class="alignright size-full wp-image-5661" title="features_filevault2_icon" src="http://static.fosketts.net/wp-content/uploads/2011/06/features_filevault2_icon.png" alt="" width="64" height="64" /></a>Face it, FileVault was little-used and unfriendly. That&#8217;s all changed with FileVault 2 in Lion. It&#8217;s encryption done right and might just push average Apple users to protect their data. Kind of like what Time Machine did for backup. That&#8217;s a good thing.</p>
<p>What&#8217;s new in FileVault 2?</p>
<ul>
<li>Full-disk encryption, rather than an <a href="http://arstechnica.com/apple/news/2006/12/6436.ars" >image-based hack</a>.</li>
<li>Encrypt in place, in the background, while you work. No more waiting and losing your computer for hours or days while encryption is installed. And the encryption process will resume even after a reboot (or two)!</li>
<li>Encrypt external drives, too. Very sweet, especially for Time Machine backups!</li>
<li>Remote wipe service, probably through iCloud</li>
<li><a rel="nofollow" href="http://en.wikipedia.org/wiki/Disk_encryption_theory#XTS" >XTS-AES 128</a> support</li>
</ul>
<p>FileVault 2 is full-disk only, but supports multiple users (each with their own encrypted home area) by storing the full-disk key in each user&#8217;s keychain. They access it by using their login password, just like they always did. Apple will offer an option to store the encryption key in iCloud, but this does not appear to be the default condition.</p>
<p>Of course, users with poor passwords, or those vulnerable to social engineering, will still be vulnerable. But some encryption beats no encryption any day! Lion uses the login window for sleep and screen savers, so FileVault 2 protects in those states as well.</p>
<h4>NFSv4</h4>
<p>Lion adds support for NFS version 4, which is a welcome update. But it doesn&#8217;t appear to support version 4.1, or parallel NFS.</p>
<h4>SMB with DFS</h4>
<p>DFS is a technology in Microsoft Windows that virtualizes file server shares, improving flexibility. Lion&#8217;s SMB client supports DFS when connecting to Windows file servers.</p>
<h4>Xsan Built In</h4>
<p>Mac OS X Server will now be an optional extra to add to the standard Lion install, rather than a separate SKU. Part of this shift is the integration of Xsan, Apple&#8217;s OEM version of Quantum&#8217;s StorNext SAN file system. Xsan has been quite popular in the media space on client machines, and it appears that it does not require a Server license.</p>
<p>Xsan also gets case-insensitive volumes, which brings it more in line with the expectations of users used to HFS+. Finally, Lion gets ALUA-compatible multipathing, supporting a wide variety of storage arrays.</p>
<h4>Probable: TRIM Support for SSDs</h4>
<p>Apple enabled TRIM in certain versions of 10.6 &#8220;Snow Leopard&#8221;, but it only supported their OEM SSDs. Lion probably gets a fully-supported TRIM implementation that works with any SSD with TRIM, though this was not mentioned at WWDC or online. This is a welcome update, and one too-long in coming!</p>
<p><span style="font-size: 15px; font-weight: bold;">Stephen&#8217;s Stance</span></p>
<p>Mac OS X &#8220;Lion&#8221; doesn&#8217;t give storage folks much, but it could be a harbinger of massive changes. Although not part of Lion per se, the iCloud Storage API will likely see much use by application developers. But even Lion&#8217;s smaller storage feature list is welcome. Simple full-disk encryption in FileVault and Core Data incremental storage are promising. Let&#8217;s just hope TRIM comes along for the ride!</p>
<p>On the enterprise side of things, Lion is improving as a storage client with NFSv4, DFS, and Xsan built in. But there&#8217;s no mention of an iSCSI client, suggesting that project is dead. Thank goodness for <a href="http://www.studionetworksolutions.com/products/product_detail.php?pi=11" >Studio Network Solutions</a>! And we&#8217;re still stuck with HFS+, suggesting the ZFS transition is off the table, too. Too bad &#8211; Versions and FileVault would have been much easier to implement on ZFS!</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/05/23/mac-osx-lion-time-machine-local-snapshots/"  rel="bookmark" class="crp_title">Local Snapshots in Mac OS X Lion Time Machine: Is It A Good Idea?</a></li><li><a href="http://blog.fosketts.net/2011/06/13/storage-features-missing-lion/"  rel="bookmark" class="crp_title">Three Key Storage Features Missing in Mac OS X &#8220;Lion&#8221;</a></li><li><a href="http://blog.fosketts.net/2011/08/04/mac-osx-lion-corestorage-volume-manager/"  rel="bookmark" class="crp_title">Mac OS X Lion Adds CoreStorage, a Volume Manager (Finally!)</a></li><li><a href="http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/"  rel="bookmark" class="crp_title">Snooping on AirDrop in Apple&#8217;s Mac OS X 10.7 &#8220;Lion&#8221;</a></li><li><a href="http://blog.fosketts.net/2011/05/27/pile-interesting-links-27-2011/"  rel="bookmark" class="crp_title">Back From the Pile: Interesting Links, May 27, 2011</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/">Key Storage Features in Mac OS X 10.7 &#8220;Lion&#8221;</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/enterprisestorage/" title="View all posts in Enterprise storage" rel="category tag">Enterprise storage</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/everything/terabytehome/" title="View all posts in Terabyte home" rel="category tag">Terabyte home</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
	
		<series:name><![CDATA[Mac OS X Lion]]></series:name>
	</item>
		<item>
		<title>Snooping on AirDrop in Apple&#8217;s Mac OS X 10.7 &#8220;Lion&#8221;</title>
		<link>http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/</link>
		<comments>http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/#comments</comments>
		<pubDate>Wed, 01 Jun 2011 15:00:37 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[Terabyte home]]></category>
		<category><![CDATA[10.7]]></category>
		<category><![CDATA[Atheros]]></category>
		<category><![CDATA[Bluetooth]]></category>
		<category><![CDATA[Broadcomm]]></category>
		<category><![CDATA[Lion]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[Wi-Fi]]></category>
		<category><![CDATA[Wi-Fi Direct]]></category>
		<category><![CDATA[Wi-Fi virtualization]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5573</guid>
		<description><![CDATA[Apple has aggressively moved to eliminate “superfluous” peripherals and connections, wiping out the floppy and now selling a number of machines without optical drives. AirDrop continues this progression, attacking the prime use case for USB flash drives.]]></description>
			<content:encoded><![CDATA[<div id="attachment_5575" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a href="http://static.fosketts.net/wp-content/uploads/2011/06/AirDrop-Screen-shot-2011-02-24-at-21.54.54.png" ><img class="size-medium wp-image-5575" title="AirDrop-Screen-shot" src="http://static.fosketts.net/wp-content/uploads/2011/06/AirDrop-Screen-shot-2011-02-24-at-21.54.54-300x155.png" alt="" width="300" height="155" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">AirDrop enables direct file sharing over Wi-Fi</p></div>
<p>As computers get easier to use, once-&#8221;magical&#8221; features like Wi-Fi and Bluetooth begin showing usability issues. Although every modern computer operating system includes <a href="http://blog.fosketts.net/2008/07/25/quick-and-easy-bluetooth-sharing-between-pc-and-mac/" >the ability to share files locally</a>, it is usually a major hassle setting things up. Apple intends to remedy this situation with <a rel="nofollow" href="http://www.apple.com/macosx/lion/" >AirDrop</a>, a Wi-Fi-based local filesharing protocol built into Mac OS X “Lion”. How exactly does AirDrop work? Let&#8217;s take a look.</p>
<h3>Wi-Fi Tricks</h3>
<p><a href="https://www.cwnp.com/index/cwnp_wifi_blog/three-spatial-streams-the-good-bad-and-ugly" >Wi-Fi is no simple technology</a>. Modern Wi-Fi radio chipsets include multiple transmitters, receivers, and antennas. These are normally used to boost throughput for a single link, but it&#8217;s possible to do much more with these radios.</p>
<p>A little-known feature in Microsoft Windows is <a href="http://www.istartedsomething.com/20090516/windows-7-native-virtual-wifi-technology-microsoft-research/" >Wi-Fi virtualization</a>, allowing certain Wi-Fi chips to act as both a client and a base station at the same time. In this way, a Windows 7 machine can simultaneously access a Wi-Fi network and share that network with multiple clients.</p>
<p>Another new trick for Wi-Fi is “<a rel="nofollow" href="http://revolutionwifi.blogspot.com/2011/05/wi-fi-direct-devices-begin-hitting.html" >Wi-Fi Direct</a>”, a short range communication protocol that functions similarly to Bluetooth but uses Wi-Fi hardware instead. The first Wi-Fi Direct hardware is just beginning to appear in early 2011, in the form of chipsets from the major vendors as well as a mouse from HP and a card from Eye-Fi.</p>
<h3>Introducing AirDrop</h3>
<p><a href="http://static.fosketts.net/wp-content/uploads/2011/06/overview_airdrop_icon20110224.jpg" ><img style=' float: right; padding: 4px; margin: 0 0 2px 7px;'  class="alignright size-full wp-image-5574" title="overview_airdrop_icon20110224" src="http://static.fosketts.net/wp-content/uploads/2011/06/overview_airdrop_icon20110224.jpg" alt="" width="60" height="60" /></a>Apple&#8217;s AirDrop is functionally similar to a hybrid between Wi-Fi direct and Wi-Fi virtualization, but it is a proprietary Apple protocol. AirDrop allows two computers (running Mac OS X “Lion” on compatible hardware) to <a href="http://www.appleinsider.com/articles/11/03/10/inside_mac_os_x_10_7_lion_airdrop_local_file_sharing.html" >transfer files</a> in a <a href="http://www.9to5mac.com/53899/mac-os-x-lions-airdrop-in-action-tests-instructions-and-video/" >friendly</a>, no-configuration-required mechanism direct from Finder.</p>
<p>AirDrop will prove useful in business and classroom settings where one must quickly and easily move files between computers. A teacher could open AirDrop and collect assignments from students or pass out new materials, and collaborators in a conference room or airport could quickly exchange information.</p>
<h3>AirDrop Concerns</h3>
<p>The AirDrop interface itself seems fairly robust and secure, with no permanent connections or authentication. AirDrop is only active when one clicks on the icon in Finder, and every file transfer requires permission on both the sending and receiving systems. AirDrop connections are firewalled, and it automatically encrypts all transactions using <a rel="nofollow" href="http://en.wikipedia.org/wiki/Transport_Layer_Security" >TLS</a>, so snooping is not much of a concern either.</p>
<p>AirDrop support is limited to Mac OS X “Lion”, and requires modern Wi-Fi hardware from Atheros or Broadcomm. Most recent machines include capable hardware, but the early Broadcomm BCM4321 found in the early 2009 Mac Mini is not supported. And AirDrop does not use Wi-Fi Direct, being a proprietary protocol developed by Apple. This makes it unlikely that it will spread beyond the Macintosh computer range.</p>
<p>Apple has not indicated that AirDrop will spread to iOS devices like the iPhone and iPad, but it seems a logical and useful addition. But <a href="http://wirelesslanprofessionals.com/apple-ipad-wi-fi-detailed-analysis/" >these devices have a very basic Wi-Fi implementation</a>, lacking multiple spatial streams. If they do not support AirDrop and Wi-Fi simultaneously, Apple may still enable it as an either/or option.</p>
<h3>Stephen&#8217;s Stance</h3>
<p>Apple has aggressively moved to eliminate “superfluous” peripherals and connections, wiping out the floppy and now selling a number of machines without optical drives. AirDrop continues this progression, attacking the prime use case for USB flash drives.</p>
<p>One can imagine an exciting use case for this technology, but it is disappointingly limited to recent Macs running the latest operating system. It would certainly be more consumer friendly if Apple had decided to leverage Wi-Fi Direct and expanded support to PCs and iOS devices. Sadly, the only likely expansion of AirDrop is to the iPad and iPhone.</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/06/06/storage-features-mac-os-107-lion/"  rel="bookmark" class="crp_title">Key Storage Features in Mac OS X 10.7 &#8220;Lion&#8221;</a></li><li><a href="http://blog.fosketts.net/2008/07/25/quick-and-easy-bluetooth-sharing-between-pc-and-mac/"  rel="bookmark" class="crp_title">Quick and Easy Bluetooth Sharing Between PC and Mac</a></li><li><a href="http://blog.fosketts.net/2011/07/13/apple-80211n-wifi-support/"  rel="bookmark" class="crp_title">Which Apple Devices Support 802.11n Wi-Fi?</a></li><li><a href="http://blog.fosketts.net/2011/05/23/mac-osx-lion-time-machine-local-snapshots/"  rel="bookmark" class="crp_title">Local Snapshots in Mac OS X Lion Time Machine: Is It A Good Idea?</a></li><li><a href="http://blog.fosketts.net/2011/07/05/eyefi-wireless-card-reader/"  rel="bookmark" class="crp_title">Eye-Fi Workflow: Wireless Card Reader</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/">Snooping on AirDrop in Apple&#8217;s Mac OS X 10.7 &#8220;Lion&#8221;</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>, <a href="http://blog.fosketts.net/category/everything/terabytehome/" title="View all posts in Terabyte home" rel="category tag">Terabyte home</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/06/01/apple-airdrop-mac-os-107-lion/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<series:name><![CDATA[Mac OS X Lion]]></series:name>
	</item>
		<item>
		<title>Dragon Dictate for Mac: Utterly Frustrating</title>
		<link>http://blog.fosketts.net/2011/05/27/review-dragon-dictate-mac/</link>
		<comments>http://blog.fosketts.net/2011/05/27/review-dragon-dictate-mac/#comments</comments>
		<pubDate>Fri, 27 May 2011 16:05:39 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Everything]]></category>
		<category><![CDATA[Features]]></category>
		<category><![CDATA[Personal]]></category>
		<category><![CDATA[dictation]]></category>
		<category><![CDATA[Dragon]]></category>
		<category><![CDATA[Dragon Dictate]]></category>
		<category><![CDATA[featured]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[Microsoft]]></category>
		<category><![CDATA[Nuance]]></category>
		<category><![CDATA[reviews]]></category>
		<category><![CDATA[Windows 7]]></category>
		<category><![CDATA[Windows Vista]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5535</guid>
		<description><![CDATA[For the last few months, I've been experimenting with dictation software on the Mac. Previously, I had used the built-in dictation software in Windows Vista and Windows 7, but it was annoying to launch a virtual machine every time I wanted to dictate something. On the Mac and purchase an (expensive) copy of Dragon Dictate for Mac. So far, my experience has been less than positive.]]></description>
			<content:encoded><![CDATA[<div id="attachment_4939" class="wp-caption aligncenter" style="width: 310px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;"><a rel="nofollow" href="http://www.amazon.com/Nuance-Communications-Inc-S601A-G00-2-0-Dictate/dp/B003YUJBXK%3FSubscriptionId%3DAKIAJYEMQAFREVFYOMPQ%26tag%3DPackrat-20%26linkCode%3Dxm2%26camp%3D2025%26creative%3D165953%26creativeASIN%3DB003YUJBXK" ><img class="size-full wp-image-4939 " title="Dragon Dictate 2.0 Mac" src="http://static.fosketts.net/wp-content/uploads/2011/02/Dragon-Dictate-2.0-Mac.jpg" alt="" width="300" height="300" /></a><p style=' padding: 0 4px 5px; margin: 0;'  class="wp-caption-text">Dragon Dictate might be the best dictation option for Mac users, but it&#39;s not very good at that</p></div>
<p>For the last few months, I&#8217;ve been experimenting with dictation software on the Mac. Previously, I had used the built-in dictation software in Windows Vista and Windows 7, but it was annoying to launch a virtual machine every time I wanted to dictate something. On the Mac and purchase an (expensive) copy of Dragon Dictate for Mac. So far, my experience has been less than positive.</p>
<h3>What I Was Looking For</h3>
<p>I use dictation software for dictation, not control my computer. I was looking for a product that would allow me to speak naturally to my computer as a way to accelerate my writing. See, I am a professional writer and regularly churn out thousands of words per week. Although I can type very fast, I find that dictation allows me to sit back and collect my thoughts without being distracted by the keyboard.</p>
<p>It is critical for me that the software I select recognizes my words consistently and does not require a great deal of post-speech editing. In other words, I want to be able to carefully talk through an entire piece without going back and doing word by word checking after the fact.</p>
<p>Since I write on technical topics (I specialize in enterprise data storage), it is critical that the dictation software is flexible in allowing bizarre spelling and capitalization combinations. It seems that every enterprise product and company has some strange combination of capital letters and made-up words!</p>
<h3>Dragon Dictates (Sometimes)</h3>
<p>Dragon Dictate has proven frustrating in regular use, especially for a software package that cost nearly $200. Although it is quite adept at recognizing flowing speech, it utterly fails to be useful in regular applications and workflows.</p>
<p>Adding new words to the Dragon dictionary is <a href="http://voicesofdragon.com/2011/05/25/ask-the-dictator-training-words-mac/" >frustratingly complicated</a>, requiring nearly a dozen steps with keyboard, mouse, and the GUI. Why can&#8217;t an end-user <em>dictate</em> a new word? Alas, this seems to be impossible. Spelled (or even typed) words don&#8217;t appear to be added to the dictionary and will be mis-recognized even in the same document.</p>
<p>Dragon stubbornly refuses to learn certain words, as well. I can&#8217;t figure out how to tell it to spell the word “minutes”, for example, rather than the abbreviation &#8220;min.&#8221; though there is such an option in Dragon&#8217;s PC product. It is frustratingly inconsistent at recognizing common punctuation marks and commands, often typing “exhalation point” or “go to and” rather than “!” or moving the cursor to the end of the line. I can&#8217;t figure out how to tell it to spell my name without a “V” and it even mis-recognizes its own name as &#8220;DragonDictate” (no space).</p>
<p>But by far the most frustrating aspect of Dragon Dictate is its incompatibility with just about every native application I use. The only satisfying dictation experience relies on the ultra basic text editor included with Dictate. Although it is supposed to be compatible with Microsoft Word, actual usage is frustrating. And attempting to use dictate in other applications makes me want to throw my computer out the window, as illustrated in the video below.</p>
<p><object width="500" height="306"><param name="movie" value="http://www.youtube.com/v/uBQY0yR1C1s?version=3"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/uBQY0yR1C1s?version=3" type="application/x-shockwave-flash" width="500" height="306" allowscriptaccess="always" allowfullscreen="true"></embed></object></p>
<h3>Stephen&#8217;s Stance</h3>
<p>I am utterly frustrated with Dragon Dictate for Mac. It shows sparks of brilliance but is so tarnished by the obnoxious experience of actually using it that I have half a mind to demand my money back. And it certainly did cost quite a bit of money, especially compared to the free and less frustrating speech recognition software bundled with Microsoft Windows!</p>
<p>I&#8217;d like to see Dragon improve this product in key areas:</p>
<ol>
<li>Add a simple, streamlined <em>dictated</em> command to add a word to the dictionary &#8211; &#8220;add that&#8221; should allow you to spell and train in one go</li>
<li>Fix the obnoxious behavior outside the Dictate &#8220;notepad&#8221; window &#8211; or add an &#8220;insert only&#8221; setting to not &#8220;guess&#8221; where you are in a document and overwrite things</li>
<li>Fix the weird &#8220;add extra characters after the cursor&#8221; bug demonstrated in my video</li>
<li>Allow me to fix one or two words, not a whole phrase, and allow me to fix it outside the Recognition window&#8217;s (limited) choices</li>
<li>Add a &#8220;no abbreviations&#8221; setting to the Mac version so it stops correcting &#8220;minutes&#8221; to &#8220;min.&#8221;</li>
<li>Allow me to say &#8220;always recognize it this way&#8221; for words like &#8220;Fibre Channel&#8221; and &#8220;Stephen&#8217;s stance&#8221; (not &#8220;fiber channel&#8221; and &#8220;Steven&#8217;s stance&#8221;)</li>
<li>Improve recognition of common commands and terms (&#8220;explanation point&#8221;? Seriously? When has anyone <em>ever</em> dictated that?)</li>
</ol>
<blockquote><p>See <a href="http://blog.fosketts.net/2011/06/03/nuance-responds-dragon-dictate-concerns/" >Nuance Responds to My Dragon Dictate Concerns</a> for detailed information on these concerns</p></blockquote>
<p>Finally, I suggest that Dragon does something to lower the price. I got mine <a href="http://blog.fosketts.net/2011/02/23/dragon-dictate-2-mac-129-today/" >at a steep discount</a>, but it was still $129, which is pretty stiff for a utility of limited utility. The included headset and USB adapter are of mediocre quality and aren&#8217;t worth more than $20. If Dragon brings Dictate to the Mac App Store at $49 and fixes the bugs it might be worthwhile. But for now, it earns a &#8220;<a rel="nofollow" href="http://en.wikipedia.org/wiki/Bluth_Company" >don&#8217;t buy</a>&#8220;.</p>
<p>One more thing: <a href="http://www.nuance.com/product-support/policy.asp" >90 days of support</a>? For a product costing over $100? Seriously? I was so frustrated at my initial attempts to use Dictate that I stuck it on the shelf for months before giving it another shot. Now I have to pay for support. Or not.</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/02/23/dragon-dictate-2-mac-129-today/"  rel="bookmark" class="crp_title">Get Dragon Dictate 2 for Mac, Just $129 Today Only!</a></li><li><a href="http://blog.fosketts.net/2012/01/16/case-missing-letters-obnoxious-bug-dragon-dictate/"  rel="bookmark" class="crp_title">The Case of the Missing Letters: Another Obnoxious Bug in Dragon Dictate</a></li><li><a href="http://blog.fosketts.net/2011/06/03/nuance-responds-dragon-dictate-concerns/"  rel="bookmark" class="crp_title">Nuance Responds to My Dragon Dictate Concerns</a></li><li><a href="http://blog.fosketts.net/2011/05/27/pile-interesting-links-27-2011/"  rel="bookmark" class="crp_title">Back From the Pile: Interesting Links, May 27, 2011</a></li><li><a href="http://blog.fosketts.net/2011/12/22/terrifying-true-story-virtual-machine-mobility/"  rel="bookmark" class="crp_title">The Terrifying True Story Of Virtual Machine Mobility</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/05/27/review-dragon-dictate-mac/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/05/27/review-dragon-dictate-mac/">Dragon Dictate for Mac: Utterly Frustrating</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>, <a href="http://blog.fosketts.net/category/everything/" title="View all posts in Everything" rel="category tag">Everything</a>, <a href="http://blog.fosketts.net/category/features/" title="View all posts in Features" rel="category tag">Features</a>, <a href="http://blog.fosketts.net/category/everything/personal/" title="View all posts in Personal" rel="category tag">Personal</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/05/27/review-dragon-dictate-mac/feed/</wfw:commentRss>
		<slash:comments>9</slash:comments>
	
		<series:name><![CDATA[Dragon Dictate]]></series:name>
	</item>
		<item>
		<title>Boxcar Growl Plugin Not Working?</title>
		<link>http://blog.fosketts.net/2011/05/24/boxcar-growl-plugin-system-preferences-pane/</link>
		<comments>http://blog.fosketts.net/2011/05/24/boxcar-growl-plugin-system-preferences-pane/#comments</comments>
		<pubDate>Tue, 24 May 2011 16:03:22 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Apple]]></category>
		<category><![CDATA[Boxcar]]></category>
		<category><![CDATA[Growl]]></category>
		<category><![CDATA[iOS]]></category>
		<category><![CDATA[Mac OS X]]></category>
		<category><![CDATA[push notification]]></category>
		<category><![CDATA[System Preferences]]></category>

		<guid isPermaLink="false">http://blog.fosketts.net/?p=5527</guid>
		<description><![CDATA[The Boxcar plugin is a "display" not an "application", so it shows up under the "Display Options" tab, not the "Applications" tab in Growl on Mac OS X. The plugin installation automatically opens the wrong tab!]]></description>
			<content:encoded><![CDATA[<p>Push notifications on the iPhone and iPad were a great idea, but the implementation stinks. All too often, I have to dismiss multiple notifications before I can actually use the device, and it&#8217;s difficult to customize which notifications show or beep me. That&#8217;s why I&#8217;m enjoying <a href="http://boxcar.io" >B0xcar</a>, a push notification service for iOS devices (and soon for Mac desktops, too!) But I just couldn&#8217;t figure out how to configure the <a href="http://growl.info/" >Growl</a> plugin for Boxcar until now!</p>
<p>The Boxcar plugin for Growl is a great idea, allowing notices on the Mac to be forwarded to an iOS device on the go. But when I downloaded and installed the plugin, nothing seemed to happen. It asked if I wanted to configure the plugin and opened the &#8220;Applications&#8221; tab in Growl&#8217;s System Preferences but didn&#8217;t show up there. And I wasn&#8217;t getting notifications. What gives?</p>
<p>The <a href="http://boxcar.io/growl" >two</a> <a href="http://boxcar.io/services/help/6" >Growl plugin support pages</a> at Boxcar.io were less than helpful. But I finally figured it out: I was looking in the wrong place.</p>
<p>The Boxcar plugin is a &#8220;display&#8221; not an &#8220;application&#8221;, so it shows up under the &#8220;Display Options&#8221; tab, not the &#8220;Applications&#8221; tab in Growl on Mac OS X. The plugin installation automatically opens the wrong tab!</p>
<p><a href="http://static.fosketts.net/wp-content/uploads/2011/05/Boxcar-Growl-settings.png" ><img style=' display: block; margin-right: auto; margin-left: auto;'  class="aligncenter size-medium wp-image-5528" title="Boxcar Growl settings" src="http://static.fosketts.net/wp-content/uploads/2011/05/Boxcar-Growl-settings-300x252.png" alt="" width="300" height="252" /></a>Now that I have that fixed, I can move on with configuring Growl to notify my iOS devices for system events. And I can experiment with the new Boxcar application for Mac OS X. Sweet!</p>
<div id="crp_related"><h3>You might also want to read these other posts...</h3><ul><li><a href="http://blog.fosketts.net/2011/05/27/pile-interesting-links-27-2011/"  rel="bookmark" class="crp_title">Back From the Pile: Interesting Links, May 27, 2011</a></li><li><a href="http://blog.fosketts.net/2010/10/06/timthumb-php-tantan-wordpress-s3-plugin/"  rel="bookmark" class="crp_title">How To Make TimThumb Play Nicely With TanTan&#8217;s WordPress S3 Plugin</a></li><li><a href="http://blog.fosketts.net/2011/10/15/siri-awol-upgrading-iphone-4s/"  rel="bookmark" class="crp_title">Siri is AWOL After Upgrading to the iPhone 4S</a></li><li><a href="http://blog.fosketts.net/2010/04/18/how-to-share-google-calendar-caldav/"  rel="bookmark" class="crp_title">How To Keep Your Family Activities In Sync With A Shared Google Calendar</a></li><li><a href="http://blog.fosketts.net/2009/07/06/install-google-gears-safari-4/"  rel="bookmark" class="crp_title">How To Install Google Gears in Safari 4</a></li></ul></div><script src="http://feeds.feedburner.com/~s/sfoskett?i=http://blog.fosketts.net/2011/05/24/boxcar-growl-plugin-system-preferences-pane/" type="text/javascript" charset="utf-8"></script><hr />
<p><small>© sfoskett for <a href="http://blog.fosketts.net">Stephen Foskett, Pack Rat</a>, 2011. |
<a href="http://blog.fosketts.net/2011/05/24/boxcar-growl-plugin-system-preferences-pane/">Boxcar Growl Plugin Not Working?</a>
<br/>
This post was categorized as <a href="http://blog.fosketts.net/category/everything/apple/" title="View all posts in Apple" rel="category tag">Apple</a>. Each of my categories has its own feed if you'd like to filter out or focus on posts like this.<br/>
</small></p>]]></content:encoded>
			<wfw:commentRss>http://blog.fosketts.net/2011/05/24/boxcar-growl-plugin-system-preferences-pane/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

