• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Symantec’s Thin API: The Plot Thickens

Symantec’s Thin API: The Plot Thickens

October 24, 2008 By Stephen 5 Comments

Last week, I lauded Symantec for introducing an API in Storage Foundation which will interact with the thin storage capabilities of supported arrays. Since then, I’ve learned more about this capability, and I am writing this update to share that knowledge. As I noted last week, the press release was a bit hard to follow and comprehend (and not just for me), and one of my initial assumptions about the API turned out to be wrong. I also received a few comments from interested folks pointing out some more pros and cons of this technology.

First, let’s clarify just which products and capabilities Symantec is offering here:

  • Veritas Storage Foundation version 5.0MP3 for Unix/Linux includes SmartMove and the Thin Reclamation API
  • Veritas Storage Foundation for Windows 5.0 only includes SmartMove at this point, but it will be updated to include Thin Reclamation at some point in the coming year

Although there is no real information on Symantec’s web site about all this yet, Symantec’s director of Storage Management and High Availability, Sean Derrington, assures me that their software is available now. Although no compatible arrays are in end-user hands, 3PAR will update their T-Class firmware to support the API shortly, and HDS and HP are on the way as well.

Thin Aware Software

Next, contrary to what I inferred from the announcement, there is no native thin provisioning capability in the file system or volume manager. So the first item in my list is right out. However, the volume manager is now “thin aware”, which means that it will communicate up to the file system and down to the array to coordinate more effective use of space.

When the volume manager is used with Veritas File System (VxFS) on UNIX or NTFS on Windows Server 2003 or 2008, it will automatically keep track of deleted files and will pass this information down the stack to the array. This is a major piece of functionality to add, especially to NTFS, “hole punching” (like NetApp) to maximize thin provisioning.

The Storage Foundation tools have also been updated to properly report on thin provisioned volumes. For example, the following screenshot shows three disk devices where encl1 supports thin reclamation and encl0 does not.

# vxdisk list
DEVICE        TYPE   DISK          GROUP         STATUS
encl0_0       auto   encl0_0       mydg online   thin
encl1_0       auto   encl1_0       mydg online   thinrclm
encl1_1       auto   ecnl1_1       mydg online   thinrclm

Thin Reclamation API

The Veritas Thin Reclamation API allows the Storage Foundation volume manager and file systems to communicate with thin-capable arrays when data is deleted on thin-ified LUNs, maintaining their thin-ness as you go. When a file is deleted, the file system will communicate to the volume manager that that space is no longer needed. When the server administrator runs the “vxdisk reclaim” or “fsadm —R” commands, the volume manager will communicate this information to the array (using SCSI commands) that any vacated disk blocks can now be reclaimed. Symantec expects folks to set up a cron job to reclaim space, or perhaps just run it when they see the need.

This is brilliant stuff, and ought to make thin provisioning shine in terms of array utilization. In an environment of thin-enabled Veritas volumes and supported storage arrays, the amount of space used on an array will be awfully close to the amount of space used in the file systems. This is a massive win – a capacity gain of on the order of 50%-70% in an average environment!

For more on this topic, see my recent post on storage utilization

If the storage array fully supports Symantec’s API, the tools will also report physically allocated storage behind thin and thin_reclaim devices.

# vxdisk —o thin list
DANAME        DISK SIZE(Mb)        PHYS_ALLOC(Mb)       DISK GROUP TYPE
encl0_0       2000                 50 mydg              thin
encl1_0       200                  50 mydg              thinrclm
encl1_1       500                  500 mydg             thinrclm

SmartMove

SmartMove is Symantec’s new capability for online migration from “thick” to thin LUNs. It is included in Storage Foundation for Unix/Linux and Windows and works with any thin storage array, not just those that support the API. This is basically a tweak to the old storage migration support we have all known and relied on in Veritas Storage Foundation for over a decade, except that it’s smart enough to not request blocks that it won’t use. One could theoretically “SmartMove” a volume regularly to reclaim space without using the API at all, but those commands are sure a lot simpler.

Note that SmartMove speeds up migration too, even for thick volumes! When you use a SmartMove-enabled version of Storage Foundation to move a volume, it will only send the blocks that have changed over the wire. This reminds me a little of VMware’s new I/O deduplication capability talked about at VMworld, but it’s focused only on migrations, not other I/O situations.

For more on this topic, see my recent post on VMware vStorage

The Plot Thickens

So I was wrong about one item, but the other two remain true. Is Symantec’s new capability a winner? I give it a silver medal – it’s good stuff, but some issues remain.

  1. My primary concern remains – thin provisioning does nothing to address the lack of storage management that is so prevalent. It enables greater utilization of capacity, but does nothing to control how that capacity is used. This isn’t a beef with Symantec’s Veritas Storage Foundation or 3PAR or HDS or EMC or anyone in the thin industry, really. Instead, it is a wake-up call to all of the storage organizations out there who have filesystems full of uncontrolled junk!
  2. My second concern is the lack of capacity management. Thin provisioning is a lie, promising more capacity than is available. This might be acceptable in certain controlled circumstances like operating system or application volumes, but telling end users that they have plenty of available space is a recipe for disaster. Storage use is like air – it expands to fill all available volume. Without capacity management, your thin volumes will be “overdrawn” and your storage “account” will be bankrupt.
  3. Then there is the issue of proprietary APIs versus standards. Let me say right away that I always support standards over proprietary technology. But, at the same time, given the choice between nothing and something, I’ll take the proprietary API. Thin provisioning is a good idea with poor implementation. This API helps to make it useful in the real world, and having a market leader like Symantec behind it makes it all the more relevant. I certainly hope the entire storage industry will come up with a standard thin API, and when that happens I hope Symantec will support it. Until then, at least we have something.

I will be writing more about thin provisioning in the coming weeks. Until then, I continue to applaud Symantec, 3PAR, HDS, and HP for their work in making this technology somewhat more practical. Now how about VMware, Microsoft, Sun, and the Linux guys get some thin technology going, too?

See my posts on Gestalt IT for similar enterprise IT infrastructure commentary

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • What You See and What You Get When You Follow Me
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • How To Connect Everything From Everywhere with ZeroTier
  • Tortoise or Hare? Nvidia Jetson TK1

Filed Under: Enterprise storage, Virtual Storage Tagged With: 3PAR, API, data growth, data management, file system, HDS, HP, I/O deduplication, SmartMove, Storage Foundation, storage management, Symantec, thin provisioning, Thin Reclamation, utilization, Veritas Storage Foundation, VMware, volume manager

Primary Sidebar

The less the prudence with which others conduct their affairs, the greater the prudence with which we must conduct our own

Warren Buffett

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    We Live in the Future: Robotic Cat Litter Boxes!

    May 8, 2010

    Ranting and Raving About the 2018 iPad Pro

    November 11, 2018

    The Terrifying True Story Of Virtual Machine Mobility

    December 22, 2011

    The Ideal pfSense Platform: Netgate RCC-VE 2440

    September 21, 2015

    Nimble Storage Rolls Out an All-Flash Array

    February 24, 2016

    Why You Should Never Again Utter The Word, “CIFS”

    February 16, 2012

    The Rack Endgame: Converged Infrastructure and Disaggregation

    September 19, 2014

    It’s Time To Move Beyond Passwords (Especially On Web Sites)

    January 8, 2016

    Debit or Credit? Always Choose Credit!

    December 19, 2013

    How Will Cisco Recover From The Consumer Strategy Blunder?

    January 2, 2013

    Footer

    Legalese

    Copyright © 2022 · Log in