Symantec announced that their widely-used Veritas Storage Foundation software (more commonly known as Veritas Volume Manager and filesystem) will now include thin provisioning, migration, and a new API which allows it to directly communicate with thin-capable storage arrays. This is a smart step, certainly, and ought to be useful, but storage managers should still try to manage storage use aggressively.
Update: More details have come out about Symantec’s new capability. Read my follow-up post on Symantec’s Thin API for clarification!
Here’s the gist (as near as I can tell). Veritas Storage Foundation will include the following capabilities:
- Native thin provisioning which will cause Veritas volume manager to allocate space in the Veritas filesystem only when data is written
- A thin provisioning API which allows the volume manager to communicate with certain storage arrays (3PAR is highlighted, along with HDS and HP) to allocate and reclaim space as it is used in the filesystem
- A thin migration API which allows the volume manager to tell these storage arrays to migrate existing volumes to thin-provisioned space
This is great stuff, and ought to allow folks to make better use of thin provisioning. In fact, it nicely addresses many of my concerns about thin provisioning, including un-provisioning. It’s especially nice to see that Symantec already has interested partners, and that one system (3PAR) is up and running already. Way to go!
Actually, it is disheartening that Symantec chose to define their own API, rather than to collaborate with the industry and the T10 standards body in defining a standard interface.
You know, you’re right about this. A standard API would have been much better, now that I think about it.
The API is in active negotiation in T10, and I’m sure that Symantec is aware of it.
The approach Symantec took is actually rather disruptive and a blatant mis-use of an existing SCSI command – used against a non-aware thin provisioning implementation, it could actually cause needless allocation of storage…having the direct opposite effect as is being marketed.
I think I heard that T10 actually rejected the very approach Symantec is using.
In the rush to be first, corners are being cut and this is never good for the industry.
I’d say more, but I don’t think I’m supposed to know what they did…but you might want to do some digging on your own.
So here we have the usual discussion about doing something today or waiting for a standard to emerge at some unknown point in the future. Standards, like SMI-S have not necessarily lived up to expectations – should we repeat the process for thin provisioning and expect the results to be better? A recent discussion on SMI-S might be worth a look:
http://dcsblog.burtongroup.com/data_center_strategies/2008/10/has-snias-smi-s.html
The way I understand it, the Symantec implementation requires an “aware” thin provisioning storage target in order to avoid the problems Anarchist raises. FWIW, I’m not sure how awareness is specified or communicated with Symantec’s API.
In the desire to be perfect and all encompassing, nobody benefits.
That said, Anarchist’s employer EMC has been very loyal to standards efforts in storage management and I think they deserve credit for their commitment. Unfortunately, herding storage cats has proven to be less than wonderfully effective.
Marc Farley here. The previous comment was mine. I expected the authentication/registration system to recognize me as I had signed in using my email address at 3PAR. Perhaps you can address this Stephen? Its more cumbersome commenting on your site than Barry Whyte’s – and THAT is saying something.
Actually, it is disheartening that Symantec chose to define their own API, rather than to collaborate with the industry and the T10 standards body in defining a standard interface.
You know, you’re right about this. A standard API would have been much better, now that I think about it.
The API is in active negotiation in T10, and I’m sure that Symantec is aware of it.
The approach Symantec took is actually rather disruptive and a blatant mis-use of an existing SCSI command – used against a non-aware thin provisioning implementation, it could actually cause needless allocation of storage…having the direct opposite effect as is being marketed.
I think I heard that T10 actually rejected the very approach Symantec is using.
In the rush to be first, corners are being cut and this is never good for the industry.
I’d say more, but I don’t think I’m supposed to know what they did…but you might want to do some digging on your own.
Sorry about the difficulties commenting – I never have to deal with it since I am logged in all the time. But I’ll try to improve it.
So here we have the usual discussion about doing something today or waiting for a standard to emerge at some unknown point in the future. Standards, like SMI-S have not necessarily lived up to expectations – should we repeat the process for thin provisioning and expect the results to be better? A recent discussion on SMI-S might be worth a look:
http://dcsblog.burtongroup.com/data_center_strategies/2008/10/has-snias-smi-s.html
The way I understand it, the Symantec implementation requires an “aware” thin provisioning storage target in order to avoid the problems Anarchist raises. FWIW, I’m not sure how awareness is specified or communicated with Symantec’s API.
In the desire to be perfect and all encompassing, nobody benefits.
That said, Anarchist’s employer EMC has been very loyal to standards efforts in storage management and I think they deserve credit for their commitment. Unfortunately, herding storage cats has proven to be less than wonderfully effective.
Marc Farley here. The previous comment was mine. I expected the authentication/registration system to recognize me as I had signed in using my email address at 3PAR. Perhaps you can address this Stephen? Its more cumbersome commenting on your site than Barry Whyte’s – and THAT is saying something.
Sorry about the difficulties commenting – I never have to deal with it since I am logged in all the time. But I’ll try to improve it.
We do have to be realistic about the standards process. Getting standards that address customer requirements in a timely manner can be difficult. As you know Symantec actively participates on numerous standards boards and continues to be a key contributor to SMI-S. And, in doing so has moved this forward as a way to monitor and report on storage environments (to varying degrees depending on the hardware providers commitment to SMI-S). As Marc points out, this has been a long process spanning Veritas co-authoring Bluefin circa 2000 to SMI-S 1.1 today. (SMI-S 1.2 is still subject to change.) And, it still doesn
Sean,
Thanks so much for the comment. I certainly appreciate this new API, and think it will do good for customers of Symantec, 3PAR, HDS, and HP. I hope other vendors also decide to implement it, and that it leads to a standard API in the future. And I very much hope that TSA is wrong about blowback from the API on non-aware systems!
Stephen
We do have to be realistic about the standards process. Getting standards that address customer requirements in a timely manner can be difficult. As you know Symantec actively participates on numerous standards boards and continues to be a key contributor to SMI-S. And, in doing so has moved this forward as a way to monitor and report on storage environments (to varying degrees depending on the hardware providers commitment to SMI-S). As Marc points out, this has been a long process spanning Veritas co-authoring Bluefin circa 2000 to SMI-S 1.1 today. (SMI-S 1.2 is still subject to change.) And, it still doesn’t fully encompass what customers need to effectively manage storage resources in their environment…which is why Symantec works with the major storage vendors and their unique APIs (including EMC DMX and CX) to augment SMI-S and provide richer detail. Unfortunately solely going through the standards process isn’t always the best way to meet customer needs in a timely fashion. There are many deeply entrenched vendor interests (some more valid than others) that play a big role in what the standards end up providing.
Enough about SMI-S; thin provisioning is a relatively new solution that IT organizations are trying to get their hands around. The capability in hardware is there (albeit with implementation details varying by vendor), but to truly maximize the TP value there must be server side awareness (file system and volume manager). Working with our partners, we’ve engineered a way for organizations to automatically, and non-disruptively, reclaim storage capacity. We’ve been very open with hardware providers (including EMC) about the Veritas Thin Reclamation API, and this has moved solutions for customers forward on a number of fronts (3Par being one case in point).
From a technical perspective, the concerns expressed by storage anarchist have been thoroughly considered and are handled by the Veritas Thin Reclamation API and Veritas Storage Foundation. Storage Foundation is specifically designed to deal with hardware specific functionality and the thin reclamation functionality can only be triggered to a storage controller that is known to comply with the API. There’s absolutely no risk of spill-over into controllers that do not support the Veritas Thin Reclamation API.
In considering vendor API collaboration versus a dedicated T10 approach, there’s a criterion of time to value for customers to consider. Organizations want to reduce storage costs immediately, and given the current business environment, they simply do not have the option to wait for draft T10 standards to be determined, ratified, and eventually implemented. That’s certainly a longer term goal, but realistically, for the next 18-24 months this is a way organizations can reclaim wasted storage that otherwise would be orphaned in a TP environment.
For those that are following the work of T10 closely, I would note that the latest proposal from EMC at T10 is very strongly inspired by the Veritas Thin Reclamation API…
Sean,
Thanks so much for the comment. I certainly appreciate this new API, and think it will do good for customers of Symantec, 3PAR, HDS, and HP. I hope other vendors also decide to implement it, and that it leads to a standard API in the future. And I very much hope that TSA is wrong about blowback from the API on non-aware systems!
Stephen
Hi everyone, I'm new out here on this forum, but I thought I'd pick up on the conversation. There are some things to clear up here.
First, T10 is the INCITS governing body over SAM (SCSI Architectural Model). The SAM includes SCSI interconnects (Serial, FC, etc), SCSI Transport (SAS, FCP, etc), SCSI Primary commands (shared command set) and Device Specific Commands (SCSI block commands, Multi-media commands, etc). AFAIK, these are commands, not APIs. So, unless I'm missing something, the API doesn't fall (or shouldn't fall) into T10. The implementation of the API *could* fall into T10, if the implementation requires an extension to either the SCSI primary or device specific commands. Otherwise, no. It's not what T10 does and is out of their scope.
Thus, any standard API should fall under SNIA's SMI-S initiative. In fact, one could make the case that thin provisioning should fall under the block service profile in SMI-S v1.2 (which BTW, cannot be altered unless INCITS doesn't ratify). SMI-S v1.3 is the current working version.
But these aren't the real issue. The real issue is whether or not Symantec should have put this API in SMI-S. (Remember it was Roger Reich from Symantec who made the initial proposal to SNIA to take on a storage management initiative, which eventually became Bluefin). IMO, I don't think they should. SMI-S, for all the good it has done (created the idea of profiles for CIM, led us to a web-services management model) has thus far, failed to meet it's original objective (storage management interoperability). Can we name one console vendor (like Symantec) that uses SMI-S to manage a heterogeneous storage environment? How many providers out there instrument ALL of the manageability on the system they were written for? Better still — how many customers rely on SMI-S to manage their environment.
Yes, standards take time (believe me, I know). But how long must we wait? SMI-S was started over 8 years ago. CIM is over 11 years old. It's high time SMI-S and CIM proved the interoperability or moved on to better territory. In fact, one can make the case that “higher causes” aren't far away. Rather than defining every intricate detail within CIM and SMI-S, why not move up the management stack and define APIs at a higher level? Rather than managing the tachometer of the fan or the RAID level of a storage pool, why not create an API that is simple and service oriented? (i.e. CreateLUN (spaceRequired, redundancyLevel) ) Let the underlying implementation remain the job of the storage vendors.
Hi everyone, I’m new out here on this forum, but I thought I’d pick up on the conversation. There are some things to clear up here.
First, T10 is the INCITS governing body over SAM (SCSI Architectural Model). The SAM includes SCSI interconnects (Serial, FC, etc), SCSI Transport (SAS, FCP, etc), SCSI Primary commands (shared command set) and Device Specific Commands (SCSI block commands, Multi-media commands, etc). AFAIK, these are commands, not APIs. So, unless I’m missing something, the API doesn’t fall (or shouldn’t fall) into T10. The implementation of the API *could* fall into T10, if the implementation requires an extension to either the SCSI primary or device specific commands. Otherwise, no. It’s not what T10 does and is out of their scope.
Thus, any standard API should fall under SNIA’s SMI-S initiative. In fact, one could make the case that thin provisioning should fall under the block service profile in SMI-S v1.2 (which BTW, cannot be altered unless INCITS doesn’t ratify). SMI-S v1.3 is the current working version.
But these aren’t the real issue. The real issue is whether or not Symantec should have put this API in SMI-S. (Remember it was Roger Reich from Symantec who made the initial proposal to SNIA to take on a storage management initiative, which eventually became Bluefin). IMO, I don’t think they should. SMI-S, for all the good it has done (created the idea of profiles for CIM, led us to a web-services management model) has thus far, failed to meet it’s original objective (storage management interoperability). Can we name one console vendor (like Symantec) that uses SMI-S to manage a heterogeneous storage environment? How many providers out there instrument ALL of the manageability on the system they were written for? Better still — how many customers rely on SMI-S to manage their environment.
Yes, standards take time (believe me, I know). But how long must we wait? SMI-S was started over 8 years ago. CIM is over 11 years old. It’s high time SMI-S and CIM proved the interoperability or moved on to better territory. In fact, one can make the case that “higher causes” aren’t far away. Rather than defining every intricate detail within CIM and SMI-S, why not move up the management stack and define APIs at a higher level? Rather than managing the tachometer of the fan or the RAID level of a storage pool, why not create an API that is simple and service oriented? (i.e. CreateLUN (spaceRequired, redundancyLevel) ) Let the underlying implementation remain the job of the storage vendors.
Easy option to get useful information as well as share good stuff with good ideas and concepts.