• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / Storage Arrays Do A Few Things Very Well

Storage Arrays Do A Few Things Very Well

May 22, 2012 By Stephen Leave a Comment

Storage arrays are big, expensive, and difficult to manage. Plus, concentrating storage in a single device puts everything at risk if there is an outage. So why buy a storage array at all? Arrays do a few things very well, and this often makes up for the difference, on balance.

Storage arrays excel at acceleration, movement, and sharing of data

Storage Arrays Accelerate Performance

One of the most important advancements in data storage in the 1990’s was the application of advanced caching algorithms to enterprise storage arrays. DRAM cache (later augmented with flash) makes a massive difference in responsiveness. And this is especially important when it comes to shared, networked storage devices.

Old school! Data General's CLARiiON did caching right!

One of the first arrays to really “get it right” in my own experience was the Data General CLARiiON. Although comparatively small in size, price, and power, the little CLARiiON really stood up and performed when it was configured correctly. I started using these little wonders while working at Texaco’s Star Enterprise subsidiary in the late 1990’s and admired their ability to keep up with our heavy workloads even though they only had a few megabytes of cache.

The fundamental “cool thing” that these arrays do is cache frequently accessed data in high-performance memory rather than going back to the disks every time. It is fairly straightforward to implement a read cache, though intelligently pre-filling it is substantially more difficult. Enterprise storage devices also implemented a “write-back cache”, meaning they acknowledged incoming writes from clients as soon as data was in cache, rather than waiting for it to be written to disk. The combination of read- and write-cache technology made up for the comparatively slow random I/O capabilities of hard disk drives.

See also The Four Horsemen of Storage System Performance: Never Enough Cache

In the 2000’s, “wide-striping” concepts really took hold. Where RAID traditionally only used a few disks in a set, modern arrays from 3PAR, XIV, and pretty much every other modern array spreads data across every spindle it can. This accelerates I/O and can reduce the amount of time required to rebuild a failed data protection set. It also increases the risk of data loss to some extent, but this is usually made up for by the advanced capabilities of the array itself.

Lately, storage arrays have implemented automated sub-LUN tiering, which places active data on high-performance flash or disk. This also does quite a bit to accelerate performance and is, in many ways, similar to caching approaches pioneered two decades earlier. Again, most modern arrays now offer some form of storage tiering, though implementations vary greatly in detail and effectiveness.

Offloaded Data Movement

I bought my NetApp Filer for its amazing snapshot skills, but "50GB of capacity" is cool too!

The “killer app” of the NetApp filer I purchased in 1996 was its ability to snapshot data throughout the day. Similarly, the winning feature of the EMC Symmetrix I purchased in 1997 was TimeFinder, which allowed the array to make perfect copies of entire LUNs on command. We later purchased a second Symmetrix and implemented SRDF, a remote equivalent.

Offloading data movement and copying operations to the storage array was perhaps the greatest selling feature of high-end enterprise devices. This feature really “separated the men from the boys” since it proved extremely difficult for buyers to manage and implement. It also took a burly, “manly” array to move data around without impacting primary access.

Today, nearly every storage array offers some form of automated data movement. From snapshots to mirrors to replication, offloaded data movement remains a key selling point for shared storage arrays.

VMware’s VAAI has many compelling benefits, but most users talk about offloaded data copying first and foremost. Similarly, one of the most exciting features of Microsoft Windows Server 2012 is ODX, which allows the operating system (as well as Hyper-V) to offload data movement to a compatible array.

Shared Storage Made Real

The third reason that many people buy a storage array is the simple ability to share storage between multiple servers. NAS arrays obviously excel at sharing with many clients, but many SAN storage arrays are used in this method as well. Some systems even allow multi-protocol access to the same data.

File-based protocols like NFS and SMB were designed to allow many clients to access a single pool of storage. This was initially intended only for client access, but NFS has seen massive uptake for servers. This is especially true for VMware storage, where NFS has become the second-most popular method of accessing shared storage. Microsoft is responding by promoting SMB for applications and Hyper-V as part of the launch of Windows Server 2012.

Today, most clustered applications use shared SCSI LUNs over Fibre Channel, iSCSI, or SAS. Most advanced features in VMware vSphere require shared storage, and this has done much to promote adoption of networked storage arrays in smaller and midsize businesses. SCSI is perhaps not the ideal protocol for shared storage access, but as long as “persistent reservations” are supported, it will do.

Stephen’s Stance

The most common objection to shared storage is a concern about “all my eggs in one basket”. While this is a very real consideration, most companies eventually accept the risk in return for better performance, offloaded data movement, or sharing of storage resources.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • How To Connect Everything From Everywhere with ZeroTier
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Introducing Rabbit: I Bought a Cloud!
  • Running Rabbits: More About My Cloud NUCs

Filed Under: Computer History, Enterprise storage, Gestalt IT, Virtual Storage Tagged With: caching, CLARiiON, Data General, NetApp, NFS, ODX, shared storage, snapshot, SRDF, Symmetrix, Texaco, tiered storage, TimeFinder, VAAI, wide striping, Windows Server 2012

Primary Sidebar

For the past 33 years, I have looked in the mirror every morning and asked myself: “If today were the last day of my life, would I want to do what I am about to do today?” And whenever the answer has been “No” for too many days in a row, I know I need to change something.

Steve Jobs

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    Go Get a ProtonMail Account and Protect Your Online Life!

    July 19, 2017

    The I/O Blender Part 1: Ye Olde Storage I/O Path

    May 23, 2012

    Storage Changes in VMware vSphere 5

    July 16, 2011

    EMC Redefine Possible (TL;DR Edition)

    July 9, 2014

    Regarding My Symbolic Links and Good Reads

    April 16, 2015

    Begun, the Patent Wars Have

    July 9, 2012

    New England Takes On Seattle To Determine Who’s Number 2 … In Tech!

    January 19, 2015

    GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

    April 6, 2019

    Storage Changes in VMware vSphere 5.1

    September 4, 2012

    Free as in Coffee – Thoughts on the State of OpenStack

    May 2, 2016

    Footer

    Legalese

    Copyright © 2022 · Log in