• Skip to content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Migrating Data With ZFS Send and Receive

Migrating Data With ZFS Send and Receive

August 18, 2016 By Stephen 5 Comments

I’m a huge fan or rsync as a migration tool, but FreeNAS is ZFS-centric so I decided to take a shot at using some of the native tools to move data. I’m not sold on it for daily use, but ZFS Send and Receive is awfully useful for “internal” maintenance tasks like moving datasets and rebuilding pools. Since this kind of migration isn’t well-documented online, I figured I would make my notes public here.

Shipping by Constance Abram

ZFS Send and Receive

ZFS is a volume manager, a file system, and a set of data management tools all bundled together.1 One of the interesting features of ZFS is the ability to backup and restore data natively using the “send” and “receive” (or “recv”) commands.

Like rsync, tar, and other UNIX backup utilities, ZFS Send and Receive is a two-part process:

  1. “zfs send” reads snapshot data and outputs a datastream to stdout
  2. “zfs recv” reads a stream and restores data to a snapshot

You’ll note that I said “snapshot” there: In the interest of consistency, ZFS Send and Receive mainly operates on ZFS snapshots. This is a huge advantage over other utilities since it gives a point-in-time copy rather than reading some data at one time and other at a different time.

It’s also important to see that ZFS Send and Receive respectively output or input a stream of data. You can redirect this to a file or send it over an ssh connection, but you can also use it locally with a simple pipe. This means that ZFS Send and Receive can be used to move data internally to a system as well as for backup or remote replication.

Copying a ZFS Dataset

Let’s say you have a dataset called gang/scooby and wanted to create a perfect copy, gang/scrappy. ZFS Send and Receive can do this quickly and easily, guaranteeing a perfect and consistent copy! Here’s how:

  1. Try to quiet gang/scooby (turn off Samba, stop applications, etc)
  2. Make a snapshot: zfs snap gang/[email protected]
  3. Send that snapshot to overwrite gang/scrappy: zfs send gang/[email protected] | zfs recv -F gang/scrappy2
  4. Check out the results: zfs list -t snapshot -r gang
  5. Promote gang/scrappy’s new snapshot to become the dataset’s data: zfs rollback gang/[email protected]

Note that this will totally blow away any existing data in the target dataset. If this is not what you want to do, you have been warned! But no one likes Scrappy Doo anyway…

Wait For It…

You can also redirect ZFS Send to a file and tell ZFS Receive to read from a file. This is handy when you need to rebuild a pool as well as for backup and replication.

In this example, we will send gang/scooby to a file and then restore that file later.

  1. Try to quiet gang/scooby
  2. Make a snapshot: zfs snap gang/[email protected]
  3. Send that snapshot to a file: zfs send gang/[email protected] > gzip /tmp/ghost.gz
  4. Do what you need to gang/scooby
  5. Restore the data to gang/scooby: gzcat /tmp/ghost.gz | zfs recv -F gang/scooby
  6. Promote gang/scooby’s new snapshot to become the dataset’s data: zfs rollback gang/[email protected]

Incremental Improvements

You can also send incremental updates of ZFS datasets. This is handy for more-active data that can’t be offline for the hours it might take to move data from place to place.

In this example, we’ll send an initial snapshot using ZFS Send and Receive, followed by an incremental snapshot of the data that changed.

  1. Make a snapshot: zfs snap gang/[email protected]
  2. Send that snapshot to overwrite gang/scrappy: zfs send gang/[email protected] | zfs recv -F gang/scrappy
  3. Check out the results: zfs list -t snapshot -r gang
  4. Try to quiet gang/scooby
  5. Make another snapshot: zfs snap gang/[email protected]
  6. Send the incremental snapshot to gang/scrappy: zfs send -i initial gang/[email protected] | zfs recv gang/scrappy
  7. Note that both gang/scooby and gang/scrappy now have both @initial and @incremental snapshots: zfs list -t snapshot -r gang
  8. Promote the latest gang/scrappy snapshot to become the dataset’s data: zfs rollback gang/[email protected]

You could script this to make sure it happens as quickly as possible. Of course it’s best not to try to do something like this on an active dataset, but it could still be useful on an active server: You only need to truly quiet the source application before the incremental snapshot is taken, and this will hopefully be much quicker than the initial data transfer.

Stephen’s Stance

I like ZFS Send and Receive, but I’m not totally sold on it. I’ve used rsync for decades, so I’m not giving it up anytime soon. Even so, I can see the value of ZFS Send and Receive for local migration and data management tasks as well as the backup and replication tasks that are typically talked about.

Here’s my entire FreeNAS series so far:

  1. Hello FreeNAS! Goodbye Drobo and Iomega…
  2. My FreeNAS Build: Supermicro X10SL7, Intel Haswell Xeon, ECC RAM
  3. 14 Drives For 14 Ports: A Case For FreeNAS
  4. It’s Fine To Mount Hard Drives On Their Side Or Even Upside-Down
  5. Migrating Data With ZFS Send and Receive
  6. FreeNAS First Impressions

Image credit: “Shipping” by Constance Abram, CC-by-NC-ND

  1. Ok, it’s not technically a volume manager, but neither is it technically a file system! ↩
  2. If you need to have root privileges, use: sudo zfs send gang/[email protected] | ( sudo zfs recv -F gang/scrappy ) ↩

You might also want to read these other posts...

  • It’s Fine To Mount Hard Drives On Their Side…
  • Add a Mirror to an Existing ZFS Drive
  • FreeNAS First Impressions
  • Hello FreeNAS! Goodbye Drobo and Iomega…
  • 14 Drives For 14 Ports: A Case For FreeNAS

Filed Under: Enterprise storage, Terabyte home Tagged With: data movement, data replication, FreeNAS, replication, rsync, ZFS

Primary Sidebar

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine.

Alan Turing

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Ranting and Raving About the 2018 iPad Pro

November 11, 2018

The 2018 iPad Pro is a Beast!

November 9, 2018

Add a Mirror to an Existing ZFS Drive

December 11, 2017

How To Remove Raw Images From Apple Photos and iCloud

July 24, 2017

Recalling An HP MediaSmart Server To Active Duty

July 21, 2017

Go Get a ProtonMail Account and Protect Your Online Life!

July 19, 2017

ZFS Is the Best Filesystem (For Now…)

July 10, 2017

Co-Processors, GPGPU, and Heterogeneous Computing

June 26, 2017

What is OCuLink?

June 22, 2017

Storage is Getting Cloudier!

June 21, 2017

Symbolic Links

  • How Reliable are SSDs?

  • Today I Am Retiring From NetApp

  • Atmosic and the Power of RF?

    Tom Hollingsworth

  • How the data science profession is growing in value and impact across the business world

    Dana Gardner

  • IPv6 in Enterprise , Should we still talk about it?

    Orhan Ergun

Featured Posts

The Fat Middle: Today’s Enterprise Storage Array

August 31, 2014

Ranting and Raving About the 2018 iPad Pro

November 11, 2018

Storage Changes in VMware vSphere 5

July 16, 2011

Microsoft’s Big Chance to Change

August 23, 2013

Scaling Storage In Conventional Arrays

November 19, 2013

My Core i7 Macintosh SE

May 25, 2017

Scaling Storage Is Hard To Do

June 4, 2013

SMB 3 is Going to be Huge, in both Scope and Impact

May 6, 2012

Infographic: Real-World Port Throughput Relative To Thunderbolt (Formerly Light Peak)

February 21, 2011

My 2012 Project: Improving Energy Efficiency

January 3, 2012

Copyright © 2019 · Log in