• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Migrating Data With ZFS Send and Receive

Migrating Data With ZFS Send and Receive

August 18, 2016 By Stephen 5 Comments

I’m a huge fan of rsync as a migration tool, but FreeNAS is ZFS-centric so I decided to take a shot at using some of the native tools to move data. I’m not sold on it for daily use, but ZFS Send and Receive is awfully useful for “internal” maintenance tasks like moving datasets and rebuilding pools. Since this kind of migration isn’t well-documented online, I figured I would make my notes public here.

Shipping by Constance Abram

ZFS Send and Receive

ZFS is a volume manager, a file system, and a set of data management tools all bundled together.1 One of the interesting features of ZFS is the ability to backup and restore data natively using the “send” and “receive” (or “recv”) commands.

Like rsync, tar, and other UNIX backup utilities, ZFS Send and Receive is a two-part process:

  1. “zfs send” reads snapshot data and outputs a datastream to stdout
  2. “zfs recv” reads a stream and restores data to a snapshot

You’ll note that I said “snapshot” there: In the interest of consistency, ZFS Send and Receive mainly operates on ZFS snapshots. This is a huge advantage over other utilities since it gives a point-in-time copy rather than reading some data at one time and other at a different time.

It’s also important to see that ZFS Send and Receive respectively output or input a stream of data. You can redirect this to a file or send it over an ssh connection, but you can also use it locally with a simple pipe. This means that ZFS Send and Receive can be used to move data internally to a system as well as for backup or remote replication.

Copying a ZFS Dataset

Let’s say you have a dataset called gang/scooby and wanted to create a perfect copy, gang/scrappy. ZFS Send and Receive can do this quickly and easily, guaranteeing a perfect and consistent copy! Here’s how:

  1. Try to quiet gang/scooby (turn off Samba, stop applications, etc)
  2. Make a snapshot: zfs snap gang/[email protected]
  3. Send that snapshot to overwrite gang/scrappy: zfs send -R gang/[email protected] | zfs recv -F gang/scrappy2
  4. Check out the results: zfs list -t snapshot -r gang
  5. Promote gang/scrappy’s new snapshot to become the dataset’s data: zfs rollback gang/[email protected]

Note that this will totally blow away any existing data in the target dataset. If this is not what you want to do, you have been warned! But no one likes Scrappy Doo anyway…

Wait For It…

You can also redirect ZFS Send to a file and tell ZFS Receive to read from a file. This is handy when you need to rebuild a pool as well as for backup and replication.

In this example, we will send gang/scooby to a file and then restore that file later.

  1. Try to quiet gang/scooby
  2. Make a snapshot: zfs snap gang/[email protected]
  3. Send that snapshot to a file: zfs send -R gang/[email protected] > gzip /tmp/ghost.gz
  4. Do what you need to gang/scooby
  5. Restore the data to gang/scooby: gzcat /tmp/ghost.gz | zfs recv -F gang/scooby
  6. Promote gang/scooby’s new snapshot to become the dataset’s data: zfs rollback gang/[email protected]

Incremental Improvements

You can also send incremental updates of ZFS datasets. This is handy for more-active data that can’t be offline for the hours it might take to move data from place to place.

In this example, we’ll send an initial snapshot using ZFS Send and Receive, followed by an incremental snapshot of the data that changed.

  1. Make a snapshot: zfs snap gang/[email protected]
  2. Send that snapshot to overwrite gang/scrappy: zfs send -R gang/[email protected] | zfs recv -F gang/scrappy
  3. Check out the results: zfs list -t snapshot -r gang
  4. Try to quiet gang/scooby
  5. Make another snapshot: zfs snap gang/[email protected]
  6. Send the incremental snapshot to gang/scrappy: zfs send -i -R initial gang/[email protected] | zfs recv gang/scrappy
  7. Note that both gang/scooby and gang/scrappy now have both @initial and @incremental snapshots: zfs list -t snapshot -r gang
  8. Promote the latest gang/scrappy snapshot to become the dataset’s data: zfs rollback gang/[email protected]

You could script this to make sure it happens as quickly as possible. Of course it’s best not to try to do something like this on an active dataset, but it could still be useful on an active server: You only need to truly quiet the source application before the incremental snapshot is taken, and this will hopefully be much quicker than the initial data transfer.

Stephen’s Stance

I like ZFS Send and Receive, but I’m not totally sold on it. I’ve used rsync for decades, so I’m not giving it up anytime soon. Even so, I can see the value of ZFS Send and Receive for local migration and data management tasks as well as the backup and replication tasks that are typically talked about.

Update: Fixed a few typos. You need to add -R to zfs send! Thanks!

Here’s my entire FreeNAS series so far:

  1. Hello FreeNAS! Goodbye Drobo and Iomega…
  2. My FreeNAS Build: Supermicro X10SL7, Intel Haswell Xeon, ECC RAM
  3. 14 Drives For 14 Ports: A Case For FreeNAS
  4. It’s Fine To Mount Hard Drives On Their Side Or Even Upside-Down
  5. Migrating Data With ZFS Send and Receive
  6. FreeNAS First Impressions

Image credit: “Shipping” by Constance Abram, CC-by-NC-ND

  1. Ok, it’s not technically a volume manager, but neither is it technically a file system! ↩
  2. If you need to have root privileges, use: sudo zfs send -R gang/[email protected] | ( sudo zfs recv -F gang/scrappy ) ↩

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • How To Install ZeroTier on TrueNAS 12
  • How To Connect Everything From Everywhere with ZeroTier
  • Powering Rabbits: The Mean Well LRS-350-12 Power Supply
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

Filed Under: Enterprise storage, Terabyte home Tagged With: data movement, data replication, FreeNAS, replication, rsync, ZFS

Primary Sidebar

An SSD is just a very small storage array

Stephen Foskett

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    Storage Changes in VMware vSphere 5.1

    September 4, 2012

    The End of Unlimited Data – Part 1: The Buffet

    June 2, 2010

    Free as in Coffee – Thoughts on the State of OpenStack

    May 2, 2016

    New England Takes On Seattle To Determine Who’s Number 2 … In Tech!

    January 19, 2015

    Co-Processors, GPGPU, and Heterogeneous Computing

    June 26, 2017

    Instapaper for iPad and iPhone Enhances My Web World

    June 1, 2010

    Virtual Machine Mobility: Of What, and to Where and in What State?

    January 16, 2012

    10 Mysteries The Lost Finale Definitively Settled

    May 25, 2010

    Faster Ethernet Gets Weird

    June 19, 2015

    Datacenter History: Through the Ages in Lego

    October 22, 2013

    Footer

    Legalese

    Copyright © 2022 · Log in