It's hard to take a product called "Robocopy" seriously!
Anyone doing much storage work on Microsoft Windows machines is familiar with Robocopy. It’s the best tool to move data between NTFS filesystems, since Robocopy maintains permissions and file attributes. It also tolerates dropped connections, resuming where it left off, and can throttle operations over slow networks.
But Robocopy was never very quick, especially when dealing with large data sets. It was single-threaded, hurting performance on high-latency networks, and startup was painfully slow on deep directory structures.
Robocopy XXVII
Windows Vista, Windows Server 2008, and later versions include a new version of Robocopy with performance tweaks designed to overcome these limitations. Right-click on the executable in Windows\System32 and make sure you are using version XP027, 5.1.10.1027.
Update: Oops! Only the “6.1″ versions of Microsoft Windows (Windows 7 and Windows Server 2008 R2) include multi-threaded robocopy!
The big deal here is multi-threading:
- The application can now run in multi-threaded mode using the /MT option. This defaults to 8 threads, but users can specify up to 128 if desired. For example, the following command would use 16 threads:
robocopy c:\ d:\ /MT:16
- Initial directory enumeration is also multi-threaded, so deep directory structures are examined much more quickly.
- XP027 also added the /EFSRAW parameter, allowing one to copy files from EFS using RAW mode. However neither this nor /IPG (inter-packet gap) can be combined with the multi-threaded option mentioned above.
Dead or alive, you’re coming with me!
I tried out the new /MT option on a Core 2 Duo laptop and was surprised by the dramatic improvement in copy performance. Copying my entire “Program Files” directory took well over a minute without multi-threading, but simply specifying “/MT” at the end of the command reduced a second copy to a different directory to about 20 seconds. Using “/MT:32″ was blazing fast – easily less than 15 seconds. I repeated the first single-threaded test again and watched it dawdle along, taking over a minute again to finish.
Microsoft suggests that multi-threaded Robocopy helps with network throughput as well. I tried a series of copies between two Windows 7 machines over Wi-Fi, to simulate a slow network. Although single-threaded Robocopy was able to saturate the network with large files, it really slowed down (thanks to latency) once it hits a patch of smaller files. Multi-threaded Robocopy was more capable of maintaining high throughput once smaller files were encountered, with a 32-thread test keeping the link at maximum pretty much the entire time. Again, a noticeable improvement.
Your Move, Creep
If you’re copying lots of Windows data on a machine running Windows 7 Vista or newer, I can confidently say that the /MT switch will speed things up dramatically. Whether you should stick with the default 8 threads or up it to 16 or 32 depends on the capabilities of your CPU, but it’s worth a try. One more tip: Use the /LOG switch or pipe the output to NULL to speed up copying even more. Displaying all that text delays the whole process!
