How to Sync Directories with Rsync: A Beginner’s Guide

Synchronizing directories is a common task for anyone who needs to keep files consistent across multiple locations, such as backups, development environments, or remote servers. rsync is a powerful and efficient command-line utility that copies and synchronizes files and directories between two locations, either locally or over a network. This guide will walk you through syncing directories with rsync step-by-step, even if you’re new to the tool.

What You’ll Need

• A computer running a Unix-like operating system (Linux, macOS, or Windows with WSL or Cygwin)
• Basic familiarity with the command line or terminal
• rsync installed (usually pre-installed on Linux/macOS; can be installed on Windows)
• Access permissions to the directories you want to sync
• For remote syncing, SSH access to the remote machine

Step-by-Step

1. Open your terminal or command prompt. On Linux or macOS, use Terminal. On Windows, open PowerShell or Command Prompt with WSL or Cygwin installed.
2. Check if rsync is installed. Run rsync --version. If it returns a version number, you’re set. If not, install it using your package manager (e.g., sudo apt install rsync on Ubuntu, brew install rsync on macOS).
3. Understand the basic syntax. The general form is:

   rsync [options] source destination

   The source and destination can be local paths or remote paths in the form user@host:/path.

4. Start with a dry run. Before making any changes, it’s safest to simulate the sync and see what will happen:

   rsync -avun source/ destination/

   Here, -a means archive mode (preserves permissions, timestamps, symbolic links), -v is verbose, -u skips files that are newer on the destination, and -n means dry run.

5. Run your first actual sync locally. For example, to sync the contents of ~/Documents/project/ to ~/Backup/project/, use:

   rsync -av --delete ~/Documents/project/ ~/Backup/project/

   The trailing slash on the source folder copies the contents inside project, not the directory itself. The --delete flag removes files in the destination that no longer exist in the source, keeping both exactly in sync.

6. Sync files to a remote server. If you want to sync to a remote machine, use:

   rsync -av --delete ~/Documents/project/ user@remotehost:/path/to/destination/

   This requires SSH access to remotehost. You may be prompted for your password unless you use SSH keys.

7. Sync files from a remote server to local. To pull files from remote to local, reverse the order:

   rsync -av --delete user@remotehost:/path/to/source/ ~/Documents/project/

8. Use SSH keys for passwordless login (optional but recommended). Generate SSH keys with ssh-keygen and copy the public key to the remote server with ssh-copy-id user@remotehost. This speeds up syncing and automates scripts.
9. Customize your command with additional options. Common options include:
   • -z: Compress data during transfer (good for slow connections)
   • -P: Show progress and allow resuming interrupted transfers
   • --exclude='pattern': Skip files matching a pattern

   Example:

   rsync -avzP --exclude='*.tmp' ~/Documents/project/ user@remotehost:/backup/project/

10. Automate syncing with scripts or cron jobs. Once comfortable, you can schedule rsync commands to run automatically at intervals using cron (Linux/macOS) or Task Scheduler (Windows).

Common Mistakes

• Forgetting the trailing slash on the source directory. This changes behavior: with a slash, the contents are copied; without it, the directory itself is copied inside the destination. For example, rsync -av source/ destination/ copies contents, but rsync -av source destination/ copies the entire source folder inside destination.
• Not using --delete when you want exact mirrors. Without --delete, files deleted in the source remain in the destination, causing clutter and confusion.
• Syncing to the wrong directory. Typos in paths can cause files to go to unexpected locations or overwrite important data. Always double-check your paths.
• Running as root unnecessarily. Avoid using sudo unless you need to sync system files. Running as root can change file ownerships unintentionally.
• Not testing with a dry run first. Skipping the -n dry run can lead to unexpected deletions or overwrites.
• Ignoring permissions and ownership. Sometimes rsync can’t preserve permissions due to user rights or different filesystems, leading to access problems.

Tips

• Use -avzP for most remote syncing tasks. This combination archives, compresses, shows progress, and allows resuming interrupted transfers.
• Exclude unnecessary files. Use --exclude to skip temporary files, caches, or other irrelevant data to speed up syncing.
• Consider using --delete-before or --delete-delay for large syncs. These options control when deletions happen during the sync process, which can improve performance or safety.
• Keep your SSH keys secure. Protect private keys with passphrases and restrict permissions to avoid unauthorized access.
• Run rsync in verbose mode -v to watch what files are being transferred and deleted. This helps understand the process and spot issues early.
• Use --checksum if you suspect file timestamps may not be reliable. This forces rsync to compare file contents, but it is slower.
• Combine rsync with cron or scheduled tasks for regular automated backups. For example, create a cron job running nightly syncs.

Troubleshooting

If your rsync command doesn’t work as expected, here are some common issues and fixes:

• Files not syncing or missing: Check your source and destination paths carefully. Use the dry run mode -n to preview changes. Look for typos or missing trailing slashes.
• Permission denied errors: Ensure you have read permissions on the source and write permissions on the destination. Avoid running as root unless necessary. For remote syncs, check SSH user privileges.
• SSH connection refused or timed out: Verify the remote server is reachable and SSH service is running. Check firewall settings and correct hostname/IP.
• Ownership or permission changes unexpected: If you don’t want to change ownership, avoid the -a flag or use --no-owner if syncing as root. Check filesystem compatibility.
• Slow transfers: Use -z to compress data. Check network speed. Exclude large unnecessary files.
• Files not deleted on destination: Confirm you used the --delete flag. Without it, rsync will not remove files.
• Interrupted transfers: Use -P to allow resuming. You can rerun the command; rsync will skip files already transferred.

Frequently Asked Questions

1. What’s the difference between rsync and cp?
rsync is designed for syncing and syncing only transfers changed files, supports remote syncing, preserves permissions, and can delete files to mirror directories. cp simply copies files without syncing or remote capabilities.

2. Can I use rsync on Windows?
Yes. Windows users can install rsync via Windows Subsystem for Linux (WSL), Cygwin, or third-party tools like DeltaCopy.

3. How do I exclude certain files or folders?
Use the --exclude='pattern' option. For example, --exclude='*.tmp' excludes all temporary files ending with .tmp.

4. Is it safe to use --delete?
Yes, but be careful. --delete removes files in the destination that do not exist in the source, so double-check your paths and run a dry run first.

5. How do I sync files securely over the internet?
Use rsync over SSH by specifying a remote path like user@host:/path. SSH encrypts the data during transfer.

6. Can rsync resume interrupted transfers?
Yes. Using the -P option (which includes --partial and --progress) allows resuming incomplete files without restarting from scratch.

By following these steps and tips, even beginners can confidently use rsync to keep directories synchronized efficiently and securely. Practice with small test directories to build familiarity before syncing important data.

Remember, rsync is highly versatile and can be tailored to many use cases, from simple backups to complex deployment workflows. Mastering it can save you time and prevent data loss in your daily file management.

Happy syncing!

Related

• Differences Between Spiders and Spider Lookalike Bugs Explained
• Bugs That Look Like Spiders Found Indoors: Identification Guide
• 15 Creepy Crawlies That Look Like Spiders but Aren’t