Rsync Cheatsheet
Syncing With QNAP¶
-
Confirm Accuracy of Transfer¶
-
Initiate Sync¶
rsync --exclude 'Spotlight-V100*' -P -av rsync://alfie.local:'/root/Volumes/<folder>' . -
Create MD5 Digest¶
find -type f -exec md5sum "{}" + > checklist.chk -
Syncing With QNAP NAS¶
rsync -P -rltDvn --delete Consolidated/ rsync://rsync@monolith.guestwifi.home:"/Public/Consolidated" -r, --recursive recurse into directories -l, --links copy symlinks as symlinks -t, --times preserve modification times -D same as --devices --specials --devices preserve device files (super-user only) --specials preserve special files -v, --verbose increase verbosity -n, --dry-run perform a trial run with no changes made -q, --quiet suppress non-error messages --no-motd suppress daemon-mode MOTD (see caveat) -c, --checksum skip based on checksum, not mod-time & size -a, --archive archive mode; equals -rltDgop (no -H,-A,-X) --no-OPTION turn off an implied OPTION (e.g. --no-D) -R, --relative use relative path names --no-implied-dirs don't send implied dirs with --relative -b, --backup make backups (see --suffix & --backup-dir) --backup-dir=DIR make backups into hierarchy based in DIR --suffix=SUFFIX backup suffix (default ~ w/o --backup-dir) -u, --update skip files that are newer on the receiver --inplace update destination files in-place --append append data onto shorter files --append-verify --append w/old data in file checksum -d, --dirs transfer directories without recursing -L, --copy-links transform symlink into referent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed --safe-links ignore symlinks that point outside the tree -k, --copy-dirlinks transform symlink to dir into referent dir -K, --keep-dirlinks treat symlinked dir on receiver as dir -H, --hard-links preserve hard links -p, --perms preserve permissions -E, --executability preserve executability --chmod=CHMOD affect file and/or directory permissions -A, --acls preserve ACLs (implies -p) -X, --xattrs preserve extended attributes -o, --owner preserve owner (super-user only) -g, --group preserve group -O, --omit-dir-times omit directories from --times --super receiver attempts super-user activities --fake-super store/recover privileged attrs using xattrs -S, --sparse handle sparse files efficiently -W, --whole-file copy files whole (w/o delta-xfer algorithm) -x, --one-file-system don't cross filesystem boundaries -B, --block-size=SIZE force a fixed checksum block-size -e, --rsh=COMMAND specify the remote shell to use --rsync-path=PROGRAM specify the rsync to run on remote machine --existing skip creating new files on receiver --ignore-existing skip updating files that exist on receiver --remove-source-files sender removes synchronized files (non-dir) --del an alias for --delete-during --delete delete extraneous files from dest dirs --delete-before receiver deletes before transfer (default) --delete-during receiver deletes during xfer, not before --delete-delay find deletions during, delete after --delete-after receiver deletes after transfer, not before --delete-excluded also delete excluded files from dest dirs --ignore-errors delete even if there are I/O errors --force force deletion of dirs even if not empty --max-delete=NUM don't delete more than NUM files --max-size=SIZE don't transfer any file larger than SIZE --min-size=SIZE don't transfer any file smaller than SIZE --partial keep partially transferred files --partial-dir=DIR put a partially transferred file into DIR --delay-updates put all updated files into place at end -m, --prune-empty-dirs prune empty directory chains from file-list --numeric-ids don't map uid/gid values by user/group name --timeout=SECONDS set I/O timeout in seconds --contimeout=SECONDS set daemon connection timeout in seconds -I, --ignore-times don't skip files that match size and time --size-only skip files that match in size --modify-window=NUM compare mod-times with reduced accuracy -T, --temp-dir=DIR create temporary files in directory DIR -y, --fuzzy find similar file for basis if no dest file --compare-dest=DIR also compare received files relative to DIR --copy-dest=DIR ... and include copies of unchanged files --link-dest=DIR hardlink to files in DIR when unchanged -z, --compress compress file data during the transfer --compress-level=NUM explicitly set compression level --skip-compress=LIST skip compressing files with suffix in LIST -C, --cvs-exclude auto-ignore files in the same way CVS does -f, --filter=RULE add a file-filtering RULE -F same as --filter='dir-merge /.rsync-filter' repeated: --filter='- .rsync-filter' --exclude=PATTERN exclude files matching PATTERN --exclude-from=FILE read exclude patterns from FILE --include=PATTERN don't exclude files matching PATTERN --include-from=FILE read include patterns from FILE --files-from=FILE read list of source-file names from FILE -0, --from0 all *from/filter files are delimited by 0s -s, --protect-args no space-splitting; wildcard chars only --address=ADDRESS bind address for outgoing socket to daemon --port=PORT specify double-colon alternate port number --sockopts=OPTIONS specify custom TCP options --blocking-io use blocking I/O for the remote shell --stats give some file-transfer stats -8, --8-bit-output leave high-bit chars unescaped in output -h, --human-readable output numbers in a human-readable format --progress show progress during transfer -P same as --partial --progress -i, --itemize-changes output a change-summary for all updates --out-format=FORMAT output updates using the specified FORMAT --log-file=FILE log what we're doing to the specified FILE --log-file-format=FMT log updates using the specified FMT --password-file=FILE read daemon-access password from FILE --list-only list the files instead of copying them --bwlimit=KBPS limit I/O bandwidth; KBytes per second --write-batch=FILE write a batched update to FILE --only-write-batch=FILE like --write-batch but w/o updating dest --read-batch=FILE read a batched update from FILE --protocol=NUM force an older protocol version to be used --iconv=CONVERT_SPEC request charset conversion of filenames --checksum-seed=NUM set block/file checksum seed (advanced) -4, --ipv4 prefer IPv4 -6, --ipv6 prefer IPv6 --version print version number (-h) --help show this help (see below for -h comment)
References¶
- https://superuser.com/questions/1034408/make-rsync-show-progress-on-checksum-comparison-even-if-no-difference-is-being
- https://blog.wirelessmoves.com/2017/03/backup-doubts-use-checksums-and-rsync.html?utm_source=pocket_mylist
Understanding the output of rsync --itemize-changes¶
As you may know the rsync's --delete options if misused could make severe damage.
To prevent this you can use the --itemize-change and the --dry-run options to figure out how the command will behave before launching the real one.
The output will be something like that:
.d..t..g... ./
.f...p.g... Something.pdf
.f.....g... md5sum-2010-02-21.txt
.f...p.g... prova.rb
.d.....g... .metadata/
.f...p.g... .metadata/.lock
.f...p.g... .metadata/.log
.f...p.g... .metadata/version.ini
>f+++++++++ Parameter_Usage.txt
Where the first field of each line tell what rsync would do to each file.
I wrote this little schema that helped me to understand this output format and I'm publishing hoping it will be useful for others.
YXcstpoguax path/to/file
|||||||||||
`----------- the type of update being done::
|||||||||| <: file is being transferred to the remote host (sent).
|||||||||| >: file is being transferred to the local host (received).
|||||||||| c: local change/creation for the item, such as:
|||||||||| - the creation of a directory
|||||||||| - the changing of a symlink,
|||||||||| - etc.
|||||||||| h: the item is a hard link to another item (requires --hard-links).
|||||||||| .: the item is not being updated (though it might have attributes that are being modified).
|||||||||| *: means that the rest of the itemized-output area contains a message (e.g. "deleting").
||||||||||
`---------- the file type:
||||||||| f for a file,
||||||||| d for a directory,
||||||||| L for a symlink,
||||||||| D for a device,
||||||||| S for a special file (e.g. named sockets and fifos).
|||||||||
`--------- c: different checksum (for regular files)
|||||||| changed value (for symlink, device, and special file)
`-------- s: Size is different
`------- t: Modification time is different
`------ p: Permission are different
`----- o: Owner is different
`---- g: Group is different
`--- u: The u slot is reserved for future use.
`-- a: The ACL information changed
References¶
Last update: 2023-03-28