TrueCloudLab/rclone
16
0
Fork 4
Code Issues 1 Pull requests Projects Releases 4 Packages Wiki Activity Actions

sync: report list of synced paths to file -- see #7282

Browse source 
Allows rclone sync to accept the same output file flags as rclone check,
for the purpose of writing results to a file.
A new --dest-after option is also supported, which writes a list file using
the same ListFormat flags as lsf (including customizable options for hash,
modtime, etc.) Conceptually it is similar to rsync's --itemize-changes, but
not identical -- it should output an accurate list of what will be on the
destination after the sync.

Note that it has a few limitations, and certain scenarios
are not currently supported:

--max-duration / CutoffModeHard
--compare-dest / --copy-dest (because equal() is called multiple times for the
    same file)
server-side moves of an entire dir at once (because we never get the individual
file objects in the dir)
High-level retries, because there would be dupes
Possibly some error scenarios that didn't come up on the tests

Note also that each file is logged during the sync, as opposed to after, so it
is most useful as a predictor of what SHOULD happen to each file
(which may or may not match what actually DID.)

Only rclone sync is currently supported -- support for copy and move may be
added in the future.
This commit is contained in:
nielash 2023-11-06 02:45:51 -05:00
parent c0968a0987
commit 3a50f35df9
10 changed files with 470 additions and 46 deletions
Download patch file Download diff file Expand all files Collapse all files

37
docs/content/commands/rclone_sync.md
View file

@ -48,6 +48,43 @@ destination that is inside the source directory.
**Note**: Use the `rclone dedupe` command to deal with "Duplicate object/directory found in source/destination - ignoring" errors.
See [this forum post](https://forum.rclone.org/t/sync-not-clearing-duplicates/14372) for more info.
## Logger Flags
The `--differ`, `--missing-on-dst`, `--missing-on-src`, `--match`
and `--error` flags write paths, one per line, to the file name (or
stdout if it is `-`) supplied. What they write is described in the
help below. For example `--differ` will write all paths which are
present on both the source and destination but different.
The `--combined` flag will write a file (or stdout) which contains all
file paths with a symbol and then a space and then the path to tell
you what happened to it. These are reminiscent of diff files.
- `= path` means path was found in source and destination and was identical
- `- path` means path was missing on the source, so only in the destination
- `+ path` means path was missing on the destination, so only in the source
- `* path` means path was present in source and destination but different.
- `! path` means there was an error reading or hashing the source or dest.
The `--dest-after` flag writes a list file using the same format flags
as [`lsf`](/commands/rclone_lsf/#synopsis) (including [customizable options
for hash, modtime, etc.](/commands/rclone_lsf/#synopsis))
Conceptually it is similar to rsync's `--itemize-changes`, but not identical
-- it should output an accurate list of what will be on the destination
after the sync.
Note that these logger flags have a few limitations, and certain scenarios
are not currently supported:
- `--max-duration` / `CutoffModeHard`
- `--compare-dest` / `--copy-dest`
- server-side moves of an entire dir at once
- High-level retries, because there would be duplicates (use `--retries 1` to disable)
- Possibly some unusual error scenarios
Note also that each file is logged during the sync, as opposed to after, so it
is most useful as a predictor of what SHOULD happen to each file
(which may or may not match what actually DID.)
```
rclone sync source:path dest:path [flags]