TrueCloudLab/rclone
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
rclone/docs/content/commands/rclone_dedupe.md

114 lines
4.4 KiB
Markdown
Raw Normal View History

Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
---
gendocs: add autogenerated header to all docs
2020-02-10 12:31:45 +00:00
date: 2020-02-10T12:28:36Z
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
title: "rclone dedupe"
slug: rclone_dedupe
url: /commands/rclone_dedupe/
gendocs: add autogenerated header to all docs
2020-02-10 12:31:45 +00:00
# autogenerated - DO NOT EDIT, instead edit the source code in cmd/dedupe/ and as part of making a release run "make commanddocs"
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
---
## rclone dedupe
Version v1.39
2017-12-23 13:07:45 +00:00
Interactively find duplicate files and delete/rename them.
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
### Synopsis
Version v1.38
2017-09-30 13:19:47 +00:00
By default `dedupe` interactively finds duplicate files and offers to
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
delete all but one or rename them to be different. Only useful with
Google Drive which can have duplicate file names.
Version v1.38
2017-09-30 13:19:47 +00:00
In the first pass it will merge directories with the same name. It
will do this iteratively until all the identical directories have been
merged.
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
The `dedupe` command will delete all but one of any identical (same
md5sum) files it finds without confirmation. This means that for most
duplicated files the `dedupe` command will not be interactive. You
can use `--dry-run` to see what would happen without doing anything.
Here is an example run.
Before - with duplicates
$ rclone lsl drive:dupes
6048320 2016-03-05 16:23:16.798000000 one.txt
6048320 2016-03-05 16:23:11.775000000 one.txt
564374 2016-03-05 16:23:06.731000000 one.txt
6048320 2016-03-05 16:18:26.092000000 one.txt
6048320 2016-03-05 16:22:46.185000000 two.txt
1744073 2016-03-05 16:22:38.104000000 two.txt
564374 2016-03-05 16:22:52.118000000 two.txt
Now the `dedupe` session
$ rclone dedupe drive:dupes
2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode.
one.txt: Found 4 duplicates - deleting identical copies
one.txt: Deleting 2/3 identical duplicates (md5sum "1eedaa9fe86fd4b8632e2ac549403b36")
one.txt: 2 duplicates remain
1: 6048320 bytes, 2016-03-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
2: 564374 bytes, 2016-03-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
s) Skip and do nothing
k) Keep just one (choose which in next step)
r) Rename all to be different (by changing file.jpg to file-1.jpg)
s/k/r> k
Enter the number of the file to keep> 1
one.txt: Deleted 1 extra copies
two.txt: Found 3 duplicates - deleting identical copies
two.txt: 3 duplicates remain
1: 564374 bytes, 2016-03-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
2: 6048320 bytes, 2016-03-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
3: 1744073 bytes, 2016-03-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802
s) Skip and do nothing
k) Keep just one (choose which in next step)
r) Rename all to be different (by changing file.jpg to file-1.jpg)
s/k/r> r
two-1.txt: renamed from: two.txt
two-2.txt: renamed from: two.txt
two-3.txt: renamed from: two.txt
The result being
$ rclone lsl drive:dupes
6048320 2016-03-05 16:23:16.798000000 one.txt
564374 2016-03-05 16:22:52.118000000 two-1.txt
6048320 2016-03-05 16:22:46.185000000 two-2.txt
1744073 2016-03-05 16:22:38.104000000 two-3.txt
Dedupe can be run non interactively using the `--dedupe-mode` flag or by using an extra parameter with the same value
* `--dedupe-mode interactive` - interactive as above.
* `--dedupe-mode skip` - removes identical files then skips anything left.
* `--dedupe-mode first` - removes identical files then keeps the first one.
* `--dedupe-mode newest` - removes identical files then keeps the newest one.
* `--dedupe-mode oldest` - removes identical files then keeps the oldest one.
Version v1.41
2018-04-28 10:46:27 +00:00
* `--dedupe-mode largest` - removes identical files then keeps the largest one.
Version v1.51.0
2020-02-01 10:31:42 +00:00
* `--dedupe-mode smallest` - removes identical files then keeps the smallest one.
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
* `--dedupe-mode rename` - removes identical files then renames the rest to be different.
For example to rename all the identically named photos in your Google Photos directory, do
rclone dedupe --dedupe-mode rename "drive:Google Photos"
Or
rclone dedupe rename "drive:Google Photos"
```
Add ftp.md to docs builder and update docs
2017-06-15 19:12:26 +00:00
rclone dedupe [mode] remote:path [flags]
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
```
### Options
```
Version v1.51.0
2020-02-01 10:31:42 +00:00
--dedupe-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename. (default "interactive")
Version v1.38
2017-09-30 13:19:47 +00:00
-h, --help help for dedupe
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
```
docs: add a new page with global flags and link to it from the command docs In f544234 we removed the global flags from each command as it was making each page very big and causing 1000s of lines of duplication in the man page. This change adds a new flags page with all the global flags on and links each command page to it. Fixes #3273
2019-06-20 15:18:02 +00:00
See the [global flags page](/flags/) for global options not listed here.
Use cobra autogenerated docs * put the most up to date docs into the code * generate command docs using rclone gendocs * put command docs into own directory * remake them into MANUAL.md
2016-08-03 20:36:28 +00:00
### SEE ALSO
Version v1.44
2018-10-15 10:03:08 +00:00
* [rclone](/commands/rclone/) - Show help for rclone commands, flags and backends.
Version v1.40
2018-03-19 10:06:13 +00:00
Reference in a new issue Copy permalink