Commit graph

92 commits

Author SHA1 Message Date
Nick Craig-Wood
3dfcfc2caa operations: document multi-thread copy and tweak defaults 2023-08-24 12:39:27 +01:00
Vitor Gomes
7701d1d33d config: add "multi-thread-chunk-size" flag #7056 2023-08-12 17:55:01 +01:00
antoinetran
db744f64f6 docs: clarify --checksum documentation - Fixes #7145 2023-08-09 20:00:46 +09:00
Nick Craig-Wood
bff702a6f1 docs: group the global flags and make them appear on command and flags pages
This adds an additional parameter to the creation of each flag. This
specifies one or more flag groups. This **must** be set for global
flags and **must not** be set for local flags.

This causes flags.md to be built with sections to aid comprehension
and it causes the documentation pages for each command (and the
`--help`) to be built showing the flags groups as specified in the
`groups` annotation on the command.

See: https://forum.rclone.org/t/make-docs-for-mortals-not-only-rclone-gurus/39476/
2023-08-02 12:53:09 +01:00
Paulo Schreiner
fcb912a664 fs: allow setting a write buffer for multithread
when multi-thread downloading is enabled, rclone used
to send a write to disk after every read, resulting in a lot
of small writes to different locations of the file.

depending on the underlying filesystem or device, it can be more
efficient to send bigger writes.
2023-06-23 18:44:43 +01:00
Janne Hellsten
5c594fea90 operations: implement uploads to temp name with --inplace to disable
When copying to a backend which has the PartialUploads feature flag
set and can Move files the file is copied into a temporary name first.
Once the copy is complete, the file is renamed to the real
destination.

This prevents other processes from seeing partially downloaded copies
of files being downloaded and prevents overwriting the old file until
the new one is complete.

This also adds --inplace flag that can be used to disable the partial
file copy/rename feature.

See #3770

Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2023-05-09 16:28:10 +01:00
Nick Craig-Wood
8e2dc069d2 fs: Add --default-time flag to control unknown modtime of files/dirs
Before this patch, files or directories with unknown modtime would
appear as the current date.

When mounted some systems look at modification dates of directories to
see if they change and having them change whenever they drop out of
the directory cache is not optimal.

See #6986
2023-05-04 15:03:11 +01:00
Leandro Sacchet
f689db4422 fs: Add --max-delete-size a delete size threshold
Fixes #3329
2023-03-08 17:12:31 +00:00
alankrit
6b17044f8e fs:Added multiple ca certificate support. 2023-01-17 12:16:11 +00:00
Kevin Verstaen
c2dfc3e5b3
fs: Add global flag '--color' to control terminal colors
* fs: add TerminalColorMode type
* fs: add new config(flags) for TerminalColorMode
* lib/terminal: use TerminalColorMode to determine how to handle colors
* Add documentation for '--terminal-color-mode'
* tree: remove obsolete --color replaced by global --color

This changes the default behaviour of tree. It now displays colors by
default instead of only displaying them when the flag -C/--color was
active. Old behaviour (no color) can be achieved by setting --color to
'never'.

Fixes: #6604
2022-12-06 12:07:06 +00:00
Nick Craig-Wood
55bbff6346 operations: add --server-side-across-configs global flag for any backend 2022-07-11 18:17:42 +01:00
Nick Craig-Wood
78d52882ca fs: add --metadata/-M flag to control whether metadata is copied 2022-06-29 14:29:36 +01:00
Nick Craig-Wood
c4451bc43a fs: add --metadata-set flag to specify metadata for uploads 2022-06-29 14:29:36 +01:00
Nick Craig-Wood
e7483b40b3 fshttp: add --disable-http-keep-alives to disable HTTP Keep Alives
See: https://forum.rclone.org/t/getting-rate-limited-before-advertised-limit-on-s3-compatible-object-storage/31010/
2022-06-16 22:13:50 +01:00
albertony
e2f47ecdeb docs: punctuation cleanup
See #5538
2021-10-20 22:56:19 +02:00
Ivan Andreev
50df8cec9c lib/kv: add key-value database api #5587
Add bolt-based key-value database support.

Quick API description:
https://github.com/rclone/rclone/pull/5587#issuecomment-942174768
2021-10-20 19:11:54 +03:00
albertony
f3e71f129c config: convert --cache-dir value to an absolute path 2021-10-11 15:08:35 +02:00
albertony
dbb6f94d95 config: make temporary directory user-configurable
See #5591
2021-10-11 15:08:35 +02:00
albertony
f1f923a986 Change byte unit format from MiByte to MiB 2021-08-31 09:57:27 +02:00
albertony
721a9786a7 ls: introduce and global option to print human-readable sizes and consider it for ls commands
Fixes #1890
2021-08-31 09:57:27 +02:00
Alex Chen
e635f4c0be
fs: make --dump imply -vv (#5418) 2021-06-23 00:32:26 +08:00
sp31415t1
f593558dc2
docs: improve --disable help 2021-05-14 15:44:58 +01:00
Nick Craig-Wood
9e6c23d9af fs: add --disable-http2 for global http2 disable #5253 2021-04-30 20:31:04 +01:00
albertony
2925e1384c Use binary prefixes for size and rate units
Includes adding support for additional size input suffix Mi and MiB, treated equivalent to M.
Extends binary suffix output with letter i, e.g. Ki and Mi.
Centralizes creation of bit/byte unit strings.
2021-04-27 02:25:52 +03:00
albertony
98579608ec docs: cleanup spelling of size and rate units 2021-04-27 02:25:52 +03:00
albertony
23a0d4a1e6 config: fix issues with memory-only config file paths
Fixes #5222
2021-04-12 18:17:19 +02:00
Nick Craig-Wood
c0c74003f2 fs/cache: add --fs-cache-expire-duration to control the fs cache
This commit makes the previously statically configured fs cache configurable.

It introduces two parameters `--fs-cache-expire-duration` and
`--fs-cache-expire-interval` to control the caching of the items.

It also adds new interfaces to lib/cache to set these.
2021-03-30 12:46:47 +01:00
Nick Craig-Wood
294f090361 fs: make sure --low-level-retries, --checkers, --transfers are > 0 2021-03-15 17:05:35 +00:00
K265
930bca2478 feat: add multiple paths support to --compare-dest and --copy-dest flag 2021-02-16 18:17:04 +00:00
Max Sum
edfe183ba2 fshttp: add DSCP support with --dscp for QoS with differentiated services 2021-02-10 18:29:18 +00:00
albertony
7a496752f3 fs: add support for flag --no-console on windows to hide the console window 2021-01-27 18:44:35 +00:00
Nick Craig-Wood
f425950a52 fs: Always show stats when using --dry-run or --interactive #4624 2020-12-29 21:11:12 +00:00
Nick Craig-Wood
c98dd8755c log: fix enabling systemd logging when using --log-file
This also moves all the systemd logging decisions to fs/log
2020-12-17 11:55:27 +00:00
Nick Craig-Wood
2e21c58e6a fs: deglobalise the config #4685
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
2020-11-26 16:40:12 +00:00
Josh Soref
e4a87f772f docs: spelling: e.g.
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
bbe7eb35f1 docs: spelling: server-side
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
LaSombra
aac2406e19
cmd: add --progress-terminal-title to print ETA to terminal title
Adds a flag, --progress-terminal-title, that when used with --progress,
will print the string `ETA: %s` to the terminal title.

This also adds WriteTerminalTitle to lib/terminal
2020-10-06 16:34:26 +01:00
Hekmon
c58023a9ba enhance systemd integration
* log level identification
* manual activation with flag
* automatic systemd launch detection
2020-10-06 16:21:30 +01:00
Nick Craig-Wood
26001d520a fs: add --bwlimit-file flag to limit speeds of individual file transfers 2020-07-28 11:46:24 +01:00
Nick Craig-Wood
8f42532b6d sync: add --track-renames-strategy leaf
See: https://forum.rclone.org/t/how-to-minimize-bandwith-w-r-t-renames-during-sync/16928/22
2020-07-28 11:34:27 +01:00
Nick Craig-Wood
3efdf5e095 operations: implement --refresh-times flag to set modtimes on hashless backends 2020-06-17 10:48:13 +01:00
fishbullet
ba5eb230fb operations: interactive mode -i/--interactive for destructive operations #3886 2020-06-10 12:33:53 +01:00
Nick Craig-Wood
518d39815c sync,copy,move: add --check-first to do all checking before starting transfers
See: https://forum.rclone.org/t/rclone-sync-doing-transfer-and-checking-in-paralel/16352/
2020-05-22 17:50:07 +01:00
Ben Zenker
899c8e0697 march: added flag to allow Unicode filenames to remain unique
If your filenames contain two near-identical Unicode characters,
rclone will normalize these, making them identical. This flag
gives you the ability to keep them unique. This might
create unintended side effects, such as duplicating files that
contain certain Unicode characters, when downloading them from
certain cloud providers to a macOS filesystem.

Fixes #4228
2020-05-15 12:28:01 +01:00
Nick Craig-Wood
f6346a4d29 fs: add --header flag to add options to every HTTP transaction #59 2020-04-23 15:24:21 +01:00
Tim Gallant
93caa459e3 fs/config: add header-download and header-upload flags 2020-04-23 11:07:21 +01:00
Jon Fautley
53874bd8ee cmd: add --error-on-no-transfer option
This allows rclone to exit with a non-zero return code if no files are
transferred. This is useful when calling rclone as part of a workflow/script
pipeline as it allows the end user to stop processing if no files have been
transferred.

NB: Enabling this option will return in rclone exiting non-zero if there are no
transfers. Depending on how your're currently using rclone in your scripts,
this may break existing setups!
2020-04-15 17:06:40 +01:00
Bernd Schoolmann
158870bcdb
fs: Add --track-renames-strategy for configurable matching criteria for --track-renames
This commit adds the `--track-renames-strategy` flag which allows the
user to choose the strategy for tracking renames when using the
`--track-renames` flag.

This can be "hash" or "modtime" or both currently.

This, when used with `--track-renames-strategy modtime` enables
support for tracking renames in encrypted remotes.

Fixes #3696
Fixes #2721
2020-03-20 13:04:56 +00:00
Franklyn Tackitt
2b3d13a841 fs: Use --cutoff-mode hard,soft,catious instead of 3 --max-transfer-mode flags
Fixes #2672
2020-03-01 09:49:55 +00:00
Shing Kit Chan
6f1766dd9e fs: Add support for --max-transfer-cutoff modes #2672
This also adds max transfer cut off check for server side copies too
2020-03-01 09:49:55 +00:00