Commit graph

5218 commits

Author SHA1 Message Date
Nick Craig-Wood
1b92e4636e rc: implement passing filter config with _filter parameter 2021-02-23 10:54:40 +00:00
Nick Craig-Wood
c5a299d5b1 rc: fix options/local to return the filter options 2021-02-23 10:33:03 +00:00
Nick Craig-Wood
04a8859d29 cmount: fix mount dropping on macOS by setting --daemon-timeout 10m
Previously rclone set --daemon-timeout to 15m by default. However
osxfuse seems to be ignoring that value since it is above the maximum
value of 10m. This is conjecture since the source of osxfuse is no
longer available.

Setting the value to 10m seems to resolve the problem.

See: https://forum.rclone.org/t/rclone-mount-frequently-drops-when-using-plex/22352
2021-02-21 12:56:19 +00:00
Nick Craig-Wood
4b5fe3adad delete,rmdirs: make --rmdirs obey the filters
See: https://forum.rclone.org/t/a-problem-with-rclone-delete-from-list/22143
2021-02-19 10:32:28 +00:00
edwardxml
7db68b72f1
docs: directory filter rules 2021-02-18 12:11:56 +01:00
edwardxml
9c667be2a1
docs: remove dead link from rc.md (#5038) 2021-02-18 01:37:17 +03:00
tYYGH
c0cf54067a
vfs: --vfs-used-is-size to report used space using recursive scan (#4043)
Some backends, most notably S3, do not report the amount of bytes used.
This patch introduces a new flag that allows instead of relying on the
backend, use recursive scan similar to `rclone size` to compute the total
used space. However, this is ineffective and should be used as a last resort.

Co-authored-by: Yves G <theYinYeti@yalis.fr>
2021-02-17 23:36:13 +03:00
Romeo Kienzler
297ca23abd
docs: fix typo in crypt.md (#5037) 2021-02-17 19:11:57 +03:00
Nick Craig-Wood
d809930e1d union: fix mkdir at root with remote:/
Before the this fix if you specified remote:/ then the union backend
would fail to notice the root directory existed.

This was fixed by stripping the trailing / from the root.

See: https://forum.rclone.org/t/upgraded-from-1-45-to-1-54-now-cant-create-new-directory-within-union-mount/22284/
2021-02-17 12:11:34 +00:00
Nick Craig-Wood
fdc0528bd5 Add Dmitry Chepurovskiy to contributors 2021-02-17 12:11:34 +00:00
Nick Craig-Wood
a0320d6e94 Add Vesnyx to contributors 2021-02-17 12:11:34 +00:00
Nick Craig-Wood
89bf036e15 Add K265 to contributors 2021-02-17 12:11:34 +00:00
Dmitry Chepurovskiy
1605f9e14d
s3: Fix shared_credentials_file auth
S3 backend shared_credentials_file option wasn't working neither from
config option nor from command line option. This was caused cause
shared_credentials_file_provider works as part of chain provider, but in
case user haven't specified access_token and access_key we had removed
(set nil) to credentials field, that may contain actual credentials got
from ChainProvider.

AWS_SHARED_CREDENTIALS_FILE env varible as far as i understood worked,
cause aws_sdk code handles it as one of default auth options, when
there's not configured credentials.
2021-02-17 12:04:26 +00:00
albertony
cd6fd4be4b
mount: docs: document the new FileSecurity option in WinFsp 2021 (#5002) 2021-02-17 03:44:28 +03:00
Vesnyx
4ea7c7aa47
crypt: add option to not encrypt data #1077 (#2981)
Co-authored-by: Ivan Andreev <ivandeex@gmail.com>
2021-02-17 03:40:37 +03:00
Ivan Andreev
5834020316
docker.bash: work correctly with multi-ip containers (#5028)
Currently if container under test has multiple IP addresses,
the `docker_ip` function from `docker.sh` will return a gibberish.
This patch makes it return the first address found.
Additionally, I apply shellcheck on `docker.sh`.
2021-02-17 03:38:02 +03:00
Ivan Andreev
f5066a09cd
build: replace go 1.16-rc1 by 1.16.x (#5036) 2021-02-17 03:37:30 +03:00
edwardxml
863bd93c30
docs: fix broken link in sftp page
Just a spare line break had crept in breaking the link form.
2021-02-16 23:24:11 +01:00
edwardxml
d96af3b005
docs: convert bogus example link to code
Convert the bogus example plex url from a url that is auto created to code format that hopefully isn't.
2021-02-16 23:20:49 +01:00
edwardxml
3280ceee3b
docs: badly formed link
Fix for a badly formed link created in earlier rewrite
2021-02-16 23:16:03 +01:00
K265
930bca2478 feat: add multiple paths support to --compare-dest and --copy-dest flag 2021-02-16 18:17:04 +00:00
edwardxml
23b12c39bd Docs: Zoho WorkDrive authorisation reword
Mainly the reference to firewalls didn't make sense. Tried to make more precise. Left z in authorize.
2021-02-16 18:07:55 +00:00
Nick Craig-Wood
9d37c208b7 vfs: document simultaneous usage with the same cache shouldn't be used
Fixes #2227
2021-02-16 17:15:05 +00:00
Nick Craig-Wood
c81311722e ftp: close idle connections after --ftp-idle-timeout (1m by default)
This fixes a problem where ftp backends live on forever when using
the rc and use more and more connections.
2021-02-16 12:39:05 +00:00
Nick Craig-Wood
843ddd9136 ftp: implement Shutdown method 2021-02-16 12:39:05 +00:00
Nick Craig-Wood
a3fcadddc8 sftp: close idle connections after --sftp-idle-timeout (1m by default)
This fixes a problem where sftp backends live on forever when using
the rc and use more and more connections.

Fixes #4883
2021-02-16 12:39:05 +00:00
Nick Craig-Wood
a63e1f1383 Add Miron Veryanskiy to contributors 2021-02-16 12:39:05 +00:00
Nick Craig-Wood
5b84adf3b9 test: add "rclone test histogram" for file name distribution stats 2021-02-13 14:24:43 +00:00
Nick Craig-Wood
f890965020 test: add makefiles test command (converted from script) 2021-02-13 14:24:43 +00:00
Nick Craig-Wood
f88a5542cf test: move test commands under "rclone test" and make them visible 2021-02-13 14:24:43 +00:00
Miron Veryanskiy
fd94b3a473 docs: replace #file-caching with #vfs-file-caching
The documentation had dead links pointing to #file-caching. They've been
moved to point to #vfs-file-caching.
2021-02-13 12:56:25 +00:00
Nick Craig-Wood
2aebeb6061 accounting: fix --bwlimit when up or down is off - fixes #5019
Before this change the core bandwidth limit was limited to upload or
download value if the other value was off.

This fix only applies a core bandwidth limit when both values are set.
2021-02-13 12:45:12 +00:00
Nick Craig-Wood
e779cacc82 fshttp: fix bandwidth limiting after bad merge
Reapply missing bwlimiting which was inserted in

0a932dc1f2 Add --bwlimit for upload and download #1873

But accidentally removed when merging

edfe183ba2 fshttp: add DSCP support with --dscp for QoS with differentiated services
2021-02-13 12:45:12 +00:00
Nick Craig-Wood
37e630178e dropbox: add scopes to oauth request and optionally "members.read"
This change adds the scopes rclone wants during the oauth request.
Previously rclone left these blank to get a default set.

This allows rclone to add the "members.read" scope which is necessary
for "impersonate" to work, but only when it is in use as it require
authorisation from a Team Admin.

See: https://forum.rclone.org/t/dropbox-no-members-read/22223/3
2021-02-13 12:35:24 +00:00
Nick Craig-Wood
2cdc071b85 Add Ankur Gupta to contributors 2021-02-13 12:35:24 +00:00
Nick Craig-Wood
496e32fd8a Add cynthia kwok to contributors 2021-02-13 12:35:24 +00:00
Nick Craig-Wood
bf3ba50a0f Add David Sze to contributors 2021-02-13 12:35:24 +00:00
Nick Craig-Wood
22c226b152 Add Alexey Tabakman to contributors 2021-02-13 12:35:23 +00:00
Klaus Post
5ca7f1fe87 encoder/filename: Wrap scsu package 2021-02-12 11:39:39 +00:00
Klaus Post
f14220ef1e encoder/filename: Add 2 more tables and tests. 2021-02-12 11:39:39 +00:00
Klaus Post
424aaac2e1 encoder/filename: Add SCSU as tables
Instead of only adding SCSU, add it as an existing table.

Allow direct SCSU and add a, perhaps, reasonable table as well.

Add byte interfaces that doesn't base64 encode the URL as well with `EncodeBytes` and `DecodeBytes`.

Fuzz tested and decode tests added.
2021-02-12 11:39:39 +00:00
Ankur Gupta
47b69d6300 operations: Made copy and sync operations obey a RetryAfterError 2021-02-11 17:47:34 +00:00
cynthia kwok
c0c2505977
build: add an rclone user to the Docker image but don't use it by default
partially addresses #4831

Co-authored-by: cynful <cynful@users.noreply.github.com>
2021-02-11 17:45:44 +00:00
David Sze
2d7afe8690 local: Add flag --no-preallocate - #3207
Some virtual filesystems (such as Google Drive File Stream) may
incorrectly set the actual file size equal to the preallocated space,
causing checksum and file size checks to fail.

This flag can be used to disable preallocation for local backends of
this type.
2021-02-11 17:25:28 +00:00
Nick Craig-Wood
92187a3b33 cmount: fix unicode issues with accented characters on macOS
This adds

    -o modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC

To the mount options if it isn't already present which fixes mounting
issues on macOS with accented characters in the finder.
2021-02-11 15:13:19 +00:00
Nick Craig-Wood
53aa4b87fd b2: fix failed to create file system with application key limited to a prefix
Before this change, if an application key limited to a prefix was in
use, with trailing `/` marking the folders then rclone would HEAD the
path without a trailing `/` to work out if it was a file or a folder.
This returned a permission denied error, which rclone returned to the
user.

    Failed to create file system for "b2:bucket/path/":
        failed to HEAD for download: Unknown 401  (401 unknown)

This change assumes any errors on HEAD will make rclone assume the
object does not exist and the path is referring to a directory.

See: https://forum.rclone.org/t/b2-error-on-application-key-limited-to-a-prefix/22159/
2021-02-11 15:13:19 +00:00
Max Sum
edfe183ba2 fshttp: add DSCP support with --dscp for QoS with differentiated services 2021-02-10 18:29:18 +00:00
edwardxml
dfc63eb8f1
docs: update filtering docs
Typos from prior major rewrite
2021-02-10 18:21:41 +00:00
edwardxml
f21f2529a3
docs: fix nesting of brackets and backticks in ftp docs 2021-02-10 18:18:01 +00:00
edwardxml
1efb543ad8
docs: add a Windows example to the filtering docs
Add an example pinched from rclone forum

https://forum.rclone.org/t/need-help-to-understand-filtering-commands/22196

Credit to @asdffdsa
2021-02-10 18:09:48 +00:00