Commit graph

5042 commits

Author SHA1 Message Date
Nick Craig-Wood
83d48f65b6 Add Manish Gupta to contributors 2020-11-12 17:58:46 +00:00
Manish Gupta
95d0410baa
local: continue listing files/folders when a circular symlink is detected
Before this change a circular symlink would cause rclone to error out from the listings.

After this change rclone will skip a circular symlink and carry on the listing,
producing an error at the end.

Fixes #4743
2020-11-12 11:32:55 +00:00
albertony
2708a7569e mount: docs: make note about mounting as network drive less confusing 2020-11-11 20:37:57 +00:00
Nick Craig-Wood
45e8bea8d0 testserver: Make Test{FTP,SFTP,Webdav}Rclone run the current rclone
Before this change the tests were run against the previous stable
rclone/rclone docker image.

This unfortunately masked errors in the integration test server.

This change uses the currently installed rclone to run "rclone serve
ftp" etc. This is installed out of the current code by the integration
test server so will make a better test.
2020-11-10 18:01:15 +00:00
Nick Craig-Wood
f980f230c5 vfs: fix virtual entries causing deleted files to still appear
Before this change, if a file was created on a remote but deleted
externally from that remote then there was potential for the delete to
never be noticed.

The sequence of events was:

- Create file on VFS - creates virtual directory entry
- File deleted externally to remote before the directory refreshed
- Now the file has a virtual add but is not in the listings so will never disappear

This patch fixes it by removing all virtual directory entries except
the following when the directory is re-read.

- On remotes which can't have empty directories: virtual directory
  adds are not flushed. These will remain virtual as long as the
  directory is empty.

- For virtual file add: files that are in the process of being
  uploaded are not flushed

This patch also adds the distinction between virtually added files and
directories.

It also refactors the virtual directory logic to make it easier to follow.

Fixes #4446
2020-11-10 16:47:25 +00:00
Nick Craig-Wood
e204f89685 servetest: add -sub-run flag for running a subset of the backend tests
Use like this (eg in cmd/serve/sftp)

    go test -v -run TestSftp/Normal -sub-run "TestIntegration/FsMkdir/FsPutFiles/FsDirMove"
2020-11-10 16:47:25 +00:00
Nick Craig-Wood
f7efce594b config: add context.Context #3257 #4685
This add config to the Config callback in the backends and the related
config functions.
2020-11-09 18:05:54 +00:00
Nick Craig-Wood
1fb6ad700f accounting: add context.Context #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood
e3fe31f7cb fs: add context.Context to fs.GetModifyWindow #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood
8b96933e58 fs: Add context to fs.Features.Fill & fs.Features.Mask #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood
d69b96a94c test: Add context to mockfs.NewFs #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood
d846210978 fs: Add context to NewFs #3257 #4685
This adds a context.Context parameter to NewFs and related calls.

This is necessary as part of reading config from the context -
backends need to be able to read the global config.
2020-11-09 18:05:54 +00:00
Anagh Kumar Baranwal
30c8b1b84f
fs: Fix nil pointer on copy & move operations directly to remote
Fix the copy and move operations that broke in 127f0fc when copying directly
to a remote without a specific destination.

Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
2020-11-09 12:36:10 +00:00
Nick Craig-Wood
43e0929339 vfs: fix vfs/refresh calls with fs= parameter
Before this change rclone gave an error when the fs parameter was
provided.

This change removes the fs parameter from the parameters once it has
been read which avoids the error.

See: https://forum.rclone.org/t/precaching-with-vfs-refresh-fails-with-an-error-when-having-multiple-cloud-drives/20267
2020-11-07 14:26:33 +00:00
albertony
6c70c42577 jottacloud: docs: add heading to the example section so that it does not seem to be part of the legacy setup section 2020-11-06 17:52:50 +00:00
Adam Plánský
cd2c06f2a7 testserver: speed up seafile integration test
Before every seafile integration test we run Seafile server in docker
environment. We don't have to sleep for 60 seconds to have everything ready before running integration test. We can assume everything is ready when Seafile webserver returns status code 200. Seafile Dockerfile (https://github.com/haiwen/seafile-docker/blob/master/image/seafile/Dockerfile) runs scripts/start.py where is defined that before init_seafile_server() we always wait for mysql wait_for_mysql() (https://github.com/haiwen/seafile-docker/blob/master/scripts/start.py)
2020-11-03 16:31:39 +00:00
Nick Craig-Wood
af55a74bd2 stats: add counter for deleted directories - fixes #4676 2020-11-03 11:47:00 +00:00
Nick Craig-Wood
d00c126cef operations: fix --cutof-mode hard not cutting off immediately
This failure was noted on the integration tests server.

The fix was to be more careful about which error message was emitted
with which --cutoff-mode
2020-11-02 17:13:19 +00:00
Nick Craig-Wood
bedf6e90d2 onedrive: warn on gateway timeout errors
It seems that when doing chunked uploads to onedrive, if the chunks
take more than 3 minutes or so to upload then they may timeout with
error 504 Gateway Timeout.

This change produces an error (just once) suggesting lowering
`--onedrive-chunk-size` or decreasing `--transfers`.

This is easy to replicate with:

    rclone copy -Pvv --bwlimit 0.05M 20M onedrive:20M

See: https://forum.rclone.org/t/default-onedrive-chunk-size-does-not-work/20010/
2020-11-02 16:53:35 +00:00
Nick Craig-Wood
e8c84d8b53 Add Adam Plánský to contributors 2020-11-02 16:53:35 +00:00
Adam Plánský
f89ff3872d ncdu: add toggle option for average size in directory - key 'a'
Add toggle option to show average size in directory. This toggle
function is for ncdu and is binded to key 'a'.
2020-10-30 15:33:54 +00:00
Adam Plánský
127f0fc64c operations: move and copy log name of the destination object in verbose
If the object is moved or copied rclone in verbose mode prints name of the
destination object into the info log.
2020-10-30 15:31:54 +00:00
edwardxml
0cfa89f316
docs: ftp: put limitations in a single section
The topic is mostly about so limitations so all of these are grouped together with a section hyperlink near the top of the page. Intention is to avoid potential duplication and make it more straightforward (there is a place and it is essentially just a list so wording doesn't need to be elegant) to add notes about limitations in future, harvested from rclone forum postings.

Minor wording alterations that do not intend to change meaning
2020-10-30 14:59:36 +00:00
Nick Craig-Wood
bfcd4113c3 mount: implement mknod to make NFS file creation work - fixes #2115
It turns out that NFS calls mknod in FUSE even though we have create
defined. This was causing EIO errors when creating files.

This patch fixes it by implementing mknod. The way it is implemented
means that to write to an NFS file system you'll need --vfs-cache-mode
writes.
2020-10-29 15:12:36 +00:00
Nick Craig-Wood
0e7fc7613f mount: make mount be cmount under macOS #4393
This also adds an alias to the mount command so it responds as `rclone
cmount` as well as `rclone mount`.
2020-10-29 13:34:39 +00:00
Nick Craig-Wood
8ac2f52b6e mount: disable bazil/fuse based mount on macOS #4393
The library is no longer supported on macOS.
2020-10-29 13:34:39 +00:00
Nick Craig-Wood
1973fc1ecc azureblob: update lib from v0.10.0 to v0.11.0 and fix API breakage
See: https://github.com/Azure/azure-storage-blob-go/issues/226
2020-10-29 13:34:39 +00:00
Nick Craig-Wood
7c39a13281 build: update all dependencies 2020-10-29 13:34:39 +00:00
Nick Craig-Wood
c5c503cbbe Add Adam Plánský to contributors 2020-10-29 13:34:39 +00:00
Josh Soref
d09488b829 docs: update: add Tencent
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
0a6196716c docs: style: avoid double-nesting parens
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
8bc9b2b883 docs: grammar: examples are examples
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
a15f50254a docs: grammar: if, then
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
5d4f77a022 docs: grammar: Oxford comma
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
a089de0964 docs: grammar: uncountable: links
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
3068ae8447 docs: grammar: count agreement: files
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
67ff153b0c docs: grammar: article: a-file
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
3e1cb8302a docs: spelling: etc.
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
e4a87f772f docs: spelling: e.g.
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
d4f38d45a5 docs: spelling: high-speed
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Josh Soref
bbe7eb35f1 docs: spelling: server-side
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
edwardxml
87e54f2dde
ftp: update wording for flags
Minor wording change to help for explicit and implicit FTPS flags. More consistent between flags. Add 's' to request because only one 'client' mentioned.
2020-10-28 15:45:52 +00:00
edwardxml
3f3afe489f
docs: mount: minor changes to wording 2020-10-28 15:43:49 +00:00
edwardxml
70b21d9c87
docs: Update mount docs
Eliminate repeat word mode
2020-10-28 15:42:33 +00:00
Adam Plánský
e00bf3d723
ncdu: add sort by average size in directory
Add keyboard shortcut 'A' which sort by average size in directory.

If files/folders have same avgSize sort by actual size

Fixes: #4699
2020-10-27 13:28:38 +00:00
Nick Craig-Wood
605f2b819a build: fix nfpm install 2020-10-27 13:15:21 +00:00
Nick Craig-Wood
bf2b975359 build: update nfpm and github-release install method to go modules 2020-10-26 19:07:59 +00:00
Nick Craig-Wood
00a5086ff2 Remove accidentally committed binary and fix formatting
These were both committed in

b7253fc1c1 mount: docs: add note that allow-root and allow-other is not relevant on windows
2020-10-26 19:07:42 +00:00
Ivan Andreev
be6a888e50 chunker: skip long local hashing, hash in-transit (fixes #4021)
PR 4614
2020-10-26 20:18:07 +03:00
Ivan Andreev
dad8447423 mailru: avoid prehashing of large local files
PR 4617
2020-10-26 20:16:52 +03:00