TrueCloudLab/rclone
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions
4133 commits 181 branches 114 tags 335 MiB
Commit graph

4133 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Craig-Wood
9c9cdf1712 fs: don't run tests for --max-duration on remote backends 
This is a timing dependent test and to make it long enough so that it
would work with the remotes would make it too long for local tests.

The code paths are identical for local vs non-local so just run on
local.

This fixes the integration tests.
 2020-01-26 09:23:03 +00:00
Nick Craig-Wood
0e5537cd25 Add unbelauscht to contributors 2020-01-26 09:23:03 +00:00
unbelauscht
151d0a274e swift: Update OVH API endpoint - Fixes #3890 2020-01-24 17:43:00 +00:00
Nick Craig-Wood
dc77ec4ba1 Add boosh to contributors 2020-01-24 13:29:36 +00:00
boosh
0d7573dd81 fs: Add --max-duration flag to control the maximum duration of a transfer session 
This gives you more control over how long rclone will run for, making
it easier to script backups, e.g. via cron. Once the `--max-duration`
time limit is reached, no new transfers will be initiated, but those
already in-flight will be allowed to complete.

Fixes #985
 2020-01-24 13:28:56 +00:00
Nick Craig-Wood
e4d2d228bd webdav: add Referer header to fix problems with WAFs - fixes #3868 2020-01-23 15:56:17 +00:00
Nick Craig-Wood
ede36b001b Add Damon Permezel to contributors 2020-01-23 15:40:23 +00:00
Nick Craig-Wood
3afb2a4798 config: use SpaceSepList for argument to --password-command 
This is to enable arguments with spaces in.
 2020-01-23 15:39:15 +00:00
Nick Craig-Wood
62dbdcdbcc config: use the environment variable which goes with --password-command 2020-01-23 15:39:15 +00:00
Damon Permezel
06df133159 config: add --password-command to allow dynamic config password - fixes #3694 2020-01-23 15:39:15 +00:00
Xiaoxing Ye
0ab2693da6 doc: add desc about gzip and http dump 
Fix #3872
 2020-01-23 12:42:44 +00:00
Nick Craig-Wood
4b1cb1be43 Add jtagcat to contributors 2020-01-23 12:38:56 +00:00
Nick Craig-Wood
9d96680329 Add thestigma to contributors 2020-01-23 12:38:56 +00:00
jtagcat
d694bb30e5 install.sh: create ~/.config/rclone directory 
This allows people with their config already made copy their config to the new server(s), without having to create the directories.
 2020-01-23 12:37:46 +00:00
buengese
9c858c3228 crypt: correctly handle trailing dot 2020-01-22 01:40:04 +01:00
thestigma
7125cb10f5 doc: Add note to filtering.md about --files-from and rclone lsf 2020-01-20 17:37:24 +00:00
Nick Craig-Wood
ba421fd069 Add landall to contributors 2020-01-20 17:30:27 +00:00
landall
77e55b8265 hashsum: Add flag --base64 flag - fixes #3663 
This flag can be used to output QuickXorHash in the same format as MS
Graph API.
 2020-01-20 17:29:58 +00:00
Nick Craig-Wood
18d26e2ddb Revert "vendor: update x/crypto/ssh - to fix Windows password length issues fixes #3798" 
This turned out to introduce a regression, not being able to press Enter.

See: #3888 and https://github.com/golang/go/issues/36609

This reverts commit 251cfc100e.
 2020-01-20 12:46:52 +00:00
Nick Craig-Wood
f338a2d907 Add Benjamin Richter to contributors 2020-01-20 12:46:46 +00:00
Benjamin Richter
77fa8194f2 onedrive: add Sites.Read.All permission - Fixes #1770 2020-01-20 12:30:19 +00:00
Xiaoxing Ye
ccaca04a5d rcd: move webgui apart; option to disable browser 
Fix #3601, #3785
 2020-01-20 12:27:55 +00:00
Nick Craig-Wood
84191ac6dc vfs: fix incorrect modtime for mv into mount with --vfs-cache-modes write 
When a file has its modtime set while it is open we delay setting the
modtime until the file is closed.

The file is then uploaded in Flush. In Release we check the cached
file has been uploaded by comparing modtimes and or hashes and upload
it again if it has changed.

Before this change we forgot to change the time on the cached file
when we updated the time file on the object, so this mean that Release
reset the time to the wrong time and uploaded the file again on
remotes which don't support hashes (eg crypt).

The fix was to set the modtime of the cached file at the same time we
set the modtime of the remote object. This means that the files check
as identical in Release so it doesn't try to upload the file.

This means that we avoid a double upload and the modtime is correct.

See: https://forum.rclone.org/t/modification-time-with-vfs-cache/13906/8
 2020-01-19 12:52:48 +00:00
Nick Craig-Wood
7cf8ea354c dedupe: add missing modes to help string 2020-01-19 11:09:45 +00:00
Nick Craig-Wood
24ef00a258 build: implement a framework for starting test servers during tests 
Test servers are implemented by docker containers and run real servers
for rclone to test against.
 2020-01-18 16:47:37 +00:00
Nick Craig-Wood
00d30ce0d7 opendrive: implement --opendrive-chunk-size #3707 2020-01-18 11:56:01 +00:00
Nick Craig-Wood
db39adeb3e sftp: open files for update write only to fix AWS SFTP interop - fixes #3776 2020-01-18 11:46:56 +00:00
Nick Craig-Wood
ef7ac088c0 operations: make NewOverrideObjectInfo public and factor uses 2020-01-18 11:41:33 +00:00
Nick Craig-Wood
08a3957880 cache: fix fatal error: concurrent map writes - fixes #2378 2020-01-18 11:27:00 +00:00
Nick Craig-Wood
4499b08afc drive: log an ERROR if an incomplete search is returned 2020-01-18 11:22:26 +00:00
Nick Craig-Wood
422ad38e5b copyurl: add --stdout flag to write to stdout 2020-01-18 11:15:51 +00:00
Nick Craig-Wood
0b7f959433 cmount: when setting dates discard out of range dates 
It appears that sometimes Windows/WinFSP/cgofuse sends dates which are
the epoch to rclone.  These dates appear as 1601-01-01 00:00:00 plus
or minus the timezone.

These dates aren't being sent from rclone.

This patch filters dates out before 1601-01-02 so rclone does not
attempt to set them.

See: https://forum.rclone.org/t/bug-corruption-of-modtime-via-vfs-layer/12204
See: https://forum.rclone.org/t/io-error-googleapi-error-403-insufficient-permission-insufficientpermissions/11372
See: https://github.com/billziss-gh/cgofuse/issues/35
 2020-01-18 11:13:35 +00:00
Nick Craig-Wood
4b9da601be dropbox: treat insufficient_space errors as non retriable errors 
Before this change rclone would keep trying to upload files after
dropbox had signalled it was full.

This change makes the relevant error a non-retriable error.

See: https://forum.rclone.org/t/why-does-a-file-transfer-continue-when-there-is-no-available-storage/13677
 2020-01-18 11:10:18 +00:00
Nick Craig-Wood
c789436580 The memory backend 
This is a bucket based remote
 2020-01-18 10:41:08 +00:00
Nick Craig-Wood
277d94feac fshttp: add --expect-continue-timeout default 1s - fixes #3835 
Before this change the expect/continue timeout was set to
--conntimeout which was 60s by default which is too long to wait.

This was noticed when using s3 with a proxy which apparently didn't
support expect / continue properly.

Set --expect-continue-timeout 0 to disable expect/continue.
 2020-01-18 09:49:22 +00:00
Nick Craig-Wood
6757244918 drive: use multipart resumable uploads for streaming and uploads in mount 
Before this change we used non multipart uploads for files of unknown
size (streaming and uploads in mount).  This is slower and less
reliable and is not recommended by Google for files smaller than 5MB.

After this change we use multipart resumable uploads for all files of
unknown length.  This will use an extra transaction so is less
efficient for files under the chunk size, however the natural
buffering in the operations.Rcat call specified by
`--streaming-upload-cutoff` will overcome this.

See: https://forum.rclone.org/t/upload-behaviour-and-speed-when-using-vfs-cache/9920/
 2020-01-17 22:03:10 +00:00
Nick Craig-Wood
36157d8ae5 vendor: update t3rm1n4l/go-mega - fixes mega: couldn't login: crypto/aes: invalid key size 0 
Fixes #3740
 2020-01-17 21:42:32 +00:00
Nick Craig-Wood
251cfc100e vendor: update x/crypto/ssh - to fix Windows password length issues fixes #3798 2020-01-17 21:33:47 +00:00
Nick Craig-Wood
9fb10064ee vendor: run go mod tidy 2020-01-17 21:19:46 +00:00
Nick Craig-Wood
bedeaf23af sugarsync: new backend - fixes #622 2020-01-17 17:39:34 +00:00
Nick Craig-Wood
14e93bfd8a rest: call the Signer with the mutex unlocked 
This enables the signer to adjust rest parameters and call rest again
if necessary.
 2020-01-17 15:00:23 +00:00
Nick Craig-Wood
65071599a2 rest: don't canonicalise headers starting with * 
This leaves a way of adding headers which shouldn't be canonicalised.
 2020-01-17 15:00:23 +00:00
Nick Craig-Wood
5403e1c79a lib/encoder: remove noencode tag and update CONTRIBUTING 2020-01-17 15:00:01 +00:00
Nick Craig-Wood
5697caf20b Add Felix Hungenberg to contributors 2020-01-17 15:00:01 +00:00
Felix Hungenberg
68056f08ab docs: Use correct link for sftp wikipedia article 2020-01-17 13:20:42 +00:00
Nick Craig-Wood
81002747c5 dedupe: implement keep smallest too 
This is to help deduping google docs and their exported versions if
they accidentally get uploaded to the source again.

See: https://forum.rclone.org/t/my-stupidity-or-a-bug/13861
 2020-01-17 13:08:37 +00:00
Nick Craig-Wood
1bd9f522e0 build: compress the test builds 
This is to use less space and to fix the caddy/rclone interaction
using too much bandwidth.

See: https://caddy.community/t/how-to-stop-caddy-sniffing-mime-types-and-return-a-default-mime-type/6819
 2020-01-17 11:47:40 +00:00
Thomas Eales
3a1b41ac22 docs: fixed small typo 
is -> it
 2020-01-16 15:24:36 +00:00
Nick Craig-Wood
375d25f158 sync: implement --order-by flag to order transfers - fixes #1205 2020-01-16 15:24:36 +00:00
Nick Craig-Wood
0e57335396 docs: document new backend encoder parameter 2020-01-16 14:40:36 +00:00