TrueCloudLab/rclone
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions
1469 commits 181 branches 114 tags 335 MiB
Commit graph

1469 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Craig-Wood
5250fcdf08 core: fix data race in walk 
This was detected by the race detector when the client of Walk() sorted entries.
 2017-06-15 16:46:32 +01:00
Edward Q. Bridges
9876ba53f8 Updated permissions 
As it happens, after testing the `GetObject` permission is also required to do `HEAD` requests on a given object.
 2017-06-14 17:29:21 +01:00
Nick Craig-Wood
64662bef8d Deprecate --old-sync-method it is replaced with --fast-list 
Remove old sync method code.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
0b8d9084fc test_all: print command line so it can be cut and pasted into bash 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
7be49249d3 Add lsjson command - fixes #1063 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
8a6a8b9623 Change List interface and add ListR optional interface 
This simplifies the implementation of remotes.  The only required
interface is now `List` which is a simple one level directory list.

Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
6fc88ff32e Use --fast-list flag for sync/copy/move - fixes #1277 
Redo test framework to take a -fast-list flag and test remotes with that flag.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
50928a5027 Implement --fast-list flag. 
This is supported remotes which can do a recursive listing.  It will
use more memory.

This is related to #1277 but doesn't fix that issue yet.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
3a431056e2 gcs, swift: increase directory listing chunk to 1000 to increase performance 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
53c3e5f0ab Add placeholder support for ListR interface. 
The ListR interface will be implemented by remotes that can do a
recursive directory listing more efficiently than just recursing
through the directories.  These include the bucket based remotes.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
0edb025257 Fixup tests with dirs vs bucket based fs 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
fded4dbea2 yandex: correct error return for listing empty directory 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
7e20e16cff core: Implement Walk directory listing and use in place of Lister 
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
1e88f0702a dropbox: fix oauth configuration 
This was broken in c59a292719
 2017-06-14 16:46:46 +01:00
Nick Craig-Wood
68333d34a1 dropbox: make setting mod time on existing files work properly 
This is a fix left over from the v2 conversion.  Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file.  This change deletes the existing file first before
re-uploading the new one.
 2017-06-13 13:58:39 +01:00
Nick Craig-Wood
740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood
28fcc53e45 mount test: retry umount as it fails occasionally 
This is because of the background releasing of files which happens
after all the files are closed.
 2017-06-13 10:52:10 +01:00
Nick Craig-Wood
2ca477c57f swift: make sensible error if the user forgets the container - fixes #1470 2017-06-10 14:44:56 +01:00
Nick Craig-Wood
9a11d3efd9 Revert "Start Cat tests from 2 as onedrive doesn't support ranging from 1" 
Now that https://github.com/OneDrive/onedrive-api-docs/issues/543 is
fixed, this can be reverted.

This reverts commit 320c53eab0.
 2017-06-10 13:48:00 +01:00
Nick Craig-Wood
10d5377ed8 acd: remove revoked credentials, allow oauth proxy config and update docs 2017-06-10 12:02:34 +01:00
Nick Craig-Wood
ee14efd3c2 config: fix menu selection when no remotes 2017-06-10 11:39:40 +01:00
Nick Craig-Wood
b4be7d65a6 Update build to go1.8.3 2017-06-09 12:06:28 +01:00
Nick Craig-Wood
52e1bfae2a oauth: Allow auth_url and token_url to be set in the config file 
If set in the config file, these override the ones configured into the
remote.  This enables alternative oauth servers to be used for all
oauth remotes.  This can only be altered by editing the config file
for the moment.
 2017-06-08 20:35:32 +01:00
Nick Craig-Wood
9c1e703777 swift, b2, gcs, s3: Fix moveto and copyto 
We now make sure the container/bucket is created before creating any objects.
 2017-06-07 14:34:59 +01:00
Nick Craig-Wood
b49821956a Fix copyto/moveto test error (see #1261) 2017-06-07 14:08:46 +01:00
Nick Craig-Wood
a61ba1e7c4 moveto, copyto: report transfers and checks as per move and copy 2017-06-07 13:02:21 +01:00
Nick Craig-Wood
d30cc1e119 Factor RemoteSplit into fs 2017-06-07 12:27:33 +01:00
Nick Craig-Wood
74a3dfc4e1 Fix TestHashSums 2017-06-06 23:21:47 +01:00
Nick Craig-Wood
3fe9448229 drive, acd, onedrive: Cache the directory IDs when reading the parent directory 
This makes directory listings much more efficient (one less
transaction needed) and also fixes #1439 (which was caused by having
to look up a directory name with quotes in which isn't dealt with well
by the list routine) by not doing a directory lookup at all.
 2017-06-05 12:26:30 +01:00
Nick Craig-Wood
a5cfdfd233 drive: add team drive support - fixes #885 2017-06-04 22:38:29 +01:00
Nick Craig-Wood
bdc19b7c8a fstests: fix -remote flag to override test target 2017-06-04 22:38:29 +01:00
Nick Craig-Wood
e92cc8fe2b Add Edward Q. Bridges to contributors 2017-06-04 22:38:10 +01:00
Edward Q. Bridges
6ee4c62cae Add section on required IAM permissions. 
cf.: https://github.com/ncw/rclone/issues/1455
 2017-06-04 22:37:17 +01:00
Nick Craig-Wood
b047402294 config: Fix save of temp file under Windows - fixes #1458 2017-06-01 16:38:19 +01:00
Nick Craig-Wood
7693cecd17 Add Fabian Möller to contributors 2017-06-01 16:23:48 +01:00
Fabian Möller
558f014d43 migrate Gopkg.toml and Gopkg.lock to new format 
Update Gopkg.toml and Gopkg.lock to follow the breaking changes
introduced by https://github.com/golang/dep/pull/644
 2017-06-01 16:23:13 +01:00
Nick Craig-Wood
48508cb5b7 Add Ruwbin to contributors 2017-06-01 09:03:56 +01:00
Ruwbin
44c98e8654 fix docs typos 2017-06-01 09:03:19 +01:00
Stefan Breunig
9782c264e9 hand dirCacheTime through again 2017-06-01 09:02:22 +01:00
Stefan
9cede6b372 fully write new config file before moving to target location (fixes #1287) 
* fully write new config file before moving to target location (fixes #1287)
* do not fail if there is no previous config; print temporary config path on failure
 2017-06-01 08:57:10 +01:00
Stefan Breunig
decd960867 make moveto/copyto no-ops when source and destination are the same (fixes #1261) 2017-05-30 23:01:19 +01:00
Nick Craig-Wood
71028e0f06 dropbox/dbhash: fix errcheck warning 2017-05-30 22:08:49 +01:00
Nick Craig-Wood
52e96bc0e2 dropbox: add missing dbhashsum command 
This was missed from 6381959850
 2017-05-30 19:26:06 +01:00
Nick Craig-Wood
178ff62d6a vendor: add github.com/ncw/dropbox-sdk-go-unofficial and remove github.com/stacktic/dropbox 
In due course this will become github.com/dropbox/dropbox-sdk-go-unofficial
when the fate of https://github.com/dropbox/dropbox-sdk-go-unofficial/pull/14
has been decided.
 2017-05-30 15:49:29 +01:00
Nick Craig-Wood
9d335eb5cb dropbox: add low level retries 2017-05-30 14:49:09 +01:00
Nick Craig-Wood
20da3e6352 Add options to Put, PutUnchecked and Update, add HashOption and speed up local 
* Add options to Put, PutUnchecked and Update for all Fses
  * Use these to create HashOption
  * Implement this in local
  * Pass the option in fs.Copy

This has the effect that we only calculate hashes we need to in the
local Fs which speeds up transfers significantly.
 2017-05-29 12:04:52 +01:00
Nick Craig-Wood
6381959850 dropbox: support Dropbox content hashing scheme - fixes #1302 
* add support to hashing module
  * add dbhashsum to list the hashes
  * add support to dropbox module

This means objects up and downloaded to/from Dropbox will have their
hashes checked.

Note after this change local objects are calculating MD5, SHA1 and
DBHASH which is excessive and needs to be fixed.
 2017-05-29 12:04:44 +01:00
Nick Craig-Wood
8916455e4f dropbox: implement dropbox hasher #1302 2017-05-29 12:04:34 +01:00
Nick Craig-Wood
8e214e838e dropbox: Update dropbox to use the v2 API #349 
This is feature complete with the old version but now supports modification times.
 2017-05-29 12:04:33 +01:00
Nick Craig-Wood
23acd3ce01 oauthutil: Don't expect tokens to have refresh URL 2017-05-29 12:04:33 +01:00