TrueCloudLab/rclone
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
1757 commits 181 branches 114 tags 335 MiB
Commit graph

142 commits

Author SHA1 Message Date
Nick Craig-Wood
e53892f53b fs,drive,dropbox: Make and use new RepeatableReader variants to lower memory use 
RepeatableReaderSized has a pre-allocated buffer which should help
with memory usage - before it grew the buffer.  Since we know the size
of the chunks, pre-allocating it should be much more efficient.

RepeatableReaderBuffer uses the buffer passed in.

RepeatableLimit* are convenience funcitions for wrapping a reader in
an io.LimitReader and then a RepeatableReader with the same buffer
size.
 2017-11-23 13:53:46 +00:00
Nick Craig-Wood
6403242f48 drive, yandex: add missing CleanUpper interface check 2017-09-30 16:34:46 +01:00
Nick Craig-Wood
e0eabc75c0 drive: change the default for --drive-use-trash to true - fixes #1661 2017-09-15 11:58:50 +01:00
ishuah91
dd975ab00d drive: implement cleanup (empty trash) - addresses #575 2017-09-06 16:12:00 +01:00
Josiah White
445fc55772 Ignore return from patch request on failure. 2017-08-31 21:39:00 +01:00
Stefan Breunig
4e189fe6e7 fstests: only test uploadswith indeterminate size on remotes that support it 2017-08-22 07:19:43 +02:00
Nick Craig-Wood
20ae7d562b fs: Add CanHaveEmptyDirectories and BucketBased feature flags to all remotes 2017-08-09 20:55:08 +01:00
Nick Craig-Wood
db1995e63a Add MergeDirs optional interface and implement it for drive 2017-08-07 15:32:47 +01:00
Nick Craig-Wood
81a2ab599f fs: add optional ID to fs.Directory and set it in the remotes which care 2017-08-07 15:31:22 +01:00
Stefan Breunig
28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood
b5bf819256 acd,b2,crypt,drive: add missing upload options 2017-07-23 11:32:34 +01:00
Nick Craig-Wood
39d09c04a2 drive: Make --drive-trashed-only show all directories - fixes #1524 
Without showing all directories it doesn't show trashed files which
are in an untrashed directory.

This isn't an ideal fix, but it makes the feature useable.
 2017-07-22 10:03:27 +01:00
Nick Craig-Wood
92294a4a92 drive: Add --drive-trashed-only and remove obsolete --drive-full-list 
* Add --drive-trashed-only to show only the contents of the trash
  * Remove --drive-full-list as it is obsolete
  * Tidy the docs for the drive options
 2017-07-06 15:32:57 +01:00
Stefan Breunig
51866fbd34 drive: add missing seek to start on retries of chunked uploads 
follow up to ee13bc6775
 2017-07-05 18:52:04 +02:00
Stefan Breunig
ee13bc6775 drive: fix stats accounting for upload - fixes #970, #968 2017-07-04 19:56:46 +02:00
Nick Craig-Wood
6c3bf629a1 yandex: fix fs.Name() 
Put in tests for fs.Root() and fs.Name() for all remotes
 2017-07-03 13:39:31 +01:00
Nick Craig-Wood
e7d04fc103 Create fs.Directory interface and use it everywhere 2017-06-30 14:26:59 +01:00
Nick Craig-Wood
b44d0ea088 drive: convert / in names to a unicode equivalent (／) - fixes #62 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
8a6a8b9623 Change List interface and add ListR optional interface 
This simplifies the implementation of remotes.  The only required
interface is now `List` which is a simple one level directory list.

Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
3fe9448229 drive, acd, onedrive: Cache the directory IDs when reading the parent directory 
This makes directory listings much more efficient (one less
transaction needed) and also fixes #1439 (which was caused by having
to look up a directory name with quotes in which isn't dealt with well
by the list routine) by not doing a directory lookup at all.
 2017-06-05 12:26:30 +01:00
Nick Craig-Wood
a5cfdfd233 drive: add team drive support - fixes #885 2017-06-04 22:38:29 +01:00
Nick Craig-Wood
20da3e6352 Add options to Put, PutUnchecked and Update, add HashOption and speed up local 
* Add options to Put, PutUnchecked and Update for all Fses
  * Use these to create HashOption
  * Implement this in local
  * Pass the option in fs.Copy

This has the effect that we only calculate hashes we need to in the
local Fs which speeds up transfers significantly.
 2017-05-29 12:04:52 +01:00
Stefan Breunig
a2e3af0523 poll for Google Drive changes when mounted 2017-05-28 17:54:52 +01:00
Nick Craig-Wood
412dacf8be Add a test for partial reads to all remotes 2017-05-19 12:28:47 +01:00
Martin Kristensen
56f017c60c drive: use explicit fields for all endpoints 
Reuses the same fields for all endpoints for simplicitys sake.
Should solve remaining part of #1346
 2017-05-02 21:30:45 +01:00
Martin Kristensen
dd968a8ccf drive: nextPageToken field was missing 
Fixes the bug found by users in #1346
 2017-04-24 19:50:51 +01:00
Martin Kristensen
0d6e1afe54 drive: only request owner field when using --drive-auth-owner-only 
This fixes the note @ncw made in #1359
 2017-04-24 10:35:42 +01:00
Martin Kristensen
d7df065320 drive: reduce bandwidth by adding fields for partial responses 
Fixes #1346
 2017-04-23 17:01:15 +01:00
Nick Craig-Wood
4b4c59a4bb crypt: add integration tests for obfuscate name encryption 2017-03-29 17:57:10 +01:00
Danny Tsai
3bab119fa5 drive: implement --drive-shared-with-me flag to view shared with me files 2017-03-29 17:23:30 +01:00
Nick Craig-Wood
9b07d32c02 onedrive, drive, amazonclouddrive: make sure we find the root 
This fixes copyto copying things to the wrong place - fixes #1231
 2017-03-16 09:42:49 +00:00
Nick Craig-Wood
79e3c67bbd local, yandex, dropbox: fix NewObject suceeding on a directory #1079 
Add tests to make it consistent across all remotes
 2017-02-25 11:09:57 +00:00
Nick Craig-Wood
f3c5745468 Add srcRemote and dstRemote parameters to DirMove #954 2017-02-16 17:42:37 +00:00
Nick Craig-Wood
33c2873ae9 drive: Fix Rmdir on directories with trashed files - fixes #1040 
When we try to delete a directory which is empty other than with
trashed files, we trash the directory rather than deleting it.
 2017-02-16 12:29:37 +00:00
Nick Craig-Wood
0366ea39c5 Reassign some logging levels 2017-02-11 17:56:05 +00:00
Nick Craig-Wood
80f53176d9 Rename log functions and factor into own file 2017-02-11 17:54:50 +00:00
Nick Craig-Wood
28f9b9b611 drive: detect files using file size as well as md5 - fixes 980 2017-02-03 08:00:03 +00:00
Nick Craig-Wood
7679620f4b drive: Experimentally add --drive-list-chunk 2017-02-02 21:49:02 +00:00
okaresz
cbc113492a Add Drive specific option: --drive-skip-gdocs - fixes #1035 2017-01-29 20:53:51 +00:00
Nick Craig-Wood
1fa258c2b4 Define a new Features() method for Fs 
Optional interfaces are becoming more important in rclone,
--track-renames and --backup-dir both rely on them.

Up to this point rclone has used interface upgrades to define optional
behaviour on Fs objects.  However when one Fs object wraps another it
is very difficult for this scheme to work accurately.  rclone has
relied on specific error messages being returned when the interface
isn't supported - this is unsatisfactory because it means you have to
call the interface to see whether it is supported.

This change enables accurate detection of optional interfaces by use
of a Features struct as returned by an obligatory Fs.Features()
method.  The Features struct contains flags and function pointers
which can be tested against nil to see whether they can be used.

As a result crypt and hubic can accurately reflect the capabilities of
the underlying Fs they are wrapping.
 2017-01-16 17:33:25 +00:00
Nick Craig-Wood
0d75d2585f Allow all options to be set from environment variables 
The option names are munged changing - to _ making upper case and
prepending RCLONE_.  The values are as parsed by pflag.
 2017-01-03 22:42:47 +00:00
Nick Craig-Wood
c55402caa2 drive: create destination directory on Move() 2016-12-09 16:57:07 +00:00
Nick Craig-Wood
d132dc7640 drive: make DirMove more efficient and complain about moving the root 2016-12-09 16:57:07 +00:00
Nick Craig-Wood
48a2e3844d Add optional interface DirCacheFlush for making the tests more reliable 
This is defined for the users of dircache drive, onedrive, and acd.

This helps fix the DirMove tests on acd.
 2016-12-09 16:57:07 +00:00
Nick Craig-Wood
aaa1370a36 Add directory parameter to Rmdir and Mkdir #100 #831 
This will enable rclone to manage directories properly in the future.
 2016-11-26 12:02:53 +00:00
Nick Craig-Wood
441951a93b Stop removing failed upload to cloud storage remotes - fixes #559 
We do remove a partially written file on local so we don't have
corrupted files lying around.
 2016-11-04 21:34:25 +00:00
Nick Craig-Wood
4803ce010e Make exponential backoff work exactly as per google specification - fixes #583 2016-10-17 17:57:09 +01:00
Nick Craig-Wood
aef2ac5c04 Add options for Open and implement Range for all remotes 2016-10-05 21:03:56 +01:00
Nick Craig-Wood
945f49ab5e Make ContentType be preserved for cloud -> cloud copies - fixes #733 2016-10-03 20:02:04 +01:00
Nick Craig-Wood
c117eaf5a2 drive: add .epub, .odp and .tsv as export formats. 2016-09-19 18:08:10 +01:00