TrueCloudLab/rclone
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions
694 commits 181 branches 114 tags 335 MiB
Commit graph

694 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Craig-Wood
8c2fc6daf8 s3: Add instructions on how to use rclone with minio 2016-07-11 12:12:28 +01:00
Nick Craig-Wood
4bd9932703 Fix wording in verbose copy logs - fixes #574 2016-07-09 10:11:57 +01:00
Nick Craig-Wood
2a1d4b7563 s3: Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions - fixes #567 2016-07-06 11:14:59 +01:00
Nick Craig-Wood
b394431f18 Improve --files-from docs - fixes #547 2016-07-05 12:33:59 +01:00
Nick Craig-Wood
cc628717d8 b2: Add --b2-versions flag so old versions can be listed and retreived. #420 2016-07-05 11:27:04 +01:00
Nick Craig-Wood
f3e00133a0 dropbox: Don't retry 461 errors - fixes #551 
461 errors from dropbox indicate some sort of copyright violation.
 2016-07-04 13:45:53 +01:00
Nick Craig-Wood
606961f49d b2: Treat 403 errors (eg cap exceeded) as fatal #420 2016-07-04 13:45:53 +01:00
Nick Craig-Wood
13591c7c00 Redo error handling for sync/copy/move 
* Factor sync/copy/move into its own file
  * Make fatal errors abort the sync
  * Make Copy return errors
  * Make Sync/Copy/Move return the last Copy error if there was one
  * Prioritise returning Fatal errors
  * NoRetry errors are returned if no other types of errors
 2016-07-04 13:45:53 +01:00
Nick Craig-Wood
28f4061892 Add two more classes of error Fatal and NoRetry 
These are for remotes to signal that they have a fatal error and don't
want to continue (eg cap exceeded) or that a particular file shouldn't
be retried for some reason.
 2016-07-04 13:45:52 +01:00
Nick Craig-Wood
018fe80bcb b2: cleanup old file versions - fixes #462 2016-07-02 17:03:08 +01:00
Nick Craig-Wood
0a43ff9c13 Modify interface for accounting to take a string not an fs.Object 2016-07-02 16:58:50 +01:00
Nick Craig-Wood
9aae143833 Implement cleanup command for emptying trash / removing old versions of files 2016-07-01 16:35:36 +01:00
Nick Craig-Wood
c8e2531c8b b2: make error handling compliant 2016-07-01 16:23:23 +01:00
Nick Craig-Wood
9290004bb8 pacer: make sleep get-able and set-able 2016-07-01 16:22:51 +01:00
Nick Craig-Wood
cbebefebc4 b2: Fix handling of token expiry #420 
Found with --b2-test-mode expire_some_account_authorization_tokens
 2016-07-01 11:47:42 +01:00
Nick Craig-Wood
6f3897ce2c b2: implement --b2-test-mode to set X-Bz-Test-Mode header #420 2016-07-01 11:30:09 +01:00
Nick Craig-Wood
ea5878f590 b2: set cutoff for chunked upload to 200MB #420 
This is the value recommended in the b2 integration checklist:

https://www.backblaze.com/b2/docs/integration_checklist.html
 2016-07-01 10:08:09 +01:00
Nick Craig-Wood
46f8e50614 b2: Make upload multi-threaded - fixes #531 2016-07-01 10:04:52 +01:00
Nick Craig-Wood
70dc97231e Convert more tests to use assert/require 2016-06-30 15:45:30 +01:00
Nick Craig-Wood
f6a053df6e Automatically set --no-traverse when copying a single file 2016-06-29 17:38:56 +01:00
Nick Craig-Wood
af4ef8ad8d Implement --no-traverse flag to stop copy traversing the destination remote. 
Refactor sync/copy/move
  * Don't load the src listing unless doing a sync and --delete-before
  * Don't load the dst listing if doing copy/move and --no-traverse is set

`rclone --no-traverse copy src dst` now won't load either of the
listings into memory so will use the minimum amount of memory.

This change will reduce the amount of memory rclone uses dramatically
too as in normal operations (copy without --notraverse or sync) as it
no longer loads the source file listing into memory at all.

Fixes #8
Fixes #544
Fixes #546
 2016-06-29 17:38:50 +01:00
Nick Craig-Wood
13797a1fb8 Make retry logs be debug in main copy routine 2016-06-28 08:51:57 +01:00
Nick Craig-Wood
3ad8fb8634 Make DeleteFile and DeleteFiles return errors 2016-06-28 08:51:57 +01:00
Nick Craig-Wood
ab43005422 Make NewObject return an error 
* make it return an error
  * make a canonical error fs.ErrorNotFound
  * make a test for it
  * remove logs/debugs of error
 2016-06-28 08:51:57 +01:00
Nick Craig-Wood
b1f131964e Rename NewFsObject to NewObject 2016-06-28 08:51:57 +01:00
Nick Craig-Wood
1a87b69376 Get rid of LimitedFs - FIXME needs docs on copying single files 
If remote:path points to a file make NewFs return a sentinel error
fs.ErrorIsFile and an Fs which points to the parent.

Use this to remove the LimitedFs and just add this file to the
--files-from list.

This means that server side operations can be used also.

Fixes #518
Fixes #545
 2016-06-28 08:51:43 +01:00
Nick Craig-Wood
5a3b109e25 Fix issues identified by go vet -shadow - fixes #530 2016-06-21 21:17:52 +01:00
Nick Craig-Wood
a67c7461ee s3: skip SetModTime for objects > 5GB - fixes #534 2016-06-19 17:26:44 +01:00
Klaus Post
e0aa4bb492 Fix incomplete local hashes. 
Fixes #533
 2016-06-19 16:51:49 +02:00
Nick Craig-Wood
ab0947ee37 Fix typo in changelog 2016-06-18 16:58:37 +01:00
Nick Craig-Wood
bd0227450e Version v1.30 2016-06-18 16:41:46 +01:00
Nick Craig-Wood
f438f1e9ef Fix stats print 2016-06-18 16:41:46 +01:00
Nick Craig-Wood
3f7b2c1ade Add Justin R. Wilson to contributors 2016-06-18 14:31:17 +01:00
Justin R. Wilson
6e35a3b3ce Add AES256 server-side encryption for s3 - Fixes #491 
Add a configuration key and support for AES256 server-side encryption.
 2016-06-18 14:28:38 +01:00
Nick Craig-Wood
d3dd672640 Document recursion requirements for Fses 2016-06-18 14:12:47 +01:00
Nick Craig-Wood
2a46be8cf3 b2: implement large file uploading - fixes #456 2016-06-18 13:38:05 +01:00
Nick Craig-Wood
1b4370bde1 Rework retry logic when copying objects 
* Fix off by one retry logic - fixes #406
  * Retry any retriable errors
  * Restructure code
 2016-06-18 10:55:58 +01:00
Nick Craig-Wood
cc6a776034 drive, acd: Tweak logging after changing Fs.Put so that it must cope with existing files 2016-06-18 10:54:42 +01:00
Nick Craig-Wood
2cfb3834f2 Log errors with %v 2016-06-18 09:36:47 +01:00
Nick Craig-Wood
46135d830e Add --ignore-size flag - fixes #399 2016-06-17 17:20:08 +01:00
Nick Craig-Wood
318e42e35b Add a section on quoting in the shell to the docs - fixes #473 2016-06-17 16:28:50 +01:00
Nick Craig-Wood
c7f04e24d3 Document that you can't repeat filter flags - fixes #506 2016-06-17 16:06:21 +01:00
Nick Craig-Wood
e4650eff58 drive: fix retry of multipart uploads - fixes #520 
Reset the reader on retry otherwise it is empty when read again.
 2016-06-15 21:48:30 +01:00
Nick Craig-Wood
869d91269d Debug cause of low level retries 2016-06-15 21:48:14 +01:00
Nick Craig-Wood
df1092ef33 Change Fs.Put so that it must cope with existing files 
This should fix duplicate files on drive and 409 errors on
amazonclouddrive however it will slow down the upload slightly as
another roundtrip will be needed.

None of the other Fses needed adjusting.

Fixes #483
 2016-06-13 19:29:10 +01:00
Nick Craig-Wood
4c5b2833b3 Convert to using github.com/pkg/errors everywhere 2016-06-13 17:43:03 +01:00
Nick Craig-Wood
7fe653c350 Unwrap errors properly for patform specific connection retry code. 
Include more possible errors for Windows.

For #442
 2016-06-10 13:48:41 +01:00
Nick Craig-Wood
661715733a Make sure we don't use conflicting content types on upload - fixes #513 2016-06-09 17:52:58 +01:00
Nick Craig-Wood
f17cb1bf50 Fix retry of Windows wsaend errors #442 
Make the test for wsaend error less specific
 2016-06-09 15:34:13 +01:00
Nick Craig-Wood
9ec06df79f Be explicit about which arch we support which fixes failure to build with new gox 2016-06-09 15:33:26 +01:00