TrueCloudLab/rclone
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
1470 commits 181 branches 114 tags 335 MiB
Commit graph

437 commits

Author SHA1 Message Date
Nick Craig-Wood
bfe812ea6b dedupe: implement merging of duplicate directories - fixes #1243 2017-08-07 15:36:41 +01:00
Nick Craig-Wood
db1995e63a Add MergeDirs optional interface and implement it for drive 2017-08-07 15:32:47 +01:00
Nick Craig-Wood
81a2ab599f fs: add optional ID to fs.Directory and set it in the remotes which care 2017-08-07 15:31:22 +01:00
Stefan Breunig
40f24e0ea3 config: use absolute ConfigPath to ensure newly written config is on the same mount - fixes #1569 2017-08-05 12:13:25 +02:00
Nick Craig-Wood
92d2e1f8d7 azureblob: rework and complete #801 
* Fixup bitrot (rclone and Azure library)
  * Implement Copy
  * Add modtime to metadata under mtime key as RFC3339Nano
  * Make multipart upload work
  * Make it pass the integration tests
  * Fix uploading of zero length blobs
  * Rename to azureblob as it seems likely we will do azurefile
  * Add docs
 2017-08-04 22:56:16 +01:00
Andrei Dragomir
036fd61a50 Added Azure Blob storage support #801 2017-08-04 22:54:27 +01:00
wuyu
ec5b72f8d5 Add new QingStor remote 
Add new package qingstor to support QingStor API.

Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.

Add new docs for it.
 2017-08-04 17:25:47 +01:00
Nick Craig-Wood
f682002b84 fs: Make tests create a new bucket rather than purging the old one 
This enables QingStor to pass the tests as it has a 2 minute lockout
on deleting the old bucket then creating it again.
 2017-08-04 17:09:28 +01:00
Stefan Breunig
28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood
3e3a59768e fs/test_all: fix after fstest factorisation 2017-08-03 20:01:05 +01:00
Nick Craig-Wood
e01741b557 fs: Cleaning up directories in test is no longer needed 
..as it is done in the finalise method.
 2017-08-03 20:01:05 +01:00
Nick Craig-Wood
71fe046937 fs: Add Find method to DirTree 2017-07-26 16:38:53 +01:00
Nick Craig-Wood
d5ff7104e5 fs: Implement NewDirTree for non --fast-list 2017-07-26 16:38:44 +01:00
Nick Craig-Wood
cd4895690a fstest: Factor test initialisation into Initialise() 2017-07-26 16:38:33 +01:00
Nick Craig-Wood
1ecf2bcbd5 fs: fix typo in --bind description 2017-07-23 23:08:33 +01:00
Nick Craig-Wood
c3d6cc91ec Fix --bind flag changes under go1.6 
Correcting 9f24639568
 2017-07-23 22:36:32 +01:00
Nick Craig-Wood
9f24639568 Add --bind flag for choosing the local addr on outgoing connections - fixes #1087 
Supported by all remotes except FTP.
 2017-07-23 16:27:39 +01:00
Nick Craig-Wood
7b81f12dad box: add docs 
* reorder remotes so they are in alphabetical order by full name everywhere
  * update CONTRIBUTING doc
 2017-07-23 11:32:34 +01:00
Nick Craig-Wood
d279161cee Implement box storage remote - #97 2017-07-23 11:32:34 +01:00
Nick Craig-Wood
db6009126d Fix test failure with new stretchr/testify - fixes #1550 2017-07-23 08:59:07 +01:00
Nick Craig-Wood
c8b5ee1e54 Start v1.37-DEV development 2017-07-22 20:43:06 +01:00
Nick Craig-Wood
a73ecec11f Version v1.37 2017-07-22 20:04:29 +01:00
Nick Craig-Wood
0cf19ef66a Make ListDirSorted check for subdirectories and write test 2017-07-19 09:36:27 +01:00
Nick Craig-Wood
655891170f Check in ListDirSorted that the directory entries all belong 2017-07-18 23:39:42 +01:00
Nick Craig-Wood
6f71260acf Add --tpslimit and --tpslimit-burst to limit transactions per second for HTTP 
This is useful if you are being rate limited or banned by your cloud
storage provider.
 2017-07-16 17:25:39 +01:00
Nick Craig-Wood
62e28d0a72 Replace token bucket limiter github.com/tsenart/tb with golang.org/x/time/rate 
In tests tsenart/tb has proved inaccurate at low rates.
 2017-07-16 16:14:44 +01:00
Nick Craig-Wood
72add5ab27 sync: state whether duplicates are objects are directories 2017-07-08 15:42:18 +01:00
Nick Craig-Wood
69ff009264 Use a stable sort for sorting directory entries 
This is useful if there are duplicates. Assuming the remote delivers
the entries in a consistent order, this will give the best user
experience in syncing as it will consistently use the first entry for
the sync comparison.
 2017-07-06 14:07:26 +01:00
Nick Craig-Wood
27b157580e Move make_test_files.go into bin 2017-07-06 11:54:57 +01:00
Nick Craig-Wood
f0512d1a52 Fix missing fs.Dir -> fs.Directory 2017-07-06 11:31:36 +01:00
Nick Craig-Wood
575e779b55 Warn about duplicate files when syncing - fixes #1506 
Error about unsorted directories and test thoroughly
 2017-06-30 21:24:13 +01:00
Nick Craig-Wood
e7d04fc103 Create fs.Directory interface and use it everywhere 2017-06-30 14:26:59 +01:00
Nick Craig-Wood
e2d7d413ef fs: rename BasicInfo to DirEntry 2017-06-30 14:26:58 +01:00
Nick Craig-Wood
e7e9aa0dfa fs: Remove unused ListFser interface 2017-06-30 14:26:58 +01:00
Nick Craig-Wood
f88300a153 Don't Mkdir at the start of sync - fixes #1131 
This is possible now that the bucket based remotes will create the
buckets on demand (9c1e703777).
 2017-06-29 12:31:53 +01:00
Nick Craig-Wood
e54087ece1 Fix config tests to save configData which fixes subsequent tests 2017-06-29 12:31:53 +01:00
Nick Craig-Wood
54561fd2bc s3: work around eventual consistency in bucket creation 
Deleting a bucket then testing its existence can give the wrong
result.  Work around by keeping a flag as to whether we have deleted
the bucket.
 2017-06-29 12:31:52 +01:00
Nick Craig-Wood
f3c7e1a9dd Debug directory creation and removal - fixes #1192 2017-06-27 22:19:35 +01:00
Nick Craig-Wood
aa20486485 Add --stats-log-level so can see --stats without -v - fixes #1180 
The most common use for this flag is likely to be showing the stats
without using -v by using `--stats-log-level NOTICE`.
 2017-06-26 22:50:37 +01:00
Stefan Breunig
52b042971a keep file permissions and try to keep user/group on supported systems (fixes #1467) 2017-06-25 09:05:24 +02:00
Stefan Breunig
2d2778eabf don't delete remote if name does not change while renaming (fixes #1495) 2017-06-25 08:55:54 +02:00
Vasiliy Tolstov
6b89e6c381 add new http remote filesystem 
Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
b76cd4abd2 Fix Range header option 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
4fd9570332 fs: Use an in place filter in ListDirSorted 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
5250fcdf08 core: fix data race in walk 
This was detected by the race detector when the client of Walk() sorted entries.
 2017-06-15 16:46:32 +01:00
Nick Craig-Wood
64662bef8d Deprecate --old-sync-method it is replaced with --fast-list 
Remove old sync method code.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
0b8d9084fc test_all: print command line so it can be cut and pasted into bash 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
8a6a8b9623 Change List interface and add ListR optional interface 
This simplifies the implementation of remotes.  The only required
interface is now `List` which is a simple one level directory list.

Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
6fc88ff32e Use --fast-list flag for sync/copy/move - fixes #1277 
Redo test framework to take a -fast-list flag and test remotes with that flag.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
50928a5027 Implement --fast-list flag. 
This is supported remotes which can do a recursive listing.  It will
use more memory.

This is related to #1277 but doesn't fix that issue yet.
 2017-06-14 16:49:40 +01:00