TrueCloudLab/rclone
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
5477 commits 181 branches 114 tags 335 MiB
Commit graph

5477 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Craig-Wood
1a82ba196b dircache: expose FoundRoot flag 2015-11-06 15:26:58 +00:00
Nick Craig-Wood
ed72c678f8 Protect accounting from being closed twice 2015-11-06 15:26:58 +00:00
Nick Craig-Wood
4ed8836a71 oauthutil: add RedirectPublicURL 2015-11-06 15:26:58 +00:00
Nick Craig-Wood
5529978fa7 dircache: make separate mutex for the cache 2015-11-06 15:26:58 +00:00
Nick Craig-Wood
66d84c9914 Document where to install root certificates - fixes #196 2015-11-05 18:09:56 +00:00
klauspost
b85ddc4e4f Extend CI tests to include formatting checks. 
CI tests now tests 'go vet', 'go fmt' (via goimports) and golint.

Adds Travis experimental OSX support.
 2015-11-03 13:50:29 +01:00
klauspost
e4a9e27a55 Add proxy information to FAQ. 
Fixes #160.
 2015-11-02 20:19:50 +00:00
Klaus Post
22645eea2e Add AppVeyor Windows CI to tests 
AppVeyor is free, and functions pretty much like Travis, only on Windows.
 2015-11-02 20:11:06 +00:00
klauspost
345c98ed62 Update to AWS SDK 0.10.0 
Tested with S3 and Dreamhost

Here is a link to the release notes:

http://aws.amazon.com/releasenotes/5476699172355228
 2015-11-02 19:52:11 +00:00
klauspost
b872ff0237 Add option to disable server certificate verification. 
The option name mirrors the 'wget' option (also `--no-check-certificate`). The cURL equivalent is called `--insecure`, which is a bit unclear.

Put in the "developers" section in documentation with proper warnings.

Fixes #168
 2015-10-29 16:42:25 +01:00
Nick Craig-Wood
1b95718460 Fix typos in filter docs and unit test assertions 2015-10-20 09:16:47 +01:00
klauspost
6a3580c556 Show status of master branch, so it doesn't show the status of the last pushed branch, 2015-10-19 18:56:03 +01:00
klauspost
16c9fba5de Fix tests failing on Windows. 
* ":" is kept when part of a drive.
  * Create tests.
  * Fix test framework.
 2015-10-19 17:36:15 +01:00
Nick Craig-Wood
4e952af614 Allow spaces in remotes and check remote names for validity at creation time - fixes #171 2015-10-12 17:54:09 +01:00
Klaus Post
6344c3051c Add async readahead buffer 
This adds an async read buffer of 4x4MB when copying files >10MB.

This fixes #164 and reduces the number of IO operations for copy/move.
 2015-10-12 08:30:27 +01:00
klauspost
ab9f521cbd Allow '&' and disallow ':' in Windows filenames. 
Fixes #161
 2015-10-05 11:04:25 +02:00
Nick Craig-Wood
3a900e5bb7 Version v1.23 2015-10-03 16:24:07 +01:00
Nick Craig-Wood
b4d7741611 Improve output of --dump-headers 2015-10-03 16:04:51 +01:00
Nick Craig-Wood
95fd79faf9 swift: use Content-Length on uploads - fixes #125 2015-10-03 16:04:51 +01:00
Nick Craig-Wood
b79dc01016 swift: stop chunked operations logging "Failed to read info: Object Not Found" 2015-10-03 16:04:51 +01:00
Nick Craig-Wood
bf562d7373 Tweak wording on client/secret ids in acd, gcs and drive - fixes #155 2015-10-03 16:04:45 +01:00
Nick Craig-Wood
2e9f2ea3d3 oauthutil: fix headless config for drive and gcs - fixes #158 2015-10-03 16:04:21 +01:00
Nick Craig-Wood
177dbbc29a Implement rclone size for measuring remotes - fixes #153 2015-10-03 16:04:21 +01:00
Nick Craig-Wood
4712043e26 Fix race condition in pacer tests - fixes #156 2015-10-03 16:04:21 +01:00
Nick Craig-Wood
852acd5e4e oauthutil: tell the user they should try again if the webserver method failed 2015-10-03 16:04:21 +01:00
Nick Craig-Wood
9f1daabb2c s3: allow anonymous access to public repositories - fixes #154 
When setting up the remote, leave both the access key and secret key
blank.
 2015-10-03 16:04:21 +01:00
Nick Craig-Wood
938dd24cc9 Fix typo 2015-09-28 22:51:33 +01:00
Nick Craig-Wood
57aad81b68 Version v1.22 2015-09-28 19:38:20 +01:00
Nick Craig-Wood
a91bcaaeb0 Implement rsync like include and exclude - fixes #27 
* Implement include/exclude
  * Implement rsync compatible file globbing
  * Implement command line filtering flags
    * --delete-excluded - Delete files on dest excluded from sync
    * --filter - Add a file-filtering rule
    * --filter-from - Read filtering patterns from a file
    * --exclude - Exclude files matching pattern
    * --exclude-from - Read exclude patterns from file
    * --include - Include files matching pattern
    * --include-from - Read include patterns from file
    * --files-from - Read list of source-file nam
    * --min-size - Don't transfer any file smaller than this in k or suffix k|M|G
    * --max-size - Don't transfer any file larger than this in k or suffix k|M|G
  * Document
 2015-09-28 19:18:21 +01:00
Nick Craig-Wood
d04c21b198 Add Sergey Tolmachev to contributors 2015-09-28 19:12:35 +01:00
Nick Craig-Wood
4a0a42c2f1 swift: large file upload fixes 
* Read metadata in file listing for 0 length files to fix syncs
  * Ignore non-existent files in isManifestFile to fix errors on copy
  * remove nsToSwiftFloatString - experiments with the swift program
    indicate that it puts a variable number of points after the
    decimal, so might as well use the one in the swift library.
  * Make sure segments get deleted properly when move from segmented
    to non segmented and vice versa
  * Use internal list routine to detect errors on listing
  * Remove the _segments container if possible
  * Remove manifest first when deleting
 2015-09-26 17:58:04 +01:00
Sergey Tolmachev
cc7b9af50e swift: support files > 5GB - fixes #46 
* Write segments to ..._segments container
  * Choice of container and segment names compatible with swift tool
    * See http://docs.openstack.org/developer/swift/overview_large_objects.html
  * Controlled by command line flag --swift-chunk-size
  * Segments removed on delete
 2015-09-26 13:03:58 +01:00
Nick Craig-Wood
68fef49c55 swift: fetch headers as the only source of metadata 2015-09-26 13:03:58 +01:00
Nick Craig-Wood
5d4b149884 Version v1.21 2015-09-22 21:05:11 +01:00
Nick Craig-Wood
5f20ae707d Make lsl output times in localtime and fix tests - fixes #141 2015-09-22 19:04:12 +01:00
Nick Craig-Wood
e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood
2ed158aba3 Fixes from go vet and errcheck 2015-09-22 07:31:12 +01:00
Nick Craig-Wood
05050d53ad Implement compliant pacing scheme for Amazon Cloud Drive 
* Implement switchable pacing algorithm
  * Add tests for pacer
 2015-09-21 21:49:54 +01:00
Nick Craig-Wood
e391311512 gofmt 2015-09-17 18:42:39 +01:00
klauspost
3234c28f7c Minor adjustments to Unc path conversion function. 2015-09-17 18:42:10 +01:00
klauspost
6fbd9cf24b Make Google Drive Directory reads concurrent for increased speed. 
If you use the -v flag then this will show the progress of directory
listing.  Fixes #143.
 2015-09-17 18:42:10 +01:00
klauspost
bc5b63ffef Move ACD Connection-limiter into Pacer. 2015-09-17 18:42:10 +01:00
klauspost
788ef76f1c Show more of the filename and align output. 
Print more of the file name, and make the output aligned, so it is nicer on frequent updates.
 2015-09-17 15:53:45 +02:00
Nick Craig-Wood
0872ec3204 Allow user to override credentials again in drive, gcs and acd - fixes #139 2015-09-16 20:08:40 +01:00
klauspost
0a5870208e Display individual transfer progress 
Improve progress printing by displaying individual file progress, as well
as a moving average speed with ETA. Example output:

2015/09/15 16:38:21
Transferred:    183599104 Bytes (4646.49 kByte/s)
Errors:                 0
Checks:                 1
Transferred:            0
Elapsed time:       38.5s
Transferring:
 * 01_06_14.mp3: 33% done. avg: 1280.5, cur: 1288.8 kByte/s. ETA: 1m12s
 * 01_12_15.mp3: 33% done. avg: 1002.2, cur:  943.4 kByte/s. ETA: 1m17s
 * 01_13_14.mp3: 48% done. avg: 1456.8, cur: 1425.2 kByte/s. ETA: 39s
 * 01_19_15.mp3: 28% done. avg: 1226.9, cur: 1114.4 kByte/s. ETA: 1m37s
 2015-09-16 19:42:46 +01:00
klauspost
3219334c3e Add "-race" test to travis. 2015-09-16 12:26:28 +02:00
klauspost
79fd662676 Protect concurrent read/writes to pacer. 
Protects all variables in the pacer from concurrent modifications. It is now safe to modify pacer settings while it is running.

I decided to not go for an RWMutex, since all accesses are very short, so the overhead of an RWMutex isn't worth it.

Fixes #138.
 2015-09-16 12:25:55 +02:00
Nick Craig-Wood
34193fd8d9 Version v1.20 2015-09-15 07:42:31 +01:00
Nick Craig-Wood
2203766f77 Retry listing in tests to work around eventual consistency 2015-09-14 21:01:25 +01:00
Nick Craig-Wood
235cbe0e57 amazon cloud drive: retry 409 errors too 2015-09-14 21:00:44 +01:00