Commit graph

1609 commits

Author SHA1 Message Date
Nick Craig-Wood
09fbbdbb04 Add Daniel Jagszent to contributors 2017-08-31 16:46:44 +01:00
Daniel Jagszent
4b0e983323 Local: Make documentation consistent with code
Change flag `--no-local-unicode-normalization` to `--local-no-unicode-normalization` since that's the way the flag is called in the source code.

Fixes #1633
2017-08-31 16:46:14 +01:00
wuyu
ee9f987234 qingstor: Support hash md5 for upload object
* Using single object to uploaded when files less than or equal to 67108864 bytes

 * Using multi-part object to uploaded when files large than 67108864 bytes, and
   calculate MD5SUMS in the upload process

 * For Mkdir and Rmdir, Add block to wait qingstor service sync status to
   handling extreme cases that try to create a just deleted bucket or delete
   a just created bucket etc
2017-08-31 16:41:08 +01:00
Nick Craig-Wood
f407e3da55 Add bpicode to contributors 2017-08-31 16:35:35 +01:00
bpicode
f1f7e0e6f9 support for zsh auto-completion - #983 2017-08-31 16:21:28 +01:00
bpicode
7e93567b18 vendor: update version of github.com/spf13/cobra for zsh support 2017-08-31 16:21:28 +01:00
Nick Craig-Wood
2c8d6e86cc fs: fix gofmt 2017-08-31 16:01:19 +01:00
cbruegg
bb6300b032 Fix bwlimit toggle in conjunction with schedules (Fixes #1607) 2017-08-31 15:33:29 +01:00
Nick Craig-Wood
e96c5b5f39 hubic: don't check the container exists before creating it
This fixes being able to create containers for Hubic.
2017-08-30 15:54:49 +01:00
Nick Craig-Wood
672c410235 Update to using go1.9 as the default go version
Get rid of Makefile spaghetti for avoiding vendor directory where
possible in make check.
2017-08-29 16:39:56 +01:00
Nick Craig-Wood
459cf64403 qingstor: fix errors in debug parameters noticed by go1.9 go vet 2017-08-29 14:19:14 +01:00
Stefan Breunig
0158ab6926 info: add check to stream files with unknown size 2017-08-22 08:00:10 +02:00
Stefan Breunig
4e189fe6e7 fstests: only test uploadswith indeterminate size on remotes that support it 2017-08-22 07:19:43 +02:00
Stefan Breunig
b78ecb1568 docs: add optional feature "streaming uploads" to overview table 2017-08-19 14:35:17 +02:00
Stefan Breunig
a122b9fa7a yandex: implement streaming uploads (see #1614) 2017-08-19 14:07:23 +02:00
Stefan Breunig
323daae63e http: immediately fail streaming uploads instead of spooling them first (see #1614) 2017-08-19 12:42:31 +02:00
Stefan Breunig
e754f50778 box: implement streaming uploads (see #1614) 2017-08-19 12:32:56 +02:00
Stefan Breunig
034cf22d4d Add Alex McGrath Kraak to contributors 2017-08-17 06:49:38 +02:00
Alex McGrath Kraak
2cc9071791 http: add --user-agent option. close #1557 2017-08-17 06:49:27 +02:00
Stefan Breunig
b510c70c1e b2: calculate missing hashes on the fly instead of spooling – fixes #1288 2017-08-12 12:57:34 +02:00
Stefan Breunig
001431d326 snapcraft: switch back to go build plugin and only build rclone – see #1188 2017-08-12 09:20:37 +02:00
Stefan Breunig
e64435a5c1 snapcraft: adjust snapcraft-dev build to allow fuse mounting – see #1188 2017-08-11 20:57:13 +02:00
Nick Craig-Wood
9c47b767b4 swift: Configure from environment vars and add endpoint_type - fixes #1542 2017-08-10 21:38:45 +01:00
Nick Craig-Wood
2870874329 azureblob: Read LastModified time of containers in root listing 2017-08-10 20:20:14 +01:00
Nick Craig-Wood
d54fca4e58 dropbox: fix entry doesn't belong in directory error - fixes #1558
This was caused by the unreliable casing in `path_lower` as returned
in the directory listings.  We now ignore everything except the last
element in `path_lower` which is guaranteed to have the correct case.
2017-08-10 13:57:06 +01:00
Nick Craig-Wood
dcbf538416 dropbox: stop using deprecated API methods 2017-08-10 13:57:06 +01:00
Nick Craig-Wood
5b79922b5e vendor: add dropbox/dropbox-sdk-go-unofficial 2017-08-10 13:57:06 +01:00
Nick Craig-Wood
41b2645dec vendor: remove ncw/dropbox-sdk-go-unofficial dependency 2017-08-10 13:57:05 +01:00
Nick Craig-Wood
76226e0147 dropbox: swap back to upstream dropbox/dropbox-sdk-go-unofficial
Now that dropbox/dropbox-sdk-go-unofficial#13 is fixed.
2017-08-10 13:57:05 +01:00
Nick Craig-Wood
76c5aa8533 gcs: Check for errors when testing bucket is OK in mkdir #1590
Previously we would check the bucket's status and on error we would
try to create it.  Now we only try to create it if we got a not found
error, otherwise we report the error to the user.
2017-08-10 10:29:21 +01:00
Nick Craig-Wood
265fb8a5e2 fs: Manage empty directories - fixes #100
During the sync we collect a list of directories which should be empty
and attempt to rmdir them at the end of the sync.  If the directories
are not empty then the rmdir will fail, logging a message but not
erroring the sync.
2017-08-09 21:07:00 +01:00
Nick Craig-Wood
8a1a900733 fstest: use Feature.CanHaveEmptyDirectories to sharpen tests
Now we actually test whether the directories are present or not,
filtering out empty directories in the test using the
CanHaveEmptyDirectories flag.
2017-08-09 20:55:08 +01:00
Nick Craig-Wood
20ae7d562b fs: Add CanHaveEmptyDirectories and BucketBased feature flags to all remotes 2017-08-09 20:55:08 +01:00
Nick Craig-Wood
c1bfdd893f vendor: update qingstor
dep ensure needed to do this, probably after various vendor merges
2017-08-09 13:03:07 +01:00
Nick Craig-Wood
ec2ea37ad2 fs: Add --disable flag to disable optional features - fixes #1551
Eg to disable server side copy use `--disable copy`, to see a list of
what you can disable, `--disable help`.
2017-08-07 21:34:45 +01:00
Nick Craig-Wood
bced73c947 sftp: fix compile for go1.6 2017-08-07 21:34:05 +01:00
Nick Craig-Wood
5b6585f57d sftp: limit new connections per second 2017-08-07 19:47:49 +01:00
Nick Craig-Wood
c6b844977a sftp: clear the cached hashes on object update 2017-08-07 17:36:59 +01:00
Nick Craig-Wood
47eab397ba sftp: implement connection pooling for multiple ssh connections
A connection may be opened for each `--transfers` and `--checkers`
now.  Connections are checked when putting them in the pool and
getting them out the pool so it should recover from network errors
much better.

This fixes #1561, fixes #1541, fixes #1381, fixes #1158, fixes #1538
2017-08-07 17:19:37 +01:00
Nick Craig-Wood
bfe812ea6b dedupe: implement merging of duplicate directories - fixes #1243 2017-08-07 15:36:41 +01:00
Nick Craig-Wood
db1995e63a Add MergeDirs optional interface and implement it for drive 2017-08-07 15:32:47 +01:00
Nick Craig-Wood
81a2ab599f fs: add optional ID to fs.Directory and set it in the remotes which care 2017-08-07 15:31:22 +01:00
Nick Craig-Wood
74687c25f5 sftp: fixup formatting and golint warnings 2017-08-07 14:50:31 +01:00
Nick Craig-Wood
d025066fae Add Christian Brüggemann to contributors 2017-08-06 11:50:20 +01:00
Christian Brüggemann
80ce569874 sftp: Add support for md5 and sha1 hashes where available 2017-08-06 11:49:52 +01:00
Nick Craig-Wood
ee13ea74f1 box: fix multipart upload giving "parts_mismatch" error #97 2017-08-05 21:01:32 +01:00
Stefan Breunig
40f24e0ea3 config: use absolute ConfigPath to ensure newly written config is on the same mount - fixes #1569 2017-08-05 12:13:25 +02:00
Stefan Breunig
b523cfc01d oauthutil: don't show "save failed" error when setting up new remote – fixes #1466 2017-08-05 12:04:42 +02:00
Nick Craig-Wood
38dabcf6b2 azure: correct docs on MD5 and chunked files 2017-08-04 23:54:57 +01:00
Nick Craig-Wood
ee6a35d750 Test compilation of all arches
* Add compile_all step to Makefile
  * Add this to travis
  * Add -compile-only flag to cross-compile.go to save time making the zips
2017-08-04 23:20:26 +01:00