Commit graph

1917 commits

Author SHA1 Message Date
Nick Craig-Wood
c54d513bdd Add ishuah91 to contributors 2017-09-06 16:12:29 +01:00
ishuah91
dd975ab00d drive: implement cleanup (empty trash) - addresses #575 2017-09-06 16:12:00 +01:00
Nick Craig-Wood
2944f7603d s3: read 1000 items in listings #1653
This fixes directory listings with wasabi which fail if you supply
more than the allowed 1000 items as a parameter.  rclone used to
supply 1024 items which exceeds the spec - this works fine with
s3/ceph/etc but fails with wasabi.
2017-09-06 11:13:28 +01:00
Nick Craig-Wood
58f7b4ed7c Clarify --filter-from docs 2017-09-01 11:35:26 +01:00
Nick Craig-Wood
cbea06026a Make check obey --ignore-size - fixes #1643 2017-09-01 11:20:41 +01:00
Nick Craig-Wood
8207af9460 b2: Fix SHA1 mismatch when downloading files with no SHA1 #678
Some large files (depending on which version of rclone they were
uploaded with and where they were uploaded from) don't have an SHA1,
so we can't check it in that case.
2017-08-31 21:39:41 +01:00
Nick Craig-Wood
921fcc0723 Add Josiah White to contributors 2017-08-31 21:39:41 +01:00
Josiah White
445fc55772 Ignore return from patch request on failure. 2017-08-31 21:39:00 +01:00
Nick Craig-Wood
09fbbdbb04 Add Daniel Jagszent to contributors 2017-08-31 16:46:44 +01:00
Daniel Jagszent
4b0e983323 Local: Make documentation consistent with code
Change flag `--no-local-unicode-normalization` to `--local-no-unicode-normalization` since that's the way the flag is called in the source code.

Fixes #1633
2017-08-31 16:46:14 +01:00
wuyu
ee9f987234 qingstor: Support hash md5 for upload object
* Using single object to uploaded when files less than or equal to 67108864 bytes

 * Using multi-part object to uploaded when files large than 67108864 bytes, and
   calculate MD5SUMS in the upload process

 * For Mkdir and Rmdir, Add block to wait qingstor service sync status to
   handling extreme cases that try to create a just deleted bucket or delete
   a just created bucket etc
2017-08-31 16:41:08 +01:00
Nick Craig-Wood
f407e3da55 Add bpicode to contributors 2017-08-31 16:35:35 +01:00
bpicode
f1f7e0e6f9 support for zsh auto-completion - #983 2017-08-31 16:21:28 +01:00
bpicode
7e93567b18 vendor: update version of github.com/spf13/cobra for zsh support 2017-08-31 16:21:28 +01:00
Nick Craig-Wood
2c8d6e86cc fs: fix gofmt 2017-08-31 16:01:19 +01:00
cbruegg
bb6300b032 Fix bwlimit toggle in conjunction with schedules (Fixes #1607) 2017-08-31 15:33:29 +01:00
Nick Craig-Wood
e96c5b5f39 hubic: don't check the container exists before creating it
This fixes being able to create containers for Hubic.
2017-08-30 15:54:49 +01:00
Nick Craig-Wood
672c410235 Update to using go1.9 as the default go version
Get rid of Makefile spaghetti for avoiding vendor directory where
possible in make check.
2017-08-29 16:39:56 +01:00
Nick Craig-Wood
459cf64403 qingstor: fix errors in debug parameters noticed by go1.9 go vet 2017-08-29 14:19:14 +01:00
Stefan Breunig
0158ab6926 info: add check to stream files with unknown size 2017-08-22 08:00:10 +02:00
Stefan Breunig
4e189fe6e7 fstests: only test uploadswith indeterminate size on remotes that support it 2017-08-22 07:19:43 +02:00
Stefan Breunig
b78ecb1568 docs: add optional feature "streaming uploads" to overview table 2017-08-19 14:35:17 +02:00
Stefan Breunig
a122b9fa7a yandex: implement streaming uploads (see #1614) 2017-08-19 14:07:23 +02:00
Stefan Breunig
323daae63e http: immediately fail streaming uploads instead of spooling them first (see #1614) 2017-08-19 12:42:31 +02:00
Stefan Breunig
e754f50778 box: implement streaming uploads (see #1614) 2017-08-19 12:32:56 +02:00
Stefan Breunig
034cf22d4d Add Alex McGrath Kraak to contributors 2017-08-17 06:49:38 +02:00
Alex McGrath Kraak
2cc9071791 http: add --user-agent option. close #1557 2017-08-17 06:49:27 +02:00
Stefan Breunig
b510c70c1e b2: calculate missing hashes on the fly instead of spooling – fixes #1288 2017-08-12 12:57:34 +02:00
Stefan Breunig
001431d326 snapcraft: switch back to go build plugin and only build rclone – see #1188 2017-08-12 09:20:37 +02:00
Stefan Breunig
e64435a5c1 snapcraft: adjust snapcraft-dev build to allow fuse mounting – see #1188 2017-08-11 20:57:13 +02:00
Nick Craig-Wood
9c47b767b4 swift: Configure from environment vars and add endpoint_type - fixes #1542 2017-08-10 21:38:45 +01:00
Nick Craig-Wood
2870874329 azureblob: Read LastModified time of containers in root listing 2017-08-10 20:20:14 +01:00
Nick Craig-Wood
d54fca4e58 dropbox: fix entry doesn't belong in directory error - fixes #1558
This was caused by the unreliable casing in `path_lower` as returned
in the directory listings.  We now ignore everything except the last
element in `path_lower` which is guaranteed to have the correct case.
2017-08-10 13:57:06 +01:00
Nick Craig-Wood
dcbf538416 dropbox: stop using deprecated API methods 2017-08-10 13:57:06 +01:00
Nick Craig-Wood
5b79922b5e vendor: add dropbox/dropbox-sdk-go-unofficial 2017-08-10 13:57:06 +01:00
Nick Craig-Wood
41b2645dec vendor: remove ncw/dropbox-sdk-go-unofficial dependency 2017-08-10 13:57:05 +01:00
Nick Craig-Wood
76226e0147 dropbox: swap back to upstream dropbox/dropbox-sdk-go-unofficial
Now that dropbox/dropbox-sdk-go-unofficial#13 is fixed.
2017-08-10 13:57:05 +01:00
Nick Craig-Wood
76c5aa8533 gcs: Check for errors when testing bucket is OK in mkdir #1590
Previously we would check the bucket's status and on error we would
try to create it.  Now we only try to create it if we got a not found
error, otherwise we report the error to the user.
2017-08-10 10:29:21 +01:00
Nick Craig-Wood
265fb8a5e2 fs: Manage empty directories - fixes #100
During the sync we collect a list of directories which should be empty
and attempt to rmdir them at the end of the sync.  If the directories
are not empty then the rmdir will fail, logging a message but not
erroring the sync.
2017-08-09 21:07:00 +01:00
Nick Craig-Wood
8a1a900733 fstest: use Feature.CanHaveEmptyDirectories to sharpen tests
Now we actually test whether the directories are present or not,
filtering out empty directories in the test using the
CanHaveEmptyDirectories flag.
2017-08-09 20:55:08 +01:00
Nick Craig-Wood
20ae7d562b fs: Add CanHaveEmptyDirectories and BucketBased feature flags to all remotes 2017-08-09 20:55:08 +01:00
Nick Craig-Wood
c1bfdd893f vendor: update qingstor
dep ensure needed to do this, probably after various vendor merges
2017-08-09 13:03:07 +01:00
Nick Craig-Wood
ec2ea37ad2 fs: Add --disable flag to disable optional features - fixes #1551
Eg to disable server side copy use `--disable copy`, to see a list of
what you can disable, `--disable help`.
2017-08-07 21:34:45 +01:00
Nick Craig-Wood
bced73c947 sftp: fix compile for go1.6 2017-08-07 21:34:05 +01:00
Nick Craig-Wood
5b6585f57d sftp: limit new connections per second 2017-08-07 19:47:49 +01:00
Nick Craig-Wood
c6b844977a sftp: clear the cached hashes on object update 2017-08-07 17:36:59 +01:00
Nick Craig-Wood
47eab397ba sftp: implement connection pooling for multiple ssh connections
A connection may be opened for each `--transfers` and `--checkers`
now.  Connections are checked when putting them in the pool and
getting them out the pool so it should recover from network errors
much better.

This fixes #1561, fixes #1541, fixes #1381, fixes #1158, fixes #1538
2017-08-07 17:19:37 +01:00
Nick Craig-Wood
bfe812ea6b dedupe: implement merging of duplicate directories - fixes #1243 2017-08-07 15:36:41 +01:00
Nick Craig-Wood
db1995e63a Add MergeDirs optional interface and implement it for drive 2017-08-07 15:32:47 +01:00
Nick Craig-Wood
81a2ab599f fs: add optional ID to fs.Directory and set it in the remotes which care 2017-08-07 15:31:22 +01:00