Commit graph

3360 commits

Author SHA1 Message Date
Nick Craig-Wood
0386d22cc9 vfs: add test for 0 length files read in the way mount does 2019-08-03 18:25:44 +01:00
Nick Craig-Wood
0be14120e4 swift: use FixRangeOption to fix 0 length files via the VFS 2019-08-03 18:25:44 +01:00
Nick Craig-Wood
95af1f9ccf fs: fix FixRangeOption so it works with 0 length files 2019-08-03 18:25:44 +01:00
Nick Craig-Wood
629b7eacd8 b2: fix integration tests after accounting changes
In 53a1a0e3ef we started returning non nil from NewObject when
an object isn't found.  This breaks the integration tests and the API
expected of a backend.

This fixes it.
2019-08-03 13:30:31 +01:00
yparitcher
d3149acc32 b2: link sharing 2019-08-03 13:30:31 +01:00
Aleksandar Jankovic
6a3e301303 accounting: add call to clear stats
- Make calls more consistent by changing path to kebab case.
- Add stacktrace information to job panics
2019-08-02 16:56:19 +01:00
Nick Craig-Wood
5be968c0ca drive: update API for teamdrive use - fixes #3348 2019-08-02 16:06:23 +01:00
Nick Craig-Wood
f1a687c540 Add justina777 to contributors 2019-08-02 15:57:25 +01:00
justina777
94ee43fe54 log: add object and objectType to json logs 2019-08-02 15:57:09 +01:00
Nick Craig-Wood
c2635e39cc build: fix appveyor secure variables after project move 2019-07-28 22:46:26 +01:00
Nick Craig-Wood
8c511ec9cd docs: fix star count on website 2019-07-28 20:58:21 +01:00
Nick Craig-Wood
ac0dce78d0 cmd: fix up stats printing on macOS after accounting change 2019-07-28 20:38:20 +01:00
Nick Craig-Wood
f347514f62 build: fix up CI and CI badges after repo move 2019-07-28 20:07:04 +01:00
Nick Craig-Wood
57d5de6fba build: fix up package paths after repo move
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
2019-07-28 18:47:38 +01:00
Aleksandar Jankovic
4ba6532915 accounting: make stats response consistent
core/stats can return two different schemas in 'transferring' field.
One is object with fields the other is just plain string.
This is confusing, unnecessary and makes defining response schema
more difficult. It also returns `lastError` as value which can be
rendered differently depending on source of error.

This change standardizes 'transferring' filed to always return
object but with reduced fields if they are not available.
Former string item is converted to {name:remote_name} object.

'lastError' is forced to be a string as in some cases it can be encoded
as an object.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
ff235e4e56 docs: update documentation for stats 2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
68e641f6cf accounting: add limits and listing to stats groups 2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
53a1a0e3ef accounting: add reference to completed transfers
Add core/transferred call that lists completed transfers and their
status.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
8243ff8bc8 accounting: isolate stats to groups
Introduce stats groups that will isolate accounting for logically
different transferring operations. That way multiple accounting
operations can be done in parallel without interfering with each other
stats.

Using groups is optional. There is dedicated global stats that will be
used by default if no group is specified. This is operating mode for CLI
usage which is just fire and forget operation.

For running rclone as rc http server each request will create it's own
group. Also there is an option to specify your own group.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
be0464f5f1 accounting: change stats interface
This is done to make clear ownership over accounting object and prepare
for removing global stats object.

Stats elapsed time calculation has been altered to account for actual
transfer time instead of stats creation time.
2019-07-28 14:48:19 +01:00
Nick Craig-Wood
2d561b51db Add EliEron to contributors 2019-07-28 12:33:21 +01:00
Nick Craig-Wood
9241a93c2d Add justinalin to contributors 2019-07-28 12:33:21 +01:00
EliEron
fb32f77bac Docs: Fix typos in filtering demonstrations 2019-07-28 12:32:53 +01:00
justinalin
520fb03bfd log: add --use-json-log for JSON logging 2019-07-28 12:05:50 +01:00
justinalin
a3449bda30 vendor: add github.com/sirupsen/logrus 2019-07-28 12:05:50 +01:00
yparitcher
ccc416e62b b2: Fix link sharing #3314 2019-07-28 11:47:31 +01:00
jaKa
a35aa1360e Support setting modification times on Koofr backend.
Configuration time option to disable the above for if using Dropbox (does not
allow setting mtime on copy) or Amazon Drive (neither on upload nor on copy).
2019-07-24 21:11:58 +01:00
jaKa
3df9dbf887 vendor: updated github.com/koofr/go-koofrclient for set mtime support. 2019-07-24 21:11:58 +01:00
Nick Craig-Wood
9af0a704af Add Paul Millar to contributors 2019-07-24 20:34:39 +01:00
Nick Craig-Wood
691e5ae5f0 Add Yi FU to contributors 2019-07-24 20:34:39 +01:00
Nick Craig-Wood
5a44bafa4e fstest: add fs.ErrorCantShareDirectories for backends which can only share files 2019-07-24 20:34:29 +01:00
Nick Craig-Wood
8fdce31700 config: Fix hiding of options from the configurator 2019-07-24 20:34:29 +01:00
Nick Craig-Wood
493dfb68fd opendrive: refactor to use existing lib/rest facilities for uploads
This also checks the return of the call to make sure the number of
bytes written was as expected.
2019-07-24 20:34:29 +01:00
Nick Craig-Wood
71587344c6 lib/rest: allow Form upload with no file to upload 2019-07-24 20:34:29 +01:00
yparitcher
8e8b78d7e5 Implement --compare-dest & --copy-dest Fixes #3278 2019-07-22 19:42:29 +01:00
Nick Craig-Wood
266600dba7 build: reduce parallelism in cross compile to reduce memory and fix Travis
Before this change Travis builds were running out of memory when cross
compiling all the OSes.
2019-07-22 17:10:26 +01:00
Paul Millar
e4f6ccbff2 webdav: add docs for using bearer_token_command with oidc-agent 2019-07-22 16:01:55 +01:00
Nick Craig-Wood
1f1ab179a6 webdav: refresh token when it expires with --webdav-bearer-token-command
Fixes #2380
2019-07-22 16:01:55 +01:00
Nick Craig-Wood
c642531a1e webdav: add --webdav-bearer-token-command - fixes #2380
This can be used with oidc-agent to get a bearer token
2019-07-22 15:59:54 +01:00
buengese
19ae053168 rcserver: remove _async key from input parameters after parsing so later operations won't get confused - fixes #3346 2019-07-20 19:35:10 +02:00
buengese
def790986c fichier: make FolderID int and adjust related code - fixes #3359 2019-07-20 02:49:08 +02:00
Yi FU
0a1169e659 ssh: opt-in support for diffie-hellman-group-exchange-sha256 diffie-hellman-group-exchange-sha1 - fixes #1810 2019-07-13 12:21:56 +02:00
Nick Craig-Wood
5433021e8b drive: fix server side copy of big files
Before this change rclone was sending a MimeType in the requests for
server side Move and Copy.

The conjecture is that if you attempt to set the MimeType to something
different in a Copy then Google Drive has to do an actual copy of the
file data.  This takes a very long time (since it is large) and fails
after a 90s timeout.

After the change we no longer set the MimeType in Move or Copy and the
copies happen instantly and correctly.

Many thanks to @darthShadow for discovering that this was causing the
problem.

Fixes #3070
Fixes #3033
Fixes #3300
Fixes #3155
2019-07-05 10:49:19 +01:00
Nick Craig-Wood
c9f77719e4 b2: enable server side copy to copy between buckets - fixes #3303 2019-07-05 10:07:05 +01:00
Nick Craig-Wood
3cd63a00be googlephotos: fix configuration walkthrough 2019-07-04 15:19:59 +01:00
Nick Craig-Wood
d7016866e0 googlephotos: fix creation of duplicated albums
Also make sure we don't list the albums twice
2019-07-04 13:45:52 +01:00
yparitcher
d72e4105fb b2: Fix link sharing #3314 2019-07-04 11:53:59 +01:00
yparitcher
b4266da4eb sync: fix SyncSuffix tests #3272 2019-07-03 17:36:22 +01:00
yparitcher
3f5767b94e b2: Implement link sharing #2178 2019-07-03 14:10:25 +01:00
Nick Craig-Wood
1510e12659 build: move race test to go modules build
In an attempt to even out the build times.
2019-07-03 12:07:29 +01:00