Commit graph

3523 commits

Author SHA1 Message Date
Nick Craig-Wood
b5ea6af6e4 build: revert back to go1.12 for the v1.49.x builds
The go1.13 build has had various problems reported so revert back to
go1.12 for the stable branch.

See: https://forum.rclone.org/t/1-49-4-plex-internal-errors-on-google-drive/12108
Fixes #3578
2019-10-02 13:53:52 +01:00
Nick Craig-Wood
d8729441db build: use the release builds not master of nfpm and github-release
Fixes #3580
2019-10-02 13:33:27 +01:00
Nick Craig-Wood
5ac39c2176 bin/get-github-release: support tar.bz2 files 2019-10-02 13:33:22 +01:00
Nick Craig-Wood
8aae04208b Version v1.49.4 2019-09-29 17:33:45 +01:00
Richard Patel
d9bdd0575e cmd/rcd: Address ZipSlip vulnerability
Don't create files outside of target
directory while unzipping.

Fixes #3529 reported by Nico Waisman at Semmle Security Team
2019-09-29 11:15:14 +01:00
Nick Craig-Wood
b3cafe8f06 s3: fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier
- Read the storage class for each object
- Implement SetTier/GetTier
- Check the storage class on the **object** before using SetModTime

This updates the fix in 1a2fb52 so that SetModTime works when you are
using objects which have been migrated to GLACIER but you aren't using
GLACIER as a storage class.

Fixes #3522
2019-09-29 10:54:37 +01:00
Nick Craig-Wood
a7a4666ddd oauthutil: fix security problem when running with two users on the same machine
Before this change two users could run `rclone config` for the same
backend on the same machine at the same time.

User A would get as far as starting the web server.  User B would then
fail to start the webserver, but it would open the browser on the
/auth URL which would redirect the user to the login.  This would then
cause user B to authenticate to user A's rclone.

This changes fixes the problem in two ways.

Firstly it passes the state to the /auth call before redirecting and
checks it there, erroring with a 403 error if it doesn't match.  This
would have fixed the problem on its own.

Secondly it delays the opening of the web browser until after the auth
webserver has started which prevents the user entering the credentials
if another auth server is running.

Fixes #3573
2019-09-29 10:53:24 +01:00
Nick Craig-Wood
54d409a7dd ftp: fix listing of an empty root returning: error dir not found
Before this change if rclone listed an empty root directory then it
would return an error dir not found.

After this change we assume the root directory exists and don't
attempt to check it which was failing before.

See: https://forum.rclone.org/t/ftp-empty-directory-yields-directory-not-found-error/12069/
2019-09-29 10:53:11 +01:00
Nick Craig-Wood
9054542be5 build: make VERSION file be master of the last release - fixes #3570
Prior to this beta releases would appear to be older than the point
release, eg v1.49.0-096-gc41812fc which was released after v1.49.3 and
contains all the patches from v1.49.3.
2019-09-29 10:52:47 +01:00
Nick Craig-Wood
4a8a8578a5 build: replace Circle CI build and make GitHub actions the default CI 2019-09-29 10:51:08 +01:00
Nick Craig-Wood
04da57fc68 build: remove Appveyor, Circle CI, Travis and Pkgr builds 2019-09-29 10:50:48 +01:00
Nick Craig-Wood
3aeb6a5a4c build: remove azure pipelines build 2019-09-29 10:49:33 +01:00
Nick Craig-Wood
c5d2da9a77 build: build rclone with github actions 2019-09-29 10:48:31 +01:00
Nick Craig-Wood
c89261bd99 accounting: fix file handle leak on errors - fixes #3547
In 53a1a0e3ef we introduced a problem where if there was an
error on the file being transferred then the file was re-opened and
the old one wasn't closed.

This was partially fixed in bfbddab46b however this didn't
address the case of the old file being closed.

This is now fixed by
- marking the file as open again in UpdateReader
- moving the stopping the accounting machinery to a new method Done
2019-09-19 16:21:20 +01:00
Nick Craig-Wood
1bdab29eab Version v1.49.3 2019-09-15 16:42:10 +01:00
Nick Craig-Wood
f77027e6b7 fs/accounting: Fix "file already closed" on transfer retries
This was caused by the recent reworking of the accounting interface.
The Transfer object was recycling the Accounting object without
resetting the stream.

See: https://forum.rclone.org/t/error-file-already-closed/11469/
See: https://forum.rclone.org/t/rclone-b2-sync-post-error-method-not-supported/11718/
2019-09-13 18:37:01 +01:00
Aleksandar Jankovic
f73d0eb920 accounting: fix total duration calculation
Fixes: #3498
2019-09-12 12:33:57 +01:00
Nick Craig-Wood
f1a9d821e4 Version v1.49.2 2019-09-08 16:48:54 +01:00
Nick Craig-Wood
5fe78936d5 test_all: write index.json and add branch, commit and Go version to report 2019-09-08 11:38:18 +01:00
Nick Craig-Wood
4f3eee8d65 build: make sure we add version info to test_all build 2019-09-08 11:38:11 +01:00
Nick Craig-Wood
f2c05bc239 operations: fix -u/--update with google photos / files of unknown size
Before this change if -u/--update was in effect we compared the size
of the files to see if the transfer should go ahead.  This was
comparing -1 with an actual size so the transfer always proceeded.

After this change we use the existing `sizeDiffers` function which
does the correct comparison with -1 for files of unknown length.

See: https://forum.rclone.org/t/sync-with-google-photos-to-local-drive-will-result-in-recoping/11605
2019-09-06 10:11:59 +01:00
Nick Craig-Wood
b463032901 accounting: fix locking in Transfer to avoid deadlock with --progress
Before this change, using -P occasionally deadlocked on the Transfer
mutex when Transfer.Done() was called with a non nil error and the
StatsInfo mutex since they mutually call each other.

This was fixed by making sure that the Transfer mutex is always
released before calling any StatsInfo methods.

This improves on: 6f87267b34

Fixes #3505
2019-09-06 10:10:53 +01:00
Nick Craig-Wood
358decb933 rc: fix docs for config/create /update /password 2019-09-03 08:33:56 +01:00
Nick Craig-Wood
cefa2df3b2 docs: add info on how to build and use the docker images 2019-09-02 14:31:19 +01:00
Alfonso Montero
52efb7e6d0 Add Docker workflow support #3460
* Use a multi-stage build to reduce final image size.
* Run 'quicktest' make target before building.
* Built binary won't run on Alpine unless statically linked.
2019-09-02 14:31:10 +01:00
Nick Craig-Wood
01fa6835c7 gcs: fix need for elevated permissions on SetModTime - fixes #3493
Before this change we used PATCH on the object to update the metadata.

Apparently this requires the "full_control" scope which Google were
unhappy with in their oauth review.

This changes it to update the metadata by copying the object ontop of
itself (which is the way s3 works).  This can be done with normal
permissions.
2019-09-02 12:04:45 +01:00
Cnly
8adf22e294 docs: fix template argument for mktemp in install.sh 2019-09-02 12:04:33 +01:00
Nick Craig-Wood
45f7c687e2 Version v1.49.1 2019-08-28 17:51:23 +01:00
Nick Craig-Wood
a05dd6fc27 config: Fix generated passwords being stored as empty password - Fixes #3492 2019-08-28 14:24:18 +01:00
Nick Craig-Wood
642cb03121 googlephotos,onedrive: fix crash on error response - fixes #3491
This fixes a crash on the google photos backend when an error is
returned from the rest.Call function.

This turned out to be a mis-understanding of the rest docs so
- improved rest.Call docs
- fixed mis-understanding in google photos backend
- fixed similar mis-understading in onedrive backend
2019-08-28 14:24:08 +01:00
Chaitanya
da4dfdc3ec rcd: Added missing parameter for web-gui info logs. 2019-08-28 14:24:04 +01:00
Nick Craig-Wood
a6387e1f81 Version v1.49.0 2019-08-26 15:25:20 +01:00
Nick Craig-Wood
a992a910ef rest: use readers.NoCloser to stop body being closed
Before this change, if you passed a io.ReadCloser to opt.Body then the
transaction would close it.  This happens as part of http.NewRequest
which documents that the io.Reader passed in will be upgraded to a
Closer if possible and closed as part of the Do call.

After this change, we wrap any io.ReadClosers to stop them being
upgraded.  This means that they will never get closed and that the
caller should always close them.

This fixes a panic in the googlephotos integration tests.
2019-08-26 12:23:31 +01:00
Nick Craig-Wood
ce3340621f lib/readers: add NoCloser to stop upgrades from io.Reader to io.ReadCloser 2019-08-26 12:23:31 +01:00
Nick Craig-Wood
73e010aff9 docs: make the config walkthroughs consistent for each backend 2019-08-26 10:47:17 +01:00
Nick Craig-Wood
a3faf98aa0 docs: add docs about GUI 2019-08-25 20:32:41 +01:00
Nick Craig-Wood
ed85092edb docs: remove social media tracking javascript and replace with links 2019-08-25 11:09:20 +01:00
Nick Craig-Wood
193c30d570 Review random string/password generation
- factor password generation into lib/random.Password
- call from appropriate places
- choose appropriate use of random.String vs random.Password
2019-08-25 11:09:19 +01:00
Nick Craig-Wood
beb8d5c134 docs: update analytics 2019-08-25 11:09:19 +01:00
Nick Craig-Wood
93810a739d docs: update fontawesome free to 5.10.2 and fixup broken images 2019-08-25 11:09:19 +01:00
Nick Craig-Wood
5d4d5d2b07 docs: update logo on website 2019-08-25 11:09:19 +01:00
Nick Craig-Wood
f02fc5d5b5 Add Andreas Chlupka to contributors 2019-08-25 11:09:19 +01:00
Andreas Chlupka
eab999f631 graphics: update rclone logos to new design
Committed-By: Nick Craig-Wood <nick@craig-wood.com>
2019-08-24 09:31:33 +01:00
Nick Craig-Wood
bd61eb89bc serve http/webdav/restic/rc: rename --prefix flag to --baseurl #3398
The name baseurl is widely accepted for this feature so I decided to
rename it before it made it into a stable release.
2019-08-24 09:10:50 +01:00
Nick Craig-Wood
077b45322d vfs: fix --vfs-cache-mode minimal,writes ignoring cached files
Before this change, with --vfs-cache-mode minimal,writes if files were
opened they would always be read from the remote, regardless of
whether they were in the cache or not.

This change checks to see if the file is in the cache when opening a
file with --vfs-cache-mode >= minimal and if so then it uses it from
the cache.

This makes --vfs-cache-mode writes in particular much more
efficient. No longer is a file uploaded (with write mode) then
immediately downloaded (with read only mode).

Fixes #3330
2019-08-23 13:58:15 +01:00
Nick Craig-Wood
67fae720d7 serve dlna: add more builtin mime types to cover standard audio/video
Add a minimal number of mime types to augment go's built in types
for environments which don't have access to a mime.types file (eg
Termux on android)

Fixes #3475
2019-08-23 13:30:48 +01:00
Nick Craig-Wood
39ae7c7ac0 serve dlna: fix missing mime types on Android causing missing videos
Before this fix serve dlna was only using the built in database of
mime types to look up the mime types of files.  On Android (and
possibly other systems) this is very small.

The symptoms of this problem was serve dlna only listing images and
not videos.

After this fix we use the backend's idea of the mime type if possible
which will be more accurate.

Fixes #3475
2019-08-23 13:30:48 +01:00
Nick Craig-Wood
f67798d73e Add Cenk Alti to contributors 2019-08-23 12:11:51 +01:00
Cenk Alti
a1ca65bd80 putio: add new backend 2019-08-23 12:11:36 +01:00
Cenk Alti
566aa0fca7 vendor: add github.com/putdotio/go-putio for putio client 2019-08-23 12:11:36 +01:00