Commit graph

2329 commits

Author SHA1 Message Date
Nick Craig-Wood
d1bb8efb88 sftp: follow symlinks correctly - fixes #2145
The sftp library delivers the attributes of the symlink rather than
the object pointed to in directory listings, however when we use Stat
from the library it points to the objects.

Previous to this fix this caused items pointed to by symlinks to be
unusable.

After the fix both symlinked files and directories work as expected.
2018-03-16 15:36:47 +00:00
Nick Craig-Wood
c19e675ca6 vfs: unify locking for RWFileHandle.openPending,.close and File.Delete #2141
Without this fix the cached file can be removed as the file is being
uploaded or downloaded.  This can cause the directory listings to
become inconsistent (this issue) or data loss (if a retry was needed
in the Copy).

Remove file needs to be excluded from running at the same time as both
openPending and close so it makes sense to unify the locking between
all 3.
2018-03-15 20:49:07 +00:00
Nick Craig-Wood
34c45a7c04 mount, cmount: remove addition of O_CREATE to flags on file open #2141
Previously this was adding it in to all file opens which was causing
inefficiencies under Windows where it stats the file using
open/fstat/close.

This change will make stat operations run much quicker under Windows
as they won't have to open the underlying file

This problem was introduced in61b6159a05336bd7ba105766de2d2ff171f7fb81
where we added O_CREATE to all file opens and creates.
2018-03-15 20:48:56 +00:00
Nick Craig-Wood
0a0318df20 Add Leo R. Lundgren to contributors 2018-03-15 20:24:42 +00:00
Leo R. Lundgren
04e055fc06 sftp: Add --sftp-ask-password flag to prompt for password when needed - #2137 2018-03-15 20:24:30 +00:00
Nick Craig-Wood
d551137635 Add Giri Badanahatti to contributors 2018-03-15 20:21:12 +00:00
Giri Badanahatti
aba43cd3a4 Documention for IBM COS (S3) configuration. 2018-03-15 20:20:43 +00:00
Oliver Heyme
7f744033d8 onedrive: Removed upload cutoff and always do session uploads
Set modtime on copy


Added versioning issue to OneDrive documentation
2018-03-15 20:18:11 +00:00
remusb
078d705dbe cache: notify vfs and support crypt in rpc - #2111 2018-03-15 11:39:16 +02:00
Nick Craig-Wood
5981f9fab5 acd: disable integration tests
We no longer have any working keys for Amazon Cloud Drive so disable
the integration tests.
2018-03-14 22:44:46 +00:00
Alexander Neumann
84776c4e43 serve/restic: Remove log message on Close 2018-03-14 21:50:33 +00:00
Nick Craig-Wood
c1a3e363a6 mount: return ENOSYS rather than EIO on attempted link
This fixes FileZilla accessing an rclone mount served over sftp.

See: https://forum.rclone.org/t/moving-files-on-rclone-mount-with-filezilla/5029
2018-03-14 21:10:20 +00:00
Nick Craig-Wood
7ccc6080b0 serve restic: add more info to GET request error 2018-03-14 21:09:47 +00:00
remusb
677971643c cache: add support for rc 2018-03-14 22:58:20 +02:00
remusb
f4a1c1163c rc: update doc with supported params 2018-03-14 22:58:20 +02:00
remusb
97b48cf988 rc: add support for Go 1.6 2018-03-14 22:58:20 +02:00
Nick Craig-Wood
86e5a35491 Implement Remote Control for rclone #2111
This implements a remote control protocol activated with the --rc flag
and a new command `rclone rc` to use that interface.

Still to do
  * docs - need finishing
  * tests
2018-03-14 22:58:20 +02:00
Nick Craig-Wood
8bb2854fe4 httplib: allow the flags to be prefixed when instantiating a server 2018-03-14 22:58:20 +02:00
Remus Bunduc
d76da1f5fd cache: fix dir cache issue - #2117 2018-03-14 11:08:30 +02:00
Nick Craig-Wood
89748feaa5 s3: update docs to discourage use of v2 auth - fixes #2120
From testing it appears that CEPH no longer works properly with v2
auth and neither does Dreamhost, so update the docs anc configuration
to recommend v4 auth.
2018-03-13 20:47:29 +00:00
Nick Craig-Wood
dfd0f4c5a4 sync: when using --backup-dir don't delete files if we can't set their modtime
This is a problem when syncing a file which just needed its modtime
set with dropbox which can't set the mod time of a file without
re-uploading it.

Before this change we would delete the file, then the server side move
would fail moving the file to the backup-dir because it no longer
existed.

After this change the destination file is moved to the backup-dir
instead of being deleted and the new file is uploaded.

Fixes #2134
2018-03-13 16:05:06 +00:00
Nick Craig-Wood
0c9dc006c5 fs: make display of default values of --min-age/--max-age be off - Fixes #2121 2018-03-13 09:06:07 +00:00
Nick Craig-Wood
4e90ad04d5 serve restic: only accept v2 API requests for list 2018-03-11 17:35:01 +00:00
Nick Craig-Wood
43c7ea81df Add Alexander Neumann to contributors 2018-03-11 17:35:01 +00:00
Alexander Neumann
fa003e89b6 serve restic: When listing return empty list instead of 'null' 2018-03-11 14:48:46 +00:00
Alexander Neumann
5114b11d6f serve restic: add http2 server on stdin/stdout 2018-03-11 14:48:46 +00:00
Alexander Neumann
f832433fa5 serve restic: Return empty list for non-existing dirs 2018-03-11 14:48:43 +00:00
Nick Craig-Wood
d073efdc6c serve restic: serves a remote in restic REST API format 2018-03-11 14:43:03 +00:00
Nick Craig-Wood
9e48748182 httplib: Note that authentication is a good idea for non localhost 2018-03-11 14:38:54 +00:00
Nick Craig-Wood
b6058e0106 docs/install.sh: don't create root owned .config/rclone directory #2127 2018-03-10 11:09:13 +00:00
Nick Craig-Wood
66c69fe620 mount: wait longer for consistency after rm in tests 2018-03-09 23:15:38 +00:00
Nick Craig-Wood
a2336ad774 vfs: fix deadlock in mount tests
This was caused by this sequence of calls

1> file.Release
1> file.close  -> takes the file lock
2> vfs.waitforWriters
2> dir.walk -> takes the dir lock
1> file.setObject
1> dir.addObject -> attempts to take the dir lock - BLOCKS
2> file.activeWriters -> tries to take file lock - BLOCKS - DEADLOCK

The fix is to make activeWriters not take the file lock and use atomic
operations to read the number of writers instead.
2018-03-09 23:15:38 +00:00
Nick Craig-Wood
7713acf23d mount: skip failing test TestFileModTimeWithOpenWriters on Windows 2018-03-09 23:15:38 +00:00
Nick Craig-Wood
473a388f6d mount: disable failing test TestWriteFileDoubleClose on OSX 2018-03-09 23:15:37 +00:00
Nick Craig-Wood
c8a4d437a0 Make travis test mount and cmount - fixes #2100
Previously FUSE wasn't found in the container so these tests weren't
run.  Move to VM based testing and install FUSE dependencies.
2018-03-09 23:15:37 +00:00
Nick Craig-Wood
09c14af6d1 cmd: Fix go routines at exit message to make it less confusing 2018-03-09 17:15:48 +00:00
Jakub Tasiemski
acae10cd6f lsjson: add --encrypted to show encrypted name #1765 2018-03-09 08:44:02 +00:00
Nick Craig-Wood
0861207ace fstest/test_all: set cache backend wait time to 30m to fix integration tests 2018-03-08 21:14:09 +00:00
Nick Craig-Wood
a7dbf32c53 cache: Implement --cache-db-wait-time flag
This can be used to make the cache wait for other running cache
backends to finish rather than erroring after 1 second.
2018-03-08 21:14:09 +00:00
Nick Craig-Wood
6025bb6ad1 local: fix race conditions updating the hashes
This was causing occasional test failures for the -race test of mount
and cmount.
2018-03-08 21:08:41 +00:00
Remus Bunduc
70f07fd3ac
fs: add ChangeNotify and backend support for it (#2094)
* fs: rename DirChangeNotify to ChangeNotify

* cache: switch to ChangeNotify

* ChangeNotify: keep order of notifications
2018-03-08 22:03:34 +02:00
Nick Craig-Wood
b3f55d6bda vendor: Update github.com/Unknwon/goconfig to fix section listing
This fixes listing sections just after creation which means the rclone
config list will have all the keys in now.
2018-03-08 13:18:27 +00:00
Nick Craig-Wood
d9094f1a45 vendor: Gopkg.lock file format changes only after go dep update 2018-03-08 13:16:59 +00:00
Nick Craig-Wood
572ee5ec96 Sign the tags as part of the release process #1449 2018-03-07 15:18:13 +00:00
Nick Craig-Wood
316dac25c2 travis: add encrypted GITHUB_USER and GITHUB_TOKEN for using the API 2018-03-07 10:18:10 +00:00
Nick Craig-Wood
ee3c45676f bin/get-github-release.go: use GITHUB_USER/GITHUB_TOKEN when available
This should help with rate limiting problems when running under
travis.
2018-03-07 10:18:09 +00:00
Nick Craig-Wood
2e7e15461b bin/get-github-release.go: report body of HTTP responses with errors 2018-03-07 10:18:06 +00:00
Nick Craig-Wood
0175332987 vfs: fix applying modtime for an open Write Handle
The symptom of this was that the time set when the file was open was
lost.  This was causing one of the mount tests to fail too.
2018-03-06 21:58:11 +00:00
Nick Craig-Wood
85e0b87c99 build: add .deb and .rpm output for the build
This uses https://github.com/goreleaser/nfpm to create the .deb and
.rpm packages from the standard build output.
2018-03-06 12:37:44 +00:00
Nick Craig-Wood
d41017a277 A script to download and install the latest release of a github package 2018-03-06 12:37:44 +00:00