Commit graph

1003 commits

Author SHA1 Message Date
Nick Craig-Wood
94947f2523 Implement -L, --copy-links flag to allow rclone to follow symlinks
Fixes #40
2017-01-29 13:43:20 +00:00
Nick Craig-Wood
29c6e22024 mount: Make fsync be a no-op rather than returning an error - fixes #1045 2017-01-29 11:29:42 +00:00
Nick Craig-Wood
390f3cf35b crypt: Add --crypt-show-mapping to show encrypted file mapping
Fixes #1004
2017-01-29 10:14:17 +00:00
Nick Craig-Wood
20c033b484 Add a writing documentation section and update document 2017-01-29 09:47:47 +00:00
Nick Craig-Wood
8068ef96b6 Add Dario Giovannetti to contributors 2017-01-26 10:29:21 +00:00
Dario Giovannetti
9d36258923 Comply with XDG Base Directory specification
Fixes #868
2017-01-26 10:22:08 +00:00
Nick Craig-Wood
9fdeb82328 Fix tests under Windows 2017-01-20 17:12:05 +00:00
Nick Craig-Wood
2abfae283c crypt: fix crypt writer getting stuck in a loop #902
This happened when the underlying reader returned io.ErrUnexpectedEOF.
The error handling for the call to io.ReadFull failed to take this
into account.

io.ErrUnexpectedEOF is reasonably common when SSL connections go wrong
when communicating with ACD, so it manifested itself as transfers from
non-encrypted ACD to encrypted ACD getting stuck.
2017-01-20 16:00:55 +00:00
Nick Craig-Wood
b6848a3edb Fix race in Lister.Finished which was causing the tests to be unreliable 2017-01-19 20:11:17 +00:00
Nick Craig-Wood
e2bf9eb8e9 Implement --suffix for use with --backup-dir only #98
This also makes sure we remove files we are about to override in the
--backup-dir properly.
2017-01-19 20:11:17 +00:00
Nick Craig-Wood
a77659e47d Make directory listing checks more reliable and easier to read 2017-01-19 20:11:17 +00:00
Nick Craig-Wood
e9da14ac2e acd: After moving a file, wait for the file to no longer be in the directory
This fixes a Move followed quickly by a Copy updating the wrong file.
2017-01-19 20:11:17 +00:00
Nick Craig-Wood
a4bf22e620 b2: fix upload url not being refreshed properly #825 2017-01-17 17:34:21 +00:00
Nick Craig-Wood
a6b4065e13 mount: fix retry on network failure when reading off crypt - fixes #1042 2017-01-17 16:32:04 +00:00
Nick Craig-Wood
07ebf35987 Clarify what happens to files already in the --backup-dir DIR 2017-01-16 19:26:56 +00:00
Nick Craig-Wood
47ebd0789c Make "make quicktest" ignore a config file if present for local running
This means "make quicktest" should give the same result as when run by
Travis.
2017-01-16 17:54:18 +00:00
Nick Craig-Wood
166fd50451 oauthutil: copy the config before modifying it
The stops simultaneous use of oauth configs with different client IDs
causing a problem.
2017-01-16 17:33:25 +00:00
Nick Craig-Wood
0604d3dbf2 acd, onedrive: make sure we have found the root before purging
If we don't, purge can try to trash the root node which fortunately
doesn't succeed.
2017-01-16 17:33:25 +00:00
Nick Craig-Wood
1fa258c2b4 Define a new Features() method for Fs
Optional interfaces are becoming more important in rclone,
--track-renames and --backup-dir both rely on them.

Up to this point rclone has used interface upgrades to define optional
behaviour on Fs objects.  However when one Fs object wraps another it
is very difficult for this scheme to work accurately.  rclone has
relied on specific error messages being returned when the interface
isn't supported - this is unsatisfactory because it means you have to
call the interface to see whether it is supported.

This change enables accurate detection of optional interfaces by use
of a Features struct as returned by an obligatory Fs.Features()
method.  The Features struct contains flags and function pointers
which can be tested against nil to see whether they can be used.

As a result crypt and hubic can accurately reflect the capabilities of
the underlying Fs they are wrapping.
2017-01-16 17:33:25 +00:00
ncw
3745c526f1 Implement --backup-dir - fixes #98
The parameter of backup-dir specifies a remote that all deleted or
overwritten files will be copied to.
2017-01-16 17:33:25 +00:00
Nick Craig-Wood
c123c702ab Fix fs.Overlapping and factor fs.SameConfig 2017-01-14 09:55:53 +00:00
ncw
4aae7bcca6 Factor server side move detection 2017-01-14 09:55:53 +00:00
Nick Craig-Wood
aa62e93094 acd: fix panic when renaming files - fixes #973
Fixed by no longer overwriting the parameters in a retry loop
2017-01-14 09:50:45 +00:00
Nick Craig-Wood
45862f4c16 Add Brandur to contributors 2017-01-12 10:08:40 +00:00
Brandur
3b1e0b66bb Return error on not found from ListFn
This changes `ListFn`'s implementation so that if it encounters a not
found error, instead of sending a fatal error to log, it coordinates the
return of the error between checker goroutines and sends it back to the
caller.

The main impetus here is that it allows an external program compiling
against rclone as a package to handle a not found, where it currently it
cannot.

This does change error output on a not found a little bit, we go from
this:

    2017/01/09 21:14:03 directory not found

To this:

    2017/01/09 21:13:44 Failed to ls: directory not found
2017-01-12 10:07:59 +00:00
Nick Craig-Wood
a7d8ccd265 Add T.C. Ferguson to contributors 2017-01-10 13:22:51 +00:00
T.C. Ferguson
d4c923a5cc Add obscure command for generating encrypted passwords for rclone's config 2017-01-10 13:18:09 +00:00
Nick Craig-Wood
e426cb1d1a Add emyarod to contributors 2017-01-10 13:13:14 +00:00
emyarod
3c87a0d0dc Update remote docs to show correct setup process 2017-01-10 13:09:52 +00:00
0xJAKE
499766f6ab Update amazonclouddrive.md
Added details about Amazon Drive's latest trash retention policy.
See:
https://www.reddit.com/r/DataHoarder/comments/5dh96j/files_in_amazon_cloud_drive_trash_now_deleted/
https://www.amazon.com/gp/help/customer/display.html?nodeId=201376760
2017-01-08 10:24:03 -06:00
Nick Craig-Wood
35a6436983 mount: implement proper directory handling (mkdir, rmdir)
Before this change mount only simulated rmdir & mkdir, now it actually
runs mkdir & rmdir on the underlying remote, using the new parmaeters
to fs.Mkdir and fs.Rmdir.

Fixes #956
2017-01-06 11:24:22 +00:00
Nick Craig-Wood
341745d4d5 Update docs on server side copy 2017-01-05 21:11:46 +00:00
Nick Craig-Wood
78c1f2839e Fix filters to add ** rules to directory rules
This fixes `--exclude ".*{,/**}"` to exclude all . files and
. directories.
2017-01-05 19:33:49 +00:00
Nick Craig-Wood
de2d967abd Stop --track-renames hashing matching files - fixes #984
Also only hash files of the correct size.

This speeds it up a lot.
2017-01-05 17:58:01 +00:00
Marco Paganini
6611d92e21 Only start bandwidth ticker when necessary.
- Only start the token ticker when the timetable entry has more than one
  entry.
- This fixes the "Scheduled bandwidth change" log message when no
  bwlimit is specified.
- Fixes #987
2017-01-04 19:03:49 -08:00
Nick Craig-Wood
e1a49ca426 Document environment variable usage 2017-01-04 21:38:54 +00:00
Nick Craig-Wood
f73ee5eade Make all config file variables be settable in the environment
These are set in the form RCLONE_CONFIG_remote_option where remote is
the uppercased remote name and option is the uppercased config file
option name.  Note that RCLONE_CONFIG_remote_TYPE must be set if
defining a new remote.

Fixes #616
2017-01-03 22:42:47 +00:00
Nick Craig-Wood
0d75d2585f Allow all options to be set from environment variables
The option names are munged changing - to _ making upper case and
prepending RCLONE_.  The values are as parsed by pflag.
2017-01-03 22:42:47 +00:00
Marco Paganini
3b0f944e23 Add time-based bandwidth limits.
- Change the --bwlimit command line parameter to accept both a limit (as
  before) or a full timetable (formatted as "hh:mm,limit
  hh:mm,limit...")
- The timetable is checked once a minute by a ticker function. A new
  tokenBucket is created every time a bandwidth change is necessary.
- This change is compatible with the SIGUSR2 change to toggle bandwidth
  limits.

This resolves #221.
2017-01-03 21:00:38 +00:00
Nick Craig-Wood
aaeab58ce6 Add Lukas Loesche to contributors 2017-01-03 20:49:04 +00:00
Lukas Loesche
5894c02a34 Typo: the the -> the in docs and comments 2017-01-03 20:48:26 +00:00
Nick Craig-Wood
f1221b510b Change --track-renames to use the length,hash pair stored in a map
This makes it much faster in the case of many files and use less
memory.

This also detects use of --no-traverse and disables it.
2017-01-03 20:37:06 +00:00
Nick Craig-Wood
274ab349f4 sync: Only allow --track-renames if have a common hash 2017-01-03 20:35:05 +00:00
Nick Craig-Wood
86392fb800 Add Bjørn Erik Pedersen to contributors 2017-01-03 20:35:05 +00:00
Bjørn Erik Pedersen
adc156ab2a docs: Document track-renames option
See #888
2017-01-03 20:35:05 +00:00
Bjørn Erik Pedersen
47d3a450a4 sync: Track and perform server-side renames
This commits adds support for tracking of file renames if `track-renames` flag is set,
and it then performs server-side renames for remotes that support it, i.e.
remotes that implement either the `Mover` or the `Copier` interface.

Fixes #888
2017-01-03 20:35:05 +00:00
Nick Craig-Wood
5c89fd679d Fix incorrect vendoring for swift library
(vendored a feature branch by accident)
2017-01-03 17:39:56 +00:00
Nick Craig-Wood
1cad759306 Update vendor directory 2017-01-02 16:12:05 +00:00
Nick Craig-Wood
5b8b379feb Version v1.35 2017-01-02 15:33:06 +00:00
Nick Craig-Wood
f538fd8eb4 Update RELEASE procedure 2017-01-02 14:38:14 +00:00