Multiple directories (up to --checkers worth) are scanned at once.
This uses much less memory than the previous scheme - only the amount
of memory needed to hold an entire directory listing of objects.
For directory based remotes the speed is unchanged.
For bucket based remotes, instead of doing one API call to list the
whole bucket, it does multiple calls, one for each pseudo directory.
However these are done in parallel so in practice this seems to speed
up directory listings.
This replaces the existing sync method as it performs faster and uses
less memory.
The old sync method is available with the temporary --old-sync-method
flag.
Fixes#517Fixes#439Fixes#236Fixes#1067
Optional interfaces are becoming more important in rclone,
--track-renames and --backup-dir both rely on them.
Up to this point rclone has used interface upgrades to define optional
behaviour on Fs objects. However when one Fs object wraps another it
is very difficult for this scheme to work accurately. rclone has
relied on specific error messages being returned when the interface
isn't supported - this is unsatisfactory because it means you have to
call the interface to see whether it is supported.
This change enables accurate detection of optional interfaces by use
of a Features struct as returned by an obligatory Fs.Features()
method. The Features struct contains flags and function pointers
which can be tested against nil to see whether they can be used.
As a result crypt and hubic can accurately reflect the capabilities of
the underlying Fs they are wrapping.
This changes `ListFn`'s implementation so that if it encounters a not
found error, instead of sending a fatal error to log, it coordinates the
return of the error between checker goroutines and sends it back to the
caller.
The main impetus here is that it allows an external program compiling
against rclone as a package to handle a not found, where it currently it
cannot.
This does change error output on a not found a little bit, we go from
this:
2017/01/09 21:14:03 directory not found
To this:
2017/01/09 21:13:44 Failed to ls: directory not found
- Only start the token ticker when the timetable entry has more than one
entry.
- This fixes the "Scheduled bandwidth change" log message when no
bwlimit is specified.
- Fixes#987
These are set in the form RCLONE_CONFIG_remote_option where remote is
the uppercased remote name and option is the uppercased config file
option name. Note that RCLONE_CONFIG_remote_TYPE must be set if
defining a new remote.
Fixes#616
- Change the --bwlimit command line parameter to accept both a limit (as
before) or a full timetable (formatted as "hh:mm,limit
hh:mm,limit...")
- The timetable is checked once a minute by a ticker function. A new
tokenBucket is created every time a bandwidth change is necessary.
- This change is compatible with the SIGUSR2 change to toggle bandwidth
limits.
This resolves#221.
This commits adds support for tracking of file renames if `track-renames` flag is set,
and it then performs server-side renames for remotes that support it, i.e.
remotes that implement either the `Mover` or the `Copier` interface.
Fixes#888
Due to a logic error files stored on remotes which support modtime but
not hashes weren't being transferred when updating with a file of the
same size but different modtime. Instead the modtime of the remote
file was being set to that of the local file.
In practice this affected crypt with all remotes except Amazon Drive
and Dropbox.
