From f96ce5674b35a457b0542adaece885a23e42be7a Mon Sep 17 00:00:00 2001 From: Nick Craig-Wood Date: Mon, 15 Oct 2018 11:03:08 +0100 Subject: [PATCH] Version v1.44 --- MANUAL.html | 3818 +++++++++-- MANUAL.md | 3786 +++++++++-- MANUAL.txt | 3673 +++++++++-- bin/make_changelog.py | 2 +- docs/content/b2.md | 2 +- docs/content/changelog.md | 101 +- docs/content/commands/rclone.md | 575 +- docs/content/commands/rclone_about.md | 526 +- docs/content/commands/rclone_authorize.md | 526 +- docs/content/commands/rclone_cachestats.md | 526 +- docs/content/commands/rclone_cat.md | 526 +- docs/content/commands/rclone_check.md | 526 +- docs/content/commands/rclone_cleanup.md | 526 +- docs/content/commands/rclone_config.md | 526 +- docs/content/commands/rclone_config_create.md | 524 +- docs/content/commands/rclone_config_delete.md | 524 +- docs/content/commands/rclone_config_dump.md | 524 +- docs/content/commands/rclone_config_edit.md | 524 +- docs/content/commands/rclone_config_file.md | 524 +- .../commands/rclone_config_password.md | 524 +- .../commands/rclone_config_providers.md | 524 +- docs/content/commands/rclone_config_show.md | 524 +- docs/content/commands/rclone_config_update.md | 524 +- docs/content/commands/rclone_copy.md | 526 +- docs/content/commands/rclone_copyto.md | 526 +- docs/content/commands/rclone_copyurl.md | 526 +- docs/content/commands/rclone_cryptcheck.md | 526 +- docs/content/commands/rclone_cryptdecode.md | 526 +- docs/content/commands/rclone_dbhashsum.md | 526 +- docs/content/commands/rclone_dedupe.md | 526 +- docs/content/commands/rclone_delete.md | 526 +- docs/content/commands/rclone_deletefile.md | 526 +- .../commands/rclone_genautocomplete.md | 526 +- .../commands/rclone_genautocomplete_bash.md | 524 +- .../commands/rclone_genautocomplete_zsh.md | 524 +- docs/content/commands/rclone_gendocs.md | 526 +- docs/content/commands/rclone_hashsum.md | 526 +- docs/content/commands/rclone_link.md | 526 +- docs/content/commands/rclone_listremotes.md | 526 +- docs/content/commands/rclone_ls.md | 526 +- docs/content/commands/rclone_lsd.md | 526 +- docs/content/commands/rclone_lsf.md | 526 +- docs/content/commands/rclone_lsjson.md | 526 +- docs/content/commands/rclone_lsl.md | 526 +- docs/content/commands/rclone_md5sum.md | 526 +- docs/content/commands/rclone_mkdir.md | 526 +- docs/content/commands/rclone_mount.md | 536 +- docs/content/commands/rclone_move.md | 526 +- docs/content/commands/rclone_moveto.md | 526 +- docs/content/commands/rclone_ncdu.md | 526 +- docs/content/commands/rclone_obscure.md | 526 +- docs/content/commands/rclone_purge.md | 526 +- docs/content/commands/rclone_rc.md | 526 +- docs/content/commands/rclone_rcat.md | 526 +- docs/content/commands/rclone_rmdir.md | 526 +- docs/content/commands/rclone_rmdirs.md | 526 +- docs/content/commands/rclone_serve.md | 527 +- docs/content/commands/rclone_serve_ftp.md | 469 ++ docs/content/commands/rclone_serve_http.md | 526 +- docs/content/commands/rclone_serve_restic.md | 524 +- docs/content/commands/rclone_serve_webdav.md | 526 +- docs/content/commands/rclone_settier.md | 325 + docs/content/commands/rclone_sha1sum.md | 526 +- docs/content/commands/rclone_size.md | 526 +- docs/content/commands/rclone_sync.md | 526 +- docs/content/commands/rclone_touch.md | 526 +- docs/content/commands/rclone_tree.md | 526 +- docs/content/commands/rclone_version.md | 526 +- docs/content/rc.md | 51 + docs/layouts/partials/version.html | 2 +- fs/version.go | 2 +- rclone.1 | 5570 +++++++++++++++-- 72 files changed, 32422 insertions(+), 16975 deletions(-) create mode 100644 docs/content/commands/rclone_serve_ftp.md create mode 100644 docs/content/commands/rclone_settier.md diff --git a/MANUAL.html b/MANUAL.html index 9648c259f..bb9b50815 100644 --- a/MANUAL.html +++ b/MANUAL.html @@ -12,7 +12,7 @@

Rclone

Logo

@@ -63,14 +63,15 @@
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, eg two different cloud accounts
    • -
  • Optional encryption (Crypt)
    • -
  • Optional cache (Cache)
    • +
  • (Encryption) backend
    • +
  • (Cache) backend
    • +
  • (Union) backend
  • Optional FUSE mount (rclone mount)

    • Links

    Environment Variables

    Rclone can be configured entirely using environment variables. These can be used to set defaults for options or config file entries.

    -

    Options

    +

    Options

    Every option in rclone can have its default set by environment variable.

    To find the name of the environment variable, first, take the long option name, strip the leading --, change - to _, make upper case and prepend RCLONE_.

    For example, to always set --stats 5s, set the environment variable RCLONE_STATS=5s. If you set stats on the command line this will override the environment variable setting.

    @@ -2202,6 +2302,14 @@ dir1/dir2/dir3/.ignore

    Eg

    rclone rc cache/expire remote=path/to/sub/folder/
 rclone rc cache/expire remote=/ withData=true
    +

    cache/fetch: Fetch file chunks

    +

    Ensure the specified file chunks are cached on disk.

    +

    The chunks= parameter specifies the file chunks to check. It takes a comma separated list of array slice indices. The slice indices are similar to Python slices: start[:end]

    +

    start is the 0 based chunk number from the beginning of the file to fetch inclusive. end is 0 based chunk number from the beginning of the file to fetch exclisive. Both values can be negative, in which case they count from the back of the file. The value "-5:" represents the last 5 chunks of a file.

    +

    Some valid examples are: ":5,-5:" -> the first and last five chunks "0,-2" -> the first and the second last chunk "0:10" -> the first ten chunks

    +

    Any parameter with a key that starts with "file" can be used to specify files to fetch, eg

    +
    rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye
    +

    File names will automatically be encrypted when the a crypt remote is used on top of the cache.

    cache/stats: Get cache stats

    Show statistics for the cache remote.

    core/bwlimit: Set the bandwidth limit.

    @@ -2231,6 +2339,8 @@ rclone rc core/bwlimit rate=off "speed": average speed in bytes/sec since start of the process, "bytes": total transferred bytes since the start of the process, "errors": number of errors, + "fatalError": whether there has been at least one FatalError, + "retryError": whether there has been at least one non-NoRetryError, "checks": number of checked files, "transfers": number of transferred files, "deletes" : number of deleted files, @@ -2264,6 +2374,13 @@ rclone rc core/bwlimit rate=off 
    rclone rc vfs/forget

    Otherwise pass files or dirs in as file=path or dir=path. Any parameter key starting with file will forget that file and any starting with dir will forget that dir, eg

    rclone rc vfs/forget file=hello file2=goodbye dir=home/junk
    +

    vfs/poll-interval: Get the status or update the value of the poll-interval option.

    +

    Without any parameter given this returns the current status of the poll-interval setting.

    +

    When the interval=duration parameter is set, the poll-interval value is updated and the polling function is notified. Setting interval=0 disables poll-interval.

    +
    rclone rc vfs/poll-interval interval=5m
    +

    The timeout=duration parameter can be used to specify a time to wait for the current poll function to apply the new value. If timeout is less or equal 0, which is the default, wait indefinitely.

    +

    The new poll-interval value will only be active when the timeout is not reached.

    +

    If poll-interval is updated or disabled temporarily, some changes might not get picked up by the polling function, depending on the used remote.

    vfs/refresh: Refresh the directory cache.

    This reads the directories for the specified paths and freshens the directory cache.

    If no paths are passed in then it will refresh the root directory.

    @@ -2279,6 +2396,7 @@ rclone rc core/bwlimit rate=off

    The input objects can be supplied using URL parameters, POST parameters or by supplying "Content-Type: application/json" and a JSON blob in the body. There are examples of these below using curl.

    The response will be a JSON blob in the body of the response. This is formatted to be reasonably human readable.

    If an error occurs then there will be an HTTP error status (usually 400) and the body of the response will contain a JSON encoded error object.

    +

    The sever implements basic CORS support and allows all origins for that. The response to a preflight OPTIONS request will echo the requested "Access-Control-Request-Headers" back.

    Using POST with URL parameters only

    curl -X POST 'http://localhost:5572/rc/noop/?potato=1&sausage=2'

    Response

    @@ -2645,7 +2763,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total No #575 No Yes -No #2178 +Yes No @@ -2727,10 +2845,10 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total Yes Yes No +Yes No -No -No -No +Yes +Yes Mega @@ -2765,7 +2883,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total No #575 No No -No #2178 +Yes Yes @@ -2985,6 +3103,18 @@ e/n/d/r/c/s/q> q 
    rclone ls remote:

    Copy another local directory to the alias directory called source

    rclone copy /home/source remote:source
    + +

    Standard Options

    +

    Here are the standard options specific to alias (Alias for a existing remote).

    +

    --alias-remote

    +

    Remote or path to alias. Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/local/path".

    + +

    Amazon Drive

    Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage service run by Amazon for consumers.

    Status

    @@ -3086,17 +3216,75 @@ y/e/d> y

    Any files you delete with rclone will end up in the trash. Amazon don't provide an API to permanently delete files, nor to empty the trash, so you will have to do that with one of Amazon's apps or via the Amazon Drive website. As of November 17, 2016, files are automatically deleted by Amazon from the trash after 30 days.

    Using with non .com Amazon accounts

    Let's say you usually use amazon.co.uk. When you authenticate with rclone it will take you to an amazon.com page to log in. Your amazon.co.uk email and password should work here just fine.

    -

    Specific options

    -

    Here are the command line options specific to this cloud storage system.

    - -

    Files this size or more will be downloaded via their tempLink. This is to work around a problem with Amazon Drive which blocks downloads of files bigger than about 10GB. The default for this is 9GB which shouldn't need to be changed.

    -

    To download files above this threshold, rclone requests a tempLink which downloads the file through a temporary URL directly from the underlying S3 storage.

    -

    --acd-upload-wait-per-gb=TIME

    + +

    Standard Options

    +

    Here are the standard options specific to amazon cloud drive (Amazon Drive).

    +

    --acd-client-id

    +

    Amazon Application Client ID.

    + +

    --acd-client-secret

    +

    Amazon Application Client Secret.

    + +

    Advanced Options

    +

    Here are the advanced options specific to amazon cloud drive (Amazon Drive).

    +

    --acd-auth-url

    +

    Auth server URL. Leave blank to use Amazon's.

    + +

    --acd-token-url

    +

    Token server url. leave blank to use Amazon's.

    + +

    --acd-checkpoint

    +

    Checkpoint for internal polling (debug).

    + +

    --acd-upload-wait-per-gb

    +

    Additional time per GB to wait after a failed complete upload to see if it appears.

    Sometimes Amazon Drive gives an error when a file has been fully uploaded but the file appears anyway after a little while. This happens sometimes for files over 1GB in size and nearly every time for files bigger than 10GB. This parameter controls the time rclone waits for the file to appear.

    The default value for this parameter is 3 minutes per GB, so by default it will wait 3 minutes for every GB uploaded to see if the file appears.

    You can disable this feature by setting it to 0. This may cause conflict errors as rclone retries the failed upload but the file will most likely appear correctly eventually.

    These values were determined empirically by observing lots of uploads of big files for a range of file sizes.

    -

    Upload with the -v flag to see more info about what rclone is doing in this situation.

    +

    Upload with the "-v" flag to see more info about what rclone is doing in this situation.

    + + +

    Files >= this size will be downloaded via their tempLink.

    +

    Files this size or more will be downloaded via their "tempLink". This is to work around a problem with Amazon Drive which blocks downloads of files bigger than about 10GB. The default for this is 9GB which shouldn't need to be changed.

    +

    To download files above this threshold, rclone requests a "tempLink" which downloads the file through a temporary URL directly from the underlying S3 storage.

    + +

    Limitations

    Note that Amazon Drive is case insensitive so you can't have a file called "Hello.doc" and one called "hello.doc".

    Amazon Drive has rate limiting so you may notice errors in the sync (429 errors). rclone will automatically retry the sync up to 3 times by default (see --retries flag) which should hopefully work around this problem.

    @@ -3327,7 +3515,7 @@ y/e/d>

    rclone supports multipart uploads with S3 which means that it can upload files bigger than 5GB. Note that files uploaded both with multipart upload and through crypt remotes do not have MD5 sums.

    Buckets and Regions

    With Amazon S3 you can list buckets (rclone lsd) using any region, but you can only access the content of a bucket from the region it was created in. If you attempt to access a bucket from the wrong region, you will get an error, incorrect region, the bucket is not in 'XXX' region.

    -

    Authentication

    +

    Authentication

    There are a number of ways to supply rclone with a set of AWS credentials, with and without using the environment.

    The different authentication methods are tried in this order:

    + +

    Standard Options

    +

    Here are the standard options specific to qingstor (QingCloud Object Storage).

    +

    --qingstor-env-auth

    +

    Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.

    + +

    --qingstor-access-key-id

    +

    QingStor Access Key ID Leave blank for anonymous access or runtime credentials.

    + +

    --qingstor-secret-access-key

    +

    QingStor Secret Access Key (password) Leave blank for anonymous access or runtime credentials.

    + +

    --qingstor-endpoint

    +

    Enter a endpoint URL to connection QingStor API. Leave blank will use the default value "https://qingstor.com:443"

    + +

    --qingstor-zone

    +

    Zone to connect to. Default is "pek3a".

    + +

    Advanced Options

    +

    Here are the advanced options specific to qingstor (QingCloud Object Storage).

    +

    --qingstor-connection-retries

    +

    Number of connnection retries.

    + +

    Swift

    Swift refers to Openstack Object Storage. Commercial implementations of that being: