29288 lines
888 KiB
Groff
Generated
29288 lines
888 KiB
Groff
Generated
.\"t
|
|
.\" Automatically generated by Pandoc 2.2.1
|
|
.\"
|
|
.TH "rclone" "1" "Oct 26, 2019" "User Manual" ""
|
|
.hy
|
|
.SH Rclone \- rsync for cloud storage
|
|
.PP
|
|
Rclone is a command line program to sync files and directories to and
|
|
from:
|
|
.IP \[bu] 2
|
|
1Fichier
|
|
.IP \[bu] 2
|
|
Alibaba Cloud (Aliyun) Object Storage System (OSS)
|
|
.IP \[bu] 2
|
|
Amazon Drive (See note (/amazonclouddrive/#status))
|
|
.IP \[bu] 2
|
|
Amazon S3
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.IP \[bu] 2
|
|
Box
|
|
.IP \[bu] 2
|
|
Ceph
|
|
.IP \[bu] 2
|
|
Citrix ShareFile
|
|
.IP \[bu] 2
|
|
C14
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces
|
|
.IP \[bu] 2
|
|
Dreamhost
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.IP \[bu] 2
|
|
FTP
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.IP \[bu] 2
|
|
Google Photos
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud
|
|
.IP \[bu] 2
|
|
Memset Memstore
|
|
.IP \[bu] 2
|
|
Mega
|
|
.IP \[bu] 2
|
|
Microsoft Azure Blob Storage
|
|
.IP \[bu] 2
|
|
Microsoft OneDrive
|
|
.IP \[bu] 2
|
|
Minio
|
|
.IP \[bu] 2
|
|
Nextcloud
|
|
.IP \[bu] 2
|
|
OVH
|
|
.IP \[bu] 2
|
|
OpenDrive
|
|
.IP \[bu] 2
|
|
Openstack Swift
|
|
.IP \[bu] 2
|
|
Oracle Cloud Storage
|
|
.IP \[bu] 2
|
|
ownCloud
|
|
.IP \[bu] 2
|
|
pCloud
|
|
.IP \[bu] 2
|
|
premiumize.me
|
|
.IP \[bu] 2
|
|
put.io
|
|
.IP \[bu] 2
|
|
QingStor
|
|
.IP \[bu] 2
|
|
Rackspace Cloud Files
|
|
.IP \[bu] 2
|
|
rsync.net
|
|
.IP \[bu] 2
|
|
Scaleway
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.IP \[bu] 2
|
|
Wasabi
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.IP \[bu] 2
|
|
Yandex Disk
|
|
.IP \[bu] 2
|
|
The local filesystem
|
|
.PP
|
|
Features
|
|
.IP \[bu] 2
|
|
MD5/SHA1 hashes checked at all times for file integrity
|
|
.IP \[bu] 2
|
|
Timestamps preserved on files
|
|
.IP \[bu] 2
|
|
Partial syncs supported on a whole file basis
|
|
.IP \[bu] 2
|
|
Copy (https://rclone.org/commands/rclone_copy/) mode to just copy
|
|
new/changed files
|
|
.IP \[bu] 2
|
|
Sync (https://rclone.org/commands/rclone_sync/) (one way) mode to make a
|
|
directory identical
|
|
.IP \[bu] 2
|
|
Check (https://rclone.org/commands/rclone_check/) mode to check for file
|
|
hash equality
|
|
.IP \[bu] 2
|
|
Can sync to and from network, eg two different cloud accounts
|
|
.IP \[bu] 2
|
|
Encryption (https://rclone.org/crypt/) backend
|
|
.IP \[bu] 2
|
|
Cache (https://rclone.org/cache/) backend
|
|
.IP \[bu] 2
|
|
Chunking (https://rclone.org/chunker/) backend
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/) backend
|
|
.IP \[bu] 2
|
|
Optional FUSE mount (rclone
|
|
mount (https://rclone.org/commands/rclone_mount/))
|
|
.IP \[bu] 2
|
|
Multi\-threaded downloads to local disk
|
|
.IP \[bu] 2
|
|
Can serve (https://rclone.org/commands/rclone_serve/) local or remote
|
|
files over
|
|
HTTP (https://rclone.org/commands/rclone_serve_http/)/WebDav (https://rclone.org/commands/rclone_serve_webdav/)/FTP (https://rclone.org/commands/rclone_serve_ftp/)/SFTP (https://rclone.org/commands/rclone_serve_sftp/)/dlna (https://rclone.org/commands/rclone_serve_dlna/)
|
|
.IP \[bu] 2
|
|
Experimental Web based GUI (https://rclone.org/gui/)
|
|
.PP
|
|
Links
|
|
.IP \[bu] 2
|
|
Home page (https://rclone.org/)
|
|
.IP \[bu] 2
|
|
GitHub project page for source and bug
|
|
tracker (https://github.com/rclone/rclone)
|
|
.IP \[bu] 2
|
|
Rclone Forum (https://forum.rclone.org)
|
|
.IP \[bu] 2
|
|
Downloads (https://rclone.org/downloads/)
|
|
.SH Install
|
|
.PP
|
|
Rclone is a Go program and comes as a single binary file.
|
|
.SS Quickstart
|
|
.IP \[bu] 2
|
|
Download (https://rclone.org/downloads/) the relevant binary.
|
|
.IP \[bu] 2
|
|
Extract the \f[C]rclone\f[] or \f[C]rclone.exe\f[] binary from the
|
|
archive
|
|
.IP \[bu] 2
|
|
Run \f[C]rclone\ config\f[] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.PP
|
|
See below for some expanded Linux / macOS instructions.
|
|
.PP
|
|
See the Usage section (https://rclone.org/docs/) of the docs for how to
|
|
use rclone, or run \f[C]rclone\ \-h\f[].
|
|
.SS Script installation
|
|
.PP
|
|
To install rclone on Linux/macOS/BSD systems, run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl\ https://rclone.org/install.sh\ |\ sudo\ bash
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
For beta installation, run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl\ https://rclone.org/install.sh\ |\ sudo\ bash\ \-s\ beta
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that this script checks the version of rclone installed first and
|
|
won't re\-download if not needed.
|
|
.SS Linux installation from precompiled binary
|
|
.PP
|
|
Fetch and unpack
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl\ \-O\ https://downloads.rclone.org/rclone\-current\-linux\-amd64.zip
|
|
unzip\ rclone\-current\-linux\-amd64.zip
|
|
cd\ rclone\-*\-linux\-amd64
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Copy binary file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo\ cp\ rclone\ /usr/bin/
|
|
sudo\ chown\ root:root\ /usr/bin/rclone
|
|
sudo\ chmod\ 755\ /usr/bin/rclone
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Install manpage
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo\ mkdir\ \-p\ /usr/local/share/man/man1
|
|
sudo\ cp\ rclone.1\ /usr/local/share/man/man1/
|
|
sudo\ mandb\
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone\ config\f[] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.SS macOS installation from precompiled binary
|
|
.PP
|
|
Download the latest version of rclone.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cd\ &&\ curl\ \-O\ https://downloads.rclone.org/rclone\-current\-osx\-amd64.zip
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Unzip the download and cd to the extracted folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
unzip\ \-a\ rclone\-current\-osx\-amd64.zip\ &&\ cd\ rclone\-*\-osx\-amd64
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Move rclone to your $PATH.
|
|
You will be prompted for your password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo\ mkdir\ \-p\ /usr/local/bin
|
|
sudo\ mv\ rclone\ /usr/local/bin/
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
(the \f[C]mkdir\f[] command is safe to run, even if the directory
|
|
already exists).
|
|
.PP
|
|
Remove the leftover files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cd\ ..\ &&\ rm\ \-rf\ rclone\-*\-osx\-amd64\ rclone\-current\-osx\-amd64.zip
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone\ config\f[] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.SS Install with docker
|
|
.PP
|
|
The rclone maintains a docker image for
|
|
rclone (https://hub.docker.com/r/rclone/rclone).
|
|
These images are autobuilt by docker hub from the rclone source based on
|
|
a minimal Alpine linux image.
|
|
.PP
|
|
The \f[C]:latest\f[] tag will always point to the latest stable release.
|
|
You can use the \f[C]:beta\f[] tag to get the latest build from master.
|
|
You can also use version tags, eg \f[C]:1.49.1\f[], \f[C]:1.49\f[] or
|
|
\f[C]:1\f[].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ docker\ pull\ rclone/rclone:latest
|
|
latest:\ Pulling\ from\ rclone/rclone
|
|
Digest:\ sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11
|
|
\&...
|
|
$\ docker\ run\ \-\-rm\ rclone/rclone:latest\ version
|
|
rclone\ v1.49.1
|
|
\-\ os/arch:\ linux/amd64
|
|
\-\ go\ version:\ go1.12.9
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
There are a few command line options to consider when starting an rclone
|
|
Docker container from the rclone image.
|
|
.IP \[bu] 2
|
|
You need to mount the host rclone config dir at \f[C]/config/rclone\f[]
|
|
into the Docker container.
|
|
Due to the fact that rclone updates tokens inside its config file, and
|
|
that the update process involves a file rename, you need to mount the
|
|
whole host rclone config dir, not just the single host rclone config
|
|
file.
|
|
.IP \[bu] 2
|
|
You need to mount a host data dir at \f[C]/data\f[] into the Docker
|
|
container.
|
|
.IP \[bu] 2
|
|
By default, the rclone binary inside a Docker container runs with UID=0
|
|
(root).
|
|
As a result, all files created in a run will have UID=0.
|
|
If your config and data files reside on the host with a non\-root
|
|
UID:GID, you need to pass these on the container start command line.
|
|
.IP \[bu] 2
|
|
It is possible to use \f[C]rclone\ mount\f[] inside a userspace Docker
|
|
container, and expose the resulting fuse mount to the host.
|
|
The exact \f[C]docker\ run\f[] options to do that might vary slightly
|
|
between hosts.
|
|
See, e.g.\ the discussion in this
|
|
thread (https://github.com/moby/moby/issues/9448).
|
|
.RS 2
|
|
.PP
|
|
You also need to mount the host \f[C]/etc/passwd\f[] and
|
|
\f[C]/etc/group\f[] for fuse to work inside the container.
|
|
.RE
|
|
.PP
|
|
Here are some commands tested on an Ubuntu 18.04.3 host:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ config\ on\ host\ at\ ~/.config/rclone/rclone.conf
|
|
#\ data\ on\ host\ at\ ~/data
|
|
|
|
#\ make\ sure\ the\ config\ is\ ok\ by\ listing\ the\ remotes
|
|
docker\ run\ \-\-rm\ \\
|
|
\ \ \ \ \-\-volume\ ~/.config/rclone:/config/rclone\ \\
|
|
\ \ \ \ \-\-volume\ ~/data:/data:shared\ \\
|
|
\ \ \ \ \-\-user\ $(id\ \-u):$(id\ \-g)\ \\
|
|
\ \ \ \ rclone/rclone\ \\
|
|
\ \ \ \ listremotes
|
|
|
|
#\ perform\ mount\ inside\ Docker\ container,\ expose\ result\ to\ host
|
|
mkdir\ \-p\ ~/data/mount
|
|
docker\ run\ \-\-rm\ \\
|
|
\ \ \ \ \-\-volume\ ~/.config/rclone:/config/rclone\ \\
|
|
\ \ \ \ \-\-volume\ ~/data:/data:shared\ \\
|
|
\ \ \ \ \-\-user\ $(id\ \-u):$(id\ \-g)\ \\
|
|
\ \ \ \ \-\-volume\ /etc/passwd:/etc/passwd:ro\ \-\-volume\ /etc/group:/etc/group:ro\ \\
|
|
\ \ \ \ \-\-device\ /dev/fuse\ \-\-cap\-add\ SYS_ADMIN\ \-\-security\-opt\ apparmor:unconfined\ \\
|
|
\ \ \ \ rclone/rclone\ \\
|
|
\ \ \ \ mount\ dropbox:Photos\ /data/mount\ &
|
|
ls\ ~/data/mount
|
|
kill\ %1
|
|
\f[]
|
|
.fi
|
|
.SS Install from source
|
|
.PP
|
|
Make sure you have at least Go (https://golang.org/) 1.7 installed.
|
|
Download go (https://golang.org/dl/) if necessary.
|
|
The latest release is recommended.
|
|
Then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
git\ clone\ https://github.com/rclone/rclone.git
|
|
cd\ rclone
|
|
go\ build
|
|
\&./rclone\ version
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You can also build and install rclone in the
|
|
GOPATH (https://github.com/golang/go/wiki/GOPATH) (which defaults to
|
|
\f[C]~/go\f[]) with:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go\ get\ \-u\ \-v\ github.com/rclone/rclone
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
and this will build the binary in \f[C]$GOPATH/bin\f[]
|
|
(\f[C]~/go/bin/rclone\f[] by default) after downloading the source to
|
|
\f[C]$GOPATH/src/github.com/rclone/rclone\f[]
|
|
(\f[C]~/go/src/github.com/rclone/rclone\f[] by default).
|
|
.SS Installation with Ansible
|
|
.PP
|
|
This can be done with Stefan Weichinger's ansible
|
|
role (https://github.com/stefangweichinger/ansible-rclone).
|
|
.PP
|
|
Instructions
|
|
.IP "1." 3
|
|
\f[C]git\ clone\ https://github.com/stefangweichinger/ansible\-rclone.git\f[]
|
|
into your local roles\-directory
|
|
.IP "2." 3
|
|
add the role to the hosts you want rclone installed to:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \-\ hosts:\ rclone\-hosts
|
|
\ \ \ \ \ \ roles:
|
|
\ \ \ \ \ \ \ \ \ \ \-\ rclone
|
|
\f[]
|
|
.fi
|
|
.SS Configure
|
|
.PP
|
|
First, you'll need to configure rclone.
|
|
As the object storage systems have quite complicated authentication
|
|
these are kept in a config file.
|
|
(See the \f[C]\-\-config\f[] entry for how to find the config file and
|
|
choose its location.)
|
|
.PP
|
|
The easiest way to make the config is to run rclone with the config
|
|
option:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the following for detailed instructions for
|
|
.IP \[bu] 2
|
|
1Fichier (https://rclone.org/fichier/)
|
|
.IP \[bu] 2
|
|
Alias (https://rclone.org/alias/)
|
|
.IP \[bu] 2
|
|
Amazon Drive (https://rclone.org/amazonclouddrive/)
|
|
.IP \[bu] 2
|
|
Amazon S3 (https://rclone.org/s3/)
|
|
.IP \[bu] 2
|
|
Backblaze B2 (https://rclone.org/b2/)
|
|
.IP \[bu] 2
|
|
Box (https://rclone.org/box/)
|
|
.IP \[bu] 2
|
|
Cache (https://rclone.org/cache/)
|
|
.IP \[bu] 2
|
|
Chunker (https://rclone.org/chunker/) \- transparently splits large
|
|
files for other remotes
|
|
.IP \[bu] 2
|
|
Citrix ShareFile (https://rclone.org/sharefile/)
|
|
.IP \[bu] 2
|
|
Crypt (https://rclone.org/crypt/) \- to encrypt other remotes
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces (/s3/#digitalocean-spaces)
|
|
.IP \[bu] 2
|
|
Dropbox (https://rclone.org/dropbox/)
|
|
.IP \[bu] 2
|
|
FTP (https://rclone.org/ftp/)
|
|
.IP \[bu] 2
|
|
Google Cloud Storage (https://rclone.org/googlecloudstorage/)
|
|
.IP \[bu] 2
|
|
Google Drive (https://rclone.org/drive/)
|
|
.IP \[bu] 2
|
|
Google Photos (https://rclone.org/googlephotos/)
|
|
.IP \[bu] 2
|
|
HTTP (https://rclone.org/http/)
|
|
.IP \[bu] 2
|
|
Hubic (https://rclone.org/hubic/)
|
|
.IP \[bu] 2
|
|
Jottacloud (https://rclone.org/jottacloud/)
|
|
.IP \[bu] 2
|
|
Koofr (https://rclone.org/koofr/)
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud (https://rclone.org/mailru/)
|
|
.IP \[bu] 2
|
|
Mega (https://rclone.org/mega/)
|
|
.IP \[bu] 2
|
|
Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
|
|
.IP \[bu] 2
|
|
Microsoft OneDrive (https://rclone.org/onedrive/)
|
|
.IP \[bu] 2
|
|
Openstack Swift / Rackspace Cloudfiles / Memset
|
|
Memstore (https://rclone.org/swift/)
|
|
.IP \[bu] 2
|
|
OpenDrive (https://rclone.org/opendrive/)
|
|
.IP \[bu] 2
|
|
Pcloud (https://rclone.org/pcloud/)
|
|
.IP \[bu] 2
|
|
premiumize.me (https://rclone.org/premiumizeme/)
|
|
.IP \[bu] 2
|
|
put.io (https://rclone.org/putio/)
|
|
.IP \[bu] 2
|
|
QingStor (https://rclone.org/qingstor/)
|
|
.IP \[bu] 2
|
|
SFTP (https://rclone.org/sftp/)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/)
|
|
.IP \[bu] 2
|
|
WebDAV (https://rclone.org/webdav/)
|
|
.IP \[bu] 2
|
|
Yandex Disk (https://rclone.org/yandex/)
|
|
.IP \[bu] 2
|
|
The local filesystem (https://rclone.org/local/)
|
|
.SS Usage
|
|
.PP
|
|
Rclone syncs a directory tree from one storage system to another.
|
|
.PP
|
|
Its syntax is like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Syntax:\ [options]\ subcommand\ <parameters>\ <parameters...>
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Source and destination paths are specified by the name you gave the
|
|
storage system in the config file then the sub path, eg
|
|
\[lq]drive:myfolder\[rq] to look at \[lq]myfolder\[rq] in Google drive.
|
|
.PP
|
|
You can define as many storage paths as you like in the config file.
|
|
.SS Subcommands
|
|
.PP
|
|
rclone uses a system of subcommands.
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:path\ #\ lists\ a\ remote
|
|
rclone\ copy\ /local/path\ remote:path\ #\ copies\ /local/path\ to\ the\ remote
|
|
rclone\ sync\ /local/path\ remote:path\ #\ syncs\ /local/path\ to\ the\ remote
|
|
\f[]
|
|
.fi
|
|
.SS rclone config
|
|
.PP
|
|
Enter an interactive configuration session.
|
|
.SS Synopsis
|
|
.PP
|
|
Enter an interactive configuration session where you can setup new
|
|
remotes and manage existing ones.
|
|
You may also set or remove a password to protect your configuration.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone config create (https://rclone.org/commands/rclone_config_create/)
|
|
\- Create a new remote with name, type and options.
|
|
.IP \[bu] 2
|
|
rclone config delete (https://rclone.org/commands/rclone_config_delete/)
|
|
\- Delete an existing remote .
|
|
.IP \[bu] 2
|
|
rclone config
|
|
disconnect (https://rclone.org/commands/rclone_config_disconnect/) \-
|
|
Disconnects user from remote
|
|
.IP \[bu] 2
|
|
rclone config dump (https://rclone.org/commands/rclone_config_dump/) \-
|
|
Dump the config file as JSON.
|
|
.IP \[bu] 2
|
|
rclone config edit (https://rclone.org/commands/rclone_config_edit/) \-
|
|
Enter an interactive configuration session.
|
|
.IP \[bu] 2
|
|
rclone config file (https://rclone.org/commands/rclone_config_file/) \-
|
|
Show path of configuration file in use.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
password (https://rclone.org/commands/rclone_config_password/) \- Update
|
|
password in an existing remote.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
providers (https://rclone.org/commands/rclone_config_providers/) \- List
|
|
in JSON format all the providers and options.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
reconnect (https://rclone.org/commands/rclone_config_reconnect/) \-
|
|
Re\-authenticates user with remote.
|
|
.IP \[bu] 2
|
|
rclone config show (https://rclone.org/commands/rclone_config_show/) \-
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.IP \[bu] 2
|
|
rclone config update (https://rclone.org/commands/rclone_config_update/)
|
|
\- Update options in an existing remote.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
userinfo (https://rclone.org/commands/rclone_config_userinfo/) \- Prints
|
|
info about logged in user of remote.
|
|
.SS rclone copy
|
|
.PP
|
|
Copy files from source to dest, skipping already copied
|
|
.SS Synopsis
|
|
.PP
|
|
Copy the source to the destination.
|
|
Doesn't transfer unchanged files, testing by size and modification time
|
|
or MD5SUM.
|
|
Doesn't delete files from the destination.
|
|
.PP
|
|
Note that it is always the contents of the directory that is synced, not
|
|
the directory so when source:path is a directory, it's the contents of
|
|
source:path that are copied, not the directory name and contents.
|
|
.PP
|
|
If dest:path doesn't exist, it is created and the source:path contents
|
|
go there.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ source:sourcepath\ dest:destpath
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Let's say there are two files in sourcepath
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sourcepath/one.txt
|
|
sourcepath/two.txt
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This copies them to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
destpath/one.txt
|
|
destpath/two.txt
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Not to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
destpath/sourcepath/one.txt
|
|
destpath/sourcepath/two.txt
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you are familiar with \f[C]rsync\f[], rclone always works as if you
|
|
had written a trailing / \- meaning \[lq]copy the contents of this
|
|
directory\[rq].
|
|
This applies to all commands and whether you are talking about the
|
|
source or destination.
|
|
.PP
|
|
See the \[en]no\-traverse (/docs/#no-traverse) option for controlling
|
|
whether rclone lists the destination directory or not.
|
|
Supplying this option when copying a small number of files into a large
|
|
destination can speed transfers up greatly.
|
|
.PP
|
|
For example, if you have many files in /path/to/src but only a few of
|
|
them change every day, you can to copy all the files which have changed
|
|
recently very efficiently like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-max\-age\ 24h\ \-\-no\-traverse\ /path/to/src\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\f[B]Note\f[]: Use the \f[C]\-P\f[]/\f[C]\-\-progress\f[] flag to view
|
|
real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ source:path\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-create\-empty\-src\-dirs\ \ \ Create\ empty\ source\ dirs\ on\ destination\ after\ copy
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ copy
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone sync
|
|
.PP
|
|
Make source and dest identical, modifying destination only.
|
|
.SS Synopsis
|
|
.PP
|
|
Sync the source to the destination, changing the destination only.
|
|
Doesn't transfer unchanged files, testing by size and modification time
|
|
or MD5SUM.
|
|
Destination is updated to match source, including deleting files if
|
|
necessary.
|
|
.PP
|
|
\f[B]Important\f[]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[] flag to see exactly what would be copied and
|
|
deleted.
|
|
.PP
|
|
Note that files in the destination won't be deleted if there were any
|
|
errors at any point.
|
|
.PP
|
|
It is always the contents of the directory that is synced, not the
|
|
directory so when source:path is a directory, it's the contents of
|
|
source:path that are copied, not the directory name and contents.
|
|
See extended explanation in the \f[C]copy\f[] command above if unsure.
|
|
.PP
|
|
If dest:path doesn't exist, it is created and the source:path contents
|
|
go there.
|
|
.PP
|
|
\f[B]Note\f[]: Use the \f[C]\-P\f[]/\f[C]\-\-progress\f[] flag to view
|
|
real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ source:path\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-create\-empty\-src\-dirs\ \ \ Create\ empty\ source\ dirs\ on\ destination\ after\ sync
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ sync
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone move
|
|
.PP
|
|
Move files from source to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
Moves the contents of the source directory to the destination directory.
|
|
Rclone will error if the source and destination overlap and the remote
|
|
does not support a server side directory move operation.
|
|
.PP
|
|
If no filters are in use and if possible this will server side move
|
|
\f[C]source:path\f[] into \f[C]dest:path\f[].
|
|
After this \f[C]source:path\f[] will no longer longer exist.
|
|
.PP
|
|
Otherwise for each file in \f[C]source:path\f[] selected by the filters
|
|
(if any) this will move it into \f[C]dest:path\f[].
|
|
If possible a server side move will be used, otherwise it will copy it
|
|
(server side if possible) into \f[C]dest:path\f[] then delete the
|
|
original (if no errors on copy) in \f[C]source:path\f[].
|
|
.PP
|
|
If you want to delete empty source directories after move, use the
|
|
\[en]delete\-empty\-src\-dirs flag.
|
|
.PP
|
|
See the \[en]no\-traverse (/docs/#no-traverse) option for controlling
|
|
whether rclone lists the destination directory or not.
|
|
Supplying this option when moving a small number of files into a large
|
|
destination can speed transfers up greatly.
|
|
.PP
|
|
\f[B]Important\f[]: Since this can cause data loss, test first with the
|
|
\[en]dry\-run flag.
|
|
.PP
|
|
\f[B]Note\f[]: Use the \f[C]\-P\f[]/\f[C]\-\-progress\f[] flag to view
|
|
real\-time transfer statistics.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ move\ source:path\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-create\-empty\-src\-dirs\ \ \ Create\ empty\ source\ dirs\ on\ destination\ after\ move
|
|
\ \ \ \ \ \ \-\-delete\-empty\-src\-dirs\ \ \ Delete\ empty\ source\ dirs\ after\ move
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ move
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone delete
|
|
.PP
|
|
Remove the contents of path.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the files in path.
|
|
Unlike \f[C]purge\f[] it obeys include/exclude filters so can be used to
|
|
selectively delete files.
|
|
.PP
|
|
\f[C]rclone\ delete\f[] only deletes objects but leaves the directory
|
|
structure alone.
|
|
If you want to delete a directory and all of its contents use
|
|
\f[C]rclone\ purge\f[]
|
|
.PP
|
|
Eg delete all files bigger than 100MBytes
|
|
.PP
|
|
Check what would be deleted first (use either)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ \-\-min\-size\ 100M\ lsl\ remote:path
|
|
rclone\ \-\-dry\-run\ \-\-min\-size\ 100M\ delete\ remote:path
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then delete
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ \-\-min\-size\ 100M\ delete\ remote:path
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
That reads \[lq]delete everything with a minimum size of 100 MB\[rq],
|
|
hence delete all files bigger than 100MBytes.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ delete\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ delete
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone purge
|
|
.PP
|
|
Remove the path and all of its contents.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the path and all of its contents.
|
|
Note that this does not obey include/exclude filters \- everything will
|
|
be removed.
|
|
Use \f[C]delete\f[] if you want to selectively delete files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ purge\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ purge
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone mkdir
|
|
.PP
|
|
Make the path if it doesn't already exist.
|
|
.SS Synopsis
|
|
.PP
|
|
Make the path if it doesn't already exist.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ mkdir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone rmdir
|
|
.PP
|
|
Remove the path if empty.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the path.
|
|
Note that you can't remove a path with objects in it, use purge for
|
|
that.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rmdir\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ rmdir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone check
|
|
.PP
|
|
Checks the files in the source and destination match.
|
|
.SS Synopsis
|
|
.PP
|
|
Checks the files in the source and destination match.
|
|
It compares sizes and hashes (MD5 or SHA1) and logs a report of files
|
|
which don't match.
|
|
It doesn't alter the source or destination.
|
|
.PP
|
|
If you supply the \[en]size\-only flag, it will only compare the sizes
|
|
not the hashes as well.
|
|
Use this for a quick check.
|
|
.PP
|
|
If you supply the \[en]download flag, it will download the data from
|
|
both remotes and check them against each other on the fly.
|
|
This can be useful for remotes that don't support hashes or if you
|
|
really want to check all the data.
|
|
.PP
|
|
If you supply the \[en]one\-way flag, it will only check that files in
|
|
source match the files in destination, not the other way around.
|
|
Meaning extra files in destination that are not in the source will not
|
|
trigger an error.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ check\ source:path\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-download\ \ \ Check\ by\ downloading\ rather\ than\ with\ hash.
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ help\ for\ check
|
|
\ \ \ \ \ \ \-\-one\-way\ \ \ \ Check\ one\ way\ only,\ source\ files\ must\ exist\ on\ remote
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone ls
|
|
.PP
|
|
List the objects in the path with size and path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the objects in the source path to standard output in a human
|
|
readable format with size and path.
|
|
Recurses by default.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ ls\ swift:bucket
|
|
\ \ \ \ 60295\ bevajer5jef
|
|
\ \ \ \ 90613\ canole
|
|
\ \ \ \ 94467\ diwogej7
|
|
\ \ \ \ 37600\ fubuwic
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[],\f[C]lsl\f[],\f[C]lsd\f[] are designed to be human readable.
|
|
\f[C]lsf\f[] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[] and \f[C]lsl\f[] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[],\f[C]lsf\f[],\f[C]lsjson\f[] do not
|
|
recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can't have empty directories (eg s3, swift, gcs, etc \-
|
|
the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ ls
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone lsd
|
|
.PP
|
|
List all directories/containers/buckets in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the directories in the source path to standard output.
|
|
Does not recurse by default.
|
|
Use the \-R flag to recurse.
|
|
.PP
|
|
This command lists the total size of the directory (if known, \-1 if
|
|
not), the modification time (if known, the current time if not), the
|
|
number of objects in the directory (if known, \-1 if not) and the name
|
|
of the directory, Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsd\ swift:
|
|
\ \ \ \ \ \ 494000\ 2018\-04\-26\ 08:43:20\ \ \ \ \ 10000\ 10000files
|
|
\ \ \ \ \ \ \ \ \ \ 65\ 2018\-04\-26\ 08:43:20\ \ \ \ \ \ \ \ \ 1\ 1File
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsd\ drive:test
|
|
\ \ \ \ \ \ \ \ \ \ \-1\ 2016\-10\-17\ 17:41:53\ \ \ \ \ \ \ \ \-1\ 1000files
|
|
\ \ \ \ \ \ \ \ \ \ \-1\ 2017\-01\-03\ 14:40:54\ \ \ \ \ \ \ \ \-1\ 2500files
|
|
\ \ \ \ \ \ \ \ \ \ \-1\ 2017\-07\-08\ 14:39:28\ \ \ \ \ \ \ \ \-1\ 4000files
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you just want the directory names use \[lq]rclone lsf
|
|
\[en]dirs\-only\[rq].
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[],\f[C]lsl\f[],\f[C]lsd\f[] are designed to be human readable.
|
|
\f[C]lsf\f[] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[] and \f[C]lsl\f[] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[],\f[C]lsf\f[],\f[C]lsjson\f[] do not
|
|
recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can't have empty directories (eg s3, swift, gcs, etc \-
|
|
the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ help\ for\ lsd
|
|
\ \ \-R,\ \-\-recursive\ \ \ Recurse\ into\ the\ listing.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone lsl
|
|
.PP
|
|
List the objects in path with modification time, size and path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the objects in the source path to standard output in a human
|
|
readable format with modification time, size and path.
|
|
Recurses by default.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsl\ swift:bucket
|
|
\ \ \ \ 60295\ 2016\-06\-25\ 18:55:41.062626927\ bevajer5jef
|
|
\ \ \ \ 90613\ 2016\-06\-25\ 18:55:43.302607074\ canole
|
|
\ \ \ \ 94467\ 2016\-06\-25\ 18:55:43.046609333\ diwogej7
|
|
\ \ \ \ 37600\ 2016\-06\-25\ 18:55:40.814629136\ fubuwic
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[],\f[C]lsl\f[],\f[C]lsd\f[] are designed to be human readable.
|
|
\f[C]lsf\f[] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[] and \f[C]lsl\f[] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[],\f[C]lsf\f[],\f[C]lsjson\f[] do not
|
|
recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can't have empty directories (eg s3, swift, gcs, etc \-
|
|
the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsl\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ lsl
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone md5sum
|
|
.PP
|
|
Produces an md5sum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces an md5sum file for all the objects in the path.
|
|
This is in the same format as the standard md5sum tool produces.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ md5sum\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ md5sum
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone sha1sum
|
|
.PP
|
|
Produces an sha1sum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces an sha1sum file for all the objects in the path.
|
|
This is in the same format as the standard sha1sum tool produces.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sha1sum\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ sha1sum
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone size
|
|
.PP
|
|
Prints the total size and number of objects in remote:path.
|
|
.SS Synopsis
|
|
.PP
|
|
Prints the total size and number of objects in remote:path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ size\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ size
|
|
\ \ \ \ \ \ \-\-json\ \ \ format\ output\ as\ JSON
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone version
|
|
.PP
|
|
Show the version number.
|
|
.SS Synopsis
|
|
.PP
|
|
Show the version number, the go version and the architecture.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ version
|
|
rclone\ v1.41
|
|
\-\ os/arch:\ linux/amd64
|
|
\-\ go\ version:\ go1.10
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you supply the \[en]check flag, then it will do an online check to
|
|
compare your version with the latest release and the latest beta.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ version\ \-\-check
|
|
yours:\ \ 1.42.0.6
|
|
latest:\ 1.42\ \ \ \ \ \ \ \ \ \ (released\ 2018\-06\-16)
|
|
beta:\ \ \ 1.42.0.5\ \ \ \ \ \ (released\ 2018\-06\-17)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ version\ \-\-check
|
|
yours:\ \ 1.41
|
|
latest:\ 1.42\ \ \ \ \ \ \ \ \ \ (released\ 2018\-06\-16)
|
|
\ \ upgrade:\ https://downloads.rclone.org/v1.42
|
|
beta:\ \ \ 1.42.0.5\ \ \ \ \ \ (released\ 2018\-06\-17)
|
|
\ \ upgrade:\ https://beta.rclone.org/v1.42\-005\-g56e1e820
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ version\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-check\ \ \ Check\ for\ new\ version.
|
|
\ \ \-h,\ \-\-help\ \ \ \ help\ for\ version
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone cleanup
|
|
.PP
|
|
Clean up the remote if possible
|
|
.SS Synopsis
|
|
.PP
|
|
Clean up the remote if possible.
|
|
Empty the trash or delete old file versions.
|
|
Not supported by all remotes.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cleanup\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ cleanup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone dedupe
|
|
.PP
|
|
Interactively find duplicate files and delete/rename them.
|
|
.SS Synopsis
|
|
.PP
|
|
By default \f[C]dedupe\f[] interactively finds duplicate files and
|
|
offers to delete all but one or rename them to be different.
|
|
Only useful with Google Drive which can have duplicate file names.
|
|
.PP
|
|
In the first pass it will merge directories with the same name.
|
|
It will do this iteratively until all the identical directories have
|
|
been merged.
|
|
.PP
|
|
The \f[C]dedupe\f[] command will delete all but one of any identical
|
|
(same md5sum) files it finds without confirmation.
|
|
This means that for most duplicated files the \f[C]dedupe\f[] command
|
|
will not be interactive.
|
|
You can use \f[C]\-\-dry\-run\f[] to see what would happen without doing
|
|
anything.
|
|
.PP
|
|
Here is an example run.
|
|
.PP
|
|
Before \- with duplicates
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsl\ drive:dupes
|
|
\ \ 6048320\ 2016\-03\-05\ 16:23:16.798000000\ one.txt
|
|
\ \ 6048320\ 2016\-03\-05\ 16:23:11.775000000\ one.txt
|
|
\ \ \ 564374\ 2016\-03\-05\ 16:23:06.731000000\ one.txt
|
|
\ \ 6048320\ 2016\-03\-05\ 16:18:26.092000000\ one.txt
|
|
\ \ 6048320\ 2016\-03\-05\ 16:22:46.185000000\ two.txt
|
|
\ \ 1744073\ 2016\-03\-05\ 16:22:38.104000000\ two.txt
|
|
\ \ \ 564374\ 2016\-03\-05\ 16:22:52.118000000\ two.txt
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Now the \f[C]dedupe\f[] session
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ dedupe\ drive:dupes
|
|
2016/03/05\ 16:24:37\ Google\ drive\ root\ \[aq]dupes\[aq]:\ Looking\ for\ duplicates\ using\ interactive\ mode.
|
|
one.txt:\ Found\ 4\ duplicates\ \-\ deleting\ identical\ copies
|
|
one.txt:\ Deleting\ 2/3\ identical\ duplicates\ (md5sum\ "1eedaa9fe86fd4b8632e2ac549403b36")
|
|
one.txt:\ 2\ duplicates\ remain
|
|
\ \ 1:\ \ \ \ \ \ 6048320\ bytes,\ 2016\-03\-05\ 16:23:16.798000000,\ md5sum\ 1eedaa9fe86fd4b8632e2ac549403b36
|
|
\ \ 2:\ \ \ \ \ \ \ 564374\ bytes,\ 2016\-03\-05\ 16:23:06.731000000,\ md5sum\ 7594e7dc9fc28f727c42ee3e0749de81
|
|
s)\ Skip\ and\ do\ nothing
|
|
k)\ Keep\ just\ one\ (choose\ which\ in\ next\ step)
|
|
r)\ Rename\ all\ to\ be\ different\ (by\ changing\ file.jpg\ to\ file\-1.jpg)
|
|
s/k/r>\ k
|
|
Enter\ the\ number\ of\ the\ file\ to\ keep>\ 1
|
|
one.txt:\ Deleted\ 1\ extra\ copies
|
|
two.txt:\ Found\ 3\ duplicates\ \-\ deleting\ identical\ copies
|
|
two.txt:\ 3\ duplicates\ remain
|
|
\ \ 1:\ \ \ \ \ \ \ 564374\ bytes,\ 2016\-03\-05\ 16:22:52.118000000,\ md5sum\ 7594e7dc9fc28f727c42ee3e0749de81
|
|
\ \ 2:\ \ \ \ \ \ 6048320\ bytes,\ 2016\-03\-05\ 16:22:46.185000000,\ md5sum\ 1eedaa9fe86fd4b8632e2ac549403b36
|
|
\ \ 3:\ \ \ \ \ \ 1744073\ bytes,\ 2016\-03\-05\ 16:22:38.104000000,\ md5sum\ 851957f7fb6f0bc4ce76be966d336802
|
|
s)\ Skip\ and\ do\ nothing
|
|
k)\ Keep\ just\ one\ (choose\ which\ in\ next\ step)
|
|
r)\ Rename\ all\ to\ be\ different\ (by\ changing\ file.jpg\ to\ file\-1.jpg)
|
|
s/k/r>\ r
|
|
two\-1.txt:\ renamed\ from:\ two.txt
|
|
two\-2.txt:\ renamed\ from:\ two.txt
|
|
two\-3.txt:\ renamed\ from:\ two.txt
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The result being
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsl\ drive:dupes
|
|
\ \ 6048320\ 2016\-03\-05\ 16:23:16.798000000\ one.txt
|
|
\ \ \ 564374\ 2016\-03\-05\ 16:22:52.118000000\ two\-1.txt
|
|
\ \ 6048320\ 2016\-03\-05\ 16:22:46.185000000\ two\-2.txt
|
|
\ \ 1744073\ 2016\-03\-05\ 16:22:38.104000000\ two\-3.txt
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Dedupe can be run non interactively using the \f[C]\-\-dedupe\-mode\f[]
|
|
flag or by using an extra parameter with the same value
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ interactive\f[] \- interactive as above.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ skip\f[] \- removes identical files then skips
|
|
anything left.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ first\f[] \- removes identical files then keeps
|
|
the first one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ newest\f[] \- removes identical files then keeps
|
|
the newest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ oldest\f[] \- removes identical files then keeps
|
|
the oldest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ largest\f[] \- removes identical files then keeps
|
|
the largest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ rename\f[] \- removes identical files then
|
|
renames the rest to be different.
|
|
.PP
|
|
For example to rename all the identically named photos in your Google
|
|
Photos directory, do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ dedupe\ \-\-dedupe\-mode\ rename\ "drive:Google\ Photos"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ dedupe\ rename\ "drive:Google\ Photos"
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ dedupe\ [mode]\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-dedupe\-mode\ string\ \ \ Dedupe\ mode\ interactive|skip|first|newest|oldest|rename.\ (default\ "interactive")
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ dedupe
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone about
|
|
.PP
|
|
Get quota information from the remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Get quota information from the remote, like bytes used/free/quota and
|
|
bytes used in the trash.
|
|
Not supported by all remotes.
|
|
.PP
|
|
This will print to stdout something like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Total:\ \ \ 17G
|
|
Used:\ \ \ \ 7.444G
|
|
Free:\ \ \ \ 1.315G
|
|
Trashed:\ 100.000M
|
|
Other:\ \ \ 8.241G
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Where the fields are:
|
|
.IP \[bu] 2
|
|
Total: total size available.
|
|
.IP \[bu] 2
|
|
Used: total size used
|
|
.IP \[bu] 2
|
|
Free: total amount this user could upload.
|
|
.IP \[bu] 2
|
|
Trashed: total amount in the trash
|
|
.IP \[bu] 2
|
|
Other: total amount in other storage (eg Gmail, Google Photos)
|
|
.IP \[bu] 2
|
|
Objects: total number of objects in the storage
|
|
.PP
|
|
Note that not all the backends provide all the fields \- they will be
|
|
missing if they are not known for that backend.
|
|
Where it is known that the value is unlimited the value will also be
|
|
omitted.
|
|
.PP
|
|
Use the \[en]full flag to see the numbers written out in full, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Total:\ \ \ 18253611008
|
|
Used:\ \ \ \ 7993453766
|
|
Free:\ \ \ \ 1411001220
|
|
Trashed:\ 104857602
|
|
Other:\ \ \ 8849156022
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Use the \[en]json flag for a computer readable output, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "total":\ 18253611008,
|
|
\ \ \ \ "used":\ 7993453766,
|
|
\ \ \ \ "trashed":\ 104857602,
|
|
\ \ \ \ "other":\ 8849156022,
|
|
\ \ \ \ "free":\ 1411001220
|
|
}
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ about\ remote:\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-full\ \ \ Full\ numbers\ instead\ of\ SI\ units
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ about
|
|
\ \ \ \ \ \ \-\-json\ \ \ Format\ output\ as\ JSON
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone authorize
|
|
.PP
|
|
Remote authorization.
|
|
.SS Synopsis
|
|
.PP
|
|
Remote authorization.
|
|
Used to authorize a remote or headless rclone from a machine with a
|
|
browser \- use as instructed by rclone config.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ authorize\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ authorize
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone cachestats
|
|
.PP
|
|
Print cache stats for a remote
|
|
.SS Synopsis
|
|
.PP
|
|
Print cache stats for a remote in JSON format
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cachestats\ source:\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ cachestats
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone cat
|
|
.PP
|
|
Concatenates any files and sends them to stdout.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cat sends any files to standard output.
|
|
.PP
|
|
You can use it like this to output a single file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cat\ remote:path/to/file
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or like this to output any file in dir or subdirectories.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cat\ remote:path/to/dir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or like this to output any .txt files in dir or subdirectories.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ \-\-include\ "*.txt"\ cat\ remote:path/to/dir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Use the \[en]head flag to print characters only at the start, \[en]tail
|
|
for the end and \[en]offset and \[en]count to print a section in the
|
|
middle.
|
|
Note that if offset is negative it will count from the end, so
|
|
\[en]offset \-1 \[en]count 1 is equivalent to \[en]tail 1.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cat\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-count\ int\ \ \ \ Only\ print\ N\ characters.\ (default\ \-1)
|
|
\ \ \ \ \ \ \-\-discard\ \ \ \ \ \ Discard\ the\ output\ instead\ of\ printing.
|
|
\ \ \ \ \ \ \-\-head\ int\ \ \ \ \ Only\ print\ the\ first\ N\ characters.
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ help\ for\ cat
|
|
\ \ \ \ \ \ \-\-offset\ int\ \ \ Start\ printing\ at\ offset\ N\ (or\ from\ end\ if\ \-ve).
|
|
\ \ \ \ \ \ \-\-tail\ int\ \ \ \ \ Only\ print\ the\ last\ N\ characters.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone config create
|
|
.PP
|
|
Create a new remote with name, type and options.
|
|
.SS Synopsis
|
|
.PP
|
|
Create a new remote of with and options.
|
|
The options should be passed in in pairs of .
|
|
.PP
|
|
For example to make a swift remote of name myremote using auto config
|
|
you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ create\ myremote\ swift\ env_auth\ true
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that if the config process would normally ask a question the
|
|
default is taken.
|
|
Each time that happens rclone will print a message saying how to affect
|
|
the value taken.
|
|
.PP
|
|
If any of the parameters passed is a password field, then rclone will
|
|
automatically obscure them before putting them in the config file.
|
|
.PP
|
|
So for example if you wanted to configure a Google Drive remote but
|
|
using remote authorization you would do this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ create\ mydrive\ drive\ config_is_local\ false
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ create\ <name>\ <type>\ [<key>\ <value>]*\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ create
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config delete
|
|
.PP
|
|
Delete an existing remote .
|
|
.SS Synopsis
|
|
.PP
|
|
Delete an existing remote .
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ delete\ <name>\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ delete
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config disconnect
|
|
.PP
|
|
Disconnects user from remote
|
|
.SS Synopsis
|
|
.PP
|
|
This disconnects the remote: passed in to the cloud storage system.
|
|
.PP
|
|
This normally means revoking the oauth token.
|
|
.PP
|
|
To reconnect use \[lq]rclone config reconnect\[rq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ disconnect\ remote:\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ disconnect
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config dump
|
|
.PP
|
|
Dump the config file as JSON.
|
|
.SS Synopsis
|
|
.PP
|
|
Dump the config file as JSON.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ dump\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ dump
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config edit
|
|
.PP
|
|
Enter an interactive configuration session.
|
|
.SS Synopsis
|
|
.PP
|
|
Enter an interactive configuration session where you can setup new
|
|
remotes and manage existing ones.
|
|
You may also set or remove a password to protect your configuration.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ edit\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ edit
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config file
|
|
.PP
|
|
Show path of configuration file in use.
|
|
.SS Synopsis
|
|
.PP
|
|
Show path of configuration file in use.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ file\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ file
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config password
|
|
.PP
|
|
Update password in an existing remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Update an existing remote's password.
|
|
The password should be passed in in pairs of .
|
|
.PP
|
|
For example to set password of a remote of name myremote you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ password\ myremote\ fieldname\ mypassword
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This command is obsolete now that \[lq]config update\[rq] and
|
|
\[lq]config create\[rq] both support obscuring passwords directly.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ password\ <name>\ [<key>\ <value>]+\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ password
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config providers
|
|
.PP
|
|
List in JSON format all the providers and options.
|
|
.SS Synopsis
|
|
.PP
|
|
List in JSON format all the providers and options.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ providers\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ providers
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config reconnect
|
|
.PP
|
|
Re\-authenticates user with remote.
|
|
.SS Synopsis
|
|
.PP
|
|
This reconnects remote: passed in to the cloud storage system.
|
|
.PP
|
|
To disconnect the remote use \[lq]rclone config disconnect\[rq].
|
|
.PP
|
|
This normally means going through the interactive oauth flow again.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ reconnect\ remote:\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ reconnect
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config show
|
|
.PP
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ show\ [<remote>]\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ show
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config update
|
|
.PP
|
|
Update options in an existing remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Update an existing remote's options.
|
|
The options should be passed in in pairs of .
|
|
.PP
|
|
For example to update the env_auth field of a remote of name myremote
|
|
you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ update\ myremote\ swift\ env_auth\ true
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If any of the parameters passed is a password field, then rclone will
|
|
automatically obscure them before putting them in the config file.
|
|
.PP
|
|
If the remote uses oauth the token will be updated, if you don't require
|
|
this add an extra parameter thus:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ update\ myremote\ swift\ env_auth\ true\ config_refresh_token\ false
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ update\ <name>\ [<key>\ <value>]+\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ update
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone config userinfo
|
|
.PP
|
|
Prints info about logged in user of remote.
|
|
.SS Synopsis
|
|
.PP
|
|
This prints the details of the person logged in to the cloud storage
|
|
system.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config\ userinfo\ remote:\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ userinfo
|
|
\ \ \ \ \ \ \-\-json\ \ \ Format\ output\ as\ JSON
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SS rclone copyto
|
|
.PP
|
|
Copy files from source to dest, skipping already copied
|
|
.SS Synopsis
|
|
.PP
|
|
If source:path is a file or directory then it copies it to a file or
|
|
directory named dest:path.
|
|
.PP
|
|
This can be used to upload single files to other than their current
|
|
name.
|
|
If the source is a directory then it acts exactly like the copy command.
|
|
.PP
|
|
So
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copyto\ src\ dst
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
where src and dst are rclone paths, either remote:path or /path/to/local
|
|
or C:.
|
|
.PP
|
|
This will:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
if\ src\ is\ file
|
|
\ \ \ \ copy\ it\ to\ dst,\ overwriting\ an\ existing\ file\ if\ it\ exists
|
|
if\ src\ is\ directory
|
|
\ \ \ \ copy\ it\ to\ dst,\ overwriting\ existing\ files\ if\ they\ exist
|
|
\ \ \ \ see\ copy\ command\ for\ full\ details
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This doesn't transfer unchanged files, testing by size and modification
|
|
time or MD5SUM.
|
|
It doesn't delete files from the destination.
|
|
.PP
|
|
\f[B]Note\f[]: Use the \f[C]\-P\f[]/\f[C]\-\-progress\f[] flag to view
|
|
real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copyto\ source:path\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ copyto
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone copyurl
|
|
.PP
|
|
Copy url content to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
Download urls content and copy it to destination without saving it in
|
|
tmp storage.
|
|
.PP
|
|
Setting \[en]auto\-filename flag will cause retrieving file name from
|
|
url and using it in destination path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copyurl\ https://example.com\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-a,\ \-\-auto\-filename\ \ \ Get\ the\ file\ name\ from\ the\ url\ and\ use\ it\ for\ destination\ file\ path
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ help\ for\ copyurl
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone cryptcheck
|
|
.PP
|
|
Cryptcheck checks the integrity of a crypted remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cryptcheck checks a remote against a crypted remote.
|
|
This is the equivalent of running rclone check, but able to check the
|
|
checksums of the crypted remote.
|
|
.PP
|
|
For it to work the underlying remote of the cryptedremote must support
|
|
some kind of checksum.
|
|
.PP
|
|
It works by reading the nonce from each file on the cryptedremote: and
|
|
using that to encrypt each file on the remote:.
|
|
It then checks the checksum of the underlying file on the cryptedremote:
|
|
against the checksum of the file it has just encrypted.
|
|
.PP
|
|
Use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cryptcheck\ /path/to/files\ encryptedremote:path
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You can use it like this also, but that will involve downloading all the
|
|
files in remote:path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cryptcheck\ remote:path\ encryptedremote:path
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
After it has run it will log the status of the encryptedremote:.
|
|
.PP
|
|
If you supply the \[en]one\-way flag, it will only check that files in
|
|
source match the files in destination, not the other way around.
|
|
Meaning extra files in destination that are not in the source will not
|
|
trigger an error.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cryptcheck\ remote:path\ cryptedremote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ help\ for\ cryptcheck
|
|
\ \ \ \ \ \ \-\-one\-way\ \ \ Check\ one\ way\ only,\ source\ files\ must\ exist\ on\ destination
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone cryptdecode
|
|
.PP
|
|
Cryptdecode returns unencrypted file names.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cryptdecode returns unencrypted file names when provided with a
|
|
list of encrypted file names.
|
|
List limit is 10 items.
|
|
.PP
|
|
If you supply the \[en]reverse flag, it will return encrypted file
|
|
names.
|
|
.PP
|
|
use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cryptdecode\ encryptedremote:\ encryptedfilename1\ encryptedfilename2
|
|
|
|
rclone\ cryptdecode\ \-\-reverse\ encryptedremote:\ filename1\ filename2
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ cryptdecode\ encryptedremote:\ encryptedfilename\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ help\ for\ cryptdecode
|
|
\ \ \ \ \ \ \-\-reverse\ \ \ Reverse\ cryptdecode,\ encrypts\ filenames
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone dbhashsum
|
|
.PP
|
|
Produces a Dropbox hash file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces a Dropbox hash file for all the objects in the path.
|
|
The hashes are calculated according to Dropbox content hash
|
|
rules (https://www.dropbox.com/developers/reference/content-hash).
|
|
The output is in the same format as md5sum and sha1sum.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ dbhashsum\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ dbhashsum
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone deletefile
|
|
.PP
|
|
Remove a single file from remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove a single file from remote.
|
|
Unlike \f[C]delete\f[] it cannot be used to remove a directory and it
|
|
doesn't obey include/exclude filters \- if the specified file exists, it
|
|
will always be removed.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ deletefile\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ deletefile
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone genautocomplete
|
|
.PP
|
|
Output completion script for a given shell.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a shell completion script for rclone.
|
|
Run with \[en]help to list the supported shells.
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ genautocomplete
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
bash (https://rclone.org/commands/rclone_genautocomplete_bash/) \-
|
|
Output bash completion script for rclone.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
zsh (https://rclone.org/commands/rclone_genautocomplete_zsh/) \- Output
|
|
zsh completion script for rclone.
|
|
.SS rclone genautocomplete bash
|
|
.PP
|
|
Output bash completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a bash shell autocompletion script for rclone.
|
|
.PP
|
|
This writes to /etc/bash_completion.d/rclone by default so will probably
|
|
need to be run with sudo or as root, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo\ rclone\ genautocomplete\ bash
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&.\ /etc/bash_completion
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ genautocomplete\ bash\ [output_file]\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ bash
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SS rclone genautocomplete zsh
|
|
.PP
|
|
Output zsh completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a zsh autocompletion script for rclone.
|
|
.PP
|
|
This writes to /usr/share/zsh/vendor\-completions/_rclone by default so
|
|
will probably need to be run with sudo or as root, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo\ rclone\ genautocomplete\ zsh
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
autoload\ \-U\ compinit\ &&\ compinit
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ genautocomplete\ zsh\ [output_file]\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ zsh
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SS rclone gendocs
|
|
.PP
|
|
Output markdown docs for rclone to the directory supplied.
|
|
.SS Synopsis
|
|
.PP
|
|
This produces markdown docs for the rclone commands to the directory
|
|
supplied.
|
|
These are in a format suitable for hugo to render into the rclone.org
|
|
website.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ gendocs\ output_directory\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ gendocs
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone hashsum
|
|
.PP
|
|
Produces an hashsum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces a hash file for all the objects in the path using the hash
|
|
named.
|
|
The output is in the same format as the standard md5sum/sha1sum tool.
|
|
.PP
|
|
Run without a hash to see the list of supported hashes, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ hashsum
|
|
Supported\ hashes\ are:
|
|
\ \ *\ MD5
|
|
\ \ *\ SHA\-1
|
|
\ \ *\ DropboxHash
|
|
\ \ *\ QuickXorHash
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ hashsum\ MD5\ remote:path
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ hashsum\ <hash>\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ hashsum
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone link
|
|
.PP
|
|
Generate public link to file/folder.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone link will create or retrieve a public link to the given file or
|
|
folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ link\ remote:path/to/file
|
|
rclone\ link\ remote:path/to/folder/
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If successful, the last line of the output will contain the link.
|
|
Exact capabilities depend on the remote, but the link will always be
|
|
created with the least constraints \[en] e.g.\ no expiry, no password
|
|
protection, accessible without account.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ link\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ link
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone listremotes
|
|
.PP
|
|
List all the remotes in the config file.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone listremotes lists all the available remotes from the config file.
|
|
.PP
|
|
When uses with the \-l flag it lists the types too.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ listremotes\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ listremotes
|
|
\ \ \ \ \ \ \-\-long\ \ \ Show\ the\ type\ as\ well\ as\ names.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone lsf
|
|
.PP
|
|
List directories and objects in remote:path formatted for parsing
|
|
.SS Synopsis
|
|
.PP
|
|
List the contents of the source path (directories and objects) to
|
|
standard output in a form which is easy to parse by scripts.
|
|
By default this will just be the names of the objects and directories,
|
|
one per line.
|
|
The directories will have a / suffix.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsf\ swift:bucket
|
|
bevajer5jef
|
|
canole
|
|
diwogej7
|
|
ferejej3gux/
|
|
fubuwic
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Use the \[en]format option to control what gets listed.
|
|
By default this is just the path, but you can use these parameters to
|
|
control the output:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
p\ \-\ path
|
|
s\ \-\ size
|
|
t\ \-\ modification\ time
|
|
h\ \-\ hash
|
|
i\ \-\ ID\ of\ object
|
|
o\ \-\ Original\ ID\ of\ underlying\ object
|
|
m\ \-\ MimeType\ of\ object\ if\ known
|
|
e\ \-\ encrypted\ name
|
|
T\ \-\ tier\ of\ storage\ if\ known,\ eg\ "Hot"\ or\ "Cool"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
So if you wanted the path, size and modification time, you would use
|
|
\[en]format \[lq]pst\[rq], or maybe \[en]format \[lq]tsp\[rq] to put the
|
|
path last.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsf\ \ \-\-format\ "tsp"\ swift:bucket
|
|
2016\-06\-25\ 18:55:41;60295;bevajer5jef
|
|
2016\-06\-25\ 18:55:43;90613;canole
|
|
2016\-06\-25\ 18:55:43;94467;diwogej7
|
|
2018\-04\-26\ 08:50:45;0;ferejej3gux/
|
|
2016\-06\-25\ 18:55:40;37600;fubuwic
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you specify \[lq]h\[rq] in the format you will get the MD5 hash by
|
|
default, use the \[lq]\[en]hash\[rq] flag to change which hash you want.
|
|
Note that this can be returned as an empty string if it isn't available
|
|
on the object (and for directories), \[lq]ERROR\[rq] if there was an
|
|
error reading it from the object and \[lq]UNSUPPORTED\[rq] if that
|
|
object does not support that hash type.
|
|
.PP
|
|
For example to emulate the md5sum command you can use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsf\ \-R\ \-\-hash\ MD5\ \-\-format\ hp\ \-\-separator\ "\ \ "\ \-\-files\-only\ .
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsf\ \-R\ \-\-hash\ MD5\ \-\-format\ hp\ \-\-separator\ "\ \ "\ \-\-files\-only\ swift:bucket\
|
|
7908e352297f0f530b84a756f188baa3\ \ bevajer5jef
|
|
cd65ac234e6fea5925974a51cdd865cc\ \ canole
|
|
03b5341b4f234b9d984d03ad076bae91\ \ diwogej7
|
|
8fd37c3810dd660778137ac3a66cc06d\ \ fubuwic
|
|
99713e14a4c4ff553acaf1930fad985b\ \ gixacuh7ku
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
(Though \[lq]rclone md5sum .\[rq] is an easier way of typing this.)
|
|
.PP
|
|
By default the separator is \[lq];\[rq] this can be changed with the
|
|
\[en]separator flag.
|
|
Note that separators aren't escaped in the path so putting it last is a
|
|
good strategy.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsf\ \ \-\-separator\ ","\ \-\-format\ "tshp"\ swift:bucket
|
|
2016\-06\-25\ 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef
|
|
2016\-06\-25\ 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole
|
|
2016\-06\-25\ 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
|
|
2018\-04\-26\ 08:52:53,0,,ferejej3gux/
|
|
2016\-06\-25\ 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You can output in CSV standard format.
|
|
This will escape things in " if they contain ,
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ lsf\ \-\-csv\ \-\-files\-only\ \-\-format\ ps\ remote:path
|
|
test.log,22355
|
|
test.sh,449
|
|
"this\ file\ contains\ a\ comma,\ in\ the\ file\ name.txt",6
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that the \[en]absolute parameter is useful for making lists of
|
|
files to pass to an rclone copy with the \[en]files\-from flag.
|
|
.PP
|
|
For example to find all the files modified within one day and copy those
|
|
only (without traversing the whole directory structure):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsf\ \-\-absolute\ \-\-files\-only\ \-\-max\-age\ 1d\ /path/to/local\ >\ new_files
|
|
rclone\ copy\ \-\-files\-from\ new_files\ /path/to/local\ remote:path
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[],\f[C]lsl\f[],\f[C]lsd\f[] are designed to be human readable.
|
|
\f[C]lsf\f[] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[] and \f[C]lsl\f[] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[],\f[C]lsf\f[],\f[C]lsjson\f[] do not
|
|
recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can't have empty directories (eg s3, swift, gcs, etc \-
|
|
the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsf\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-absolute\ \ \ \ \ \ \ \ \ \ \ Put\ a\ leading\ /\ in\ front\ of\ path\ names.
|
|
\ \ \ \ \ \ \-\-csv\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Output\ in\ CSV\ format.
|
|
\ \ \-d,\ \-\-dir\-slash\ \ \ \ \ \ \ \ \ \ Append\ a\ slash\ to\ directory\ names.\ (default\ true)
|
|
\ \ \ \ \ \ \-\-dirs\-only\ \ \ \ \ \ \ \ \ \ Only\ list\ directories.
|
|
\ \ \ \ \ \ \-\-files\-only\ \ \ \ \ \ \ \ \ Only\ list\ files.
|
|
\ \ \-F,\ \-\-format\ string\ \ \ \ \ \ Output\ format\ \-\ see\ \ help\ for\ details\ (default\ "p")
|
|
\ \ \ \ \ \ \-\-hash\ h\ \ \ \ \ \ \ \ \ \ \ \ \ Use\ this\ hash\ when\ h\ is\ used\ in\ the\ format\ MD5|SHA\-1|DropboxHash\ (default\ "MD5")
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ lsf
|
|
\ \ \-R,\ \-\-recursive\ \ \ \ \ \ \ \ \ \ Recurse\ into\ the\ listing.
|
|
\ \ \-s,\ \-\-separator\ string\ \ \ Separator\ for\ the\ items\ in\ the\ format.\ (default\ ";")
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone lsjson
|
|
.PP
|
|
List directories and objects in the path in JSON format.
|
|
.SS Synopsis
|
|
.PP
|
|
List directories and objects in the path in JSON format.
|
|
.PP
|
|
The output is an array of Items, where each Item looks like this
|
|
.PP
|
|
{ \[lq]Hashes\[rq] : { \[lq]SHA\-1\[rq] :
|
|
\[lq]f572d396fae9206628714fb2ce00f72e94f2258f\[rq], \[lq]MD5\[rq] :
|
|
\[lq]b1946ac92492d2347c6235b4d2611184\[rq], \[lq]DropboxHash\[rq] :
|
|
\[lq]ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc\[rq]
|
|
}, \[lq]ID\[rq]: \[lq]y2djkhiujf83u33\[rq], \[lq]OrigID\[rq]:
|
|
\[lq]UYOJVTUW00Q1RzTDA\[rq], \[lq]IsBucket\[rq] : false, \[lq]IsDir\[rq]
|
|
: false, \[lq]MimeType\[rq] : \[lq]application/octet\-stream\[rq],
|
|
\[lq]ModTime\[rq] : \[lq]2017\-05\-31T16:15:57.034468261+01:00\[rq],
|
|
\[lq]Name\[rq] : \[lq]file.txt\[rq], \[lq]Encrypted\[rq] :
|
|
\[lq]v0qpsdq8anpci8n929v3uu9338\[rq], \[lq]EncryptedPath\[rq] :
|
|
\[lq]kja9098349023498/v0qpsdq8anpci8n929v3uu9338\[rq], \[lq]Path\[rq] :
|
|
\[lq]full/path/goes/here/file.txt\[rq], \[lq]Size\[rq] : 6,
|
|
\[lq]Tier\[rq] : \[lq]hot\[rq], }
|
|
.PP
|
|
If \[en]hash is not specified the Hashes property won't be emitted.
|
|
.PP
|
|
If \[en]no\-modtime is specified then ModTime will be blank.
|
|
.PP
|
|
If \[en]encrypted is not specified the Encrypted won't be emitted.
|
|
.PP
|
|
If \[en]dirs\-only is not specified files in addition to directories are
|
|
returned
|
|
.PP
|
|
If \[en]files\-only is not specified directories in addition to the
|
|
files will be returned.
|
|
.PP
|
|
The Path field will only show folders below the remote path being
|
|
listed.
|
|
If \[lq]remote:path\[rq] contains the file \[lq]subfolder/file.txt\[rq],
|
|
the Path for \[lq]file.txt\[rq] will be \[lq]subfolder/file.txt\[rq],
|
|
not \[lq]remote:path/subfolder/file.txt\[rq].
|
|
When used without \[en]recursive the Path will always be the same as
|
|
Name.
|
|
.PP
|
|
If the directory is a bucket in a bucket based backend, then
|
|
\[lq]IsBucket\[rq] will be set to true.
|
|
This key won't be present unless it is \[lq]true\[rq].
|
|
.PP
|
|
The time is in RFC3339 format with up to nanosecond precision.
|
|
The number of decimal digits in the seconds will depend on the precision
|
|
that the remote can hold the times, so if times are accurate to the
|
|
nearest millisecond (eg Google Drive) then 3 digits will always be shown
|
|
(\[lq]2017\-05\-31T16:15:57.034+01:00\[rq]) whereas if the times are
|
|
accurate to the nearest second (Dropbox, Box, WebDav etc) no digits will
|
|
be shown (\[lq]2017\-05\-31T16:15:57+01:00\[rq]).
|
|
.PP
|
|
The whole output can be processed as a JSON blob, or alternatively it
|
|
can be processed line by line as each item is written one to a line.
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[],\f[C]lsl\f[],\f[C]lsd\f[] are designed to be human readable.
|
|
\f[C]lsf\f[] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[] and \f[C]lsl\f[] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[],\f[C]lsf\f[],\f[C]lsjson\f[] do not
|
|
recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can't have empty directories (eg s3, swift, gcs, etc \-
|
|
the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsjson\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-dirs\-only\ \ \ \ Show\ only\ directories\ in\ the\ listing.
|
|
\ \ \-M,\ \-\-encrypted\ \ \ \ Show\ the\ encrypted\ names.
|
|
\ \ \ \ \ \ \-\-files\-only\ \ \ Show\ only\ files\ in\ the\ listing.
|
|
\ \ \ \ \ \ \-\-hash\ \ \ \ \ \ \ \ \ Include\ hashes\ in\ the\ output\ (may\ take\ longer).
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ help\ for\ lsjson
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ Don\[aq]t\ read\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-original\ \ \ \ \ Show\ the\ ID\ of\ the\ underlying\ Object.
|
|
\ \ \-R,\ \-\-recursive\ \ \ \ Recurse\ into\ the\ listing.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone mount
|
|
.PP
|
|
Mount the remote as file system on a mountpoint.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone's cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
First set up your remote using \f[C]rclone\ config\f[].
|
|
Check it works with \f[C]rclone\ ls\f[] etc.
|
|
.PP
|
|
Start the mount like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mount\ remote:path/to/files\ /path/to/local/mount
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or on Windows like this where X: is an unused drive letter
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mount\ remote:path/to/files\ X:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
When the program ends, either via Ctrl+C or receiving a SIGINT or
|
|
SIGTERM signal, the mount is automatically stopped.
|
|
.PP
|
|
The umount operation can fail, for example when the mountpoint is busy.
|
|
When that happens, it is the user's responsibility to stop the mount
|
|
manually with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ Linux
|
|
fusermount\ \-u\ /path/to/local/mount
|
|
#\ OS\ X
|
|
umount\ /path/to/local/mount
|
|
\f[]
|
|
.fi
|
|
.SS Installing on Windows
|
|
.PP
|
|
To run rclone mount on Windows, you will need to download and install
|
|
WinFsp (http://www.secfs.net/winfsp/).
|
|
.PP
|
|
WinFsp is an open source (https://github.com/billziss-gh/winfsp) Windows
|
|
File System Proxy which makes it easy to write user space file systems
|
|
for Windows.
|
|
It provides a FUSE emulation layer which rclone uses combination with
|
|
cgofuse (https://github.com/billziss-gh/cgofuse).
|
|
Both of these packages are by Bill Zissimopoulos who was very helpful
|
|
during the implementation of rclone mount for Windows.
|
|
.SS Windows caveats
|
|
.PP
|
|
Note that drives created as Administrator are not visible by other
|
|
accounts (including the account that was elevated as Administrator).
|
|
So if you start a Windows drive from an Administrative Command Prompt
|
|
and then try to access the same drive from Explorer (which does not run
|
|
as Administrator), you will not be able to see the new drive.
|
|
.PP
|
|
The easiest way around this is to start the drive from a normal command
|
|
prompt.
|
|
It is also possible to start a drive from the SYSTEM account (using the
|
|
WinFsp.Launcher
|
|
infrastructure (https://github.com/billziss-gh/winfsp/wiki/WinFsp-Service-Architecture))
|
|
which creates drives accessible for everyone on the system or
|
|
alternatively using the nssm service manager (https://nssm.cc/usage).
|
|
.SS Limitations
|
|
.PP
|
|
Without the use of \[lq]\[en]vfs\-cache\-mode\[rq] this can only write
|
|
files sequentially, it can only seek when reading.
|
|
This means that many applications won't work with their files on an
|
|
rclone mount without \[lq]\[en]vfs\-cache\-mode writes\[rq] or
|
|
\[lq]\[en]vfs\-cache\-mode full\[rq].
|
|
See the File Caching section for more info.
|
|
.PP
|
|
The bucket based remotes (eg Swift, S3, Google Compute Storage, B2,
|
|
Hubic) do not support the concept of empty directories, so empty
|
|
directories will have a tendency to disappear once they fall out of the
|
|
directory cache.
|
|
.PP
|
|
Only supported on Linux, FreeBSD, OS X and Windows at the moment.
|
|
.SS rclone mount vs rclone sync/copy
|
|
.PP
|
|
File systems expect things to be 100% reliable, whereas cloud storage
|
|
systems are a long way from 100% reliable.
|
|
The rclone sync/copy commands cope with this with lots of retries.
|
|
However rclone mount can't use retries in the same way without making
|
|
local copies of the uploads.
|
|
Look at the file caching for solutions to make mount more reliable.
|
|
.SS Attribute caching
|
|
.PP
|
|
You can use the flag \[en]attr\-timeout to set the time the kernel
|
|
caches the attributes (size, modification time etc) for directory
|
|
entries.
|
|
.PP
|
|
The default is \[lq]1s\[rq] which caches files just long enough to avoid
|
|
too many callbacks to rclone from the kernel.
|
|
.PP
|
|
In theory 0s should be the correct value for filesystems which can
|
|
change outside the control of the kernel.
|
|
However this causes quite a few problems such as rclone using too much
|
|
memory (https://github.com/rclone/rclone/issues/2157), rclone not
|
|
serving files to
|
|
samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-issue/5112)
|
|
and excessive time listing
|
|
directories (https://github.com/rclone/rclone/issues/2095#issuecomment-371141147).
|
|
.PP
|
|
The kernel can cache the info about a file for the time given by
|
|
\[lq]\[en]attr\-timeout\[rq].
|
|
You may see corruption if the remote file changes length during this
|
|
window.
|
|
It will show up as either a truncated file or a file with garbage on the
|
|
end.
|
|
With \[lq]\[en]attr\-timeout 1s\[rq] this is very unlikely but not
|
|
impossible.
|
|
The higher you set \[lq]\[en]attr\-timeout\[rq] the more likely it is.
|
|
The default setting of \[lq]1s\[rq] is the lowest setting which
|
|
mitigates the problems above.
|
|
.PP
|
|
If you set it higher (`10s' or `1m' say) then the kernel will call back
|
|
to rclone less often making it more efficient, however there is more
|
|
chance of the corruption issue above.
|
|
.PP
|
|
If files don't change on the remote outside of the control of rclone
|
|
then there is no chance of corruption.
|
|
.PP
|
|
This is the same as setting the attr_timeout option in mount.fuse.
|
|
.SS Filters
|
|
.PP
|
|
Note that all the rclone filters can be used to select a subset of the
|
|
files to be visible in the mount.
|
|
.SS systemd
|
|
.PP
|
|
When running rclone mount as a systemd service, it is possible to use
|
|
Type=notify.
|
|
In this case the service will enter the started state after the
|
|
mountpoint has been successfully set up.
|
|
Units having the rclone mount service specified as a requirement will
|
|
see all files and folders immediately in this mode.
|
|
.SS chunked reading
|
|
.PP
|
|
\[en]vfs\-read\-chunk\-size will enable reading the source objects in
|
|
parts.
|
|
This can reduce the used download quota for some remotes by requesting
|
|
only chunks from the remote that are actually read at the cost of an
|
|
increased number of requests.
|
|
.PP
|
|
When \[en]vfs\-read\-chunk\-size\-limit is also specified and greater
|
|
than \[en]vfs\-read\-chunk\-size, the chunk size for each open file will
|
|
get doubled for each chunk read, until the specified value is reached.
|
|
A value of \-1 will disable the limit and the chunk size will grow
|
|
indefinitely.
|
|
.PP
|
|
With \[en]vfs\-read\-chunk\-size 100M and
|
|
\[en]vfs\-read\-chunk\-size\-limit 0 the following parts will be
|
|
downloaded: 0\-100M, 100M\-200M, 200M\-300M, 300M\-400M and so on.
|
|
When \[en]vfs\-read\-chunk\-size\-limit 500M is specified, the result
|
|
would be 0\-100M, 100M\-300M, 300M\-700M, 700M\-1200M, 1200M\-1700M and
|
|
so on.
|
|
.PP
|
|
Chunked reading will only work with \[en]vfs\-cache\-mode < full, as the
|
|
file will always be copied to the vfs cache before opening with
|
|
\[en]vfs\-cache\-mode full.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
|
flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGHUP\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget\ file=path/to/file\ dir=path/to/dir
|
|
\f[]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[] flag determines the amount of memory, that
|
|
will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won't be shared
|
|
between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size\ *\ open\ files\f[].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You'll need to enable VFS caching if you want, for example, to read and
|
|
write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
|
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
|
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\-\-vfs\-cache\-max\-size\ int\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won't get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can't be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mount\ remote:path\ /path/to/mountpoint\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-allow\-non\-empty\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ mounting\ over\ a\ non\-empty\ directory.
|
|
\ \ \ \ \ \ \-\-allow\-other\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ access\ to\ other\ users.
|
|
\ \ \ \ \ \ \-\-allow\-root\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ access\ to\ root\ user.
|
|
\ \ \ \ \ \ \-\-attr\-timeout\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ for\ which\ file/directory\ attributes\ are\ cached.\ (default\ 1s)
|
|
\ \ \ \ \ \ \-\-daemon\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Run\ mount\ as\ a\ daemon\ (background\ mode).
|
|
\ \ \ \ \ \ \-\-daemon\-timeout\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ limit\ for\ rclone\ to\ respond\ to\ kernel\ (not\ supported\ by\ all\ OSes).
|
|
\ \ \ \ \ \ \-\-debug\-fuse\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Debug\ the\ FUSE\ internals\ \-\ needs\ \-v.
|
|
\ \ \ \ \ \ \-\-default\-permissions\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Makes\ kernel\ enforce\ access\ control\ based\ on\ the\ file\ mode.
|
|
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
|
\ \ \ \ \ \ \-\-dir\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ permissions\ (default\ 0777)
|
|
\ \ \ \ \ \ \-\-file\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ File\ permissions\ (default\ 0666)
|
|
\ \ \ \ \ \ \-\-fuse\-flag\ stringArray\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Flags\ or\ arguments\ to\ be\ passed\ direct\ to\ libfuse/WinFsp.\ Repeat\ if\ required.
|
|
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ mount
|
|
\ \ \ \ \ \ \-\-max\-read\-ahead\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ The\ number\ of\ bytes\ that\ can\ be\ prefetched\ for\ sequential\ reads.\ (default\ 128k)
|
|
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
|
\ \ \-o,\ \-\-option\ stringArray\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Option\ for\ libfuse/WinFsp.\ Repeat\ if\ required.
|
|
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
|
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ CacheMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ \ \ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-case\-insensitive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ a\ file\ name\ not\ found,\ find\ a\ case\ insensitive\ match.
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ Read\ the\ source\ objects\ in\ chunks.\ (default\ 128M)
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\-limit\ SizeSuffix\ \ \ If\ greater\ than\ \-\-vfs\-read\-chunk\-size,\ double\ the\ chunk\ size\ after\ each\ chunk\ read,\ until\ the\ limit\ is\ reached.\ \[aq]off\[aq]\ is\ unlimited.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-volname\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Set\ the\ volume\ name\ (not\ supported\ by\ all\ OSes).
|
|
\ \ \ \ \ \ \-\-write\-back\-cache\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Makes\ kernel\ buffer\ writes\ before\ sending\ them\ to\ rclone.\ Without\ this,\ writethrough\ caching\ is\ used.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone moveto
|
|
.PP
|
|
Move file or directory from source to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
If source:path is a file or directory then it moves it to a file or
|
|
directory named dest:path.
|
|
.PP
|
|
This can be used to rename files or upload single files to other than
|
|
their existing name.
|
|
If the source is a directory then it acts exactly like the move command.
|
|
.PP
|
|
So
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ moveto\ src\ dst
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
where src and dst are rclone paths, either remote:path or /path/to/local
|
|
or C:.
|
|
.PP
|
|
This will:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
if\ src\ is\ file
|
|
\ \ \ \ move\ it\ to\ dst,\ overwriting\ an\ existing\ file\ if\ it\ exists
|
|
if\ src\ is\ directory
|
|
\ \ \ \ move\ it\ to\ dst,\ overwriting\ existing\ files\ if\ they\ exist
|
|
\ \ \ \ see\ move\ command\ for\ full\ details
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This doesn't transfer unchanged files, testing by size and modification
|
|
time or MD5SUM.
|
|
src will be deleted on successful transfer.
|
|
.PP
|
|
\f[B]Important\f[]: Since this can cause data loss, test first with the
|
|
\[en]dry\-run flag.
|
|
.PP
|
|
\f[B]Note\f[]: Use the \f[C]\-P\f[]/\f[C]\-\-progress\f[] flag to view
|
|
real\-time transfer statistics.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ moveto\ source:path\ dest:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ moveto
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone ncdu
|
|
.PP
|
|
Explore a remote with a text based user interface.
|
|
.SS Synopsis
|
|
.PP
|
|
This displays a text based user interface allowing the navigation of a
|
|
remote.
|
|
It is most useful for answering the question \- \[lq]What is using all
|
|
my disk space?\[rq].
|
|
.PP
|
|
To make the user interface it first scans the entire remote given and
|
|
builds an in memory representation.
|
|
rclone ncdu can be used during this scanning phase and you will see it
|
|
building up the directory structure as it goes along.
|
|
.PP
|
|
Here are the keys \- press `?' to toggle the help on and off
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ ↑,↓\ or\ k,j\ to\ Move
|
|
\ →,l\ to\ enter
|
|
\ ←,h\ to\ return
|
|
\ c\ toggle\ counts
|
|
\ g\ toggle\ graph
|
|
\ n,s,C\ sort\ by\ name,size,count
|
|
\ d\ delete\ file/directory
|
|
\ Y\ display\ current\ path
|
|
\ ^L\ refresh\ screen
|
|
\ ?\ to\ toggle\ help\ on\ and\ off
|
|
\ q/ESC/c\-C\ to\ quit
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
|
|
rclone remotes.
|
|
It is missing lots of features at the moment but is useful as it stands.
|
|
.PP
|
|
Note that it might take some time to delete big files/folders.
|
|
The UI won't respond in the meantime since the deletion is done
|
|
synchronously.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ncdu\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ ncdu
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone obscure
|
|
.PP
|
|
Obscure password for use in the rclone.conf
|
|
.SS Synopsis
|
|
.PP
|
|
Obscure password for use in the rclone.conf
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ obscure\ password\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ obscure
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone rc
|
|
.PP
|
|
Run a command against a running rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs a command against a running rclone.
|
|
Use the \[en]url flag to specify an non default URL to connect on.
|
|
This can be either a \[lq]:port\[rq] which is taken to mean
|
|
\[lq]http://localhost:port\[rq] or a \[lq]host:port\[rq] which is taken
|
|
to mean \[lq]http://host:port\[rq]
|
|
.PP
|
|
A username and password can be passed in with \[en]user and \[en]pass.
|
|
.PP
|
|
Note that \[en]rc\-addr, \[en]rc\-user, \[en]rc\-pass will be read also
|
|
for \[en]url, \[en]user, \[en]pass.
|
|
.PP
|
|
Arguments should be passed in as parameter=value.
|
|
.PP
|
|
The result will be returned as a JSON object by default.
|
|
.PP
|
|
The \[en]json parameter can be used to pass in a JSON blob as an input
|
|
instead of key=value arguments.
|
|
This is the only way of passing in more complicated values.
|
|
.PP
|
|
Use \[en]loopback to connect to the rclone instance running \[lq]rclone
|
|
rc\[rq].
|
|
This is very useful for testing commands without having to run an rclone
|
|
rc server, eg:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ \-\-loopback\ operations/about\ fs=/
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Use \[lq]rclone rc\[rq] to see a list of all possible commands.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ commands\ parameter\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ help\ for\ rc
|
|
\ \ \ \ \ \ \-\-json\ string\ \ \ Input\ JSON\ \-\ use\ instead\ of\ key=value\ args.
|
|
\ \ \ \ \ \ \-\-loopback\ \ \ \ \ \ If\ set\ connect\ to\ this\ rclone\ instance\ not\ via\ HTTP.
|
|
\ \ \ \ \ \ \-\-no\-output\ \ \ \ \ If\ set\ don\[aq]t\ output\ the\ JSON\ result.
|
|
\ \ \ \ \ \ \-\-pass\ string\ \ \ Password\ to\ use\ to\ connect\ to\ rclone\ remote\ control.
|
|
\ \ \ \ \ \ \-\-url\ string\ \ \ \ URL\ to\ connect\ to\ rclone\ remote\ control.\ (default\ "http://localhost:5572/")
|
|
\ \ \ \ \ \ \-\-user\ string\ \ \ Username\ to\ use\ to\ rclone\ remote\ control.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone rcat
|
|
.PP
|
|
Copies standard input to file on remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone rcat reads from standard input (stdin) and copies it to a single
|
|
remote file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
echo\ "hello\ world"\ |\ rclone\ rcat\ remote:path/to/file
|
|
ffmpeg\ \-\ |\ rclone\ rcat\ remote:path/to/file
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If the remote file already exists, it will be overwritten.
|
|
.PP
|
|
rcat will try to upload small files in a single request, which is
|
|
usually more efficient than the streaming/chunked upload endpoints,
|
|
which use multiple requests.
|
|
Exact behaviour depends on the remote.
|
|
What is considered a small file may be set through
|
|
\f[C]\-\-streaming\-upload\-cutoff\f[].
|
|
Uploading only starts after the cutoff is reached or if the file ends
|
|
before that.
|
|
The data must fit into RAM.
|
|
The cutoff needs to be small enough to adhere the limits of your remote,
|
|
please see there.
|
|
Generally speaking, setting this cutoff too high will decrease your
|
|
performance.
|
|
.PP
|
|
Note that the upload can also not be retried because the data is not
|
|
kept around until the upload succeeds.
|
|
If you need to transfer a lot of data, you're better off caching locally
|
|
and then \f[C]rclone\ move\f[] it to the destination.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rcat\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ rcat
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone rcd
|
|
.PP
|
|
Run rclone listening to remote control commands only.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs rclone so that it only listens to remote control commands.
|
|
.PP
|
|
This is useful if you are controlling rclone via the rc API.
|
|
.PP
|
|
If you pass in a path to a directory, rclone will serve that directory
|
|
for GET requests on the URL passed in.
|
|
It will also open the URL in the browser when rclone is run.
|
|
.PP
|
|
See the rc documentation (https://rclone.org/rc/) for more info on the
|
|
rc flags.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rcd\ <path\ to\ files\ to\ serve>*\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ rcd
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone rmdirs
|
|
.PP
|
|
Remove empty directories under the path.
|
|
.SS Synopsis
|
|
.PP
|
|
This removes any empty directories (or directories that only contain
|
|
empty directories) under the path that it finds, including the path if
|
|
it has nothing in.
|
|
.PP
|
|
If you supply the \[en]leave\-root flag, it will not remove the root
|
|
directory.
|
|
.PP
|
|
This is useful for tidying up remotes that rclone has left a lot of
|
|
empty directories in.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rmdirs\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ help\ for\ rmdirs
|
|
\ \ \ \ \ \ \-\-leave\-root\ \ \ Do\ not\ remove\ root\ directory\ if\ empty
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone serve
|
|
.PP
|
|
Serve a remote over a protocol.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve is used to serve a remote over a given protocol.
|
|
This command requires the use of a subcommand to specify the protocol,
|
|
eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ http\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Each subcommand has its own options which you can see in their help.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ <protocol>\ [opts]\ <remote>\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ serve
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) \-
|
|
Serve remote:path over DLNA
|
|
.IP \[bu] 2
|
|
rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) \-
|
|
Serve remote:path over FTP.
|
|
.IP \[bu] 2
|
|
rclone serve http (https://rclone.org/commands/rclone_serve_http/) \-
|
|
Serve the remote over HTTP.
|
|
.IP \[bu] 2
|
|
rclone serve restic (https://rclone.org/commands/rclone_serve_restic/)
|
|
\- Serve the remote for restic's REST API.
|
|
.IP \[bu] 2
|
|
rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) \-
|
|
Serve the remote over SFTP.
|
|
.IP \[bu] 2
|
|
rclone serve webdav (https://rclone.org/commands/rclone_serve_webdav/)
|
|
\- Serve remote:path over webdav.
|
|
.SS rclone serve dlna
|
|
.PP
|
|
Serve remote:path over DLNA
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve dlna is a DLNA media server for media stored in a rclone
|
|
remote.
|
|
Many devices, such as the Xbox and PlayStation, can automatically
|
|
discover this server in the LAN and play audio/video from it.
|
|
VLC is also supported.
|
|
Service discovery uses UDP multicast packets (SSDP) and will thus only
|
|
work on LANs.
|
|
.PP
|
|
Rclone will list all files present in the remote, without filtering
|
|
based on media formats or file extensions.
|
|
Additionally, there is no media transcoding support.
|
|
This means that some players might show files that they are not able to
|
|
play back correctly.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
.PP
|
|
Use \[en]name to choose the friendly server name, which is by default
|
|
\[lq]rclone (hostname)\[rq].
|
|
.PP
|
|
Use \[en]log\-trace in conjunction with \-vv to enable additional debug
|
|
logging of all UPNP traffic.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
|
flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGHUP\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget\ file=path/to/file\ dir=path/to/dir
|
|
\f[]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[] flag determines the amount of memory, that
|
|
will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won't be shared
|
|
between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size\ *\ open\ files\f[].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You'll need to enable VFS caching if you want, for example, to read and
|
|
write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
|
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
|
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\-\-vfs\-cache\-max\-size\ int\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won't get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can't be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ dlna\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ip:port\ or\ :port\ to\ bind\ the\ DLNA\ http\ server\ to.\ (default\ ":7879")
|
|
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
|
\ \ \ \ \ \ \-\-dir\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ permissions\ (default\ 0777)
|
|
\ \ \ \ \ \ \-\-file\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ File\ permissions\ (default\ 0666)
|
|
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ dlna
|
|
\ \ \ \ \ \ \-\-log\-trace\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ enable\ trace\ logging\ of\ SOAP\ traffic
|
|
\ \ \ \ \ \ \-\-name\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ name\ of\ DLNA\ server
|
|
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
|
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
|
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ CacheMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ \ \ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-case\-insensitive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ a\ file\ name\ not\ found,\ find\ a\ case\ insensitive\ match.
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ Read\ the\ source\ objects\ in\ chunks.\ (default\ 128M)
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\-limit\ SizeSuffix\ \ \ If\ greater\ than\ \-\-vfs\-read\-chunk\-size,\ double\ the\ chunk\ size\ after\ each\ chunk\ read,\ until\ the\ limit\ is\ reached.\ \[aq]off\[aq]\ is\ unlimited.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SS rclone serve ftp
|
|
.PP
|
|
Serve remote:path over FTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve ftp implements a basic ftp server to serve the remote over
|
|
FTP protocol.
|
|
This can be viewed with a ftp client or you can make a remote of type
|
|
ftp to read and write it.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can set a single username and password with the \[en]user and
|
|
\[en]pass flags.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
|
flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGHUP\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget\ file=path/to/file\ dir=path/to/dir
|
|
\f[]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[] flag determines the amount of memory, that
|
|
will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won't be shared
|
|
between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size\ *\ open\ files\f[].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You'll need to enable VFS caching if you want, for example, to read and
|
|
write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
|
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
|
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\-\-vfs\-cache\-max\-size\ int\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won't get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can't be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy\ /path/to/program\f[]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocl with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program's job is to take a \f[C]user\f[] and \f[C]pass\f[] on the
|
|
input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won't use configuration from environment variables or command line
|
|
options \- it is the job of the proxy program to make a complete config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
For example the program might take this on STDIN
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "user":\ "me",
|
|
\ \ \ \ "pass":\ "mypassword"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
And return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "type":\ "sftp",
|
|
\ \ \ \ "_root":\ "",
|
|
\ \ \ \ "_obscure":\ "pass",
|
|
\ \ \ \ "user":\ "me",
|
|
\ \ \ \ "pass":\ "mypassword",
|
|
\ \ \ \ "host":\ "sftp.example.com"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[] and \f[C]pass\f[] returned in the output to the host
|
|
given.
|
|
Note that since \f[C]_obscure\f[] is set to \f[C]pass\f[], rclone will
|
|
obscure the \f[C]pass\f[] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The progam can manipulate the supplied \f[C]user\f[] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[] be \f[C]user\@example.com\f[] and then set the
|
|
\f[C]host\f[] to \f[C]example.com\f[] in the output and the user to
|
|
\f[C]user\f[].
|
|
For security you'd probably want to restrict the \f[C]host\f[] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[] so only use that
|
|
for configuration, don't use \f[C]pass\f[].
|
|
This also means that if a user's password is changed the cache will need
|
|
to expire (which takes 5 mins) before it takes effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ ftp\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ or\ :Port\ to\ bind\ server\ to.\ (default\ "localhost:2121")
|
|
\ \ \ \ \ \ \-\-auth\-proxy\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ A\ program\ to\ use\ to\ create\ the\ backend\ from\ the\ auth.
|
|
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
|
\ \ \ \ \ \ \-\-dir\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ permissions\ (default\ 0777)
|
|
\ \ \ \ \ \ \-\-file\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ File\ permissions\ (default\ 0666)
|
|
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ ftp
|
|
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
|
\ \ \ \ \ \ \-\-pass\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Password\ for\ authentication.\ (empty\ value\ allow\ every\ password)
|
|
\ \ \ \ \ \ \-\-passive\-port\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Passive\ port\ range\ to\ use.\ (default\ "30000\-32000")
|
|
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-public\-ip\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Public\ IP\ address\ to\ advertise\ for\ passive\ connections.
|
|
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
|
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
|
\ \ \ \ \ \ \-\-user\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ User\ name\ for\ authentication.\ (default\ "anonymous")
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ CacheMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ \ \ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-case\-insensitive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ a\ file\ name\ not\ found,\ find\ a\ case\ insensitive\ match.
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ Read\ the\ source\ objects\ in\ chunks.\ (default\ 128M)
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\-limit\ SizeSuffix\ \ \ If\ greater\ than\ \-\-vfs\-read\-chunk\-size,\ double\ the\ chunk\ size\ after\ each\ chunk\ read,\ until\ the\ limit\ is\ reached.\ \[aq]off\[aq]\ is\ unlimited.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SS rclone serve http
|
|
.PP
|
|
Serve the remote over HTTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve http implements a basic web server to serve the remote over
|
|
HTTP.
|
|
This can be viewed in a web browser or you can make a remote of type
|
|
http read from it.
|
|
.PP
|
|
You can use the filter flags (eg \[en]include, \[en]exclude) to control
|
|
what is served.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\[en]bwlimit will be respected for file transfers.
|
|
Use \[en]stats to control the stats printing.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\[en]server\-read\-timeout and \[en]server\-write\-timeout can be used
|
|
to control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\[en]max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\[en]baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \[en]baseurl \[lq]/rclone\[rq] then rclone would serve from
|
|
a URL starting with \[lq]/rclone/\[rq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[lq]/\[rq] on
|
|
\[en]baseurl, so \[en]baseurl \[lq]rclone\[rq], \[en]baseurl
|
|
\[lq]/rclone\[rq] and \[en]baseurl \[lq]/rclone/\[rq] are all treated
|
|
identically.
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \[en]user and \[en]pass flags.
|
|
.PP
|
|
Use \[en]htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch\ htpasswd
|
|
htpasswd\ \-B\ htpasswd\ user
|
|
htpasswd\ \-B\ htpasswd\ anotherUser
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \[en]realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \[en]cert and \[en]key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \[en]client\-ca also.
|
|
.PP
|
|
\[en]cert should be a either a PEM encoded certificate or a
|
|
concatenation of that with the CA certificate.
|
|
\[en]key should be the PEM encoded private key and \[en]client\-ca
|
|
should be the PEM encoded client certificate authority certificate.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
|
flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGHUP\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget\ file=path/to/file\ dir=path/to/dir
|
|
\f[]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[] flag determines the amount of memory, that
|
|
will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won't be shared
|
|
between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size\ *\ open\ files\f[].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You'll need to enable VFS caching if you want, for example, to read and
|
|
write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
|
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
|
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\-\-vfs\-cache\-max\-size\ int\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won't get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can't be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ http\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ or\ :Port\ to\ bind\ server\ to.\ (default\ "localhost:8080")
|
|
\ \ \ \ \ \ \-\-baseurl\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Prefix\ for\ URLs\ \-\ leave\ blank\ for\ root.
|
|
\ \ \ \ \ \ \-\-cert\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSL\ PEM\ key\ (concatenation\ of\ certificate\ and\ CA\ certificate)
|
|
\ \ \ \ \ \ \-\-client\-ca\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Client\ certificate\ authority\ to\ verify\ clients\ with
|
|
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
|
\ \ \ \ \ \ \-\-dir\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ permissions\ (default\ 0777)
|
|
\ \ \ \ \ \ \-\-file\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ File\ permissions\ (default\ 0666)
|
|
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ http
|
|
\ \ \ \ \ \ \-\-htpasswd\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ htpasswd\ file\ \-\ if\ not\ provided\ no\ authentication\ is\ done
|
|
\ \ \ \ \ \ \-\-key\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSL\ PEM\ Private\ key
|
|
\ \ \ \ \ \ \-\-max\-header\-bytes\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Maximum\ size\ of\ request\ header\ (default\ 4096)
|
|
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
|
\ \ \ \ \ \ \-\-pass\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Password\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
|
\ \ \ \ \ \ \-\-realm\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ realm\ for\ authentication\ (default\ "rclone")
|
|
\ \ \ \ \ \ \-\-server\-read\-timeout\ duration\ \ \ \ \ \ \ \ \ \ \ Timeout\ for\ server\ reading\ data\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-server\-write\-timeout\ duration\ \ \ \ \ \ \ \ \ \ Timeout\ for\ server\ writing\ data\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
|
\ \ \ \ \ \ \-\-user\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ User\ name\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ CacheMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ \ \ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-case\-insensitive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ a\ file\ name\ not\ found,\ find\ a\ case\ insensitive\ match.
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ Read\ the\ source\ objects\ in\ chunks.\ (default\ 128M)
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\-limit\ SizeSuffix\ \ \ If\ greater\ than\ \-\-vfs\-read\-chunk\-size,\ double\ the\ chunk\ size\ after\ each\ chunk\ read,\ until\ the\ limit\ is\ reached.\ \[aq]off\[aq]\ is\ unlimited.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SS rclone serve restic
|
|
.PP
|
|
Serve the remote for restic's REST API.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve restic implements restic's REST backend API over HTTP.
|
|
This allows restic to use rclone as a data storage mechanism for cloud
|
|
providers that restic does not support directly.
|
|
.PP
|
|
Restic (https://restic.net/) is a command line program for doing
|
|
backups.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\[en]bwlimit will be respected for file transfers.
|
|
Use \[en]stats to control the stats printing.
|
|
.SS Setting up rclone for use by restic
|
|
.PP
|
|
First set up a remote for your chosen cloud provider (/docs/#configure).
|
|
.PP
|
|
Once you have set up the remote, check it is working with, for example
|
|
\[lq]rclone lsd remote:\[rq].
|
|
You may have called the remote something other than \[lq]remote:\[rq] \-
|
|
just substitute whatever you called it in the following instructions.
|
|
.PP
|
|
Now start the rclone restic server
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ restic\ \-v\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Where you can replace \[lq]backup\[rq] in the above by whatever path in
|
|
the remote you wish to use.
|
|
.PP
|
|
By default this will serve on \[lq]localhost:8080\[rq] you can change
|
|
this with use of the \[lq]\[en]addr\[rq] flag.
|
|
.PP
|
|
You might wish to start this server on boot.
|
|
.SS Setting up restic to use rclone
|
|
.PP
|
|
Now you can follow the restic
|
|
instructions (http://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server)
|
|
on setting up restic.
|
|
.PP
|
|
Note that you will need restic 0.8.2 or later to interoperate with
|
|
rclone.
|
|
.PP
|
|
For the example above you will want to use
|
|
\[lq]http://localhost:8080/\[rq] as the URL for the REST server.
|
|
.PP
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ export\ RESTIC_REPOSITORY=rest:http://localhost:8080/
|
|
$\ export\ RESTIC_PASSWORD=yourpassword
|
|
$\ restic\ init
|
|
created\ restic\ backend\ 8b1a4b56ae\ at\ rest:http://localhost:8080/
|
|
|
|
Please\ note\ that\ knowledge\ of\ your\ password\ is\ required\ to\ access
|
|
the\ repository.\ Losing\ your\ password\ means\ that\ your\ data\ is
|
|
irrecoverably\ lost.
|
|
$\ restic\ backup\ /path/to/files/to/backup
|
|
scan\ [/path/to/files/to/backup]
|
|
scanned\ 189\ directories,\ 312\ files\ in\ 0:00
|
|
[0:00]\ 100.00%\ \ 38.128\ MiB\ /\ 38.128\ MiB\ \ 501\ /\ 501\ items\ \ 0\ errors\ \ ETA\ 0:00
|
|
duration:\ 0:00
|
|
snapshot\ 45c8fdd8\ saved
|
|
\f[]
|
|
.fi
|
|
.SS Multiple repositories
|
|
.PP
|
|
Note that you can use the endpoint to host multiple repositories.
|
|
Do this by adding a directory name or path after the URL.
|
|
Note that these \f[B]must\f[] end with /.
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ export\ RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
|
|
#\ backup\ user1\ stuff
|
|
$\ export\ RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
|
|
#\ backup\ user2\ stuff
|
|
\f[]
|
|
.fi
|
|
.SS Private repositories
|
|
.PP
|
|
The \[lq]\[en]private\-repos\[rq] flag can be used to limit users to
|
|
repositories starting with a path of \[lq]//\[rq].
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\[en]server\-read\-timeout and \[en]server\-write\-timeout can be used
|
|
to control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\[en]max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\[en]baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \[en]baseurl \[lq]/rclone\[rq] then rclone would serve from
|
|
a URL starting with \[lq]/rclone/\[rq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[lq]/\[rq] on
|
|
\[en]baseurl, so \[en]baseurl \[lq]rclone\[rq], \[en]baseurl
|
|
\[lq]/rclone\[rq] and \[en]baseurl \[lq]/rclone/\[rq] are all treated
|
|
identically.
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \[en]user and \[en]pass flags.
|
|
.PP
|
|
Use \[en]htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch\ htpasswd
|
|
htpasswd\ \-B\ htpasswd\ user
|
|
htpasswd\ \-B\ htpasswd\ anotherUser
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \[en]realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \[en]cert and \[en]key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \[en]client\-ca also.
|
|
.PP
|
|
\[en]cert should be a either a PEM encoded certificate or a
|
|
concatenation of that with the CA certificate.
|
|
\[en]key should be the PEM encoded private key and \[en]client\-ca
|
|
should be the PEM encoded client certificate authority certificate.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ restic\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ or\ :Port\ to\ bind\ server\ to.\ (default\ "localhost:8080")
|
|
\ \ \ \ \ \ \-\-append\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ disallow\ deletion\ of\ repository\ data
|
|
\ \ \ \ \ \ \-\-baseurl\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Prefix\ for\ URLs\ \-\ leave\ blank\ for\ root.
|
|
\ \ \ \ \ \ \-\-cert\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSL\ PEM\ key\ (concatenation\ of\ certificate\ and\ CA\ certificate)
|
|
\ \ \ \ \ \ \-\-client\-ca\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Client\ certificate\ authority\ to\ verify\ clients\ with
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ restic
|
|
\ \ \ \ \ \ \-\-htpasswd\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ htpasswd\ file\ \-\ if\ not\ provided\ no\ authentication\ is\ done
|
|
\ \ \ \ \ \ \-\-key\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSL\ PEM\ Private\ key
|
|
\ \ \ \ \ \ \-\-max\-header\-bytes\ int\ \ \ \ \ \ \ \ \ \ \ \ Maximum\ size\ of\ request\ header\ (default\ 4096)
|
|
\ \ \ \ \ \ \-\-pass\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Password\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-private\-repos\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ users\ can\ only\ access\ their\ private\ repo
|
|
\ \ \ \ \ \ \-\-realm\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ realm\ for\ authentication\ (default\ "rclone")
|
|
\ \ \ \ \ \ \-\-server\-read\-timeout\ duration\ \ \ \ Timeout\ for\ server\ reading\ data\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-server\-write\-timeout\ duration\ \ \ Timeout\ for\ server\ writing\ data\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-stdio\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ run\ an\ HTTP2\ server\ on\ stdin/stdout
|
|
\ \ \ \ \ \ \-\-user\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ User\ name\ for\ authentication.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SS rclone serve sftp
|
|
.PP
|
|
Serve the remote over SFTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve sftp implements an SFTP server to serve the remote over
|
|
SFTP.
|
|
This can be used with an SFTP client or you can make a remote of type
|
|
sftp to use with it.
|
|
.PP
|
|
You can use the filter flags (eg \[en]include, \[en]exclude) to control
|
|
what is served.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\[en]bwlimit will be respected for file transfers.
|
|
Use \[en]stats to control the stats printing.
|
|
.PP
|
|
You must provide some means of authentication, either with
|
|
\[en]user/\[en]pass, an authorized keys file (specify location with
|
|
\[en]authorized\-keys \- the default is the same as ssh) or set the
|
|
\[en]no\-auth flag for no authentication when logging in.
|
|
.PP
|
|
Note that this also implements a small number of shell commands so that
|
|
it can provide md5sum/sha1sum/df information for the rclone sftp
|
|
backend.
|
|
This means that is can support SHA1SUMs, MD5SUMs and the about command
|
|
when paired with the rclone sftp backend.
|
|
.PP
|
|
If you don't supply a \[en]key then rclone will generate one and cache
|
|
it for later use.
|
|
.PP
|
|
By default the server binds to localhost:2022 \- if you want it to be
|
|
reachable externally then supply \[lq]\[en]addr :2022\[rq] for example.
|
|
.PP
|
|
Note that the default of \[lq]\[en]vfs\-cache\-mode off\[rq] is fine for
|
|
the rclone sftp backend, but it may not be with other SFTP clients.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
|
flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGHUP\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget\ file=path/to/file\ dir=path/to/dir
|
|
\f[]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[] flag determines the amount of memory, that
|
|
will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won't be shared
|
|
between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size\ *\ open\ files\f[].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You'll need to enable VFS caching if you want, for example, to read and
|
|
write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
|
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
|
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\-\-vfs\-cache\-max\-size\ int\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won't get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can't be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy\ /path/to/program\f[]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocl with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program's job is to take a \f[C]user\f[] and \f[C]pass\f[] on the
|
|
input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won't use configuration from environment variables or command line
|
|
options \- it is the job of the proxy program to make a complete config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
For example the program might take this on STDIN
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "user":\ "me",
|
|
\ \ \ \ "pass":\ "mypassword"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
And return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "type":\ "sftp",
|
|
\ \ \ \ "_root":\ "",
|
|
\ \ \ \ "_obscure":\ "pass",
|
|
\ \ \ \ "user":\ "me",
|
|
\ \ \ \ "pass":\ "mypassword",
|
|
\ \ \ \ "host":\ "sftp.example.com"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[] and \f[C]pass\f[] returned in the output to the host
|
|
given.
|
|
Note that since \f[C]_obscure\f[] is set to \f[C]pass\f[], rclone will
|
|
obscure the \f[C]pass\f[] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The progam can manipulate the supplied \f[C]user\f[] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[] be \f[C]user\@example.com\f[] and then set the
|
|
\f[C]host\f[] to \f[C]example.com\f[] in the output and the user to
|
|
\f[C]user\f[].
|
|
For security you'd probably want to restrict the \f[C]host\f[] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[] so only use that
|
|
for configuration, don't use \f[C]pass\f[].
|
|
This also means that if a user's password is changed the cache will need
|
|
to expire (which takes 5 mins) before it takes effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ sftp\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ or\ :Port\ to\ bind\ server\ to.\ (default\ "localhost:2022")
|
|
\ \ \ \ \ \ \-\-auth\-proxy\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ A\ program\ to\ use\ to\ create\ the\ backend\ from\ the\ auth.
|
|
\ \ \ \ \ \ \-\-authorized\-keys\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Authorized\ keys\ file\ (default\ "~/.ssh/authorized_keys")
|
|
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
|
\ \ \ \ \ \ \-\-dir\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ permissions\ (default\ 0777)
|
|
\ \ \ \ \ \ \-\-file\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ File\ permissions\ (default\ 0666)
|
|
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ sftp
|
|
\ \ \ \ \ \ \-\-key\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSH\ private\ key\ file\ (leave\ blank\ to\ auto\ generate)
|
|
\ \ \ \ \ \ \-\-no\-auth\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ connections\ with\ no\ authentication\ if\ set.
|
|
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
|
\ \ \ \ \ \ \-\-pass\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Password\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
|
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
|
\ \ \ \ \ \ \-\-user\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ User\ name\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ CacheMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ \ \ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-case\-insensitive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ a\ file\ name\ not\ found,\ find\ a\ case\ insensitive\ match.
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ Read\ the\ source\ objects\ in\ chunks.\ (default\ 128M)
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\-limit\ SizeSuffix\ \ \ If\ greater\ than\ \-\-vfs\-read\-chunk\-size,\ double\ the\ chunk\ size\ after\ each\ chunk\ read,\ until\ the\ limit\ is\ reached.\ \[aq]off\[aq]\ is\ unlimited.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SS rclone serve webdav
|
|
.PP
|
|
Serve remote:path over webdav.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve webdav implements a basic webdav server to serve the remote
|
|
over HTTP via the webdav protocol.
|
|
This can be viewed with a webdav client, through a web browser, or you
|
|
can make a remote of type webdav to read and write it.
|
|
.SS Webdav options
|
|
.SS \[en]etag\-hash
|
|
.PP
|
|
This controls the ETag header.
|
|
Without this flag the ETag will be based on the ModTime and Size of the
|
|
object.
|
|
.PP
|
|
If this flag is set to \[lq]auto\[rq] then rclone will choose the first
|
|
supported hash on the backend or you can use a named hash such as
|
|
\[lq]MD5\[rq] or \[lq]SHA\-1\[rq].
|
|
.PP
|
|
Use \[lq]rclone hashsum\[rq] to see the full list.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\[en]server\-read\-timeout and \[en]server\-write\-timeout can be used
|
|
to control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\[en]max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\[en]baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \[en]baseurl \[lq]/rclone\[rq] then rclone would serve from
|
|
a URL starting with \[lq]/rclone/\[rq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[lq]/\[rq] on
|
|
\[en]baseurl, so \[en]baseurl \[lq]rclone\[rq], \[en]baseurl
|
|
\[lq]/rclone\[rq] and \[en]baseurl \[lq]/rclone/\[rq] are all treated
|
|
identically.
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \[en]user and \[en]pass flags.
|
|
.PP
|
|
Use \[en]htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch\ htpasswd
|
|
htpasswd\ \-B\ htpasswd\ user
|
|
htpasswd\ \-B\ htpasswd\ anotherUser
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \[en]realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \[en]cert and \[en]key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \[en]client\-ca also.
|
|
.PP
|
|
\[en]cert should be a either a PEM encoded certificate or a
|
|
concatenation of that with the CA certificate.
|
|
\[en]key should be the PEM encoded private key and \[en]client\-ca
|
|
should be the PEM encoded client certificate authority certificate.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
|
flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGHUP\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ vfs/forget\ file=path/to/file\ dir=path/to/dir
|
|
\f[]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[] flag determines the amount of memory, that
|
|
will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won't be shared
|
|
between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size\ *\ open\ files\f[].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You'll need to enable VFS caching if you want, for example, to read and
|
|
write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
|
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
|
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\-\-vfs\-cache\-max\-size\ int\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won't get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can't be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can't be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can't be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy\ /path/to/program\f[]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocl with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program's job is to take a \f[C]user\f[] and \f[C]pass\f[] on the
|
|
input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won't use configuration from environment variables or command line
|
|
options \- it is the job of the proxy program to make a complete config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
For example the program might take this on STDIN
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "user":\ "me",
|
|
\ \ \ \ "pass":\ "mypassword"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
And return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "type":\ "sftp",
|
|
\ \ \ \ "_root":\ "",
|
|
\ \ \ \ "_obscure":\ "pass",
|
|
\ \ \ \ "user":\ "me",
|
|
\ \ \ \ "pass":\ "mypassword",
|
|
\ \ \ \ "host":\ "sftp.example.com"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[] and \f[C]pass\f[] returned in the output to the host
|
|
given.
|
|
Note that since \f[C]_obscure\f[] is set to \f[C]pass\f[], rclone will
|
|
obscure the \f[C]pass\f[] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The progam can manipulate the supplied \f[C]user\f[] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[] be \f[C]user\@example.com\f[] and then set the
|
|
\f[C]host\f[] to \f[C]example.com\f[] in the output and the user to
|
|
\f[C]user\f[].
|
|
For security you'd probably want to restrict the \f[C]host\f[] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[] so only use that
|
|
for configuration, don't use \f[C]pass\f[].
|
|
This also means that if a user's password is changed the cache will need
|
|
to expire (which takes 5 mins) before it takes effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ serve\ webdav\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ or\ :Port\ to\ bind\ server\ to.\ (default\ "localhost:8080")
|
|
\ \ \ \ \ \ \-\-auth\-proxy\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ A\ program\ to\ use\ to\ create\ the\ backend\ from\ the\ auth.
|
|
\ \ \ \ \ \ \-\-baseurl\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Prefix\ for\ URLs\ \-\ leave\ blank\ for\ root.
|
|
\ \ \ \ \ \ \-\-cert\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSL\ PEM\ key\ (concatenation\ of\ certificate\ and\ CA\ certificate)
|
|
\ \ \ \ \ \ \-\-client\-ca\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Client\ certificate\ authority\ to\ verify\ clients\ with
|
|
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
|
\ \ \ \ \ \ \-\-dir\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ permissions\ (default\ 0777)
|
|
\ \ \ \ \ \ \-\-disable\-dir\-list\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Disable\ HTML\ directory\ list\ on\ GET\ request\ for\ a\ directory
|
|
\ \ \ \ \ \ \-\-etag\-hash\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Which\ hash\ to\ use\ for\ the\ ETag,\ or\ auto\ or\ blank\ for\ off
|
|
\ \ \ \ \ \ \-\-file\-perms\ FileMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ File\ permissions\ (default\ 0666)
|
|
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ webdav
|
|
\ \ \ \ \ \ \-\-htpasswd\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ htpasswd\ file\ \-\ if\ not\ provided\ no\ authentication\ is\ done
|
|
\ \ \ \ \ \ \-\-key\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SSL\ PEM\ Private\ key
|
|
\ \ \ \ \ \ \-\-max\-header\-bytes\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Maximum\ size\ of\ request\ header\ (default\ 4096)
|
|
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
|
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
|
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
|
\ \ \ \ \ \ \-\-pass\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Password\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
|
\ \ \ \ \ \ \-\-realm\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ realm\ for\ authentication\ (default\ "rclone")
|
|
\ \ \ \ \ \ \-\-server\-read\-timeout\ duration\ \ \ \ \ \ \ \ \ \ \ Timeout\ for\ server\ reading\ data\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-server\-write\-timeout\ duration\ \ \ \ \ \ \ \ \ \ Timeout\ for\ server\ writing\ data\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 1000)
|
|
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
|
\ \ \ \ \ \ \-\-user\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ User\ name\ for\ authentication.
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-max\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ Max\ total\ size\ of\ objects\ in\ the\ cache.\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ CacheMode\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ off)
|
|
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ \ \ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
|
\ \ \ \ \ \ \-\-vfs\-case\-insensitive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ a\ file\ name\ not\ found,\ find\ a\ case\ insensitive\ match.
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ Read\ the\ source\ objects\ in\ chunks.\ (default\ 128M)
|
|
\ \ \ \ \ \ \-\-vfs\-read\-chunk\-size\-limit\ SizeSuffix\ \ \ If\ greater\ than\ \-\-vfs\-read\-chunk\-size,\ double\ the\ chunk\ size\ after\ each\ chunk\ read,\ until\ the\ limit\ is\ reached.\ \[aq]off\[aq]\ is\ unlimited.\ (default\ off)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SS rclone settier
|
|
.PP
|
|
Changes storage class/tier of objects in remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone settier changes storage tier or class at remote if supported.
|
|
Few cloud storage services provides different storage classes on
|
|
objects, for example AWS S3 and Glacier, Azure Blob storage \- Hot, Cool
|
|
and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline
|
|
etc.
|
|
.PP
|
|
Note that, certain tier changes make objects not available to access
|
|
immediately.
|
|
For example tiering to archive in azure blob storage makes objects in
|
|
frozen state, user can restore by setting tier to Hot/Cool, similarly S3
|
|
to Glacier makes object inaccessible.true
|
|
.PP
|
|
You can use it to tier single object
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ settier\ Cool\ remote:path/file
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or use rclone filters to set tier on only specific files
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ \-\-include\ "*.txt"\ settier\ Hot\ remote:path/dir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Or just provide remote directory and all files in directory will be
|
|
tiered
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ settier\ tier\ remote:path/dir
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ settier\ tier\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ help\ for\ settier
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone touch
|
|
.PP
|
|
Create new file or change file modification time.
|
|
.SS Synopsis
|
|
.PP
|
|
Create new file or change file modification time.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ touch\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ touch
|
|
\ \ \-C,\ \-\-no\-create\ \ \ \ \ \ \ \ \ \ Do\ not\ create\ the\ file\ if\ it\ does\ not\ exist.
|
|
\ \ \-t,\ \-\-timestamp\ string\ \ \ Change\ the\ modification\ times\ to\ the\ specified\ time\ instead\ of\ the\ current\ time\ of\ day.\ The\ argument\ is\ of\ the\ form\ \[aq]YYMMDD\[aq]\ (ex.\ 17.10.30)\ or\ \[aq]YYYY\-MM\-DDTHH:MM:SS\[aq]\ (ex.\ 2006\-01\-02T15:04:05)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS rclone tree
|
|
.PP
|
|
List the contents of the remote in a tree like fashion.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone tree lists the contents of a remote in a similar way to the unix
|
|
tree command.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ tree\ remote:path
|
|
/
|
|
├──\ file1
|
|
├──\ file2
|
|
├──\ file3
|
|
└──\ subdir
|
|
\ \ \ \ ├──\ file4
|
|
\ \ \ \ └──\ file5
|
|
|
|
1\ directories,\ 5\ files
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You can use any of the filtering options with the tree command (eg
|
|
\[en]include and \[en]exclude).
|
|
You can also use \[en]fast\-list.
|
|
.PP
|
|
The tree command has many options for controlling the listing which are
|
|
compatible with the tree command.
|
|
Note that not all of them have short options as they conflict with
|
|
rclone's short options.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ tree\ remote:path\ [flags]
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-a,\ \-\-all\ \ \ \ \ \ \ \ \ \ \ \ \ All\ files\ are\ listed\ (list\ .\ files\ too).
|
|
\ \ \-C,\ \-\-color\ \ \ \ \ \ \ \ \ \ \ Turn\ colorization\ on\ always.
|
|
\ \ \-d,\ \-\-dirs\-only\ \ \ \ \ \ \ List\ directories\ only.
|
|
\ \ \ \ \ \ \-\-dirsfirst\ \ \ \ \ \ \ List\ directories\ before\ files\ (\-U\ disables).
|
|
\ \ \ \ \ \ \-\-full\-path\ \ \ \ \ \ \ Print\ the\ full\ path\ prefix\ for\ each\ file.
|
|
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ help\ for\ tree
|
|
\ \ \ \ \ \ \-\-human\ \ \ \ \ \ \ \ \ \ \ Print\ the\ size\ in\ a\ more\ human\ readable\ way.
|
|
\ \ \ \ \ \ \-\-level\ int\ \ \ \ \ \ \ Descend\ only\ level\ directories\ deep.
|
|
\ \ \-D,\ \-\-modtime\ \ \ \ \ \ \ \ \ Print\ the\ date\ of\ last\ modification.
|
|
\ \ \-i,\ \-\-noindent\ \ \ \ \ \ \ \ Don\[aq]t\ print\ indentation\ lines.
|
|
\ \ \ \ \ \ \-\-noreport\ \ \ \ \ \ \ \ Turn\ off\ file/directory\ count\ at\ end\ of\ tree\ listing.
|
|
\ \ \-o,\ \-\-output\ string\ \ \ Output\ to\ file\ instead\ of\ stdout.
|
|
\ \ \-p,\ \-\-protections\ \ \ \ \ Print\ the\ protections\ for\ each\ file.
|
|
\ \ \-Q,\ \-\-quote\ \ \ \ \ \ \ \ \ \ \ Quote\ filenames\ with\ double\ quotes.
|
|
\ \ \-s,\ \-\-size\ \ \ \ \ \ \ \ \ \ \ \ Print\ the\ size\ in\ bytes\ of\ each\ file.
|
|
\ \ \ \ \ \ \-\-sort\ string\ \ \ \ \ Select\ sort:\ name,version,size,mtime,ctime.
|
|
\ \ \ \ \ \ \-\-sort\-ctime\ \ \ \ \ \ Sort\ files\ by\ last\ status\ change\ time.
|
|
\ \ \-t,\ \-\-sort\-modtime\ \ \ \ Sort\ files\ by\ last\ modification\ time.
|
|
\ \ \-r,\ \-\-sort\-reverse\ \ \ \ Reverse\ the\ order\ of\ the\ sort.
|
|
\ \ \-U,\ \-\-unsorted\ \ \ \ \ \ \ \ Leave\ files\ unsorted.
|
|
\ \ \ \ \ \ \-\-version\ \ \ \ \ \ \ \ \ Sort\ files\ alphanumerically\ by\ version.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS Copying single files
|
|
.PP
|
|
rclone normally syncs or copies directories.
|
|
However, if the source remote points to a file, rclone will just copy
|
|
that file.
|
|
The destination remote must point to a directory \- rclone will give the
|
|
error
|
|
\f[C]Failed\ to\ create\ file\ system\ for\ "remote:file":\ is\ a\ file\ not\ a\ directory\f[]
|
|
if it isn't.
|
|
.PP
|
|
For example, suppose you have a remote with a file in called
|
|
\f[C]test.jpg\f[], then you could copy just that file like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ remote:test.jpg\ /tmp/download
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The file \f[C]test.jpg\f[] will be placed inside \f[C]/tmp/download\f[].
|
|
.PP
|
|
This is equivalent to specifying
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-files\-from\ /tmp/files\ remote:\ /tmp/download
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Where \f[C]/tmp/files\f[] contains the single line
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
test.jpg
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
It is recommended to use \f[C]copy\f[] when copying individual files,
|
|
not \f[C]sync\f[].
|
|
They have pretty much the same effect but \f[C]copy\f[] will use a lot
|
|
less memory.
|
|
.SS Syntax of remote paths
|
|
.PP
|
|
The syntax of the paths passed to the rclone command are as follows.
|
|
.SS /path/to/dir
|
|
.PP
|
|
This refers to the local file system.
|
|
.PP
|
|
On Windows only \f[C]\\\f[] may be used instead of \f[C]/\f[] in local
|
|
paths \f[B]only\f[], non local paths must use \f[C]/\f[].
|
|
.PP
|
|
These paths needn't start with a leading \f[C]/\f[] \- if they don't
|
|
then they will be relative to the current directory.
|
|
.SS remote:path/to/dir
|
|
.PP
|
|
This refers to a directory \f[C]path/to/dir\f[] on \f[C]remote:\f[] as
|
|
defined in the config file (configured with \f[C]rclone\ config\f[]).
|
|
.SS remote:/path/to/dir
|
|
.PP
|
|
On most backends this is refers to the same directory as
|
|
\f[C]remote:path/to/dir\f[] and that format should be preferred.
|
|
On a very small number of remotes (FTP, SFTP, Dropbox for business) this
|
|
will refer to a different directory.
|
|
On these, paths without a leading \f[C]/\f[] will refer to your
|
|
\[lq]home\[rq] directory and paths with a leading \f[C]/\f[] will refer
|
|
to the root.
|
|
.SS :backend:path/to/dir
|
|
.PP
|
|
This is an advanced form for creating remotes on the fly.
|
|
\f[C]backend\f[] should be the name or prefix of a backend (the
|
|
\f[C]type\f[] in the config file) and all the configuration for the
|
|
backend should be provided on the command line (or in environment
|
|
variables).
|
|
.PP
|
|
Here are some examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ \-\-http\-url\ https://pub.rclone.org\ :http:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To list all the directories in the root of
|
|
\f[C]https://pub.rclone.org/\f[].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsf\ \-\-http\-url\ https://example.com\ :http:path/to/dir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To list files and directories in
|
|
\f[C]https://example.com/path/to/dir/\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-http\-url\ https://example.com\ :http:path/to/dir\ /tmp/dir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy files and directories in
|
|
\f[C]https://example.com/path/to/dir\f[] to \f[C]/tmp/dir\f[].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-sftp\-host\ example.com\ :sftp:path/to/dir\ /tmp/dir
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy files and directories from \f[C]example.com\f[] in the relative
|
|
directory \f[C]path/to/dir\f[] to \f[C]/tmp/dir\f[] using sftp.
|
|
.SS Quoting and the shell
|
|
.PP
|
|
When you are typing commands to your computer you are using something
|
|
called the command line shell.
|
|
This interprets various characters in an OS specific way.
|
|
.PP
|
|
Here are some gotchas which may help users unfamiliar with the shell
|
|
rules
|
|
.SS Linux / OSX
|
|
.PP
|
|
If your names have spaces or shell metacharacters (eg \f[C]*\f[],
|
|
\f[C]?\f[], \f[C]$\f[], \f[C]\[aq]\f[], \f[C]"\f[] etc) then you must
|
|
quote them.
|
|
Use single quotes \f[C]\[aq]\f[] by default.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \[aq]Important\ files?\[aq]\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you want to send a \f[C]\[aq]\f[] you will need to use \f[C]"\f[], eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ "O\[aq]Reilly\ Reviews"\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The rules for quoting metacharacters are complicated and if you want the
|
|
full details you'll have to consult the manual page for your shell.
|
|
.SS Windows
|
|
.PP
|
|
If your names have spaces in you need to put them in \f[C]"\f[], eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ "E:\\folder\ name\\folder\ name\\folder\ name"\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you are using the root directory on its own then don't quote it (see
|
|
#464 (https://github.com/rclone/rclone/issues/464) for why), eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ E:\\\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Copying files or directories with \f[C]:\f[] in the names
|
|
.PP
|
|
rclone uses \f[C]:\f[] to mark a remote name.
|
|
This is, however, a valid filename component in non\-Windows OSes.
|
|
The remote name parser will only search for a \f[C]:\f[] up to the first
|
|
\f[C]/\f[] so if you need to act on a file or directory like this then
|
|
use the full path starting with a \f[C]/\f[], or use \f[C]\&./\f[] as a
|
|
current directory prefix.
|
|
.PP
|
|
So to sync a directory called \f[C]sync:me\f[] to a remote called
|
|
\f[C]remote:\f[] use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ ./sync:me\ remote:path
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /full/path/to/sync:me\ remote:path
|
|
\f[]
|
|
.fi
|
|
.SS Server Side Copy
|
|
.PP
|
|
Most remotes (but not all \- see the
|
|
overview (/overview/#optional-features)) support server side copy.
|
|
.PP
|
|
This means if you want to copy one folder to another then rclone won't
|
|
download all the files and re\-upload them; it will instruct the server
|
|
to copy them in place.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ s3:oldbucket\ s3:newbucket
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Will copy the contents of \f[C]oldbucket\f[] to \f[C]newbucket\f[]
|
|
without downloading and re\-uploading.
|
|
.PP
|
|
Remotes which don't support server side copy \f[B]will\f[] download and
|
|
re\-upload in this case.
|
|
.PP
|
|
Server side copies are used with \f[C]sync\f[] and \f[C]copy\f[] and
|
|
will be identified in the log when using the \f[C]\-v\f[] flag.
|
|
The \f[C]move\f[] command may also use them if remote doesn't support
|
|
server side move directly.
|
|
This is done by issuing a server side copy then a delete which is much
|
|
quicker than a download and re\-upload.
|
|
.PP
|
|
Server side copies will only be attempted if the remote names are the
|
|
same.
|
|
.PP
|
|
This can be used when scripting to make aged backups efficiently, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ remote:current\-backup\ remote:previous\-backup
|
|
rclone\ sync\ /path/to/files\ remote:current\-backup
|
|
\f[]
|
|
.fi
|
|
.SS Options
|
|
.PP
|
|
Rclone has a number of options to control its behaviour.
|
|
.PP
|
|
Options that take parameters can have the values passed in two ways,
|
|
\f[C]\-\-option=value\f[] or \f[C]\-\-option\ value\f[].
|
|
However boolean (true/false) options behave slightly differently to the
|
|
other options in that \f[C]\-\-boolean\f[] sets the option to
|
|
\f[C]true\f[] and the absence of the flag sets it to \f[C]false\f[].
|
|
It is also possible to specify \f[C]\-\-boolean=false\f[] or
|
|
\f[C]\-\-boolean=true\f[].
|
|
Note that \f[C]\-\-boolean\ false\f[] is not valid \- this is parsed as
|
|
\f[C]\-\-boolean\f[] and the \f[C]false\f[] is parsed as an extra
|
|
command line argument for rclone.
|
|
.PP
|
|
Options which use TIME use the go time parser.
|
|
A duration string is a possibly signed sequence of decimal numbers, each
|
|
with optional fraction and a unit suffix, such as \[lq]300ms\[rq],
|
|
\[lq]\-1.5h\[rq] or \[lq]2h45m\[rq].
|
|
Valid time units are \[lq]ns\[rq], \[lq]us\[rq] (or \[lq]µs\[rq]),
|
|
\[lq]ms\[rq], \[lq]s\[rq], \[lq]m\[rq], \[lq]h\[rq].
|
|
.PP
|
|
Options which use SIZE use kByte by default.
|
|
However, a suffix of \f[C]b\f[] for bytes, \f[C]k\f[] for kBytes,
|
|
\f[C]M\f[] for MBytes, \f[C]G\f[] for GBytes, \f[C]T\f[] for TBytes and
|
|
\f[C]P\f[] for PBytes may be used.
|
|
These are the binary units, eg 1, 2**10, 2**20, 2**30 respectively.
|
|
.SS \[en]backup\-dir=DIR
|
|
.PP
|
|
When using \f[C]sync\f[], \f[C]copy\f[] or \f[C]move\f[] any files which
|
|
would have been overwritten or deleted are moved in their original
|
|
hierarchy into this directory.
|
|
.PP
|
|
If \f[C]\-\-suffix\f[] is set, then the moved files will have the suffix
|
|
added to them.
|
|
If there is a file with the same path (after the suffix has been added)
|
|
in DIR, then it will be overwritten.
|
|
.PP
|
|
The remote in use must support server side move or copy and you must use
|
|
the same remote as the destination of the sync.
|
|
The backup directory must not overlap the destination directory.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /path/to/local\ remote:current\ \-\-backup\-dir\ remote:old
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
will sync \f[C]/path/to/local\f[] to \f[C]remote:current\f[], but for
|
|
any files which would have been updated or deleted will be stored in
|
|
\f[C]remote:old\f[].
|
|
.PP
|
|
If running rclone from a script you might want to use today's date as
|
|
the directory name passed to \f[C]\-\-backup\-dir\f[] to store the old
|
|
files, or you might want to pass \f[C]\-\-suffix\f[] with today's date.
|
|
.PP
|
|
See \f[C]\-\-compare\-dest\f[] and \f[C]\-\-copy\-dest\f[].
|
|
.SS \[en]bind string
|
|
.PP
|
|
Local address to bind to for outgoing connections.
|
|
This can be an IPv4 address (1.2.3.4), an IPv6 address (1234::789A) or
|
|
host name.
|
|
If the host name doesn't resolve or resolves to more than one IP address
|
|
it will give an error.
|
|
.SS \[en]bwlimit=BANDWIDTH_SPEC
|
|
.PP
|
|
This option controls the bandwidth limit.
|
|
Limits can be specified in two ways: As a single limit, or as a
|
|
timetable.
|
|
.PP
|
|
Single limits last for the duration of the session.
|
|
To use a single limit, specify the desired bandwidth in kBytes/s, or use
|
|
a suffix b|k|M|G.
|
|
The default is \f[C]0\f[] which means to not limit bandwidth.
|
|
.PP
|
|
For example, to limit bandwidth usage to 10 MBytes/s use
|
|
\f[C]\-\-bwlimit\ 10M\f[]
|
|
.PP
|
|
It is also possible to specify a \[lq]timetable\[rq] of limits, which
|
|
will cause certain limits to be applied at certain times.
|
|
To specify a timetable, format your entries as
|
|
\[lq]WEEKDAY\-HH:MM,BANDWIDTH WEEKDAY\-HH:MM,BANDWIDTH\&...\[rq] where:
|
|
WEEKDAY is optional element.
|
|
It could be written as whole world or only using 3 first characters.
|
|
HH:MM is an hour from 00:00 to 23:59.
|
|
.PP
|
|
An example of a typical timetable to avoid link saturation during
|
|
daytime working hours could be:
|
|
.PP
|
|
\f[C]\-\-bwlimit\ "08:00,512\ 12:00,10M\ 13:00,512\ 18:00,30M\ 23:00,off"\f[]
|
|
.PP
|
|
In this example, the transfer bandwidth will be every day set to
|
|
512kBytes/sec at 8am.
|
|
At noon, it will raise to 10Mbytes/s, and drop back to 512kBytes/sec at
|
|
1pm.
|
|
At 6pm, the bandwidth limit will be set to 30MBytes/s, and at 11pm it
|
|
will be completely disabled (full speed).
|
|
Anything between 11pm and 8am will remain unlimited.
|
|
.PP
|
|
An example of timetable with WEEKDAY could be:
|
|
.PP
|
|
\f[C]\-\-bwlimit\ "Mon\-00:00,512\ Fri\-23:59,10M\ Sat\-10:00,1M\ Sun\-20:00,off"\f[]
|
|
.PP
|
|
It mean that, the transfer bandwidth will be set to 512kBytes/sec on
|
|
Monday.
|
|
It will raise to 10Mbytes/s before the end of Friday.
|
|
At 10:00 on Sunday it will be set to 1Mbyte/s.
|
|
From 20:00 at Sunday will be unlimited.
|
|
.PP
|
|
Timeslots without weekday are extended to whole week.
|
|
So this one example:
|
|
.PP
|
|
\f[C]\-\-bwlimit\ "Mon\-00:00,512\ 12:00,1M\ Sun\-20:00,off"\f[]
|
|
.PP
|
|
Is equal to this:
|
|
.PP
|
|
\f[C]\-\-bwlimit\ "Mon\-00:00,512Mon\-12:00,1M\ Tue\-12:00,1M\ Wed\-12:00,1M\ Thu\-12:00,1M\ Fri\-12:00,1M\ Sat\-12:00,1M\ Sun\-12:00,1M\ Sun\-20:00,off"\f[]
|
|
.PP
|
|
Bandwidth limits only apply to the data transfer.
|
|
They don't apply to the bandwidth of the directory listings etc.
|
|
.PP
|
|
Note that the units are Bytes/s, not Bits/s.
|
|
Typically connections are measured in Bits/s \- to convert divide by 8.
|
|
For example, let's say you have a 10 Mbit/s connection and you wish
|
|
rclone to use half of it \- 5 Mbit/s.
|
|
This is 5/8 = 0.625MByte/s so you would use a
|
|
\f[C]\-\-bwlimit\ 0.625M\f[] parameter for rclone.
|
|
.PP
|
|
On Unix systems (Linux, MacOS, \&...) the bandwidth limiter can be
|
|
toggled by sending a \f[C]SIGUSR2\f[] signal to rclone.
|
|
This allows to remove the limitations of a long running rclone transfer
|
|
and to restore it back to the value specified with \f[C]\-\-bwlimit\f[]
|
|
quickly when needed.
|
|
Assuming there is only one rclone instance running, you can toggle the
|
|
limiter like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill\ \-SIGUSR2\ $(pidof\ rclone)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
change the bwlimit dynamically:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ core/bwlimit\ rate=1M
|
|
\f[]
|
|
.fi
|
|
.SS \[en]buffer\-size=SIZE
|
|
.PP
|
|
Use this sized buffer to speed up file transfers.
|
|
Each \f[C]\-\-transfer\f[] will use this much memory for buffering.
|
|
.PP
|
|
When using \f[C]mount\f[] or \f[C]cmount\f[] each open file descriptor
|
|
will use this much memory for buffering.
|
|
See the mount (/commands/rclone_mount/#file-buffering) documentation for
|
|
more details.
|
|
.PP
|
|
Set to 0 to disable the buffering for the minimum memory usage.
|
|
.PP
|
|
Note that the memory allocation of the buffers is influenced by the
|
|
\[en]use\-mmap flag.
|
|
.SS \[en]checkers=N
|
|
.PP
|
|
The number of checkers to run in parallel.
|
|
Checkers do the equality checking of files during a sync.
|
|
For some storage systems (eg S3, Swift, Dropbox) this can take a
|
|
significant amount of time so they are run in parallel.
|
|
.PP
|
|
The default is to run 8 checkers in parallel.
|
|
.SS \-c, \[en]checksum
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check the file hash and size to
|
|
determine if files are equal.
|
|
.PP
|
|
This is useful when the remote doesn't support setting modified time and
|
|
a more accurate sync is desired than just checking the file size.
|
|
.PP
|
|
This is very useful when transferring between remotes which store the
|
|
same hash type on the object, eg Drive and Swift.
|
|
For details of which remotes support which hash type see the table in
|
|
the overview section (https://rclone.org/overview/).
|
|
.PP
|
|
Eg \f[C]rclone\ \-\-checksum\ sync\ s3:/bucket\ swift:/bucket\f[] would
|
|
run much quicker than without the \f[C]\-\-checksum\f[] flag.
|
|
.PP
|
|
When using this flag, rclone won't update mtimes of remote files if they
|
|
are incorrect as it would normally.
|
|
.SS \[en]compare\-dest=DIR
|
|
.PP
|
|
When using \f[C]sync\f[], \f[C]copy\f[] or \f[C]move\f[] DIR is checked
|
|
in addition to the destination for files.
|
|
If a file identical to the source is found that file is NOT copied from
|
|
source.
|
|
This is useful to copy just files that have changed since the last
|
|
backup.
|
|
.PP
|
|
You must use the same remote as the destination of the sync.
|
|
The compare directory must not overlap the destination directory.
|
|
.PP
|
|
See \f[C]\-\-copy\-dest\f[] and \f[C]\-\-backup\-dir\f[].
|
|
.SS \[en]config=CONFIG_FILE
|
|
.PP
|
|
Specify the location of the rclone config file.
|
|
.PP
|
|
Normally the config file is in your home directory as a file called
|
|
\f[C]\&.config/rclone/rclone.conf\f[] (or \f[C]\&.rclone.conf\f[] if
|
|
created with an older version).
|
|
If \f[C]$XDG_CONFIG_HOME\f[] is set it will be at
|
|
\f[C]$XDG_CONFIG_HOME/rclone/rclone.conf\f[].
|
|
.PP
|
|
If there is a file \f[C]rclone.conf\f[] in the same directory as the
|
|
rclone executable it will be preferred.
|
|
This file must be created manually for Rclone to use it, it will never
|
|
be created automatically.
|
|
.PP
|
|
If you run \f[C]rclone\ config\ file\f[] you will see where the default
|
|
location is for you.
|
|
.PP
|
|
Use this flag to override the config location, eg
|
|
\f[C]rclone\ \-\-config=".myconfig"\ .config\f[].
|
|
.SS \[en]contimeout=TIME
|
|
.PP
|
|
Set the connection timeout.
|
|
This should be in go time format which looks like \f[C]5s\f[] for 5
|
|
seconds, \f[C]10m\f[] for 10 minutes, or \f[C]3h30m\f[].
|
|
.PP
|
|
The connection timeout is the amount of time rclone will wait for a
|
|
connection to go through to a remote object storage system.
|
|
It is \f[C]1m\f[] by default.
|
|
.SS \[en]copy\-dest=DIR
|
|
.PP
|
|
When using \f[C]sync\f[], \f[C]copy\f[] or \f[C]move\f[] DIR is checked
|
|
in addition to the destination for files.
|
|
If a file identical to the source is found that file is server side
|
|
copied from DIR to the destination.
|
|
This is useful for incremental backup.
|
|
.PP
|
|
The remote in use must support server side copy and you must use the
|
|
same remote as the destination of the sync.
|
|
The compare directory must not overlap the destination directory.
|
|
.PP
|
|
See \f[C]\-\-compare\-dest\f[] and \f[C]\-\-backup\-dir\f[].
|
|
.SS \[en]dedupe\-mode MODE
|
|
.PP
|
|
Mode to run dedupe command in.
|
|
One of \f[C]interactive\f[], \f[C]skip\f[], \f[C]first\f[],
|
|
\f[C]newest\f[], \f[C]oldest\f[], \f[C]rename\f[].
|
|
The default is \f[C]interactive\f[].
|
|
See the dedupe command for more information as to what these options
|
|
mean.
|
|
.SS \[en]disable FEATURE,FEATURE,\&...
|
|
.PP
|
|
This disables a comma separated list of optional features.
|
|
For example to disable server side move and server side copy use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-disable\ move,copy
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The features can be put in in any case.
|
|
.PP
|
|
To see a list of which features can be disabled use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-disable\ help
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the overview features (/overview/#features) and optional
|
|
features (/overview/#optional-features) to get an idea of which feature
|
|
does what.
|
|
.PP
|
|
This flag can be useful for debugging and in exceptional circumstances
|
|
(eg Google Drive limiting the total volume of Server Side Copies to
|
|
100GB/day).
|
|
.SS \-n, \[en]dry\-run
|
|
.PP
|
|
Do a trial run with no permanent changes.
|
|
Use this to see what rclone would do without actually doing it.
|
|
Useful when setting up the \f[C]sync\f[] command which deletes files in
|
|
the destination.
|
|
.SS \[en]ignore\-case\-sync
|
|
.PP
|
|
Using this option will cause rclone to ignore the case of the files when
|
|
synchronizing so files will not be copied/synced when the existing
|
|
filenames are the same, even if the casing is different.
|
|
.SS \[en]ignore\-checksum
|
|
.PP
|
|
Normally rclone will check that the checksums of transferred files
|
|
match, and give an error \[lq]corrupted on transfer\[rq] if they don't.
|
|
.PP
|
|
You can use this option to skip that check.
|
|
You should only use it if you have had the \[lq]corrupted on
|
|
transfer\[rq] error message and you are sure you might want to transfer
|
|
potentially corrupted data.
|
|
.SS \[en]ignore\-existing
|
|
.PP
|
|
Using this option will make rclone unconditionally skip all files that
|
|
exist on the destination, no matter the content of these files.
|
|
.PP
|
|
While this isn't a generally recommended option, it can be useful in
|
|
cases where your files change due to encryption.
|
|
However, it cannot correct partial transfers in case a transfer was
|
|
interrupted.
|
|
.SS \[en]ignore\-size
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check only the modification time.
|
|
If \f[C]\-\-checksum\f[] is set then it only checks the checksum.
|
|
.PP
|
|
It will also cause rclone to skip verifying the sizes are the same after
|
|
transfer.
|
|
.PP
|
|
This can be useful for transferring files to and from OneDrive which
|
|
occasionally misreports the size of image files (see
|
|
#399 (https://github.com/rclone/rclone/issues/399) for more info).
|
|
.SS \-I, \[en]ignore\-times
|
|
.PP
|
|
Using this option will cause rclone to unconditionally upload all files
|
|
regardless of the state of files on the destination.
|
|
.PP
|
|
Normally rclone would skip any files that have the same modification
|
|
time and are the same size (or have the same checksum if using
|
|
\f[C]\-\-checksum\f[]).
|
|
.SS \[en]immutable
|
|
.PP
|
|
Treat source and destination files as immutable and disallow
|
|
modification.
|
|
.PP
|
|
With this option set, files will be created and deleted as requested,
|
|
but existing files will never be updated.
|
|
If an existing file does not match between the source and destination,
|
|
rclone will give the error
|
|
\f[C]Source\ and\ destination\ exist\ but\ do\ not\ match:\ immutable\ file\ modified\f[].
|
|
.PP
|
|
Note that only commands which transfer files (e.g.
|
|
\f[C]sync\f[], \f[C]copy\f[], \f[C]move\f[]) are affected by this
|
|
behavior, and only modification is disallowed.
|
|
Files may still be deleted explicitly (e.g.
|
|
\f[C]delete\f[], \f[C]purge\f[]) or implicitly (e.g.
|
|
\f[C]sync\f[], \f[C]move\f[]).
|
|
Use \f[C]copy\ \-\-immutable\f[] if it is desired to avoid deletion as
|
|
well as modification.
|
|
.PP
|
|
This can be useful as an additional layer of protection for immutable or
|
|
append\-only data sets (notably backup archives), where modification
|
|
implies corruption and should not be propagated.
|
|
.SS \[en]leave\-root
|
|
.PP
|
|
During rmdirs it will not remove root directory, even if it's empty.
|
|
.SS \[en]log\-file=FILE
|
|
.PP
|
|
Log all of rclone's output to FILE.
|
|
This is not active by default.
|
|
This can be useful for tracking down problems with syncs in combination
|
|
with the \f[C]\-v\f[] flag.
|
|
See the Logging section for more info.
|
|
.PP
|
|
Note that if you are using the \f[C]logrotate\f[] program to manage
|
|
rclone's logs, then you should use the \f[C]copytruncate\f[] option as
|
|
rclone doesn't have a signal to rotate logs.
|
|
.SS \[en]log\-format LIST
|
|
.PP
|
|
Comma separated list of log format options.
|
|
\f[C]date\f[], \f[C]time\f[], \f[C]microseconds\f[], \f[C]longfile\f[],
|
|
\f[C]shortfile\f[], \f[C]UTC\f[].
|
|
The default is \[lq]\f[C]date\f[],\f[C]time\f[]\[rq].
|
|
.SS \[en]log\-level LEVEL
|
|
.PP
|
|
This sets the log level for rclone.
|
|
The default log level is \f[C]NOTICE\f[].
|
|
.PP
|
|
\f[C]DEBUG\f[] is equivalent to \f[C]\-vv\f[].
|
|
It outputs lots of debug info \- useful for bug reports and really
|
|
finding out what rclone is doing.
|
|
.PP
|
|
\f[C]INFO\f[] is equivalent to \f[C]\-v\f[].
|
|
It outputs information about each transfer and prints stats once a
|
|
minute by default.
|
|
.PP
|
|
\f[C]NOTICE\f[] is the default log level if no logging flags are
|
|
supplied.
|
|
It outputs very little when things are working normally.
|
|
It outputs warnings and significant events.
|
|
.PP
|
|
\f[C]ERROR\f[] is equivalent to \f[C]\-q\f[].
|
|
It only outputs error messages.
|
|
.SS \[en]use\-json\-log
|
|
.PP
|
|
This switches the log format to JSON for rclone.
|
|
The fields of json log are level, msg, source, time.
|
|
.SS \[en]low\-level\-retries NUMBER
|
|
.PP
|
|
This controls the number of low level retries rclone does.
|
|
.PP
|
|
A low level retry is used to retry a failing operation \- typically one
|
|
HTTP request.
|
|
This might be uploading a chunk of a big file for example.
|
|
You will see low level retries in the log with the \f[C]\-v\f[] flag.
|
|
.PP
|
|
This shouldn't need to be changed from the default in normal operations.
|
|
However, if you get a lot of low level retries you may wish to reduce
|
|
the value so rclone moves on to a high level retry (see the
|
|
\f[C]\-\-retries\f[] flag) quicker.
|
|
.PP
|
|
Disable low level retries with \f[C]\-\-low\-level\-retries\ 1\f[].
|
|
.SS \[en]max\-backlog=N
|
|
.PP
|
|
This is the maximum allowable backlog of files in a sync/copy/move
|
|
queued for being checked or transferred.
|
|
.PP
|
|
This can be set arbitrarily large.
|
|
It will only use memory when the queue is in use.
|
|
Note that it will use in the order of N kB of memory when the backlog is
|
|
in use.
|
|
.PP
|
|
Setting this large allows rclone to calculate how many files are pending
|
|
more accurately and give a more accurate estimated finish time.
|
|
.PP
|
|
Setting this small will make rclone more synchronous to the listings of
|
|
the remote which may be desirable.
|
|
.SS \[en]max\-delete=N
|
|
.PP
|
|
This tells rclone not to delete more than N files.
|
|
If that limit is exceeded then a fatal error will be generated and
|
|
rclone will stop the operation in progress.
|
|
.SS \[en]max\-depth=N
|
|
.PP
|
|
This modifies the recursion depth for all the commands except purge.
|
|
.PP
|
|
So if you do \f[C]rclone\ \-\-max\-depth\ 1\ ls\ remote:path\f[] you
|
|
will see only the files in the top level directory.
|
|
Using \f[C]\-\-max\-depth\ 2\f[] means you will see all the files in
|
|
first two directory levels and so on.
|
|
.PP
|
|
For historical reasons the \f[C]lsd\f[] command defaults to using a
|
|
\f[C]\-\-max\-depth\f[] of 1 \- you can override this with the command
|
|
line flag.
|
|
.PP
|
|
You can use this command to disable recursion (with
|
|
\f[C]\-\-max\-depth\ 1\f[]).
|
|
.PP
|
|
Note that if you use this with \f[C]sync\f[] and
|
|
\f[C]\-\-delete\-excluded\f[] the files not recursed through are
|
|
considered excluded and will be deleted on the destination.
|
|
Test first with \f[C]\-\-dry\-run\f[] if you are not sure what will
|
|
happen.
|
|
.SS \[en]max\-transfer=SIZE
|
|
.PP
|
|
Rclone will stop transferring when it has reached the size specified.
|
|
Defaults to off.
|
|
.PP
|
|
When the limit is reached all transfers will stop immediately.
|
|
.PP
|
|
Rclone will exit with exit code 8 if the transfer limit is reached.
|
|
.SS \[en]modify\-window=TIME
|
|
.PP
|
|
When checking whether a file has been modified, this is the maximum
|
|
allowed time difference that a file can have and still be considered
|
|
equivalent.
|
|
.PP
|
|
The default is \f[C]1ns\f[] unless this is overridden by a remote.
|
|
For example OS X only stores modification times to the nearest second so
|
|
if you are reading and writing to an OS X filing system this will be
|
|
\f[C]1s\f[] by default.
|
|
.PP
|
|
This command line flag allows you to override that computed default.
|
|
.SS \[en]multi\-thread\-cutoff=SIZE
|
|
.PP
|
|
When downloading files to the local backend above this size, rclone will
|
|
use multiple threads to download the file.
|
|
(default 250M)
|
|
.PP
|
|
Rclone preallocates the file (using
|
|
\f[C]fallocate(FALLOC_FL_KEEP_SIZE)\f[] on unix or
|
|
\f[C]NTSetInformationFile\f[] on Windows both of which takes no time)
|
|
then each thread writes directly into the file at the correct place.
|
|
This means that rclone won't create fragmented or sparse files and there
|
|
won't be any assembly time at the end of the transfer.
|
|
.PP
|
|
The number of threads used to dowload is controlled by
|
|
\f[C]\-\-multi\-thread\-streams\f[].
|
|
.PP
|
|
Use \f[C]\-vv\f[] if you wish to see info about the threads.
|
|
.PP
|
|
This will work with the \f[C]sync\f[]/\f[C]copy\f[]/\f[C]move\f[]
|
|
commands and friends \f[C]copyto\f[]/\f[C]moveto\f[].
|
|
Multi thread downloads will be used with \f[C]rclone\ mount\f[] and
|
|
\f[C]rclone\ serve\f[] if \f[C]\-\-vfs\-cache\-mode\f[] is set to
|
|
\f[C]writes\f[] or above.
|
|
.PP
|
|
\f[B]NB\f[] that this \f[B]only\f[] works for a local destination but
|
|
will work with any source.
|
|
.PP
|
|
\f[B]NB\f[] that multi thread copies are disabled for local to local
|
|
copies as they are faster without unless
|
|
\f[C]\-\-multi\-thread\-streams\f[] is set explicitly.
|
|
.SS \[en]multi\-thread\-streams=N
|
|
.PP
|
|
When using multi thread downloads (see above
|
|
\f[C]\-\-multi\-thread\-cutoff\f[]) this sets the maximum number of
|
|
streams to use.
|
|
Set to \f[C]0\f[] to disable multi thread downloads.
|
|
(Default 4)
|
|
.PP
|
|
Exactly how many streams rclone uses for the download depends on the
|
|
size of the file.
|
|
To calculate the number of download streams Rclone divides the size of
|
|
the file by the \f[C]\-\-multi\-thread\-cutoff\f[] and rounds up, up to
|
|
the maximum set with \f[C]\-\-multi\-thread\-streams\f[].
|
|
.PP
|
|
So if \f[C]\-\-multi\-thread\-cutoff\ 250MB\f[] and
|
|
\f[C]\-\-multi\-thread\-streams\ 4\f[] are in effect (the defaults):
|
|
.IP \[bu] 2
|
|
0MB.250MB files will be downloaded with 1 stream
|
|
.IP \[bu] 2
|
|
250MB..500MB files will be downloaded with 2 streams
|
|
.IP \[bu] 2
|
|
500MB..750MB files will be downloaded with 3 streams
|
|
.IP \[bu] 2
|
|
750MB+ files will be downloaded with 4 streams
|
|
.SS \[en]no\-gzip\-encoding
|
|
.PP
|
|
Don't set \f[C]Accept\-Encoding:\ gzip\f[].
|
|
This means that rclone won't ask the server for compressed files
|
|
automatically.
|
|
Useful if you've set the server to return files with
|
|
\f[C]Content\-Encoding:\ gzip\f[] but you uploaded compressed files.
|
|
.PP
|
|
There is no need to set this in normal operation, and doing so will
|
|
decrease the network transfer efficiency of rclone.
|
|
.SS \[en]no\-traverse
|
|
.PP
|
|
The \f[C]\-\-no\-traverse\f[] flag controls whether the destination file
|
|
system is traversed when using the \f[C]copy\f[] or \f[C]move\f[]
|
|
commands.
|
|
\f[C]\-\-no\-traverse\f[] is not compatible with \f[C]sync\f[] and will
|
|
be ignored if you supply it with \f[C]sync\f[].
|
|
.PP
|
|
If you are only copying a small number of files (or are filtering most
|
|
of the files) and/or have a large number of files on the destination
|
|
then \f[C]\-\-no\-traverse\f[] will stop rclone listing the destination
|
|
and save time.
|
|
.PP
|
|
However, if you are copying a large number of files, especially if you
|
|
are doing a copy where lots of the files under consideration haven't
|
|
changed and won't need copying then you shouldn't use
|
|
\f[C]\-\-no\-traverse\f[].
|
|
.PP
|
|
See rclone copy (https://rclone.org/commands/rclone_copy/) for an
|
|
example of how to use it.
|
|
.SS \[en]no\-update\-modtime
|
|
.PP
|
|
When using this flag, rclone won't update modification times of remote
|
|
files if they are incorrect as it would normally.
|
|
.PP
|
|
This can be used if the remote is being synced with another tool also
|
|
(eg the Google Drive client).
|
|
.SS \-P, \[en]progress
|
|
.PP
|
|
This flag makes rclone update the stats in a static block in the
|
|
terminal providing a realtime overview of the transfer.
|
|
.PP
|
|
Any log messages will scroll above the static block.
|
|
Log messages will push the static block down to the bottom of the
|
|
terminal where it will stay.
|
|
.PP
|
|
Normally this is updated every 500mS but this period can be overridden
|
|
with the \f[C]\-\-stats\f[] flag.
|
|
.PP
|
|
This can be used with the \f[C]\-\-stats\-one\-line\f[] flag for a
|
|
simpler display.
|
|
.PP
|
|
Note: On Windows untilthis
|
|
bug (https://github.com/Azure/go-ansiterm/issues/26) is fixed all
|
|
non\-ASCII characters will be replaced with \f[C]\&.\f[] when
|
|
\f[C]\-\-progress\f[] is in use.
|
|
.SS \-q, \[en]quiet
|
|
.PP
|
|
Normally rclone outputs stats and a completion message.
|
|
If you set this flag it will make as little output as possible.
|
|
.SS \[en]retries int
|
|
.PP
|
|
Retry the entire sync if it fails this many times it fails (default 3).
|
|
.PP
|
|
Some remotes can be unreliable and a few retries help pick up the files
|
|
which didn't get transferred because of errors.
|
|
.PP
|
|
Disable retries with \f[C]\-\-retries\ 1\f[].
|
|
.SS \[en]retries\-sleep=TIME
|
|
.PP
|
|
This sets the interval between each retry specified by
|
|
\f[C]\-\-retries\f[]
|
|
.PP
|
|
The default is 0.
|
|
Use 0 to disable.
|
|
.SS \[en]size\-only
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check only the size.
|
|
.PP
|
|
This can be useful transferring files from Dropbox which have been
|
|
modified by the desktop sync client which doesn't set checksums of
|
|
modification times in the same way as rclone.
|
|
.SS \[en]stats=TIME
|
|
.PP
|
|
Commands which transfer data (\f[C]sync\f[], \f[C]copy\f[],
|
|
\f[C]copyto\f[], \f[C]move\f[], \f[C]moveto\f[]) will print data
|
|
transfer stats at regular intervals to show their progress.
|
|
.PP
|
|
This sets the interval.
|
|
.PP
|
|
The default is \f[C]1m\f[].
|
|
Use 0 to disable.
|
|
.PP
|
|
If you set the stats interval then all commands can show stats.
|
|
This can be useful when running other commands, \f[C]check\f[] or
|
|
\f[C]mount\f[] for example.
|
|
.PP
|
|
Stats are logged at \f[C]INFO\f[] level by default which means they
|
|
won't show at default log level \f[C]NOTICE\f[].
|
|
Use \f[C]\-\-stats\-log\-level\ NOTICE\f[] or \f[C]\-v\f[] to make them
|
|
show.
|
|
See the Logging section for more info on log levels.
|
|
.PP
|
|
Note that on macOS you can send a SIGINFO (which is normally ctrl\-T in
|
|
the terminal) to make the stats print immediately.
|
|
.SS \[en]stats\-file\-name\-length integer
|
|
.PP
|
|
By default, the \f[C]\-\-stats\f[] output will truncate file names and
|
|
paths longer than 40 characters.
|
|
This is equivalent to providing
|
|
\f[C]\-\-stats\-file\-name\-length\ 40\f[].
|
|
Use \f[C]\-\-stats\-file\-name\-length\ 0\f[] to disable any truncation
|
|
of file names printed by stats.
|
|
.SS \[en]stats\-log\-level string
|
|
.PP
|
|
Log level to show \f[C]\-\-stats\f[] output at.
|
|
This can be \f[C]DEBUG\f[], \f[C]INFO\f[], \f[C]NOTICE\f[], or
|
|
\f[C]ERROR\f[].
|
|
The default is \f[C]INFO\f[].
|
|
This means at the default level of logging which is \f[C]NOTICE\f[] the
|
|
stats won't show \- if you want them to then use
|
|
\f[C]\-\-stats\-log\-level\ NOTICE\f[].
|
|
See the Logging section for more info on log levels.
|
|
.SS \[en]stats\-one\-line
|
|
.PP
|
|
When this is specified, rclone condenses the stats into a single line
|
|
showing the most important stats only.
|
|
.SS \[en]stats\-one\-line\-date
|
|
.PP
|
|
When this is specified, rclone enables the single\-line stats and
|
|
prepends the display with a date string.
|
|
The default is \f[C]2006/01/02\ 15:04:05\ \-\f[]
|
|
.SS \[en]stats\-one\-line\-date\-format
|
|
.PP
|
|
When this is specified, rclone enables the single\-line stats and
|
|
prepends the display with a user\-supplied date string.
|
|
The date string MUST be enclosed in quotes.
|
|
Follow golang specs (https://golang.org/pkg/time/#Time.Format) for date
|
|
formatting syntax.
|
|
.SS \[en]stats\-unit=bits|bytes
|
|
.PP
|
|
By default, data transfer rates will be printed in bytes/second.
|
|
.PP
|
|
This option allows the data rate to be printed in bits/second.
|
|
.PP
|
|
Data transfer volume will still be reported in bytes.
|
|
.PP
|
|
The rate is reported as a binary unit, not SI unit.
|
|
So 1 Mbit/s equals 1,048,576 bits/s and not 1,000,000 bits/s.
|
|
.PP
|
|
The default is \f[C]bytes\f[].
|
|
.SS \[en]suffix=SUFFIX
|
|
.PP
|
|
When using \f[C]sync\f[], \f[C]copy\f[] or \f[C]move\f[] any files which
|
|
would have been overwritten or deleted will have the suffix added to
|
|
them.
|
|
If there is a file with the same path (after the suffix has been added),
|
|
then it will be overwritten.
|
|
.PP
|
|
The remote in use must support server side move or copy and you must use
|
|
the same remote as the destination of the sync.
|
|
.PP
|
|
This is for use with files to add the suffix in the current directory or
|
|
with \f[C]\-\-backup\-dir\f[].
|
|
See \f[C]\-\-backup\-dir\f[] for more info.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /path/to/local/file\ remote:current\ \-\-suffix\ .bak
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
will sync \f[C]/path/to/local\f[] to \f[C]remote:current\f[], but for
|
|
any files which would have been updated or deleted have .bak added.
|
|
.SS \[en]suffix\-keep\-extension
|
|
.PP
|
|
When using \f[C]\-\-suffix\f[], setting this causes rclone put the
|
|
SUFFIX before the extension of the files that it backs up rather than
|
|
after.
|
|
.PP
|
|
So let's say we had \f[C]\-\-suffix\ \-2019\-01\-01\f[], without the
|
|
flag \f[C]file.txt\f[] would be backed up to
|
|
\f[C]file.txt\-2019\-01\-01\f[] and with the flag it would be backed up
|
|
to \f[C]file\-2019\-01\-01.txt\f[].
|
|
This can be helpful to make sure the suffixed files can still be opened.
|
|
.SS \[en]syslog
|
|
.PP
|
|
On capable OSes (not Windows or Plan9) send all log output to syslog.
|
|
.PP
|
|
This can be useful for running rclone in a script or
|
|
\f[C]rclone\ mount\f[].
|
|
.SS \[en]syslog\-facility string
|
|
.PP
|
|
If using \f[C]\-\-syslog\f[] this sets the syslog facility (eg
|
|
\f[C]KERN\f[], \f[C]USER\f[]).
|
|
See \f[C]man\ syslog\f[] for a list of possible facilities.
|
|
The default facility is \f[C]DAEMON\f[].
|
|
.SS \[en]tpslimit float
|
|
.PP
|
|
Limit HTTP transactions per second to this.
|
|
Default is 0 which is used to mean unlimited transactions per second.
|
|
.PP
|
|
For example to limit rclone to 10 HTTP transactions per second use
|
|
\f[C]\-\-tpslimit\ 10\f[], or to 1 transaction every 2 seconds use
|
|
\f[C]\-\-tpslimit\ 0.5\f[].
|
|
.PP
|
|
Use this when the number of transactions per second from rclone is
|
|
causing a problem with the cloud storage provider (eg getting you banned
|
|
or rate limited).
|
|
.PP
|
|
This can be very useful for \f[C]rclone\ mount\f[] to control the
|
|
behaviour of applications using it.
|
|
.PP
|
|
See also \f[C]\-\-tpslimit\-burst\f[].
|
|
.SS \[en]tpslimit\-burst int
|
|
.PP
|
|
Max burst of transactions for \f[C]\-\-tpslimit\f[].
|
|
(default 1)
|
|
.PP
|
|
Normally \f[C]\-\-tpslimit\f[] will do exactly the number of transaction
|
|
per second specified.
|
|
However if you supply \f[C]\-\-tps\-burst\f[] then rclone can save up
|
|
some transactions from when it was idle giving a burst of up to the
|
|
parameter supplied.
|
|
.PP
|
|
For example if you provide \f[C]\-\-tpslimit\-burst\ 10\f[] then if
|
|
rclone has been idle for more than 10*\f[C]\-\-tpslimit\f[] then it can
|
|
do 10 transactions very quickly before they are limited again.
|
|
.PP
|
|
This may be used to increase performance of \f[C]\-\-tpslimit\f[]
|
|
without changing the long term average number of transactions per
|
|
second.
|
|
.SS \[en]track\-renames
|
|
.PP
|
|
By default, rclone doesn't keep track of renamed files, so if you rename
|
|
a file locally then sync it to a remote, rclone will delete the old file
|
|
on the remote and upload a new copy.
|
|
.PP
|
|
If you use this flag, and the remote supports server side copy or server
|
|
side move, and the source and destination have a compatible hash, then
|
|
this will track renames during \f[C]sync\f[] operations and perform
|
|
renaming server\-side.
|
|
.PP
|
|
Files will be matched by size and hash \- if both match then a rename
|
|
will be considered.
|
|
.PP
|
|
If the destination does not support server\-side copy or move, rclone
|
|
will fall back to the default behaviour and log an error level message
|
|
to the console.
|
|
Note: Encrypted destinations are not supported by
|
|
\f[C]\-\-track\-renames\f[].
|
|
.PP
|
|
Note that \f[C]\-\-track\-renames\f[] is incompatible with
|
|
\f[C]\-\-no\-traverse\f[] and that it uses extra memory to keep track of
|
|
all the rename candidates.
|
|
.PP
|
|
Note also that \f[C]\-\-track\-renames\f[] is incompatible with
|
|
\f[C]\-\-delete\-before\f[] and will select \f[C]\-\-delete\-after\f[]
|
|
instead of \f[C]\-\-delete\-during\f[].
|
|
.SS \[en]delete\-(before,during,after)
|
|
.PP
|
|
This option allows you to specify when files on your destination are
|
|
deleted when you sync folders.
|
|
.PP
|
|
Specifying the value \f[C]\-\-delete\-before\f[] will delete all files
|
|
present on the destination, but not on the source \f[I]before\f[]
|
|
starting the transfer of any new or updated files.
|
|
This uses two passes through the file systems, one for the deletions and
|
|
one for the copies.
|
|
.PP
|
|
Specifying \f[C]\-\-delete\-during\f[] will delete files while checking
|
|
and uploading files.
|
|
This is the fastest option and uses the least memory.
|
|
.PP
|
|
Specifying \f[C]\-\-delete\-after\f[] (the default value) will delay
|
|
deletion of files until all new/updated files have been successfully
|
|
transferred.
|
|
The files to be deleted are collected in the copy pass then deleted
|
|
after the copy pass has completed successfully.
|
|
The files to be deleted are held in memory so this mode may use more
|
|
memory.
|
|
This is the safest mode as it will only delete files if there have been
|
|
no errors subsequent to that.
|
|
If there have been errors before the deletions start then you will get
|
|
the message \f[C]not\ deleting\ files\ as\ there\ were\ IO\ errors\f[].
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
When doing anything which involves a directory listing (eg
|
|
\f[C]sync\f[], \f[C]copy\f[], \f[C]ls\f[] \- in fact nearly every
|
|
command), rclone normally lists a directory and processes it before
|
|
using more directory lists to process any subdirectories.
|
|
This can be parallelised and works very quickly using the least amount
|
|
of memory.
|
|
.PP
|
|
However, some remotes have a way of listing all files beneath a
|
|
directory in one (or a small number) of transactions.
|
|
These tend to be the bucket based remotes (eg S3, B2, GCS, Swift,
|
|
Hubic).
|
|
.PP
|
|
If you use the \f[C]\-\-fast\-list\f[] flag then rclone will use this
|
|
method for listing directories.
|
|
This will have the following consequences for the listing:
|
|
.IP \[bu] 2
|
|
It \f[B]will\f[] use fewer transactions (important if you pay for them)
|
|
.IP \[bu] 2
|
|
It \f[B]will\f[] use more memory.
|
|
Rclone has to load the whole listing into memory.
|
|
.IP \[bu] 2
|
|
It \f[I]may\f[] be faster because it uses fewer transactions
|
|
.IP \[bu] 2
|
|
It \f[I]may\f[] be slower because it can't be parallelized
|
|
.PP
|
|
rclone should always give identical results with and without
|
|
\f[C]\-\-fast\-list\f[].
|
|
.PP
|
|
If you pay for transactions and can fit your entire sync listing into
|
|
memory then \f[C]\-\-fast\-list\f[] is recommended.
|
|
If you have a very big sync to do then don't use \f[C]\-\-fast\-list\f[]
|
|
otherwise you will run out of memory.
|
|
.PP
|
|
If you use \f[C]\-\-fast\-list\f[] on a remote which doesn't support it,
|
|
then rclone will just ignore it.
|
|
.SS \[en]timeout=TIME
|
|
.PP
|
|
This sets the IO idle timeout.
|
|
If a transfer has started but then becomes idle for this long it is
|
|
considered broken and disconnected.
|
|
.PP
|
|
The default is \f[C]5m\f[].
|
|
Set to 0 to disable.
|
|
.SS \[en]transfers=N
|
|
.PP
|
|
The number of file transfers to run in parallel.
|
|
It can sometimes be useful to set this to a smaller number if the remote
|
|
is giving a lot of timeouts or bigger if you have lots of bandwidth and
|
|
a fast remote.
|
|
.PP
|
|
The default is to run 4 file transfers in parallel.
|
|
.SS \-u, \[en]update
|
|
.PP
|
|
This forces rclone to skip any files which exist on the destination and
|
|
have a modified time that is newer than the source file.
|
|
.PP
|
|
This can be useful when transferring to a remote which doesn't support
|
|
mod times directly (or when using \f[C]\-\-use\-server\-modtime\f[] to
|
|
avoid extra API calls) as it is more accurate than a
|
|
\f[C]\-\-size\-only\f[] check and faster than using
|
|
\f[C]\-\-checksum\f[].
|
|
.PP
|
|
If an existing destination file has a modification time equal (within
|
|
the computed modify window precision) to the source file's, it will be
|
|
updated if the sizes are different.
|
|
If \f[C]\-\-checksum\f[] is set then rclone will update the destination
|
|
if the checksums differ too.
|
|
.PP
|
|
If an existing destination file is older than the source file then it
|
|
will be updated if the size or checksum differs from the source file.
|
|
.PP
|
|
On remotes which don't support mod time directly (or when using
|
|
\f[C]\-\-use\-server\-modtime\f[]) the time checked will be the uploaded
|
|
time.
|
|
This means that if uploading to one of these remotes, rclone will skip
|
|
any files which exist on the destination and have an uploaded time that
|
|
is newer than the modification time of the source file.
|
|
.SS \[en]use\-mmap
|
|
.PP
|
|
If this flag is set then rclone will use anonymous memory allocated by
|
|
mmap on Unix based platforms and VirtualAlloc on Windows for its
|
|
transfer buffers (size controlled by \f[C]\-\-buffer\-size\f[]).
|
|
Memory allocated like this does not go on the Go heap and can be
|
|
returned to the OS immediately when it is finished with.
|
|
.PP
|
|
If this flag is not set then rclone will allocate and free the buffers
|
|
using the Go memory allocator which may use more memory as memory pages
|
|
are returned less aggressively to the OS.
|
|
.PP
|
|
It is possible this does not work well on all platforms so it is
|
|
disabled by default; in the future it may be enabled by default.
|
|
.SS \[en]use\-server\-modtime
|
|
.PP
|
|
Some object\-store backends (e.g, Swift, S3) do not preserve file
|
|
modification times (modtime).
|
|
On these backends, rclone stores the original modtime as additional
|
|
metadata on the object.
|
|
By default it will make an API call to retrieve the metadata when the
|
|
modtime is needed by an operation.
|
|
.PP
|
|
Use this flag to disable the extra API call and rely instead on the
|
|
server's modified time.
|
|
In cases such as a local to remote sync using \f[C]\-\-update\f[],
|
|
knowing the local file is newer than the time it was last uploaded to
|
|
the remote is sufficient.
|
|
In those cases, this flag can speed up the process and reduce the number
|
|
of API calls necessary.
|
|
.PP
|
|
Using this flag on a sync operation without also using
|
|
\f[C]\-\-update\f[] would cause all files modified at any time other
|
|
than the last upload time to be uploaded again, which is probably not
|
|
what you want.
|
|
.SS \-v, \-vv, \[en]verbose
|
|
.PP
|
|
With \f[C]\-v\f[] rclone will tell you about each file that is
|
|
transferred and a small number of significant events.
|
|
.PP
|
|
With \f[C]\-vv\f[] rclone will become very verbose telling you about
|
|
every file it considers and transfers.
|
|
Please send bug reports with a log with this setting.
|
|
.SS \-V, \[en]version
|
|
.PP
|
|
Prints the version number
|
|
.SS SSL/TLS options
|
|
.PP
|
|
The outoing SSL/TLS connections rclone makes can be controlled with
|
|
these options.
|
|
For example this can be very useful with the HTTP or WebDAV backends.
|
|
Rclone HTTP servers have their own set of configuration for SSL/TLS
|
|
which you can find in their documentation.
|
|
.SS \[en]ca\-cert string
|
|
.PP
|
|
This loads the PEM encoded certificate authority certificate and uses it
|
|
to verify the certificates of the servers rclone connects to.
|
|
.PP
|
|
If you have generated certificates signed with a local CA then you will
|
|
need this flag to connect to servers using those certificates.
|
|
.SS \[en]client\-cert string
|
|
.PP
|
|
This loads the PEM encoded client side certificate.
|
|
.PP
|
|
This is used for mutual TLS
|
|
authentication (https://en.wikipedia.org/wiki/Mutual_authentication).
|
|
.PP
|
|
The \f[C]\-\-client\-key\f[] flag is required too when using this.
|
|
.SS \[en]client\-key string
|
|
.PP
|
|
This loads the PEM encoded client side private key used for mutual TLS
|
|
authentication.
|
|
Used in conjunction with \f[C]\-\-client\-cert\f[].
|
|
.SS \[en]no\-check\-certificate=true/false
|
|
.PP
|
|
\f[C]\-\-no\-check\-certificate\f[] controls whether a client verifies
|
|
the server's certificate chain and host name.
|
|
If \f[C]\-\-no\-check\-certificate\f[] is true, TLS accepts any
|
|
certificate presented by the server and any host name in that
|
|
certificate.
|
|
In this mode, TLS is susceptible to man\-in\-the\-middle attacks.
|
|
.PP
|
|
This option defaults to \f[C]false\f[].
|
|
.PP
|
|
\f[B]This should be used only for testing.\f[]
|
|
.SS Configuration Encryption
|
|
.PP
|
|
Your configuration file contains information for logging in to your
|
|
cloud services.
|
|
This means that you should keep your \f[C]\&.rclone.conf\f[] file in a
|
|
secure location.
|
|
.PP
|
|
If you are in an environment where that isn't possible, you can add a
|
|
password to your configuration.
|
|
This means that you will have to enter the password every time you start
|
|
rclone.
|
|
.PP
|
|
To add a password to your rclone configuration, execute
|
|
\f[C]rclone\ config\f[].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
>rclone\ config
|
|
Current\ remotes:
|
|
|
|
e)\ Edit\ existing\ remote
|
|
n)\ New\ remote
|
|
d)\ Delete\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
e/n/d/s/q>
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Go into \f[C]s\f[], Set configuration password:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
e/n/d/s/q>\ s
|
|
Your\ configuration\ is\ not\ encrypted.
|
|
If\ you\ add\ a\ password,\ you\ will\ protect\ your\ login\ information\ to\ cloud\ services.
|
|
a)\ Add\ Password
|
|
q)\ Quit\ to\ main\ menu
|
|
a/q>\ a
|
|
Enter\ NEW\ configuration\ password:
|
|
password:
|
|
Confirm\ NEW\ password:
|
|
password:
|
|
Password\ set
|
|
Your\ configuration\ is\ encrypted.
|
|
c)\ Change\ Password
|
|
u)\ Unencrypt\ configuration
|
|
q)\ Quit\ to\ main\ menu
|
|
c/u/q>
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Your configuration is now encrypted, and every time you start rclone you
|
|
will now be asked for the password.
|
|
In the same menu, you can change the password or completely remove
|
|
encryption from your configuration.
|
|
.PP
|
|
There is no way to recover the configuration if you lose your password.
|
|
.PP
|
|
rclone uses nacl
|
|
secretbox (https://godoc.org/golang.org/x/crypto/nacl/secretbox) which
|
|
in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your
|
|
configuration with secret\-key cryptography.
|
|
The password is SHA\-256 hashed, which produces the key for secretbox.
|
|
The hashed password is not stored.
|
|
.PP
|
|
While this provides very good security, we do not recommend storing your
|
|
encrypted rclone configuration in public if it contains sensitive
|
|
information, maybe except if you use a very strong password.
|
|
.PP
|
|
If it is safe in your environment, you can set the
|
|
\f[C]RCLONE_CONFIG_PASS\f[] environment variable to contain your
|
|
password, in which case it will be used for decrypting the
|
|
configuration.
|
|
.PP
|
|
You can set this for a session from a script.
|
|
For unix like systems save this to a file called
|
|
\f[C]set\-rclone\-password\f[]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#!/bin/echo\ Source\ this\ file\ don\[aq]t\ run\ it
|
|
|
|
read\ \-s\ RCLONE_CONFIG_PASS
|
|
export\ RCLONE_CONFIG_PASS
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then source the file when you want to use it.
|
|
From the shell you would do \f[C]source\ set\-rclone\-password\f[].
|
|
It will then ask you for the password and set it in the environment
|
|
variable.
|
|
.PP
|
|
If you are running rclone inside a script, you might want to disable
|
|
password prompts.
|
|
To do that, pass the parameter \f[C]\-\-ask\-password=false\f[] to
|
|
rclone.
|
|
This will make rclone fail instead of asking for a password if
|
|
\f[C]RCLONE_CONFIG_PASS\f[] doesn't contain a valid password.
|
|
.SS Developer options
|
|
.PP
|
|
These options are useful when developing or debugging rclone.
|
|
There are also some more remote specific options which aren't documented
|
|
here which are used for testing.
|
|
These start with remote name eg \f[C]\-\-drive\-test\-option\f[] \- see
|
|
the docs for the remote in question.
|
|
.SS \[en]cpuprofile=FILE
|
|
.PP
|
|
Write CPU profile to file.
|
|
This can be analysed with \f[C]go\ tool\ pprof\f[].
|
|
.SS \[en]dump flag,flag,flag
|
|
.PP
|
|
The \f[C]\-\-dump\f[] flag takes a comma separated list of flags to dump
|
|
info about.
|
|
These are:
|
|
.SS \[en]dump headers
|
|
.PP
|
|
Dump HTTP headers with \f[C]Authorization:\f[] lines removed.
|
|
May still contain sensitive info.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.PP
|
|
Use \f[C]\-\-dump\ auth\f[] if you do want the \f[C]Authorization:\f[]
|
|
headers.
|
|
.SS \[en]dump bodies
|
|
.PP
|
|
Dump HTTP headers and bodies \- may contain sensitive info.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.PP
|
|
Note that the bodies are buffered in memory so don't use this for
|
|
enormous files.
|
|
.SS \[en]dump requests
|
|
.PP
|
|
Like \f[C]\-\-dump\ bodies\f[] but dumps the request bodies and the
|
|
response headers.
|
|
Useful for debugging download problems.
|
|
.SS \[en]dump responses
|
|
.PP
|
|
Like \f[C]\-\-dump\ bodies\f[] but dumps the response bodies and the
|
|
request headers.
|
|
Useful for debugging upload problems.
|
|
.SS \[en]dump auth
|
|
.PP
|
|
Dump HTTP headers \- will contain sensitive info such as
|
|
\f[C]Authorization:\f[] headers \- use \f[C]\-\-dump\ headers\f[] to
|
|
dump without \f[C]Authorization:\f[] headers.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.SS \[en]dump filters
|
|
.PP
|
|
Dump the filters to the output.
|
|
Useful to see exactly what include and exclude options are filtering on.
|
|
.SS \[en]dump goroutines
|
|
.PP
|
|
This dumps a list of the running go\-routines at the end of the command
|
|
to standard output.
|
|
.SS \[en]dump openfiles
|
|
.PP
|
|
This dumps a list of the open files at the end of the command.
|
|
It uses the \f[C]lsof\f[] command to do that so you'll need that
|
|
installed to use it.
|
|
.SS \[en]memprofile=FILE
|
|
.PP
|
|
Write memory profile to file.
|
|
This can be analysed with \f[C]go\ tool\ pprof\f[].
|
|
.SS Filtering
|
|
.PP
|
|
For the filtering options
|
|
.IP \[bu] 2
|
|
\f[C]\-\-delete\-excluded\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\-from\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\-from\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-files\-from\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-min\-size\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-size\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-min\-age\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-age\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dump\ filters\f[]
|
|
.PP
|
|
See the filtering section (https://rclone.org/filtering/).
|
|
.SS Remote control
|
|
.PP
|
|
For the remote control options and for instructions on how to remote
|
|
control rclone
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\f[]
|
|
.IP \[bu] 2
|
|
and anything starting with \f[C]\-\-rc\-\f[]
|
|
.PP
|
|
See the remote control section (https://rclone.org/rc/).
|
|
.SS Logging
|
|
.PP
|
|
rclone has 4 levels of logging, \f[C]ERROR\f[], \f[C]NOTICE\f[],
|
|
\f[C]INFO\f[] and \f[C]DEBUG\f[].
|
|
.PP
|
|
By default, rclone logs to standard error.
|
|
This means you can redirect standard error and still see the normal
|
|
output of rclone commands (eg \f[C]rclone\ ls\f[]).
|
|
.PP
|
|
By default, rclone will produce \f[C]Error\f[] and \f[C]Notice\f[] level
|
|
messages.
|
|
.PP
|
|
If you use the \f[C]\-q\f[] flag, rclone will only produce
|
|
\f[C]Error\f[] messages.
|
|
.PP
|
|
If you use the \f[C]\-v\f[] flag, rclone will produce \f[C]Error\f[],
|
|
\f[C]Notice\f[] and \f[C]Info\f[] messages.
|
|
.PP
|
|
If you use the \f[C]\-vv\f[] flag, rclone will produce \f[C]Error\f[],
|
|
\f[C]Notice\f[], \f[C]Info\f[] and \f[C]Debug\f[] messages.
|
|
.PP
|
|
You can also control the log levels with the \f[C]\-\-log\-level\f[]
|
|
flag.
|
|
.PP
|
|
If you use the \f[C]\-\-log\-file=FILE\f[] option, rclone will redirect
|
|
\f[C]Error\f[], \f[C]Info\f[] and \f[C]Debug\f[] messages along with
|
|
standard error to FILE.
|
|
.PP
|
|
If you use the \f[C]\-\-syslog\f[] flag then rclone will log to syslog
|
|
and the \f[C]\-\-syslog\-facility\f[] control which facility it uses.
|
|
.PP
|
|
Rclone prefixes all log messages with their level in capitals, eg INFO
|
|
which makes it easy to grep the log file for different kinds of
|
|
information.
|
|
.SS Exit Code
|
|
.PP
|
|
If any errors occur during the command execution, rclone will exit with
|
|
a non\-zero exit code.
|
|
This allows scripts to detect when rclone operations have failed.
|
|
.PP
|
|
During the startup phase, rclone will exit immediately if an error is
|
|
detected in the configuration.
|
|
There will always be a log message immediately before exiting.
|
|
.PP
|
|
When rclone is running it will accumulate errors as it goes along, and
|
|
only exit with a non\-zero exit code if (after retries) there were still
|
|
failed transfers.
|
|
For every error counted there will be a high priority log message
|
|
(visible with \f[C]\-q\f[]) showing the message and which file caused
|
|
the problem.
|
|
A high priority message is also shown when starting a retry so the user
|
|
can see that any previous error messages may not be valid after the
|
|
retry.
|
|
If rclone has done a retry it will log a high priority message if the
|
|
retry was successful.
|
|
.SS List of exit codes
|
|
.IP \[bu] 2
|
|
\f[C]0\f[] \- success
|
|
.IP \[bu] 2
|
|
\f[C]1\f[] \- Syntax or usage error
|
|
.IP \[bu] 2
|
|
\f[C]2\f[] \- Error not otherwise categorised
|
|
.IP \[bu] 2
|
|
\f[C]3\f[] \- Directory not found
|
|
.IP \[bu] 2
|
|
\f[C]4\f[] \- File not found
|
|
.IP \[bu] 2
|
|
\f[C]5\f[] \- Temporary error (one that more retries might fix) (Retry
|
|
errors)
|
|
.IP \[bu] 2
|
|
\f[C]6\f[] \- Less serious errors (like 461 errors from dropbox)
|
|
(NoRetry errors)
|
|
.IP \[bu] 2
|
|
\f[C]7\f[] \- Fatal error (one that more retries won't fix, like account
|
|
suspended) (Fatal errors)
|
|
.IP \[bu] 2
|
|
\f[C]8\f[] \- Transfer exceeded \- limit set by \[en]max\-transfer
|
|
reached
|
|
.SS Environment Variables
|
|
.PP
|
|
Rclone can be configured entirely using environment variables.
|
|
These can be used to set defaults for options or config file entries.
|
|
.SS Options
|
|
.PP
|
|
Every option in rclone can have its default set by environment variable.
|
|
.PP
|
|
To find the name of the environment variable, first, take the long
|
|
option name, strip the leading \f[C]\-\-\f[], change \f[C]\-\f[] to
|
|
\f[C]_\f[], make upper case and prepend \f[C]RCLONE_\f[].
|
|
.PP
|
|
For example, to always set \f[C]\-\-stats\ 5s\f[], set the environment
|
|
variable \f[C]RCLONE_STATS=5s\f[].
|
|
If you set stats on the command line this will override the environment
|
|
variable setting.
|
|
.PP
|
|
Or to always use the trash in drive \f[C]\-\-drive\-use\-trash\f[], set
|
|
\f[C]RCLONE_DRIVE_USE_TRASH=true\f[].
|
|
.PP
|
|
The same parser is used for the options and the environment variables so
|
|
they take exactly the same form.
|
|
.SS Config file
|
|
.PP
|
|
You can set defaults for values in the config file on an individual
|
|
remote basis.
|
|
If you want to use this feature, you will need to discover the name of
|
|
the config items that you want.
|
|
The easiest way is to run through \f[C]rclone\ config\f[] by hand, then
|
|
look in the config file to see what the values are (the config file can
|
|
be found by looking at the help for \f[C]\-\-config\f[] in
|
|
\f[C]rclone\ help\f[]).
|
|
.PP
|
|
To find the name of the environment variable, you need to set, take
|
|
\f[C]RCLONE_CONFIG_\f[] + name of remote + \f[C]_\f[] + name of config
|
|
file option and make it all uppercase.
|
|
.PP
|
|
For example, to configure an S3 remote named \f[C]mys3:\f[] without a
|
|
config file (using unix ways of setting environment variables):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ export\ RCLONE_CONFIG_MYS3_TYPE=s3
|
|
$\ export\ RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
|
|
$\ export\ RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
|
|
$\ rclone\ lsd\ MYS3:
|
|
\ \ \ \ \ \ \ \ \ \ \-1\ 2016\-09\-21\ 12:54:21\ \ \ \ \ \ \ \ \-1\ my\-bucket
|
|
$\ rclone\ listremotes\ |\ grep\ mys3
|
|
mys3:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that if you want to create a remote using environment variables you
|
|
must create the \f[C]\&..._TYPE\f[] variable as above.
|
|
.SS Other environment variables
|
|
.IP \[bu] 2
|
|
RCLONE_CONFIG_PASS` set to contain your config file password (see
|
|
Configuration Encryption section)
|
|
.IP \[bu] 2
|
|
HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions
|
|
thereof).
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.
|
|
.IP \[bu] 2
|
|
The environment values may be either a complete URL or a
|
|
\[lq]host[:port]\[rq] for, in which case the \[lq]http\[rq] scheme is
|
|
assumed.
|
|
.RE
|
|
.SH Configuring rclone on a remote / headless machine
|
|
.PP
|
|
Some of the configurations (those involving oauth2) require an Internet
|
|
connected web browser.
|
|
.PP
|
|
If you are trying to set rclone up on a remote or headless box with no
|
|
browser available on it (eg a NAS or a server in a datacenter) then you
|
|
will need to use an alternative means of configuration.
|
|
There are two ways of doing it, described below.
|
|
.SS Configuring using rclone authorize
|
|
.PP
|
|
On the headless box
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&...
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ n
|
|
For\ this\ to\ work,\ you\ will\ need\ rclone\ available\ on\ a\ machine\ that\ has\ a\ web\ browser\ available.
|
|
Execute\ the\ following\ on\ your\ machine:
|
|
\ \ \ \ rclone\ authorize\ "amazon\ cloud\ drive"
|
|
Then\ paste\ the\ result\ below:
|
|
result>
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then on your main desktop machine
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ authorize\ "amazon\ cloud\ drive"
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
Paste\ the\ following\ into\ your\ remote\ machine\ \-\-\->
|
|
SECRET_TOKEN
|
|
<\-\-\-End\ paste
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then back to the headless box, paste in the code
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
result>\ SECRET_TOKEN
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[acd12]
|
|
client_id\ =\
|
|
client_secret\ =\
|
|
token\ =\ SECRET_TOKEN
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>
|
|
\f[]
|
|
.fi
|
|
.SS Configuring by copying the config file
|
|
.PP
|
|
Rclone stores all of its config in a single configuration file.
|
|
This can easily be copied to configure a remote rclone.
|
|
.PP
|
|
So first configure rclone on your desktop machine
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
to set up the config file.
|
|
.PP
|
|
Find the config file by running \f[C]rclone\ config\ file\f[], for
|
|
example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ config\ file
|
|
Configuration\ file\ is\ stored\ at:
|
|
/home/user/.rclone.conf
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Now transfer it to the remote box (scp, cut paste, ftp, sftp etc) and
|
|
place it in the correct place (use \f[C]rclone\ config\ file\f[] on the
|
|
remote box to find out where).
|
|
.SH Filtering, includes and excludes
|
|
.PP
|
|
Rclone has a sophisticated set of include and exclude rules.
|
|
Some of these are based on patterns and some on other things like file
|
|
size.
|
|
.PP
|
|
The filters are applied for the \f[C]copy\f[], \f[C]sync\f[],
|
|
\f[C]move\f[], \f[C]ls\f[], \f[C]lsl\f[], \f[C]md5sum\f[],
|
|
\f[C]sha1sum\f[], \f[C]size\f[], \f[C]delete\f[] and \f[C]check\f[]
|
|
operations.
|
|
Note that \f[C]purge\f[] does not obey the filters.
|
|
.PP
|
|
Each path as it passes through rclone is matched against the include and
|
|
exclude rules like \f[C]\-\-include\f[], \f[C]\-\-exclude\f[],
|
|
\f[C]\-\-include\-from\f[], \f[C]\-\-exclude\-from\f[],
|
|
\f[C]\-\-filter\f[], or \f[C]\-\-filter\-from\f[].
|
|
The simplest way to try them out is using the \f[C]ls\f[] command, or
|
|
\f[C]\-\-dry\-run\f[] together with \f[C]\-v\f[].
|
|
.SS Patterns
|
|
.PP
|
|
The patterns used to match files for inclusion or exclusion are based on
|
|
\[lq]file globs\[rq] as used by the unix shell.
|
|
.PP
|
|
If the pattern starts with a \f[C]/\f[] then it only matches at the top
|
|
level of the directory tree, \f[B]relative to the root of the remote\f[]
|
|
(not necessarily the root of the local drive).
|
|
If it doesn't start with \f[C]/\f[] then it is matched starting at the
|
|
\f[B]end of the path\f[], but it will only match a complete path
|
|
element:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
file.jpg\ \ \-\ matches\ "file.jpg"
|
|
\ \ \ \ \ \ \ \ \ \ \-\ matches\ "directory/file.jpg"
|
|
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "afile.jpg"
|
|
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "directory/afile.jpg"
|
|
/file.jpg\ \-\ matches\ "file.jpg"\ in\ the\ root\ directory\ of\ the\ remote
|
|
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "afile.jpg"
|
|
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "directory/file.jpg"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\f[B]Important\f[] Note that you must use \f[C]/\f[] in patterns and not
|
|
\f[C]\\\f[] even if running on Windows.
|
|
.PP
|
|
A \f[C]*\f[] matches anything but not a \f[C]/\f[].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
*.jpg\ \ \-\ matches\ "file.jpg"
|
|
\ \ \ \ \ \ \ \-\ matches\ "directory/file.jpg"
|
|
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "file.jpg/something"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Use \f[C]**\f[] to match anything, including slashes (\f[C]/\f[]).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
dir/**\ \-\ matches\ "dir/file.jpg"
|
|
\ \ \ \ \ \ \ \-\ matches\ "dir/dir1/dir2/file.jpg"
|
|
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "directory/file.jpg"
|
|
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "adir/file.jpg"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
A \f[C]?\f[] matches any character except a slash \f[C]/\f[].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
l?ss\ \ \-\ matches\ "less"
|
|
\ \ \ \ \ \ \-\ matches\ "lass"
|
|
\ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "floss"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
A \f[C][\f[] and \f[C]]\f[] together make a character class, such as
|
|
\f[C][a\-z]\f[] or \f[C][aeiou]\f[] or \f[C][[:alpha:]]\f[].
|
|
See the go regexp docs (https://golang.org/pkg/regexp/syntax/) for more
|
|
info on these.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
h[ae]llo\ \-\ matches\ "hello"
|
|
\ \ \ \ \ \ \ \ \ \-\ matches\ "hallo"
|
|
\ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "hullo"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
A \f[C]{\f[] and \f[C]}\f[] define a choice between elements.
|
|
It should contain a comma separated list of patterns, any of which might
|
|
match.
|
|
These patterns can contain wildcards.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{one,two}_potato\ \-\ matches\ "one_potato"
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \-\ matches\ "two_potato"
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "three_potato"
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "_potato"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Special characters can be escaped with a \f[C]\\\f[] before them.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\\*.jpg\ \ \ \ \ \ \ \-\ matches\ "*.jpg"
|
|
\\\\.jpg\ \ \ \ \ \ \ \-\ matches\ "\\.jpg"
|
|
\\[one\\].jpg\ \ \-\ matches\ "[one].jpg"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Patterns are case sensitive unless the \f[C]\-\-ignore\-case\f[] flag is
|
|
used.
|
|
.PP
|
|
Without \f[C]\-\-ignore\-case\f[] (default)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
potato\ \-\ matches\ "potato"
|
|
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "POTATO"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
With \f[C]\-\-ignore\-case\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
potato\ \-\ matches\ "potato"
|
|
\ \ \ \ \ \ \ \-\ matches\ "POTATO"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note also that rclone filter globs can only be used in one of the filter
|
|
command line flags, not in the specification of the remote, so
|
|
\f[C]rclone\ copy\ "remote:dir*.jpg"\ /path/to/dir\f[] won't work \-
|
|
what is required is
|
|
\f[C]rclone\ \-\-include\ "*.jpg"\ copy\ remote:dir\ /path/to/dir\f[]
|
|
.SS Directories
|
|
.PP
|
|
Rclone keeps track of directories that could match any file patterns.
|
|
.PP
|
|
Eg if you add the include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/a/*.jpg
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Rclone will synthesize the directory include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/a/
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you put any rules which end in \f[C]/\f[] then it will only match
|
|
directories.
|
|
.PP
|
|
Directory matches are \f[B]only\f[] used to optimise directory access
|
|
patterns \- you must still match the files that you want to match.
|
|
Directory matches won't optimise anything on bucket based remotes (eg
|
|
s3, swift, google compute storage, b2) which don't have a concept of
|
|
directory.
|
|
.SS Differences between rsync and rclone patterns
|
|
.PP
|
|
Rclone implements bash style \f[C]{a,b,c}\f[] glob matching which rsync
|
|
doesn't.
|
|
.PP
|
|
Rclone always does a wildcard match so \f[C]\\\f[] must always escape a
|
|
\f[C]\\\f[].
|
|
.SS How the rules are used
|
|
.PP
|
|
Rclone maintains a combined list of include rules and exclude rules.
|
|
.PP
|
|
Each file is matched in order, starting from the top, against the rule
|
|
in the list until it finds a match.
|
|
The file is then included or excluded according to the rule type.
|
|
.PP
|
|
If the matcher fails to find a match after testing against all the
|
|
entries in the list then the path is included.
|
|
.PP
|
|
For example given the following rules, \f[C]+\f[] being include,
|
|
\f[C]\-\f[] being exclude,
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\ secret*.jpg
|
|
+\ *.jpg
|
|
+\ *.png
|
|
+\ file2.avi
|
|
\-\ *
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This would include
|
|
.IP \[bu] 2
|
|
\f[C]file1.jpg\f[]
|
|
.IP \[bu] 2
|
|
\f[C]file3.png\f[]
|
|
.IP \[bu] 2
|
|
\f[C]file2.avi\f[]
|
|
.PP
|
|
This would exclude
|
|
.IP \[bu] 2
|
|
\f[C]secret17.jpg\f[]
|
|
.IP \[bu] 2
|
|
non \f[C]*.jpg\f[] and \f[C]*.png\f[]
|
|
.PP
|
|
A similar process is done on directory entries before recursing into
|
|
them.
|
|
This only works on remotes which have a concept of directory (Eg local,
|
|
google drive, onedrive, amazon drive) and not on bucket based remotes
|
|
(eg s3, swift, google compute storage, b2).
|
|
.SS Adding filtering rules
|
|
.PP
|
|
Filtering rules are added with the following command line flags.
|
|
.SS Repeating options
|
|
.PP
|
|
You can repeat the following options to add more than one rule of that
|
|
type.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\-from\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\-from\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\f[]
|
|
.PP
|
|
\f[B]Important\f[] You should not use \f[C]\-\-include*\f[] together
|
|
with \f[C]\-\-exclude*\f[].
|
|
It may produce different results than you expected.
|
|
In that case try to use: \f[C]\-\-filter*\f[].
|
|
.PP
|
|
Note that all the options of the same type are processed together in the
|
|
order above, regardless of what order they were placed on the command
|
|
line.
|
|
.PP
|
|
So all \f[C]\-\-include\f[] options are processed first in the order
|
|
they appeared on the command line, then all \f[C]\-\-include\-from\f[]
|
|
options etc.
|
|
.PP
|
|
To mix up the order includes and excludes, the \f[C]\-\-filter\f[] flag
|
|
can be used.
|
|
.SS \f[C]\-\-exclude\f[] \- Exclude files matching pattern
|
|
.PP
|
|
Add a single exclude rule with \f[C]\-\-exclude\f[].
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Eg \f[C]\-\-exclude\ *.bak\f[] to exclude all bak files from the sync.
|
|
.SS \f[C]\-\-exclude\-from\f[] \- Read exclude patterns from file
|
|
.PP
|
|
Add exclude rules from a file.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Prepare a file like this \f[C]exclude\-file.txt\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ a\ sample\ exclude\ rule\ file
|
|
*.bak
|
|
file2.jpg
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then use as \f[C]\-\-exclude\-from\ exclude\-file.txt\f[].
|
|
This will sync all files except those ending in \f[C]bak\f[] and
|
|
\f[C]file2.jpg\f[].
|
|
.PP
|
|
This is useful if you have a lot of rules.
|
|
.SS \f[C]\-\-include\f[] \- Include files matching pattern
|
|
.PP
|
|
Add a single include rule with \f[C]\-\-include\f[].
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Eg \f[C]\-\-include\ *.{png,jpg}\f[] to include all \f[C]png\f[] and
|
|
\f[C]jpg\f[] files in the backup and no others.
|
|
.PP
|
|
This adds an implicit \f[C]\-\-exclude\ *\f[] at the very end of the
|
|
filter list.
|
|
This means you can mix \f[C]\-\-include\f[] and
|
|
\f[C]\-\-include\-from\f[] with the other filters (eg
|
|
\f[C]\-\-exclude\f[]) but you must include all the files you want in the
|
|
include statement.
|
|
If this doesn't provide enough flexibility then you must use
|
|
\f[C]\-\-filter\-from\f[].
|
|
.SS \f[C]\-\-include\-from\f[] \- Read include patterns from file
|
|
.PP
|
|
Add include rules from a file.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Prepare a file like this \f[C]include\-file.txt\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ a\ sample\ include\ rule\ file
|
|
*.jpg
|
|
*.png
|
|
file2.avi
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then use as \f[C]\-\-include\-from\ include\-file.txt\f[].
|
|
This will sync all \f[C]jpg\f[], \f[C]png\f[] files and
|
|
\f[C]file2.avi\f[].
|
|
.PP
|
|
This is useful if you have a lot of rules.
|
|
.PP
|
|
This adds an implicit \f[C]\-\-exclude\ *\f[] at the very end of the
|
|
filter list.
|
|
This means you can mix \f[C]\-\-include\f[] and
|
|
\f[C]\-\-include\-from\f[] with the other filters (eg
|
|
\f[C]\-\-exclude\f[]) but you must include all the files you want in the
|
|
include statement.
|
|
If this doesn't provide enough flexibility then you must use
|
|
\f[C]\-\-filter\-from\f[].
|
|
.SS \f[C]\-\-filter\f[] \- Add a file\-filtering rule
|
|
.PP
|
|
This can be used to add a single include or exclude rule.
|
|
Include rules start with \f[C]+\f[] and exclude rules start with
|
|
\f[C]\-\f[].
|
|
A special rule called \f[C]!\f[] can be used to clear the existing
|
|
rules.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Eg \f[C]\-\-filter\ "\-\ *.bak"\f[] to exclude all bak files from the
|
|
sync.
|
|
.SS \f[C]\-\-filter\-from\f[] \- Read filtering patterns from a file
|
|
.PP
|
|
Add include/exclude rules from a file.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Prepare a file like this \f[C]filter\-file.txt\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ a\ sample\ filter\ rule\ file
|
|
\-\ secret*.jpg
|
|
+\ *.jpg
|
|
+\ *.png
|
|
+\ file2.avi
|
|
\-\ /dir/Trash/**
|
|
+\ /dir/**
|
|
#\ exclude\ everything\ else
|
|
\-\ *
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then use as \f[C]\-\-filter\-from\ filter\-file.txt\f[].
|
|
The rules are processed in the order that they are defined.
|
|
.PP
|
|
This example will include all \f[C]jpg\f[] and \f[C]png\f[] files,
|
|
exclude any files matching \f[C]secret*.jpg\f[] and include
|
|
\f[C]file2.avi\f[].
|
|
It will also include everything in the directory \f[C]dir\f[] at the
|
|
root of the sync, except \f[C]dir/Trash\f[] which it will exclude.
|
|
Everything else will be excluded from the sync.
|
|
.SS \f[C]\-\-files\-from\f[] \- Read list of source\-file names
|
|
.PP
|
|
This reads a list of file names from the file passed in and
|
|
\f[B]only\f[] these files are transferred.
|
|
The \f[B]filtering rules are ignored\f[] completely if you use this
|
|
option.
|
|
.PP
|
|
Rclone will traverse the file system if you use
|
|
\f[C]\-\-files\-from\f[], effectively using the files in
|
|
\f[C]\-\-files\-from\f[] as a set of filters.
|
|
Rclone will not error if any of the files are missing.
|
|
.PP
|
|
If you use \f[C]\-\-no\-traverse\f[] as well as \f[C]\-\-files\-from\f[]
|
|
then rclone will not traverse the destination file system, it will find
|
|
each file individually using approximately 1 API call.
|
|
This can be more efficient for small lists of files.
|
|
.PP
|
|
This option can be repeated to read from more than one file.
|
|
These are read in the order that they are placed on the command line.
|
|
.PP
|
|
Paths within the \f[C]\-\-files\-from\f[] file will be interpreted as
|
|
starting with the root specified in the command.
|
|
Leading \f[C]/\f[] characters are ignored.
|
|
.PP
|
|
For example, suppose you had \f[C]files\-from.txt\f[] with this content:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ comment
|
|
file1.jpg
|
|
subdir/file2.jpg
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You could then use it like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-files\-from\ files\-from.txt\ /home/me/pics\ remote:pics
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will transfer these files only (if they exist)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/me/pics/file1.jpg\ \ \ \ \ \ \ \ →\ remote:pics/file1.jpg
|
|
/home/me/pics/subdir/file2.jpg\ →\ remote:pics/subdir/file2.jpg
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To take a more complicated example, let's say you had a few files you
|
|
want to back up regularly with these absolute paths:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important
|
|
/home/user1/dir/file
|
|
/home/user2/stuff
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy these you'd find a common subdirectory \- in this case
|
|
\f[C]/home\f[] and put the remaining files in \f[C]files\-from.txt\f[]
|
|
with or without leading \f[C]/\f[], eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
user1/important
|
|
user1/dir/file
|
|
user2/stuff
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You could then copy these to a remote like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-files\-from\ files\-from.txt\ /home\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The 3 files will arrive in \f[C]remote:backup\f[] with the paths as in
|
|
the \f[C]files\-from.txt\f[] like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important\ →\ remote:backup/user1/important
|
|
/home/user1/dir/file\ \ →\ remote:backup/user1/dir/file
|
|
/home/user2/stuff\ \ \ \ \ →\ remote:backup/user2/stuff
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You could of course choose \f[C]/\f[] as the root too in which case your
|
|
\f[C]files\-from.txt\f[] might look like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important
|
|
/home/user1/dir/file
|
|
/home/user2/stuff
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
And you would transfer it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ \-\-files\-from\ files\-from.txt\ /\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
In this case there will be an extra \f[C]home\f[] directory on the
|
|
remote:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important\ →\ remote:backup/home/user1/important
|
|
/home/user1/dir/file\ \ →\ remote:backup/home/user1/dir/file
|
|
/home/user2/stuff\ \ \ \ \ →\ remote:backup/home/user2/stuff
|
|
\f[]
|
|
.fi
|
|
.SS \f[C]\-\-min\-size\f[] \- Don't transfer any file smaller than this
|
|
.PP
|
|
This option controls the minimum size file which will be transferred.
|
|
This defaults to \f[C]kBytes\f[] but a suffix of \f[C]k\f[], \f[C]M\f[],
|
|
or \f[C]G\f[] can be used.
|
|
.PP
|
|
For example \f[C]\-\-min\-size\ 50k\f[] means no files smaller than
|
|
50kByte will be transferred.
|
|
.SS \f[C]\-\-max\-size\f[] \- Don't transfer any file larger than this
|
|
.PP
|
|
This option controls the maximum size file which will be transferred.
|
|
This defaults to \f[C]kBytes\f[] but a suffix of \f[C]k\f[], \f[C]M\f[],
|
|
or \f[C]G\f[] can be used.
|
|
.PP
|
|
For example \f[C]\-\-max\-size\ 1G\f[] means no files larger than 1GByte
|
|
will be transferred.
|
|
.SS \f[C]\-\-max\-age\f[] \- Don't transfer any file older than this
|
|
.PP
|
|
This option controls the maximum age of files to transfer.
|
|
Give in seconds or with a suffix of:
|
|
.IP \[bu] 2
|
|
\f[C]ms\f[] \- Milliseconds
|
|
.IP \[bu] 2
|
|
\f[C]s\f[] \- Seconds
|
|
.IP \[bu] 2
|
|
\f[C]m\f[] \- Minutes
|
|
.IP \[bu] 2
|
|
\f[C]h\f[] \- Hours
|
|
.IP \[bu] 2
|
|
\f[C]d\f[] \- Days
|
|
.IP \[bu] 2
|
|
\f[C]w\f[] \- Weeks
|
|
.IP \[bu] 2
|
|
\f[C]M\f[] \- Months
|
|
.IP \[bu] 2
|
|
\f[C]y\f[] \- Years
|
|
.PP
|
|
For example \f[C]\-\-max\-age\ 2d\f[] means no files older than 2 days
|
|
will be transferred.
|
|
.SS \f[C]\-\-min\-age\f[] \- Don't transfer any file younger than this
|
|
.PP
|
|
This option controls the minimum age of files to transfer.
|
|
Give in seconds or with a suffix (see \f[C]\-\-max\-age\f[] for list of
|
|
suffixes)
|
|
.PP
|
|
For example \f[C]\-\-min\-age\ 2d\f[] means no files younger than 2 days
|
|
will be transferred.
|
|
.SS \f[C]\-\-delete\-excluded\f[] \- Delete files on dest excluded from
|
|
sync
|
|
.PP
|
|
\f[B]Important\f[] this flag is dangerous \- use with
|
|
\f[C]\-\-dry\-run\f[] and \f[C]\-v\f[] first.
|
|
.PP
|
|
When doing \f[C]rclone\ sync\f[] this will delete any files which are
|
|
excluded from the sync on the destination.
|
|
.PP
|
|
If for example you did a sync from \f[C]A\f[] to \f[C]B\f[] without the
|
|
\f[C]\-\-min\-size\ 50k\f[] flag
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ A:\ B:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then you repeated it like this with the \f[C]\-\-delete\-excluded\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ \-\-min\-size\ 50k\ \-\-delete\-excluded\ sync\ A:\ B:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This would delete all files on \f[C]B\f[] which are less than 50 kBytes
|
|
as these are now excluded from the sync.
|
|
.PP
|
|
Always test first with \f[C]\-\-dry\-run\f[] and \f[C]\-v\f[] before
|
|
using this flag.
|
|
.SS \f[C]\-\-dump\ filters\f[] \- dump the filters to the output
|
|
.PP
|
|
This dumps the defined filters to the output as regular expressions.
|
|
.PP
|
|
Useful for debugging.
|
|
.SS \f[C]\-\-ignore\-case\f[] \- make searches case insensitive
|
|
.PP
|
|
Normally filter patterns are case sensitive.
|
|
If this flag is supplied then filter patterns become case insensitive.
|
|
.PP
|
|
Normally a \f[C]\-\-include\ "file.txt"\f[] will not match a file called
|
|
\f[C]FILE.txt\f[].
|
|
However if you use the \f[C]\-\-ignore\-case\f[] flag then
|
|
\f[C]\-\-include\ "file.txt"\f[] this will match a file called
|
|
\f[C]FILE.txt\f[].
|
|
.SS Quoting shell metacharacters
|
|
.PP
|
|
The examples above may not work verbatim in your shell as they have
|
|
shell metacharacters in them (eg \f[C]*\f[]), and may require quoting.
|
|
.PP
|
|
Eg linux, OSX
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\ \\*.jpg\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\ \[aq]*.jpg\[aq]\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include=\[aq]*.jpg\[aq]\f[]
|
|
.PP
|
|
In Windows the expansion is done by the command not the shell so this
|
|
should work fine
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\ *.jpg\f[]
|
|
.SS Exclude directory based on a file
|
|
.PP
|
|
It is possible to exclude a directory based on a file, which is present
|
|
in this directory.
|
|
Filename should be specified using the \f[C]\-\-exclude\-if\-present\f[]
|
|
flag.
|
|
This flag has a priority over the other filtering flags.
|
|
.PP
|
|
Imagine, you have the following directory structure:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
dir1/file1
|
|
dir1/dir2/file2
|
|
dir1/dir2/dir3/file3
|
|
dir1/dir2/dir3/.ignore
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You can exclude \f[C]dir3\f[] from sync by running the following
|
|
command:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ \-\-exclude\-if\-present\ .ignore\ dir1\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Currently only one filename is supported, i.e.
|
|
\f[C]\-\-exclude\-if\-present\f[] should not be used multiple times.
|
|
.SH GUI (Experimental)
|
|
.PP
|
|
Rclone can serve a web based GUI (graphical user interface).
|
|
This is somewhat experimental at the moment so things may be subject to
|
|
change.
|
|
.PP
|
|
Run this command in a terminal and rclone will download and then display
|
|
the GUI in a web browser.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rcd\ \-\-rc\-web\-gui
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will produce logs like this and rclone needs to continue to run to
|
|
serve the GUI:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2019/08/25\ 11:40:14\ NOTICE:\ A\ new\ release\ for\ gui\ is\ present\ at\ https://github.com/rclone/rclone\-webui\-react/releases/download/v0.0.6/currentbuild.zip
|
|
2019/08/25\ 11:40:14\ NOTICE:\ Downloading\ webgui\ binary.\ Please\ wait.\ [Size:\ 3813937,\ Path\ :\ \ /home/USER/.cache/rclone/webgui/v0.0.6.zip]
|
|
2019/08/25\ 11:40:16\ NOTICE:\ Unzipping
|
|
2019/08/25\ 11:40:16\ NOTICE:\ Serving\ remote\ control\ on\ http://127.0.0.1:5572/
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This assumes you are running rclone locally on your machine.
|
|
It is possible to separate the rclone and the GUI \- see below for
|
|
details.
|
|
.PP
|
|
If you wish to update to the latest API version then you can add
|
|
\f[C]\-\-rc\-web\-gui\-update\f[] to the command line.
|
|
.SS Using the GUI
|
|
.PP
|
|
Once the GUI opens, you will be looking at the dashboard which has an
|
|
overall overview.
|
|
.PP
|
|
On the left hand side you will see a series of view buttons you can
|
|
click on:
|
|
.IP \[bu] 2
|
|
Dashboard \- main overview
|
|
.IP \[bu] 2
|
|
Configs \- examine and create new configurations
|
|
.IP \[bu] 2
|
|
Explorer \- view, download and upload files to the cloud storage systems
|
|
.IP \[bu] 2
|
|
Backend \- view or alter the backend config
|
|
.IP \[bu] 2
|
|
Log out
|
|
.PP
|
|
(More docs and walkthrough video to come!)
|
|
.SS How it works
|
|
.PP
|
|
When you run the \f[C]rclone\ rcd\ \-\-rc\-web\-gui\f[] this is what
|
|
happens
|
|
.IP \[bu] 2
|
|
Rclone starts but only runs the remote control API (\[lq]rc\[rq]).
|
|
.IP \[bu] 2
|
|
The API is bound to localhost with an auto generated username and
|
|
password.
|
|
.IP \[bu] 2
|
|
If the API bundle is missing then rclone will download it.
|
|
.IP \[bu] 2
|
|
rclone will start serving the files from the API bundle over the same
|
|
port as the API
|
|
.IP \[bu] 2
|
|
rclone will open the browser with a \f[C]login_token\f[] so it can log
|
|
straight in.
|
|
.SS Advanced use
|
|
.PP
|
|
The \f[C]rclone\ rcd\f[] may use any of the flags documented on the rc
|
|
page (https://rclone.org/rc/#supported-parameters).
|
|
.PP
|
|
The flag \f[C]\-\-rc\-web\-gui\f[] is shorthand for
|
|
.IP \[bu] 2
|
|
Download the web GUI if necessary
|
|
.IP \[bu] 2
|
|
Check we are using some authentication
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-user\ gui\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-pass\ <random\ password>\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-serve\f[]
|
|
.PP
|
|
These flags can be overidden as desired.
|
|
.PP
|
|
See also the rclone rcd
|
|
documentation (https://rclone.org/commands/rclone_rcd/).
|
|
.SS Example: Running a public GUI
|
|
.PP
|
|
For example the GUI could be served on a public port over SSL using an
|
|
htpasswd file using the following flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-web\-gui\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-addr\ :443\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-htpasswd\ /path/to/htpasswd\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-cert\ /path/to/ssl.crt\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-key\ /path/to/ssl.key\f[]
|
|
.SS Example: Running a GUI behind a proxy
|
|
.PP
|
|
If you want to run the GUI behind a proxy at \f[C]/rclone\f[] you could
|
|
use these flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-web\-gui\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-baseurl\ rclone\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-htpasswd\ /path/to/htpasswd\f[]
|
|
.PP
|
|
Or instead of htpassword if you just want a single user and password:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-user\ me\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-pass\ mypassword\f[]
|
|
.SS Project
|
|
.PP
|
|
The GUI is being developed in the: rclone/rclone\-webui\-react
|
|
respository (https://github.com/rclone/rclone-webui-react).
|
|
.PP
|
|
Bug reports and contributions very welcome welcome :\-)
|
|
.PP
|
|
If you have questions then please ask them on the rclone
|
|
forum (https://forum.rclone.org/).
|
|
.SH Remote controlling rclone
|
|
.PP
|
|
If rclone is run with the \f[C]\-\-rc\f[] flag then it starts an http
|
|
server which can be used to remote control rclone.
|
|
.PP
|
|
If you just want to run a remote control then see the rcd
|
|
command (https://rclone.org/commands/rclone_rcd/).
|
|
.PP
|
|
\f[B]NB\f[] this is experimental and everything here is subject to
|
|
change!
|
|
.SS Supported parameters
|
|
.SS \[en]rc
|
|
.PP
|
|
Flag to start the http server listen on remote requests
|
|
.SS \[en]rc\-addr=IP
|
|
.PP
|
|
IPaddress:Port or :Port to bind server to.
|
|
(default \[lq]localhost:5572\[rq])
|
|
.SS \[en]rc\-cert=KEY
|
|
.PP
|
|
SSL PEM key (concatenation of certificate and CA certificate)
|
|
.SS \[en]rc\-client\-ca=PATH
|
|
.PP
|
|
Client certificate authority to verify clients with
|
|
.SS \[en]rc\-htpasswd=PATH
|
|
.PP
|
|
htpasswd file \- if not provided no authentication is done
|
|
.SS \[en]rc\-key=PATH
|
|
.PP
|
|
SSL PEM Private key
|
|
.SS \[en]rc\-max\-header\-bytes=VALUE
|
|
.PP
|
|
Maximum size of request header (default 4096)
|
|
.SS \[en]rc\-user=VALUE
|
|
.PP
|
|
User name for authentication.
|
|
.SS \[en]rc\-pass=VALUE
|
|
.PP
|
|
Password for authentication.
|
|
.SS \[en]rc\-realm=VALUE
|
|
.PP
|
|
Realm for authentication (default \[lq]rclone\[rq])
|
|
.SS \[en]rc\-server\-read\-timeout=DURATION
|
|
.PP
|
|
Timeout for server reading data (default 1h0m0s)
|
|
.SS \[en]rc\-server\-write\-timeout=DURATION
|
|
.PP
|
|
Timeout for server writing data (default 1h0m0s)
|
|
.SS \[en]rc\-serve
|
|
.PP
|
|
Enable the serving of remote objects via the HTTP interface.
|
|
This means objects will be accessible at http://127.0.0.1:5572/ by
|
|
default, so you can browse to http://127.0.0.1:5572/ or
|
|
http://127.0.0.1:5572/* to see a listing of the remotes.
|
|
Objects may be requested from remotes using this syntax
|
|
http://127.0.0.1:5572/[remote:path]/path/to/object
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-files /path/to/directory
|
|
.PP
|
|
Path to local files to serve on the HTTP server.
|
|
.PP
|
|
If this is set then rclone will serve the files in that directory.
|
|
It will also open the root in the web browser if specified.
|
|
This is for implementing browser based GUIs for rclone functions.
|
|
.PP
|
|
If \f[C]\-\-rc\-user\f[] or \f[C]\-\-rc\-pass\f[] is set then the URL
|
|
that is opened will have the authorization in the URL in the
|
|
\f[C]http://user:pass\@localhost/\f[] style.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-web\-gui
|
|
.PP
|
|
Set this flag to serve the default web gui on the same port as rclone.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-allow\-origin
|
|
.PP
|
|
Set the allowed Access\-Control\-Allow\-Origin for rc requests.
|
|
.PP
|
|
Can be used with \[en]rc\-web\-gui if the rclone is running on different
|
|
IP than the web\-gui.
|
|
.PP
|
|
Default is IP address on which rc is running.
|
|
.SS \[en]rc\-web\-fetch\-url
|
|
.PP
|
|
Set the URL to fetch the rclone\-web\-gui files from.
|
|
.PP
|
|
Default
|
|
https://api.github.com/repos/rclone/rclone\-webui\-react/releases/latest.
|
|
.SS \[en]rc\-web\-gui\-update
|
|
.PP
|
|
Set this flag to Download / Force update rclone\-webui\-react from the
|
|
rc\-web\-fetch\-url.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-job\-expire\-duration=DURATION
|
|
.PP
|
|
Expire finished async jobs older than DURATION (default 60s).
|
|
.SS \[en]rc\-job\-expire\-interval=DURATION
|
|
.PP
|
|
Interval duration to check for expired async jobs (default 10s).
|
|
.SS \[en]rc\-no\-auth
|
|
.PP
|
|
By default rclone will require authorisation to have been set up on the
|
|
rc interface in order to use any methods which access any rclone
|
|
remotes.
|
|
Eg \f[C]operations/list\f[] is denied as it involved creating a remote
|
|
as is \f[C]sync/copy\f[].
|
|
.PP
|
|
If this is set then no authorisation will be required on the server to
|
|
use these methods.
|
|
The alternative is to use \f[C]\-\-rc\-user\f[] and
|
|
\f[C]\-\-rc\-pass\f[] and use these credentials in the request.
|
|
.PP
|
|
Default Off.
|
|
.SS Accessing the remote control via the rclone rc command
|
|
.PP
|
|
Rclone itself implements the remote control protocol in its
|
|
\f[C]rclone\ rc\f[] command.
|
|
.PP
|
|
You can use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ rc\ rc/noop\ param1=one\ param2=two
|
|
{
|
|
\ \ \ \ "param1":\ "one",
|
|
\ \ \ \ "param2":\ "two"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone\ rc\f[] on its own to see the help for the installed
|
|
remote control commands.
|
|
.PP
|
|
\f[C]rclone\ rc\f[] also supports a \f[C]\-\-json\f[] flag which can be
|
|
used to send more complicated input parameters.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ rc\ \-\-json\ \[aq]{\ "p1":\ [1,"2",null,4],\ "p2":\ {\ "a":1,\ "b":2\ }\ }\[aq]\ rc/noop
|
|
{
|
|
\ \ \ \ "p1":\ [
|
|
\ \ \ \ \ \ \ \ 1,
|
|
\ \ \ \ \ \ \ \ "2",
|
|
\ \ \ \ \ \ \ \ null,
|
|
\ \ \ \ \ \ \ \ 4
|
|
\ \ \ \ ],
|
|
\ \ \ \ "p2":\ {
|
|
\ \ \ \ \ \ \ \ "a":\ 1,
|
|
\ \ \ \ \ \ \ \ "b":\ 2
|
|
\ \ \ \ }
|
|
}
|
|
\f[]
|
|
.fi
|
|
.SS Special parameters
|
|
.PP
|
|
The rc interface supports some special parameters which apply to
|
|
\f[B]all\f[] commands.
|
|
These start with \f[C]_\f[] to show they are different.
|
|
.SS Running asynchronous jobs with _async = true
|
|
.PP
|
|
Each rc call is classified as a job and it is assigned its own id.
|
|
By default jobs are executed immediately as they are created or
|
|
synchronously.
|
|
.PP
|
|
If \f[C]_async\f[] has a true value when supplied to an rc call then it
|
|
will return immediately with a job id and the task will be run in the
|
|
background.
|
|
The \f[C]job/status\f[] call can be used to get information of the
|
|
background job.
|
|
The job can be queried for up to 1 minute after it has finished.
|
|
.PP
|
|
It is recommended that potentially long running jobs, eg
|
|
\f[C]sync/sync\f[], \f[C]sync/copy\f[], \f[C]sync/move\f[],
|
|
\f[C]operations/purge\f[] are run with the \f[C]_async\f[] flag to avoid
|
|
any potential problems with the HTTP request and response timing out.
|
|
.PP
|
|
Starting a job with the \f[C]_async\f[] flag:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ rc\ \-\-json\ \[aq]{\ "p1":\ [1,"2",null,4],\ "p2":\ {\ "a":1,\ "b":2\ },\ "_async":\ true\ }\[aq]\ rc/noop
|
|
{
|
|
\ \ \ \ "jobid":\ 2
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Query the status to see if the job has finished.
|
|
For more information on the meaning of these return parameters see the
|
|
\f[C]job/status\f[] call.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ rc\ \-\-json\ \[aq]{\ "jobid":2\ }\[aq]\ job/status
|
|
{
|
|
\ \ \ \ "duration":\ 0.000124163,
|
|
\ \ \ \ "endTime":\ "2018\-10\-27T11:38:07.911245881+01:00",
|
|
\ \ \ \ "error":\ "",
|
|
\ \ \ \ "finished":\ true,
|
|
\ \ \ \ "id":\ 2,
|
|
\ \ \ \ "output":\ {
|
|
\ \ \ \ \ \ \ \ "_async":\ true,
|
|
\ \ \ \ \ \ \ \ "p1":\ [
|
|
\ \ \ \ \ \ \ \ \ \ \ \ 1,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ "2",
|
|
\ \ \ \ \ \ \ \ \ \ \ \ null,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ 4
|
|
\ \ \ \ \ \ \ \ ],
|
|
\ \ \ \ \ \ \ \ "p2":\ {
|
|
\ \ \ \ \ \ \ \ \ \ \ \ "a":\ 1,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ "b":\ 2
|
|
\ \ \ \ \ \ \ \ }
|
|
\ \ \ \ },
|
|
\ \ \ \ "startTime":\ "2018\-10\-27T11:38:07.911121728+01:00",
|
|
\ \ \ \ "success":\ true
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\f[C]job/list\f[] can be used to show the running or recently completed
|
|
jobs
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ rc\ job/list
|
|
{
|
|
\ \ \ \ "jobids":\ [
|
|
\ \ \ \ \ \ \ \ 2
|
|
\ \ \ \ ]
|
|
}
|
|
\f[]
|
|
.fi
|
|
.SS Assigning operations to groups with _group =
|
|
.PP
|
|
Each rc call has it's own stats group for tracking it's metrics.
|
|
By default grouping is done by the composite group name from prefix
|
|
\f[C]job/\f[] and id of the job like so \f[C]job/1\f[].
|
|
.PP
|
|
If \f[C]_group\f[] has a value then stats for that request will be
|
|
grouped under that value.
|
|
This allows caller to group stats under their own name.
|
|
.PP
|
|
Stats for specific group can be accessed by passing \f[C]group\f[] to
|
|
\f[C]core/stats\f[]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ rc\ \-\-json\ \[aq]{\ "group":\ "job/1"\ }\[aq]\ core/stats
|
|
{
|
|
\ \ \ \ "speed":\ 12345
|
|
\ \ \ \ ...
|
|
}
|
|
\f[]
|
|
.fi
|
|
.SS Supported commands
|
|
.SS cache/expire: Purge a remote from cache {#cache/expire}
|
|
.PP
|
|
Purge a remote from the cache backend.
|
|
Supports either a directory or a file.
|
|
Params: \- remote = path to remote (required) \- withData = true/false
|
|
to delete cached data (chunks) as well (optional)
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ cache/expire\ remote=path/to/sub/folder/
|
|
rclone\ rc\ cache/expire\ remote=/\ withData=true
|
|
\f[]
|
|
.fi
|
|
.SS cache/fetch: Fetch file chunks {#cache/fetch}
|
|
.PP
|
|
Ensure the specified file chunks are cached on disk.
|
|
.PP
|
|
The chunks= parameter specifies the file chunks to check.
|
|
It takes a comma separated list of array slice indices.
|
|
The slice indices are similar to Python slices: start[:end]
|
|
.PP
|
|
start is the 0 based chunk number from the beginning of the file to
|
|
fetch inclusive.
|
|
end is 0 based chunk number from the beginning of the file to fetch
|
|
exclusive.
|
|
Both values can be negative, in which case they count from the back of
|
|
the file.
|
|
The value \[lq]\-5:\[rq] represents the last 5 chunks of a file.
|
|
.PP
|
|
Some valid examples are: \[lq]:5,\-5:\[rq] \-> the first and last five
|
|
chunks \[lq]0,\-2\[rq] \-> the first and the second last chunk
|
|
\[lq]0:10\[rq] \-> the first ten chunks
|
|
.PP
|
|
Any parameter with a key that starts with \[lq]file\[rq] can be used to
|
|
specify files to fetch, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ cache/fetch\ chunks=0\ file=hello\ file2=home/goodbye
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
File names will automatically be encrypted when the a crypt remote is
|
|
used on top of the cache.
|
|
.SS cache/stats: Get cache stats {#cache/stats}
|
|
.PP
|
|
Show statistics for the cache remote.
|
|
.SS config/create: create the config for a remote. {#config/create}
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[lq]key\[rq]: \[lq]value\[rq] } pairs
|
|
.IP \[bu] 2
|
|
type \- type of the new remote
|
|
.PP
|
|
See the config create
|
|
command (https://rclone.org/commands/rclone_config_create/) command for
|
|
more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/delete: Delete a remote in the config file. {#config/delete}
|
|
.PP
|
|
Parameters:
|
|
.IP \[bu] 2
|
|
name \- name of remote to delete
|
|
.PP
|
|
See the config delete
|
|
command (https://rclone.org/commands/rclone_config_delete/) command for
|
|
more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/dump: Dumps the config file. {#config/dump}
|
|
.PP
|
|
Returns a JSON object: \- key: value
|
|
.PP
|
|
Where keys are remote names and values are the config parameters.
|
|
.PP
|
|
See the config dump
|
|
command (https://rclone.org/commands/rclone_config_dump/) command for
|
|
more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/get: Get a remote in the config file. {#config/get}
|
|
.PP
|
|
Parameters: \- name \- name of remote to get
|
|
.PP
|
|
See the config dump
|
|
command (https://rclone.org/commands/rclone_config_dump/) command for
|
|
more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/listremotes: Lists the remotes in the config file.
|
|
{#config/listremotes}
|
|
.PP
|
|
Returns \- remotes \- array of remote names
|
|
.PP
|
|
See the listremotes
|
|
command (https://rclone.org/commands/rclone_listremotes/) command for
|
|
more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/password: password the config for a remote.
|
|
{#config/password}
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[lq]key\[rq]: \[lq]value\[rq] } pairs
|
|
.PP
|
|
See the config password
|
|
command (https://rclone.org/commands/rclone_config_password/) command
|
|
for more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/providers: Shows how providers are configured in the config
|
|
file. {#config/providers}
|
|
.PP
|
|
Returns a JSON object: \- providers \- array of objects
|
|
.PP
|
|
See the config providers
|
|
command (https://rclone.org/commands/rclone_config_providers/) command
|
|
for more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS config/update: update the config for a remote. {#config/update}
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[lq]key\[rq]: \[lq]value\[rq] } pairs
|
|
.PP
|
|
See the config update
|
|
command (https://rclone.org/commands/rclone_config_update/) command for
|
|
more information on the above.
|
|
.PP
|
|
Authentication is required for this call.
|
|
.SS core/bwlimit: Set the bandwidth limit. {#core/bwlimit}
|
|
.PP
|
|
This sets the bandwidth limit to that passed in.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ core/bwlimit\ rate=off
|
|
{
|
|
\ \ \ \ "bytesPerSecond":\ \-1,
|
|
\ \ \ \ "rate":\ "off"
|
|
}
|
|
rclone\ rc\ core/bwlimit\ rate=1M
|
|
{
|
|
\ \ \ \ "bytesPerSecond":\ 1048576,
|
|
\ \ \ \ "rate":\ "1M"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If the rate parameter is not suppied then the bandwidth is queried
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ core/bwlimit
|
|
{
|
|
\ \ \ \ "bytesPerSecond":\ 1048576,
|
|
\ \ \ \ "rate":\ "1M"
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The format of the parameter is exactly the same as passed to
|
|
\[en]bwlimit except only one bandwidth may be specified.
|
|
.PP
|
|
In either case \[lq]rate\[rq] is returned as a human readable string,
|
|
and \[lq]bytesPerSecond\[rq] is returned as a number.
|
|
.SS core/gc: Runs a garbage collection. {#core/gc}
|
|
.PP
|
|
This tells the go runtime to do a garbage collection run.
|
|
It isn't necessary to call this normally, but it can be useful for
|
|
debugging memory problems.
|
|
.SS core/group\-list: Returns list of stats. {#core/group\-list}
|
|
.PP
|
|
This returns list of stats groups currently in memory.
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "groups":\ \ an\ array\ of\ group\ names:
|
|
\ \ \ \ \ \ \ \ [
|
|
\ \ \ \ \ \ \ \ \ \ \ \ "group1",
|
|
\ \ \ \ \ \ \ \ \ \ \ \ "group2",
|
|
\ \ \ \ \ \ \ \ \ \ \ \ ...
|
|
\ \ \ \ \ \ \ \ ]
|
|
}
|
|
\f[]
|
|
.fi
|
|
.SS core/memstats: Returns the memory statistics {#core/memstats}
|
|
.PP
|
|
This returns the memory statistics of the running program.
|
|
What the values mean are explained in the go docs:
|
|
https://golang.org/pkg/runtime/#MemStats
|
|
.PP
|
|
The most interesting values for most people are:
|
|
.IP \[bu] 2
|
|
HeapAlloc: This is the amount of memory rclone is actually using
|
|
.IP \[bu] 2
|
|
HeapSys: This is the amount of memory rclone has obtained from the OS
|
|
.IP \[bu] 2
|
|
Sys: this is the total amount of memory requested from the OS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
It is virtual memory so may include unused memory
|
|
.RE
|
|
.SS core/obscure: Obscures a string passed in. {#core/obscure}
|
|
.PP
|
|
Pass a clear string and rclone will obscure it for the config file: \-
|
|
clear \- string
|
|
.PP
|
|
Returns \- obscured \- string
|
|
.SS core/pid: Return PID of current process {#core/pid}
|
|
.PP
|
|
This returns PID of current process.
|
|
Useful for stopping rclone process.
|
|
.SS core/quit: Terminates the app. {#core/quit}
|
|
.PP
|
|
(optional) Pass an exit code to be used for terminating the app: \-
|
|
exitCode \- int
|
|
.SS core/stats: Returns stats about current transfers. {#core/stats}
|
|
.PP
|
|
This returns all available stats:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ rc\ core/stats
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If group is not provided then summed up stats for all groups will be
|
|
returned.
|
|
.PP
|
|
Parameters \- group \- name of the stats group (string)
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\ \ \ \ "speed":\ average\ speed\ in\ bytes/sec\ since\ start\ of\ the\ process,
|
|
\ \ \ \ "bytes":\ total\ transferred\ bytes\ since\ the\ start\ of\ the\ process,
|
|
\ \ \ \ "errors":\ number\ of\ errors,
|
|
\ \ \ \ "fatalError":\ whether\ there\ has\ been\ at\ least\ one\ FatalError,
|
|
\ \ \ \ "retryError":\ whether\ there\ has\ been\ at\ least\ one\ non\-NoRetryError,
|
|
\ \ \ \ "checks":\ number\ of\ checked\ files,
|
|
\ \ \ \ "transfers":\ number\ of\ transferred\ files,
|
|
\ \ \ \ "deletes"\ :\ number\ of\ deleted\ files,
|
|
\ \ \ \ "elapsedTime":\ time\ in\ seconds\ since\ the\ start\ of\ the\ process,
|
|
\ \ \ \ "lastError":\ last\ occurred\ error,
|
|
\ \ \ \ "transferring":\ an\ array\ of\ currently\ active\ file\ transfers:
|
|
\ \ \ \ \ \ \ \ [
|
|
\ \ \ \ \ \ \ \ \ \ \ \ {
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "bytes":\ total\ transferred\ bytes\ for\ this\ file,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "eta":\ estimated\ time\ in\ seconds\ until\ file\ transfer\ completion
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "name":\ name\ of\ the\ file,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "percentage":\ progress\ of\ the\ file\ transfer\ in\ percent,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "speed":\ speed\ in\ bytes/sec,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "speedAvg":\ speed\ in\ bytes/sec\ as\ an\ exponentially\ weighted\ moving\ average,
|
|
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "size":\ size\ of\ the\ file\ in\ bytes
|
|
\ \ \ \ \ \ \ \ \ \ \ \ }
|
|
\ \ \ \ \ \ \ \ ],
|
|
\ \ \ \ "checking":\ an\ array\ of\ names\ of\ currently\ active\ file\ checks
|
|
\ \ \ \ \ \ \ \ []
|
|
}
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Values for \[lq]transferring\[rq], \[lq]checking\[rq] and
|
|
\[lq]lastError\[rq] are only assigned if data is available.
|
|
The value for \[lq]eta\[rq] is null if an eta cannot be determined.
|
|
.SS core/stats\-reset: Reset stats. {#core/stats\-reset}
|
|
.PP
|
|
This clears counters and errors for all stats or specific stats group if
|
|
group is provided.
|
|
.PP
|
|
Parameters \- group \- name of the stats group (string)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ core/transferred:\ Returns\ stats\ about\ completed\ transfers.\ {#core/transferred}
|
|
|
|
This\ returns\ stats\ about\ completed\ transfers:
|
|
|
|
\ \ \ \ rclone\ rc\ core/transferred
|
|
|
|
If\ group\ is\ not\ provided\ then\ completed\ transfers\ for\ all\ groups\ will\ be
|
|
returned.
|
|
|
|
Note\ only\ the\ last\ 100\ completed\ transfers\ are\ returned.
|
|
|
|
Parameters
|
|
\-\ group\ \-\ name\ of\ the\ stats\ group\ (string)
|
|
|
|
Returns\ the\ following\ values:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]transferred\[rq]: an array of completed transfers (including
|
|
failed ones): [ { \[lq]name\[rq]: name of the file, \[lq]size\[rq]: size
|
|
of the file in bytes, \[lq]bytes\[rq]: total transferred bytes for this
|
|
file, \[lq]checked\[rq]: if the transfer is only checked (skipped,
|
|
deleted), \[lq]timestamp\[rq]: integer representing millisecond unix
|
|
epoch, \[lq]error\[rq]: string description of the error (empty if
|
|
successfull), \[lq]jobid\[rq]: id of the job that this transfer belongs
|
|
to } ] }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ core/version:\ Shows\ the\ current\ version\ of\ rclone\ and\ the\ go\ runtime.\ {#core/version}
|
|
|
|
This\ shows\ the\ current\ version\ of\ go\ and\ the\ go\ runtime
|
|
\-\ version\ \-\ rclone\ version,\ eg\ "v1.44"
|
|
\-\ decomposed\ \-\ version\ number\ as\ [major,\ minor,\ patch,\ subpatch]
|
|
\ \ \ \ \-\ note\ patch\ and\ subpatch\ will\ be\ 999\ for\ a\ git\ compiled\ version
|
|
\-\ isGit\ \-\ boolean\ \-\ true\ if\ this\ was\ compiled\ from\ the\ git\ version
|
|
\-\ os\ \-\ OS\ in\ use\ as\ according\ to\ Go
|
|
\-\ arch\ \-\ cpu\ architecture\ in\ use\ according\ to\ Go
|
|
\-\ goVersion\ \-\ version\ of\ Go\ runtime\ in\ use
|
|
|
|
###\ job/list:\ Lists\ the\ IDs\ of\ the\ running\ jobs\ {#job/list}
|
|
|
|
Parameters\ \-\ None
|
|
|
|
Results
|
|
\-\ jobids\ \-\ array\ of\ integer\ job\ ids
|
|
|
|
###\ job/status:\ Reads\ the\ status\ of\ the\ job\ ID\ {#job/status}
|
|
|
|
Parameters
|
|
\-\ jobid\ \-\ id\ of\ the\ job\ (integer)
|
|
|
|
Results
|
|
\-\ finished\ \-\ boolean
|
|
\-\ duration\ \-\ time\ in\ seconds\ that\ the\ job\ ran\ for
|
|
\-\ endTime\ \-\ time\ the\ job\ finished\ (eg\ "2018\-10\-26T18:50:20.528746884+01:00")
|
|
\-\ error\ \-\ error\ from\ the\ job\ or\ empty\ string\ for\ no\ error
|
|
\-\ finished\ \-\ boolean\ whether\ the\ job\ has\ finished\ or\ not
|
|
\-\ id\ \-\ as\ passed\ in\ above
|
|
\-\ startTime\ \-\ time\ the\ job\ started\ (eg\ "2018\-10\-26T18:50:20.528336039+01:00")
|
|
\-\ success\ \-\ boolean\ \-\ true\ for\ success\ false\ otherwise
|
|
\-\ output\ \-\ output\ of\ the\ job\ as\ would\ have\ been\ returned\ if\ called\ synchronously
|
|
\-\ progress\ \-\ output\ of\ the\ progress\ related\ to\ the\ underlying\ job
|
|
|
|
###\ job/stop:\ Stop\ the\ running\ job\ {#job/stop}
|
|
|
|
Parameters
|
|
\-\ jobid\ \-\ id\ of\ the\ job\ (integer)
|
|
|
|
###\ operations/about:\ Return\ the\ space\ used\ on\ the\ remote\ {#operations/about}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
|
|
The\ result\ is\ as\ returned\ from\ rclone\ about\ \-\-json
|
|
|
|
See\ the\ [about\ command](https://rclone.org/commands/rclone_size/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/cleanup:\ Remove\ trashed\ files\ in\ the\ remote\ or\ path\ {#operations/cleanup}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
|
|
See\ the\ [cleanup\ command](https://rclone.org/commands/rclone_cleanup/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/copyfile:\ Copy\ a\ file\ from\ source\ remote\ to\ destination\ remote\ {#operations/copyfile}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ srcFs\ \-\ a\ remote\ name\ string\ eg\ "drive:"\ for\ the\ source
|
|
\-\ srcRemote\ \-\ a\ path\ within\ that\ remote\ eg\ "file.txt"\ for\ the\ source
|
|
\-\ dstFs\ \-\ a\ remote\ name\ string\ eg\ "drive2:"\ for\ the\ destination
|
|
\-\ dstRemote\ \-\ a\ path\ within\ that\ remote\ eg\ "file2.txt"\ for\ the\ destination
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/copyurl:\ Copy\ the\ URL\ to\ the\ object\ {#operations/copyurl}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
\-\ url\ \-\ string,\ URL\ to\ read\ from
|
|
\ \-\ autoFilename\ \-\ boolean,\ set\ to\ true\ to\ retrieve\ destination\ file\ name\ from\ url
|
|
See\ the\ [copyurl\ command](https://rclone.org/commands/rclone_copyurl/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/delete:\ Remove\ files\ in\ the\ path\ {#operations/delete}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
|
|
See\ the\ [delete\ command](https://rclone.org/commands/rclone_delete/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/deletefile:\ Remove\ the\ single\ file\ pointed\ to\ {#operations/deletefile}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
|
|
See\ the\ [deletefile\ command](https://rclone.org/commands/rclone_deletefile/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/fsinfo:\ Return\ information\ about\ the\ remote\ {#operations/fsinfo}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
|
|
This\ returns\ info\ about\ the\ remote\ passed\ in;
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ // optional features and whether they are available or not
|
|
\[lq]Features\[rq]: { \[lq]About\[rq]: true, \[lq]BucketBased\[rq]:
|
|
false, \[lq]CanHaveEmptyDirectories\[rq]: true,
|
|
\[lq]CaseInsensitive\[rq]: false, \[lq]ChangeNotify\[rq]: false,
|
|
\[lq]CleanUp\[rq]: false, \[lq]Copy\[rq]: false,
|
|
\[lq]DirCacheFlush\[rq]: false, \[lq]DirMove\[rq]: true,
|
|
\[lq]DuplicateFiles\[rq]: false, \[lq]GetTier\[rq]: false,
|
|
\[lq]ListR\[rq]: false, \[lq]MergeDirs\[rq]: false, \[lq]Move\[rq]:
|
|
true, \[lq]OpenWriterAt\[rq]: true, \[lq]PublicLink\[rq]: false,
|
|
\[lq]Purge\[rq]: true, \[lq]PutStream\[rq]: true,
|
|
\[lq]PutUnchecked\[rq]: false, \[lq]ReadMimeType\[rq]: false,
|
|
\[lq]ServerSideAcrossConfigs\[rq]: false, \[lq]SetTier\[rq]: false,
|
|
\[lq]SetWrapper\[rq]: false, \[lq]UnWrap\[rq]: false, \[lq]WrapFs\[rq]:
|
|
false, \[lq]WriteMimeType\[rq]: false }, // Names of hashes available
|
|
\[lq]Hashes\[rq]: [ \[lq]MD5\[rq], \[lq]SHA\-1\[rq],
|
|
\[lq]DropboxHash\[rq], \[lq]QuickXorHash\[rq] ], \[lq]Name\[rq]:
|
|
\[lq]local\[rq], // Name as created \[lq]Precision\[rq]: 1, // Precision
|
|
of timestamps in ns \[lq]Root\[rq]: \[lq]/\[rq], // Path as created
|
|
\[lq]String\[rq]: \[lq]Local file system at /\[rq] // how the remote
|
|
will appear in logs }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
This\ command\ does\ not\ have\ a\ command\ line\ equivalent\ so\ use\ this\ instead:
|
|
|
|
\ \ \ \ rclone\ rc\ \-\-loopback\ operations/fsinfo\ fs=remote:
|
|
|
|
###\ operations/list:\ List\ the\ given\ remote\ and\ path\ in\ JSON\ format\ {#operations/list}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
\-\ opt\ \-\ a\ dictionary\ of\ options\ to\ control\ the\ listing\ (optional)
|
|
\ \ \ \ \-\ recurse\ \-\ If\ set\ recurse\ directories
|
|
\ \ \ \ \-\ noModTime\ \-\ If\ set\ return\ modification\ time
|
|
\ \ \ \ \-\ showEncrypted\ \-\ \ If\ set\ show\ decrypted\ names
|
|
\ \ \ \ \-\ showOrigIDs\ \-\ If\ set\ show\ the\ IDs\ for\ each\ item\ if\ known
|
|
\ \ \ \ \-\ showHash\ \-\ If\ set\ return\ a\ dictionary\ of\ hashes
|
|
|
|
The\ result\ is
|
|
|
|
\-\ list
|
|
\ \ \ \ \-\ This\ is\ an\ array\ of\ objects\ as\ described\ in\ the\ lsjson\ command
|
|
|
|
See\ the\ [lsjson\ command](https://rclone.org/commands/rclone_lsjson/)\ for\ more\ information\ on\ the\ above\ and\ examples.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/mkdir:\ Make\ a\ destination\ directory\ or\ container\ {#operations/mkdir}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
|
|
See\ the\ [mkdir\ command](https://rclone.org/commands/rclone_mkdir/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/movefile:\ Move\ a\ file\ from\ source\ remote\ to\ destination\ remote\ {#operations/movefile}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ srcFs\ \-\ a\ remote\ name\ string\ eg\ "drive:"\ for\ the\ source
|
|
\-\ srcRemote\ \-\ a\ path\ within\ that\ remote\ eg\ "file.txt"\ for\ the\ source
|
|
\-\ dstFs\ \-\ a\ remote\ name\ string\ eg\ "drive2:"\ for\ the\ destination
|
|
\-\ dstRemote\ \-\ a\ path\ within\ that\ remote\ eg\ "file2.txt"\ for\ the\ destination
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/publiclink:\ Create\ or\ retrieve\ a\ public\ link\ to\ the\ given\ file\ or\ folder.\ {#operations/publiclink}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
|
|
Returns
|
|
|
|
\-\ url\ \-\ URL\ of\ the\ resource
|
|
|
|
See\ the\ [link\ command](https://rclone.org/commands/rclone_link/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/purge:\ Remove\ a\ directory\ or\ container\ and\ all\ of\ its\ contents\ {#operations/purge}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
|
|
See\ the\ [purge\ command](https://rclone.org/commands/rclone_purge/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/rmdir:\ Remove\ an\ empty\ directory\ or\ container\ {#operations/rmdir}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
|
|
See\ the\ [rmdir\ command](https://rclone.org/commands/rclone_rmdir/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/rmdirs:\ Remove\ all\ the\ empty\ directories\ in\ the\ path\ {#operations/rmdirs}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:"
|
|
\-\ remote\ \-\ a\ path\ within\ that\ remote\ eg\ "dir"
|
|
\-\ leaveRoot\ \-\ boolean,\ set\ to\ true\ not\ to\ delete\ the\ root
|
|
|
|
See\ the\ [rmdirs\ command](https://rclone.org/commands/rclone_rmdirs/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ operations/size:\ Count\ the\ number\ of\ bytes\ and\ files\ in\ remote\ {#operations/size}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ fs\ \-\ a\ remote\ name\ string\ eg\ "drive:path/to/dir"
|
|
|
|
Returns
|
|
|
|
\-\ count\ \-\ number\ of\ files
|
|
\-\ bytes\ \-\ number\ of\ bytes\ in\ those\ files
|
|
|
|
See\ the\ [size\ command](https://rclone.org/commands/rclone_size/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ options/blocks:\ List\ all\ the\ option\ blocks\ {#options/blocks}
|
|
|
|
Returns
|
|
\-\ options\ \-\ a\ list\ of\ the\ options\ block\ names
|
|
|
|
###\ options/get:\ Get\ all\ the\ options\ {#options/get}
|
|
|
|
Returns\ an\ object\ where\ keys\ are\ option\ block\ names\ and\ values\ are\ an
|
|
object\ with\ the\ current\ option\ values\ in.
|
|
|
|
This\ shows\ the\ internal\ names\ of\ the\ option\ within\ rclone\ which\ should
|
|
map\ to\ the\ external\ options\ very\ easily\ with\ a\ few\ exceptions.
|
|
|
|
###\ options/set:\ Set\ an\ option\ {#options/set}
|
|
|
|
Parameters
|
|
|
|
\-\ option\ block\ name\ containing\ an\ object\ with
|
|
\ \ \-\ key:\ value
|
|
|
|
Repeated\ as\ often\ as\ required.
|
|
|
|
Only\ supply\ the\ options\ you\ wish\ to\ change.\ \ If\ an\ option\ is\ unknown
|
|
it\ will\ be\ silently\ ignored.\ \ Not\ all\ options\ will\ have\ an\ effect\ when
|
|
changed\ like\ this.
|
|
|
|
For\ example:
|
|
|
|
This\ sets\ DEBUG\ level\ logs\ (\-vv)
|
|
|
|
\ \ \ \ rclone\ rc\ options/set\ \-\-json\ \[aq]{"main":\ {"LogLevel":\ 8}}\[aq]
|
|
|
|
And\ this\ sets\ INFO\ level\ logs\ (\-v)
|
|
|
|
\ \ \ \ rclone\ rc\ options/set\ \-\-json\ \[aq]{"main":\ {"LogLevel":\ 7}}\[aq]
|
|
|
|
And\ this\ sets\ NOTICE\ level\ logs\ (normal\ without\ \-v)
|
|
|
|
\ \ \ \ rclone\ rc\ options/set\ \-\-json\ \[aq]{"main":\ {"LogLevel":\ 6}}\[aq]
|
|
|
|
###\ rc/error:\ This\ returns\ an\ error\ {#rc/error}
|
|
|
|
This\ returns\ an\ error\ with\ the\ input\ as\ part\ of\ its\ error\ string.
|
|
Useful\ for\ testing\ error\ handling.
|
|
|
|
###\ rc/list:\ List\ all\ the\ registered\ remote\ control\ commands\ {#rc/list}
|
|
|
|
This\ lists\ all\ the\ registered\ remote\ control\ commands\ as\ a\ JSON\ map\ in
|
|
the\ commands\ response.
|
|
|
|
###\ rc/noop:\ Echo\ the\ input\ to\ the\ output\ parameters\ {#rc/noop}
|
|
|
|
This\ echoes\ the\ input\ parameters\ to\ the\ output\ parameters\ for\ testing
|
|
purposes.\ \ It\ can\ be\ used\ to\ check\ that\ rclone\ is\ still\ alive\ and\ to
|
|
check\ that\ parameter\ passing\ is\ working\ properly.
|
|
|
|
###\ rc/noopauth:\ Echo\ the\ input\ to\ the\ output\ parameters\ requiring\ auth\ {#rc/noopauth}
|
|
|
|
This\ echoes\ the\ input\ parameters\ to\ the\ output\ parameters\ for\ testing
|
|
purposes.\ \ It\ can\ be\ used\ to\ check\ that\ rclone\ is\ still\ alive\ and\ to
|
|
check\ that\ parameter\ passing\ is\ working\ properly.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ sync/copy:\ copy\ a\ directory\ from\ source\ remote\ to\ destination\ remote\ {#sync/copy}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ srcFs\ \-\ a\ remote\ name\ string\ eg\ "drive:src"\ for\ the\ source
|
|
\-\ dstFs\ \-\ a\ remote\ name\ string\ eg\ "drive:dst"\ for\ the\ destination
|
|
|
|
|
|
See\ the\ [copy\ command](https://rclone.org/commands/rclone_copy/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ sync/move:\ move\ a\ directory\ from\ source\ remote\ to\ destination\ remote\ {#sync/move}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ srcFs\ \-\ a\ remote\ name\ string\ eg\ "drive:src"\ for\ the\ source
|
|
\-\ dstFs\ \-\ a\ remote\ name\ string\ eg\ "drive:dst"\ for\ the\ destination
|
|
\-\ deleteEmptySrcDirs\ \-\ delete\ empty\ src\ directories\ if\ set
|
|
|
|
|
|
See\ the\ [move\ command](https://rclone.org/commands/rclone_move/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ sync/sync:\ sync\ a\ directory\ from\ source\ remote\ to\ destination\ remote\ {#sync/sync}
|
|
|
|
This\ takes\ the\ following\ parameters
|
|
|
|
\-\ srcFs\ \-\ a\ remote\ name\ string\ eg\ "drive:src"\ for\ the\ source
|
|
\-\ dstFs\ \-\ a\ remote\ name\ string\ eg\ "drive:dst"\ for\ the\ destination
|
|
|
|
|
|
See\ the\ [sync\ command](https://rclone.org/commands/rclone_sync/)\ command\ for\ more\ information\ on\ the\ above.
|
|
|
|
Authentication\ is\ required\ for\ this\ call.
|
|
|
|
###\ vfs/forget:\ Forget\ files\ or\ directories\ in\ the\ directory\ cache.\ {#vfs/forget}
|
|
|
|
This\ forgets\ the\ paths\ in\ the\ directory\ cache\ causing\ them\ to\ be
|
|
re\-read\ from\ the\ remote\ when\ needed.
|
|
|
|
If\ no\ paths\ are\ passed\ in\ then\ it\ will\ forget\ all\ the\ paths\ in\ the
|
|
directory\ cache.
|
|
|
|
\ \ \ \ rclone\ rc\ vfs/forget
|
|
|
|
Otherwise\ pass\ files\ or\ dirs\ in\ as\ file=path\ or\ dir=path.\ \ Any
|
|
parameter\ key\ starting\ with\ file\ will\ forget\ that\ file\ and\ any
|
|
starting\ with\ dir\ will\ forget\ that\ dir,\ eg
|
|
|
|
\ \ \ \ rclone\ rc\ vfs/forget\ file=hello\ file2=goodbye\ dir=home/junk
|
|
|
|
###\ vfs/poll\-interval:\ Get\ the\ status\ or\ update\ the\ value\ of\ the\ poll\-interval\ option.\ {#vfs/poll\-interval}
|
|
|
|
Without\ any\ parameter\ given\ this\ returns\ the\ current\ status\ of\ the
|
|
poll\-interval\ setting.
|
|
|
|
When\ the\ interval=duration\ parameter\ is\ set,\ the\ poll\-interval\ value
|
|
is\ updated\ and\ the\ polling\ function\ is\ notified.
|
|
Setting\ interval=0\ disables\ poll\-interval.
|
|
|
|
\ \ \ \ rclone\ rc\ vfs/poll\-interval\ interval=5m
|
|
|
|
The\ timeout=duration\ parameter\ can\ be\ used\ to\ specify\ a\ time\ to\ wait
|
|
for\ the\ current\ poll\ function\ to\ apply\ the\ new\ value.
|
|
If\ timeout\ is\ less\ or\ equal\ 0,\ which\ is\ the\ default,\ wait\ indefinitely.
|
|
|
|
The\ new\ poll\-interval\ value\ will\ only\ be\ active\ when\ the\ timeout\ is
|
|
not\ reached.
|
|
|
|
If\ poll\-interval\ is\ updated\ or\ disabled\ temporarily,\ some\ changes
|
|
might\ not\ get\ picked\ up\ by\ the\ polling\ function,\ depending\ on\ the
|
|
used\ remote.
|
|
|
|
###\ vfs/refresh:\ Refresh\ the\ directory\ cache.\ {#vfs/refresh}
|
|
|
|
This\ reads\ the\ directories\ for\ the\ specified\ paths\ and\ freshens\ the
|
|
directory\ cache.
|
|
|
|
If\ no\ paths\ are\ passed\ in\ then\ it\ will\ refresh\ the\ root\ directory.
|
|
|
|
\ \ \ \ rclone\ rc\ vfs/refresh
|
|
|
|
Otherwise\ pass\ directories\ in\ as\ dir=path.\ Any\ parameter\ key
|
|
starting\ with\ dir\ will\ refresh\ that\ directory,\ eg
|
|
|
|
\ \ \ \ rclone\ rc\ vfs/refresh\ dir=home/junk\ dir2=data/misc
|
|
|
|
If\ the\ parameter\ recursive=true\ is\ given\ the\ whole\ directory\ tree
|
|
will\ get\ refreshed.\ This\ refresh\ will\ use\ \-\-fast\-list\ if\ enabled.
|
|
|
|
<!\-\-\-\ autogenerated\ stop\ \-\->
|
|
|
|
##\ Accessing\ the\ remote\ control\ via\ HTTP
|
|
|
|
Rclone\ implements\ a\ simple\ HTTP\ based\ protocol.
|
|
|
|
Each\ endpoint\ takes\ an\ JSON\ object\ and\ returns\ a\ JSON\ object\ or\ an
|
|
error.\ \ The\ JSON\ objects\ are\ essentially\ a\ map\ of\ string\ names\ to
|
|
values.
|
|
|
|
All\ calls\ must\ made\ using\ POST.
|
|
|
|
The\ input\ objects\ can\ be\ supplied\ using\ URL\ parameters,\ POST
|
|
parameters\ or\ by\ supplying\ "Content\-Type:\ application/json"\ and\ a\ JSON
|
|
blob\ in\ the\ body.\ \ There\ are\ examples\ of\ these\ below\ using\ `curl`.
|
|
|
|
The\ response\ will\ be\ a\ JSON\ blob\ in\ the\ body\ of\ the\ response.\ \ This\ is
|
|
formatted\ to\ be\ reasonably\ human\ readable.
|
|
|
|
###\ Error\ returns
|
|
|
|
If\ an\ error\ occurs\ then\ there\ will\ be\ an\ HTTP\ error\ status\ (eg\ 500)
|
|
and\ the\ body\ of\ the\ response\ will\ contain\ a\ JSON\ encoded\ error\ object,
|
|
eg
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]error\[rq]: \[lq]Expecting string value for key "remote" (was
|
|
float64)\[rq], \[lq]input\[rq]: { \[lq]fs\[rq]: \[lq]/tmp\[rq],
|
|
\[lq]remote\[rq]: 3 }, \[lq]status\[rq]: 400 \[lq]path\[rq]:
|
|
\[lq]operations/rmdir\[rq], }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
The\ keys\ in\ the\ error\ response\ are
|
|
\-\ error\ \-\ error\ string
|
|
\-\ input\ \-\ the\ input\ parameters\ to\ the\ call
|
|
\-\ status\ \-\ the\ HTTP\ status\ code
|
|
\-\ path\ \-\ the\ path\ of\ the\ call
|
|
|
|
###\ CORS
|
|
|
|
The\ sever\ implements\ basic\ CORS\ support\ and\ allows\ all\ origins\ for\ that.
|
|
The\ response\ to\ a\ preflight\ OPTIONS\ request\ will\ echo\ the\ requested\ "Access\-Control\-Request\-Headers"\ back.
|
|
|
|
###\ Using\ POST\ with\ URL\ parameters\ only
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
curl \-X POST `http://localhost:5572/rc/noop?potato=1&sausage=2'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Response
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]potato\[rq]: \[lq]1\[rq], \[lq]sausage\[rq]: \[lq]2\[rq] }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Here\ is\ what\ an\ error\ response\ looks\ like:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
curl \-X POST `http://localhost:5572/rc/error?potato=1&sausage=2'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]error\[rq]: \[lq]arbitrary error on input map[potato:1
|
|
sausage:2]\[rq], \[lq]input\[rq]: { \[lq]potato\[rq]: \[lq]1\[rq],
|
|
\[lq]sausage\[rq]: \[lq]2\[rq] } }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Note\ that\ curl\ doesn\[aq]t\ return\ errors\ to\ the\ shell\ unless\ you\ use\ the\ `\-f`\ option
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ curl \-f \-X POST `http://localhost:5572/rc/error?potato=1&sausage=2'
|
|
curl: (22) The requested URL returned error: 400 Bad Request $ echo $?
|
|
22
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Using\ POST\ with\ a\ form
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
curl \[en]data \[lq]potato=1\[rq] \[en]data \[lq]sausage=2\[rq]
|
|
http://localhost:5572/rc/noop
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Response
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]potato\[rq]: \[lq]1\[rq], \[lq]sausage\[rq]: \[lq]2\[rq] }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Note\ that\ you\ can\ combine\ these\ with\ URL\ parameters\ too\ with\ the\ POST
|
|
parameters\ taking\ precedence.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
curl \[en]data \[lq]potato=1\[rq] \[en]data \[lq]sausage=2\[rq]
|
|
\[lq]http://localhost:5572/rc/noop?rutabaga=3&sausage=4\[rq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Response
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]potato\[rq]: \[lq]1\[rq], \[lq]rutabaga\[rq]: \[lq]3\[rq],
|
|
\[lq]sausage\[rq]: \[lq]4\[rq] }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Using\ POST\ with\ a\ JSON\ blob
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
curl \-H \[lq]Content\-Type: application/json\[rq] \-X POST \-d
|
|
`{\[lq]potato\[rq]:2,\[lq]sausage\[rq]:1}' http://localhost:5572/rc/noop
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
response
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]password\[rq]: \[lq]xyz\[rq], \[lq]username\[rq]: \[lq]xyz\[rq] }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
This\ can\ be\ combined\ with\ URL\ parameters\ too\ if\ required.\ \ The\ JSON
|
|
blob\ takes\ precedence.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
curl \-H \[lq]Content\-Type: application/json\[rq] \-X POST \-d
|
|
`{\[lq]potato\[rq]:2,\[lq]sausage\[rq]:1}'
|
|
`http://localhost:5572/rc/noop?rutabaga=3&potato=4'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]potato\[rq]: 2, \[lq]rutabaga\[rq]: \[lq]3\[rq],
|
|
\[lq]sausage\[rq]: 1 }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
##\ Debugging\ rclone\ with\ pprof\ ##
|
|
|
|
If\ you\ use\ the\ `\-\-rc`\ flag\ this\ will\ also\ enable\ the\ use\ of\ the\ go
|
|
profiling\ tools\ on\ the\ same\ port.
|
|
|
|
To\ use\ these,\ first\ [install\ go](https://golang.org/doc/install).
|
|
|
|
###\ Debugging\ memory\ use
|
|
|
|
To\ profile\ rclone\[aq]s\ memory\ use\ you\ can\ run:
|
|
|
|
\ \ \ \ go\ tool\ pprof\ \-web\ http://localhost:5572/debug/pprof/heap
|
|
|
|
This\ should\ open\ a\ page\ in\ your\ browser\ showing\ what\ is\ using\ what
|
|
memory.
|
|
|
|
You\ can\ also\ use\ the\ `\-text`\ flag\ to\ produce\ a\ textual\ summary
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ go tool pprof \-text http://localhost:5572/debug/pprof/heap Showing
|
|
nodes accounting for 1537.03kB, 100% of 1537.03kB total flat flat% sum%
|
|
cum cum% 1024.03kB 66.62% 66.62% 1024.03kB 66.62%
|
|
github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
|
|
513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize 0 0% 100%
|
|
1024.03kB 66.62% github.com/rclone/rclone/cmd/all.init 0 0% 100%
|
|
1024.03kB 66.62% github.com/rclone/rclone/cmd/serve.init 0 0% 100%
|
|
1024.03kB 66.62% github.com/rclone/rclone/cmd/serve/restic.init 0 0%
|
|
100% 1024.03kB 66.62%
|
|
github.com/rclone/rclone/vendor/golang.org/x/net/http2.init 0 0% 100%
|
|
1024.03kB 66.62%
|
|
github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init 0 0%
|
|
100% 1024.03kB 66.62%
|
|
github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init.0 0 0%
|
|
100% 1024.03kB 66.62% main.init 0 0% 100% 513kB 33.38%
|
|
net/http.(\f[I]conn).readRequest 0 0% 100% 513kB 33.38%
|
|
net/http.(\f[]conn).serve 0 0% 100% 1024.03kB 66.62% runtime.main
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Debugging\ go\ routine\ leaks
|
|
|
|
Memory\ leaks\ are\ most\ often\ caused\ by\ go\ routine\ leaks\ keeping\ memory
|
|
alive\ which\ should\ have\ been\ garbage\ collected.
|
|
|
|
See\ all\ active\ go\ routines\ using
|
|
|
|
\ \ \ \ curl\ http://localhost:5572/debug/pprof/goroutine?debug=1
|
|
|
|
Or\ go\ to\ http://localhost:5572/debug/pprof/goroutine?debug=1\ in\ your\ browser.
|
|
|
|
###\ Other\ profiles\ to\ look\ at
|
|
|
|
You\ can\ see\ a\ summary\ of\ profiles\ available\ at\ http://localhost:5572/debug/pprof/
|
|
|
|
Here\ is\ how\ to\ use\ some\ of\ them:
|
|
|
|
\ \ *\ Memory:\ `go\ tool\ pprof\ http://localhost:5572/debug/pprof/heap`
|
|
\ \ *\ Go\ routines:\ `curl\ http://localhost:5572/debug/pprof/goroutine?debug=1`
|
|
\ \ *\ 30\-second\ CPU\ profile:\ `go\ tool\ pprof\ http://localhost:5572/debug/pprof/profile`
|
|
\ \ *\ 5\-second\ execution\ trace:\ `wget\ http://localhost:5572/debug/pprof/trace?seconds=5`
|
|
|
|
See\ the\ [net/http/pprof\ docs](https://golang.org/pkg/net/http/pprof/)
|
|
for\ more\ info\ on\ how\ to\ use\ the\ profiling\ and\ for\ a\ general\ overview
|
|
see\ [the\ Go\ team\[aq]s\ blog\ post\ on\ profiling\ go\ programs](https://blog.golang.org/profiling\-go\-programs).
|
|
|
|
The\ profiling\ hook\ is\ [zero\ overhead\ unless\ it\ is\ used](https://stackoverflow.com/q/26545159/164234).
|
|
|
|
#\ Overview\ of\ cloud\ storage\ systems\ #
|
|
|
|
Each\ cloud\ storage\ system\ is\ slightly\ different.\ \ Rclone\ attempts\ to
|
|
provide\ a\ unified\ interface\ to\ them,\ but\ some\ underlying\ differences
|
|
show\ through.
|
|
|
|
##\ Features\ ##
|
|
|
|
Here\ is\ an\ overview\ of\ the\ major\ features\ of\ each\ cloud\ storage\ system.
|
|
|
|
|\ Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Hash\ \ \ \ \ \ \ \ |\ ModTime\ |\ Case\ Insensitive\ |\ Duplicate\ Files\ |\ MIME\ Type\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-\-\-\-\-\-\-:|:\-\-\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-:|
|
|
|\ 1Fichier\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Whirlpool\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ Amazon\ Drive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ Amazon\ S3\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Backblaze\ B2\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ SHA1\ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Box\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ SHA1\ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Citrix\ ShareFile\ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Dropbox\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ DBHASH\ †\ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ FTP\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Google\ Cloud\ Storage\ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Google\ Drive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Google\ Photos\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ HTTP\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ Hubic\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Jottacloud\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Koofr\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Mail.ru\ Cloud\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Mailru\ ‡‡‡\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Mega\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Microsoft\ Azure\ Blob\ Storage\ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ Microsoft\ OneDrive\ \ \ \ \ \ \ \ \ \ \ |\ SHA1\ ‡‡\ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ OpenDrive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Openstack\ Swift\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ pCloud\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5,\ SHA1\ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ W\ \ \ \ \ \ \ \ \ |
|
|
|\ premiumize.me\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ put.io\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ CRC\-32\ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ |\ R\ \ \ \ \ \ \ \ \ |
|
|
|\ QingStor\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ SFTP\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5,\ SHA1\ ‡\ |\ Yes\ \ \ \ \ |\ Depends\ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ WebDAV\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5,\ SHA1\ ††|\ Yes\ †††\ |\ Depends\ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|\ Yandex\ Disk\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ MD5\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ R/W\ \ \ \ \ \ \ |
|
|
|\ The\ local\ filesystem\ \ \ \ \ \ \ \ \ |\ All\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ Depends\ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \-\ \ \ \ \ \ \ \ \ |
|
|
|
|
###\ Hash\ ###
|
|
|
|
The\ cloud\ storage\ system\ supports\ various\ hash\ types\ of\ the\ objects.
|
|
The\ hashes\ are\ used\ when\ transferring\ data\ as\ an\ integrity\ check\ and
|
|
can\ be\ specifically\ used\ with\ the\ `\-\-checksum`\ flag\ in\ syncs\ and\ in
|
|
the\ `check`\ command.
|
|
|
|
To\ use\ the\ verify\ checksums\ when\ transferring\ between\ cloud\ storage
|
|
systems\ they\ must\ support\ a\ common\ hash\ type.
|
|
|
|
†\ Note\ that\ Dropbox\ supports\ [its\ own\ custom
|
|
hash](https://www.dropbox.com/developers/reference/content\-hash).
|
|
This\ is\ an\ SHA256\ sum\ of\ all\ the\ 4MB\ block\ SHA256s.
|
|
|
|
‡\ SFTP\ supports\ checksums\ if\ the\ same\ login\ has\ shell\ access\ and\ `md5sum`
|
|
or\ `sha1sum`\ as\ well\ as\ `echo`\ are\ in\ the\ remote\[aq]s\ PATH.
|
|
|
|
††\ WebDAV\ supports\ hashes\ when\ used\ with\ Owncloud\ and\ Nextcloud\ only.
|
|
|
|
†††\ WebDAV\ supports\ modtimes\ when\ used\ with\ Owncloud\ and\ Nextcloud\ only.
|
|
|
|
‡‡\ Microsoft\ OneDrive\ Personal\ supports\ SHA1\ hashes,\ whereas\ OneDrive
|
|
for\ business\ and\ SharePoint\ server\ support\ Microsoft\[aq]s\ own
|
|
[QuickXorHash](https://docs.microsoft.com/en\-us/onedrive/developer/code\-snippets/quickxorhash).
|
|
|
|
‡‡‡\ Mail.ru\ uses\ its\ own\ modified\ SHA1\ hash
|
|
|
|
###\ ModTime\ ###
|
|
|
|
The\ cloud\ storage\ system\ supports\ setting\ modification\ times\ on
|
|
objects.\ \ If\ it\ does\ then\ this\ enables\ a\ using\ the\ modification\ times
|
|
as\ part\ of\ the\ sync.\ \ If\ not\ then\ only\ the\ size\ will\ be\ checked\ by
|
|
default,\ though\ the\ MD5SUM\ can\ be\ checked\ with\ the\ `\-\-checksum`\ flag.
|
|
|
|
All\ cloud\ storage\ systems\ support\ some\ kind\ of\ date\ on\ the\ object\ and
|
|
these\ will\ be\ set\ when\ transferring\ from\ the\ cloud\ storage\ system.
|
|
|
|
###\ Case\ Insensitive\ ###
|
|
|
|
If\ a\ cloud\ storage\ systems\ is\ case\ sensitive\ then\ it\ is\ possible\ to
|
|
have\ two\ files\ which\ differ\ only\ in\ case,\ eg\ `file.txt`\ and
|
|
`FILE.txt`.\ \ If\ a\ cloud\ storage\ system\ is\ case\ insensitive\ then\ that
|
|
isn\[aq]t\ possible.
|
|
|
|
This\ can\ cause\ problems\ when\ syncing\ between\ a\ case\ insensitive
|
|
system\ and\ a\ case\ sensitive\ system.\ \ The\ symptom\ of\ this\ is\ that\ no
|
|
matter\ how\ many\ times\ you\ run\ the\ sync\ it\ never\ completes\ fully.
|
|
|
|
The\ local\ filesystem\ and\ SFTP\ may\ or\ may\ not\ be\ case\ sensitive
|
|
depending\ on\ OS.
|
|
|
|
\ \ *\ Windows\ \-\ usually\ case\ insensitive,\ though\ case\ is\ preserved
|
|
\ \ *\ OSX\ \-\ usually\ case\ insensitive,\ though\ it\ is\ possible\ to\ format\ case\ sensitive
|
|
\ \ *\ Linux\ \-\ usually\ case\ sensitive,\ but\ there\ are\ case\ insensitive\ file\ systems\ (eg\ FAT\ formatted\ USB\ keys)
|
|
|
|
Most\ of\ the\ time\ this\ doesn\[aq]t\ cause\ any\ problems\ as\ people\ tend\ to
|
|
avoid\ files\ whose\ name\ differs\ only\ by\ case\ even\ on\ case\ sensitive
|
|
systems.
|
|
|
|
###\ Duplicate\ files\ ###
|
|
|
|
If\ a\ cloud\ storage\ system\ allows\ duplicate\ files\ then\ it\ can\ have\ two
|
|
objects\ with\ the\ same\ name.
|
|
|
|
This\ confuses\ rclone\ greatly\ when\ syncing\ \-\ use\ the\ `rclone\ dedupe`
|
|
command\ to\ rename\ or\ remove\ duplicates.
|
|
|
|
###\ Restricted\ filenames\ ###
|
|
|
|
Some\ cloud\ storage\ systems\ might\ have\ restrictions\ on\ the\ characters
|
|
that\ are\ usable\ in\ file\ or\ directory\ names.
|
|
When\ `rclone`\ detects\ such\ a\ name\ during\ a\ file\ upload,\ it\ will
|
|
transparently\ replace\ the\ restricted\ characters\ with\ similar\ looking
|
|
Unicode\ characters.
|
|
|
|
This\ process\ is\ designed\ to\ avoid\ ambiguous\ file\ names\ as\ much\ as
|
|
possible\ and\ allow\ to\ move\ files\ between\ many\ cloud\ storage\ systems
|
|
transparently.
|
|
|
|
The\ name\ shown\ by\ `rclone`\ to\ the\ user\ or\ during\ log\ output\ will\ only
|
|
contain\ a\ minimal\ set\ of\ [replaced\ characters](#restricted\-characters)
|
|
to\ ensure\ correct\ formatting\ and\ not\ necessarily\ the\ actual\ name\ used
|
|
on\ the\ cloud\ storage.
|
|
|
|
This\ transformation\ is\ reversed\ when\ downloading\ a\ file\ or\ parsing
|
|
`rclone`\ arguments.
|
|
For\ example,\ when\ uploading\ a\ file\ named\ `my\ file?.txt`\ to\ Onedrive
|
|
will\ be\ displayed\ as\ `my\ file?.txt`\ on\ the\ console,\ but\ stored\ as
|
|
`my\ file？.txt`\ (the\ `?`\ gets\ replaced\ by\ the\ similar\ looking\ `？`
|
|
character)\ to\ Onedrive.
|
|
The\ reverse\ transformation\ allows\ to\ read\ a\ file`unusual/name.txt`
|
|
from\ Google\ Drive,\ by\ passing\ the\ name\ `unusual／name.txt`\ (the\ `/`\ needs
|
|
to\ be\ replaced\ by\ the\ similar\ looking\ `／`\ character)\ on\ the\ command\ line.
|
|
|
|
####\ Default\ restricted\ characters\ {#restricted\-characters}
|
|
|
|
The\ table\ below\ shows\ the\ characters\ that\ are\ replaced\ by\ default.
|
|
|
|
When\ a\ replacement\ character\ is\ found\ in\ a\ filename,\ this\ character
|
|
will\ be\ escaped\ with\ the\ `‛`\ character\ to\ avoid\ ambiguous\ file\ names.
|
|
(e.g.\ a\ file\ named\ `␀.txt`\ would\ shown\ as\ `‛␀.txt`)
|
|
|
|
Each\ cloud\ storage\ backend\ can\ use\ a\ different\ set\ of\ characters,
|
|
which\ will\ be\ specified\ in\ the\ documentation\ for\ each\ backend.
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ NUL\ \ \ \ \ \ \ |\ 0x00\ \ |\ ␀\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ SOH\ \ \ \ \ \ \ |\ 0x01\ \ |\ ␁\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ STX\ \ \ \ \ \ \ |\ 0x02\ \ |\ ␂\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ ETX\ \ \ \ \ \ \ |\ 0x03\ \ |\ ␃\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ EOT\ \ \ \ \ \ \ |\ 0x04\ \ |\ ␄\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ ENQ\ \ \ \ \ \ \ |\ 0x05\ \ |\ ␅\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ ACK\ \ \ \ \ \ \ |\ 0x06\ \ |\ ␆\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ BEL\ \ \ \ \ \ \ |\ 0x07\ \ |\ ␇\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ BS\ \ \ \ \ \ \ \ |\ 0x08\ \ |\ ␈\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ HT\ \ \ \ \ \ \ \ |\ 0x09\ \ |\ ␉\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ LF\ \ \ \ \ \ \ \ |\ 0x0A\ \ |\ ␊\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ VT\ \ \ \ \ \ \ \ |\ 0x0B\ \ |\ ␋\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ FF\ \ \ \ \ \ \ \ |\ 0x0C\ \ |\ ␌\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ CR\ \ \ \ \ \ \ \ |\ 0x0D\ \ |\ ␍\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ SO\ \ \ \ \ \ \ \ |\ 0x0E\ \ |\ ␎\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ SI\ \ \ \ \ \ \ \ |\ 0x0F\ \ |\ ␏\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DLE\ \ \ \ \ \ \ |\ 0x10\ \ |\ ␐\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DC1\ \ \ \ \ \ \ |\ 0x11\ \ |\ ␑\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DC2\ \ \ \ \ \ \ |\ 0x12\ \ |\ ␒\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DC3\ \ \ \ \ \ \ |\ 0x13\ \ |\ ␓\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DC4\ \ \ \ \ \ \ |\ 0x14\ \ |\ ␔\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ NAK\ \ \ \ \ \ \ |\ 0x15\ \ |\ ␕\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ SYN\ \ \ \ \ \ \ |\ 0x16\ \ |\ ␖\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ ETB\ \ \ \ \ \ \ |\ 0x17\ \ |\ ␗\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ CAN\ \ \ \ \ \ \ |\ 0x18\ \ |\ ␘\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ EM\ \ \ \ \ \ \ \ |\ 0x19\ \ |\ ␙\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ SUB\ \ \ \ \ \ \ |\ 0x1A\ \ |\ ␚\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ ESC\ \ \ \ \ \ \ |\ 0x1B\ \ |\ ␛\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ FS\ \ \ \ \ \ \ \ |\ 0x1C\ \ |\ ␜\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ GS\ \ \ \ \ \ \ \ |\ 0x1D\ \ |\ ␝\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ RS\ \ \ \ \ \ \ \ |\ 0x1E\ \ |\ ␞\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ US\ \ \ \ \ \ \ \ |\ 0x1F\ \ |\ ␟\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ /\ \ \ \ \ \ \ \ \ |\ 0x2F\ \ |\ ／\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DEL\ \ \ \ \ \ \ |\ 0x7F\ \ |\ ␡\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
The\ default\ encoding\ will\ also\ encode\ these\ file\ names\ as\ they\ are
|
|
problematic\ with\ many\ cloud\ storage\ systems.
|
|
|
|
|\ File\ name\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ .\ \ \ \ \ \ \ \ \ |\ ．\ \ \ \ \ \ \ \ \ \ |
|
|
|\ ..\ \ \ \ \ \ \ \ |\ ．．\ \ \ \ \ \ \ \ \ |
|
|
|
|
####\ Invalid\ UTF\-8\ bytes\ {#invalid\-utf8}
|
|
|
|
Some\ backends\ only\ support\ a\ sequence\ of\ well\ formed\ UTF\-8\ bytes
|
|
as\ file\ or\ directory\ names.
|
|
|
|
In\ this\ case\ all\ invalid\ UTF\-8\ bytes\ will\ be\ replaced\ with\ a\ quoted
|
|
representation\ of\ the\ byte\ value\ to\ allow\ uploading\ a\ file\ to\ such\ a
|
|
backend.\ For\ example,\ the\ invalid\ byte\ `0xFE`\ will\ be\ encoded\ as\ `‛FE`.
|
|
|
|
A\ common\ source\ of\ invalid\ UTF\-8\ bytes\ are\ local\ filesystems,\ that\ store
|
|
names\ in\ a\ different\ encoding\ than\ UTF\-8\ or\ UTF\-16,\ like\ latin1.\ See\ the
|
|
[local\ filenames](/local/#filenames)\ section\ for\ details.
|
|
|
|
###\ MIME\ Type\ ###
|
|
|
|
MIME\ types\ (also\ known\ as\ media\ types)\ classify\ types\ of\ documents
|
|
using\ a\ simple\ text\ classification,\ eg\ `text/html`\ or
|
|
`application/pdf`.
|
|
|
|
Some\ cloud\ storage\ systems\ support\ reading\ (`R`)\ the\ MIME\ type\ of
|
|
objects\ and\ some\ support\ writing\ (`W`)\ the\ MIME\ type\ of\ objects.
|
|
|
|
The\ MIME\ type\ can\ be\ important\ if\ you\ are\ serving\ files\ directly\ to
|
|
HTTP\ from\ the\ storage\ system.
|
|
|
|
If\ you\ are\ copying\ from\ a\ remote\ which\ supports\ reading\ (`R`)\ to\ a
|
|
remote\ which\ supports\ writing\ (`W`)\ then\ rclone\ will\ preserve\ the\ MIME
|
|
types.\ \ Otherwise\ they\ will\ be\ guessed\ from\ the\ extension,\ or\ the
|
|
remote\ itself\ may\ assign\ the\ MIME\ type.
|
|
|
|
##\ Optional\ Features\ ##
|
|
|
|
All\ the\ remotes\ support\ a\ basic\ set\ of\ features,\ but\ there\ are\ some
|
|
optional\ features\ supported\ by\ some\ remotes\ used\ to\ make\ some
|
|
operations\ more\ efficient.
|
|
|
|
|\ Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Purge\ |\ Copy\ |\ Move\ |\ DirMove\ |\ CleanUp\ |\ ListR\ |\ StreamUpload\ |\ LinkSharing\ |\ About\ |\ EmptyDir\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-:|:\-\-\-\-:|:\-\-\-\-\-\-\-:|:\-\-\-\-\-\-\-:|:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-\-:|:\-\-\-\-\-:|\ :\-\-\-\-\-\-:\ |
|
|
|\ 1Fichier\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ \ \ No\ \ |\ \ Yes\ |
|
|
|\ Amazon\ Drive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ [#575](https://github.com/rclone/rclone/issues/575)\ |\ No\ \ |\ No\ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ Yes\ |
|
|
|\ Amazon\ S3\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ No\ |
|
|
|\ Backblaze\ B2\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ Yes\ |\ No\ \ |\ No\ |
|
|
|\ Box\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ [#575](https://github.com/rclone/rclone/issues/575)\ |\ No\ \ |\ Yes\ |\ Yes\ |\ No\ \ |\ Yes\ |
|
|
|\ Citrix\ ShareFile\ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ |\ No\ \ |\ Yes\ |
|
|
|\ Dropbox\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ [#575](https://github.com/rclone/rclone/issues/575)\ |\ No\ \ |\ Yes\ |\ Yes\ |\ Yes\ |\ Yes\ |
|
|
|\ FTP\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ Yes\ |
|
|
|\ Google\ Cloud\ Storage\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ No\ |
|
|
|\ Google\ Drive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ |\ Yes\ |\ Yes\ |
|
|
|\ Google\ Photos\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ |\ No\ |\ No\ |
|
|
|\ HTTP\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ Yes\ |
|
|
|\ Hubic\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ †\ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ |\ No\ |
|
|
|\ Jottacloud\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ |\ Yes\ |
|
|
|\ Mail.ru\ Cloud\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ |\ Yes\ |
|
|
|\ Mega\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ |\ Yes\ |
|
|
|\ Microsoft\ Azure\ Blob\ Storage\ |\ Yes\ \ \ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ No\ |
|
|
|\ Microsoft\ OneDrive\ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ [#575](https://github.com/rclone/rclone/issues/575)\ |\ No\ |\ No\ |\ Yes\ |\ Yes\ |\ Yes\ |
|
|
|\ OpenDrive\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ |\ Yes\ |
|
|
|\ Openstack\ Swift\ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ †\ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ |\ No\ |
|
|
|\ pCloud\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ |\ Yes\ |
|
|
|\ premiumize.me\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ |\ Yes\ |\ Yes\ |
|
|
|\ put.io\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ |\ Yes\ |
|
|
|\ QingStor\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ |\ No\ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ No\ \ |\ No\ |
|
|
|\ SFTP\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ \ |\ Yes\ |
|
|
|\ WebDAV\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ ‡\ \ \ \ \ \ \ \ |\ No\ [#2178](https://github.com/rclone/rclone/issues/2178)\ |\ Yes\ \ |\ Yes\ |
|
|
|\ Yandex\ Disk\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ Yes\ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ |\ Yes\ |\ Yes\ |
|
|
|\ The\ local\ filesystem\ \ \ \ \ \ \ \ \ |\ Yes\ \ \ |\ No\ \ \ |\ Yes\ \ |\ Yes\ \ \ \ \ |\ No\ \ \ \ \ \ |\ No\ \ \ \ |\ Yes\ \ \ \ \ \ \ \ \ \ |\ No\ \ \ \ \ \ \ \ \ \ |\ Yes\ |\ Yes\ |
|
|
|
|
###\ Purge\ ###
|
|
|
|
This\ deletes\ a\ directory\ quicker\ than\ just\ deleting\ all\ the\ files\ in
|
|
the\ directory.
|
|
|
|
†\ Note\ Swift\ and\ Hubic\ implement\ this\ in\ order\ to\ delete\ directory
|
|
markers\ but\ they\ don\[aq]t\ actually\ have\ a\ quicker\ way\ of\ deleting\ files
|
|
other\ than\ deleting\ them\ individually.
|
|
|
|
‡\ StreamUpload\ is\ not\ supported\ with\ Nextcloud
|
|
|
|
###\ Copy\ ###
|
|
|
|
Used\ when\ copying\ an\ object\ to\ and\ from\ the\ same\ remote.\ \ This\ known
|
|
as\ a\ server\ side\ copy\ so\ you\ can\ copy\ a\ file\ without\ downloading\ it
|
|
and\ uploading\ it\ again.\ \ It\ is\ used\ if\ you\ use\ `rclone\ copy`\ or
|
|
`rclone\ move`\ if\ the\ remote\ doesn\[aq]t\ support\ `Move`\ directly.
|
|
|
|
If\ the\ server\ doesn\[aq]t\ support\ `Copy`\ directly\ then\ for\ copy\ operations
|
|
the\ file\ is\ downloaded\ then\ re\-uploaded.
|
|
|
|
###\ Move\ ###
|
|
|
|
Used\ when\ moving/renaming\ an\ object\ on\ the\ same\ remote.\ \ This\ is\ known
|
|
as\ a\ server\ side\ move\ of\ a\ file.\ \ This\ is\ used\ in\ `rclone\ move`\ if\ the
|
|
server\ doesn\[aq]t\ support\ `DirMove`.
|
|
|
|
If\ the\ server\ isn\[aq]t\ capable\ of\ `Move`\ then\ rclone\ simulates\ it\ with
|
|
`Copy`\ then\ delete.\ \ If\ the\ server\ doesn\[aq]t\ support\ `Copy`\ then\ rclone
|
|
will\ download\ the\ file\ and\ re\-upload\ it.
|
|
|
|
###\ DirMove\ ###
|
|
|
|
This\ is\ used\ to\ implement\ `rclone\ move`\ to\ move\ a\ directory\ if
|
|
possible.\ \ If\ it\ isn\[aq]t\ then\ it\ will\ use\ `Move`\ on\ each\ file\ (which
|
|
falls\ back\ to\ `Copy`\ then\ download\ and\ upload\ \-\ see\ `Move`\ section).
|
|
|
|
###\ CleanUp\ ###
|
|
|
|
This\ is\ used\ for\ emptying\ the\ trash\ for\ a\ remote\ by\ `rclone\ cleanup`.
|
|
|
|
If\ the\ server\ can\[aq]t\ do\ `CleanUp`\ then\ `rclone\ cleanup`\ will\ return\ an
|
|
error.
|
|
|
|
###\ ListR\ ###
|
|
|
|
The\ remote\ supports\ a\ recursive\ list\ to\ list\ all\ the\ contents\ beneath
|
|
a\ directory\ quickly.\ \ This\ enables\ the\ `\-\-fast\-list`\ flag\ to\ work.
|
|
See\ the\ [rclone\ docs](/docs/#fast\-list)\ for\ more\ details.
|
|
|
|
###\ StreamUpload\ ###
|
|
|
|
Some\ remotes\ allow\ files\ to\ be\ uploaded\ without\ knowing\ the\ file\ size
|
|
in\ advance.\ This\ allows\ certain\ operations\ to\ work\ without\ spooling\ the
|
|
file\ to\ local\ disk\ first,\ e.g.\ `rclone\ rcat`.
|
|
|
|
###\ LinkSharing\ ###
|
|
|
|
Sets\ the\ necessary\ permissions\ on\ a\ file\ or\ folder\ and\ prints\ a\ link
|
|
that\ allows\ others\ to\ access\ them,\ even\ if\ they\ don\[aq]t\ have\ an\ account
|
|
on\ the\ particular\ cloud\ provider.
|
|
|
|
###\ About\ ###
|
|
|
|
This\ is\ used\ to\ fetch\ quota\ information\ from\ the\ remote,\ like\ bytes
|
|
used/free/quota\ and\ bytes\ used\ in\ the\ trash.
|
|
|
|
This\ is\ also\ used\ to\ return\ the\ space\ used,\ available\ for\ `rclone\ mount`.
|
|
|
|
If\ the\ server\ can\[aq]t\ do\ `About`\ then\ `rclone\ about`\ will\ return\ an
|
|
error.
|
|
|
|
###\ EmptyDir\ ###
|
|
|
|
The\ remote\ supports\ empty\ directories.\ See\ [Limitations](/bugs/#limitations)
|
|
\ for\ details.\ Most\ Object/Bucket\ based\ remotes\ do\ not\ support\ this.
|
|
|
|
#\ Global\ Flags
|
|
|
|
This\ describes\ the\ global\ flags\ available\ to\ every\ rclone\ command
|
|
split\ into\ two\ groups,\ non\ backend\ and\ backend\ flags.
|
|
|
|
##\ Non\ Backend\ Flags
|
|
|
|
These\ flags\ are\ available\ for\ every\ command.
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-\-ask\-password\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ prompt\ for\ password\ for\ encrypted\ configuration.\ (default\ true)
|
|
\ \ \-\-auto\-confirm\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ If\ enabled,\ do\ not\ request\ console\ confirmation.
|
|
\ \ \-\-backup\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Make\ backups\ into\ hierarchy\ based\ in\ DIR.
|
|
\ \ \-\-bind\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Local\ address\ to\ bind\ to\ for\ outgoing\ connections,\ IPv4,\ IPv6\ or\ name.
|
|
\ \ \-\-buffer\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ In\ memory\ buffer\ size\ when\ reading\ files\ for\ each\ \-\-transfer.\ (default\ 16M)
|
|
\ \ \-\-bwlimit\ BwTimetable\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Bandwidth\ limit\ in\ kBytes/s,\ or\ use\ suffix\ b|k|M|G\ or\ a\ full\ timetable.
|
|
\ \ \-\-ca\-cert\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CA\ certificate\ used\ to\ verify\ servers
|
|
\ \ \-\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.\ (default\ "$HOME/.cache/rclone")
|
|
\ \ \-\-checkers\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Number\ of\ checkers\ to\ run\ in\ parallel.\ (default\ 8)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\-c, \[en]checksum Skip based on checksum (if available) & size, not
|
|
mod\-time & size \[en]client\-cert string Client SSL certificate (PEM)
|
|
for mutual TLS auth \[en]client\-key string Client SSL private key (PEM)
|
|
for mutual TLS auth \[en]compare\-dest string use DIR to server side
|
|
copy flies from.
|
|
\[en]config string Config file.
|
|
(default \[lq]$HOME/.config/rclone/rclone.conf\[rq]) \[en]contimeout
|
|
duration Connect timeout (default 1m0s) \[en]copy\-dest string Compare
|
|
dest to DIR also.
|
|
\[en]cpuprofile string Write cpu profile to file \[en]delete\-after When
|
|
synchronizing, delete files on destination after transferring (default)
|
|
\[en]delete\-before When synchronizing, delete files on destination
|
|
before transferring \[en]delete\-during When synchronizing, delete files
|
|
during transfer \[en]delete\-excluded Delete files on dest excluded from
|
|
sync \[en]disable string Disable a comma separated list of features.
|
|
Use help to see a list.
|
|
\-n, \[en]dry\-run Do a trial run with no permanent changes \[en]dump
|
|
DumpFlags List of items to dump from:
|
|
headers,bodies,requests,responses,auth,filters,goroutines,openfiles
|
|
\[en]dump\-bodies Dump HTTP headers and bodies \- may contain sensitive
|
|
info \[en]dump\-headers Dump HTTP headers \- may contain sensitive info
|
|
\[en]exclude stringArray Exclude files matching pattern
|
|
\[en]exclude\-from stringArray Read exclude patterns from file
|
|
\[en]exclude\-if\-present string Exclude directories if filename is
|
|
present \[en]fast\-list Use recursive list if available.
|
|
Uses more memory but fewer transactions.
|
|
\[en]files\-from stringArray Read list of source\-file names from file
|
|
\-f, \[en]filter stringArray Add a file\-filtering rule
|
|
\[en]filter\-from stringArray Read filtering patterns from a file
|
|
\[en]ignore\-case Ignore case in filters (case insensitive)
|
|
\[en]ignore\-case\-sync Ignore case when synchronizing
|
|
\[en]ignore\-checksum Skip post copy check of checksums.
|
|
\[en]ignore\-errors delete even if there are I/O errors
|
|
\[en]ignore\-existing Skip all files that exist on destination
|
|
\[en]ignore\-size Ignore size when skipping use mod\-time or checksum.
|
|
\-I, \[en]ignore\-times Don't skip files that match size and time \-
|
|
transfer all files \[en]immutable Do not modify files.
|
|
Fail if existing files have been modified.
|
|
\[en]include stringArray Include files matching pattern
|
|
\[en]include\-from stringArray Read include patterns from file
|
|
\[en]log\-file string Log everything to this file \[en]log\-format
|
|
string Comma separated list of log format options (default
|
|
\[lq]date,time\[rq]) \[en]log\-level string Log level
|
|
DEBUG|INFO|NOTICE|ERROR (default \[lq]NOTICE\[rq])
|
|
\[en]low\-level\-retries int Number of low level retries to do.
|
|
(default 10) \[en]max\-age Duration Only transfer files younger than
|
|
this in s or suffix ms|s|m|h|d|w|M|y (default off) \[en]max\-backlog int
|
|
Maximum number of objects in sync or check backlog.
|
|
(default 10000) \[en]max\-delete int When synchronizing, limit the
|
|
number of deletes (default \-1) \[en]max\-depth int If set limits the
|
|
recursion depth to this.
|
|
(default \-1) \[en]max\-size SizeSuffix Only transfer files smaller than
|
|
this in k or suffix b|k|M|G (default off) \[en]max\-stats\-groups int
|
|
Maximum number of stats groups to keep in memory.
|
|
On max oldest is discarded.
|
|
(default 1000) \[en]max\-transfer SizeSuffix Maximum size of data to
|
|
transfer.
|
|
(default off) \[en]memprofile string Write memory profile to file
|
|
\[en]min\-age Duration Only transfer files older than this in s or
|
|
suffix ms|s|m|h|d|w|M|y (default off) \[en]min\-size SizeSuffix Only
|
|
transfer files bigger than this in k or suffix b|k|M|G (default off)
|
|
\[en]modify\-window duration Max time diff to be considered the same
|
|
(default 1ns) \[en]multi\-thread\-cutoff SizeSuffix Use multi\-thread
|
|
downloads for files above this size.
|
|
(default 250M) \[en]multi\-thread\-streams int Max number of streams to
|
|
use for multi\-thread downloads.
|
|
(default 4) \[en]no\-check\-certificate Do not verify the server SSL
|
|
certificate.
|
|
Insecure.
|
|
\[en]no\-gzip\-encoding Don't set Accept\-Encoding: gzip.
|
|
\[en]no\-traverse Don't traverse destination file system on copy.
|
|
\[en]no\-update\-modtime Don't update destination mod\-time if files
|
|
identical.
|
|
\-P, \[en]progress Show progress during transfer.
|
|
\-q, \[en]quiet Print as little stuff as possible \[en]rc Enable the
|
|
remote control server.
|
|
\[en]rc\-addr string IPaddress:Port or :Port to bind server to.
|
|
(default \[lq]localhost:5572\[rq]) \[en]rc\-allow\-origin string Set the
|
|
allowed origin for CORS.
|
|
\[en]rc\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\[en]rc\-cert string SSL PEM key (concatenation of certificate and CA
|
|
certificate) \[en]rc\-client\-ca string Client certificate authority to
|
|
verify clients with \[en]rc\-files string Path to local files to serve
|
|
on the HTTP server.
|
|
\[en]rc\-htpasswd string htpasswd file \- if not provided no
|
|
authentication is done \[en]rc\-job\-expire\-duration duration expire
|
|
finished async jobs older than this value (default 1m0s)
|
|
\[en]rc\-job\-expire\-interval duration interval to check for expired
|
|
async jobs (default 10s) \[en]rc\-key string SSL PEM Private key
|
|
\[en]rc\-max\-header\-bytes int Maximum size of request header (default
|
|
4096) \[en]rc\-no\-auth Don't require auth for certain methods.
|
|
\[en]rc\-pass string Password for authentication.
|
|
\[en]rc\-realm string realm for authentication (default
|
|
\[lq]rclone\[rq]) \[en]rc\-serve Enable the serving of remote objects.
|
|
\[en]rc\-server\-read\-timeout duration Timeout for server reading data
|
|
(default 1h0m0s) \[en]rc\-server\-write\-timeout duration Timeout for
|
|
server writing data (default 1h0m0s) \[en]rc\-user string User name for
|
|
authentication.
|
|
\[en]rc\-web\-fetch\-url string URL to fetch the releases for webgui.
|
|
(default
|
|
\[lq]https://api.github.com/repos/rclone/rclone\-webui\-react/releases/latest\[rq])
|
|
\[en]rc\-web\-gui Launch WebGUI on localhost \[en]rc\-web\-gui\-update
|
|
Update / Force update to latest version of web gui \[en]retries int
|
|
Retry operations this many times if they fail (default 3)
|
|
\[en]retries\-sleep duration Interval between retrying operations if
|
|
they fail, e.g 500ms, 60s, 5m.
|
|
(0 to disable) \[en]size\-only Skip based on size only, not mod\-time or
|
|
checksum \[en]stats duration Interval between printing stats, e.g 500ms,
|
|
60s, 5m.
|
|
(0 to disable) (default 1m0s) \[en]stats\-file\-name\-length int Max
|
|
file name length in stats.
|
|
0 for no limit (default 45) \[en]stats\-log\-level string Log level to
|
|
show \[en]stats output DEBUG|INFO|NOTICE|ERROR (default \[lq]INFO\[rq])
|
|
\[en]stats\-one\-line Make the stats fit on one line.
|
|
\[en]stats\-one\-line\-date Enables \[en]stats\-one\-line and add
|
|
current date/time prefix.
|
|
\[en]stats\-one\-line\-date\-format string Enables
|
|
\[en]stats\-one\-line\-date and uses custom formatted date.
|
|
Enclose date string in double quotes (\[lq]). See
|
|
https://golang.org/pkg/time/#Time.Format \[en]stats\-unit string Show
|
|
data rate in stats as either `bits' or `bytes'/s
|
|
(default\[rq]bytes\[lq]) \[en]streaming\-upload\-cutoff SizeSuffix
|
|
Cutoff for switching to chunked upload if file size is unknown. Upload
|
|
starts after reaching cutoff or when file ends. (default 100k)
|
|
\[en]suffix string Suffix to add to changed files.
|
|
\[en]suffix\-keep\-extension Preserve the extension when using
|
|
\[en]suffix. \[en]syslog Use Syslog for logging \[en]syslog\-facility
|
|
string Facility for syslog, eg KERN,USER,\&... (default\[rq]DAEMON\[lq])
|
|
\[en]timeout duration IO idle timeout (default 5m0s) \[en]tpslimit float
|
|
Limit HTTP transactions per second to this. \[en]tpslimit\-burst int Max
|
|
burst of transactions for \[en]tpslimit. (default 1) \[en]track\-renames
|
|
When synchronizing, track file renames and do a server side move if
|
|
possible \[en]transfers int Number of file transfers to run in parallel.
|
|
(default 4) \-u, \[en]update Skip files that are newer on the
|
|
destination. \[en]use\-cookies Enable session cookiejar.
|
|
\[en]use\-json\-log Use json log format. \[en]use\-mmap Use mmap
|
|
allocator (see docs). \[en]use\-server\-modtime Use server modified time
|
|
instead of object metadata \[en]user\-agent string Set the user\-agent
|
|
to a specified string. The default is rclone/ version
|
|
(default\[rq]rclone/v1.50.0") \-v, \[en]verbose count Print lots more
|
|
stuff (repeat for more)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
##\ Backend\ Flags
|
|
|
|
These\ flags\ are\ available\ for\ every\ command.\ They\ control\ the\ backends
|
|
and\ may\ be\ set\ in\ the\ config\ file.
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ \ \-\-acd\-auth\-url\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Auth\ server\ URL.
|
|
\ \ \-\-acd\-client\-id\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Amazon\ Application\ Client\ ID.
|
|
\ \ \-\-acd\-client\-secret\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Amazon\ Application\ Client\ Secret.
|
|
\ \ \-\-acd\-templink\-threshold\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ Files\ >=\ this\ size\ will\ be\ downloaded\ via\ their\ tempLink.\ (default\ 9G)
|
|
\ \ \-\-acd\-token\-url\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Token\ server\ url.
|
|
\ \ \-\-acd\-upload\-wait\-per\-gb\ Duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ Additional\ time\ per\ GB\ to\ wait\ after\ a\ failed\ complete\ upload\ to\ see\ if\ it\ appears.\ (default\ 3m0s)
|
|
\ \ \-\-alias\-remote\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Remote\ or\ path\ to\ alias.
|
|
\ \ \-\-azureblob\-access\-tier\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Access\ tier\ of\ blob:\ hot,\ cool\ or\ archive.
|
|
\ \ \-\-azureblob\-account\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Storage\ Account\ Name\ (leave\ blank\ to\ use\ SAS\ URL\ or\ Emulator)
|
|
\ \ \-\-azureblob\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ Upload\ chunk\ size\ (<=\ 100MB).\ (default\ 4M)
|
|
\ \ \-\-azureblob\-endpoint\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Endpoint\ for\ the\ service
|
|
\ \ \-\-azureblob\-key\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Storage\ Account\ Key\ (leave\ blank\ to\ use\ SAS\ URL\ or\ Emulator)
|
|
\ \ \-\-azureblob\-list\-chunk\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Size\ of\ blob\ list.\ (default\ 5000)
|
|
\ \ \-\-azureblob\-sas\-url\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ SAS\ URL\ for\ container\ level\ access\ only
|
|
\ \ \-\-azureblob\-upload\-cutoff\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ Cutoff\ for\ switching\ to\ chunked\ upload\ (<=\ 256MB).\ (default\ 256M)
|
|
\ \ \-\-azureblob\-use\-emulator\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Uses\ local\ storage\ emulator\ if\ provided\ as\ \[aq]true\[aq]\ (leave\ blank\ if\ using\ real\ azure\ storage\ endpoint)
|
|
\ \ \-\-b2\-account\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Account\ ID\ or\ Application\ Key\ ID
|
|
\ \ \-\-b2\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Upload\ chunk\ size.\ Must\ fit\ in\ memory.\ (default\ 96M)
|
|
\ \ \-\-b2\-disable\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Disable\ checksums\ for\ large\ (>\ upload\ cutoff)\ files
|
|
\ \ \-\-b2\-download\-auth\-duration\ Duration\ \ \ \ \ \ \ \ \ \ \ Time\ before\ the\ authorization\ token\ will\ expire\ in\ s\ or\ suffix\ ms|s|m|h|d.\ (default\ 1w)
|
|
\ \ \-\-b2\-download\-url\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Custom\ endpoint\ for\ downloads.
|
|
\ \ \-\-b2\-endpoint\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Endpoint\ for\ the\ service.
|
|
\ \ \-\-b2\-hard\-delete\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Permanently\ delete\ files\ on\ remote\ removal,\ otherwise\ hide\ files.
|
|
\ \ \-\-b2\-key\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Application\ Key
|
|
\ \ \-\-b2\-test\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ A\ flag\ string\ for\ X\-Bz\-Test\-Mode\ header\ for\ debugging.
|
|
\ \ \-\-b2\-upload\-cutoff\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cutoff\ for\ switching\ to\ chunked\ upload.\ (default\ 200M)
|
|
\ \ \-\-b2\-versions\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Include\ old\ versions\ in\ directory\ listings.
|
|
\ \ \-\-box\-box\-config\-file\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Box\ App\ config.json\ location
|
|
\ \ \-\-box\-box\-sub\-type\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (default\ "user")
|
|
\ \ \-\-box\-client\-id\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Box\ App\ Client\ Id.
|
|
\ \ \-\-box\-client\-secret\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Box\ App\ Client\ Secret
|
|
\ \ \-\-box\-commit\-retries\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Max\ number\ of\ times\ to\ try\ committing\ a\ multipart\ file.\ (default\ 100)
|
|
\ \ \-\-box\-upload\-cutoff\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cutoff\ for\ switching\ to\ multipart\ upload\ (>=\ 50MB).\ (default\ 50M)
|
|
\ \ \-\-cache\-chunk\-clean\-interval\ Duration\ \ \ \ \ \ \ \ \ \ How\ often\ should\ the\ cache\ perform\ cleanups\ of\ the\ chunk\ storage.\ (default\ 1m0s)
|
|
\ \ \-\-cache\-chunk\-no\-memory\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Disable\ the\ in\-memory\ cache\ for\ storing\ chunks\ during\ streaming.
|
|
\ \ \-\-cache\-chunk\-path\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ to\ cache\ chunk\ files.\ (default\ "$HOME/.cache/rclone/cache\-backend")
|
|
\ \ \-\-cache\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ The\ size\ of\ a\ chunk\ (partial\ file\ data).\ (default\ 5M)
|
|
\ \ \-\-cache\-chunk\-total\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ The\ total\ size\ that\ the\ chunks\ can\ take\ up\ on\ the\ local\ disk.\ (default\ 10G)
|
|
\ \ \-\-cache\-db\-path\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ to\ store\ file\ structure\ metadata\ DB.\ (default\ "$HOME/.cache/rclone/cache\-backend")
|
|
\ \ \-\-cache\-db\-purge\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Clear\ all\ the\ cached\ data\ for\ this\ remote\ on\ start.
|
|
\ \ \-\-cache\-db\-wait\-time\ Duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ How\ long\ to\ wait\ for\ the\ DB\ to\ be\ available\ \-\ 0\ is\ unlimited\ (default\ 1s)
|
|
\ \ \-\-cache\-info\-age\ Duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ How\ long\ to\ cache\ file\ structure\ information\ (directory\ listings,\ file\ size,\ times\ etc).\ (default\ 6h0m0s)
|
|
\ \ \-\-cache\-plex\-insecure\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Skip\ all\ certificate\ verifications\ when\ connecting\ to\ the\ Plex\ server
|
|
\ \ \-\-cache\-plex\-password\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ The\ password\ of\ the\ Plex\ user
|
|
\ \ \-\-cache\-plex\-url\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ The\ URL\ of\ the\ Plex\ server
|
|
\ \ \-\-cache\-plex\-username\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ The\ username\ of\ the\ Plex\ user
|
|
\ \ \-\-cache\-read\-retries\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ How\ many\ times\ to\ retry\ a\ read\ from\ a\ cache\ storage.\ (default\ 10)
|
|
\ \ \-\-cache\-remote\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Remote\ to\ cache.
|
|
\ \ \-\-cache\-rps\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Limits\ the\ number\ of\ requests\ per\ second\ to\ the\ source\ FS\ (\-1\ to\ disable)\ (default\ \-1)
|
|
\ \ \-\-cache\-tmp\-upload\-path\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ to\ keep\ temporary\ files\ until\ they\ are\ uploaded.
|
|
\ \ \-\-cache\-tmp\-wait\-time\ Duration\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ How\ long\ should\ files\ be\ stored\ in\ local\ cache\ before\ being\ uploaded\ (default\ 15s)
|
|
\ \ \-\-cache\-workers\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ How\ many\ workers\ should\ run\ in\ parallel\ to\ download\ chunks.\ (default\ 4)
|
|
\ \ \-\-cache\-writes\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ file\ data\ on\ writes\ through\ the\ FS
|
|
\ \ \-\-chunker\-chunk\-size\ SizeSuffix\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Files\ larger\ than\ chunk\ size\ will\ be\ split\ in\ chunks.\ (default\ 2G)
|
|
\ \ \-\-chunker\-fail\-hard\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Choose\ how\ chunker\ should\ handle\ files\ with\ missing\ or\ invalid\ chunks.
|
|
\ \ \-\-chunker\-hash\-type\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Choose\ how\ chunker\ handles\ hash\ sums.\ All\ modes\ but\ "none"\ require\ metadata.\ (default\ "md5")
|
|
\ \ \-\-chunker\-meta\-format\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Format\ of\ the\ metadata\ object\ or\ "none".\ By\ default\ "simplejson".\ (default\ "simplejson")
|
|
\ \ \-\-chunker\-name\-format\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ String\ format\ of\ chunk\ file\ names.\ (default\ "*.rclone_chunk.###")
|
|
\ \ \-\-chunker\-remote\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Remote\ to\ chunk/unchunk.
|
|
\ \ \-\-chunker\-start\-from\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Minimum\ valid\ chunk\ number.\ Usually\ 0\ or\ 1.\ (default\ 1)
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\-L, \[en]copy\-links Follow symlinks and copy the pointed to item.
|
|
\[en]crypt\-directory\-name\-encryption Option to either encrypt
|
|
directory names or leave them intact.
|
|
(default true) \[en]crypt\-filename\-encryption string How to encrypt
|
|
the filenames.
|
|
(default \[lq]standard\[rq]) \[en]crypt\-password string Password or
|
|
pass phrase for encryption.
|
|
\[en]crypt\-password2 string Password or pass phrase for salt.
|
|
Optional but recommended.
|
|
\[en]crypt\-remote string Remote to encrypt/decrypt.
|
|
\[en]crypt\-show\-mapping For all files listed show how the names
|
|
encrypt.
|
|
\[en]drive\-acknowledge\-abuse Set to allow files which return
|
|
cannotDownloadAbusiveFile to be downloaded.
|
|
\[en]drive\-allow\-import\-name\-change Allow the filetype to change
|
|
when uploading Google docs (e.g.\ file.doc to file.docx).
|
|
This will confuse sync and reupload every time.
|
|
\[en]drive\-alternate\-export Use alternate export URLs for google
|
|
documents export., \[en]drive\-auth\-owner\-only Only consider files
|
|
owned by the authenticated user.
|
|
\[en]drive\-chunk\-size SizeSuffix Upload chunk size.
|
|
Must a power of 2 >= 256k.
|
|
(default 8M) \[en]drive\-client\-id string Google Application Client Id
|
|
\[en]drive\-client\-secret string Google Application Client Secret
|
|
\[en]drive\-disable\-http2 Disable drive using http2 (default true)
|
|
\[en]drive\-export\-formats string Comma separated list of preferred
|
|
formats for downloading Google docs.
|
|
(default \[lq]docx,xlsx,pptx,svg\[rq]) \[en]drive\-formats string
|
|
Deprecated: see export_formats \[en]drive\-impersonate string
|
|
Impersonate this user when using a service account.
|
|
\[en]drive\-import\-formats string Comma separated list of preferred
|
|
formats for uploading Google docs.
|
|
\[en]drive\-keep\-revision\-forever Keep new head revision of each file
|
|
forever.
|
|
\[en]drive\-list\-chunk int Size of listing chunk 100\-1000.
|
|
0 to disable.
|
|
(default 1000) \[en]drive\-pacer\-burst int Number of API calls to allow
|
|
without sleeping.
|
|
(default 100) \[en]drive\-pacer\-min\-sleep Duration Minimum time to
|
|
sleep between API calls.
|
|
(default 100ms) \[en]drive\-root\-folder\-id string ID of the root
|
|
folder \[en]drive\-scope string Scope that rclone should use when
|
|
requesting access from drive.
|
|
\[en]drive\-server\-side\-across\-configs Allow server side operations
|
|
(eg copy) to work across different drive configs.
|
|
\[en]drive\-service\-account\-credentials string Service Account
|
|
Credentials JSON blob \[en]drive\-service\-account\-file string Service
|
|
Account Credentials JSON file path \[en]drive\-shared\-with\-me Only
|
|
show files that are shared with me.
|
|
\[en]drive\-size\-as\-quota Show storage quota usage for file size.
|
|
\[en]drive\-skip\-checksum\-gphotos Skip MD5 checksum on Google photos
|
|
and videos only.
|
|
\[en]drive\-skip\-gdocs Skip google documents in all listings.
|
|
\[en]drive\-team\-drive string ID of the Team Drive
|
|
\[en]drive\-trashed\-only Only show files that are in the trash.
|
|
\[en]drive\-upload\-cutoff SizeSuffix Cutoff for switching to chunked
|
|
upload (default 8M) \[en]drive\-use\-created\-date Use file created date
|
|
instead of modified date., \[en]drive\-use\-trash Send files to the
|
|
trash instead of deleting permanently.
|
|
(default true) \[en]drive\-v2\-download\-min\-size SizeSuffix If
|
|
Object's are greater, use drive v2 API to download.
|
|
(default off) \[en]dropbox\-chunk\-size SizeSuffix Upload chunk size.
|
|
(< 150M).
|
|
(default 48M) \[en]dropbox\-client\-id string Dropbox App Client Id
|
|
\[en]dropbox\-client\-secret string Dropbox App Client Secret
|
|
\[en]dropbox\-impersonate string Impersonate this user when using a
|
|
business account.
|
|
\[en]fichier\-api\-key string Your API Key, get it from
|
|
https://1fichier.com/console/params.pl \[en]fichier\-shared\-folder
|
|
string If you want to download a shared folder, add this parameter
|
|
\[en]ftp\-concurrency int Maximum number of FTP simultaneous
|
|
connections, 0 for unlimited \[en]ftp\-disable\-epsv Disable using EPSV
|
|
even if server advertises support \[en]ftp\-host string FTP host to
|
|
connect to \[en]ftp\-no\-check\-certificate Do not verify the TLS
|
|
certificate of the server \[en]ftp\-pass string FTP password
|
|
\[en]ftp\-port string FTP port, leave blank to use default (21)
|
|
\[en]ftp\-tls Use FTP over TLS (Implicit) \[en]ftp\-user string FTP
|
|
username, leave blank for current username, $USER \[en]gcs\-bucket\-acl
|
|
string Access Control List for new buckets.
|
|
\[en]gcs\-bucket\-policy\-only Access checks should use bucket\-level
|
|
IAM policies.
|
|
\[en]gcs\-client\-id string Google Application Client Id
|
|
\[en]gcs\-client\-secret string Google Application Client Secret
|
|
\[en]gcs\-location string Location for the newly created buckets.
|
|
\[en]gcs\-object\-acl string Access Control List for new objects.
|
|
\[en]gcs\-project\-number string Project number.
|
|
\[en]gcs\-service\-account\-file string Service Account Credentials JSON
|
|
file path \[en]gcs\-storage\-class string The storage class to use when
|
|
storing objects in Google Cloud Storage.
|
|
\[en]gphotos\-client\-id string Google Application Client Id
|
|
\[en]gphotos\-client\-secret string Google Application Client Secret
|
|
\[en]gphotos\-read\-only Set to make the Google Photos backend read
|
|
only.
|
|
\[en]gphotos\-read\-size Set to read the size of media items.
|
|
\[en]http\-headers CommaSepList Set HTTP headers for all transactions
|
|
\[en]http\-no\-head Don't use HEAD requests to find file sizes in dir
|
|
listing \[en]http\-no\-slash Set this if the site doesn't end
|
|
directories with / \[en]http\-url string URL of http host to connect to
|
|
\[en]hubic\-chunk\-size SizeSuffix Above this size files will be chunked
|
|
into a _segments container.
|
|
(default 5G) \[en]hubic\-client\-id string Hubic Client Id
|
|
\[en]hubic\-client\-secret string Hubic Client Secret
|
|
\[en]hubic\-no\-chunk Don't chunk files during streaming upload.
|
|
\[en]jottacloud\-hard\-delete Delete files permanently rather than
|
|
putting them into the trash.
|
|
\[en]jottacloud\-md5\-memory\-limit SizeSuffix Files bigger than this
|
|
will be cached on disk to calculate the MD5 if required.
|
|
(default 10M) \[en]jottacloud\-unlink Remove existing public link to
|
|
file/folder with link command rather than creating.
|
|
\[en]jottacloud\-upload\-resume\-limit SizeSuffix Files bigger than this
|
|
can be resumed if the upload fail's.
|
|
(default 10M) \[en]koofr\-endpoint string The Koofr API endpoint to use
|
|
(default \[lq]https://app.koofr.net\[rq]) \[en]koofr\-mountid string
|
|
Mount ID of the mount to use.
|
|
If omitted, the primary mount is used.
|
|
\[en]koofr\-password string Your Koofr password for rclone (generate one
|
|
at https://app.koofr.net/app/admin/preferences/password)
|
|
\[en]koofr\-setmtime Does the backend support setting modification time.
|
|
Set this to false if you use a mount ID that points to a Dropbox or
|
|
Amazon Drive backend.
|
|
(default true) \[en]koofr\-user string Your Koofr user name \-l,
|
|
\[en]links Translate symlinks to/from regular files with a `.rclonelink'
|
|
extension \[en]local\-case\-insensitive Force the filesystem to report
|
|
itself as case insensitive \[en]local\-case\-sensitive Force the
|
|
filesystem to report itself as case sensitive.
|
|
\[en]local\-no\-check\-updated Don't check to see if the files change
|
|
during upload \[en]local\-no\-unicode\-normalization Don't apply unicode
|
|
normalization to paths and filenames (Deprecated) \[en]local\-nounc
|
|
string Disable UNC (long path names) conversion on Windows
|
|
\[en]mailru\-check\-hash What should copy do if file checksum is
|
|
mismatched or invalid (default true) \[en]mailru\-pass string Password
|
|
\[en]mailru\-speedup\-enable Skip full upload if there is another file
|
|
with same data hash.
|
|
(default true) \[en]mailru\-speedup\-file\-patterns string Comma
|
|
separated list of file name patterns eligible for speedup (put by hash).
|
|
(default
|
|
\[lq]\f[I].mkv,\f[].avi,\f[I].mp4,\f[].mp3,\f[I].zip,\f[].gz,\f[I].rar,\f[].pdf\[rq])
|
|
\[en]mailru\-speedup\-max\-disk SizeSuffix This option allows you to
|
|
disable speedup (put by hash) for large files (default 3G)
|
|
\[en]mailru\-speedup\-max\-memory SizeSuffix Files larger than the size
|
|
given below will always be hashed on disk.
|
|
(default 32M) \[en]mailru\-user string User name (usually email)
|
|
\[en]mega\-debug Output more debug from Mega.
|
|
\[en]mega\-hard\-delete Delete files permanently rather than putting
|
|
them into the trash.
|
|
\[en]mega\-pass string Password.
|
|
\[en]mega\-user string User name \-x, \[en]one\-file\-system Don't cross
|
|
filesystem boundaries (unix/macOS only).
|
|
\[en]onedrive\-chunk\-size SizeSuffix Chunk size to upload files with \-
|
|
must be multiple of 320k (327,680 bytes).
|
|
(default 10M) \[en]onedrive\-client\-id string Microsoft App Client Id
|
|
\[en]onedrive\-client\-secret string Microsoft App Client Secret
|
|
\[en]onedrive\-drive\-id string The ID of the drive to use
|
|
\[en]onedrive\-drive\-type string The type of the drive ( personal |
|
|
business | documentLibrary ) \[en]onedrive\-expose\-onenote\-files Set
|
|
to make OneNote files show up in directory listings.
|
|
\[en]opendrive\-password string Password.
|
|
\[en]opendrive\-username string Username \[en]pcloud\-client\-id string
|
|
Pcloud App Client Id \[en]pcloud\-client\-secret string Pcloud App
|
|
Client Secret \[en]qingstor\-access\-key\-id string QingStor Access Key
|
|
ID \[en]qingstor\-chunk\-size SizeSuffix Chunk size to use for
|
|
uploading.
|
|
(default 4M) \[en]qingstor\-connection\-retries int Number of connection
|
|
retries.
|
|
(default 3) \[en]qingstor\-endpoint string Enter a endpoint URL to
|
|
connection QingStor API.
|
|
\[en]qingstor\-env\-auth Get QingStor credentials from runtime.
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
\[en]qingstor\-secret\-access\-key string QingStor Secret Access Key
|
|
(password) \[en]qingstor\-upload\-concurrency int Concurrency for
|
|
multipart uploads.
|
|
(default 1) \[en]qingstor\-upload\-cutoff SizeSuffix Cutoff for
|
|
switching to chunked upload (default 200M) \[en]qingstor\-zone string
|
|
Zone to connect to.
|
|
\[en]s3\-access\-key\-id string AWS Access Key ID.
|
|
\[en]s3\-acl string Canned ACL used when creating buckets and storing or
|
|
copying objects.
|
|
\[en]s3\-bucket\-acl string Canned ACL used when creating buckets.
|
|
\[en]s3\-chunk\-size SizeSuffix Chunk size to use for uploading.
|
|
(default 5M) \[en]s3\-disable\-checksum Don't store MD5 checksum with
|
|
object metadata \[en]s3\-endpoint string Endpoint for S3 API.
|
|
\[en]s3\-env\-auth Get AWS credentials from runtime (environment
|
|
variables or EC2/ECS meta data if no env vars).
|
|
\[en]s3\-force\-path\-style If true use path style access if false use
|
|
virtual hosted style.
|
|
(default true) \[en]s3\-leave\-parts\-on\-error If true avoid calling
|
|
abort upload on a failure, leaving all successfully uploaded parts on S3
|
|
for manual recovery.
|
|
\[en]s3\-location\-constraint string Location constraint \- must be set
|
|
to match the Region.
|
|
\[en]s3\-provider string Choose your S3 provider.
|
|
\[en]s3\-region string Region to connect to.
|
|
\[en]s3\-secret\-access\-key string AWS Secret Access Key (password)
|
|
\[en]s3\-server\-side\-encryption string The server\-side encryption
|
|
algorithm used when storing this object in S3.
|
|
\[en]s3\-session\-token string An AWS session token
|
|
\[en]s3\-sse\-kms\-key\-id string If using KMS ID you must provide the
|
|
ARN of Key.
|
|
\[en]s3\-storage\-class string The storage class to use when storing new
|
|
objects in S3.
|
|
\[en]s3\-upload\-concurrency int Concurrency for multipart uploads.
|
|
(default 4) \[en]s3\-upload\-cutoff SizeSuffix Cutoff for switching to
|
|
chunked upload (default 200M) \[en]s3\-use\-accelerate\-endpoint If true
|
|
use the AWS S3 accelerated endpoint.
|
|
\[en]s3\-v2\-auth If true use v2 authentication.
|
|
\[en]sftp\-ask\-password Allow asking for SFTP password when needed.
|
|
\[en]sftp\-disable\-hashcheck Disable the execution of SSH commands to
|
|
determine if remote file hashing is available.
|
|
\[en]sftp\-host string SSH host to connect to \[en]sftp\-key\-file
|
|
string Path to PEM\-encoded private key file, leave blank or set
|
|
key\-use\-agent to use ssh\-agent.
|
|
\[en]sftp\-key\-file\-pass string The passphrase to decrypt the
|
|
PEM\-encoded private key file.
|
|
\[en]sftp\-key\-use\-agent When set forces the usage of the ssh\-agent.
|
|
\[en]sftp\-md5sum\-command string The command used to read md5 hashes.
|
|
Leave blank for autodetect.
|
|
\[en]sftp\-pass string SSH password, leave blank to use ssh\-agent.
|
|
\[en]sftp\-path\-override string Override path used by SSH connection.
|
|
\[en]sftp\-port string SSH port, leave blank to use default (22)
|
|
\[en]sftp\-set\-modtime Set the modified time on the remote if set.
|
|
(default true) \[en]sftp\-sha1sum\-command string The command used to
|
|
read sha1 hashes.
|
|
Leave blank for autodetect.
|
|
\[en]sftp\-use\-insecure\-cipher Enable the use of insecure ciphers and
|
|
key exchange methods.
|
|
\[en]sftp\-user string SSH username, leave blank for current username,
|
|
ncw \[en]sharefile\-chunk\-size SizeSuffix Upload chunk size.
|
|
Must a power of 2 >= 256k.
|
|
(default 64M) \[en]sharefile\-endpoint string Endpoint for API calls.
|
|
\[en]sharefile\-root\-folder\-id string ID of the root folder
|
|
\[en]sharefile\-upload\-cutoff SizeSuffix Cutoff for switching to
|
|
multipart upload.
|
|
(default 128M) \[en]skip\-links Don't warn about skipped symlinks.
|
|
\[en]swift\-application\-credential\-id string Application Credential ID
|
|
(OS_APPLICATION_CREDENTIAL_ID) \[en]swift\-application\-credential\-name
|
|
string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
|
|
\[en]swift\-application\-credential\-secret string Application
|
|
Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET) \[en]swift\-auth
|
|
string Authentication URL for server (OS_AUTH_URL).
|
|
\[en]swift\-auth\-token string Auth Token from alternate authentication
|
|
\- optional (OS_AUTH_TOKEN) \[en]swift\-auth\-version int AuthVersion \-
|
|
optional \- set to (1,2,3) if your auth URL has no version
|
|
(ST_AUTH_VERSION) \[en]swift\-chunk\-size SizeSuffix Above this size
|
|
files will be chunked into a _segments container.
|
|
(default 5G) \[en]swift\-domain string User domain \- optional (v3 auth)
|
|
(OS_USER_DOMAIN_NAME) \[en]swift\-endpoint\-type string Endpoint type to
|
|
choose from the service catalogue (OS_ENDPOINT_TYPE) (default
|
|
\[lq]public\[rq]) \[en]swift\-env\-auth Get swift credentials from
|
|
environment variables in standard OpenStack form.
|
|
\[en]swift\-key string API key or password (OS_PASSWORD).
|
|
\[en]swift\-no\-chunk Don't chunk files during streaming upload.
|
|
\[en]swift\-region string Region name \- optional (OS_REGION_NAME)
|
|
\[en]swift\-storage\-policy string The storage policy to use when
|
|
creating a new container \[en]swift\-storage\-url string Storage URL \-
|
|
optional (OS_STORAGE_URL) \[en]swift\-tenant string Tenant name \-
|
|
optional for v1 auth, this or tenant_id required otherwise
|
|
(OS_TENANT_NAME or OS_PROJECT_NAME) \[en]swift\-tenant\-domain string
|
|
Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
\[en]swift\-tenant\-id string Tenant ID \- optional for v1 auth, this or
|
|
tenant required otherwise (OS_TENANT_ID) \[en]swift\-user string User
|
|
name to log in (OS_USERNAME).
|
|
\[en]swift\-user\-id string User ID to log in \- optional \- most swift
|
|
systems use user and leave this blank (v3 auth) (OS_USER_ID).
|
|
\[en]union\-remotes string List of space separated remotes.
|
|
\[en]webdav\-bearer\-token string Bearer token instead of user/pass (eg
|
|
a Macaroon) \[en]webdav\-bearer\-token\-command string Command to run to
|
|
get a bearer token \[en]webdav\-pass string Password.
|
|
\[en]webdav\-url string URL of http host to connect to \[en]webdav\-user
|
|
string User name \[en]webdav\-vendor string Name of the Webdav
|
|
site/service/software you are using \[en]yandex\-client\-id string
|
|
Yandex Client Id \[en]yandex\-client\-secret string Yandex Client Secret
|
|
\[en]yandex\-unlink Remove existing public link to file/folder with link
|
|
command rather than creating.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
1Fichier
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
This\ is\ a\ backend\ for\ the\ [1ficher](https://1fichier.com)\ cloud
|
|
storage\ service.\ Note\ that\ a\ Premium\ subscription\ is\ required\ to\ use
|
|
the\ API.
|
|
|
|
Paths\ are\ specified\ as\ `remote:path`
|
|
|
|
Paths\ may\ be\ as\ deep\ as\ required,\ eg\ `remote:directory/subdirectory`.
|
|
|
|
The\ initial\ setup\ for\ 1Fichier\ involves\ getting\ the\ API\ key\ from\ the\ website\ which\ you
|
|
need\ to\ do\ in\ your\ browser.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> remote Type of storage to
|
|
configure.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value [snip] XX / 1Fichier \ \[rq]fichier" [snip]
|
|
Storage> fichier ** See help for fichier backend at:
|
|
https://rclone.org/fichier/ **
|
|
.PP
|
|
Your API Key, get it from https://1fichier.com/console/params.pl Enter a
|
|
string value.
|
|
Press Enter for the default ("").
|
|
api_key> example_key
|
|
.PP
|
|
Edit advanced config?
|
|
(y/n) y) Yes n) No y/n> Remote config
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] type = fichier api_key =
|
|
example_key \[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e)
|
|
Edit this remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Once\ configured\ you\ can\ then\ use\ `rclone`\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ 1Fichier\ account
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ your\ 1Fichier\ account
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
To\ copy\ a\ local\ directory\ to\ a\ 1Fichier\ directory\ called\ backup
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:backup
|
|
|
|
###\ Modified\ time\ and\ hashes\ ###
|
|
|
|
1Fichier\ does\ not\ support\ modification\ times.\ It\ supports\ the\ Whirlpool\ hash\ algorithm.
|
|
|
|
###\ Duplicated\ files\ ###
|
|
|
|
1Fichier\ can\ have\ two\ files\ with\ exactly\ the\ same\ name\ and\ path\ (unlike\ a
|
|
normal\ file\ system).
|
|
|
|
Duplicated\ files\ cause\ problems\ with\ the\ syncing\ and\ you\ will\ see
|
|
messages\ in\ the\ log\ about\ duplicates.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
In\ addition\ to\ the\ [default\ restricted\ characters\ set](/overview/#restricted\-characters)
|
|
the\ following\ characters\ are\ also\ replaced:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ \\\ \ \ \ \ \ \ \ \ |\ 0x5C\ \ |\ ＼\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ <\ \ \ \ \ \ \ \ \ |\ 0x3C\ \ |\ ＜\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ >\ \ \ \ \ \ \ \ \ |\ 0x3E\ \ |\ ＞\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ "\ \ \ \ \ \ \ \ \ |\ 0x22\ \ |\ ＂\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ $\ \ \ \ \ \ \ \ \ |\ 0x24\ \ |\ ＄\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ `\ \ \ \ \ \ \ \ \ |\ 0x60\ \ |\ ｀\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ \[aq]\ \ \ \ \ \ \ \ \ |\ 0x27\ \ |\ ＇\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
File\ names\ can\ also\ not\ start\ or\ end\ with\ the\ following\ characters.
|
|
These\ only\ get\ replaced\ if\ they\ are\ first\ or\ last\ character\ in\ the
|
|
name:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ SP\ \ \ \ \ \ \ \ |\ 0x20\ \ |\ ␠\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/fichier/fichier.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ fichier\ (1Fichier).
|
|
|
|
####\ \-\-fichier\-api\-key
|
|
|
|
Your\ API\ Key,\ get\ it\ from\ https://1fichier.com/console/params.pl
|
|
|
|
\-\ Config:\ \ \ \ \ \ api_key
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FICHIER_API_KEY
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ fichier\ (1Fichier).
|
|
|
|
####\ \-\-fichier\-shared\-folder
|
|
|
|
If\ you\ want\ to\ download\ a\ shared\ folder,\ add\ this\ parameter
|
|
|
|
\-\ Config:\ \ \ \ \ \ shared_folder
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FICHIER_SHARED_FOLDER
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
Alias
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
The\ `alias`\ remote\ provides\ a\ new\ name\ for\ another\ remote.
|
|
|
|
Paths\ may\ be\ as\ deep\ as\ required\ or\ a\ local\ path,\
|
|
eg\ `remote:directory/subdirectory`\ or\ `/directory/subdirectory`.
|
|
|
|
During\ the\ initial\ setup\ with\ `rclone\ config`\ you\ will\ specify\ the\ target
|
|
remote.\ The\ target\ remote\ can\ either\ be\ a\ local\ path\ or\ another\ remote.
|
|
|
|
Subfolders\ can\ be\ used\ in\ target\ remote.\ Assume\ a\ alias\ remote\ named\ `backup`
|
|
with\ the\ target\ `mydrive:private/backup`.\ Invoking\ `rclone\ mkdir\ backup:desktop`
|
|
is\ exactly\ the\ same\ as\ invoking\ `rclone\ mkdir\ mydrive:private/backup/desktop`.
|
|
|
|
There\ will\ be\ no\ special\ handling\ of\ paths\ containing\ `..`\ segments.
|
|
Invoking\ `rclone\ mkdir\ backup:../desktop`\ is\ exactly\ the\ same\ as\ invoking
|
|
`rclone\ mkdir\ mydrive:private/backup/../desktop`.
|
|
The\ empty\ path\ is\ not\ allowed\ as\ a\ remote.\ To\ alias\ the\ current\ directory
|
|
use\ `.`\ instead.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ alias\ called\ `remote`\ for\ local\ folder.
|
|
First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> remote Type of storage to
|
|
configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Alias
|
|
for an existing remote \ \[lq]alias\[rq] [snip] Storage> alias Remote or
|
|
path to alias.
|
|
Can be \[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq],
|
|
\[lq]myremote:\[rq] or \[lq]/local/path\[rq].
|
|
remote> /mnt/storage/backup Remote config
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] remote =
|
|
/mnt/storage/backup \[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is
|
|
OK e) Edit this remote d) Delete this remote y/e/d> y Current remotes:
|
|
.PP
|
|
Name Type ==== ==== remote alias
|
|
.IP "e)" 3
|
|
Edit existing remote
|
|
.IP "f)" 3
|
|
New remote
|
|
.IP "g)" 3
|
|
Delete remote
|
|
.IP "h)" 3
|
|
Rename remote
|
|
.IP "i)" 3
|
|
Copy remote
|
|
.IP "j)" 3
|
|
Set configuration password
|
|
.IP "k)" 3
|
|
Quit config e/n/d/r/c/s/q> q
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Once\ configured\ you\ can\ then\ use\ `rclone`\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ in\ `/mnt/storage/backup`
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ `/mnt/storage/backup`
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
Copy\ another\ local\ directory\ to\ the\ alias\ directory\ called\ source
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:source
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/alias/alias.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ alias\ (Alias\ for\ an\ existing\ remote).
|
|
|
|
####\ \-\-alias\-remote
|
|
|
|
Remote\ or\ path\ to\ alias.
|
|
Can\ be\ "myremote:path/to/dir",\ "myremote:bucket",\ "myremote:"\ or\ "/local/path".
|
|
|
|
\-\ Config:\ \ \ \ \ \ remote
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ALIAS_REMOTE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
Amazon\ Drive
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
Amazon\ Drive,\ formerly\ known\ as\ Amazon\ Cloud\ Drive,\ is\ a\ cloud\ storage
|
|
service\ run\ by\ Amazon\ for\ consumers.
|
|
|
|
##\ Status
|
|
|
|
**Important:**\ rclone\ supports\ Amazon\ Drive\ only\ if\ you\ have\ your\ own
|
|
set\ of\ API\ keys.\ Unfortunately\ the\ [Amazon\ Drive\ developer
|
|
program](https://developer.amazon.com/amazon\-drive)\ is\ now\ closed\ to
|
|
new\ entries\ so\ if\ you\ don\[aq]t\ already\ have\ your\ own\ set\ of\ keys\ you\ will
|
|
not\ be\ able\ to\ use\ rclone\ with\ Amazon\ Drive.
|
|
|
|
For\ the\ history\ on\ why\ rclone\ no\ longer\ has\ a\ set\ of\ Amazon\ Drive\ API
|
|
keys\ see\ [the\ forum](https://forum.rclone.org/t/rclone\-has\-been\-banned\-from\-amazon\-drive/2314).
|
|
|
|
If\ you\ happen\ to\ know\ anyone\ who\ works\ at\ Amazon\ then\ please\ ask\ them
|
|
to\ re\-instate\ rclone\ into\ the\ Amazon\ Drive\ developer\ program\ \-\ thanks!
|
|
|
|
##\ Setup
|
|
|
|
The\ initial\ setup\ for\ Amazon\ Drive\ involves\ getting\ a\ token\ from
|
|
Amazon\ which\ you\ need\ to\ do\ in\ your\ browser.\ \ `rclone\ config`\ walks
|
|
you\ through\ it.
|
|
|
|
The\ configuration\ process\ for\ Amazon\ Drive\ may\ involve\ using\ an\ [oauth
|
|
proxy](https://github.com/ncw/oauthproxy).\ This\ is\ used\ to\ keep\ the
|
|
Amazon\ credentials\ out\ of\ the\ source\ code.\ \ The\ proxy\ runs\ in\ Google\[aq]s
|
|
very\ secure\ App\ Engine\ environment\ and\ doesn\[aq]t\ store\ any\ credentials
|
|
which\ pass\ through\ it.
|
|
|
|
Since\ rclone\ doesn\[aq]t\ currently\ have\ its\ own\ Amazon\ Drive\ credentials
|
|
so\ you\ will\ either\ need\ to\ have\ your\ own\ `client_id`\ and
|
|
`client_secret`\ with\ Amazon\ Drive,\ or\ use\ a\ a\ third\ party\ ouath\ proxy
|
|
in\ which\ case\ you\ will\ need\ to\ enter\ `client_id`,\ `client_secret`,
|
|
`auth_url`\ and\ `token_url`.
|
|
|
|
Note\ also\ if\ you\ are\ not\ using\ Amazon\[aq]s\ `auth_url`\ and\ `token_url`,
|
|
(ie\ you\ filled\ in\ something\ for\ those)\ then\ if\ setting\ up\ on\ a\ remote
|
|
machine\ you\ can\ only\ use\ the\ [copying\ the\ config\ method\ of
|
|
configuration](https://rclone.org/remote_setup/#configuring\-by\-copying\-the\-config\-file)
|
|
\-\ `rclone\ authorize`\ will\ not\ work.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote r) Rename remote c)
|
|
Copy remote s) Set configuration password q) Quit config n/r/c/s/q> n
|
|
name> remote Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Amazon
|
|
Drive \ \[lq]amazon cloud drive\[rq] [snip] Storage> amazon cloud drive
|
|
Amazon Application Client Id \- required.
|
|
client_id> your client ID goes here Amazon Application Client Secret \-
|
|
required.
|
|
client_secret> your client secret goes here Auth server URL \- leave
|
|
blank to use Amazon's.
|
|
auth_url> Optional auth URL Token server url \- leave blank to use
|
|
Amazon's.
|
|
token_url> Optional token URL Remote config Make sure your Redirect URL
|
|
is set to \[lq]http://127.0.0.1:53682/\[rq] in your custom config.
|
|
Use auto config?
|
|
* Say Y if not sure * Say N if you are working on a remote or headless
|
|
machine y) Yes n) No y/n> y If your browser doesn't open automatically
|
|
go to the following link: http://127.0.0.1:53682/auth Log in and
|
|
authorize rclone for access Waiting for code\&... Got code
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] client_id = your client ID
|
|
goes here client_secret = your client secret goes here auth_url =
|
|
Optional auth URL token_url = Optional token URL token =
|
|
{\[lq]access_token\[rq]:\[lq]xxxxxxxxxxxxxxxxxxxxxxx\[rq],\[lq]token_type\[rq]:\[lq]bearer\[rq],\[lq]refresh_token\[rq]:\[lq]xxxxxxxxxxxxxxxxxx\[rq],\[lq]expiry\[rq]:\[lq]2015\-09\-06T16:07:39.658438471+01:00\[rq]}
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
See\ the\ [remote\ setup\ docs](https://rclone.org/remote_setup/)\ for\ how\ to\ set\ it\ up\ on\ a
|
|
machine\ with\ no\ Internet\ browser\ available.
|
|
|
|
Note\ that\ rclone\ runs\ a\ webserver\ on\ your\ local\ machine\ to\ collect\ the
|
|
token\ as\ returned\ from\ Amazon.\ This\ only\ runs\ from\ the\ moment\ it
|
|
opens\ your\ browser\ to\ the\ moment\ you\ get\ back\ the\ verification
|
|
code.\ \ This\ is\ on\ `http://127.0.0.1:53682/`\ and\ this\ it\ may\ require
|
|
you\ to\ unblock\ it\ temporarily\ if\ you\ are\ running\ a\ host\ firewall.
|
|
|
|
Once\ configured\ you\ can\ then\ use\ `rclone`\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ Amazon\ Drive
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ your\ Amazon\ Drive
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
To\ copy\ a\ local\ directory\ to\ an\ Amazon\ Drive\ directory\ called\ backup
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:backup
|
|
|
|
###\ Modified\ time\ and\ MD5SUMs\ ###
|
|
|
|
Amazon\ Drive\ doesn\[aq]t\ allow\ modification\ times\ to\ be\ changed\ via
|
|
the\ API\ so\ these\ won\[aq]t\ be\ accurate\ or\ used\ for\ syncing.
|
|
|
|
It\ does\ store\ MD5SUMs\ so\ for\ a\ more\ accurate\ sync,\ you\ can\ use\ the
|
|
`\-\-checksum`\ flag.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ NUL\ \ \ \ \ \ \ |\ 0x00\ \ |\ ␀\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ /\ \ \ \ \ \ \ \ \ |\ 0x2F\ \ |\ ／\ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
###\ Deleting\ files\ ###
|
|
|
|
Any\ files\ you\ delete\ with\ rclone\ will\ end\ up\ in\ the\ trash.\ \ Amazon
|
|
don\[aq]t\ provide\ an\ API\ to\ permanently\ delete\ files,\ nor\ to\ empty\ the
|
|
trash,\ so\ you\ will\ have\ to\ do\ that\ with\ one\ of\ Amazon\[aq]s\ apps\ or\ via
|
|
the\ Amazon\ Drive\ website.\ As\ of\ November\ 17,\ 2016,\ files\ are\
|
|
automatically\ deleted\ by\ Amazon\ from\ the\ trash\ after\ 30\ days.
|
|
|
|
###\ Using\ with\ non\ `.com`\ Amazon\ accounts\ ###
|
|
|
|
Let\[aq]s\ say\ you\ usually\ use\ `amazon.co.uk`.\ When\ you\ authenticate\ with
|
|
rclone\ it\ will\ take\ you\ to\ an\ `amazon.com`\ page\ to\ log\ in.\ \ Your
|
|
`amazon.co.uk`\ email\ and\ password\ should\ work\ here\ just\ fine.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/amazonclouddrive/amazonclouddrive.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ amazon\ cloud\ drive\ (Amazon\ Drive).
|
|
|
|
####\ \-\-acd\-client\-id
|
|
|
|
Amazon\ Application\ Client\ ID.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_CLIENT_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-acd\-client\-secret
|
|
|
|
Amazon\ Application\ Client\ Secret.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_secret
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_CLIENT_SECRET
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ amazon\ cloud\ drive\ (Amazon\ Drive).
|
|
|
|
####\ \-\-acd\-auth\-url
|
|
|
|
Auth\ server\ URL.
|
|
Leave\ blank\ to\ use\ Amazon\[aq]s.
|
|
|
|
\-\ Config:\ \ \ \ \ \ auth_url
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_AUTH_URL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-acd\-token\-url
|
|
|
|
Token\ server\ url.
|
|
leave\ blank\ to\ use\ Amazon\[aq]s.
|
|
|
|
\-\ Config:\ \ \ \ \ \ token_url
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_TOKEN_URL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-acd\-checkpoint
|
|
|
|
Checkpoint\ for\ internal\ polling\ (debug).
|
|
|
|
\-\ Config:\ \ \ \ \ \ checkpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_CHECKPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-acd\-upload\-wait\-per\-gb
|
|
|
|
Additional\ time\ per\ GB\ to\ wait\ after\ a\ failed\ complete\ upload\ to\ see\ if\ it\ appears.
|
|
|
|
Sometimes\ Amazon\ Drive\ gives\ an\ error\ when\ a\ file\ has\ been\ fully
|
|
uploaded\ but\ the\ file\ appears\ anyway\ after\ a\ little\ while.\ \ This
|
|
happens\ sometimes\ for\ files\ over\ 1GB\ in\ size\ and\ nearly\ every\ time\ for
|
|
files\ bigger\ than\ 10GB.\ This\ parameter\ controls\ the\ time\ rclone\ waits
|
|
for\ the\ file\ to\ appear.
|
|
|
|
The\ default\ value\ for\ this\ parameter\ is\ 3\ minutes\ per\ GB,\ so\ by
|
|
default\ it\ will\ wait\ 3\ minutes\ for\ every\ GB\ uploaded\ to\ see\ if\ the
|
|
file\ appears.
|
|
|
|
You\ can\ disable\ this\ feature\ by\ setting\ it\ to\ 0.\ This\ may\ cause
|
|
conflict\ errors\ as\ rclone\ retries\ the\ failed\ upload\ but\ the\ file\ will
|
|
most\ likely\ appear\ correctly\ eventually.
|
|
|
|
These\ values\ were\ determined\ empirically\ by\ observing\ lots\ of\ uploads
|
|
of\ big\ files\ for\ a\ range\ of\ file\ sizes.
|
|
|
|
Upload\ with\ the\ "\-v"\ flag\ to\ see\ more\ info\ about\ what\ rclone\ is\ doing
|
|
in\ this\ situation.
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_wait_per_gb
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_UPLOAD_WAIT_PER_GB
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 3m0s
|
|
|
|
####\ \-\-acd\-templink\-threshold
|
|
|
|
Files\ >=\ this\ size\ will\ be\ downloaded\ via\ their\ tempLink.
|
|
|
|
Files\ this\ size\ or\ more\ will\ be\ downloaded\ via\ their\ "tempLink".\ This
|
|
is\ to\ work\ around\ a\ problem\ with\ Amazon\ Drive\ which\ blocks\ downloads
|
|
of\ files\ bigger\ than\ about\ 10GB.\ \ The\ default\ for\ this\ is\ 9GB\ which
|
|
shouldn\[aq]t\ need\ to\ be\ changed.
|
|
|
|
To\ download\ files\ above\ this\ threshold,\ rclone\ requests\ a\ "tempLink"
|
|
which\ downloads\ the\ file\ through\ a\ temporary\ URL\ directly\ from\ the
|
|
underlying\ S3\ storage.
|
|
|
|
\-\ Config:\ \ \ \ \ \ templink_threshold
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_ACD_TEMPLINK_THRESHOLD
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 9G
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
###\ Limitations\ ###
|
|
|
|
Note\ that\ Amazon\ Drive\ is\ case\ insensitive\ so\ you\ can\[aq]t\ have\ a
|
|
file\ called\ "Hello.doc"\ and\ one\ called\ "hello.doc".
|
|
|
|
Amazon\ Drive\ has\ rate\ limiting\ so\ you\ may\ notice\ errors\ in\ the
|
|
sync\ (429\ errors).\ \ rclone\ will\ automatically\ retry\ the\ sync\ up\ to\ 3
|
|
times\ by\ default\ (see\ `\-\-retries`\ flag)\ which\ should\ hopefully\ work
|
|
around\ this\ problem.
|
|
|
|
Amazon\ Drive\ has\ an\ internal\ limit\ of\ file\ sizes\ that\ can\ be\ uploaded
|
|
to\ the\ service.\ This\ limit\ is\ not\ officially\ published,\ but\ all\ files
|
|
larger\ than\ this\ will\ fail.
|
|
|
|
At\ the\ time\ of\ writing\ (Jan\ 2016)\ is\ in\ the\ area\ of\ 50GB\ per\ file.
|
|
This\ means\ that\ larger\ files\ are\ likely\ to\ fail.
|
|
|
|
Unfortunately\ there\ is\ no\ way\ for\ rclone\ to\ see\ that\ this\ failure\ is
|
|
because\ of\ file\ size,\ so\ it\ will\ retry\ the\ operation,\ as\ any\ other
|
|
failure.\ To\ avoid\ this\ problem,\ use\ `\-\-max\-size\ 50000M`\ option\ to\ limit
|
|
the\ maximum\ size\ of\ uploaded\ files.\ Note\ that\ `\-\-max\-size`\ does\ not\ split
|
|
files\ into\ segments,\ it\ only\ ignores\ files\ over\ this\ size.
|
|
|
|
Amazon\ S3\ Storage\ Providers
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
The\ S3\ backend\ can\ be\ used\ with\ a\ number\ of\ different\ providers:
|
|
|
|
*\ AWS\ S3
|
|
*\ Alibaba\ Cloud\ (Aliyun)\ Object\ Storage\ System\ (OSS)
|
|
*\ Ceph
|
|
*\ DigitalOcean\ Spaces
|
|
*\ Dreamhost
|
|
*\ IBM\ COS\ S3
|
|
*\ Minio
|
|
*\ Wasabi
|
|
|
|
Paths\ are\ specified\ as\ `remote:bucket`\ (or\ `remote:`\ for\ the\ `lsd`
|
|
command.)\ \ You\ may\ put\ subdirectories\ in\ too,\ eg\ `remote:bucket/path/to/dir`.
|
|
|
|
Once\ you\ have\ made\ a\ remote\ (see\ the\ provider\ specific\ section\ above)
|
|
you\ can\ use\ it\ like\ this:
|
|
|
|
See\ all\ buckets
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
Make\ a\ new\ bucket
|
|
|
|
\ \ \ \ rclone\ mkdir\ remote:bucket
|
|
|
|
List\ the\ contents\ of\ a\ bucket
|
|
|
|
\ \ \ \ rclone\ ls\ remote:bucket
|
|
|
|
Sync\ `/home/local/directory`\ to\ the\ remote\ bucket,\ deleting\ any\ excess
|
|
files\ in\ the\ bucket.
|
|
|
|
\ \ \ \ rclone\ sync\ /home/local/directory\ remote:bucket
|
|
|
|
##\ AWS\ S3\ {#amazon\-s3}
|
|
|
|
Here\ is\ an\ example\ of\ making\ an\ s3\ configuration.\ \ First\ run
|
|
|
|
\ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> remote Type of storage to
|
|
configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Amazon
|
|
S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
|
|
\ \[lq]s3\[rq] [snip] Storage> s3 Choose your S3 provider.
|
|
Choose a number from below, or type in your own value 1 / Amazon Web
|
|
Services (AWS) S3 \ \[lq]AWS\[rq] 2 / Ceph Object Storage
|
|
\ \[lq]Ceph\[rq] 3 / Digital Ocean Spaces \ \[lq]DigitalOcean\[rq] 4 /
|
|
Dreamhost DreamObjects \ \[lq]Dreamhost\[rq] 5 / IBM COS S3
|
|
\ \[lq]IBMCOS\[rq] 6 / Minio Object Storage \ \[lq]Minio\[rq] 7 / Wasabi
|
|
Object Storage \ \[lq]Wasabi\[rq] 8 / Any other S3 compatible provider
|
|
\ \[lq]Other\[rq] provider> 1 Get AWS credentials from runtime
|
|
(environment variables or EC2/ECS meta data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value 1 / Enter AWS
|
|
credentials in the next step \ \[lq]false\[rq] 2 / Get AWS credentials
|
|
from the environment (env vars or IAM) \ \[lq]true\[rq] env_auth> 1 AWS
|
|
Access Key ID \- leave blank for anonymous access or runtime
|
|
credentials.
|
|
access_key_id> XXX AWS Secret Access Key (password) \- leave blank for
|
|
anonymous access or runtime credentials.
|
|
secret_access_key> YYY Region to connect to.
|
|
Choose a number from below, or type in your own value / The default
|
|
endpoint \- a good choice if you are unsure.
|
|
1 | US Region, Northern Virginia or Pacific Northwest.
|
|
| Leave location constraint empty.
|
|
\ \[lq]us\-east\-1\[rq] / US East (Ohio) Region 2 | Needs location
|
|
constraint us\-east\-2.
|
|
\ \[lq]us\-east\-2\[rq] / US West (Oregon) Region 3 | Needs location
|
|
constraint us\-west\-2.
|
|
\ \[lq]us\-west\-2\[rq] / US West (Northern California) Region 4 | Needs
|
|
location constraint us\-west\-1.
|
|
\ \[lq]us\-west\-1\[rq] / Canada (Central) Region 5 | Needs location
|
|
constraint ca\-central\-1.
|
|
\ \[lq]ca\-central\-1\[rq] / EU (Ireland) Region 6 | Needs location
|
|
constraint EU or eu\-west\-1.
|
|
\ \[lq]eu\-west\-1\[rq] / EU (London) Region 7 | Needs location
|
|
constraint eu\-west\-2.
|
|
\ \[lq]eu\-west\-2\[rq] / EU (Frankfurt) Region 8 | Needs location
|
|
constraint eu\-central\-1.
|
|
\ \[lq]eu\-central\-1\[rq] / Asia Pacific (Singapore) Region 9 | Needs
|
|
location constraint ap\-southeast\-1.
|
|
\ \[lq]ap\-southeast\-1\[rq] / Asia Pacific (Sydney) Region 10 | Needs
|
|
location constraint ap\-southeast\-2.
|
|
\ \[lq]ap\-southeast\-2\[rq] / Asia Pacific (Tokyo) Region 11 | Needs
|
|
location constraint ap\-northeast\-1.
|
|
\ \[lq]ap\-northeast\-1\[rq] / Asia Pacific (Seoul) 12 | Needs location
|
|
constraint ap\-northeast\-2.
|
|
\ \[lq]ap\-northeast\-2\[rq] / Asia Pacific (Mumbai) 13 | Needs location
|
|
constraint ap\-south\-1.
|
|
\ \[lq]ap\-south\-1\[rq] / South America (Sao Paulo) Region 14 | Needs
|
|
location constraint sa\-east\-1.
|
|
\ \[lq]sa\-east\-1\[rq] region> 1 Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
endpoint> Location constraint \- must be set to match the Region.
|
|
Used when creating buckets only.
|
|
Choose a number from below, or type in your own value 1 / Empty for US
|
|
Region, Northern Virginia or Pacific Northwest.
|
|
\ "" 2 / US East (Ohio) Region.
|
|
\ \[lq]us\-east\-2\[rq] 3 / US West (Oregon) Region.
|
|
\ \[lq]us\-west\-2\[rq] 4 / US West (Northern California) Region.
|
|
\ \[lq]us\-west\-1\[rq] 5 / Canada (Central) Region.
|
|
\ \[lq]ca\-central\-1\[rq] 6 / EU (Ireland) Region.
|
|
\ \[lq]eu\-west\-1\[rq] 7 / EU (London) Region.
|
|
\ \[lq]eu\-west\-2\[rq] 8 / EU Region.
|
|
\ \[lq]EU\[rq] 9 / Asia Pacific (Singapore) Region.
|
|
\ \[lq]ap\-southeast\-1\[rq] 10 / Asia Pacific (Sydney) Region.
|
|
\ \[lq]ap\-southeast\-2\[rq] 11 / Asia Pacific (Tokyo) Region.
|
|
\ \[lq]ap\-northeast\-1\[rq] 12 / Asia Pacific (Seoul)
|
|
\ \[lq]ap\-northeast\-2\[rq] 13 / Asia Pacific (Mumbai)
|
|
\ \[lq]ap\-south\-1\[rq] 14 / South America (Sao Paulo) Region.
|
|
\ \[lq]sa\-east\-1\[rq] location_constraint> 1 Canned ACL used when
|
|
creating buckets and/or storing objects in S3.
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value 1 / Owner gets
|
|
FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
\ \[lq]private\[rq] 2 / Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
\ \[lq]public\-read\[rq] / Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
3 | Granting this on a bucket is generally not recommended.
|
|
\ \[lq]public\-read\-write\[rq] 4 / Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
\ \[lq]authenticated\-read\[rq] / Object owner gets FULL_CONTROL.
|
|
Bucket owner gets READ access.
|
|
5 | If you specify this canned ACL when creating a bucket, Amazon S3
|
|
ignores it.
|
|
\ \[lq]bucket\-owner\-read\[rq] / Both the object owner and the bucket
|
|
owner get FULL_CONTROL over the object.
|
|
6 | If you specify this canned ACL when creating a bucket, Amazon S3
|
|
ignores it.
|
|
\ \[lq]bucket\-owner\-full\-control\[rq] acl> 1 The server\-side
|
|
encryption algorithm used when storing this object in S3.
|
|
Choose a number from below, or type in your own value 1 / None \ "" 2 /
|
|
AES256 \ \[lq]AES256\[rq] server_side_encryption> 1 The storage class to
|
|
use when storing objects in S3.
|
|
Choose a number from below, or type in your own value 1 / Default \ "" 2
|
|
/ Standard storage class \ \[lq]STANDARD\[rq] 3 / Reduced redundancy
|
|
storage class \ \[lq]REDUCED_REDUNDANCY\[rq] 4 / Standard Infrequent
|
|
Access storage class \ \[lq]STANDARD_IA\[rq] 5 / One Zone Infrequent
|
|
Access storage class \ \[lq]ONEZONE_IA\[rq] 6 / Glacier storage class
|
|
\ \[lq]GLACIER\[rq] 7 / Glacier Deep Archive storage class
|
|
\ \[lq]DEEP_ARCHIVE\[rq] 8 / Intelligent\-Tiering storage class
|
|
\ \[lq]INTELLIGENT_TIERING\[rq] storage_class> 1 Remote config
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] type = s3 provider = AWS
|
|
env_auth = false access_key_id = XXX secret_access_key = YYY region =
|
|
us\-east\-1 endpoint = location_constraint = acl = private
|
|
server_side_encryption = storage_class =
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d>
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ \-\-fast\-list\ ###
|
|
|
|
This\ remote\ supports\ `\-\-fast\-list`\ which\ allows\ you\ to\ use\ fewer
|
|
transactions\ in\ exchange\ for\ more\ memory.\ See\ the\ [rclone
|
|
docs](/docs/#fast\-list)\ for\ more\ details.
|
|
|
|
###\ \-\-update\ and\ \-\-use\-server\-modtime\ ###
|
|
|
|
As\ noted\ below,\ the\ modified\ time\ is\ stored\ on\ metadata\ on\ the\ object.\ It\ is
|
|
used\ by\ default\ for\ all\ operations\ that\ require\ checking\ the\ time\ a\ file\ was
|
|
last\ updated.\ It\ allows\ rclone\ to\ treat\ the\ remote\ more\ like\ a\ true\ filesystem,
|
|
but\ it\ is\ inefficient\ because\ it\ requires\ an\ extra\ API\ call\ to\ retrieve\ the
|
|
metadata.
|
|
|
|
For\ many\ operations,\ the\ time\ the\ object\ was\ last\ uploaded\ to\ the\ remote\ is
|
|
sufficient\ to\ determine\ if\ it\ is\ "dirty".\ By\ using\ `\-\-update`\ along\ with
|
|
`\-\-use\-server\-modtime`,\ you\ can\ avoid\ the\ extra\ API\ call\ and\ simply\ upload
|
|
files\ whose\ local\ modtime\ is\ newer\ than\ the\ time\ it\ was\ last\ uploaded.
|
|
|
|
###\ Modified\ time\ ###
|
|
|
|
The\ modified\ time\ is\ stored\ as\ metadata\ on\ the\ object\ as
|
|
`X\-Amz\-Meta\-Mtime`\ as\ floating\ point\ since\ the\ epoch\ accurate\ to\ 1\ ns.
|
|
|
|
If\ the\ modification\ time\ needs\ to\ be\ updated\ rclone\ will\ attempt\ to\ perform\ a\ server
|
|
side\ copy\ to\ update\ the\ modification\ if\ the\ object\ can\ be\ copied\ in\ a\ single\ part.\ \
|
|
In\ the\ case\ the\ object\ is\ larger\ than\ 5Gb\ or\ is\ in\ Glacier\ or\ Glacier\ Deep\ Archive\
|
|
storage\ the\ object\ will\ be\ uploaded\ rather\ than\ copied.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
S3\ allows\ any\ valid\ UTF\-8\ string\ as\ a\ key.
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ be\ [replaced](/overview/#invalid\-utf8),\ as
|
|
they\ can\[aq]t\ be\ used\ in\ XML.
|
|
|
|
The\ following\ characters\ are\ replaced\ since\ these\ are\ problematic\ when
|
|
dealing\ with\ the\ REST\ API:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ NUL\ \ \ \ \ \ \ |\ 0x00\ \ |\ ␀\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ /\ \ \ \ \ \ \ \ \ |\ 0x2F\ \ |\ ／\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
The\ encoding\ will\ also\ encode\ these\ file\ names\ as\ they\ don\[aq]t\ seem\ to
|
|
work\ with\ the\ SDK\ properly:
|
|
|
|
|\ File\ name\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ .\ \ \ \ \ \ \ \ \ |\ ．\ \ \ \ \ \ \ \ \ \ |
|
|
|\ ..\ \ \ \ \ \ \ \ |\ ．．\ \ \ \ \ \ \ \ \ |
|
|
|
|
###\ Multipart\ uploads\ ###
|
|
|
|
rclone\ supports\ multipart\ uploads\ with\ S3\ which\ means\ that\ it\ can
|
|
upload\ files\ bigger\ than\ 5GB.
|
|
|
|
Note\ that\ files\ uploaded\ *both*\ with\ multipart\ upload\ *and*\ through
|
|
crypt\ remotes\ do\ not\ have\ MD5\ sums.
|
|
|
|
rclone\ switches\ from\ single\ part\ uploads\ to\ multipart\ uploads\ at\ the
|
|
point\ specified\ by\ `\-\-s3\-upload\-cutoff`.\ \ This\ can\ be\ a\ maximum\ of\ 5GB
|
|
and\ a\ minimum\ of\ 0\ (ie\ always\ upload\ multipart\ files).
|
|
|
|
The\ chunk\ sizes\ used\ in\ the\ multipart\ upload\ are\ specified\ by
|
|
`\-\-s3\-chunk\-size`\ and\ the\ number\ of\ chunks\ uploaded\ concurrently\ is
|
|
specified\ by\ `\-\-s3\-upload\-concurrency`.
|
|
|
|
Multipart\ uploads\ will\ use\ `\-\-transfers`\ *\ `\-\-s3\-upload\-concurrency`\ *
|
|
`\-\-s3\-chunk\-size`\ extra\ memory.\ \ Single\ part\ uploads\ to\ not\ use\ extra
|
|
memory.
|
|
|
|
Single\ part\ transfers\ can\ be\ faster\ than\ multipart\ transfers\ or\ slower
|
|
depending\ on\ your\ latency\ from\ S3\ \-\ the\ more\ latency,\ the\ more\ likely
|
|
single\ part\ transfers\ will\ be\ faster.
|
|
|
|
Increasing\ `\-\-s3\-upload\-concurrency`\ will\ increase\ throughput\ (8\ would
|
|
be\ a\ sensible\ value)\ and\ increasing\ `\-\-s3\-chunk\-size`\ also\ increases
|
|
throughput\ (16M\ would\ be\ sensible).\ \ Increasing\ either\ of\ these\ will
|
|
use\ more\ memory.\ \ The\ default\ values\ are\ high\ enough\ to\ gain\ most\ of
|
|
the\ possible\ performance\ without\ using\ too\ much\ memory.
|
|
|
|
|
|
###\ Buckets\ and\ Regions\ ###
|
|
|
|
With\ Amazon\ S3\ you\ can\ list\ buckets\ (`rclone\ lsd`)\ using\ any\ region,
|
|
but\ you\ can\ only\ access\ the\ content\ of\ a\ bucket\ from\ the\ region\ it\ was
|
|
created\ in.\ \ If\ you\ attempt\ to\ access\ a\ bucket\ from\ the\ wrong\ region,
|
|
you\ will\ get\ an\ error,\ `incorrect\ region,\ the\ bucket\ is\ not\ in\ \[aq]XXX\[aq]
|
|
region`.
|
|
|
|
###\ Authentication\ ###
|
|
|
|
There\ are\ a\ number\ of\ ways\ to\ supply\ `rclone`\ with\ a\ set\ of\ AWS
|
|
credentials,\ with\ and\ without\ using\ the\ environment.
|
|
|
|
The\ different\ authentication\ methods\ are\ tried\ in\ this\ order:
|
|
|
|
\ \-\ Directly\ in\ the\ rclone\ configuration\ file\ (`env_auth\ =\ false`\ in\ the\ config\ file):
|
|
\ \ \ \-\ `access_key_id`\ and\ `secret_access_key`\ are\ required.
|
|
\ \ \ \-\ `session_token`\ can\ be\ optionally\ set\ when\ using\ AWS\ STS.
|
|
\ \-\ Runtime\ configuration\ (`env_auth\ =\ true`\ in\ the\ config\ file):
|
|
\ \ \ \-\ Export\ the\ following\ environment\ variables\ before\ running\ `rclone`:
|
|
\ \ \ \ \ \-\ Access\ Key\ ID:\ `AWS_ACCESS_KEY_ID`\ or\ `AWS_ACCESS_KEY`
|
|
\ \ \ \ \ \-\ Secret\ Access\ Key:\ `AWS_SECRET_ACCESS_KEY`\ or\ `AWS_SECRET_KEY`
|
|
\ \ \ \ \ \-\ Session\ Token:\ `AWS_SESSION_TOKEN`\ (optional)
|
|
\ \ \ \-\ Or,\ use\ a\ [named\ profile](https://docs.aws.amazon.com/cli/latest/userguide/cli\-multiple\-profiles.html):
|
|
\ \ \ \ \ \-\ Profile\ files\ are\ standard\ files\ used\ by\ AWS\ CLI\ tools
|
|
\ \ \ \ \ \-\ By\ default\ it\ will\ use\ the\ profile\ in\ your\ home\ directory\ (eg\ `~/.aws/credentials`\ on\ unix\ based\ systems)\ file\ and\ the\ "default"\ profile,\ to\ change\ set\ these\ environment\ variables:
|
|
\ \ \ \ \ \ \ \ \ \-\ `AWS_SHARED_CREDENTIALS_FILE`\ to\ control\ which\ file.
|
|
\ \ \ \ \ \ \ \ \ \-\ `AWS_PROFILE`\ to\ control\ which\ profile\ to\ use.
|
|
\ \ \ \-\ Or,\ run\ `rclone`\ in\ an\ ECS\ task\ with\ an\ IAM\ role\ (AWS\ only).
|
|
\ \ \ \-\ Or,\ run\ `rclone`\ on\ an\ EC2\ instance\ with\ an\ IAM\ role\ (AWS\ only).
|
|
|
|
If\ none\ of\ these\ option\ actually\ end\ up\ providing\ `rclone`\ with\ AWS
|
|
credentials\ then\ S3\ interaction\ will\ be\ non\-authenticated\ (see\ below).
|
|
|
|
###\ S3\ Permissions\ ###
|
|
|
|
When\ using\ the\ `sync`\ subcommand\ of\ `rclone`\ the\ following\ minimum
|
|
permissions\ are\ required\ to\ be\ available\ on\ the\ bucket\ being\ written\ to:
|
|
|
|
*\ `ListBucket`
|
|
*\ `DeleteObject`
|
|
*\ `GetObject`
|
|
*\ `PutObject`
|
|
*\ `PutObjectACL`
|
|
|
|
When\ using\ the\ `lsd`\ subcommand,\ the\ `ListAllMyBuckets`\ permission\ is\ required.
|
|
|
|
Example\ policy:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]Version\[rq]: \[lq]2012\-10\-17\[rq], \[lq]Statement\[rq]: [ {
|
|
\[lq]Effect\[rq]: \[lq]Allow\[rq], \[lq]Principal\[rq]: { \[lq]AWS\[rq]:
|
|
\[lq]arn:aws:iam::USER_SID:user/USER_NAME\[rq] }, \[lq]Action\[rq]: [
|
|
\[lq]s3:ListBucket\[rq], \[lq]s3:DeleteObject\[rq],
|
|
\[lq]s3:GetObject\[rq], \[lq]s3:PutObject\[rq],
|
|
\[lq]s3:PutObjectAcl\[rq] ], \[lq]Resource\[rq]: [
|
|
"arn:aws:s3:::BUCKET_NAME/*\[lq],\[rq]arn:aws:s3:::BUCKET_NAME" ] }, {
|
|
\[lq]Effect\[rq]: \[lq]Allow\[rq], \[lq]Action\[rq]:
|
|
\[lq]s3:ListAllMyBuckets\[rq], \[lq]Resource\[rq]: "arn:aws:s3:::*" }
|
|
.PD 0
|
|
.P
|
|
.PD
|
|
] }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Notes\ on\ above:
|
|
|
|
1.\ This\ is\ a\ policy\ that\ can\ be\ used\ when\ creating\ bucket.\ It\ assumes
|
|
\ \ \ that\ `USER_NAME`\ has\ been\ created.
|
|
2.\ The\ Resource\ entry\ must\ include\ both\ resource\ ARNs,\ as\ one\ implies
|
|
\ \ \ the\ bucket\ and\ the\ other\ implies\ the\ bucket\[aq]s\ objects.
|
|
|
|
For\ reference,\ [here\[aq]s\ an\ Ansible\ script](https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
|
|
that\ will\ generate\ one\ or\ more\ buckets\ that\ will\ work\ with\ `rclone\ sync`.
|
|
|
|
###\ Key\ Management\ System\ (KMS)\ ###
|
|
|
|
If\ you\ are\ using\ server\ side\ encryption\ with\ KMS\ then\ you\ will\ find
|
|
you\ can\[aq]t\ transfer\ small\ objects.\ \ As\ a\ work\-around\ you\ can\ use\ the
|
|
`\-\-ignore\-checksum`\ flag.
|
|
|
|
A\ proper\ fix\ is\ being\ worked\ on\ in\ [issue\ #1824](https://github.com/rclone/rclone/issues/1824).
|
|
|
|
###\ Glacier\ and\ Glacier\ Deep\ Archive\ ###
|
|
|
|
You\ can\ upload\ objects\ using\ the\ glacier\ storage\ class\ or\ transition\ them\ to\ glacier\ using\ a\ [lifecycle\ policy](http://docs.aws.amazon.com/AmazonS3/latest/user\-guide/create\-lifecycle.html).
|
|
The\ bucket\ can\ still\ be\ synced\ or\ copied\ into\ normally,\ but\ if\ rclone
|
|
tries\ to\ access\ data\ from\ the\ glacier\ storage\ class\ you\ will\ see\ an\ error\ like\ below.
|
|
|
|
\ \ \ \ 2017/09/11\ 19:07:43\ Failed\ to\ sync:\ failed\ to\ open\ source\ object:\ Object\ in\ GLACIER,\ restore\ first:\ path/to/file
|
|
|
|
In\ this\ case\ you\ need\ to\ [restore](http://docs.aws.amazon.com/AmazonS3/latest/user\-guide/restore\-archived\-objects.html)
|
|
the\ object(s)\ in\ question\ before\ using\ rclone.
|
|
|
|
Note\ that\ rclone\ only\ speaks\ the\ S3\ API\ it\ does\ not\ speak\ the\ Glacier
|
|
Vault\ API,\ so\ rclone\ cannot\ directly\ access\ Glacier\ Vaults.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/s3/s3.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ s3\ (Amazon\ S3\ Compliant\ Storage\ Provider\ (AWS,\ Alibaba,\ Ceph,\ Digital\ Ocean,\ Dreamhost,\ IBM\ COS,\ Minio,\ etc)).
|
|
|
|
####\ \-\-s3\-provider
|
|
|
|
Choose\ your\ S3\ provider.
|
|
|
|
\-\ Config:\ \ \ \ \ \ provider
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_PROVIDER
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "AWS"
|
|
\ \ \ \ \ \ \ \ \-\ Amazon\ Web\ Services\ (AWS)\ S3
|
|
\ \ \ \ \-\ "Alibaba"
|
|
\ \ \ \ \ \ \ \ \-\ Alibaba\ Cloud\ Object\ Storage\ System\ (OSS)\ formerly\ Aliyun
|
|
\ \ \ \ \-\ "Ceph"
|
|
\ \ \ \ \ \ \ \ \-\ Ceph\ Object\ Storage
|
|
\ \ \ \ \-\ "DigitalOcean"
|
|
\ \ \ \ \ \ \ \ \-\ Digital\ Ocean\ Spaces
|
|
\ \ \ \ \-\ "Dreamhost"
|
|
\ \ \ \ \ \ \ \ \-\ Dreamhost\ DreamObjects
|
|
\ \ \ \ \-\ "IBMCOS"
|
|
\ \ \ \ \ \ \ \ \-\ IBM\ COS\ S3
|
|
\ \ \ \ \-\ "Minio"
|
|
\ \ \ \ \ \ \ \ \-\ Minio\ Object\ Storage
|
|
\ \ \ \ \-\ "Netease"
|
|
\ \ \ \ \ \ \ \ \-\ Netease\ Object\ Storage\ (NOS)
|
|
\ \ \ \ \-\ "Wasabi"
|
|
\ \ \ \ \ \ \ \ \-\ Wasabi\ Object\ Storage
|
|
\ \ \ \ \-\ "Other"
|
|
\ \ \ \ \ \ \ \ \-\ Any\ other\ S3\ compatible\ provider
|
|
|
|
####\ \-\-s3\-env\-auth
|
|
|
|
Get\ AWS\ credentials\ from\ runtime\ (environment\ variables\ or\ EC2/ECS\ meta\ data\ if\ no\ env\ vars).
|
|
Only\ applies\ if\ access_key_id\ and\ secret_access_key\ is\ blank.
|
|
|
|
\-\ Config:\ \ \ \ \ \ env_auth
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ENV_AUTH
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "false"
|
|
\ \ \ \ \ \ \ \ \-\ Enter\ AWS\ credentials\ in\ the\ next\ step
|
|
\ \ \ \ \-\ "true"
|
|
\ \ \ \ \ \ \ \ \-\ Get\ AWS\ credentials\ from\ the\ environment\ (env\ vars\ or\ IAM)
|
|
|
|
####\ \-\-s3\-access\-key\-id
|
|
|
|
AWS\ Access\ Key\ ID.
|
|
Leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
|
|
|
\-\ Config:\ \ \ \ \ \ access_key_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ACCESS_KEY_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-s3\-secret\-access\-key
|
|
|
|
AWS\ Secret\ Access\ Key\ (password)
|
|
Leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
|
|
|
\-\ Config:\ \ \ \ \ \ secret_access_key
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_SECRET_ACCESS_KEY
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-s3\-region
|
|
|
|
Region\ to\ connect\ to.
|
|
|
|
\-\ Config:\ \ \ \ \ \ region
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_REGION
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "us\-east\-1"
|
|
\ \ \ \ \ \ \ \ \-\ The\ default\ endpoint\ \-\ a\ good\ choice\ if\ you\ are\ unsure.
|
|
\ \ \ \ \ \ \ \ \-\ US\ Region,\ Northern\ Virginia\ or\ Pacific\ Northwest.
|
|
\ \ \ \ \ \ \ \ \-\ Leave\ location\ constraint\ empty.
|
|
\ \ \ \ \-\ "us\-east\-2"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ (Ohio)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ us\-east\-2.
|
|
\ \ \ \ \-\ "us\-west\-2"
|
|
\ \ \ \ \ \ \ \ \-\ US\ West\ (Oregon)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ us\-west\-2.
|
|
\ \ \ \ \-\ "us\-west\-1"
|
|
\ \ \ \ \ \ \ \ \-\ US\ West\ (Northern\ California)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ us\-west\-1.
|
|
\ \ \ \ \-\ "ca\-central\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Canada\ (Central)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ ca\-central\-1.
|
|
\ \ \ \ \-\ "eu\-west\-1"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (Ireland)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ EU\ or\ eu\-west\-1.
|
|
\ \ \ \ \-\ "eu\-west\-2"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (London)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ eu\-west\-2.
|
|
\ \ \ \ \-\ "eu\-north\-1"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (Stockholm)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ eu\-north\-1.
|
|
\ \ \ \ \-\ "eu\-central\-1"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (Frankfurt)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ eu\-central\-1.
|
|
\ \ \ \ \-\ "ap\-southeast\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Singapore)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ ap\-southeast\-1.
|
|
\ \ \ \ \-\ "ap\-southeast\-2"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Sydney)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ ap\-southeast\-2.
|
|
\ \ \ \ \-\ "ap\-northeast\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Tokyo)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ ap\-northeast\-1.
|
|
\ \ \ \ \-\ "ap\-northeast\-2"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Seoul)
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ ap\-northeast\-2.
|
|
\ \ \ \ \-\ "ap\-south\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Mumbai)
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ ap\-south\-1.
|
|
\ \ \ \ \-\ "sa\-east\-1"
|
|
\ \ \ \ \ \ \ \ \-\ South\ America\ (Sao\ Paulo)\ Region
|
|
\ \ \ \ \ \ \ \ \-\ Needs\ location\ constraint\ sa\-east\-1.
|
|
|
|
####\ \-\-s3\-region
|
|
|
|
Region\ to\ connect\ to.
|
|
Leave\ blank\ if\ you\ are\ using\ an\ S3\ clone\ and\ you\ don\[aq]t\ have\ a\ region.
|
|
|
|
\-\ Config:\ \ \ \ \ \ region
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_REGION
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Use\ this\ if\ unsure.\ Will\ use\ v4\ signatures\ and\ an\ empty\ region.
|
|
\ \ \ \ \-\ "other\-v2\-signature"
|
|
\ \ \ \ \ \ \ \ \-\ Use\ this\ only\ if\ v4\ signatures\ don\[aq]t\ work,\ eg\ pre\ Jewel/v10\ CEPH.
|
|
|
|
####\ \-\-s3\-endpoint
|
|
|
|
Endpoint\ for\ S3\ API.
|
|
Leave\ blank\ if\ using\ AWS\ to\ use\ the\ default\ endpoint\ for\ the\ region.
|
|
|
|
\-\ Config:\ \ \ \ \ \ endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-s3\-endpoint
|
|
|
|
Endpoint\ for\ IBM\ COS\ S3\ API.
|
|
Specify\ if\ using\ an\ IBM\ COS\ On\ Premise.
|
|
|
|
\-\ Config:\ \ \ \ \ \ endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "s3\-api.us\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.dal.us\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Dallas\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.wdc\-us\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Washington\ DC\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.sjc\-us\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ San\ Jose\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.us\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.dal\-us\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Dallas\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.wdc\-us\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Washington\ DC\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3\-api.sjc\-us\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ San\ Jose\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.us\-east.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Region\ East\ Endpoint
|
|
\ \ \ \ \-\ "s3.us\-east.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Region\ East\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.us\-south.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Region\ South\ Endpoint
|
|
\ \ \ \ \-\ "s3.us\-south.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Region\ South\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.eu\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Endpoint
|
|
\ \ \ \ \-\ "s3.fra\-eu\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Frankfurt\ Endpoint
|
|
\ \ \ \ \-\ "s3.mil\-eu\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Milan\ Endpoint
|
|
\ \ \ \ \-\ "s3.ams\-eu\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Amsterdam\ Endpoint
|
|
\ \ \ \ \-\ "s3.eu\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.fra\-eu\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Frankfurt\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.mil\-eu\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Milan\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.ams\-eu\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Amsterdam\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.eu\-gb.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ Great\ Britain\ Endpoint
|
|
\ \ \ \ \-\ "s3.eu\-gb.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ Great\ Britain\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.ap\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ Endpoint
|
|
\ \ \ \ \-\ "s3.tok\-ap\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ Tokyo\ Endpoint
|
|
\ \ \ \ \-\ "s3.hkg\-ap\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ HongKong\ Endpoint
|
|
\ \ \ \ \-\ "s3.seo\-ap\-geo.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ Seoul\ Endpoint
|
|
\ \ \ \ \-\ "s3.ap\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.tok\-ap\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ Tokyo\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.hkg\-ap\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ HongKong\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.seo\-ap\-geo.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cross\ Regional\ Seoul\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.mel01.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ Melbourne\ Single\ Site\ Endpoint
|
|
\ \ \ \ \-\ "s3.mel01.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ Melbourne\ Single\ Site\ Private\ Endpoint
|
|
\ \ \ \ \-\ "s3.tor01.objectstorage.softlayer.net"
|
|
\ \ \ \ \ \ \ \ \-\ Toronto\ Single\ Site\ Endpoint
|
|
\ \ \ \ \-\ "s3.tor01.objectstorage.service.networklayer.com"
|
|
\ \ \ \ \ \ \ \ \-\ Toronto\ Single\ Site\ Private\ Endpoint
|
|
|
|
####\ \-\-s3\-endpoint
|
|
|
|
Endpoint\ for\ OSS\ API.
|
|
|
|
\-\ Config:\ \ \ \ \ \ endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "oss\-cn\-hangzhou.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ East\ China\ 1\ (Hangzhou)
|
|
\ \ \ \ \-\ "oss\-cn\-shanghai.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ East\ China\ 2\ (Shanghai)
|
|
\ \ \ \ \-\ "oss\-cn\-qingdao.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ North\ China\ 1\ (Qingdao)
|
|
\ \ \ \ \-\ "oss\-cn\-beijing.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ North\ China\ 2\ (Beijing)
|
|
\ \ \ \ \-\ "oss\-cn\-zhangjiakou.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ North\ China\ 3\ (Zhangjiakou)
|
|
\ \ \ \ \-\ "oss\-cn\-huhehaote.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ North\ China\ 5\ (Huhehaote)
|
|
\ \ \ \ \-\ "oss\-cn\-shenzhen.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ South\ China\ 1\ (Shenzhen)
|
|
\ \ \ \ \-\ "oss\-cn\-hongkong.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Hong\ Kong\ (Hong\ Kong)
|
|
\ \ \ \ \-\ "oss\-us\-west\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ West\ 1\ (Silicon\ Valley)
|
|
\ \ \ \ \-\ "oss\-us\-east\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ 1\ (Virginia)
|
|
\ \ \ \ \-\ "oss\-ap\-southeast\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Southeast\ Asia\ Southeast\ 1\ (Singapore)
|
|
\ \ \ \ \-\ "oss\-ap\-southeast\-2.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ Southeast\ 2\ (Sydney)
|
|
\ \ \ \ \-\ "oss\-ap\-southeast\-3.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Southeast\ Asia\ Southeast\ 3\ (Kuala\ Lumpur)
|
|
\ \ \ \ \-\ "oss\-ap\-southeast\-5.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ Southeast\ 5\ (Jakarta)
|
|
\ \ \ \ \-\ "oss\-ap\-northeast\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ Northeast\ 1\ (Japan)
|
|
\ \ \ \ \-\ "oss\-ap\-south\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ South\ 1\ (Mumbai)
|
|
\ \ \ \ \-\ "oss\-eu\-central\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Central\ Europe\ 1\ (Frankfurt)
|
|
\ \ \ \ \-\ "oss\-eu\-west\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ West\ Europe\ (London)
|
|
\ \ \ \ \-\ "oss\-me\-east\-1.aliyuncs.com"
|
|
\ \ \ \ \ \ \ \ \-\ Middle\ East\ 1\ (Dubai)
|
|
|
|
####\ \-\-s3\-endpoint
|
|
|
|
Endpoint\ for\ S3\ API.
|
|
Required\ when\ using\ an\ S3\ clone.
|
|
|
|
\-\ Config:\ \ \ \ \ \ endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "objects\-us\-east\-1.dream.io"
|
|
\ \ \ \ \ \ \ \ \-\ Dream\ Objects\ endpoint
|
|
\ \ \ \ \-\ "nyc3.digitaloceanspaces.com"
|
|
\ \ \ \ \ \ \ \ \-\ Digital\ Ocean\ Spaces\ New\ York\ 3
|
|
\ \ \ \ \-\ "ams3.digitaloceanspaces.com"
|
|
\ \ \ \ \ \ \ \ \-\ Digital\ Ocean\ Spaces\ Amsterdam\ 3
|
|
\ \ \ \ \-\ "sgp1.digitaloceanspaces.com"
|
|
\ \ \ \ \ \ \ \ \-\ Digital\ Ocean\ Spaces\ Singapore\ 1
|
|
\ \ \ \ \-\ "s3.wasabisys.com"
|
|
\ \ \ \ \ \ \ \ \-\ Wasabi\ US\ East\ endpoint
|
|
\ \ \ \ \-\ "s3.us\-west\-1.wasabisys.com"
|
|
\ \ \ \ \ \ \ \ \-\ Wasabi\ US\ West\ endpoint
|
|
\ \ \ \ \-\ "s3.eu\-central\-1.wasabisys.com"
|
|
\ \ \ \ \ \ \ \ \-\ Wasabi\ EU\ Central\ endpoint
|
|
|
|
####\ \-\-s3\-location\-constraint
|
|
|
|
Location\ constraint\ \-\ must\ be\ set\ to\ match\ the\ Region.
|
|
Used\ when\ creating\ buckets\ only.
|
|
|
|
\-\ Config:\ \ \ \ \ \ location_constraint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_LOCATION_CONSTRAINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Empty\ for\ US\ Region,\ Northern\ Virginia\ or\ Pacific\ Northwest.
|
|
\ \ \ \ \-\ "us\-east\-2"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ (Ohio)\ Region.
|
|
\ \ \ \ \-\ "us\-west\-2"
|
|
\ \ \ \ \ \ \ \ \-\ US\ West\ (Oregon)\ Region.
|
|
\ \ \ \ \-\ "us\-west\-1"
|
|
\ \ \ \ \ \ \ \ \-\ US\ West\ (Northern\ California)\ Region.
|
|
\ \ \ \ \-\ "ca\-central\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Canada\ (Central)\ Region.
|
|
\ \ \ \ \-\ "eu\-west\-1"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (Ireland)\ Region.
|
|
\ \ \ \ \-\ "eu\-west\-2"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (London)\ Region.
|
|
\ \ \ \ \-\ "eu\-north\-1"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ (Stockholm)\ Region.
|
|
\ \ \ \ \-\ "EU"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Region.
|
|
\ \ \ \ \-\ "ap\-southeast\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Singapore)\ Region.
|
|
\ \ \ \ \-\ "ap\-southeast\-2"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Sydney)\ Region.
|
|
\ \ \ \ \-\ "ap\-northeast\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Tokyo)\ Region.
|
|
\ \ \ \ \-\ "ap\-northeast\-2"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Seoul)
|
|
\ \ \ \ \-\ "ap\-south\-1"
|
|
\ \ \ \ \ \ \ \ \-\ Asia\ Pacific\ (Mumbai)
|
|
\ \ \ \ \-\ "sa\-east\-1"
|
|
\ \ \ \ \ \ \ \ \-\ South\ America\ (Sao\ Paulo)\ Region.
|
|
|
|
####\ \-\-s3\-location\-constraint
|
|
|
|
Location\ constraint\ \-\ must\ match\ endpoint\ when\ using\ IBM\ Cloud\ Public.
|
|
For\ on\-prem\ COS,\ do\ not\ make\ a\ selection\ from\ this\ list,\ hit\ enter
|
|
|
|
\-\ Config:\ \ \ \ \ \ location_constraint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_LOCATION_CONSTRAINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "us\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Standard
|
|
\ \ \ \ \-\ "us\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Vault
|
|
\ \ \ \ \-\ "us\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Cold
|
|
\ \ \ \ \-\ "us\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ US\ Cross\ Region\ Flex
|
|
\ \ \ \ \-\ "us\-east\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ Region\ Standard
|
|
\ \ \ \ \-\ "us\-east\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ Region\ Vault
|
|
\ \ \ \ \-\ "us\-east\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ Region\ Cold
|
|
\ \ \ \ \-\ "us\-east\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ US\ East\ Region\ Flex
|
|
\ \ \ \ \-\ "us\-south\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ US\ South\ Region\ Standard
|
|
\ \ \ \ \-\ "us\-south\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ US\ South\ Region\ Vault
|
|
\ \ \ \ \-\ "us\-south\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ US\ South\ Region\ Cold
|
|
\ \ \ \ \-\ "us\-south\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ US\ South\ Region\ Flex
|
|
\ \ \ \ \-\ "eu\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Standard
|
|
\ \ \ \ \-\ "eu\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Vault
|
|
\ \ \ \ \-\ "eu\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Cold
|
|
\ \ \ \ \-\ "eu\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ EU\ Cross\ Region\ Flex
|
|
\ \ \ \ \-\ "eu\-gb\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ Great\ Britain\ Standard
|
|
\ \ \ \ \-\ "eu\-gb\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ Great\ Britain\ Vault
|
|
\ \ \ \ \-\ "eu\-gb\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ Great\ Britain\ Cold
|
|
\ \ \ \ \-\ "eu\-gb\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ Great\ Britain\ Flex
|
|
\ \ \ \ \-\ "ap\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Standard
|
|
\ \ \ \ \-\ "ap\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Vault
|
|
\ \ \ \ \-\ "ap\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Cold
|
|
\ \ \ \ \-\ "ap\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ APAC\ Flex
|
|
\ \ \ \ \-\ "mel01\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ Melbourne\ Standard
|
|
\ \ \ \ \-\ "mel01\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ Melbourne\ Vault
|
|
\ \ \ \ \-\ "mel01\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ Melbourne\ Cold
|
|
\ \ \ \ \-\ "mel01\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ Melbourne\ Flex
|
|
\ \ \ \ \-\ "tor01\-standard"
|
|
\ \ \ \ \ \ \ \ \-\ Toronto\ Standard
|
|
\ \ \ \ \-\ "tor01\-vault"
|
|
\ \ \ \ \ \ \ \ \-\ Toronto\ Vault
|
|
\ \ \ \ \-\ "tor01\-cold"
|
|
\ \ \ \ \ \ \ \ \-\ Toronto\ Cold
|
|
\ \ \ \ \-\ "tor01\-flex"
|
|
\ \ \ \ \ \ \ \ \-\ Toronto\ Flex
|
|
|
|
####\ \-\-s3\-location\-constraint
|
|
|
|
Location\ constraint\ \-\ must\ be\ set\ to\ match\ the\ Region.
|
|
Leave\ blank\ if\ not\ sure.\ Used\ when\ creating\ buckets\ only.
|
|
|
|
\-\ Config:\ \ \ \ \ \ location_constraint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_LOCATION_CONSTRAINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-s3\-acl
|
|
|
|
Canned\ ACL\ used\ when\ creating\ buckets\ and\ storing\ or\ copying\ objects.
|
|
|
|
This\ ACL\ is\ used\ for\ creating\ objects\ and\ if\ bucket_acl\ isn\[aq]t\ set,\ for\ creating\ buckets\ too.
|
|
|
|
For\ more\ info\ visit\ https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
|
|
Note\ that\ this\ ACL\ is\ applied\ when\ server\ side\ copying\ objects\ as\ S3
|
|
doesn\[aq]t\ copy\ the\ ACL\ from\ the\ source\ but\ rather\ writes\ a\ fresh\ one.
|
|
|
|
\-\ Config:\ \ \ \ \ \ acl
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_ACL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "private"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ No\ one\ else\ has\ access\ rights\ (default).
|
|
\ \ \ \ \-\ "public\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ access.
|
|
\ \ \ \ \-\ "public\-read\-write"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ and\ WRITE\ access.
|
|
\ \ \ \ \ \ \ \ \-\ Granting\ this\ on\ a\ bucket\ is\ generally\ not\ recommended.
|
|
\ \ \ \ \-\ "authenticated\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AuthenticatedUsers\ group\ gets\ READ\ access.
|
|
\ \ \ \ \-\ "bucket\-owner\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ FULL_CONTROL.\ Bucket\ owner\ gets\ READ\ access.
|
|
\ \ \ \ \ \ \ \ \-\ If\ you\ specify\ this\ canned\ ACL\ when\ creating\ a\ bucket,\ Amazon\ S3\ ignores\ it.
|
|
\ \ \ \ \-\ "bucket\-owner\-full\-control"
|
|
\ \ \ \ \ \ \ \ \-\ Both\ the\ object\ owner\ and\ the\ bucket\ owner\ get\ FULL_CONTROL\ over\ the\ object.
|
|
\ \ \ \ \ \ \ \ \-\ If\ you\ specify\ this\ canned\ ACL\ when\ creating\ a\ bucket,\ Amazon\ S3\ ignores\ it.
|
|
\ \ \ \ \-\ "private"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ No\ one\ else\ has\ access\ rights\ (default).\ This\ acl\ is\ available\ on\ IBM\ Cloud\ (Infra),\ IBM\ Cloud\ (Storage),\ On\-Premise\ COS
|
|
\ \ \ \ \-\ "public\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ access.\ This\ acl\ is\ available\ on\ IBM\ Cloud\ (Infra),\ IBM\ Cloud\ (Storage),\ On\-Premise\ IBM\ COS
|
|
\ \ \ \ \-\ "public\-read\-write"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ and\ WRITE\ access.\ This\ acl\ is\ available\ on\ IBM\ Cloud\ (Infra),\ On\-Premise\ IBM\ COS
|
|
\ \ \ \ \-\ "authenticated\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AuthenticatedUsers\ group\ gets\ READ\ access.\ Not\ supported\ on\ Buckets.\ This\ acl\ is\ available\ on\ IBM\ Cloud\ (Infra)\ and\ On\-Premise\ IBM\ COS
|
|
|
|
####\ \-\-s3\-server\-side\-encryption
|
|
|
|
The\ server\-side\ encryption\ algorithm\ used\ when\ storing\ this\ object\ in\ S3.
|
|
|
|
\-\ Config:\ \ \ \ \ \ server_side_encryption
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_SERVER_SIDE_ENCRYPTION
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ None
|
|
\ \ \ \ \-\ "AES256"
|
|
\ \ \ \ \ \ \ \ \-\ AES256
|
|
\ \ \ \ \-\ "aws:kms"
|
|
\ \ \ \ \ \ \ \ \-\ aws:kms
|
|
|
|
####\ \-\-s3\-sse\-kms\-key\-id
|
|
|
|
If\ using\ KMS\ ID\ you\ must\ provide\ the\ ARN\ of\ Key.
|
|
|
|
\-\ Config:\ \ \ \ \ \ sse_kms_key_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_SSE_KMS_KEY_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ None
|
|
\ \ \ \ \-\ "arn:aws:kms:us\-east\-1:*"
|
|
\ \ \ \ \ \ \ \ \-\ arn:aws:kms:*
|
|
|
|
####\ \-\-s3\-storage\-class
|
|
|
|
The\ storage\ class\ to\ use\ when\ storing\ new\ objects\ in\ S3.
|
|
|
|
\-\ Config:\ \ \ \ \ \ storage_class
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_STORAGE_CLASS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Default
|
|
\ \ \ \ \-\ "STANDARD"
|
|
\ \ \ \ \ \ \ \ \-\ Standard\ storage\ class
|
|
\ \ \ \ \-\ "REDUCED_REDUNDANCY"
|
|
\ \ \ \ \ \ \ \ \-\ Reduced\ redundancy\ storage\ class
|
|
\ \ \ \ \-\ "STANDARD_IA"
|
|
\ \ \ \ \ \ \ \ \-\ Standard\ Infrequent\ Access\ storage\ class
|
|
\ \ \ \ \-\ "ONEZONE_IA"
|
|
\ \ \ \ \ \ \ \ \-\ One\ Zone\ Infrequent\ Access\ storage\ class
|
|
\ \ \ \ \-\ "GLACIER"
|
|
\ \ \ \ \ \ \ \ \-\ Glacier\ storage\ class
|
|
\ \ \ \ \-\ "DEEP_ARCHIVE"
|
|
\ \ \ \ \ \ \ \ \-\ Glacier\ Deep\ Archive\ storage\ class
|
|
\ \ \ \ \-\ "INTELLIGENT_TIERING"
|
|
\ \ \ \ \ \ \ \ \-\ Intelligent\-Tiering\ storage\ class
|
|
|
|
####\ \-\-s3\-storage\-class
|
|
|
|
The\ storage\ class\ to\ use\ when\ storing\ new\ objects\ in\ OSS.
|
|
|
|
\-\ Config:\ \ \ \ \ \ storage_class
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_STORAGE_CLASS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Default
|
|
\ \ \ \ \-\ "STANDARD"
|
|
\ \ \ \ \ \ \ \ \-\ Standard\ storage\ class
|
|
\ \ \ \ \-\ "GLACIER"
|
|
\ \ \ \ \ \ \ \ \-\ Archive\ storage\ mode.
|
|
\ \ \ \ \-\ "STANDARD_IA"
|
|
\ \ \ \ \ \ \ \ \-\ Infrequent\ access\ storage\ mode.
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ s3\ (Amazon\ S3\ Compliant\ Storage\ Provider\ (AWS,\ Alibaba,\ Ceph,\ Digital\ Ocean,\ Dreamhost,\ IBM\ COS,\ Minio,\ etc)).
|
|
|
|
####\ \-\-s3\-bucket\-acl
|
|
|
|
Canned\ ACL\ used\ when\ creating\ buckets.
|
|
|
|
For\ more\ info\ visit\ https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
|
|
Note\ that\ this\ ACL\ is\ applied\ when\ only\ when\ creating\ buckets.\ \ If\ it
|
|
isn\[aq]t\ set\ then\ "acl"\ is\ used\ instead.
|
|
|
|
\-\ Config:\ \ \ \ \ \ bucket_acl
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_BUCKET_ACL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "private"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ No\ one\ else\ has\ access\ rights\ (default).
|
|
\ \ \ \ \-\ "public\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ access.
|
|
\ \ \ \ \-\ "public\-read\-write"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ and\ WRITE\ access.
|
|
\ \ \ \ \ \ \ \ \-\ Granting\ this\ on\ a\ bucket\ is\ generally\ not\ recommended.
|
|
\ \ \ \ \-\ "authenticated\-read"
|
|
\ \ \ \ \ \ \ \ \-\ Owner\ gets\ FULL_CONTROL.\ The\ AuthenticatedUsers\ group\ gets\ READ\ access.
|
|
|
|
####\ \-\-s3\-upload\-cutoff
|
|
|
|
Cutoff\ for\ switching\ to\ chunked\ upload
|
|
|
|
Any\ files\ larger\ than\ this\ will\ be\ uploaded\ in\ chunks\ of\ chunk_size.
|
|
The\ minimum\ is\ 0\ and\ the\ maximum\ is\ 5GB.
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_cutoff
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_UPLOAD_CUTOFF
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 200M
|
|
|
|
####\ \-\-s3\-chunk\-size
|
|
|
|
Chunk\ size\ to\ use\ for\ uploading.
|
|
|
|
When\ uploading\ files\ larger\ than\ upload_cutoff\ they\ will\ be\ uploaded
|
|
as\ multipart\ uploads\ using\ this\ chunk\ size.
|
|
|
|
Note\ that\ "\-\-s3\-upload\-concurrency"\ chunks\ of\ this\ size\ are\ buffered
|
|
in\ memory\ per\ transfer.
|
|
|
|
If\ you\ are\ transferring\ large\ files\ over\ high\ speed\ links\ and\ you\ have
|
|
enough\ memory,\ then\ increasing\ this\ will\ speed\ up\ the\ transfers.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 5M
|
|
|
|
####\ \-\-s3\-disable\-checksum
|
|
|
|
Don\[aq]t\ store\ MD5\ checksum\ with\ object\ metadata
|
|
|
|
\-\ Config:\ \ \ \ \ \ disable_checksum
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_DISABLE_CHECKSUM
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-s3\-session\-token
|
|
|
|
An\ AWS\ session\ token
|
|
|
|
\-\ Config:\ \ \ \ \ \ session_token
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_SESSION_TOKEN
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-s3\-upload\-concurrency
|
|
|
|
Concurrency\ for\ multipart\ uploads.
|
|
|
|
This\ is\ the\ number\ of\ chunks\ of\ the\ same\ file\ that\ are\ uploaded
|
|
concurrently.
|
|
|
|
If\ you\ are\ uploading\ small\ numbers\ of\ large\ file\ over\ high\ speed\ link
|
|
and\ these\ uploads\ do\ not\ fully\ utilize\ your\ bandwidth,\ then\ increasing
|
|
this\ may\ help\ to\ speed\ up\ the\ transfers.
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_concurrency
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_UPLOAD_CONCURRENCY
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 4
|
|
|
|
####\ \-\-s3\-force\-path\-style
|
|
|
|
If\ true\ use\ path\ style\ access\ if\ false\ use\ virtual\ hosted\ style.
|
|
|
|
If\ this\ is\ true\ (the\ default)\ then\ rclone\ will\ use\ path\ style\ access,
|
|
if\ false\ then\ rclone\ will\ use\ virtual\ path\ style.\ See\ [the\ AWS\ S3
|
|
docs](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access\-bucket\-intro)
|
|
for\ more\ info.
|
|
|
|
Some\ providers\ (eg\ Aliyun\ OSS\ or\ Netease\ COS)\ require\ this\ set\ to\ false.
|
|
|
|
\-\ Config:\ \ \ \ \ \ force_path_style
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_FORCE_PATH_STYLE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ true
|
|
|
|
####\ \-\-s3\-v2\-auth
|
|
|
|
If\ true\ use\ v2\ authentication.
|
|
|
|
If\ this\ is\ false\ (the\ default)\ then\ rclone\ will\ use\ v4\ authentication.
|
|
If\ it\ is\ set\ then\ rclone\ will\ use\ v2\ authentication.
|
|
|
|
Use\ this\ only\ if\ v4\ signatures\ don\[aq]t\ work,\ eg\ pre\ Jewel/v10\ CEPH.
|
|
|
|
\-\ Config:\ \ \ \ \ \ v2_auth
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_V2_AUTH
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-s3\-use\-accelerate\-endpoint
|
|
|
|
If\ true\ use\ the\ AWS\ S3\ accelerated\ endpoint.
|
|
|
|
See:\ [AWS\ S3\ Transfer\ acceleration](https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer\-acceleration\-examples.html)
|
|
|
|
\-\ Config:\ \ \ \ \ \ use_accelerate_endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_USE_ACCELERATE_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-s3\-leave\-parts\-on\-error
|
|
|
|
If\ true\ avoid\ calling\ abort\ upload\ on\ a\ failure,\ leaving\ all\ successfully\ uploaded\ parts\ on\ S3\ for\ manual\ recovery.
|
|
|
|
It\ should\ be\ set\ to\ true\ for\ resuming\ uploads\ across\ different\ sessions.
|
|
|
|
WARNING:\ Storing\ parts\ of\ an\ incomplete\ multipart\ upload\ counts\ towards\ space\ usage\ on\ S3\ and\ will\ add\ additional\ costs\ if\ not\ cleaned\ up.
|
|
|
|
|
|
\-\ Config:\ \ \ \ \ \ leave_parts_on_error
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_S3_LEAVE_PARTS_ON_ERROR
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
###\ Anonymous\ access\ to\ public\ buckets\ ###
|
|
|
|
If\ you\ want\ to\ use\ rclone\ to\ access\ a\ public\ bucket,\ configure\ with\ a
|
|
blank\ `access_key_id`\ and\ `secret_access_key`.\ \ Your\ config\ should\ end
|
|
up\ looking\ like\ this:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[anons3] type = s3 provider = AWS env_auth = false access_key_id =
|
|
secret_access_key = region = us\-east\-1 endpoint = location_constraint
|
|
= acl = private server_side_encryption = storage_class =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Then\ use\ it\ as\ normal\ with\ the\ name\ of\ the\ public\ bucket,\ eg
|
|
|
|
\ \ \ \ rclone\ lsd\ anons3:1000genomes
|
|
|
|
You\ will\ be\ able\ to\ list\ and\ copy\ data\ but\ not\ upload\ it.
|
|
|
|
###\ Ceph\ ###
|
|
|
|
[Ceph](https://ceph.com/)\ is\ an\ open\ source\ unified,\ distributed
|
|
storage\ system\ designed\ for\ excellent\ performance,\ reliability\ and
|
|
scalability.\ \ It\ has\ an\ S3\ compatible\ object\ storage\ interface.
|
|
|
|
To\ use\ rclone\ with\ Ceph,\ configure\ as\ above\ but\ leave\ the\ region\ blank
|
|
and\ set\ the\ endpoint.\ \ You\ should\ end\ up\ with\ something\ like\ this\ in
|
|
your\ config:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[ceph] type = s3 provider = Ceph env_auth = false access_key_id = XXX
|
|
secret_access_key = YYY region = endpoint =
|
|
https://ceph.endpoint.example.com location_constraint = acl =
|
|
server_side_encryption = storage_class =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
If\ you\ are\ using\ an\ older\ version\ of\ CEPH,\ eg\ 10.2.x\ Jewel,\ then\ you
|
|
may\ need\ to\ supply\ the\ parameter\ `\-\-s3\-upload\-cutoff\ 0`\ or\ put\ this\ in
|
|
the\ config\ file\ as\ `upload_cutoff\ 0`\ to\ work\ around\ a\ bug\ which\ causes
|
|
uploading\ of\ small\ files\ to\ fail.
|
|
|
|
Note\ also\ that\ Ceph\ sometimes\ puts\ `/`\ in\ the\ passwords\ it\ gives
|
|
users.\ \ If\ you\ read\ the\ secret\ access\ key\ using\ the\ command\ line\ tools
|
|
you\ will\ get\ a\ JSON\ blob\ with\ the\ `/`\ escaped\ as\ `\\/`.\ \ Make\ sure\ you
|
|
only\ write\ `/`\ in\ the\ secret\ access\ key.
|
|
|
|
Eg\ the\ dump\ from\ Ceph\ looks\ something\ like\ this\ (irrelevant\ keys
|
|
removed).
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
{ \[lq]user_id\[rq]: \[lq]xxx\[rq], \[lq]display_name\[rq]:
|
|
\[lq]xxxx\[rq], \[lq]keys\[rq]: [ { \[lq]user\[rq]: \[lq]xxx\[rq],
|
|
\[lq]access_key\[rq]: \[lq]xxxxxx\[rq], \[lq]secret_key\[rq]:
|
|
\[lq]xxxxxx/xxxx\[rq] } ], }
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Because\ this\ is\ a\ json\ dump,\ it\ is\ encoding\ the\ `/`\ as\ `\\/`,\ so\ if\ you
|
|
use\ the\ secret\ key\ as\ `xxxxxx/xxxx`\ \ it\ will\ work\ fine.
|
|
|
|
###\ Dreamhost\ ###
|
|
|
|
Dreamhost\ [DreamObjects](https://www.dreamhost.com/cloud/storage/)\ is
|
|
an\ object\ storage\ system\ based\ on\ CEPH.
|
|
|
|
To\ use\ rclone\ with\ Dreamhost,\ configure\ as\ above\ but\ leave\ the\ region\ blank
|
|
and\ set\ the\ endpoint.\ \ You\ should\ end\ up\ with\ something\ like\ this\ in
|
|
your\ config:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[dreamobjects] type = s3 provider = DreamHost env_auth = false
|
|
access_key_id = your_access_key secret_access_key = your_secret_key
|
|
region = endpoint = objects\-us\-west\-1.dream.io location_constraint =
|
|
acl = private server_side_encryption = storage_class =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ DigitalOcean\ Spaces\ ###
|
|
|
|
[Spaces](https://www.digitalocean.com/products/object\-storage/)\ is\ an\ [S3\-interoperable](https://developers.digitalocean.com/documentation/spaces/)\ object\ storage\ service\ from\ cloud\ provider\ DigitalOcean.
|
|
|
|
To\ connect\ to\ DigitalOcean\ Spaces\ you\ will\ need\ an\ access\ key\ and\ secret\ key.\ These\ can\ be\ retrieved\ on\ the\ "[Applications\ &\ API](https://cloud.digitalocean.com/settings/api/tokens)"\ page\ of\ the\ DigitalOcean\ control\ panel.\ They\ will\ be\ needed\ when\ promted\ by\ `rclone\ config`\ for\ your\ `access_key_id`\ and\ `secret_access_key`.
|
|
|
|
When\ prompted\ for\ a\ `region`\ or\ `location_constraint`,\ press\ enter\ to\ use\ the\ default\ value.\ The\ region\ must\ be\ included\ in\ the\ `endpoint`\ setting\ (e.g.\ `nyc3.digitaloceanspaces.com`).\ The\ default\ values\ can\ be\ used\ for\ other\ settings.
|
|
|
|
Going\ through\ the\ whole\ process\ of\ creating\ a\ new\ remote\ by\ running\ `rclone\ config`,\ each\ prompt\ should\ be\ answered\ as\ shown\ below:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Storage> s3 env_auth> 1 access_key_id> YOUR_ACCESS_KEY
|
|
secret_access_key> YOUR_SECRET_KEY region> endpoint>
|
|
nyc3.digitaloceanspaces.com location_constraint> acl> storage_class>
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
The\ resulting\ configuration\ file\ should\ look\ like:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[spaces] type = s3 provider = DigitalOcean env_auth = false
|
|
access_key_id = YOUR_ACCESS_KEY secret_access_key = YOUR_SECRET_KEY
|
|
region = endpoint = nyc3.digitaloceanspaces.com location_constraint =
|
|
acl = server_side_encryption = storage_class =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Once\ configured,\ you\ can\ create\ a\ new\ Space\ and\ begin\ copying\ files.\ For\ example:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
rclone mkdir spaces:my\-new\-space rclone copy /path/to/files
|
|
spaces:my\-new\-space
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ IBM\ COS\ (S3)\ ###
|
|
|
|
Information\ stored\ with\ IBM\ Cloud\ Object\ Storage\ is\ encrypted\ and\ dispersed\ across\ multiple\ geographic\ locations,\ and\ accessed\ through\ an\ implementation\ of\ the\ S3\ API.\ This\ service\ makes\ use\ of\ the\ distributed\ storage\ technologies\ provided\ by\ IBM's\ Cloud\ Object\ Storage\ System\ (formerly\ Cleversafe).\ For\ more\ information\ visit:\ (http://www.ibm.com/cloud/object\-storage)
|
|
|
|
To\ configure\ access\ to\ IBM\ COS\ S3,\ follow\ the\ steps\ below:
|
|
|
|
1.\ Run\ rclone\ config\ and\ select\ n\ for\ a\ new\ remote.
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2018/02/14\ 14:13:11\ NOTICE:\ Config\ file\ "C:\\\\Users\\\\a\\\\.config\\\\rclone\\\\rclone.conf"\ not\ found\ \-\ using\ defaults
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
2.\ Enter\ the\ name\ for\ the\ configuration
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
name>\ <YOUR\ NAME>
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
3.\ Select\ "s3"\ storage.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Choose a number from below, or type in your own value 1 / Alias for an
|
|
existing remote \ \[lq]alias\[rq] 2 / Amazon Drive \ \[lq]amazon cloud
|
|
drive\[rq] 3 / Amazon S3 Complaint Storage Providers (Dreamhost, Ceph,
|
|
Minio, IBM COS) \ \[lq]s3\[rq] 4 / Backblaze B2 \ \[lq]b2\[rq] [snip] 23
|
|
/ http Connection \ \[lq]http\[rq] Storage> 3
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
4.\ Select\ IBM\ COS\ as\ the\ S3\ Storage\ Provider.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Choose the S3 provider.
|
|
Choose a number from below, or type in your own value 1 / Choose this
|
|
option to configure Storage to AWS S3 \ \[lq]AWS\[rq] 2 / Choose this
|
|
option to configure Storage to Ceph Systems \ \[lq]Ceph\[rq] 3 / Choose
|
|
this option to configure Storage to Dreamhost \ \[lq]Dreamhost\[rq] 4 /
|
|
Choose this option to the configure Storage to IBM COS S3
|
|
\ \[lq]IBMCOS\[rq] 5 / Choose this option to the configure Storage to
|
|
Minio \ \[lq]Minio\[rq] Provider>4
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
5.\ Enter\ the\ Access\ Key\ and\ Secret.
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
AWS\ Access\ Key\ ID\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
|
access_key_id>\ <>
|
|
AWS\ Secret\ Access\ Key\ (password)\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
|
secret_access_key>\ <>
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
6.\ Specify\ the\ endpoint\ for\ IBM\ COS.\ For\ Public\ IBM\ COS,\ choose\ from\ the\ option\ below.\ For\ On\ Premise\ IBM\ COS,\ enter\ an\ enpoint\ address.
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Endpoint\ for\ IBM\ COS\ S3\ API.
|
|
Specify\ if\ using\ an\ IBM\ COS\ On\ Premise.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ US\ Cross\ Region\ Endpoint
|
|
\ \ \ \\\ "s3\-api.us\-geo.objectstorage.softlayer.net"
|
|
\ 2\ /\ US\ Cross\ Region\ Dallas\ Endpoint
|
|
\ \ \ \\\ "s3\-api.dal.us\-geo.objectstorage.softlayer.net"
|
|
\ 3\ /\ US\ Cross\ Region\ Washington\ DC\ Endpoint
|
|
\ \ \ \\\ "s3\-api.wdc\-us\-geo.objectstorage.softlayer.net"
|
|
\ 4\ /\ US\ Cross\ Region\ San\ Jose\ Endpoint
|
|
\ \ \ \\\ "s3\-api.sjc\-us\-geo.objectstorage.softlayer.net"
|
|
\ 5\ /\ US\ Cross\ Region\ Private\ Endpoint
|
|
\ \ \ \\\ "s3\-api.us\-geo.objectstorage.service.networklayer.com"
|
|
\ 6\ /\ US\ Cross\ Region\ Dallas\ Private\ Endpoint
|
|
\ \ \ \\\ "s3\-api.dal\-us\-geo.objectstorage.service.networklayer.com"
|
|
\ 7\ /\ US\ Cross\ Region\ Washington\ DC\ Private\ Endpoint
|
|
\ \ \ \\\ "s3\-api.wdc\-us\-geo.objectstorage.service.networklayer.com"
|
|
\ 8\ /\ US\ Cross\ Region\ San\ Jose\ Private\ Endpoint
|
|
\ \ \ \\\ "s3\-api.sjc\-us\-geo.objectstorage.service.networklayer.com"
|
|
\ 9\ /\ US\ Region\ East\ Endpoint
|
|
\ \ \ \\\ "s3.us\-east.objectstorage.softlayer.net"
|
|
10\ /\ US\ Region\ East\ Private\ Endpoint
|
|
\ \ \ \\\ "s3.us\-east.objectstorage.service.networklayer.com"
|
|
11\ /\ US\ Region\ South\ Endpoint
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[snip] 34 / Toronto Single Site Private Endpoint
|
|
\ \[lq]s3.tor01.objectstorage.service.networklayer.com\[rq] endpoint>1
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
|
|
7.\ Specify\ a\ IBM\ COS\ Location\ Constraint.\ The\ location\ constraint\ must\ match\ endpoint\ when\ using\ IBM\ Cloud\ Public.\ For\ on\-prem\ COS,\ do\ not\ make\ a\ selection\ from\ this\ list,\ hit\ enter
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ 1\ /\ US\ Cross\ Region\ Standard
|
|
\ \ \ \\\ "us\-standard"
|
|
\ 2\ /\ US\ Cross\ Region\ Vault
|
|
\ \ \ \\\ "us\-vault"
|
|
\ 3\ /\ US\ Cross\ Region\ Cold
|
|
\ \ \ \\\ "us\-cold"
|
|
\ 4\ /\ US\ Cross\ Region\ Flex
|
|
\ \ \ \\\ "us\-flex"
|
|
\ 5\ /\ US\ East\ Region\ Standard
|
|
\ \ \ \\\ "us\-east\-standard"
|
|
\ 6\ /\ US\ East\ Region\ Vault
|
|
\ \ \ \\\ "us\-east\-vault"
|
|
\ 7\ /\ US\ East\ Region\ Cold
|
|
\ \ \ \\\ "us\-east\-cold"
|
|
\ 8\ /\ US\ East\ Region\ Flex
|
|
\ \ \ \\\ "us\-east\-flex"
|
|
\ 9\ /\ US\ South\ Region\ Standard
|
|
\ \ \ \\\ "us\-south\-standard"
|
|
10\ /\ US\ South\ Region\ Vault
|
|
\ \ \ \\\ "us\-south\-vault"
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[snip] 32 / Toronto Flex \ \[lq]tor01\-flex\[rq] location_constraint>1
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
9.\ Specify\ a\ canned\ ACL.\ IBM\ Cloud\ (Strorage)\ supports\ "public\-read"\ and\ "private".\ IBM\ Cloud(Infra)\ supports\ all\ the\ canned\ ACLs.\ On\-Premise\ COS\ supports\ all\ the\ canned\ ACLs.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value 1 / Owner gets
|
|
FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
|
|
On\-Premise COS \ \[lq]private\[rq] 2 / Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
|
|
On\-Premise IBM COS \ \[lq]public\-read\[rq] 3 / Owner gets
|
|
FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
This acl is available on IBM Cloud (Infra), On\-Premise IBM COS
|
|
\ \[lq]public\-read\-write\[rq] 4 / Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
Not supported on Buckets.
|
|
This acl is available on IBM Cloud (Infra) and On\-Premise IBM COS
|
|
\ \[lq]authenticated\-read\[rq] acl> 1
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
|
|
12.\ Review\ the\ displayed\ configuration\ and\ accept\ to\ save\ the\ "remote"\ then\ quit.\ The\ config\ file\ should\ look\ like\ this
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[xxx]
|
|
type\ =\ s3
|
|
Provider\ =\ IBMCOS
|
|
access_key_id\ =\ xxx
|
|
secret_access_key\ =\ yyy
|
|
endpoint\ =\ s3\-api.us\-geo.objectstorage.softlayer.net
|
|
location_constraint\ =\ us\-standard
|
|
acl\ =\ private
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
13.\ Execute\ rclone\ commands
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
1)\ \ Create\ a\ bucket.
|
|
\ \ \ \ rclone\ mkdir\ IBM\-COS\-XREGION:newbucket
|
|
2)\ \ List\ available\ buckets.
|
|
\ \ \ \ rclone\ lsd\ IBM\-COS\-XREGION:
|
|
\ \ \ \ \-1\ 2017\-11\-08\ 21:16:22\ \ \ \ \ \ \ \ \-1\ test
|
|
\ \ \ \ \-1\ 2018\-02\-14\ 20:16:39\ \ \ \ \ \ \ \ \-1\ newbucket
|
|
3)\ \ List\ contents\ of\ a\ bucket.
|
|
\ \ \ \ rclone\ ls\ IBM\-COS\-XREGION:newbucket
|
|
\ \ \ \ 18685952\ test.exe
|
|
4)\ \ Copy\ a\ file\ from\ local\ to\ remote.
|
|
\ \ \ \ rclone\ copy\ /Users/file.txt\ IBM\-COS\-XREGION:newbucket
|
|
5)\ \ Copy\ a\ file\ from\ remote\ to\ local.
|
|
\ \ \ \ rclone\ copy\ IBM\-COS\-XREGION:newbucket/file.txt\ .
|
|
6)\ \ Delete\ a\ file\ on\ remote.
|
|
\ \ \ \ rclone\ delete\ IBM\-COS\-XREGION:newbucket/file.txt
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Minio\ ###
|
|
|
|
[Minio](https://minio.io/)\ is\ an\ object\ storage\ server\ built\ for\ cloud\ application\ developers\ and\ devops.
|
|
|
|
It\ is\ very\ easy\ to\ install\ and\ provides\ an\ S3\ compatible\ server\ which\ can\ be\ used\ by\ rclone.
|
|
|
|
To\ use\ it,\ install\ Minio\ following\ the\ instructions\ [here](https://docs.minio.io/docs/minio\-quickstart\-guide).
|
|
|
|
When\ it\ configures\ itself\ Minio\ will\ print\ something\ like\ this
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000 AccessKey:
|
|
USWUXHGYZQYFYFFIT3RE SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
Region: us\-east\-1 SQS ARNs: arn:minio:sqs:us\-east\-1:1:redis
|
|
arn:minio:sqs:us\-east\-1:2:redis
|
|
.PP
|
|
Browser Access: http://192.168.1.106:9000 http://172.23.0.1:9000
|
|
.PP
|
|
Command\-line Access:
|
|
https://docs.minio.io/docs/minio\-client\-quickstart\-guide $ mc config
|
|
host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE
|
|
MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
.PP
|
|
Object API (Amazon S3 compatible): Go:
|
|
https://docs.minio.io/docs/golang\-client\-quickstart\-guide Java:
|
|
https://docs.minio.io/docs/java\-client\-quickstart\-guide Python:
|
|
https://docs.minio.io/docs/python\-client\-quickstart\-guide JavaScript:
|
|
https://docs.minio.io/docs/javascript\-client\-quickstart\-guide .NET:
|
|
https://docs.minio.io/docs/dotnet\-client\-quickstart\-guide
|
|
.PP
|
|
Drive Capacity: 26 GiB Free, 165 GiB Total
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
These\ details\ need\ to\ go\ into\ `rclone\ config`\ like\ this.\ \ Note\ that\ it
|
|
is\ important\ to\ put\ the\ region\ in\ as\ stated\ above.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
env_auth> 1 access_key_id> USWUXHGYZQYFYFFIT3RE secret_access_key>
|
|
MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03 region> us\-east\-1 endpoint>
|
|
http://192.168.1.106:9000 location_constraint> server_side_encryption>
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Which\ makes\ the\ config\ file\ look\ like\ this
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[minio] type = s3 provider = Minio env_auth = false access_key_id =
|
|
USWUXHGYZQYFYFFIT3RE secret_access_key =
|
|
MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03 region = us\-east\-1 endpoint =
|
|
http://192.168.1.106:9000 location_constraint = server_side_encryption =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
So\ once\ set\ up,\ for\ example\ to\ copy\ files\ into\ a\ bucket
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
rclone copy /path/to/files minio:bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Scaleway\ {#scaleway}
|
|
|
|
[Scaleway](https://www.scaleway.com/object\-storage/)\ The\ Object\ Storage\ platform\ allows\ you\ to\ store\ anything\ from\ backups,\ logs\ and\ web\ assets\ to\ documents\ and\ photos.
|
|
Files\ can\ be\ dropped\ from\ the\ Scaleway\ console\ or\ transferred\ through\ our\ API\ and\ CLI\ or\ using\ any\ S3\-compatible\ tool.
|
|
|
|
Scaleway\ provides\ an\ S3\ interface\ which\ can\ be\ configured\ for\ use\ with\ rclone\ like\ this:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[scaleway] type = s3 env_auth = false endpoint = s3.nl\-ams.scw.cloud
|
|
access_key_id = SCWXXXXXXXXXXXXXX secret_access_key =
|
|
1111111\-2222\-3333\-44444\-55555555555555 region = nl\-ams
|
|
location_constraint = acl = private force_path_style = false
|
|
server_side_encryption = storage_class =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Wasabi\ ###
|
|
|
|
[Wasabi](https://wasabi.com)\ is\ a\ cloud\-based\ object\ storage\ service\ for\ a
|
|
broad\ range\ of\ applications\ and\ use\ cases.\ Wasabi\ is\ designed\ for
|
|
individuals\ and\ organizations\ that\ require\ a\ high\-performance,
|
|
reliable,\ and\ secure\ data\ storage\ infrastructure\ at\ minimal\ cost.
|
|
|
|
Wasabi\ provides\ an\ S3\ interface\ which\ can\ be\ configured\ for\ use\ with
|
|
rclone\ like\ this.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password n/s> n name> wasabi Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Amazon
|
|
S3 (also Dreamhost, Ceph, Minio) \ \[lq]s3\[rq] [snip] Storage> s3 Get
|
|
AWS credentials from runtime (environment variables or EC2/ECS meta data
|
|
if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value 1 / Enter AWS
|
|
credentials in the next step \ \[lq]false\[rq] 2 / Get AWS credentials
|
|
from the environment (env vars or IAM) \ \[lq]true\[rq] env_auth> 1 AWS
|
|
Access Key ID \- leave blank for anonymous access or runtime
|
|
credentials.
|
|
access_key_id> YOURACCESSKEY AWS Secret Access Key (password) \- leave
|
|
blank for anonymous access or runtime credentials.
|
|
secret_access_key> YOURSECRETACCESSKEY Region to connect to.
|
|
Choose a number from below, or type in your own value / The default
|
|
endpoint \- a good choice if you are unsure.
|
|
1 | US Region, Northern Virginia or Pacific Northwest.
|
|
| Leave location constraint empty.
|
|
\ \[lq]us\-east\-1\[rq] [snip] region> us\-east\-1 Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
Specify if using an S3 clone such as Ceph.
|
|
endpoint> s3.wasabisys.com Location constraint \- must be set to match
|
|
the Region.
|
|
Used when creating buckets only.
|
|
Choose a number from below, or type in your own value 1 / Empty for US
|
|
Region, Northern Virginia or Pacific Northwest.
|
|
\ "" [snip] location_constraint> Canned ACL used when creating buckets
|
|
and/or storing objects in S3.
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value 1 / Owner gets
|
|
FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
\ \[lq]private\[rq] [snip] acl> The server\-side encryption algorithm
|
|
used when storing this object in S3.
|
|
Choose a number from below, or type in your own value 1 / None \ "" 2 /
|
|
AES256 \ \[lq]AES256\[rq] server_side_encryption> The storage class to
|
|
use when storing objects in S3.
|
|
Choose a number from below, or type in your own value 1 / Default \ "" 2
|
|
/ Standard storage class \ \[lq]STANDARD\[rq] 3 / Reduced redundancy
|
|
storage class \ \[lq]REDUCED_REDUNDANCY\[rq] 4 / Standard Infrequent
|
|
Access storage class \ \[lq]STANDARD_IA\[rq] storage_class> Remote
|
|
config \[em]\[em]\[em]\[em]\[em]\[em]\[en] [wasabi] env_auth = false
|
|
access_key_id = YOURACCESSKEY secret_access_key = YOURSECRETACCESSKEY
|
|
region = us\-east\-1 endpoint = s3.wasabisys.com location_constraint =
|
|
acl = server_side_encryption = storage_class =
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
This\ will\ leave\ the\ config\ file\ looking\ like\ this.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
[wasabi] type = s3 provider = Wasabi env_auth = false access_key_id =
|
|
YOURACCESSKEY secret_access_key = YOURSECRETACCESSKEY region = endpoint
|
|
= s3.wasabisys.com location_constraint = acl = server_side_encryption =
|
|
storage_class =
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Alibaba\ OSS\ {#alibaba\-oss}
|
|
|
|
Here\ is\ an\ example\ of\ making\ an\ [Alibaba\ Cloud\ (Aliyun)\ OSS](https://www.alibabacloud.com/product/oss/)
|
|
configuration.\ \ First\ run:
|
|
|
|
\ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> oss Type of storage to configure.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value [snip] 4 / Amazon S3 Compliant Storage Provider
|
|
(AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
|
|
\ \[rq]s3" [snip] Storage> s3 Choose your S3 provider.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value 1 / Amazon Web Services (AWS) S3 \ \[rq]AWS" 2 /
|
|
Alibaba Cloud Object Storage System (OSS) formerly Aliyun
|
|
\ \[lq]Alibaba\[rq] 3 / Ceph Object Storage \ \[lq]Ceph\[rq] [snip]
|
|
provider> Alibaba Get AWS credentials from runtime (environment
|
|
variables or EC2/ECS meta data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
Enter a boolean value (true or false).
|
|
Press Enter for the default (\[lq]false\[rq]).
|
|
Choose a number from below, or type in your own value 1 / Enter AWS
|
|
credentials in the next step \ \[lq]false\[rq] 2 / Get AWS credentials
|
|
from the environment (env vars or IAM) \ \[lq]true\[rq] env_auth> 1 AWS
|
|
Access Key ID.
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). access_key_id> accesskeyid AWS
|
|
Secret Access Key (password) Leave blank for anonymous access or runtime
|
|
credentials. Enter a string value. Press Enter for the default
|
|
(\[rq]\[lq]). secret_access_key> secretaccesskey Endpoint for OSS API.
|
|
Enter a string value. Press Enter for the default (\[rq]\[lq]). Choose a
|
|
number from below, or type in your own value 1 / East China 1 (Hangzhou)
|
|
\ \[rq]oss\-cn\-hangzhou.aliyuncs.com" 2 / East China 2 (Shanghai)
|
|
\ \[lq]oss\-cn\-shanghai.aliyuncs.com\[rq] 3 / North China 1 (Qingdao)
|
|
\ \[lq]oss\-cn\-qingdao.aliyuncs.com\[rq] [snip] endpoint> 1 Canned ACL
|
|
used when creating buckets and storing or copying objects.
|
|
.PP
|
|
Note that this ACL is applied when server side copying objects as S3
|
|
doesn't copy the ACL from the source but rather writes a fresh one.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value 1 / Owner gets FULL_CONTROL. No one else has
|
|
access rights (default). \ \[rq]private" 2 / Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
\ \[lq]public\-read\[rq] / Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
[snip] acl> 1 The storage class to use when storing new objects in OSS.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value 1 / Default \ \[rq]" 2 / Standard storage class
|
|
\ \[lq]STANDARD\[rq] 3 / Archive storage mode.
|
|
\ \[lq]GLACIER\[rq] 4 / Infrequent access storage mode.
|
|
\ \[lq]STANDARD_IA\[rq] storage_class> 1 Edit advanced config?
|
|
(y/n) y) Yes n) No y/n> n Remote config
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [oss] type = s3 provider = Alibaba
|
|
env_auth = false access_key_id = accesskeyid secret_access_key =
|
|
secretaccesskey endpoint = oss\-cn\-hangzhou.aliyuncs.com acl = private
|
|
storage_class = Standard \[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this
|
|
is OK e) Edit this remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Netease\ NOS\ \ ###
|
|
|
|
For\ Netease\ NOS\ configure\ as\ per\ the\ configurator\ `rclone\ config`
|
|
setting\ the\ provider\ `Netease`.\ \ This\ will\ automatically\ set
|
|
`force_path_style\ =\ false`\ which\ is\ necessary\ for\ it\ to\ run\ properly.
|
|
|
|
Backblaze\ B2
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
B2\ is\ [Backblaze\[aq]s\ cloud\ storage\ system](https://www.backblaze.com/b2/).
|
|
|
|
Paths\ are\ specified\ as\ `remote:bucket`\ (or\ `remote:`\ for\ the\ `lsd`
|
|
command.)\ \ You\ may\ put\ subdirectories\ in\ too,\ eg\ `remote:bucket/path/to/dir`.
|
|
|
|
Here\ is\ an\ example\ of\ making\ a\ b2\ configuration.\ \ First\ run
|
|
|
|
\ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process.\ \ To\ authenticate
|
|
you\ will\ either\ need\ your\ Account\ ID\ (a\ short\ hex\ number)\ and\ Master
|
|
Application\ Key\ (a\ long\ hex\ number)\ OR\ an\ Application\ Key,\ which\ is\ the
|
|
recommended\ method.\ See\ below\ for\ further\ details\ on\ generating\ and\ using
|
|
an\ Application\ Key.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote q) Quit config n/q> n
|
|
name> remote Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX /
|
|
Backblaze B2 \ \[lq]b2\[rq] [snip] Storage> b2 Account ID or Application
|
|
Key ID account> 123456789abc Application Key key>
|
|
0123456789abcdef0123456789abcdef0123456789 Endpoint for the service \-
|
|
leave blank normally.
|
|
endpoint> Remote config \[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote]
|
|
account = 123456789abc key = 0123456789abcdef0123456789abcdef0123456789
|
|
endpoint = \[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit
|
|
this remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
This\ remote\ is\ called\ `remote`\ and\ can\ now\ be\ used\ like\ this
|
|
|
|
See\ all\ buckets
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
Create\ a\ new\ bucket
|
|
|
|
\ \ \ \ rclone\ mkdir\ remote:bucket
|
|
|
|
List\ the\ contents\ of\ a\ bucket
|
|
|
|
\ \ \ \ rclone\ ls\ remote:bucket
|
|
|
|
Sync\ `/home/local/directory`\ to\ the\ remote\ bucket,\ deleting\ any
|
|
excess\ files\ in\ the\ bucket.
|
|
|
|
\ \ \ \ rclone\ sync\ /home/local/directory\ remote:bucket
|
|
|
|
###\ Application\ Keys\ ###
|
|
|
|
B2\ supports\ multiple\ [Application\ Keys\ for\ different\ access\ permission
|
|
to\ B2\ Buckets](https://www.backblaze.com/b2/docs/application_keys.html).
|
|
|
|
You\ can\ use\ these\ with\ rclone\ too;\ you\ will\ need\ to\ use\ rclone\ version\ 1.43
|
|
or\ later.
|
|
|
|
Follow\ Backblaze\[aq]s\ docs\ to\ create\ an\ Application\ Key\ with\ the\ required
|
|
permission\ and\ add\ the\ `applicationKeyId`\ as\ the\ `account`\ and\ the
|
|
`Application\ Key`\ itself\ as\ the\ `key`.
|
|
|
|
Note\ that\ you\ must\ put\ the\ _applicationKeyId_\ as\ the\ `account`\ \[en]\ you
|
|
can\[aq]t\ use\ the\ master\ Account\ ID.\ \ If\ you\ try\ then\ B2\ will\ return\ 401
|
|
errors.
|
|
|
|
###\ \-\-fast\-list\ ###
|
|
|
|
This\ remote\ supports\ `\-\-fast\-list`\ which\ allows\ you\ to\ use\ fewer
|
|
transactions\ in\ exchange\ for\ more\ memory.\ See\ the\ [rclone
|
|
docs](/docs/#fast\-list)\ for\ more\ details.
|
|
|
|
###\ Modified\ time\ ###
|
|
|
|
The\ modified\ time\ is\ stored\ as\ metadata\ on\ the\ object\ as
|
|
`X\-Bz\-Info\-src_last_modified_millis`\ as\ milliseconds\ since\ 1970\-01\-01
|
|
in\ the\ Backblaze\ standard.\ \ Other\ tools\ should\ be\ able\ to\ use\ this\ as
|
|
a\ modified\ time.
|
|
|
|
Modified\ times\ are\ used\ in\ syncing\ and\ are\ fully\ supported.\ Note\ that
|
|
if\ a\ modification\ time\ needs\ to\ be\ updated\ on\ an\ object\ then\ it\ will
|
|
create\ a\ new\ version\ of\ the\ object.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
In\ addition\ to\ the\ [default\ restricted\ characters\ set](/overview/#restricted\-characters)
|
|
the\ following\ characters\ are\ also\ replaced:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ \\\ \ \ \ \ \ \ \ \ |\ 0x5C\ \ |\ ＼\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
###\ SHA1\ checksums\ ###
|
|
|
|
The\ SHA1\ checksums\ of\ the\ files\ are\ checked\ on\ upload\ and\ download\ and
|
|
will\ be\ used\ in\ the\ syncing\ process.
|
|
|
|
Large\ files\ (bigger\ than\ the\ limit\ in\ `\-\-b2\-upload\-cutoff`)\ which\ are
|
|
uploaded\ in\ chunks\ will\ store\ their\ SHA1\ on\ the\ object\ as
|
|
`X\-Bz\-Info\-large_file_sha1`\ as\ recommended\ by\ Backblaze.
|
|
|
|
For\ a\ large\ file\ to\ be\ uploaded\ with\ an\ SHA1\ checksum,\ the\ source
|
|
needs\ to\ support\ SHA1\ checksums.\ The\ local\ disk\ supports\ SHA1
|
|
checksums\ so\ large\ file\ transfers\ from\ local\ disk\ will\ have\ an\ SHA1.
|
|
See\ [the\ overview](/overview/#features)\ for\ exactly\ which\ remotes
|
|
support\ SHA1.
|
|
|
|
Sources\ which\ don\[aq]t\ support\ SHA1,\ in\ particular\ `crypt`\ will\ upload
|
|
large\ files\ without\ SHA1\ checksums.\ \ This\ may\ be\ fixed\ in\ the\ future
|
|
(see\ [#1767](https://github.com/rclone/rclone/issues/1767)).
|
|
|
|
Files\ sizes\ below\ `\-\-b2\-upload\-cutoff`\ will\ always\ have\ an\ SHA1
|
|
regardless\ of\ the\ source.
|
|
|
|
###\ Transfers\ ###
|
|
|
|
Backblaze\ recommends\ that\ you\ do\ lots\ of\ transfers\ simultaneously\ for
|
|
maximum\ speed.\ \ In\ tests\ from\ my\ SSD\ equipped\ laptop\ the\ optimum
|
|
setting\ is\ about\ `\-\-transfers\ 32`\ though\ higher\ numbers\ may\ be\ used
|
|
for\ a\ slight\ speed\ improvement.\ The\ optimum\ number\ for\ you\ may\ vary
|
|
depending\ on\ your\ hardware,\ how\ big\ the\ files\ are,\ how\ much\ you\ want
|
|
to\ load\ your\ computer,\ etc.\ \ The\ default\ of\ `\-\-transfers\ 4`\ is
|
|
definitely\ too\ low\ for\ Backblaze\ B2\ though.
|
|
|
|
Note\ that\ uploading\ big\ files\ (bigger\ than\ 200\ MB\ by\ default)\ will\ use
|
|
a\ 96\ MB\ RAM\ buffer\ by\ default.\ \ There\ can\ be\ at\ most\ `\-\-transfers`\ of
|
|
these\ in\ use\ at\ any\ moment,\ so\ this\ sets\ the\ upper\ limit\ on\ the\ memory
|
|
used.
|
|
|
|
###\ Versions\ ###
|
|
|
|
When\ rclone\ uploads\ a\ new\ version\ of\ a\ file\ it\ creates\ a\ [new\ version
|
|
of\ it](https://www.backblaze.com/b2/docs/file_versions.html).
|
|
Likewise\ when\ you\ delete\ a\ file,\ the\ old\ version\ will\ be\ marked\ hidden
|
|
and\ still\ be\ available.\ \ Conversely,\ you\ may\ opt\ in\ to\ a\ "hard\ delete"
|
|
of\ files\ with\ the\ `\-\-b2\-hard\-delete`\ flag\ which\ would\ permanently\ remove
|
|
the\ file\ instead\ of\ hiding\ it.
|
|
|
|
Old\ versions\ of\ files,\ where\ available,\ are\ visible\ using\ the\
|
|
`\-\-b2\-versions`\ flag.
|
|
|
|
**NB**\ Note\ that\ `\-\-b2\-versions`\ does\ not\ work\ with\ crypt\ at\ the
|
|
moment\ [#1627](https://github.com/rclone/rclone/issues/1627).\ Using
|
|
[\-\-backup\-dir](/docs/#backup\-dir\-dir)\ with\ rclone\ is\ the\ recommended
|
|
way\ of\ working\ around\ this.
|
|
|
|
If\ you\ wish\ to\ remove\ all\ the\ old\ versions\ then\ you\ can\ use\ the
|
|
`rclone\ cleanup\ remote:bucket`\ command\ which\ will\ delete\ all\ the\ old
|
|
versions\ of\ files,\ leaving\ the\ current\ ones\ intact.\ \ You\ can\ also
|
|
supply\ a\ path\ and\ only\ old\ versions\ under\ that\ path\ will\ be\ deleted,
|
|
eg\ `rclone\ cleanup\ remote:bucket/path/to/stuff`.
|
|
|
|
Note\ that\ `cleanup`\ will\ remove\ partially\ uploaded\ files\ from\ the\ bucket
|
|
if\ they\ are\ more\ than\ a\ day\ old.
|
|
|
|
When\ you\ `purge`\ a\ bucket,\ the\ current\ and\ the\ old\ versions\ will\ be
|
|
deleted\ then\ the\ bucket\ will\ be\ deleted.
|
|
|
|
However\ `delete`\ will\ cause\ the\ current\ versions\ of\ the\ files\ to
|
|
become\ hidden\ old\ versions.
|
|
|
|
Here\ is\ a\ session\ showing\ the\ listing\ and\ retrieval\ of\ an\ old
|
|
version\ followed\ by\ a\ `cleanup`\ of\ the\ old\ versions.
|
|
|
|
Show\ current\ version\ and\ all\ the\ versions\ with\ `\-\-b2\-versions`\ flag.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q ls b2:cleanup\-test 9 one.txt
|
|
.PP
|
|
$ rclone \-q \[en]b2\-versions ls b2:cleanup\-test 9 one.txt 8
|
|
one\-v2016\-07\-04\-141032\-000.txt 16
|
|
one\-v2016\-07\-04\-141003\-000.txt 15
|
|
one\-v2016\-07\-02\-155621\-000.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Retrieve\ an\ old\ version
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q \[en]b2\-versions copy
|
|
b2:cleanup\-test/one\-v2016\-07\-04\-141003\-000.txt /tmp
|
|
.PP
|
|
$ ls \-l /tmp/one\-v2016\-07\-04\-141003\-000.txt \-rw\-rw\-r\[en] 1 ncw
|
|
ncw 16 Jul 2 17:46 /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Clean\ up\ all\ the\ old\ versions\ and\ show\ that\ they\[aq]ve\ gone.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q cleanup b2:cleanup\-test
|
|
.PP
|
|
$ rclone \-q ls b2:cleanup\-test 9 one.txt
|
|
.PP
|
|
$ rclone \-q \[en]b2\-versions ls b2:cleanup\-test 9 one.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Data\ usage\ ###
|
|
|
|
It\ is\ useful\ to\ know\ how\ many\ requests\ are\ sent\ to\ the\ server\ in\ different\ scenarios.
|
|
|
|
All\ copy\ commands\ send\ the\ following\ 4\ requests:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
/b2api/v1/b2_authorize_account /b2api/v1/b2_create_bucket
|
|
/b2api/v1/b2_list_buckets /b2api/v1/b2_list_file_names
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
The\ `b2_list_file_names`\ request\ will\ be\ sent\ once\ for\ every\ 1k\ files
|
|
in\ the\ remote\ path,\ providing\ the\ checksum\ and\ modification\ time\ of
|
|
the\ listed\ files.\ As\ of\ version\ 1.33\ issue
|
|
[#818](https://github.com/rclone/rclone/issues/818)\ causes\ extra\ requests
|
|
to\ be\ sent\ when\ using\ B2\ with\ Crypt.\ When\ a\ copy\ operation\ does\ not
|
|
require\ any\ files\ to\ be\ uploaded,\ no\ more\ requests\ will\ be\ sent.
|
|
|
|
Uploading\ files\ that\ do\ not\ require\ chunking,\ will\ send\ 2\ requests\ per
|
|
file\ upload:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
/b2api/v1/b2_get_upload_url /b2api/v1/b2_upload_file/
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Uploading\ files\ requiring\ chunking,\ will\ send\ 2\ requests\ (one\ each\ to
|
|
start\ and\ finish\ the\ upload)\ and\ another\ 2\ requests\ for\ each\ chunk:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
/b2api/v1/b2_start_large_file /b2api/v1/b2_get_upload_part_url
|
|
/b2api/v1/b2_upload_part/ /b2api/v1/b2_finish_large_file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
####\ Versions\ ####
|
|
|
|
Versions\ can\ be\ viewed\ with\ the\ `\-\-b2\-versions`\ flag.\ When\ it\ is\ set
|
|
rclone\ will\ show\ and\ act\ on\ older\ versions\ of\ files.\ \ For\ example
|
|
|
|
Listing\ without\ `\-\-b2\-versions`
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q ls b2:cleanup\-test 9 one.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
And\ with
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q \[en]b2\-versions ls b2:cleanup\-test 9 one.txt 8
|
|
one\-v2016\-07\-04\-141032\-000.txt 16
|
|
one\-v2016\-07\-04\-141003\-000.txt 15
|
|
one\-v2016\-07\-02\-155621\-000.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Showing\ that\ the\ current\ version\ is\ unchanged\ but\ older\ versions\ can
|
|
be\ seen.\ \ These\ have\ the\ UTC\ date\ that\ they\ were\ uploaded\ to\ the
|
|
server\ to\ the\ nearest\ millisecond\ appended\ to\ them.
|
|
|
|
Note\ that\ when\ using\ `\-\-b2\-versions`\ no\ file\ write\ operations\ are
|
|
permitted,\ so\ you\ can\[aq]t\ upload\ files\ or\ delete\ them.
|
|
|
|
###\ B2\ and\ rclone\ link\ ###
|
|
|
|
Rclone\ supports\ generating\ file\ share\ links\ for\ private\ B2\ buckets.
|
|
They\ can\ either\ be\ for\ a\ file\ for\ example:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\&./rclone link B2:bucket/path/to/file.txt
|
|
https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
or\ if\ run\ on\ a\ directory\ you\ will\ get:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\&./rclone link B2:bucket/path
|
|
https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
you\ can\ then\ use\ the\ authorization\ token\ (the\ part\ of\ the\ url\ from\ the
|
|
\ `?Authorization=`\ on)\ on\ any\ file\ path\ under\ that\ directory.\ For\ example:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx
|
|
https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx
|
|
https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/b2/b2.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ b2\ (Backblaze\ B2).
|
|
|
|
####\ \-\-b2\-account
|
|
|
|
Account\ ID\ or\ Application\ Key\ ID
|
|
|
|
\-\ Config:\ \ \ \ \ \ account
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_ACCOUNT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-b2\-key
|
|
|
|
Application\ Key
|
|
|
|
\-\ Config:\ \ \ \ \ \ key
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_KEY
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-b2\-hard\-delete
|
|
|
|
Permanently\ delete\ files\ on\ remote\ removal,\ otherwise\ hide\ files.
|
|
|
|
\-\ Config:\ \ \ \ \ \ hard_delete
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_HARD_DELETE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ b2\ (Backblaze\ B2).
|
|
|
|
####\ \-\-b2\-endpoint
|
|
|
|
Endpoint\ for\ the\ service.
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-b2\-test\-mode
|
|
|
|
A\ flag\ string\ for\ X\-Bz\-Test\-Mode\ header\ for\ debugging.
|
|
|
|
This\ is\ for\ debugging\ purposes\ only.\ Setting\ it\ to\ one\ of\ the\ strings
|
|
below\ will\ cause\ b2\ to\ return\ specific\ errors:
|
|
|
|
\ \ *\ "fail_some_uploads"
|
|
\ \ *\ "expire_some_account_authorization_tokens"
|
|
\ \ *\ "force_cap_exceeded"
|
|
|
|
These\ will\ be\ set\ in\ the\ "X\-Bz\-Test\-Mode"\ header\ which\ is\ documented
|
|
in\ the\ [b2\ integrations\ checklist](https://www.backblaze.com/b2/docs/integration_checklist.html).
|
|
|
|
\-\ Config:\ \ \ \ \ \ test_mode
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_TEST_MODE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-b2\-versions
|
|
|
|
Include\ old\ versions\ in\ directory\ listings.
|
|
Note\ that\ when\ using\ this\ no\ file\ write\ operations\ are\ permitted,
|
|
so\ you\ can\[aq]t\ upload\ files\ or\ delete\ them.
|
|
|
|
\-\ Config:\ \ \ \ \ \ versions
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_VERSIONS
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-b2\-upload\-cutoff
|
|
|
|
Cutoff\ for\ switching\ to\ chunked\ upload.
|
|
|
|
Files\ above\ this\ size\ will\ be\ uploaded\ in\ chunks\ of\ "\-\-b2\-chunk\-size".
|
|
|
|
This\ value\ should\ be\ set\ no\ larger\ than\ 4.657GiB\ (==\ 5GB).
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_cutoff
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_UPLOAD_CUTOFF
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 200M
|
|
|
|
####\ \-\-b2\-chunk\-size
|
|
|
|
Upload\ chunk\ size.\ Must\ fit\ in\ memory.
|
|
|
|
When\ uploading\ large\ files,\ chunk\ the\ file\ into\ this\ size.\ \ Note\ that
|
|
these\ chunks\ are\ buffered\ in\ memory\ and\ there\ might\ a\ maximum\ of
|
|
"\-\-transfers"\ chunks\ in\ progress\ at\ once.\ \ 5,000,000\ Bytes\ is\ the
|
|
minimum\ size.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 96M
|
|
|
|
####\ \-\-b2\-disable\-checksum
|
|
|
|
Disable\ checksums\ for\ large\ (>\ upload\ cutoff)\ files
|
|
|
|
\-\ Config:\ \ \ \ \ \ disable_checksum
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_DISABLE_CHECKSUM
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-b2\-download\-url
|
|
|
|
Custom\ endpoint\ for\ downloads.
|
|
|
|
This\ is\ usually\ set\ to\ a\ Cloudflare\ CDN\ URL\ as\ Backblaze\ offers
|
|
free\ egress\ for\ data\ downloaded\ through\ the\ Cloudflare\ network.
|
|
This\ is\ probably\ only\ useful\ for\ a\ public\ bucket.
|
|
Leave\ blank\ if\ you\ want\ to\ use\ the\ endpoint\ provided\ by\ Backblaze.
|
|
|
|
\-\ Config:\ \ \ \ \ \ download_url
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_DOWNLOAD_URL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-b2\-download\-auth\-duration
|
|
|
|
Time\ before\ the\ authorization\ token\ will\ expire\ in\ s\ or\ suffix\ ms|s|m|h|d.
|
|
|
|
The\ duration\ before\ the\ download\ authorization\ token\ will\ expire.
|
|
The\ minimum\ value\ is\ 1\ second.\ The\ maximum\ value\ is\ one\ week.
|
|
|
|
\-\ Config:\ \ \ \ \ \ download_auth_duration
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_B2_DOWNLOAD_AUTH_DURATION
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 1w
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
Box
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
Paths\ are\ specified\ as\ `remote:path`
|
|
|
|
Paths\ may\ be\ as\ deep\ as\ required,\ eg\ `remote:directory/subdirectory`.
|
|
|
|
The\ initial\ setup\ for\ Box\ involves\ getting\ a\ token\ from\ Box\ which\ you
|
|
can\ do\ either\ in\ your\ browser,\ or\ with\ a\ config.json\ downloaded\ from\ Box
|
|
to\ use\ JWT\ authentication.\ \ `rclone\ config`\ walks\ you\ through\ it.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> remote Type of storage to
|
|
configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Box
|
|
\ \[lq]box\[rq] [snip] Storage> box Box App Client Id \- leave blank
|
|
normally.
|
|
client_id> Box App Client Secret \- leave blank normally.
|
|
client_secret> Box App config.json location Leave blank normally.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). config_json> `enterprise' or
|
|
`user' depending on the type of token being requested. Enter a string
|
|
value. Press Enter for the default (\[rq]user\[lq]). box_sub_type>
|
|
Remote config Use auto config? * Say Y if not sure * Say N if you are
|
|
working on a remote or headless machine y) Yes n) No y/n> y If your
|
|
browser doesn't open automatically go to the following link:
|
|
http://127.0.0.1:53682/auth Log in and authorize rclone for access
|
|
Waiting for code\&... Got code \[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
[remote] client_id = client_secret = token =
|
|
{\[rq]access_token\[lq]:\[rq]XXX\[lq],\[rq]token_type\[lq]:\[rq]bearer\[lq],\[rq]refresh_token\[lq]:\[rq]XXX\[lq],\[rq]expiry\[lq]:\[rq]XXX"}
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
See\ the\ [remote\ setup\ docs](https://rclone.org/remote_setup/)\ for\ how\ to\ set\ it\ up\ on\ a
|
|
machine\ with\ no\ Internet\ browser\ available.
|
|
|
|
Note\ that\ rclone\ runs\ a\ webserver\ on\ your\ local\ machine\ to\ collect\ the
|
|
token\ as\ returned\ from\ Box.\ This\ only\ runs\ from\ the\ moment\ it\ opens
|
|
your\ browser\ to\ the\ moment\ you\ get\ back\ the\ verification\ code.\ \ This
|
|
is\ on\ `http://127.0.0.1:53682/`\ and\ this\ it\ may\ require\ you\ to\ unblock
|
|
it\ temporarily\ if\ you\ are\ running\ a\ host\ firewall.
|
|
|
|
Once\ configured\ you\ can\ then\ use\ `rclone`\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ Box
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ your\ Box
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
To\ copy\ a\ local\ directory\ to\ an\ Box\ directory\ called\ backup
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:backup
|
|
|
|
###\ Using\ rclone\ with\ an\ Enterprise\ account\ with\ SSO\ ###
|
|
|
|
If\ you\ have\ an\ "Enterprise"\ account\ type\ with\ Box\ with\ single\ sign\ on
|
|
(SSO),\ you\ need\ to\ create\ a\ password\ to\ use\ Box\ with\ rclone.\ This\ can
|
|
be\ done\ at\ your\ Enterprise\ Box\ account\ by\ going\ to\ Settings,\ "Account"
|
|
Tab,\ and\ then\ set\ the\ password\ in\ the\ "Authentication"\ field.
|
|
|
|
Once\ you\ have\ done\ this,\ you\ can\ setup\ your\ Enterprise\ Box\ account
|
|
using\ the\ same\ procedure\ detailed\ above\ in\ the,\ using\ the\ password\ you
|
|
have\ just\ set.
|
|
|
|
###\ Invalid\ refresh\ token\ ###
|
|
|
|
According\ to\ the\ [box\ docs](https://developer.box.com/v2.0/docs/oauth\-20#section\-6\-using\-the\-access\-and\-refresh\-tokens):
|
|
|
|
>\ Each\ refresh_token\ is\ valid\ for\ one\ use\ in\ 60\ days.
|
|
|
|
This\ means\ that\ if\ you
|
|
|
|
\ \ *\ Don\[aq]t\ use\ the\ box\ remote\ for\ 60\ days
|
|
\ \ *\ Copy\ the\ config\ file\ with\ a\ box\ refresh\ token\ in\ and\ use\ it\ in\ two\ places
|
|
\ \ *\ Get\ an\ error\ on\ a\ token\ refresh
|
|
|
|
then\ rclone\ will\ return\ an\ error\ which\ includes\ the\ text\ `Invalid
|
|
refresh\ token`.
|
|
|
|
To\ fix\ this\ you\ will\ need\ to\ use\ oauth2\ again\ to\ update\ the\ refresh
|
|
token.\ \ You\ can\ use\ the\ methods\ in\ [the\ remote\ setup
|
|
docs](https://rclone.org/remote_setup/),\ bearing\ in\ mind\ that\ if\ you\ use\ the\ copy\ the
|
|
config\ file\ method,\ you\ should\ not\ use\ that\ remote\ on\ the\ computer\ you
|
|
did\ the\ authentication\ on.
|
|
|
|
Here\ is\ how\ to\ do\ it.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone config Current remotes:
|
|
.PP
|
|
Name Type ==== ==== remote box
|
|
.IP "e)" 3
|
|
Edit existing remote
|
|
.IP "f)" 3
|
|
New remote
|
|
.IP "g)" 3
|
|
Delete remote
|
|
.IP "h)" 3
|
|
Rename remote
|
|
.IP "i)" 3
|
|
Copy remote
|
|
.IP "j)" 3
|
|
Set configuration password
|
|
.IP "k)" 3
|
|
Quit config e/n/d/r/c/s/q> e Choose a number from below, or type in an
|
|
existing value 1 > remote remote> remote
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] type = box token =
|
|
{\[lq]access_token\[rq]:\[lq]XXX\[rq],\[lq]token_type\[rq]:\[lq]bearer\[rq],\[lq]refresh_token\[rq]:\[lq]XXX\[rq],\[lq]expiry\[rq]:\[lq]2017\-07\-08T23:40:08.059167677+01:00\[rq]}
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] Edit remote Value
|
|
\[lq]client_id\[rq] = "" Edit?
|
|
(y/n)>
|
|
.IP "l)" 3
|
|
Yes
|
|
.IP "m)" 3
|
|
No y/n> n Value \[lq]client_secret\[rq] = "" Edit?
|
|
(y/n)>
|
|
.IP "n)" 3
|
|
Yes
|
|
.IP "o)" 3
|
|
No y/n> n Remote config Already have a token \- refresh?
|
|
.IP "p)" 3
|
|
Yes
|
|
.IP "q)" 3
|
|
No y/n> y Use auto config?
|
|
.IP \[bu] 2
|
|
Say Y if not sure
|
|
.IP \[bu] 2
|
|
Say N if you are working on a remote or headless machine
|
|
.IP "y)" 3
|
|
Yes
|
|
.IP "z)" 3
|
|
No y/n> y If your browser doesn't open automatically go to the following
|
|
link: http://127.0.0.1:53682/auth Log in and authorize rclone for access
|
|
Waiting for code\&... Got code \[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
[remote] type = box token =
|
|
{\[lq]access_token\[rq]:\[lq]YYY\[rq],\[lq]token_type\[rq]:\[lq]bearer\[rq],\[lq]refresh_token\[rq]:\[lq]YYY\[rq],\[lq]expiry\[rq]:\[lq]2017\-07\-23T12:22:29.259137901+01:00\[rq]}
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
.IP "a)" 3
|
|
Yes this is OK
|
|
.IP "b)" 3
|
|
Edit this remote
|
|
.IP "c)" 3
|
|
Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Modified\ time\ and\ hashes\ ###
|
|
|
|
Box\ allows\ modification\ times\ to\ be\ set\ on\ objects\ accurate\ to\ 1
|
|
second.\ \ These\ will\ be\ used\ to\ detect\ whether\ objects\ need\ syncing\ or
|
|
not.
|
|
|
|
Box\ supports\ SHA1\ type\ hashes,\ so\ you\ can\ use\ the\ `\-\-checksum`
|
|
flag.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
In\ addition\ to\ the\ [default\ restricted\ characters\ set](/overview/#restricted\-characters)
|
|
the\ following\ characters\ are\ also\ replaced:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ \\\ \ \ \ \ \ \ \ \ |\ 0x5C\ \ |\ ＼\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
File\ names\ can\ also\ not\ end\ with\ the\ following\ characters.
|
|
These\ only\ get\ replaced\ if\ they\ are\ last\ character\ in\ the\ name:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ SP\ \ \ \ \ \ \ \ |\ 0x20\ \ |\ ␠\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
###\ Transfers\ ###
|
|
|
|
For\ files\ above\ 50MB\ rclone\ will\ use\ a\ chunked\ transfer.\ \ Rclone\ will
|
|
upload\ up\ to\ `\-\-transfers`\ chunks\ at\ the\ same\ time\ (shared\ among\ all
|
|
the\ multipart\ uploads).\ \ Chunks\ are\ buffered\ in\ memory\ and\ are
|
|
normally\ 8MB\ so\ increasing\ `\-\-transfers`\ will\ increase\ memory\ use.
|
|
|
|
###\ Deleting\ files\ ###
|
|
|
|
Depending\ on\ the\ enterprise\ settings\ for\ your\ user,\ the\ item\ will
|
|
either\ be\ actually\ deleted\ from\ Box\ or\ moved\ to\ the\ trash.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/box/box.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ box\ (Box).
|
|
|
|
####\ \-\-box\-client\-id
|
|
|
|
Box\ App\ Client\ Id.
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_BOX_CLIENT_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-box\-client\-secret
|
|
|
|
Box\ App\ Client\ Secret
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_secret
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_BOX_CLIENT_SECRET
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-box\-box\-config\-file
|
|
|
|
Box\ App\ config.json\ location
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ box_config_file
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_BOX_BOX_CONFIG_FILE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-box\-box\-sub\-type
|
|
|
|
|
|
|
|
\-\ Config:\ \ \ \ \ \ box_sub_type
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_BOX_BOX_SUB_TYPE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "user"
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "user"
|
|
\ \ \ \ \ \ \ \ \-\ Rclone\ should\ act\ on\ behalf\ of\ a\ user
|
|
\ \ \ \ \-\ "enterprise"
|
|
\ \ \ \ \ \ \ \ \-\ Rclone\ should\ act\ on\ behalf\ of\ a\ service\ account
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ box\ (Box).
|
|
|
|
####\ \-\-box\-upload\-cutoff
|
|
|
|
Cutoff\ for\ switching\ to\ multipart\ upload\ (>=\ 50MB).
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_cutoff
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_BOX_UPLOAD_CUTOFF
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 50M
|
|
|
|
####\ \-\-box\-commit\-retries
|
|
|
|
Max\ number\ of\ times\ to\ try\ committing\ a\ multipart\ file.
|
|
|
|
\-\ Config:\ \ \ \ \ \ commit_retries
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_BOX_COMMIT_RETRIES
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 100
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
###\ Limitations\ ###
|
|
|
|
Note\ that\ Box\ is\ case\ insensitive\ so\ you\ can\[aq]t\ have\ a\ file\ called
|
|
"Hello.doc"\ and\ one\ called\ "hello.doc".
|
|
|
|
Box\ file\ names\ can\[aq]t\ have\ the\ `\\`\ character\ in.\ \ rclone\ maps\ this\ to
|
|
and\ from\ an\ identical\ looking\ unicode\ equivalent\ `＼`.
|
|
|
|
Box\ only\ supports\ filenames\ up\ to\ 255\ characters\ in\ length.
|
|
|
|
Cache\ (BETA)
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
The\ `cache`\ remote\ wraps\ another\ existing\ remote\ and\ stores\ file\ structure
|
|
and\ its\ data\ for\ long\ running\ tasks\ like\ `rclone\ mount`.
|
|
|
|
To\ get\ started\ you\ just\ need\ to\ have\ an\ existing\ remote\ which\ can\ be\ configured
|
|
with\ `cache`.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `test\-cache`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote r) Rename remote c)
|
|
Copy remote s) Set configuration password q) Quit config n/r/c/s/q> n
|
|
name> test\-cache Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Cache
|
|
a remote \ \[lq]cache\[rq] [snip] Storage> cache Remote to cache.
|
|
Normally should contain a `:' and a path, eg
|
|
\[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq] or maybe
|
|
\[lq]myremote:\[rq] (not recommended).
|
|
remote> local:/test Optional: The URL of the Plex server plex_url>
|
|
http://127.0.0.1:32400 Optional: The username of the Plex user
|
|
plex_username> dummyusername Optional: The password of the Plex user y)
|
|
Yes type in my own password g) Generate random password n) No leave this
|
|
optional password blank y/g/n> y Enter the password: password: Confirm
|
|
the password: password: The size of a chunk.
|
|
Lower value good for slow connections but can affect seamless reading.
|
|
Default: 5M Choose a number from below, or type in your own value 1 /
|
|
1MB \ \[lq]1m\[rq] 2 / 5 MB \ \[lq]5M\[rq] 3 / 10 MB \ \[lq]10M\[rq]
|
|
chunk_size> 2 How much time should object info (file size, file hashes
|
|
etc) be stored in cache.
|
|
Use a very high value if you don't plan on changing the source FS from
|
|
outside the cache.
|
|
Accepted units are: \[lq]s\[rq], \[lq]m\[rq], \[lq]h\[rq].
|
|
Default: 5m Choose a number from below, or type in your own value 1 / 1
|
|
hour \ \[lq]1h\[rq] 2 / 24 hours \ \[lq]24h\[rq] 3 / 24 hours
|
|
\ \[lq]48h\[rq] info_age> 2 The maximum size of stored chunks.
|
|
When the storage grows beyond this size, the oldest chunks will be
|
|
deleted.
|
|
Default: 10G Choose a number from below, or type in your own value 1 /
|
|
500 MB \ \[lq]500M\[rq] 2 / 1 GB \ \[lq]1G\[rq] 3 / 10 GB
|
|
\ \[lq]10G\[rq] chunk_total_size> 3 Remote config
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [test\-cache] remote = local:/test
|
|
plex_url = http://127.0.0.1:32400 plex_username = dummyusername
|
|
plex_password = *** ENCRYPTED *** chunk_size = 5M info_age = 48h
|
|
chunk_total_size = 10G
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
You\ can\ then\ use\ it\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ drive
|
|
|
|
\ \ \ \ rclone\ lsd\ test\-cache:
|
|
|
|
List\ all\ the\ files\ in\ your\ drive
|
|
|
|
\ \ \ \ rclone\ ls\ test\-cache:
|
|
|
|
To\ start\ a\ cached\ mount
|
|
|
|
\ \ \ \ rclone\ mount\ \-\-allow\-other\ test\-cache:\ /var/tmp/test\-cache
|
|
|
|
###\ Write\ Features\ ###
|
|
|
|
###\ Offline\ uploading\ ###
|
|
|
|
In\ an\ effort\ to\ make\ writing\ through\ cache\ more\ reliable,\ the\ backend\
|
|
now\ supports\ this\ feature\ which\ can\ be\ activated\ by\ specifying\ a
|
|
`cache\-tmp\-upload\-path`.
|
|
|
|
A\ files\ goes\ through\ these\ states\ when\ using\ this\ feature:
|
|
|
|
1.\ An\ upload\ is\ started\ (usually\ by\ copying\ a\ file\ on\ the\ cache\ remote)
|
|
2.\ When\ the\ copy\ to\ the\ temporary\ location\ is\ complete\ the\ file\ is\ part\
|
|
of\ the\ cached\ remote\ and\ looks\ and\ behaves\ like\ any\ other\ file\ (reading\ included)
|
|
3.\ After\ `cache\-tmp\-wait\-time`\ passes\ and\ the\ file\ is\ next\ in\ line,\ `rclone\ move`\
|
|
is\ used\ to\ move\ the\ file\ to\ the\ cloud\ provider
|
|
4.\ Reading\ the\ file\ still\ works\ during\ the\ upload\ but\ most\ modifications\ on\ it\ will\ be\ prohibited
|
|
5.\ Once\ the\ move\ is\ complete\ the\ file\ is\ unlocked\ for\ modifications\ as\ it
|
|
becomes\ as\ any\ other\ regular\ file
|
|
6.\ If\ the\ file\ is\ being\ read\ through\ `cache`\ when\ it\[aq]s\ actually
|
|
deleted\ from\ the\ temporary\ path\ then\ `cache`\ will\ simply\ swap\ the\ source
|
|
to\ the\ cloud\ provider\ without\ interrupting\ the\ reading\ (small\ blip\ can\ happen\ though)
|
|
|
|
Files\ are\ uploaded\ in\ sequence\ and\ only\ one\ file\ is\ uploaded\ at\ a\ time.
|
|
Uploads\ will\ be\ stored\ in\ a\ queue\ and\ be\ processed\ based\ on\ the\ order\ they\ were\ added.
|
|
The\ queue\ and\ the\ temporary\ storage\ is\ persistent\ across\ restarts\ but
|
|
can\ be\ cleared\ on\ startup\ with\ the\ `\-\-cache\-db\-purge`\ flag.
|
|
|
|
###\ Write\ Support\ ###
|
|
|
|
Writes\ are\ supported\ through\ `cache`.
|
|
One\ caveat\ is\ that\ a\ mounted\ cache\ remote\ does\ not\ add\ any\ retry\ or\ fallback
|
|
mechanism\ to\ the\ upload\ operation.\ This\ will\ depend\ on\ the\ implementation
|
|
of\ the\ wrapped\ remote.\ Consider\ using\ `Offline\ uploading`\ for\ reliable\ writes.
|
|
|
|
One\ special\ case\ is\ covered\ with\ `cache\-writes`\ which\ will\ cache\ the\ file
|
|
data\ at\ the\ same\ time\ as\ the\ upload\ when\ it\ is\ enabled\ making\ it\ available
|
|
from\ the\ cache\ store\ immediately\ once\ the\ upload\ is\ finished.
|
|
|
|
###\ Read\ Features\ ###
|
|
|
|
####\ Multiple\ connections\ ####
|
|
|
|
To\ counter\ the\ high\ latency\ between\ a\ local\ PC\ where\ rclone\ is\ running
|
|
and\ cloud\ providers,\ the\ cache\ remote\ can\ split\ multiple\ requests\ to\ the
|
|
cloud\ provider\ for\ smaller\ file\ chunks\ and\ combines\ them\ together\ locally
|
|
where\ they\ can\ be\ available\ almost\ immediately\ before\ the\ reader\ usually
|
|
needs\ them.
|
|
|
|
This\ is\ similar\ to\ buffering\ when\ media\ files\ are\ played\ online.\ Rclone
|
|
will\ stay\ around\ the\ current\ marker\ but\ always\ try\ its\ best\ to\ stay\ ahead
|
|
and\ prepare\ the\ data\ before.
|
|
|
|
####\ Plex\ Integration\ ####
|
|
|
|
There\ is\ a\ direct\ integration\ with\ Plex\ which\ allows\ cache\ to\ detect\ during\ reading
|
|
if\ the\ file\ is\ in\ playback\ or\ not.\ This\ helps\ cache\ to\ adapt\ how\ it\ queries
|
|
the\ cloud\ provider\ depending\ on\ what\ is\ needed\ for.
|
|
|
|
Scans\ will\ have\ a\ minimum\ amount\ of\ workers\ (1)\ while\ in\ a\ confirmed\ playback\ cache
|
|
will\ deploy\ the\ configured\ number\ of\ workers.
|
|
|
|
This\ integration\ opens\ the\ doorway\ to\ additional\ performance\ improvements
|
|
which\ will\ be\ explored\ in\ the\ near\ future.
|
|
|
|
**Note:**\ If\ Plex\ options\ are\ not\ configured,\ `cache`\ will\ function\ with\ its
|
|
configured\ options\ without\ adapting\ any\ of\ its\ settings.
|
|
|
|
How\ to\ enable?\ Run\ `rclone\ config`\ and\ add\ all\ the\ Plex\ options\ (endpoint,\ username
|
|
and\ password)\ in\ your\ remote\ and\ it\ will\ be\ automatically\ enabled.
|
|
|
|
Affected\ settings:
|
|
\-\ `cache\-workers`:\ _Configured\ value_\ during\ confirmed\ playback\ or\ _1_\ all\ the\ other\ times
|
|
|
|
#####\ Certificate\ Validation\ #####
|
|
|
|
When\ the\ Plex\ server\ is\ configured\ to\ only\ accept\ secure\ connections,\ it\ is
|
|
possible\ to\ use\ `.plex.direct`\ URL\[aq]s\ to\ ensure\ certificate\ validation\ succeeds.
|
|
These\ URL\[aq]s\ are\ used\ by\ Plex\ internally\ to\ connect\ to\ the\ Plex\ server\ securely.
|
|
|
|
The\ format\ for\ this\ URL\[aq]s\ is\ the\ following:
|
|
|
|
https://ip\-with\-dots\-replaced.server\-hash.plex.direct:32400/
|
|
|
|
The\ `ip\-with\-dots\-replaced`\ part\ can\ be\ any\ IPv4\ address,\ where\ the\ dots
|
|
have\ been\ replaced\ with\ dashes,\ e.g.\ `127.0.0.1`\ becomes\ `127\-0\-0\-1`.
|
|
|
|
To\ get\ the\ `server\-hash`\ part,\ the\ easiest\ way\ is\ to\ visit
|
|
|
|
https://plex.tv/api/resources?includeHttps=1&X\-Plex\-Token=your\-plex\-token
|
|
|
|
This\ page\ will\ list\ all\ the\ available\ Plex\ servers\ for\ your\ account
|
|
with\ at\ least\ one\ `.plex.direct`\ link\ for\ each.\ Copy\ one\ URL\ and\ replace
|
|
the\ IP\ address\ with\ the\ desired\ address.\ This\ can\ be\ used\ as\ the
|
|
`plex_url`\ value.
|
|
|
|
###\ Known\ issues\ ###
|
|
|
|
####\ Mount\ and\ \-\-dir\-cache\-time\ ####
|
|
|
|
\-\-dir\-cache\-time\ controls\ the\ first\ layer\ of\ directory\ caching\ which\ works\ at\ the\ mount\ layer.
|
|
Being\ an\ independent\ caching\ mechanism\ from\ the\ `cache`\ backend,\ it\ will\ manage\ its\ own\ entries
|
|
based\ on\ the\ configured\ time.
|
|
|
|
To\ avoid\ getting\ in\ a\ scenario\ where\ dir\ cache\ has\ obsolete\ data\ and\ cache\ would\ have\ the\ correct
|
|
one,\ try\ to\ set\ `\-\-dir\-cache\-time`\ to\ a\ lower\ time\ than\ `\-\-cache\-info\-age`.\ Default\ values\ are
|
|
already\ configured\ in\ this\ way.\
|
|
|
|
####\ Windows\ support\ \-\ Experimental\ ####
|
|
|
|
There\ are\ a\ couple\ of\ issues\ with\ Windows\ `mount`\ functionality\ that\ still\ require\ some\ investigations.
|
|
It\ should\ be\ considered\ as\ experimental\ thus\ far\ as\ fixes\ come\ in\ for\ this\ OS.
|
|
|
|
Most\ of\ the\ issues\ seem\ to\ be\ related\ to\ the\ difference\ between\ filesystems
|
|
on\ Linux\ flavors\ and\ Windows\ as\ cache\ is\ heavily\ dependant\ on\ them.
|
|
|
|
Any\ reports\ or\ feedback\ on\ how\ cache\ behaves\ on\ this\ OS\ is\ greatly\ appreciated.
|
|
\
|
|
\-\ https://github.com/rclone/rclone/issues/1935
|
|
\-\ https://github.com/rclone/rclone/issues/1907
|
|
\-\ https://github.com/rclone/rclone/issues/1834\
|
|
|
|
####\ Risk\ of\ throttling\ ####
|
|
|
|
Future\ iterations\ of\ the\ cache\ backend\ will\ make\ use\ of\ the\ pooling\ functionality
|
|
of\ the\ cloud\ provider\ to\ synchronize\ and\ at\ the\ same\ time\ make\ writing\ through\ it
|
|
more\ tolerant\ to\ failures.\
|
|
|
|
There\ are\ a\ couple\ of\ enhancements\ in\ track\ to\ add\ these\ but\ in\ the\ meantime
|
|
there\ is\ a\ valid\ concern\ that\ the\ expiring\ cache\ listings\ can\ lead\ to\ cloud\ provider
|
|
throttles\ or\ bans\ due\ to\ repeated\ queries\ on\ it\ for\ very\ large\ mounts.
|
|
|
|
Some\ recommendations:
|
|
\-\ don\[aq]t\ use\ a\ very\ small\ interval\ for\ entry\ informations\ (`\-\-cache\-info\-age`)
|
|
\-\ while\ writes\ aren\[aq]t\ yet\ optimised,\ you\ can\ still\ write\ through\ `cache`\ which\ gives\ you\ the\ advantage
|
|
of\ adding\ the\ file\ in\ the\ cache\ at\ the\ same\ time\ if\ configured\ to\ do\ so.
|
|
|
|
Future\ enhancements:
|
|
|
|
\-\ https://github.com/rclone/rclone/issues/1937
|
|
\-\ https://github.com/rclone/rclone/issues/1936\
|
|
|
|
####\ cache\ and\ crypt\ ####
|
|
|
|
One\ common\ scenario\ is\ to\ keep\ your\ data\ encrypted\ in\ the\ cloud\ provider
|
|
using\ the\ `crypt`\ remote.\ `crypt`\ uses\ a\ similar\ technique\ to\ wrap\ around
|
|
an\ existing\ remote\ and\ handles\ this\ translation\ in\ a\ seamless\ way.
|
|
|
|
There\ is\ an\ issue\ with\ wrapping\ the\ remotes\ in\ this\ order:
|
|
<span\ style="color:red">**cloud\ remote**\ \->\ **crypt**\ \->\ **cache**</span>
|
|
|
|
During\ testing,\ I\ experienced\ a\ lot\ of\ bans\ with\ the\ remotes\ in\ this\ order.
|
|
I\ suspect\ it\ might\ be\ related\ to\ how\ crypt\ opens\ files\ on\ the\ cloud\ provider
|
|
which\ makes\ it\ think\ we\[aq]re\ downloading\ the\ full\ file\ instead\ of\ small\ chunks.
|
|
Organizing\ the\ remotes\ in\ this\ order\ yields\ better\ results:
|
|
<span\ style="color:green">**cloud\ remote**\ \->\ **cache**\ \->\ **crypt**</span>
|
|
|
|
####\ absolute\ remote\ paths\ ####
|
|
|
|
`cache`\ can\ not\ differentiate\ between\ relative\ and\ absolute\ paths\ for\ the\ wrapped\ remote.
|
|
Any\ path\ given\ in\ the\ `remote`\ config\ setting\ and\ on\ the\ command\ line\ will\ be\ passed\ to
|
|
the\ wrapped\ remote\ as\ is,\ but\ for\ storing\ the\ chunks\ on\ disk\ the\ path\ will\ be\ made
|
|
relative\ by\ removing\ any\ leading\ `/`\ character.
|
|
|
|
This\ behavior\ is\ irrelevant\ for\ most\ backend\ types,\ but\ there\ are\ backends\ where\ a\ leading\ `/`
|
|
changes\ the\ effective\ directory,\ e.g.\ in\ the\ `sftp`\ backend\ paths\ starting\ with\ a\ `/`\ are
|
|
relative\ to\ the\ root\ of\ the\ SSH\ server\ and\ paths\ without\ are\ relative\ to\ the\ user\ home\ directory.
|
|
As\ a\ result\ `sftp:bin`\ and\ `sftp:/bin`\ will\ share\ the\ same\ cache\ folder,\ even\ if\ they\ represent
|
|
a\ different\ directory\ on\ the\ SSH\ server.
|
|
|
|
###\ Cache\ and\ Remote\ Control\ (\-\-rc)\ ###
|
|
Cache\ supports\ the\ new\ `\-\-rc`\ mode\ in\ rclone\ and\ can\ be\ remote\ controlled\ through\ the\ following\ end\ points:
|
|
By\ default,\ the\ listener\ is\ disabled\ if\ you\ do\ not\ add\ the\ flag.
|
|
|
|
###\ rc\ cache/expire
|
|
Purge\ a\ remote\ from\ the\ cache\ backend.\ Supports\ either\ a\ directory\ or\ a\ file.
|
|
It\ supports\ both\ encrypted\ and\ unencrypted\ file\ names\ if\ cache\ is\ wrapped\ by\ crypt.
|
|
|
|
Params:
|
|
\ \ \-\ **remote**\ =\ path\ to\ remote\ **(required)**
|
|
\ \ \-\ **withData**\ =\ true/false\ to\ delete\ cached\ data\ (chunks)\ as\ well\ _(optional,\ false\ by\ default)_
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/cache/cache.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ cache\ (Cache\ a\ remote).
|
|
|
|
####\ \-\-cache\-remote
|
|
|
|
Remote\ to\ cache.
|
|
Normally\ should\ contain\ a\ \[aq]:\[aq]\ and\ a\ path,\ eg\ "myremote:path/to/dir",
|
|
"myremote:bucket"\ or\ maybe\ "myremote:"\ (not\ recommended).
|
|
|
|
\-\ Config:\ \ \ \ \ \ remote
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_REMOTE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-plex\-url
|
|
|
|
The\ URL\ of\ the\ Plex\ server
|
|
|
|
\-\ Config:\ \ \ \ \ \ plex_url
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_PLEX_URL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-plex\-username
|
|
|
|
The\ username\ of\ the\ Plex\ user
|
|
|
|
\-\ Config:\ \ \ \ \ \ plex_username
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_PLEX_USERNAME
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-plex\-password
|
|
|
|
The\ password\ of\ the\ Plex\ user
|
|
|
|
\-\ Config:\ \ \ \ \ \ plex_password
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_PLEX_PASSWORD
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-chunk\-size
|
|
|
|
The\ size\ of\ a\ chunk\ (partial\ file\ data).
|
|
|
|
Use\ lower\ numbers\ for\ slower\ connections.\ If\ the\ chunk\ size\ is
|
|
changed,\ any\ downloaded\ chunks\ will\ be\ invalid\ and\ cache\-chunk\-path
|
|
will\ need\ to\ be\ cleared\ or\ unexpected\ EOF\ errors\ will\ occur.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 5M
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "1m"
|
|
\ \ \ \ \ \ \ \ \-\ 1MB
|
|
\ \ \ \ \-\ "5M"
|
|
\ \ \ \ \ \ \ \ \-\ 5\ MB
|
|
\ \ \ \ \-\ "10M"
|
|
\ \ \ \ \ \ \ \ \-\ 10\ MB
|
|
|
|
####\ \-\-cache\-info\-age
|
|
|
|
How\ long\ to\ cache\ file\ structure\ information\ (directory\ listings,\ file\ size,\ times\ etc).\
|
|
If\ all\ write\ operations\ are\ done\ through\ the\ cache\ then\ you\ can\ safely\ make
|
|
this\ value\ very\ large\ as\ the\ cache\ store\ will\ also\ be\ updated\ in\ real\ time.
|
|
|
|
\-\ Config:\ \ \ \ \ \ info_age
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_INFO_AGE
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 6h0m0s
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "1h"
|
|
\ \ \ \ \ \ \ \ \-\ 1\ hour
|
|
\ \ \ \ \-\ "24h"
|
|
\ \ \ \ \ \ \ \ \-\ 24\ hours
|
|
\ \ \ \ \-\ "48h"
|
|
\ \ \ \ \ \ \ \ \-\ 48\ hours
|
|
|
|
####\ \-\-cache\-chunk\-total\-size
|
|
|
|
The\ total\ size\ that\ the\ chunks\ can\ take\ up\ on\ the\ local\ disk.
|
|
|
|
If\ the\ cache\ exceeds\ this\ value\ then\ it\ will\ start\ to\ delete\ the
|
|
oldest\ chunks\ until\ it\ goes\ under\ this\ value.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_total_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_CHUNK_TOTAL_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 10G
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "500M"
|
|
\ \ \ \ \ \ \ \ \-\ 500\ MB
|
|
\ \ \ \ \-\ "1G"
|
|
\ \ \ \ \ \ \ \ \-\ 1\ GB
|
|
\ \ \ \ \-\ "10G"
|
|
\ \ \ \ \ \ \ \ \-\ 10\ GB
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ cache\ (Cache\ a\ remote).
|
|
|
|
####\ \-\-cache\-plex\-token
|
|
|
|
The\ plex\ token\ for\ authentication\ \-\ auto\ set\ normally
|
|
|
|
\-\ Config:\ \ \ \ \ \ plex_token
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_PLEX_TOKEN
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-plex\-insecure
|
|
|
|
Skip\ all\ certificate\ verifications\ when\ connecting\ to\ the\ Plex\ server
|
|
|
|
\-\ Config:\ \ \ \ \ \ plex_insecure
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_PLEX_INSECURE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-db\-path
|
|
|
|
Directory\ to\ store\ file\ structure\ metadata\ DB.
|
|
The\ remote\ name\ is\ used\ as\ the\ DB\ file\ name.
|
|
|
|
\-\ Config:\ \ \ \ \ \ db_path
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_DB_PATH
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "$HOME/.cache/rclone/cache\-backend"
|
|
|
|
####\ \-\-cache\-chunk\-path
|
|
|
|
Directory\ to\ cache\ chunk\ files.
|
|
|
|
Path\ to\ where\ partial\ file\ data\ (chunks)\ are\ stored\ locally.\ The\ remote
|
|
name\ is\ appended\ to\ the\ final\ path.
|
|
|
|
This\ config\ follows\ the\ "\-\-cache\-db\-path".\ If\ you\ specify\ a\ custom
|
|
location\ for\ "\-\-cache\-db\-path"\ and\ don\[aq]t\ specify\ one\ for\ "\-\-cache\-chunk\-path"
|
|
then\ "\-\-cache\-chunk\-path"\ will\ use\ the\ same\ path\ as\ "\-\-cache\-db\-path".
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_path
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_CHUNK_PATH
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "$HOME/.cache/rclone/cache\-backend"
|
|
|
|
####\ \-\-cache\-db\-purge
|
|
|
|
Clear\ all\ the\ cached\ data\ for\ this\ remote\ on\ start.
|
|
|
|
\-\ Config:\ \ \ \ \ \ db_purge
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_DB_PURGE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-cache\-chunk\-clean\-interval
|
|
|
|
How\ often\ should\ the\ cache\ perform\ cleanups\ of\ the\ chunk\ storage.
|
|
The\ default\ value\ should\ be\ ok\ for\ most\ people.\ If\ you\ find\ that\ the
|
|
cache\ goes\ over\ "cache\-chunk\-total\-size"\ too\ often\ then\ try\ to\ lower
|
|
this\ value\ to\ force\ it\ to\ perform\ cleanups\ more\ often.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_clean_interval
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_CHUNK_CLEAN_INTERVAL
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 1m0s
|
|
|
|
####\ \-\-cache\-read\-retries
|
|
|
|
How\ many\ times\ to\ retry\ a\ read\ from\ a\ cache\ storage.
|
|
|
|
Since\ reading\ from\ a\ cache\ stream\ is\ independent\ from\ downloading\ file
|
|
data,\ readers\ can\ get\ to\ a\ point\ where\ there\[aq]s\ no\ more\ data\ in\ the
|
|
cache.\ \ Most\ of\ the\ times\ this\ can\ indicate\ a\ connectivity\ issue\ if
|
|
cache\ isn\[aq]t\ able\ to\ provide\ file\ data\ anymore.
|
|
|
|
For\ really\ slow\ connections,\ increase\ this\ to\ a\ point\ where\ the\ stream\ is
|
|
able\ to\ provide\ data\ but\ your\ experience\ will\ be\ very\ stuttering.
|
|
|
|
\-\ Config:\ \ \ \ \ \ read_retries
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_READ_RETRIES
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 10
|
|
|
|
####\ \-\-cache\-workers
|
|
|
|
How\ many\ workers\ should\ run\ in\ parallel\ to\ download\ chunks.
|
|
|
|
Higher\ values\ will\ mean\ more\ parallel\ processing\ (better\ CPU\ needed)
|
|
and\ more\ concurrent\ requests\ on\ the\ cloud\ provider.\ \ This\ impacts
|
|
several\ aspects\ like\ the\ cloud\ provider\ API\ limits,\ more\ stress\ on\ the
|
|
hardware\ that\ rclone\ runs\ on\ but\ it\ also\ means\ that\ streams\ will\ be
|
|
more\ fluid\ and\ data\ will\ be\ available\ much\ more\ faster\ to\ readers.
|
|
|
|
**Note**:\ If\ the\ optional\ Plex\ integration\ is\ enabled\ then\ this
|
|
setting\ will\ adapt\ to\ the\ type\ of\ reading\ performed\ and\ the\ value
|
|
specified\ here\ will\ be\ used\ as\ a\ maximum\ number\ of\ workers\ to\ use.
|
|
|
|
\-\ Config:\ \ \ \ \ \ workers
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_WORKERS
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 4
|
|
|
|
####\ \-\-cache\-chunk\-no\-memory
|
|
|
|
Disable\ the\ in\-memory\ cache\ for\ storing\ chunks\ during\ streaming.
|
|
|
|
By\ default,\ cache\ will\ keep\ file\ data\ during\ streaming\ in\ RAM\ as\ well
|
|
to\ provide\ it\ to\ readers\ as\ fast\ as\ possible.
|
|
|
|
This\ transient\ data\ is\ evicted\ as\ soon\ as\ it\ is\ read\ and\ the\ number\ of
|
|
chunks\ stored\ doesn\[aq]t\ exceed\ the\ number\ of\ workers.\ However,\ depending
|
|
on\ other\ settings\ like\ "cache\-chunk\-size"\ and\ "cache\-workers"\ this\ footprint
|
|
can\ increase\ if\ there\ are\ parallel\ streams\ too\ (multiple\ files\ being\ read
|
|
at\ the\ same\ time).
|
|
|
|
If\ the\ hardware\ permits\ it,\ use\ this\ feature\ to\ provide\ an\ overall\ better
|
|
performance\ during\ streaming\ but\ it\ can\ also\ be\ disabled\ if\ RAM\ is\ not
|
|
available\ on\ the\ local\ machine.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_no_memory
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_CHUNK_NO_MEMORY
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-cache\-rps
|
|
|
|
Limits\ the\ number\ of\ requests\ per\ second\ to\ the\ source\ FS\ (\-1\ to\ disable)
|
|
|
|
This\ setting\ places\ a\ hard\ limit\ on\ the\ number\ of\ requests\ per\ second
|
|
that\ cache\ will\ be\ doing\ to\ the\ cloud\ provider\ remote\ and\ try\ to
|
|
respect\ that\ value\ by\ setting\ waits\ between\ reads.
|
|
|
|
If\ you\ find\ that\ you\[aq]re\ getting\ banned\ or\ limited\ on\ the\ cloud
|
|
provider\ through\ cache\ and\ know\ that\ a\ smaller\ number\ of\ requests\ per
|
|
second\ will\ allow\ you\ to\ work\ with\ it\ then\ you\ can\ use\ this\ setting
|
|
for\ that.
|
|
|
|
A\ good\ balance\ of\ all\ the\ other\ settings\ should\ make\ this\ setting
|
|
useless\ but\ it\ is\ available\ to\ set\ for\ more\ special\ cases.
|
|
|
|
**NOTE**:\ This\ will\ limit\ the\ number\ of\ requests\ during\ streams\ but
|
|
other\ API\ calls\ to\ the\ cloud\ provider\ like\ directory\ listings\ will
|
|
still\ pass.
|
|
|
|
\-\ Config:\ \ \ \ \ \ rps
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_RPS
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ \-1
|
|
|
|
####\ \-\-cache\-writes
|
|
|
|
Cache\ file\ data\ on\ writes\ through\ the\ FS
|
|
|
|
If\ you\ need\ to\ read\ files\ immediately\ after\ you\ upload\ them\ through
|
|
cache\ you\ can\ enable\ this\ flag\ to\ have\ their\ data\ stored\ in\ the
|
|
cache\ store\ at\ the\ same\ time\ during\ upload.
|
|
|
|
\-\ Config:\ \ \ \ \ \ writes
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_WRITES
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-cache\-tmp\-upload\-path
|
|
|
|
Directory\ to\ keep\ temporary\ files\ until\ they\ are\ uploaded.
|
|
|
|
This\ is\ the\ path\ where\ cache\ will\ use\ as\ a\ temporary\ storage\ for\ new
|
|
files\ that\ need\ to\ be\ uploaded\ to\ the\ cloud\ provider.
|
|
|
|
Specifying\ a\ value\ will\ enable\ this\ feature.\ Without\ it,\ it\ is
|
|
completely\ disabled\ and\ files\ will\ be\ uploaded\ directly\ to\ the\ cloud
|
|
provider
|
|
|
|
\-\ Config:\ \ \ \ \ \ tmp_upload_path
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_TMP_UPLOAD_PATH
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-cache\-tmp\-wait\-time
|
|
|
|
How\ long\ should\ files\ be\ stored\ in\ local\ cache\ before\ being\ uploaded
|
|
|
|
This\ is\ the\ duration\ that\ a\ file\ must\ wait\ in\ the\ temporary\ location
|
|
_cache\-tmp\-upload\-path_\ before\ it\ is\ selected\ for\ upload.
|
|
|
|
Note\ that\ only\ one\ file\ is\ uploaded\ at\ a\ time\ and\ it\ can\ take\ longer
|
|
to\ start\ the\ upload\ if\ a\ queue\ formed\ for\ this\ purpose.
|
|
|
|
\-\ Config:\ \ \ \ \ \ tmp_wait_time
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_TMP_WAIT_TIME
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 15s
|
|
|
|
####\ \-\-cache\-db\-wait\-time
|
|
|
|
How\ long\ to\ wait\ for\ the\ DB\ to\ be\ available\ \-\ 0\ is\ unlimited
|
|
|
|
Only\ one\ process\ can\ have\ the\ DB\ open\ at\ any\ one\ time,\ so\ rclone\ waits
|
|
for\ this\ duration\ for\ the\ DB\ to\ become\ available\ before\ it\ gives\ an
|
|
error.
|
|
|
|
If\ you\ set\ it\ to\ 0\ then\ it\ will\ wait\ forever.
|
|
|
|
\-\ Config:\ \ \ \ \ \ db_wait_time
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CACHE_DB_WAIT_TIME
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 1s
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
Chunker\ (BETA)
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
The\ `chunker`\ overlay\ transparently\ splits\ large\ files\ into\ smaller\ chunks
|
|
during\ upload\ to\ wrapped\ remote\ and\ transparently\ assembles\ them\ back
|
|
when\ the\ file\ is\ downloaded.\ This\ allows\ to\ effectively\ overcome\ size\ limits
|
|
imposed\ by\ storage\ providers.
|
|
|
|
To\ use\ it,\ first\ set\ up\ the\ underlying\ remote\ following\ the\ configuration
|
|
instructions\ for\ that\ remote.\ You\ can\ also\ use\ a\ local\ pathname\ instead\ of
|
|
a\ remote.
|
|
|
|
First\ check\ your\ chosen\ remote\ is\ working\ \-\ we\[aq]ll\ call\ it\ `remote:path`\ here.
|
|
Note\ that\ anything\ inside\ `remote:path`\ will\ be\ chunked\ and\ anything\ outside
|
|
won\[aq]t.\ This\ means\ that\ if\ you\ are\ using\ a\ bucket\ based\ remote\ (eg\ S3,\ B2,\ swift)
|
|
then\ you\ should\ probably\ put\ the\ bucket\ in\ the\ remote\ `s3:bucket`.
|
|
|
|
Now\ configure\ `chunker`\ using\ `rclone\ config`.\ We\ will\ call\ this\ one\ `overlay`
|
|
to\ separate\ it\ from\ the\ `remote`\ itself.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> overlay Type of storage to
|
|
configure.
|
|
Choose a number from below, or type in your own value [snip] XX /
|
|
Transparently chunk/split large files \ \[lq]chunker\[rq] [snip]
|
|
Storage> chunker Remote to chunk/unchunk.
|
|
Normally should contain a `:' and a path, eg
|
|
\[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq] or maybe
|
|
\[lq]myremote:\[rq] (not recommended).
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). remote> remote:path Files larger
|
|
than chunk size will be split in chunks. Enter a size with suffix
|
|
k,M,G,T. Press Enter for the default (\[rq]2G\[lq]). chunk_size> 100M
|
|
Choose how chunker handles hash sums. All modes but\[rq]none" require
|
|
metadata.
|
|
Enter a string value.
|
|
Press Enter for the default (\[lq]md5\[rq]).
|
|
Choose a number from below, or type in your own value 1 / Pass any hash
|
|
supported by wrapped remote for non\-chunked files, return nothing
|
|
otherwise \ \[lq]none\[rq] 2 / MD5 for composite files \ \[lq]md5\[rq] 3
|
|
/ SHA1 for composite files \ \[lq]sha1\[rq] 4 / MD5 for all files
|
|
\ \[lq]md5all\[rq] 5 / SHA1 for all files \ \[lq]sha1all\[rq] 6 /
|
|
Copying a file to chunker will request MD5 from the source falling back
|
|
to SHA1 if unsupported \ \[lq]md5quick\[rq] 7 / Similar to
|
|
\[lq]md5quick\[rq] but prefers SHA1 over MD5 \ \[lq]sha1quick\[rq]
|
|
hash_type> md5 Edit advanced config?
|
|
(y/n) y) Yes n) No y/n> n Remote config
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [overlay] type = chunker remote =
|
|
remote:bucket chunk_size = 100M hash_type = md5
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ Specifying\ the\ remote
|
|
|
|
In\ normal\ use,\ make\ sure\ the\ remote\ has\ a\ `:`\ in.\ If\ you\ specify\ the\ remote
|
|
without\ a\ `:`\ then\ rclone\ will\ use\ a\ local\ directory\ of\ that\ name.
|
|
So\ if\ you\ use\ a\ remote\ of\ `/path/to/secret/files`\ then\ rclone\ will
|
|
chunk\ stuff\ in\ that\ directory.\ If\ you\ use\ a\ remote\ of\ `name`\ then\ rclone
|
|
will\ put\ files\ in\ a\ directory\ called\ `name`\ in\ the\ current\ directory.
|
|
|
|
|
|
###\ Chunking
|
|
|
|
When\ rclone\ starts\ a\ file\ upload,\ chunker\ checks\ the\ file\ size.\ If\ it
|
|
doesn\[aq]t\ exceed\ the\ configured\ chunk\ size,\ chunker\ will\ just\ pass\ the\ file
|
|
to\ the\ wrapped\ remote.\ If\ a\ file\ is\ large,\ chunker\ will\ transparently\ cut
|
|
data\ in\ pieces\ with\ temporary\ names\ and\ stream\ them\ one\ by\ one,\ on\ the\ fly.
|
|
Each\ data\ chunk\ will\ contain\ the\ specified\ number\ of\ bytes,\ except\ for\ the
|
|
last\ one\ which\ may\ have\ less\ data.\ If\ file\ size\ is\ unknown\ in\ advance
|
|
(this\ is\ called\ a\ streaming\ upload),\ chunker\ will\ internally\ create
|
|
a\ temporary\ copy,\ record\ its\ size\ and\ repeat\ the\ above\ process.
|
|
|
|
When\ upload\ completes,\ temporary\ chunk\ files\ are\ finally\ renamed.
|
|
This\ scheme\ guarantees\ that\ operations\ can\ be\ run\ in\ parallel\ and\ look
|
|
from\ outside\ as\ atomic.
|
|
A\ similar\ method\ with\ hidden\ temporary\ chunks\ is\ used\ for\ other\ operations
|
|
(copy/move/rename\ etc).\ If\ an\ operation\ fails,\ hidden\ chunks\ are\ normally
|
|
destroyed,\ and\ the\ target\ composite\ file\ stays\ intact.
|
|
|
|
When\ a\ composite\ file\ download\ is\ requested,\ chunker\ transparently
|
|
assembles\ it\ by\ concatenating\ data\ chunks\ in\ order.\ As\ the\ split\ is\ trivial
|
|
one\ could\ even\ manually\ concatenate\ data\ chunks\ together\ to\ obtain\ the
|
|
original\ content.
|
|
|
|
When\ the\ `list`\ rclone\ command\ scans\ a\ directory\ on\ wrapped\ remote,
|
|
the\ potential\ chunk\ files\ are\ accounted\ for,\ grouped\ and\ assembled\ into
|
|
composite\ directory\ entries.\ Any\ temporary\ chunks\ are\ hidden.
|
|
|
|
List\ and\ other\ commands\ can\ sometimes\ come\ across\ composite\ files\ with
|
|
missing\ or\ invalid\ chunks,\ eg.\ shadowed\ by\ like\-named\ directory\ or
|
|
another\ file.\ This\ usually\ means\ that\ wrapped\ file\ system\ has\ been\ directly
|
|
tampered\ with\ or\ damaged.\ If\ chunker\ detects\ a\ missing\ chunk\ it\ will
|
|
by\ default\ print\ warning,\ skip\ the\ whole\ incomplete\ group\ of\ chunks\ but
|
|
proceed\ with\ current\ command.
|
|
You\ can\ set\ the\ `\-\-chunker\-fail\-hard`\ flag\ to\ have\ commands\ abort\ with
|
|
error\ message\ in\ such\ cases.
|
|
|
|
|
|
####\ Chunk\ names
|
|
|
|
The\ default\ chunk\ name\ format\ is\ `*.rclone\-chunk.###`,\ hence\ by\ default
|
|
chunk\ names\ are\ `BIG_FILE_NAME.rclone\-chunk.001`,
|
|
`BIG_FILE_NAME.rclone\-chunk.002`\ etc.\ You\ can\ configure\ a\ different\ name
|
|
format\ using\ the\ `\-\-chunker\-name\-format`\ option.\ The\ format\ uses\ asterisk
|
|
`*`\ as\ a\ placeholder\ for\ the\ base\ file\ name\ and\ one\ or\ more\ consecutive
|
|
hash\ characters\ `#`\ as\ a\ placeholder\ for\ sequential\ chunk\ number.
|
|
There\ must\ be\ one\ and\ only\ one\ asterisk.\ The\ number\ of\ consecutive\ hash
|
|
characters\ defines\ the\ minimum\ length\ of\ a\ string\ representing\ a\ chunk\ number.
|
|
If\ decimal\ chunk\ number\ has\ less\ digits\ than\ the\ number\ of\ hashes,\ it\ is
|
|
left\-padded\ by\ zeros.\ If\ the\ decimal\ string\ is\ longer,\ it\ is\ left\ intact.
|
|
By\ default\ numbering\ starts\ from\ 1\ but\ there\ is\ another\ option\ that\ allows
|
|
user\ to\ start\ from\ 0,\ eg.\ for\ compatibility\ with\ legacy\ software.
|
|
|
|
For\ example,\ if\ name\ format\ is\ `big_*\-##.part`\ and\ original\ file\ name\ is
|
|
`data.txt`\ and\ numbering\ starts\ from\ 0,\ then\ the\ first\ chunk\ will\ be\ named
|
|
`big_data.txt\-00.part`,\ the\ 99th\ chunk\ will\ be\ `big_data.txt\-98.part`
|
|
and\ the\ 302nd\ chunk\ will\ become\ `big_data.txt\-301.part`.
|
|
|
|
Note\ that\ `list`\ assembles\ composite\ directory\ entries\ only\ when\ chunk\ names
|
|
match\ the\ configured\ format\ and\ treats\ non\-conforming\ file\ names\ as\ normal
|
|
non\-chunked\ files.
|
|
|
|
|
|
###\ Metadata
|
|
|
|
Besides\ data\ chunks\ chunker\ will\ by\ default\ create\ metadata\ object\ for
|
|
a\ composite\ file.\ The\ object\ is\ named\ after\ the\ original\ file.
|
|
Chunker\ allows\ user\ to\ disable\ metadata\ completely\ (the\ `none`\ format).
|
|
Note\ that\ metadata\ is\ normally\ not\ created\ for\ files\ smaller\ than\ the
|
|
configured\ chunk\ size.\ This\ may\ change\ in\ future\ rclone\ releases.
|
|
|
|
####\ Simple\ JSON\ metadata\ format
|
|
|
|
This\ is\ the\ default\ format.\ It\ supports\ hash\ sums\ and\ chunk\ validation
|
|
for\ composite\ files.\ Meta\ objects\ carry\ the\ following\ fields:
|
|
|
|
\-\ `ver`\ \ \ \ \ \-\ version\ of\ format,\ currently\ `1`
|
|
\-\ `size`\ \ \ \ \-\ total\ size\ of\ composite\ file
|
|
\-\ `nchunks`\ \-\ number\ of\ data\ chunks\ in\ file
|
|
\-\ `md5`\ \ \ \ \ \-\ MD5\ hashsum\ of\ composite\ file\ (if\ present)
|
|
\-\ `sha1`\ \ \ \ \-\ SHA1\ hashsum\ (if\ present)
|
|
|
|
There\ is\ no\ field\ for\ composite\ file\ name\ as\ it\[aq]s\ simply\ equal\ to\ the\ name
|
|
of\ meta\ object\ on\ the\ wrapped\ remote.\ Please\ refer\ to\ respective\ sections
|
|
for\ details\ on\ hashsums\ and\ modified\ time\ handling.
|
|
|
|
####\ No\ metadata
|
|
|
|
You\ can\ disable\ meta\ objects\ by\ setting\ the\ meta\ format\ option\ to\ `none`.
|
|
In\ this\ mode\ chunker\ will\ scan\ directory\ for\ all\ files\ that\ follow
|
|
configured\ chunk\ name\ format,\ group\ them\ by\ detecting\ chunks\ with\ the\ same
|
|
base\ name\ and\ show\ group\ names\ as\ virtual\ composite\ files.
|
|
This\ method\ is\ more\ prone\ to\ missing\ chunk\ errors\ (especially\ missing
|
|
last\ chunk)\ than\ format\ with\ metadata\ enabled.
|
|
|
|
|
|
###\ Hashsums
|
|
|
|
Chunker\ supports\ hashsums\ only\ when\ a\ compatible\ metadata\ is\ present.
|
|
Hence,\ if\ you\ choose\ metadata\ format\ of\ `none`,\ chunker\ will\ report\ hashsum
|
|
as\ `UNSUPPORTED`.
|
|
|
|
Please\ note\ that\ by\ default\ metadata\ is\ stored\ only\ for\ composite\ files.
|
|
If\ a\ file\ is\ smaller\ than\ configured\ chunk\ size,\ chunker\ will\ transparently
|
|
redirect\ hash\ requests\ to\ wrapped\ remote,\ so\ support\ depends\ on\ that.
|
|
You\ will\ see\ the\ empty\ string\ as\ a\ hashsum\ of\ requested\ type\ for\ small
|
|
files\ if\ the\ wrapped\ remote\ doesn\[aq]t\ support\ it.
|
|
|
|
Many\ storage\ backends\ support\ MD5\ and\ SHA1\ hash\ types,\ so\ does\ chunker.
|
|
With\ chunker\ you\ can\ choose\ one\ or\ another\ but\ not\ both.
|
|
MD5\ is\ set\ by\ default\ as\ the\ most\ supported\ type.
|
|
Since\ chunker\ keeps\ hashes\ for\ composite\ files\ and\ falls\ back\ to\ the
|
|
wrapped\ remote\ hash\ for\ non\-chunked\ ones,\ we\ advise\ you\ to\ choose\ the\ same
|
|
hash\ type\ as\ supported\ by\ wrapped\ remote\ so\ that\ your\ file\ listings
|
|
look\ coherent.
|
|
|
|
If\ your\ storage\ backend\ does\ not\ support\ MD5\ or\ SHA1\ but\ you\ need\ consistent
|
|
file\ hashing,\ configure\ chunker\ with\ `md5all`\ or\ `sha1all`.\ These\ two\ modes
|
|
guarantee\ given\ hash\ for\ all\ files.\ If\ wrapped\ remote\ doesn\[aq]t\ support\ it,
|
|
chunker\ will\ then\ add\ metadata\ to\ all\ files,\ even\ small.\ However,\ this\ can
|
|
double\ the\ amount\ of\ small\ files\ in\ storage\ and\ incur\ additional\ service\ charges.
|
|
|
|
Normally,\ when\ a\ file\ is\ copied\ to\ chunker\ controlled\ remote,\ chunker
|
|
will\ ask\ the\ file\ source\ for\ compatible\ file\ hash\ and\ revert\ to\ on\-the\-fly
|
|
calculation\ if\ none\ is\ found.\ This\ involves\ some\ CPU\ overhead\ but\ provides
|
|
a\ guarantee\ that\ given\ hashsum\ is\ available.\ Also,\ chunker\ will\ reject
|
|
a\ server\-side\ copy\ or\ move\ operation\ if\ source\ and\ destination\ hashsum
|
|
types\ are\ different\ resulting\ in\ the\ extra\ network\ bandwidth,\ too.
|
|
In\ some\ rare\ cases\ this\ may\ be\ undesired,\ so\ chunker\ provides\ two\ optional
|
|
choices:\ `sha1quick`\ and\ `md5quick`.\ If\ the\ source\ does\ not\ support\ primary
|
|
hash\ type\ and\ the\ quick\ mode\ is\ enabled,\ chunker\ will\ try\ to\ fall\ back\ to
|
|
the\ secondary\ type.\ This\ will\ save\ CPU\ and\ bandwidth\ but\ can\ result\ in\ empty
|
|
hashsums\ at\ destination.\ Beware\ of\ consequences:\ the\ `sync`\ command\ will
|
|
revert\ (sometimes\ silently)\ to\ time/size\ comparison\ if\ compatible\ hashsums
|
|
between\ source\ and\ target\ are\ not\ found.
|
|
|
|
|
|
###\ Modified\ time
|
|
|
|
Chunker\ stores\ modification\ times\ using\ the\ wrapped\ remote\ so\ support
|
|
depends\ on\ that.\ For\ a\ small\ non\-chunked\ file\ the\ chunker\ overlay\ simply
|
|
manipulates\ modification\ time\ of\ the\ wrapped\ remote\ file.
|
|
For\ a\ composite\ file\ with\ metadata\ chunker\ will\ get\ and\ set
|
|
modification\ time\ of\ the\ metadata\ object\ on\ the\ wrapped\ remote.
|
|
If\ file\ is\ chunked\ but\ metadata\ format\ is\ `none`\ then\ chunker\ will
|
|
use\ modification\ time\ of\ the\ first\ data\ chunk.
|
|
|
|
|
|
###\ Migrations
|
|
|
|
The\ idiomatic\ way\ to\ migrate\ to\ a\ different\ chunk\ size,\ hash\ type\ or
|
|
chunk\ naming\ scheme\ is\ to:
|
|
|
|
\-\ Collect\ all\ your\ chunked\ files\ under\ a\ directory\ and\ have\ your
|
|
\ \ chunker\ remote\ point\ to\ it.
|
|
\-\ Create\ another\ directory\ (most\ probably\ on\ the\ same\ cloud\ storage)
|
|
\ \ and\ configure\ a\ new\ remote\ with\ desired\ metadata\ format,
|
|
\ \ hash\ type,\ chunk\ naming\ etc.
|
|
\-\ Now\ run\ `rclone\ sync\ oldchunks:\ newchunks:`\ and\ all\ your\ data
|
|
\ \ will\ be\ transparently\ converted\ in\ transfer.
|
|
\ \ This\ may\ take\ some\ time,\ yet\ chunker\ will\ try\ server\-side
|
|
\ \ copy\ if\ possible.
|
|
\-\ After\ checking\ data\ integrity\ you\ may\ remove\ configuration\ section
|
|
\ \ of\ the\ old\ remote.
|
|
|
|
If\ rclone\ gets\ killed\ during\ a\ long\ operation\ on\ a\ big\ composite\ file,
|
|
hidden\ temporary\ chunks\ may\ stay\ in\ the\ directory.\ They\ will\ not\ be
|
|
shown\ by\ the\ `list`\ command\ but\ will\ eat\ up\ your\ account\ quota.
|
|
Please\ note\ that\ the\ `deletefile`\ command\ deletes\ only\ active
|
|
chunks\ of\ a\ file.\ As\ a\ workaround,\ you\ can\ use\ remote\ of\ the\ wrapped
|
|
file\ system\ to\ see\ them.
|
|
An\ easy\ way\ to\ get\ rid\ of\ hidden\ garbage\ is\ to\ copy\ littered\ directory
|
|
somewhere\ using\ the\ chunker\ remote\ and\ purge\ the\ original\ directory.
|
|
The\ `copy`\ command\ will\ copy\ only\ active\ chunks\ while\ the\ `purge`\ will
|
|
remove\ everything\ including\ garbage.
|
|
|
|
|
|
###\ Caveats\ and\ Limitations
|
|
|
|
Chunker\ requires\ wrapped\ remote\ to\ support\ server\ side\ `move`\ (or\ `copy`\ +
|
|
`delete`)\ operations,\ otherwise\ it\ will\ explicitly\ refuse\ to\ start.
|
|
This\ is\ because\ it\ internally\ renames\ temporary\ chunk\ files\ to\ their\ final
|
|
names\ when\ an\ operation\ completes\ successfully.
|
|
|
|
Note\ that\ a\ move\ implemented\ using\ the\ copy\-and\-delete\ method\ may\ incur
|
|
double\ charging\ with\ some\ cloud\ storage\ providers.
|
|
|
|
Chunker\ will\ not\ automatically\ rename\ existing\ chunks\ when\ you\ run
|
|
`rclone\ config`\ on\ a\ live\ remote\ and\ change\ the\ chunk\ name\ format.
|
|
Beware\ that\ in\ result\ of\ this\ some\ files\ which\ have\ been\ treated\ as\ chunks
|
|
before\ the\ change\ can\ pop\ up\ in\ directory\ listings\ as\ normal\ files
|
|
and\ vice\ versa.\ The\ same\ warning\ holds\ for\ the\ chunk\ size.
|
|
If\ you\ desperately\ need\ to\ change\ critical\ chunking\ setings,\ you\ should
|
|
run\ data\ migration\ as\ described\ above.
|
|
|
|
If\ wrapped\ remote\ is\ case\ insensitive,\ the\ chunker\ overlay\ will\ inherit
|
|
that\ property\ (so\ you\ can\[aq]t\ have\ a\ file\ called\ "Hello.doc"\ and\ "hello.doc"
|
|
in\ the\ same\ directory).
|
|
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/chunker/chunker.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ chunker\ (Transparently\ chunk/split\ large\ files).
|
|
|
|
####\ \-\-chunker\-remote
|
|
|
|
Remote\ to\ chunk/unchunk.
|
|
Normally\ should\ contain\ a\ \[aq]:\[aq]\ and\ a\ path,\ eg\ "myremote:path/to/dir",
|
|
"myremote:bucket"\ or\ maybe\ "myremote:"\ (not\ recommended).
|
|
|
|
\-\ Config:\ \ \ \ \ \ remote
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_REMOTE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-chunker\-chunk\-size
|
|
|
|
Files\ larger\ than\ chunk\ size\ will\ be\ split\ in\ chunks.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 2G
|
|
|
|
####\ \-\-chunker\-hash\-type
|
|
|
|
Choose\ how\ chunker\ handles\ hash\ sums.\ All\ modes\ but\ "none"\ require\ metadata.
|
|
|
|
\-\ Config:\ \ \ \ \ \ hash_type
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_HASH_TYPE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "md5"
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "none"
|
|
\ \ \ \ \ \ \ \ \-\ Pass\ any\ hash\ supported\ by\ wrapped\ remote\ for\ non\-chunked\ files,\ return\ nothing\ otherwise
|
|
\ \ \ \ \-\ "md5"
|
|
\ \ \ \ \ \ \ \ \-\ MD5\ for\ composite\ files
|
|
\ \ \ \ \-\ "sha1"
|
|
\ \ \ \ \ \ \ \ \-\ SHA1\ for\ composite\ files
|
|
\ \ \ \ \-\ "md5all"
|
|
\ \ \ \ \ \ \ \ \-\ MD5\ for\ all\ files
|
|
\ \ \ \ \-\ "sha1all"
|
|
\ \ \ \ \ \ \ \ \-\ SHA1\ for\ all\ files
|
|
\ \ \ \ \-\ "md5quick"
|
|
\ \ \ \ \ \ \ \ \-\ Copying\ a\ file\ to\ chunker\ will\ request\ MD5\ from\ the\ source\ falling\ back\ to\ SHA1\ if\ unsupported
|
|
\ \ \ \ \-\ "sha1quick"
|
|
\ \ \ \ \ \ \ \ \-\ Similar\ to\ "md5quick"\ but\ prefers\ SHA1\ over\ MD5
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ chunker\ (Transparently\ chunk/split\ large\ files).
|
|
|
|
####\ \-\-chunker\-name\-format
|
|
|
|
String\ format\ of\ chunk\ file\ names.
|
|
The\ two\ placeholders\ are:\ base\ file\ name\ (*)\ and\ chunk\ number\ (#...).
|
|
There\ must\ be\ one\ and\ only\ one\ asterisk\ and\ one\ or\ more\ consecutive\ hash\ characters.
|
|
If\ chunk\ number\ has\ less\ digits\ than\ the\ number\ of\ hashes,\ it\ is\ left\-padded\ by\ zeros.
|
|
If\ there\ are\ more\ digits\ in\ the\ number,\ they\ are\ left\ as\ is.
|
|
Possible\ chunk\ files\ are\ ignored\ if\ their\ name\ does\ not\ match\ given\ format.
|
|
|
|
\-\ Config:\ \ \ \ \ \ name_format
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_NAME_FORMAT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "*.rclone_chunk.###"
|
|
|
|
####\ \-\-chunker\-start\-from
|
|
|
|
Minimum\ valid\ chunk\ number.\ Usually\ 0\ or\ 1.
|
|
By\ default\ chunk\ numbers\ start\ from\ 1.
|
|
|
|
\-\ Config:\ \ \ \ \ \ start_from
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_START_FROM
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 1
|
|
|
|
####\ \-\-chunker\-meta\-format
|
|
|
|
Format\ of\ the\ metadata\ object\ or\ "none".\ By\ default\ "simplejson".
|
|
Metadata\ is\ a\ small\ JSON\ file\ named\ after\ the\ composite\ file.
|
|
|
|
\-\ Config:\ \ \ \ \ \ meta_format
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_META_FORMAT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "simplejson"
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "none"
|
|
\ \ \ \ \ \ \ \ \-\ Do\ not\ use\ metadata\ files\ at\ all.\ Requires\ hash\ type\ "none".
|
|
\ \ \ \ \-\ "simplejson"
|
|
\ \ \ \ \ \ \ \ \-\ Simple\ JSON\ supports\ hash\ sums\ and\ chunk\ validation.
|
|
\ \ \ \ \ \ \ \ \-\ It\ has\ the\ following\ fields:\ ver,\ size,\ nchunks,\ md5,\ sha1.
|
|
|
|
####\ \-\-chunker\-fail\-hard
|
|
|
|
Choose\ how\ chunker\ should\ handle\ files\ with\ missing\ or\ invalid\ chunks.
|
|
|
|
\-\ Config:\ \ \ \ \ \ fail_hard
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CHUNKER_FAIL_HARD
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "true"
|
|
\ \ \ \ \ \ \ \ \-\ Report\ errors\ and\ abort\ current\ command.
|
|
\ \ \ \ \-\ "false"
|
|
\ \ \ \ \ \ \ \ \-\ Warn\ user,\ skip\ incomplete\ file\ and\ proceed.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
##\ Citrix\ ShareFile
|
|
|
|
[Citrix\ ShareFile](https://sharefile.com)\ is\ a\ secure\ file\ sharing\ and\ transfer\ service\ aimed\ as\ business.
|
|
|
|
The\ initial\ setup\ for\ Citrix\ ShareFile\ involves\ getting\ a\ token\ from
|
|
Citrix\ ShareFile\ which\ you\ can\ in\ your\ browser.\ \ `rclone\ config`\ walks\ you
|
|
through\ it.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> remote Type of storage to
|
|
configure.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value XX / Citrix Sharefile \ \[rq]sharefile" Storage>
|
|
sharefile ** See help for sharefile backend at:
|
|
https://rclone.org/sharefile/ **
|
|
.PP
|
|
ID of the root folder
|
|
.PP
|
|
Leave blank to access \[lq]Personal Folders\[rq].
|
|
You can use one of the standard values here or any folder ID (long hex
|
|
number ID).
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value 1 / Access the Personal Folders. (Default)
|
|
\ \[rq]" 2 / Access the Favorites folder.
|
|
\ \[lq]favorites\[rq] 3 / Access all the shared folders.
|
|
\ \[lq]allshared\[rq] 4 / Access all the individual connectors.
|
|
\ \[lq]connectors\[rq] 5 / Access the home, favorites, and shared
|
|
folders as well as the connectors.
|
|
\ \[lq]top\[rq] root_folder_id> Edit advanced config?
|
|
(y/n) y) Yes n) No y/n> n Remote config Use auto config?
|
|
* Say Y if not sure * Say N if you are working on a remote or headless
|
|
machine y) Yes n) No y/n> y If your browser doesn't open automatically
|
|
go to the following link: http://127.0.0.1:53682/auth?state=XXX Log in
|
|
and authorize rclone for access Waiting for code\&... Got code
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] type = sharefile endpoint =
|
|
https://XXX.sharefile.com token =
|
|
{\[lq]access_token\[rq]:\[lq]XXX\[rq],\[lq]token_type\[rq]:\[lq]bearer\[rq],\[lq]refresh_token\[rq]:\[lq]XXX\[rq],\[lq]expiry\[rq]:\[lq]2019\-09\-30T19:41:45.878561877+01:00\[rq]}
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
See\ the\ [remote\ setup\ docs](https://rclone.org/remote_setup/)\ for\ how\ to\ set\ it\ up\ on\ a
|
|
machine\ with\ no\ Internet\ browser\ available.
|
|
|
|
Note\ that\ rclone\ runs\ a\ webserver\ on\ your\ local\ machine\ to\ collect\ the
|
|
token\ as\ returned\ from\ Citrix\ ShareFile.\ This\ only\ runs\ from\ the\ moment\ it\ opens
|
|
your\ browser\ to\ the\ moment\ you\ get\ back\ the\ verification\ code.\ \ This
|
|
is\ on\ `http://127.0.0.1:53682/`\ and\ this\ it\ may\ require\ you\ to\ unblock
|
|
it\ temporarily\ if\ you\ are\ running\ a\ host\ firewall.
|
|
|
|
Once\ configured\ you\ can\ then\ use\ `rclone`\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ ShareFile
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ your\ ShareFile
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
To\ copy\ a\ local\ directory\ to\ an\ ShareFile\ directory\ called\ backup
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:backup
|
|
|
|
Paths\ may\ be\ as\ deep\ as\ required,\ eg\ `remote:directory/subdirectory`.
|
|
|
|
###\ Modified\ time\ and\ hashes\ ###
|
|
|
|
ShareFile\ allows\ modification\ times\ to\ be\ set\ on\ objects\ accurate\ to\ 1
|
|
second.\ \ These\ will\ be\ used\ to\ detect\ whether\ objects\ need\ syncing\ or
|
|
not.
|
|
|
|
ShareFile\ supports\ MD5\ type\ hashes,\ so\ you\ can\ use\ the\ `\-\-checksum`
|
|
flag.
|
|
|
|
###\ Transfers\ ###
|
|
|
|
For\ files\ above\ 128MB\ rclone\ will\ use\ a\ chunked\ transfer.\ \ Rclone\ will
|
|
upload\ up\ to\ `\-\-transfers`\ chunks\ at\ the\ same\ time\ (shared\ among\ all
|
|
the\ multipart\ uploads).\ \ Chunks\ are\ buffered\ in\ memory\ and\ are
|
|
normally\ 64MB\ so\ increasing\ `\-\-transfers`\ will\ increase\ memory\ use.
|
|
|
|
###\ Limitations\ ###
|
|
|
|
Note\ that\ ShareFile\ is\ case\ insensitive\ so\ you\ can\[aq]t\ have\ a\ file\ called
|
|
"Hello.doc"\ and\ one\ called\ "hello.doc".
|
|
|
|
ShareFile\ only\ supports\ filenames\ up\ to\ 256\ characters\ in\ length.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
In\ addition\ to\ the\ [default\ restricted\ characters\ set](/overview/#restricted\-characters)
|
|
the\ following\ characters\ are\ also\ replaced:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ \\\\\ \ \ \ \ \ \ \ |\ 0x5C\ \ |\ ＼\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ *\ \ \ \ \ \ \ \ \ |\ 0x2A\ \ |\ ＊\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ <\ \ \ \ \ \ \ \ \ |\ 0x3C\ \ |\ ＜\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ >\ \ \ \ \ \ \ \ \ |\ 0x3E\ \ |\ ＞\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ ?\ \ \ \ \ \ \ \ \ |\ 0x3F\ \ |\ ？\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ :\ \ \ \ \ \ \ \ \ |\ 0x3A\ \ |\ ：\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ \\|\ \ \ \ \ \ \ \ |\ 0x7C\ \ |\ ｜\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ "\ \ \ \ \ \ \ \ \ |\ 0x22\ \ |\ ＂\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
File\ names\ can\ also\ not\ start\ or\ end\ with\ the\ following\ characters.
|
|
These\ only\ get\ replaced\ if\ they\ are\ first\ or\ last\ character\ in\ the
|
|
name:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ SP\ \ \ \ \ \ \ \ |\ 0x20\ \ |\ ␠\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ .\ \ \ \ \ \ \ \ \ |\ 0x2E\ \ |\ ．\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/sharefile/sharefile.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ sharefile\ (Citrix\ Sharefile).
|
|
|
|
####\ \-\-sharefile\-root\-folder\-id
|
|
|
|
ID\ of\ the\ root\ folder
|
|
|
|
Leave\ blank\ to\ access\ "Personal\ Folders".\ \ You\ can\ use\ one\ of\ the
|
|
standard\ values\ here\ or\ any\ folder\ ID\ (long\ hex\ number\ ID).
|
|
|
|
\-\ Config:\ \ \ \ \ \ root_folder_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_SHAREFILE_ROOT_FOLDER_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Access\ the\ Personal\ Folders.\ (Default)
|
|
\ \ \ \ \-\ "favorites"
|
|
\ \ \ \ \ \ \ \ \-\ Access\ the\ Favorites\ folder.
|
|
\ \ \ \ \-\ "allshared"
|
|
\ \ \ \ \ \ \ \ \-\ Access\ all\ the\ shared\ folders.
|
|
\ \ \ \ \-\ "connectors"
|
|
\ \ \ \ \ \ \ \ \-\ Access\ all\ the\ individual\ connectors.
|
|
\ \ \ \ \-\ "top"
|
|
\ \ \ \ \ \ \ \ \-\ Access\ the\ home,\ favorites,\ and\ shared\ folders\ as\ well\ as\ the\ connectors.
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ sharefile\ (Citrix\ Sharefile).
|
|
|
|
####\ \-\-sharefile\-upload\-cutoff
|
|
|
|
Cutoff\ for\ switching\ to\ multipart\ upload.
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_cutoff
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_SHAREFILE_UPLOAD_CUTOFF
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 128M
|
|
|
|
####\ \-\-sharefile\-chunk\-size
|
|
|
|
Upload\ chunk\ size.\ Must\ a\ power\ of\ 2\ >=\ 256k.
|
|
|
|
Making\ this\ larger\ will\ improve\ performance,\ but\ note\ that\ each\ chunk
|
|
is\ buffered\ in\ memory\ one\ per\ transfer.
|
|
|
|
Reducing\ this\ will\ reduce\ memory\ usage\ but\ decrease\ performance.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_SHAREFILE_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 64M
|
|
|
|
####\ \-\-sharefile\-endpoint
|
|
|
|
Endpoint\ for\ API\ calls.
|
|
|
|
This\ is\ usually\ auto\ discovered\ as\ part\ of\ the\ oauth\ process,\ but\ can
|
|
be\ set\ manually\ to\ something\ like:\ https://XXX.sharefile.com
|
|
|
|
|
|
\-\ Config:\ \ \ \ \ \ endpoint
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_SHAREFILE_ENDPOINT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
Crypt
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
The\ `crypt`\ remote\ encrypts\ and\ decrypts\ another\ remote.
|
|
|
|
To\ use\ it\ first\ set\ up\ the\ underlying\ remote\ following\ the\ config
|
|
instructions\ for\ that\ remote.\ \ You\ can\ also\ use\ a\ local\ pathname
|
|
instead\ of\ a\ remote\ which\ will\ encrypt\ and\ decrypt\ from\ that\ directory
|
|
which\ might\ be\ useful\ for\ encrypting\ onto\ a\ USB\ stick\ for\ example.
|
|
|
|
First\ check\ your\ chosen\ remote\ is\ working\ \-\ we\[aq]ll\ call\ it
|
|
`remote:path`\ in\ these\ docs.\ \ Note\ that\ anything\ inside\ `remote:path`
|
|
will\ be\ encrypted\ and\ anything\ outside\ won\[aq]t.\ \ This\ means\ that\ if\ you
|
|
are\ using\ a\ bucket\ based\ remote\ (eg\ S3,\ B2,\ swift)\ then\ you\ should
|
|
probably\ put\ the\ bucket\ in\ the\ remote\ `s3:bucket`.\ If\ you\ just\ use
|
|
`s3:`\ then\ rclone\ will\ make\ encrypted\ bucket\ names\ too\ (if\ using\ file
|
|
name\ encryption)\ which\ may\ or\ may\ not\ be\ what\ you\ want.
|
|
|
|
Now\ configure\ `crypt`\ using\ `rclone\ config`.\ We\ will\ call\ this\ one
|
|
`secret`\ to\ differentiate\ it\ from\ the\ `remote`.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n
|
|
.PD 0
|
|
.P
|
|
.PD
|
|
name> secret Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX /
|
|
Encrypt/Decrypt a remote \ \[lq]crypt\[rq] [snip] Storage> crypt Remote
|
|
to encrypt/decrypt.
|
|
Normally should contain a `:' and a path, eg
|
|
\[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq] or maybe
|
|
\[lq]myremote:\[rq] (not recommended).
|
|
remote> remote:path How to encrypt the filenames.
|
|
Choose a number from below, or type in your own value 1 / Don't encrypt
|
|
the file names.
|
|
Adds a \[lq].bin\[rq] extension only.
|
|
\ \[lq]off\[rq] 2 / Encrypt the filenames see the docs for the details.
|
|
\ \[lq]standard\[rq] 3 / Very simple filename obfuscation.
|
|
\ \[lq]obfuscate\[rq] filename_encryption> 2 Option to either encrypt
|
|
directory names or leave them intact.
|
|
Choose a number from below, or type in your own value 1 / Encrypt
|
|
directory names.
|
|
\ \[lq]true\[rq] 2 / Don't encrypt directory names, leave them intact.
|
|
\ \[lq]false\[rq] filename_encryption> 1 Password or pass phrase for
|
|
encryption.
|
|
y) Yes type in my own password g) Generate random password y/g> y Enter
|
|
the password: password: Confirm the password: password: Password or pass
|
|
phrase for salt.
|
|
Optional but recommended.
|
|
Should be different to the previous password.
|
|
y) Yes type in my own password g) Generate random password n) No leave
|
|
this optional password blank y/g/n> g Password strength in bits.
|
|
64 is just about memorable 128 is secure 1024 is the maximum Bits> 128
|
|
Your password is: JAsJvRcgR\-_veXNfy_sGmQ Use this password?
|
|
y) Yes n) No y/n> y Remote config \[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
[secret] remote = remote:path filename_encryption = standard password =
|
|
*** ENCRYPTED \f[B]\f[BI] password2 = \f[B]\f[] ENCRYPTED ***
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
**Important**\ The\ password\ is\ stored\ in\ the\ config\ file\ is\ lightly
|
|
obscured\ so\ it\ isn\[aq]t\ immediately\ obvious\ what\ it\ is.\ \ It\ is\ in\ no\ way
|
|
secure\ unless\ you\ use\ config\ file\ encryption.
|
|
|
|
A\ long\ passphrase\ is\ recommended,\ or\ you\ can\ use\ a\ random\ one.\ \ Note
|
|
that\ if\ you\ reconfigure\ rclone\ with\ the\ same\ passwords/passphrases
|
|
elsewhere\ it\ will\ be\ compatible\ \-\ all\ the\ secrets\ used\ are\ derived
|
|
from\ those\ two\ passwords/passphrases.
|
|
|
|
Note\ that\ rclone\ does\ not\ encrypt
|
|
|
|
\ \ *\ file\ length\ \-\ this\ can\ be\ calcuated\ within\ 16\ bytes
|
|
\ \ *\ modification\ time\ \-\ used\ for\ syncing
|
|
|
|
##\ Specifying\ the\ remote\ ##
|
|
|
|
In\ normal\ use,\ make\ sure\ the\ remote\ has\ a\ `:`\ in.\ If\ you\ specify\ the
|
|
remote\ without\ a\ `:`\ then\ rclone\ will\ use\ a\ local\ directory\ of\ that
|
|
name.\ \ So\ if\ you\ use\ a\ remote\ of\ `/path/to/secret/files`\ then\ rclone
|
|
will\ encrypt\ stuff\ to\ that\ directory.\ \ If\ you\ use\ a\ remote\ of\ `name`
|
|
then\ rclone\ will\ put\ files\ in\ a\ directory\ called\ `name`\ in\ the\ current
|
|
directory.
|
|
|
|
If\ you\ specify\ the\ remote\ as\ `remote:path/to/dir`\ then\ rclone\ will
|
|
store\ encrypted\ files\ in\ `path/to/dir`\ on\ the\ remote.\ If\ you\ are\ using
|
|
file\ name\ encryption,\ then\ when\ you\ save\ files\ to
|
|
`secret:subdir/subfile`\ this\ will\ store\ them\ in\ the\ unencrypted\ path
|
|
`path/to/dir`\ but\ the\ `subdir/subpath`\ bit\ will\ be\ encrypted.
|
|
|
|
Note\ that\ unless\ you\ want\ encrypted\ bucket\ names\ (which\ are\ difficult
|
|
to\ manage\ because\ you\ won\[aq]t\ know\ what\ directory\ they\ represent\ in\ web
|
|
interfaces\ etc),\ you\ should\ probably\ specify\ a\ bucket,\ eg
|
|
`remote:secretbucket`\ when\ using\ bucket\ based\ remotes\ such\ as\ S3,
|
|
Swift,\ Hubic,\ B2,\ GCS.
|
|
|
|
##\ Example\ ##
|
|
|
|
To\ test\ I\ made\ a\ little\ directory\ of\ files\ using\ "standard"\ file\ name
|
|
encryption.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
plaintext/ ├── file0.txt ├── file1.txt └── subdir ├── file2.txt ├──
|
|
file3.txt └── subsubdir └── file4.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Copy\ these\ to\ the\ remote\ and\ list\ them\ back
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q copy plaintext secret: $ rclone \-q ls secret: 7 file1.txt
|
|
6 file0.txt 8 subdir/file2.txt 10 subdir/subsubdir/file4.txt 9
|
|
subdir/file3.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Now\ see\ what\ that\ looked\ like\ when\ encrypted
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q ls remote:path 55 hagjclgavj2mbiqm6u6cnjjqcg 54
|
|
v05749mltvv1tf4onltun46gls 57
|
|
86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo 58
|
|
86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
|
|
56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Note\ that\ this\ retains\ the\ directory\ structure\ which\ means\ you\ can\ do\ this
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q ls secret:subdir 8 file2.txt 9 file3.txt 10
|
|
subsubdir/file4.txt
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
If\ don\[aq]t\ use\ file\ name\ encryption\ then\ the\ remote\ will\ look\ like\ this
|
|
\-\ note\ the\ `.bin`\ extensions\ added\ to\ prevent\ the\ cloud\ provider
|
|
attempting\ to\ interpret\ the\ data.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
$ rclone \-q ls remote:path 54 file0.txt.bin 57 subdir/file3.txt.bin 56
|
|
subdir/file2.txt.bin 58 subdir/subsubdir/file4.txt.bin 55 file1.txt.bin
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ File\ name\ encryption\ modes\ ###
|
|
|
|
Here\ are\ some\ of\ the\ features\ of\ the\ file\ name\ encryption\ modes
|
|
|
|
Off
|
|
|
|
\ \ *\ doesn\[aq]t\ hide\ file\ names\ or\ directory\ structure
|
|
\ \ *\ allows\ for\ longer\ file\ names\ (~246\ characters)
|
|
\ \ *\ can\ use\ sub\ paths\ and\ copy\ single\ files
|
|
|
|
Standard
|
|
|
|
\ \ *\ file\ names\ encrypted
|
|
\ \ *\ file\ names\ can\[aq]t\ be\ as\ long\ (~143\ characters)
|
|
\ \ *\ can\ use\ sub\ paths\ and\ copy\ single\ files
|
|
\ \ *\ directory\ structure\ visible
|
|
\ \ *\ identical\ files\ names\ will\ have\ identical\ uploaded\ names
|
|
\ \ *\ can\ use\ shortcuts\ to\ shorten\ the\ directory\ recursion
|
|
|
|
Obfuscation
|
|
|
|
This\ is\ a\ simple\ "rotate"\ of\ the\ filename,\ with\ each\ file\ having\ a\ rot
|
|
distance\ based\ on\ the\ filename.\ We\ store\ the\ distance\ at\ the\ beginning
|
|
of\ the\ filename.\ So\ a\ file\ called\ "hello"\ may\ become\ "53.jgnnq"
|
|
|
|
This\ is\ not\ a\ strong\ encryption\ of\ filenames,\ but\ it\ may\ stop\ automated
|
|
scanning\ tools\ from\ picking\ up\ on\ filename\ patterns.\ As\ such\ it\[aq]s\ an
|
|
intermediate\ between\ "off"\ and\ "standard".\ The\ advantage\ is\ that\ it
|
|
allows\ for\ longer\ path\ segment\ names.
|
|
|
|
There\ is\ a\ possibility\ with\ some\ unicode\ based\ filenames\ that\ the
|
|
obfuscation\ is\ weak\ and\ may\ map\ lower\ case\ characters\ to\ upper\ case
|
|
equivalents.\ \ You\ can\ not\ rely\ on\ this\ for\ strong\ protection.
|
|
|
|
\ \ *\ file\ names\ very\ lightly\ obfuscated
|
|
\ \ *\ file\ names\ can\ be\ longer\ than\ standard\ encryption
|
|
\ \ *\ can\ use\ sub\ paths\ and\ copy\ single\ files
|
|
\ \ *\ directory\ structure\ visible
|
|
\ \ *\ identical\ files\ names\ will\ have\ identical\ uploaded\ names
|
|
|
|
Cloud\ storage\ systems\ have\ various\ limits\ on\ file\ name\ length\ and
|
|
total\ path\ length\ which\ you\ are\ more\ likely\ to\ hit\ using\ "Standard"
|
|
file\ name\ encryption.\ \ If\ you\ keep\ your\ file\ names\ to\ below\ 156
|
|
characters\ in\ length\ then\ you\ should\ be\ OK\ on\ all\ providers.
|
|
|
|
There\ may\ be\ an\ even\ more\ secure\ file\ name\ encryption\ mode\ in\ the
|
|
future\ which\ will\ address\ the\ long\ file\ name\ problem.
|
|
|
|
###\ Directory\ name\ encryption\ ###
|
|
Crypt\ offers\ the\ option\ of\ encrypting\ dir\ names\ or\ leaving\ them\ intact.
|
|
There\ are\ two\ options:
|
|
|
|
True
|
|
|
|
Encrypts\ the\ whole\ file\ path\ including\ directory\ names
|
|
Example:
|
|
`1/12/123.txt`\ is\ encrypted\ to
|
|
`p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0`
|
|
|
|
False
|
|
|
|
Only\ encrypts\ file\ names,\ skips\ directory\ names
|
|
Example:
|
|
`1/12/123.txt`\ is\ encrypted\ to
|
|
`1/12/qgm4avr35m5loi1th53ato71v0`
|
|
|
|
|
|
###\ Modified\ time\ and\ hashes\ ###
|
|
|
|
Crypt\ stores\ modification\ times\ using\ the\ underlying\ remote\ so\ support
|
|
depends\ on\ that.
|
|
|
|
Hashes\ are\ not\ stored\ for\ crypt.\ \ However\ the\ data\ integrity\ is
|
|
protected\ by\ an\ extremely\ strong\ crypto\ authenticator.
|
|
|
|
Note\ that\ you\ should\ use\ the\ `rclone\ cryptcheck`\ command\ to\ check\ the
|
|
integrity\ of\ a\ crypted\ remote\ instead\ of\ `rclone\ check`\ which\ can\[aq]t
|
|
check\ the\ checksums\ properly.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/crypt/crypt.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ crypt\ (Encrypt/Decrypt\ a\ remote).
|
|
|
|
####\ \-\-crypt\-remote
|
|
|
|
Remote\ to\ encrypt/decrypt.
|
|
Normally\ should\ contain\ a\ \[aq]:\[aq]\ and\ a\ path,\ eg\ "myremote:path/to/dir",
|
|
"myremote:bucket"\ or\ maybe\ "myremote:"\ (not\ recommended).
|
|
|
|
\-\ Config:\ \ \ \ \ \ remote
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CRYPT_REMOTE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-crypt\-filename\-encryption
|
|
|
|
How\ to\ encrypt\ the\ filenames.
|
|
|
|
\-\ Config:\ \ \ \ \ \ filename_encryption
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CRYPT_FILENAME_ENCRYPTION
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "standard"
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "off"
|
|
\ \ \ \ \ \ \ \ \-\ Don\[aq]t\ encrypt\ the\ file\ names.\ \ Adds\ a\ ".bin"\ extension\ only.
|
|
\ \ \ \ \-\ "standard"
|
|
\ \ \ \ \ \ \ \ \-\ Encrypt\ the\ filenames\ see\ the\ docs\ for\ the\ details.
|
|
\ \ \ \ \-\ "obfuscate"
|
|
\ \ \ \ \ \ \ \ \-\ Very\ simple\ filename\ obfuscation.
|
|
|
|
####\ \-\-crypt\-directory\-name\-encryption
|
|
|
|
Option\ to\ either\ encrypt\ directory\ names\ or\ leave\ them\ intact.
|
|
|
|
\-\ Config:\ \ \ \ \ \ directory_name_encryption
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ true
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "true"
|
|
\ \ \ \ \ \ \ \ \-\ Encrypt\ directory\ names.
|
|
\ \ \ \ \-\ "false"
|
|
\ \ \ \ \ \ \ \ \-\ Don\[aq]t\ encrypt\ directory\ names,\ leave\ them\ intact.
|
|
|
|
####\ \-\-crypt\-password
|
|
|
|
Password\ or\ pass\ phrase\ for\ encryption.
|
|
|
|
\-\ Config:\ \ \ \ \ \ password
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CRYPT_PASSWORD
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-crypt\-password2
|
|
|
|
Password\ or\ pass\ phrase\ for\ salt.\ Optional\ but\ recommended.
|
|
Should\ be\ different\ to\ the\ previous\ password.
|
|
|
|
\-\ Config:\ \ \ \ \ \ password2
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CRYPT_PASSWORD2
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ crypt\ (Encrypt/Decrypt\ a\ remote).
|
|
|
|
####\ \-\-crypt\-show\-mapping
|
|
|
|
For\ all\ files\ listed\ show\ how\ the\ names\ encrypt.
|
|
|
|
If\ this\ flag\ is\ set\ then\ for\ each\ file\ that\ the\ remote\ is\ asked\ to
|
|
list,\ it\ will\ log\ (at\ level\ INFO)\ a\ line\ stating\ the\ decrypted\ file
|
|
name\ and\ the\ encrypted\ file\ name.
|
|
|
|
This\ is\ so\ you\ can\ work\ out\ which\ encrypted\ names\ are\ which\ decrypted
|
|
names\ just\ in\ case\ you\ need\ to\ do\ something\ with\ the\ encrypted\ file
|
|
names,\ or\ for\ debugging\ purposes.
|
|
|
|
\-\ Config:\ \ \ \ \ \ show_mapping
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_CRYPT_SHOW_MAPPING
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
##\ Backing\ up\ a\ crypted\ remote\ ##
|
|
|
|
If\ you\ wish\ to\ backup\ a\ crypted\ remote,\ it\ it\ recommended\ that\ you\ use
|
|
`rclone\ sync`\ on\ the\ encrypted\ files,\ and\ make\ sure\ the\ passwords\ are
|
|
the\ same\ in\ the\ new\ encrypted\ remote.
|
|
|
|
This\ will\ have\ the\ following\ advantages
|
|
|
|
\ \ *\ `rclone\ sync`\ will\ check\ the\ checksums\ while\ copying
|
|
\ \ *\ you\ can\ use\ `rclone\ check`\ between\ the\ encrypted\ remotes
|
|
\ \ *\ you\ don\[aq]t\ decrypt\ and\ encrypt\ unnecessarily
|
|
|
|
For\ example,\ let\[aq]s\ say\ you\ have\ your\ original\ remote\ at\ `remote:`\ with
|
|
the\ encrypted\ version\ at\ `eremote:`\ with\ path\ `remote:crypt`.\ \ You
|
|
would\ then\ set\ up\ the\ new\ remote\ `remote2:`\ and\ then\ the\ encrypted
|
|
version\ `eremote2:`\ with\ path\ `remote2:crypt`\ using\ the\ same\ passwords
|
|
as\ `eremote:`.
|
|
|
|
To\ sync\ the\ two\ remotes\ you\ would\ do
|
|
|
|
\ \ \ \ rclone\ sync\ remote:crypt\ remote2:crypt
|
|
|
|
And\ to\ check\ the\ integrity\ you\ would\ do
|
|
|
|
\ \ \ \ rclone\ check\ remote:crypt\ remote2:crypt
|
|
|
|
##\ File\ formats\ ##
|
|
|
|
###\ File\ encryption\ ###
|
|
|
|
Files\ are\ encrypted\ 1:1\ source\ file\ to\ destination\ object.\ \ The\ file
|
|
has\ a\ header\ and\ is\ divided\ into\ chunks.
|
|
|
|
####\ Header\ ####
|
|
|
|
\ \ *\ 8\ bytes\ magic\ string\ `RCLONE\\x00\\x00`
|
|
\ \ *\ 24\ bytes\ Nonce\ (IV)
|
|
|
|
The\ initial\ nonce\ is\ generated\ from\ the\ operating\ systems\ crypto
|
|
strong\ random\ number\ generator.\ \ The\ nonce\ is\ incremented\ for\ each
|
|
chunk\ read\ making\ sure\ each\ nonce\ is\ unique\ for\ each\ block\ written.
|
|
The\ chance\ of\ a\ nonce\ being\ re\-used\ is\ minuscule.\ \ If\ you\ wrote\ an
|
|
exabyte\ of\ data\ (10¹⁸\ bytes)\ you\ would\ have\ a\ probability\ of
|
|
approximately\ 2×10⁻³²\ of\ re\-using\ a\ nonce.
|
|
|
|
####\ Chunk\ ####
|
|
|
|
Each\ chunk\ will\ contain\ 64kB\ of\ data,\ except\ for\ the\ last\ one\ which
|
|
may\ have\ less\ data.\ \ The\ data\ chunk\ is\ in\ standard\ NACL\ secretbox
|
|
format.\ Secretbox\ uses\ XSalsa20\ and\ Poly1305\ to\ encrypt\ and
|
|
authenticate\ messages.
|
|
|
|
Each\ chunk\ contains:
|
|
|
|
\ \ *\ 16\ Bytes\ of\ Poly1305\ authenticator
|
|
\ \ *\ 1\ \-\ 65536\ bytes\ XSalsa20\ encrypted\ data
|
|
|
|
64k\ chunk\ size\ was\ chosen\ as\ the\ best\ performing\ chunk\ size\ (the
|
|
authenticator\ takes\ too\ much\ time\ below\ this\ and\ the\ performance\ drops
|
|
off\ due\ to\ cache\ effects\ above\ this).\ \ Note\ that\ these\ chunks\ are
|
|
buffered\ in\ memory\ so\ they\ can\[aq]t\ be\ too\ big.
|
|
|
|
This\ uses\ a\ 32\ byte\ (256\ bit\ key)\ key\ derived\ from\ the\ user\ password.
|
|
|
|
####\ Examples\ ####
|
|
|
|
1\ byte\ file\ will\ encrypt\ to
|
|
|
|
\ \ *\ 32\ bytes\ header
|
|
\ \ *\ 17\ bytes\ data\ chunk
|
|
|
|
49\ bytes\ total
|
|
|
|
1MB\ (1048576\ bytes)\ file\ will\ encrypt\ to
|
|
|
|
\ \ *\ 32\ bytes\ header
|
|
\ \ *\ 16\ chunks\ of\ 65568\ bytes
|
|
|
|
1049120\ bytes\ total\ (a\ 0.05%\ overhead).\ \ This\ is\ the\ overhead\ for\ big
|
|
files.
|
|
|
|
###\ Name\ encryption\ ###
|
|
|
|
File\ names\ are\ encrypted\ segment\ by\ segment\ \-\ the\ path\ is\ broken\ up
|
|
into\ `/`\ separated\ strings\ and\ these\ are\ encrypted\ individually.
|
|
|
|
File\ segments\ are\ padded\ using\ using\ PKCS#7\ to\ a\ multiple\ of\ 16\ bytes
|
|
before\ encryption.
|
|
|
|
They\ are\ then\ encrypted\ with\ EME\ using\ AES\ with\ 256\ bit\ key.\ EME
|
|
(ECB\-Mix\-ECB)\ is\ a\ wide\-block\ encryption\ mode\ presented\ in\ the\ 2003
|
|
paper\ "A\ Parallelizable\ Enciphering\ Mode"\ by\ Halevi\ and\ Rogaway.
|
|
|
|
This\ makes\ for\ deterministic\ encryption\ which\ is\ what\ we\ want\ \-\ the
|
|
same\ filename\ must\ encrypt\ to\ the\ same\ thing\ otherwise\ we\ can\[aq]t\ find
|
|
it\ on\ the\ cloud\ storage\ system.
|
|
|
|
This\ means\ that
|
|
|
|
\ \ *\ filenames\ with\ the\ same\ name\ will\ encrypt\ the\ same
|
|
\ \ *\ filenames\ which\ start\ the\ same\ won\[aq]t\ have\ a\ common\ prefix
|
|
|
|
This\ uses\ a\ 32\ byte\ key\ (256\ bits)\ and\ a\ 16\ byte\ (128\ bits)\ IV\ both\ of
|
|
which\ are\ derived\ from\ the\ user\ password.
|
|
|
|
After\ encryption\ they\ are\ written\ out\ using\ a\ modified\ version\ of
|
|
standard\ `base32`\ encoding\ as\ described\ in\ RFC4648.\ \ The\ standard
|
|
encoding\ is\ modified\ in\ two\ ways:
|
|
|
|
\ \ *\ it\ becomes\ lower\ case\ (no\-one\ likes\ upper\ case\ filenames!)
|
|
\ \ *\ we\ strip\ the\ padding\ character\ `=`
|
|
|
|
`base32`\ is\ used\ rather\ than\ the\ more\ efficient\ `base64`\ so\ rclone\ can\ be
|
|
used\ on\ case\ insensitive\ remotes\ (eg\ Windows,\ Amazon\ Drive).
|
|
|
|
###\ Key\ derivation\ ###
|
|
|
|
Rclone\ uses\ `scrypt`\ with\ parameters\ `N=16384,\ r=8,\ p=1`\ with\ an
|
|
optional\ user\ supplied\ salt\ (password2)\ to\ derive\ the\ 32+32+16\ =\ 80
|
|
bytes\ of\ key\ material\ required.\ \ If\ the\ user\ doesn\[aq]t\ supply\ a\ salt
|
|
then\ rclone\ uses\ an\ internal\ one.
|
|
|
|
`scrypt`\ makes\ it\ impractical\ to\ mount\ a\ dictionary\ attack\ on\ rclone
|
|
encrypted\ data.\ \ For\ full\ protection\ against\ this\ you\ should\ always\ use
|
|
a\ salt.
|
|
|
|
Dropbox
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
Paths\ are\ specified\ as\ `remote:path`
|
|
|
|
Dropbox\ paths\ may\ be\ as\ deep\ as\ required,\ eg
|
|
`remote:directory/subdirectory`.
|
|
|
|
The\ initial\ setup\ for\ dropbox\ involves\ getting\ a\ token\ from\ Dropbox
|
|
which\ you\ need\ to\ do\ in\ your\ browser.\ \ `rclone\ config`\ walks\ you
|
|
through\ it.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.IP "n)" 3
|
|
New remote
|
|
.IP "o)" 3
|
|
Delete remote
|
|
.IP "p)" 3
|
|
Quit config e/n/d/q> n name> remote Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX /
|
|
Dropbox \ \[lq]dropbox\[rq] [snip] Storage> dropbox Dropbox App Key \-
|
|
leave blank normally.
|
|
app_key> Dropbox App Secret \- leave blank normally.
|
|
app_secret> Remote config Please visit:
|
|
https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
|
|
Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] app_key = app_secret =
|
|
token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
.IP "q)" 3
|
|
Yes this is OK
|
|
.IP "r)" 3
|
|
Edit this remote
|
|
.IP "s)" 3
|
|
Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
You\ can\ then\ use\ it\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ dropbox
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ your\ dropbox
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
To\ copy\ a\ local\ directory\ to\ a\ dropbox\ directory\ called\ backup
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:backup
|
|
|
|
###\ Dropbox\ for\ business\ ###
|
|
|
|
Rclone\ supports\ Dropbox\ for\ business\ and\ Team\ Folders.
|
|
|
|
When\ using\ Dropbox\ for\ business\ `remote:`\ and\ `remote:path/to/file`
|
|
will\ refer\ to\ your\ personal\ folder.
|
|
|
|
If\ you\ wish\ to\ see\ Team\ Folders\ you\ must\ use\ a\ leading\ `/`\ in\ the
|
|
path,\ so\ `rclone\ lsd\ remote:/`\ will\ refer\ to\ the\ root\ and\ show\ you\ all
|
|
Team\ Folders\ and\ your\ User\ Folder.
|
|
|
|
You\ can\ then\ use\ team\ folders\ like\ this\ `remote:/TeamFolder`\ and
|
|
`remote:/TeamFolder/path/to/file`.
|
|
|
|
A\ leading\ `/`\ for\ a\ Dropbox\ personal\ account\ will\ do\ nothing,\ but\ it
|
|
will\ take\ an\ extra\ HTTP\ transaction\ so\ it\ should\ be\ avoided.
|
|
|
|
###\ Modified\ time\ and\ Hashes\ ###
|
|
|
|
Dropbox\ supports\ modified\ times,\ but\ the\ only\ way\ to\ set\ a
|
|
modification\ time\ is\ to\ re\-upload\ the\ file.
|
|
|
|
This\ means\ that\ if\ you\ uploaded\ your\ data\ with\ an\ older\ version\ of
|
|
rclone\ which\ didn\[aq]t\ support\ the\ v2\ API\ and\ modified\ times,\ rclone\ will
|
|
decide\ to\ upload\ all\ your\ old\ data\ to\ fix\ the\ modification\ times.\ \ If
|
|
you\ don\[aq]t\ want\ this\ to\ happen\ use\ `\-\-size\-only`\ or\ `\-\-checksum`\ flag
|
|
to\ stop\ it.
|
|
|
|
Dropbox\ supports\ [its\ own\ hash
|
|
type](https://www.dropbox.com/developers/reference/content\-hash)\ which
|
|
is\ checked\ for\ all\ transfers.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ NUL\ \ \ \ \ \ \ |\ 0x00\ \ |\ ␀\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ /\ \ \ \ \ \ \ \ \ |\ 0x2F\ \ |\ ／\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ DEL\ \ \ \ \ \ \ |\ 0x7F\ \ |\ ␡\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ \\\ \ \ \ \ \ \ \ \ |\ 0x5C\ \ |\ ＼\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
File\ names\ can\ also\ not\ end\ with\ the\ following\ characters.
|
|
These\ only\ get\ replaced\ if\ they\ are\ last\ character\ in\ the\ name:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ SP\ \ \ \ \ \ \ \ |\ 0x20\ \ |\ ␠\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/dropbox/dropbox.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ dropbox\ (Dropbox).
|
|
|
|
####\ \-\-dropbox\-client\-id
|
|
|
|
Dropbox\ App\ Client\ Id
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DROPBOX_CLIENT_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-dropbox\-client\-secret
|
|
|
|
Dropbox\ App\ Client\ Secret
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_secret
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DROPBOX_CLIENT_SECRET
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ dropbox\ (Dropbox).
|
|
|
|
####\ \-\-dropbox\-chunk\-size
|
|
|
|
Upload\ chunk\ size.\ (<\ 150M).
|
|
|
|
Any\ files\ larger\ than\ this\ will\ be\ uploaded\ in\ chunks\ of\ this\ size.
|
|
|
|
Note\ that\ chunks\ are\ buffered\ in\ memory\ (one\ at\ a\ time)\ so\ rclone\ can
|
|
deal\ with\ retries.\ \ Setting\ this\ larger\ will\ increase\ the\ speed
|
|
slightly\ (at\ most\ 10%\ for\ 128MB\ in\ tests)\ at\ the\ cost\ of\ using\ more
|
|
memory.\ \ It\ can\ be\ set\ smaller\ if\ you\ are\ tight\ on\ memory.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DROPBOX_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 48M
|
|
|
|
####\ \-\-dropbox\-impersonate
|
|
|
|
Impersonate\ this\ user\ when\ using\ a\ business\ account.
|
|
|
|
\-\ Config:\ \ \ \ \ \ impersonate
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DROPBOX_IMPERSONATE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
###\ Limitations\ ###
|
|
|
|
Note\ that\ Dropbox\ is\ case\ insensitive\ so\ you\ can\[aq]t\ have\ a\ file\ called
|
|
"Hello.doc"\ and\ one\ called\ "hello.doc".
|
|
|
|
There\ are\ some\ file\ names\ such\ as\ `thumbs.db`\ which\ Dropbox\ can\[aq]t
|
|
store.\ \ There\ is\ a\ full\ list\ of\ them\ in\ the\ ["Ignored\ Files"\ section
|
|
of\ this\ document](https://www.dropbox.com/en/help/145).\ \ Rclone\ will
|
|
issue\ an\ error\ message\ `File\ name\ disallowed\ \-\ not\ uploading`\ if\ it
|
|
attempts\ to\ upload\ one\ of\ those\ file\ names,\ but\ the\ sync\ won\[aq]t\ fail.
|
|
|
|
If\ you\ have\ more\ than\ 10,000\ files\ in\ a\ directory\ then\ `rclone\ purge
|
|
dropbox:dir`\ will\ return\ the\ error\ `Failed\ to\ purge:\ There\ are\ too
|
|
many\ files\ involved\ in\ this\ operation`.\ \ As\ a\ work\-around\ do\ an
|
|
`rclone\ delete\ dropbox:dir`\ followed\ by\ an\ `rclone\ rmdir\ dropbox:dir`.
|
|
|
|
FTP
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
FTP\ is\ the\ File\ Transfer\ Protocol.\ FTP\ support\ is\ provided\ using\ the
|
|
[github.com/jlaffaye/ftp](https://godoc.org/github.com/jlaffaye/ftp)
|
|
package.
|
|
|
|
Here\ is\ an\ example\ of\ making\ an\ FTP\ configuration.\ \ First\ run
|
|
|
|
\ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process.\ An\ FTP\ remote\ only
|
|
needs\ a\ host\ together\ with\ and\ a\ username\ and\ a\ password.\ With\ anonymous\ FTP
|
|
server,\ you\ will\ need\ to\ use\ `anonymous`\ as\ username\ and\ your\ email\ address\ as
|
|
the\ password.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote r) Rename remote c)
|
|
Copy remote s) Set configuration password q) Quit config n/r/c/s/q> n
|
|
name> remote Type of storage to configure.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value [snip] XX / FTP Connection \ \[rq]ftp" [snip]
|
|
Storage> ftp ** See help for ftp backend at: https://rclone.org/ftp/ **
|
|
.PP
|
|
FTP host to connect to Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value 1 / Connect to ftp.example.com
|
|
\ \[rq]ftp.example.com" host> ftp.example.com FTP username, leave blank
|
|
for current username, ncw Enter a string value.
|
|
Press Enter for the default ("\[lq]). user> FTP port, leave blank to use
|
|
default (21) Enter a string value. Press Enter for the default
|
|
(\[rq]\[lq]). port> FTP password y) Yes type in my own password g)
|
|
Generate random password y/g> y Enter the password: password: Confirm
|
|
the password: password: Use FTP over TLS (Implicit) Enter a boolean
|
|
value (true or false). Press Enter for the default (\[rq]false").
|
|
tls> Remote config \[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] type =
|
|
ftp host = ftp.example.com pass = *** ENCRYPTED ***
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
This\ remote\ is\ called\ `remote`\ and\ can\ now\ be\ used\ like\ this
|
|
|
|
See\ all\ directories\ in\ the\ home\ directory
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
Make\ a\ new\ directory
|
|
|
|
\ \ \ \ rclone\ mkdir\ remote:path/to/directory
|
|
|
|
List\ the\ contents\ of\ a\ directory
|
|
|
|
\ \ \ \ rclone\ ls\ remote:path/to/directory
|
|
|
|
Sync\ `/home/local/directory`\ to\ the\ remote\ directory,\ deleting\ any
|
|
excess\ files\ in\ the\ directory.
|
|
|
|
\ \ \ \ rclone\ sync\ /home/local/directory\ remote:directory
|
|
|
|
###\ Modified\ time\ ###
|
|
|
|
FTP\ does\ not\ support\ modified\ times.\ \ Any\ times\ you\ see\ on\ the\ server
|
|
will\ be\ time\ of\ upload.
|
|
|
|
###\ Checksums\ ###
|
|
|
|
FTP\ does\ not\ support\ any\ checksums.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
In\ addition\ to\ the\ [default\ restricted\ characters\ set](/overview/#restricted\-characters)
|
|
the\ following\ characters\ are\ also\ replaced:
|
|
|
|
File\ names\ can\ also\ not\ end\ with\ the\ following\ characters.
|
|
These\ only\ get\ replaced\ if\ they\ are\ last\ character\ in\ the\ name:
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ SP\ \ \ \ \ \ \ \ |\ 0x20\ \ |\ ␠\ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Note\ that\ not\ all\ FTP\ servers\ can\ have\ all\ characters\ in\ file\ names,\ for\ example:
|
|
|
|
|\ FTP\ Server|\ Forbidden\ characters\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ proftpd\ \ \ |\ `*`\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ pureftpd\ \ |\ `\\\ [\ ]`\ \ \ \ \ \ \ \ \ \ \ \ \ \ |
|
|
|
|
###\ Implicit\ TLS\ ###
|
|
|
|
FTP\ supports\ implicit\ FTP\ over\ TLS\ servers\ (FTPS).\ This\ has\ to\ be\ enabled
|
|
in\ the\ config\ for\ the\ remote.\ The\ default\ FTPS\ port\ is\ `990`\ so\ the
|
|
port\ will\ likely\ have\ to\ be\ explictly\ set\ in\ the\ config\ for\ the\ remote.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/ftp/ftp.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ ftp\ (FTP\ Connection).
|
|
|
|
####\ \-\-ftp\-host
|
|
|
|
FTP\ host\ to\ connect\ to
|
|
|
|
\-\ Config:\ \ \ \ \ \ host
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_HOST
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "ftp.example.com"
|
|
\ \ \ \ \ \ \ \ \-\ Connect\ to\ ftp.example.com
|
|
|
|
####\ \-\-ftp\-user
|
|
|
|
FTP\ username,\ leave\ blank\ for\ current\ username,\ $USER
|
|
|
|
\-\ Config:\ \ \ \ \ \ user
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_USER
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-ftp\-port
|
|
|
|
FTP\ port,\ leave\ blank\ to\ use\ default\ (21)
|
|
|
|
\-\ Config:\ \ \ \ \ \ port
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_PORT
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-ftp\-pass
|
|
|
|
FTP\ password
|
|
|
|
\-\ Config:\ \ \ \ \ \ pass
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_PASS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-ftp\-tls
|
|
|
|
Use\ FTP\ over\ TLS\ (Implicit)
|
|
|
|
\-\ Config:\ \ \ \ \ \ tls
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_TLS
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ ftp\ (FTP\ Connection).
|
|
|
|
####\ \-\-ftp\-concurrency
|
|
|
|
Maximum\ number\ of\ FTP\ simultaneous\ connections,\ 0\ for\ unlimited
|
|
|
|
\-\ Config:\ \ \ \ \ \ concurrency
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_CONCURRENCY
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 0
|
|
|
|
####\ \-\-ftp\-no\-check\-certificate
|
|
|
|
Do\ not\ verify\ the\ TLS\ certificate\ of\ the\ server
|
|
|
|
\-\ Config:\ \ \ \ \ \ no_check_certificate
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_NO_CHECK_CERTIFICATE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-ftp\-disable\-epsv
|
|
|
|
Disable\ using\ EPSV\ even\ if\ server\ advertises\ support
|
|
|
|
\-\ Config:\ \ \ \ \ \ disable_epsv
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_FTP_DISABLE_EPSV
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
###\ Limitations\ ###
|
|
|
|
Note\ that\ since\ FTP\ isn\[aq]t\ HTTP\ based\ the\ following\ flags\ don\[aq]t\ work
|
|
with\ it:\ `\-\-dump\-headers`,\ `\-\-dump\-bodies`,\ `\-\-dump\-auth`
|
|
|
|
Note\ that\ `\-\-timeout`\ isn\[aq]t\ supported\ (but\ `\-\-contimeout`\ is).
|
|
|
|
Note\ that\ `\-\-bind`\ isn\[aq]t\ supported.
|
|
|
|
FTP\ could\ support\ server\ side\ move\ but\ doesn\[aq]t\ yet.
|
|
|
|
Note\ that\ the\ ftp\ backend\ does\ not\ support\ the\ `ftp_proxy`\ environment
|
|
variable\ yet.
|
|
|
|
Note\ that\ while\ implicit\ FTP\ over\ TLS\ is\ supported,
|
|
explicit\ FTP\ over\ TLS\ is\ not.
|
|
|
|
Google\ Cloud\ Storage
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
Paths\ are\ specified\ as\ `remote:bucket`\ (or\ `remote:`\ for\ the\ `lsd`
|
|
command.)\ \ You\ may\ put\ subdirectories\ in\ too,\ eg\ `remote:bucket/path/to/dir`.
|
|
|
|
The\ initial\ setup\ for\ google\ cloud\ storage\ involves\ getting\ a\ token\ from\ Google\ Cloud\ Storage
|
|
which\ you\ need\ to\ do\ in\ your\ browser.\ \ `rclone\ config`\ walks\ you
|
|
through\ it.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.IP "n)" 3
|
|
New remote
|
|
.IP "o)" 3
|
|
Delete remote
|
|
.IP "p)" 3
|
|
Quit config e/n/d/q> n name> remote Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Google
|
|
Cloud Storage (this is not Google Drive) \ \[lq]google cloud
|
|
storage\[rq] [snip] Storage> google cloud storage Google Application
|
|
Client Id \- leave blank normally.
|
|
client_id> Google Application Client Secret \- leave blank normally.
|
|
client_secret> Project number optional \- needed only for
|
|
list/create/delete buckets \- see your developer console.
|
|
project_number> 12345678 Service Account Credentials JSON file path \-
|
|
needed only if you want use SA instead of interactive login.
|
|
service_account_file> Access Control List for new objects.
|
|
Choose a number from below, or type in your own value 1 / Object owner
|
|
gets OWNER access, and all Authenticated Users get READER access.
|
|
\ \[lq]authenticatedRead\[rq] 2 / Object owner gets OWNER access, and
|
|
project team owners get OWNER access.
|
|
\ \[lq]bucketOwnerFullControl\[rq] 3 / Object owner gets OWNER access,
|
|
and project team owners get READER access.
|
|
\ \[lq]bucketOwnerRead\[rq] 4 / Object owner gets OWNER access [default
|
|
if left blank].
|
|
\ \[lq]private\[rq] 5 / Object owner gets OWNER access, and project team
|
|
members get access according to their roles.
|
|
\ \[lq]projectPrivate\[rq] 6 / Object owner gets OWNER access, and all
|
|
Users get READER access.
|
|
\ \[lq]publicRead\[rq] object_acl> 4 Access Control List for new
|
|
buckets.
|
|
Choose a number from below, or type in your own value 1 / Project team
|
|
owners get OWNER access, and all Authenticated Users get READER access.
|
|
\ \[lq]authenticatedRead\[rq] 2 / Project team owners get OWNER access
|
|
[default if left blank].
|
|
\ \[lq]private\[rq] 3 / Project team members get access according to
|
|
their roles.
|
|
\ \[lq]projectPrivate\[rq] 4 / Project team owners get OWNER access, and
|
|
all Users get READER access.
|
|
\ \[lq]publicRead\[rq] 5 / Project team owners get OWNER access, and all
|
|
Users get WRITER access.
|
|
\ \[lq]publicReadWrite\[rq] bucket_acl> 2 Location for the newly created
|
|
buckets.
|
|
Choose a number from below, or type in your own value 1 / Empty for
|
|
default location (US).
|
|
\ "" 2 / Multi\-regional location for Asia.
|
|
\ \[lq]asia\[rq] 3 / Multi\-regional location for Europe.
|
|
\ \[lq]eu\[rq] 4 / Multi\-regional location for United States.
|
|
\ \[lq]us\[rq] 5 / Taiwan.
|
|
\ \[lq]asia\-east1\[rq] 6 / Tokyo.
|
|
\ \[lq]asia\-northeast1\[rq] 7 / Singapore.
|
|
\ \[lq]asia\-southeast1\[rq] 8 / Sydney.
|
|
\ \[lq]australia\-southeast1\[rq] 9 / Belgium.
|
|
\ \[lq]europe\-west1\[rq] 10 / London.
|
|
\ \[lq]europe\-west2\[rq] 11 / Iowa.
|
|
\ \[lq]us\-central1\[rq] 12 / South Carolina.
|
|
\ \[lq]us\-east1\[rq] 13 / Northern Virginia.
|
|
\ \[lq]us\-east4\[rq] 14 / Oregon.
|
|
\ \[lq]us\-west1\[rq] location> 12 The storage class to use when storing
|
|
objects in Google Cloud Storage.
|
|
Choose a number from below, or type in your own value 1 / Default \ "" 2
|
|
/ Multi\-regional storage class \ \[lq]MULTI_REGIONAL\[rq] 3 / Regional
|
|
storage class \ \[lq]REGIONAL\[rq] 4 / Nearline storage class
|
|
\ \[lq]NEARLINE\[rq] 5 / Coldline storage class \ \[lq]COLDLINE\[rq] 6 /
|
|
Durable reduced availability storage class
|
|
\ \[lq]DURABLE_REDUCED_AVAILABILITY\[rq] storage_class> 5 Remote config
|
|
Use auto config?
|
|
.IP \[bu] 2
|
|
Say Y if not sure
|
|
.IP \[bu] 2
|
|
Say N if you are working on a remote or headless machine or Y didn't
|
|
work
|
|
.IP "y)" 3
|
|
Yes
|
|
.IP "z)" 3
|
|
No y/n> y If your browser doesn't open automatically go to the following
|
|
link: http://127.0.0.1:53682/auth Log in and authorize rclone for access
|
|
Waiting for code\&... Got code \[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
[remote] type = google cloud storage client_id = client_secret = token =
|
|
{\[lq]AccessToken\[rq]:\[lq]xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[rq],\[lq]RefreshToken\[rq]:\[lq]x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx\[rq],\[lq]Expiry\[rq]:\[lq]2014\-07\-17T20:49:14.929208288+01:00\[rq],\[lq]Extra\[rq]:null}
|
|
project_number = 12345678 object_acl = private bucket_acl = private
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en]
|
|
.IP "a)" 3
|
|
Yes this is OK
|
|
.IP "b)" 3
|
|
Edit this remote
|
|
.IP "c)" 3
|
|
Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Note\ that\ rclone\ runs\ a\ webserver\ on\ your\ local\ machine\ to\ collect\ the
|
|
token\ as\ returned\ from\ Google\ if\ you\ use\ auto\ config\ mode.\ This\ only
|
|
runs\ from\ the\ moment\ it\ opens\ your\ browser\ to\ the\ moment\ you\ get\ back
|
|
the\ verification\ code.\ \ This\ is\ on\ `http://127.0.0.1:53682/`\ and\ this
|
|
it\ may\ require\ you\ to\ unblock\ it\ temporarily\ if\ you\ are\ running\ a\ host
|
|
firewall,\ or\ use\ manual\ mode.
|
|
|
|
This\ remote\ is\ called\ `remote`\ and\ can\ now\ be\ used\ like\ this
|
|
|
|
See\ all\ the\ buckets\ in\ your\ project
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
Make\ a\ new\ bucket
|
|
|
|
\ \ \ \ rclone\ mkdir\ remote:bucket
|
|
|
|
List\ the\ contents\ of\ a\ bucket
|
|
|
|
\ \ \ \ rclone\ ls\ remote:bucket
|
|
|
|
Sync\ `/home/local/directory`\ to\ the\ remote\ bucket,\ deleting\ any\ excess
|
|
files\ in\ the\ bucket.
|
|
|
|
\ \ \ \ rclone\ sync\ /home/local/directory\ remote:bucket
|
|
|
|
###\ Service\ Account\ support\ ###
|
|
|
|
You\ can\ set\ up\ rclone\ with\ Google\ Cloud\ Storage\ in\ an\ unattended\ mode,
|
|
i.e.\ not\ tied\ to\ a\ specific\ end\-user\ Google\ account.\ This\ is\ useful
|
|
when\ you\ want\ to\ synchronise\ files\ onto\ machines\ that\ don\[aq]t\ have
|
|
actively\ logged\-in\ users,\ for\ example\ build\ machines.
|
|
|
|
To\ get\ credentials\ for\ Google\ Cloud\ Platform
|
|
[IAM\ Service\ Accounts](https://cloud.google.com/iam/docs/service\-accounts),
|
|
please\ head\ to\ the
|
|
[Service\ Account](https://console.cloud.google.com/permissions/serviceaccounts)
|
|
section\ of\ the\ Google\ Developer\ Console.\ Service\ Accounts\ behave\ just
|
|
like\ normal\ `User`\ permissions\ in
|
|
[Google\ Cloud\ Storage\ ACLs](https://cloud.google.com/storage/docs/access\-control),
|
|
so\ you\ can\ limit\ their\ access\ (e.g.\ make\ them\ read\ only).\ After
|
|
creating\ an\ account,\ a\ JSON\ file\ containing\ the\ Service\ Account\[aq]s
|
|
credentials\ will\ be\ downloaded\ onto\ your\ machines.\ These\ credentials
|
|
are\ what\ rclone\ will\ use\ for\ authentication.
|
|
|
|
To\ use\ a\ Service\ Account\ instead\ of\ OAuth2\ token\ flow,\ enter\ the\ path
|
|
to\ your\ Service\ Account\ credentials\ at\ the\ `service_account_file`
|
|
prompt\ and\ rclone\ won\[aq]t\ use\ the\ browser\ based\ authentication
|
|
flow.\ If\ you\[aq]d\ rather\ stuff\ the\ contents\ of\ the\ credentials\ file\ into
|
|
the\ rclone\ config\ file,\ you\ can\ set\ `service_account_credentials`\ with
|
|
the\ actual\ contents\ of\ the\ file\ instead,\ or\ set\ the\ equivalent
|
|
environment\ variable.
|
|
|
|
###\ Application\ Default\ Credentials\ ###
|
|
|
|
If\ no\ other\ source\ of\ credentials\ is\ provided,\ rclone\ will\ fall\ back
|
|
to
|
|
[Application\ Default\ Credentials](https://cloud.google.com/video\-intelligence/docs/common/auth#authenticating_with_application_default_credentials)
|
|
this\ is\ useful\ both\ when\ you\ already\ have\ configured\ authentication
|
|
for\ your\ developer\ account,\ or\ in\ production\ when\ running\ on\ a\ google
|
|
compute\ host.\ Note\ that\ if\ running\ in\ docker,\ you\ may\ need\ to\ run
|
|
additional\ commands\ on\ your\ google\ compute\ machine\ \-
|
|
[see\ this\ page](https://cloud.google.com/container\-registry/docs/advanced\-authentication#gcloud_as_a_docker_credential_helper).
|
|
|
|
Note\ that\ in\ the\ case\ application\ default\ credentials\ are\ used,\ there
|
|
is\ no\ need\ to\ explicitly\ configure\ a\ project\ number.
|
|
|
|
###\ \-\-fast\-list\ ###
|
|
|
|
This\ remote\ supports\ `\-\-fast\-list`\ which\ allows\ you\ to\ use\ fewer
|
|
transactions\ in\ exchange\ for\ more\ memory.\ See\ the\ [rclone
|
|
docs](/docs/#fast\-list)\ for\ more\ details.
|
|
|
|
###\ Modified\ time\ ###
|
|
|
|
Google\ google\ cloud\ storage\ stores\ md5sums\ natively\ and\ rclone\ stores
|
|
modification\ times\ as\ metadata\ on\ the\ object,\ under\ the\ "mtime"\ key\ in
|
|
RFC3339\ format\ accurate\ to\ 1ns.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
|\ Character\ |\ Value\ |\ Replacement\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |:\-\-\-\-\-:|:\-\-\-\-\-\-\-\-\-\-\-:|
|
|
|\ NUL\ \ \ \ \ \ \ |\ 0x00\ \ |\ ␀\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ LF\ \ \ \ \ \ \ \ |\ 0x0A\ \ |\ ␊\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ CR\ \ \ \ \ \ \ \ |\ 0x0D\ \ |\ ␍\ \ \ \ \ \ \ \ \ \ \ |
|
|
|\ /\ \ \ \ \ \ \ \ \ |\ 0x2F\ \ |\ ／\ \ \ \ \ \ \ \ \ \ |
|
|
|
|
Invalid\ UTF\-8\ bytes\ will\ also\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/googlecloudstorage/googlecloudstorage.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ google\ cloud\ storage\ (Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)).
|
|
|
|
####\ \-\-gcs\-client\-id
|
|
|
|
Google\ Application\ Client\ Id
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_CLIENT_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-gcs\-client\-secret
|
|
|
|
Google\ Application\ Client\ Secret
|
|
Leave\ blank\ normally.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_secret
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_CLIENT_SECRET
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-gcs\-project\-number
|
|
|
|
Project\ number.
|
|
Optional\ \-\ needed\ only\ for\ list/create/delete\ buckets\ \-\ see\ your\ developer\ console.
|
|
|
|
\-\ Config:\ \ \ \ \ \ project_number
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_PROJECT_NUMBER
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-gcs\-service\-account\-file
|
|
|
|
Service\ Account\ Credentials\ JSON\ file\ path
|
|
Leave\ blank\ normally.
|
|
Needed\ only\ if\ you\ want\ use\ SA\ instead\ of\ interactive\ login.
|
|
|
|
\-\ Config:\ \ \ \ \ \ service_account_file
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_SERVICE_ACCOUNT_FILE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-gcs\-service\-account\-credentials
|
|
|
|
Service\ Account\ Credentials\ JSON\ blob
|
|
Leave\ blank\ normally.
|
|
Needed\ only\ if\ you\ want\ use\ SA\ instead\ of\ interactive\ login.
|
|
|
|
\-\ Config:\ \ \ \ \ \ service_account_credentials
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-gcs\-object\-acl
|
|
|
|
Access\ Control\ List\ for\ new\ objects.
|
|
|
|
\-\ Config:\ \ \ \ \ \ object_acl
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_OBJECT_ACL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "authenticatedRead"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ OWNER\ access,\ and\ all\ Authenticated\ Users\ get\ READER\ access.
|
|
\ \ \ \ \-\ "bucketOwnerFullControl"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ OWNER\ access,\ and\ project\ team\ owners\ get\ OWNER\ access.
|
|
\ \ \ \ \-\ "bucketOwnerRead"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ OWNER\ access,\ and\ project\ team\ owners\ get\ READER\ access.
|
|
\ \ \ \ \-\ "private"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ OWNER\ access\ [default\ if\ left\ blank].
|
|
\ \ \ \ \-\ "projectPrivate"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ OWNER\ access,\ and\ project\ team\ members\ get\ access\ according\ to\ their\ roles.
|
|
\ \ \ \ \-\ "publicRead"
|
|
\ \ \ \ \ \ \ \ \-\ Object\ owner\ gets\ OWNER\ access,\ and\ all\ Users\ get\ READER\ access.
|
|
|
|
####\ \-\-gcs\-bucket\-acl
|
|
|
|
Access\ Control\ List\ for\ new\ buckets.
|
|
|
|
\-\ Config:\ \ \ \ \ \ bucket_acl
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_BUCKET_ACL
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "authenticatedRead"
|
|
\ \ \ \ \ \ \ \ \-\ Project\ team\ owners\ get\ OWNER\ access,\ and\ all\ Authenticated\ Users\ get\ READER\ access.
|
|
\ \ \ \ \-\ "private"
|
|
\ \ \ \ \ \ \ \ \-\ Project\ team\ owners\ get\ OWNER\ access\ [default\ if\ left\ blank].
|
|
\ \ \ \ \-\ "projectPrivate"
|
|
\ \ \ \ \ \ \ \ \-\ Project\ team\ members\ get\ access\ according\ to\ their\ roles.
|
|
\ \ \ \ \-\ "publicRead"
|
|
\ \ \ \ \ \ \ \ \-\ Project\ team\ owners\ get\ OWNER\ access,\ and\ all\ Users\ get\ READER\ access.
|
|
\ \ \ \ \-\ "publicReadWrite"
|
|
\ \ \ \ \ \ \ \ \-\ Project\ team\ owners\ get\ OWNER\ access,\ and\ all\ Users\ get\ WRITER\ access.
|
|
|
|
####\ \-\-gcs\-bucket\-policy\-only
|
|
|
|
Access\ checks\ should\ use\ bucket\-level\ IAM\ policies.
|
|
|
|
If\ you\ want\ to\ upload\ objects\ to\ a\ bucket\ with\ Bucket\ Policy\ Only\ set
|
|
then\ you\ will\ need\ to\ set\ this.
|
|
|
|
When\ it\ is\ set,\ rclone:
|
|
|
|
\-\ ignores\ ACLs\ set\ on\ buckets
|
|
\-\ ignores\ ACLs\ set\ on\ objects
|
|
\-\ creates\ buckets\ with\ Bucket\ Policy\ Only\ set
|
|
|
|
Docs:\ https://cloud.google.com/storage/docs/bucket\-policy\-only
|
|
|
|
|
|
\-\ Config:\ \ \ \ \ \ bucket_policy_only
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_BUCKET_POLICY_ONLY
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-gcs\-location
|
|
|
|
Location\ for\ the\ newly\ created\ buckets.
|
|
|
|
\-\ Config:\ \ \ \ \ \ location
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_LOCATION
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Empty\ for\ default\ location\ (US).
|
|
\ \ \ \ \-\ "asia"
|
|
\ \ \ \ \ \ \ \ \-\ Multi\-regional\ location\ for\ Asia.
|
|
\ \ \ \ \-\ "eu"
|
|
\ \ \ \ \ \ \ \ \-\ Multi\-regional\ location\ for\ Europe.
|
|
\ \ \ \ \-\ "us"
|
|
\ \ \ \ \ \ \ \ \-\ Multi\-regional\ location\ for\ United\ States.
|
|
\ \ \ \ \-\ "asia\-east1"
|
|
\ \ \ \ \ \ \ \ \-\ Taiwan.
|
|
\ \ \ \ \-\ "asia\-east2"
|
|
\ \ \ \ \ \ \ \ \-\ Hong\ Kong.
|
|
\ \ \ \ \-\ "asia\-northeast1"
|
|
\ \ \ \ \ \ \ \ \-\ Tokyo.
|
|
\ \ \ \ \-\ "asia\-south1"
|
|
\ \ \ \ \ \ \ \ \-\ Mumbai.
|
|
\ \ \ \ \-\ "asia\-southeast1"
|
|
\ \ \ \ \ \ \ \ \-\ Singapore.
|
|
\ \ \ \ \-\ "australia\-southeast1"
|
|
\ \ \ \ \ \ \ \ \-\ Sydney.
|
|
\ \ \ \ \-\ "europe\-north1"
|
|
\ \ \ \ \ \ \ \ \-\ Finland.
|
|
\ \ \ \ \-\ "europe\-west1"
|
|
\ \ \ \ \ \ \ \ \-\ Belgium.
|
|
\ \ \ \ \-\ "europe\-west2"
|
|
\ \ \ \ \ \ \ \ \-\ London.
|
|
\ \ \ \ \-\ "europe\-west3"
|
|
\ \ \ \ \ \ \ \ \-\ Frankfurt.
|
|
\ \ \ \ \-\ "europe\-west4"
|
|
\ \ \ \ \ \ \ \ \-\ Netherlands.
|
|
\ \ \ \ \-\ "us\-central1"
|
|
\ \ \ \ \ \ \ \ \-\ Iowa.
|
|
\ \ \ \ \-\ "us\-east1"
|
|
\ \ \ \ \ \ \ \ \-\ South\ Carolina.
|
|
\ \ \ \ \-\ "us\-east4"
|
|
\ \ \ \ \ \ \ \ \-\ Northern\ Virginia.
|
|
\ \ \ \ \-\ "us\-west1"
|
|
\ \ \ \ \ \ \ \ \-\ Oregon.
|
|
\ \ \ \ \-\ "us\-west2"
|
|
\ \ \ \ \ \ \ \ \-\ California.
|
|
|
|
####\ \-\-gcs\-storage\-class
|
|
|
|
The\ storage\ class\ to\ use\ when\ storing\ objects\ in\ Google\ Cloud\ Storage.
|
|
|
|
\-\ Config:\ \ \ \ \ \ storage_class
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_GCS_STORAGE_CLASS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ ""
|
|
\ \ \ \ \ \ \ \ \-\ Default
|
|
\ \ \ \ \-\ "MULTI_REGIONAL"
|
|
\ \ \ \ \ \ \ \ \-\ Multi\-regional\ storage\ class
|
|
\ \ \ \ \-\ "REGIONAL"
|
|
\ \ \ \ \ \ \ \ \-\ Regional\ storage\ class
|
|
\ \ \ \ \-\ "NEARLINE"
|
|
\ \ \ \ \ \ \ \ \-\ Nearline\ storage\ class
|
|
\ \ \ \ \-\ "COLDLINE"
|
|
\ \ \ \ \ \ \ \ \-\ Coldline\ storage\ class
|
|
\ \ \ \ \-\ "DURABLE_REDUCED_AVAILABILITY"
|
|
\ \ \ \ \ \ \ \ \-\ Durable\ reduced\ availability\ storage\ class
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
Google\ Drive
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
Paths\ are\ specified\ as\ `drive:path`
|
|
|
|
Drive\ paths\ may\ be\ as\ deep\ as\ required,\ eg\ `drive:directory/subdirectory`.
|
|
|
|
The\ initial\ setup\ for\ drive\ involves\ getting\ a\ token\ from\ Google\ drive
|
|
which\ you\ need\ to\ do\ in\ your\ browser.\ \ `rclone\ config`\ walks\ you
|
|
through\ it.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote r) Rename remote c)
|
|
Copy remote s) Set configuration password q) Quit config n/r/c/s/q> n
|
|
name> remote Type of storage to configure.
|
|
Choose a number from below, or type in your own value [snip] XX / Google
|
|
Drive \ \[lq]drive\[rq] [snip] Storage> drive Google Application Client
|
|
Id \- leave blank normally.
|
|
client_id> Google Application Client Secret \- leave blank normally.
|
|
client_secret> Scope that rclone should use when requesting access from
|
|
drive.
|
|
Choose a number from below, or type in your own value 1 / Full access
|
|
all files, excluding Application Data Folder.
|
|
\ \[lq]drive\[rq] 2 / Read\-only access to file metadata and file
|
|
contents.
|
|
\ \[lq]drive.readonly\[rq] / Access to files created by rclone only.
|
|
3 | These are visible in the drive website.
|
|
| File authorization is revoked when the user deauthorizes the app.
|
|
\ \[lq]drive.file\[rq] / Allows read and write access to the Application
|
|
Data folder.
|
|
4 | This is not visible in the drive website.
|
|
\ \[lq]drive.appfolder\[rq] / Allows read\-only access to file metadata
|
|
but 5 | does not allow any access to read or download file content.
|
|
\ \[lq]drive.metadata.readonly\[rq] scope> 1 ID of the root folder \-
|
|
leave blank normally.
|
|
Fill in to access \[lq]Computers\[rq] folders.
|
|
(see docs).
|
|
root_folder_id> Service Account Credentials JSON file path \- needed
|
|
only if you want use SA instead of interactive login.
|
|
service_account_file> Remote config Use auto config?
|
|
* Say Y if not sure * Say N if you are working on a remote or headless
|
|
machine or Y didn't work y) Yes n) No y/n> y If your browser doesn't
|
|
open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access Waiting for code\&... Got code
|
|
Configure this as a team drive?
|
|
y) Yes n) No y/n> n \[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote]
|
|
client_id = client_secret = scope = drive root_folder_id =
|
|
service_account_file = token =
|
|
{\[lq]access_token\[rq]:\[lq]XXX\[rq],\[lq]token_type\[rq]:\[lq]Bearer\[rq],\[lq]refresh_token\[rq]:\[lq]XXX\[rq],\[lq]expiry\[rq]:\[lq]2014\-03\-16T13:57:58.955387075Z\[rq]}
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] y) Yes this is OK e) Edit this
|
|
remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
Note\ that\ rclone\ runs\ a\ webserver\ on\ your\ local\ machine\ to\ collect\ the
|
|
token\ as\ returned\ from\ Google\ if\ you\ use\ auto\ config\ mode.\ This\ only
|
|
runs\ from\ the\ moment\ it\ opens\ your\ browser\ to\ the\ moment\ you\ get\ back
|
|
the\ verification\ code.\ \ This\ is\ on\ `http://127.0.0.1:53682/`\ and\ this
|
|
it\ may\ require\ you\ to\ unblock\ it\ temporarily\ if\ you\ are\ running\ a\ host
|
|
firewall,\ or\ use\ manual\ mode.
|
|
|
|
You\ can\ then\ use\ it\ like\ this,
|
|
|
|
List\ directories\ in\ top\ level\ of\ your\ drive
|
|
|
|
\ \ \ \ rclone\ lsd\ remote:
|
|
|
|
List\ all\ the\ files\ in\ your\ drive
|
|
|
|
\ \ \ \ rclone\ ls\ remote:
|
|
|
|
To\ copy\ a\ local\ directory\ to\ a\ drive\ directory\ called\ backup
|
|
|
|
\ \ \ \ rclone\ copy\ /home/source\ remote:backup
|
|
|
|
###\ Scopes\ ###
|
|
|
|
Rclone\ allows\ you\ to\ select\ which\ scope\ you\ would\ like\ for\ rclone\ to
|
|
use.\ \ This\ changes\ what\ type\ of\ token\ is\ granted\ to\ rclone.\ \ [The
|
|
scopes\ are\ defined
|
|
here.](https://developers.google.com/drive/v3/web/about\-auth).
|
|
|
|
The\ scope\ are
|
|
|
|
####\ drive\ ####
|
|
|
|
This\ is\ the\ default\ scope\ and\ allows\ full\ access\ to\ all\ files,\ except
|
|
for\ the\ Application\ Data\ Folder\ (see\ below).
|
|
|
|
Choose\ this\ one\ if\ you\ aren\[aq]t\ sure.
|
|
|
|
####\ drive.readonly\ ####
|
|
|
|
This\ allows\ read\ only\ access\ to\ all\ files.\ \ Files\ may\ be\ listed\ and
|
|
downloaded\ but\ not\ uploaded,\ renamed\ or\ deleted.
|
|
|
|
####\ drive.file\ ####
|
|
|
|
With\ this\ scope\ rclone\ can\ read/view/modify\ only\ those\ files\ and
|
|
folders\ it\ creates.
|
|
|
|
So\ if\ you\ uploaded\ files\ to\ drive\ via\ the\ web\ interface\ (or\ any\ other
|
|
means)\ they\ will\ not\ be\ visible\ to\ rclone.
|
|
|
|
This\ can\ be\ useful\ if\ you\ are\ using\ rclone\ to\ backup\ data\ and\ you\ want
|
|
to\ be\ sure\ confidential\ data\ on\ your\ drive\ is\ not\ visible\ to\ rclone.
|
|
|
|
Files\ created\ with\ this\ scope\ are\ visible\ in\ the\ web\ interface.
|
|
|
|
####\ drive.appfolder\ ####
|
|
|
|
This\ gives\ rclone\ its\ own\ private\ area\ to\ store\ files.\ \ Rclone\ will
|
|
not\ be\ able\ to\ see\ any\ other\ files\ on\ your\ drive\ and\ you\ won\[aq]t\ be\ able
|
|
to\ see\ rclone\[aq]s\ files\ from\ the\ web\ interface\ either.
|
|
|
|
####\ drive.metadata.readonly\ ####
|
|
|
|
This\ allows\ read\ only\ access\ to\ file\ names\ only.\ \ It\ does\ not\ allow
|
|
rclone\ to\ download\ or\ upload\ data,\ or\ rename\ or\ delete\ files\ or
|
|
directories.
|
|
|
|
###\ Root\ folder\ ID\ ###
|
|
|
|
You\ can\ set\ the\ `root_folder_id`\ for\ rclone.\ \ This\ is\ the\ directory
|
|
(identified\ by\ its\ `Folder\ ID`)\ that\ rclone\ considers\ to\ be\ the\ root
|
|
of\ your\ drive.
|
|
|
|
Normally\ you\ will\ leave\ this\ blank\ and\ rclone\ will\ determine\ the
|
|
correct\ root\ to\ use\ itself.
|
|
|
|
However\ you\ can\ set\ this\ to\ restrict\ rclone\ to\ a\ specific\ folder
|
|
hierarchy\ or\ to\ access\ data\ within\ the\ "Computers"\ tab\ on\ the\ drive
|
|
web\ interface\ (where\ files\ from\ Google\[aq]s\ Backup\ and\ Sync\ desktop
|
|
program\ go).
|
|
|
|
In\ order\ to\ do\ this\ you\ will\ have\ to\ find\ the\ `Folder\ ID`\ of\ the
|
|
directory\ you\ wish\ rclone\ to\ display.\ \ This\ will\ be\ the\ last\ segment
|
|
of\ the\ URL\ when\ you\ open\ the\ relevant\ folder\ in\ the\ drive\ web
|
|
interface.
|
|
|
|
So\ if\ the\ folder\ you\ want\ rclone\ to\ use\ has\ a\ URL\ which\ looks\ like
|
|
`https://drive.google.com/drive/folders/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh`
|
|
in\ the\ browser,\ then\ you\ use\ `1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh`\ as
|
|
the\ `root_folder_id`\ in\ the\ config.
|
|
|
|
**NB**\ folders\ under\ the\ "Computers"\ tab\ seem\ to\ be\ read\ only\ (drive
|
|
gives\ a\ 500\ error)\ when\ using\ rclone.
|
|
|
|
There\ doesn\[aq]t\ appear\ to\ be\ an\ API\ to\ discover\ the\ folder\ IDs\ of\ the
|
|
"Computers"\ tab\ \-\ please\ contact\ us\ if\ you\ know\ otherwise!
|
|
|
|
Note\ also\ that\ rclone\ can\[aq]t\ access\ any\ data\ under\ the\ "Backups"\ tab\ on
|
|
the\ google\ drive\ web\ interface\ yet.
|
|
|
|
###\ Service\ Account\ support\ ###
|
|
|
|
You\ can\ set\ up\ rclone\ with\ Google\ Drive\ in\ an\ unattended\ mode,
|
|
i.e.\ not\ tied\ to\ a\ specific\ end\-user\ Google\ account.\ This\ is\ useful
|
|
when\ you\ want\ to\ synchronise\ files\ onto\ machines\ that\ don\[aq]t\ have
|
|
actively\ logged\-in\ users,\ for\ example\ build\ machines.
|
|
|
|
To\ use\ a\ Service\ Account\ instead\ of\ OAuth2\ token\ flow,\ enter\ the\ path
|
|
to\ your\ Service\ Account\ credentials\ at\ the\ `service_account_file`
|
|
prompt\ during\ `rclone\ config`\ and\ rclone\ won\[aq]t\ use\ the\ browser\ based
|
|
authentication\ flow.\ If\ you\[aq]d\ rather\ stuff\ the\ contents\ of\ the
|
|
credentials\ file\ into\ the\ rclone\ config\ file,\ you\ can\ set
|
|
`service_account_credentials`\ with\ the\ actual\ contents\ of\ the\ file
|
|
instead,\ or\ set\ the\ equivalent\ environment\ variable.
|
|
|
|
####\ Use\ case\ \-\ Google\ Apps/G\-suite\ account\ and\ individual\ Drive\ ####
|
|
|
|
Let\[aq]s\ say\ that\ you\ are\ the\ administrator\ of\ a\ Google\ Apps\ (old)\ or
|
|
G\-suite\ account.
|
|
The\ goal\ is\ to\ store\ data\ on\ an\ individual\[aq]s\ Drive\ account,\ who\ IS
|
|
a\ member\ of\ the\ domain.
|
|
We\[aq]ll\ call\ the\ domain\ **example.com**,\ and\ the\ user
|
|
**foo\@example.com**.
|
|
|
|
There\[aq]s\ a\ few\ steps\ we\ need\ to\ go\ through\ to\ accomplish\ this:
|
|
|
|
#####\ 1.\ Create\ a\ service\ account\ for\ example.com\ #####
|
|
\ \ \-\ To\ create\ a\ service\ account\ and\ obtain\ its\ credentials,\ go\ to\ the
|
|
[Google\ Developer\ Console](https://console.developers.google.com).
|
|
\ \ \-\ You\ must\ have\ a\ project\ \-\ create\ one\ if\ you\ don\[aq]t.
|
|
\ \ \-\ Then\ go\ to\ "IAM\ &\ admin"\ \->\ "Service\ Accounts".
|
|
\ \ \-\ Use\ the\ "Create\ Credentials"\ button.\ Fill\ in\ "Service\ account\ name"
|
|
with\ something\ that\ identifies\ your\ client.\ "Role"\ can\ be\ empty.
|
|
\ \ \-\ Tick\ "Furnish\ a\ new\ private\ key"\ \-\ select\ "Key\ type\ JSON".
|
|
\ \ \-\ Tick\ "Enable\ G\ Suite\ Domain\-wide\ Delegation".\ This\ option\ makes
|
|
"impersonation"\ possible,\ as\ documented\ here:
|
|
[Delegating\ domain\-wide\ authority\ to\ the\ service\ account](https://developers.google.com/identity/protocols/OAuth2ServiceAccount#delegatingauthority)
|
|
\ \ \-\ These\ credentials\ are\ what\ rclone\ will\ use\ for\ authentication.
|
|
If\ you\ ever\ need\ to\ remove\ access,\ press\ the\ "Delete\ service
|
|
account\ key"\ button.
|
|
|
|
#####\ 2.\ Allowing\ API\ access\ to\ example.com\ Google\ Drive\ #####
|
|
\ \ \-\ Go\ to\ example.com\[aq]s\ admin\ console
|
|
\ \ \-\ Go\ into\ "Security"\ (or\ use\ the\ search\ bar)
|
|
\ \ \-\ Select\ "Show\ more"\ and\ then\ "Advanced\ settings"
|
|
\ \ \-\ Select\ "Manage\ API\ client\ access"\ in\ the\ "Authentication"\ section
|
|
\ \ \-\ In\ the\ "Client\ Name"\ field\ enter\ the\ service\ account\[aq]s
|
|
"Client\ ID"\ \-\ this\ can\ be\ found\ in\ the\ Developer\ Console\ under
|
|
"IAM\ &\ Admin"\ \->\ "Service\ Accounts",\ then\ "View\ Client\ ID"\ for
|
|
the\ newly\ created\ service\ account.
|
|
It\ is\ a\ ~21\ character\ numerical\ string.
|
|
\ \ \-\ In\ the\ next\ field,\ "One\ or\ More\ API\ Scopes",\ enter
|
|
`https://www.googleapis.com/auth/drive`
|
|
to\ grant\ access\ to\ Google\ Drive\ specifically.
|
|
|
|
#####\ 3.\ Configure\ rclone,\ assuming\ a\ new\ install\ #####
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
rclone config
|
|
.PP
|
|
n/s/q> n # New name>gdrive # Gdrive is an example name Storage> # Select
|
|
the number shown for Google Drive client_id> # Can be left blank
|
|
client_secret> # Can be left blank scope> # Select your scope, 1 for
|
|
example root_folder_id> # Can be left blank service_account_file>
|
|
/home/foo/myJSONfile.json # This is where the JSON file goes! y/n> #
|
|
Auto config, y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
#####\ 4.\ Verify\ that\ it\[aq]s\ working\ #####
|
|
\ \ \-\ `rclone\ \-v\ \-\-drive\-impersonate\ foo\@example.com\ lsf\ gdrive:backup`
|
|
\ \ \-\ The\ arguments\ do:
|
|
\ \ \ \ \-\ `\-v`\ \-\ verbose\ logging
|
|
\ \ \ \ \-\ `\-\-drive\-impersonate\ foo\@example.com`\ \-\ this\ is\ what\ does
|
|
the\ magic,\ pretending\ to\ be\ user\ foo.
|
|
\ \ \ \ \-\ `lsf`\ \-\ list\ files\ in\ a\ parsing\ friendly\ way
|
|
\ \ \ \ \-\ `gdrive:backup`\ \-\ use\ the\ remote\ called\ gdrive,\ work\ in
|
|
the\ folder\ named\ backup.
|
|
|
|
###\ Team\ drives\ ###
|
|
|
|
If\ you\ want\ to\ configure\ the\ remote\ to\ point\ to\ a\ Google\ Team\ Drive
|
|
then\ answer\ `y`\ to\ the\ question\ `Configure\ this\ as\ a\ team\ drive?`.
|
|
|
|
This\ will\ fetch\ the\ list\ of\ Team\ Drives\ from\ google\ and\ allow\ you\ to
|
|
configure\ which\ one\ you\ want\ to\ use.\ \ You\ can\ also\ type\ in\ a\ team
|
|
drive\ ID\ if\ you\ prefer.
|
|
|
|
For\ example:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Configure this as a team drive?
|
|
y) Yes n) No y/n> y Fetching team drive list\&... Choose a number from
|
|
below, or type in your own value 1 / Rclone Test
|
|
\ \[lq]xxxxxxxxxxxxxxxxxxxx\[rq] 2 / Rclone Test 2
|
|
\ \[lq]yyyyyyyyyyyyyyyyyyyy\[rq] 3 / Rclone Test 3
|
|
\ \[lq]zzzzzzzzzzzzzzzzzzzz\[rq] Enter a Team Drive ID> 1
|
|
\[em]\[em]\[em]\[em]\[em]\[em]\[en] [remote] client_id = client_secret =
|
|
token =
|
|
{\[lq]AccessToken\[rq]:\[lq]xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[rq],\[lq]RefreshToken\[rq]:\[lq]1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx\[rq],\[lq]Expiry\[rq]:\[lq]2014\-03\-16T13:57:58.955387075Z\[rq],\[lq]Extra\[rq]:null}
|
|
team_drive = xxxxxxxxxxxxxxxxxxxx \[em]\[em]\[em]\[em]\[em]\[em]\[en] y)
|
|
Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
###\ \-\-fast\-list\ ###
|
|
|
|
This\ remote\ supports\ `\-\-fast\-list`\ which\ allows\ you\ to\ use\ fewer
|
|
transactions\ in\ exchange\ for\ more\ memory.\ See\ the\ [rclone
|
|
docs](/docs/#fast\-list)\ for\ more\ details.
|
|
|
|
It\ does\ this\ by\ combining\ multiple\ `list`\ calls\ into\ a\ single\ API\ request.
|
|
|
|
This\ works\ by\ combining\ many\ `\[aq]%s\[aq]\ in\ parents`\ filters\ into\ one\ expression.
|
|
To\ list\ the\ contents\ of\ directories\ a,\ b\ and\ c,\ the\ following\ requests\ will\ be\ send\ by\ the\ regular\ `List`\ function:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
trashed=false and `a' in parents trashed=false and `b' in parents
|
|
trashed=false and `c' in parents
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
These\ can\ now\ be\ combined\ into\ a\ single\ request:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
trashed=false and (`a' in parents or `b' in parents or `c' in parents)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
The\ implementation\ of\ `ListR`\ will\ put\ up\ to\ 50\ `parents`\ filters\ into\ one\ request.
|
|
It\ will\ \ use\ the\ `\-\-checkers`\ value\ to\ specify\ the\ number\ of\ requests\ to\ run\ in\ parallel.
|
|
|
|
In\ tests,\ these\ batch\ requests\ were\ up\ to\ 20x\ faster\ than\ the\ regular\ method.
|
|
Running\ the\ following\ command\ against\ different\ sized\ folders\ gives:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
rclone lsjson \-vv \-R \[en]checkers=6 gdrive:folder
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
|
|
small\ folder\ (220\ directories,\ 700\ files):
|
|
|
|
\-\ without\ `\-\-fast\-list`:\ 38s
|
|
\-\ with\ `\-\-fast\-list`:\ 10s
|
|
|
|
large\ folder\ (10600\ directories,\ 39000\ files):
|
|
|
|
\-\ without\ `\-\-fast\-list`:\ 22:05\ min
|
|
\-\ with\ `\-\-fast\-list`:\ 58s
|
|
|
|
###\ Modified\ time\ ###
|
|
|
|
Google\ drive\ stores\ modification\ times\ accurate\ to\ 1\ ms.
|
|
|
|
####\ Restricted\ filename\ characters
|
|
|
|
Only\ Invalid\ UTF\-8\ bytes\ will\ be\ [replaced](/overview/#invalid\-utf8),
|
|
as\ they\ can\[aq]t\ be\ used\ in\ JSON\ strings.
|
|
|
|
In\ contrast\ to\ other\ backends,\ `/`\ can\ also\ be\ used\ in\ names\ and\ `.`
|
|
or\ `..`\ are\ valid\ names.
|
|
|
|
###\ Revisions\ ###
|
|
|
|
Google\ drive\ stores\ revisions\ of\ files.\ \ When\ you\ upload\ a\ change\ to
|
|
an\ existing\ file\ to\ google\ drive\ using\ rclone\ it\ will\ create\ a\ new
|
|
revision\ of\ that\ file.
|
|
|
|
Revisions\ follow\ the\ standard\ google\ policy\ which\ at\ time\ of\ writing
|
|
was
|
|
|
|
\ \ *\ They\ are\ deleted\ after\ 30\ days\ or\ 100\ revisions\ (whatever\ comes\ first).
|
|
\ \ *\ They\ do\ not\ count\ towards\ a\ user\ storage\ quota.
|
|
|
|
###\ Deleting\ files\ ###
|
|
|
|
By\ default\ rclone\ will\ send\ all\ files\ to\ the\ trash\ when\ deleting
|
|
files.\ \ If\ deleting\ them\ permanently\ is\ required\ then\ use\ the
|
|
`\-\-drive\-use\-trash=false`\ flag,\ or\ set\ the\ equivalent\ environment
|
|
variable.
|
|
|
|
###\ Emptying\ trash\ ###
|
|
|
|
If\ you\ wish\ to\ empty\ your\ trash\ you\ can\ use\ the\ `rclone\ cleanup\ remote:`
|
|
command\ which\ will\ permanently\ delete\ all\ your\ trashed\ files.\ This\ command
|
|
does\ not\ take\ any\ path\ arguments.
|
|
|
|
Note\ that\ Google\ Drive\ takes\ some\ time\ (minutes\ to\ days)\ to\ empty\ the
|
|
trash\ even\ though\ the\ command\ returns\ within\ a\ few\ seconds.\ \ No\ output
|
|
is\ echoed,\ so\ there\ will\ be\ no\ confirmation\ even\ using\ \-v\ or\ \-vv.
|
|
|
|
###\ Quota\ information\ ###
|
|
|
|
To\ view\ your\ current\ quota\ you\ can\ use\ the\ `rclone\ about\ remote:`
|
|
command\ which\ will\ display\ your\ usage\ limit\ (quota),\ the\ usage\ in\ Google
|
|
Drive,\ the\ size\ of\ all\ files\ in\ the\ Trash\ and\ the\ space\ used\ by\ other
|
|
Google\ services\ such\ as\ Gmail.\ This\ command\ does\ not\ take\ any\ path
|
|
arguments.
|
|
|
|
####\ Import/Export\ of\ google\ documents\ ####
|
|
|
|
Google\ documents\ can\ be\ exported\ from\ and\ uploaded\ to\ Google\ Drive.
|
|
|
|
When\ rclone\ downloads\ a\ Google\ doc\ it\ chooses\ a\ format\ to\ download
|
|
depending\ upon\ the\ `\-\-drive\-export\-formats`\ setting.
|
|
By\ default\ the\ export\ formats\ are\ `docx,xlsx,pptx,svg`\ which\ are\ a
|
|
sensible\ default\ for\ an\ editable\ document.
|
|
|
|
When\ choosing\ a\ format,\ rclone\ runs\ down\ the\ list\ provided\ in\ order
|
|
and\ chooses\ the\ first\ file\ format\ the\ doc\ can\ be\ exported\ as\ from\ the
|
|
list.\ If\ the\ file\ can\[aq]t\ be\ exported\ to\ a\ format\ on\ the\ formats\ list,
|
|
then\ rclone\ will\ choose\ a\ format\ from\ the\ default\ list.
|
|
|
|
If\ you\ prefer\ an\ archive\ copy\ then\ you\ might\ use\ `\-\-drive\-export\-formats
|
|
pdf`,\ or\ if\ you\ prefer\ openoffice/libreoffice\ formats\ you\ might\ use
|
|
`\-\-drive\-export\-formats\ ods,odt,odp`.
|
|
|
|
Note\ that\ rclone\ adds\ the\ extension\ to\ the\ google\ doc,\ so\ if\ it\ is
|
|
called\ `My\ Spreadsheet`\ on\ google\ docs,\ it\ will\ be\ exported\ as\ `My
|
|
Spreadsheet.xlsx`\ or\ `My\ Spreadsheet.pdf`\ etc.
|
|
|
|
When\ importing\ files\ into\ Google\ Drive,\ rclone\ will\ convert\ all
|
|
files\ with\ an\ extension\ in\ `\-\-drive\-import\-formats`\ to\ their
|
|
associated\ document\ type.
|
|
rclone\ will\ not\ convert\ any\ files\ by\ default,\ since\ the\ conversion
|
|
is\ lossy\ process.
|
|
|
|
The\ conversion\ must\ result\ in\ a\ file\ with\ the\ same\ extension\ when
|
|
the\ `\-\-drive\-export\-formats`\ rules\ are\ applied\ to\ the\ uploaded\ document.
|
|
|
|
Here\ are\ some\ examples\ for\ allowed\ and\ prohibited\ conversions.
|
|
|
|
|\ export\-formats\ |\ import\-formats\ |\ Upload\ Ext\ |\ Document\ Ext\ |\ Allowed\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\-\-\-\-\-\ |\ \-\-\-\-\-\-\-\-\-\-\-\-\-\-\ |\ \-\-\-\-\-\-\-\-\-\-\ |\ \-\-\-\-\-\-\-\-\-\-\-\-\ |\ \-\-\-\-\-\-\-\ |
|
|
|\ odt\ |\ odt\ |\ odt\ |\ odt\ |\ Yes\ |
|
|
|\ odt\ |\ docx,odt\ |\ odt\ |\ odt\ |\ Yes\ |
|
|
|\ \ |\ docx\ |\ docx\ |\ docx\ |\ Yes\ |
|
|
|\ \ |\ odt\ |\ odt\ |\ docx\ |\ No\ |
|
|
|\ odt,docx\ |\ docx,odt\ |\ docx\ |\ odt\ |\ No\ |
|
|
|\ docx,odt\ |\ docx,odt\ |\ docx\ |\ docx\ |\ Yes\ |
|
|
|\ docx,odt\ |\ docx,odt\ |\ odt\ |\ docx\ |\ No\ |
|
|
|
|
This\ limitation\ can\ be\ disabled\ by\ specifying\ `\-\-drive\-allow\-import\-name\-change`.
|
|
When\ using\ this\ flag,\ rclone\ can\ convert\ multiple\ files\ types\ resulting
|
|
in\ the\ same\ document\ type\ at\ once,\ eg\ with\ `\-\-drive\-import\-formats\ docx,odt,txt`,
|
|
all\ files\ having\ these\ extension\ would\ result\ in\ a\ document\ represented\ as\ a\ docx\ file.
|
|
This\ brings\ the\ additional\ risk\ of\ overwriting\ a\ document,\ if\ multiple\ files
|
|
have\ the\ same\ stem.\ Many\ rclone\ operations\ will\ not\ handle\ this\ name\ change
|
|
in\ any\ way.\ They\ assume\ an\ equal\ name\ when\ copying\ files\ and\ might\ copy\ the
|
|
file\ again\ or\ delete\ them\ when\ the\ name\ changes.\
|
|
|
|
Here\ are\ the\ possible\ export\ extensions\ with\ their\ corresponding\ mime\ types.
|
|
Most\ of\ these\ can\ also\ be\ used\ for\ importing,\ but\ there\ more\ that\ are\ not
|
|
listed\ here.\ Some\ of\ these\ additional\ ones\ might\ only\ be\ available\ when
|
|
the\ operating\ system\ provides\ the\ correct\ MIME\ type\ entries.
|
|
|
|
This\ list\ can\ be\ changed\ by\ Google\ Drive\ at\ any\ time\ and\ might\ not
|
|
represent\ the\ currently\ available\ conversions.
|
|
|
|
|\ Extension\ |\ Mime\ Type\ |\ Description\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |\-\-\-\-\-\-\-\-\-\-\-|\ \-\-\-\-\-\-\-\-\-\-\-\-|
|
|
|\ csv\ \ |\ text/csv\ |\ Standard\ CSV\ format\ for\ Spreadsheets\ |
|
|
|\ docx\ |\ application/vnd.openxmlformats\-officedocument.wordprocessingml.document\ |\ Microsoft\ Office\ Document\ |
|
|
|\ epub\ |\ application/epub+zip\ |\ E\-book\ format\ |
|
|
|\ html\ |\ text/html\ |\ An\ HTML\ Document\ |
|
|
|\ jpg\ \ |\ image/jpeg\ |\ A\ JPEG\ Image\ File\ |
|
|
|\ json\ |\ application/vnd.google\-apps.script+json\ |\ JSON\ Text\ Format\ |
|
|
|\ odp\ \ |\ application/vnd.oasis.opendocument.presentation\ |\ Openoffice\ Presentation\ |
|
|
|\ ods\ \ |\ application/vnd.oasis.opendocument.spreadsheet\ |\ Openoffice\ Spreadsheet\ |
|
|
|\ ods\ \ |\ application/x\-vnd.oasis.opendocument.spreadsheet\ |\ Openoffice\ Spreadsheet\ |
|
|
|\ odt\ \ |\ application/vnd.oasis.opendocument.text\ |\ Openoffice\ Document\ |
|
|
|\ pdf\ \ |\ application/pdf\ |\ Adobe\ PDF\ Format\ |
|
|
|\ png\ \ |\ image/png\ |\ PNG\ Image\ Format|
|
|
|\ pptx\ |\ application/vnd.openxmlformats\-officedocument.presentationml.presentation\ |\ Microsoft\ Office\ Powerpoint\ |
|
|
|\ rtf\ \ |\ application/rtf\ |\ Rich\ Text\ Format\ |
|
|
|\ svg\ \ |\ image/svg+xml\ |\ Scalable\ Vector\ Graphics\ Format\ |
|
|
|\ tsv\ \ |\ text/tab\-separated\-values\ |\ Standard\ TSV\ format\ for\ spreadsheets\ |
|
|
|\ txt\ \ |\ text/plain\ |\ Plain\ Text\ |
|
|
|\ xlsx\ |\ application/vnd.openxmlformats\-officedocument.spreadsheetml.sheet\ |\ Microsoft\ Office\ Spreadsheet\ |
|
|
|\ zip\ \ |\ application/zip\ |\ A\ ZIP\ file\ of\ HTML,\ Images\ CSS\ |
|
|
|
|
Google\ documents\ can\ also\ be\ exported\ as\ link\ files.\ These\ files\ will
|
|
open\ a\ browser\ window\ for\ the\ Google\ Docs\ website\ of\ that\ document
|
|
when\ opened.\ The\ link\ file\ extension\ has\ to\ be\ specified\ as\ a
|
|
`\-\-drive\-export\-formats`\ parameter.\ They\ will\ match\ all\ available
|
|
Google\ Documents.
|
|
|
|
|\ Extension\ |\ Description\ |\ OS\ Support\ |
|
|
|\ \-\-\-\-\-\-\-\-\-\ |\ \-\-\-\-\-\-\-\-\-\-\-\ |\ \-\-\-\-\-\-\-\-\-\-\ |
|
|
|\ desktop\ |\ freedesktop.org\ specified\ desktop\ entry\ |\ Linux\ |
|
|
|\ link.html\ |\ An\ HTML\ Document\ with\ a\ redirect\ |\ All\ |
|
|
|\ url\ |\ INI\ style\ link\ file\ |\ macOS,\ Windows\ |
|
|
|\ webloc\ |\ macOS\ specific\ XML\ format\ |\ macOS\ |
|
|
|
|
<!\-\-\-\ autogenerated\ options\ start\ \-\ DO\ NOT\ EDIT,\ instead\ edit\ fs.RegInfo\ in\ backend/drive/drive.go\ then\ run\ make\ backenddocs\ \-\->
|
|
###\ Standard\ Options
|
|
|
|
Here\ are\ the\ standard\ options\ specific\ to\ drive\ (Google\ Drive).
|
|
|
|
####\ \-\-drive\-client\-id
|
|
|
|
Google\ Application\ Client\ Id
|
|
Setting\ your\ own\ is\ recommended.
|
|
See\ https://rclone.org/drive/#making\-your\-own\-client\-id\ for\ how\ to\ create\ your\ own.
|
|
If\ you\ leave\ this\ blank,\ it\ will\ use\ an\ internal\ key\ which\ is\ low\ performance.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_CLIENT_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-client\-secret
|
|
|
|
Google\ Application\ Client\ Secret
|
|
Setting\ your\ own\ is\ recommended.
|
|
|
|
\-\ Config:\ \ \ \ \ \ client_secret
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_CLIENT_SECRET
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-scope
|
|
|
|
Scope\ that\ rclone\ should\ use\ when\ requesting\ access\ from\ drive.
|
|
|
|
\-\ Config:\ \ \ \ \ \ scope
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SCOPE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
\-\ Examples:
|
|
\ \ \ \ \-\ "drive"
|
|
\ \ \ \ \ \ \ \ \-\ Full\ access\ all\ files,\ excluding\ Application\ Data\ Folder.
|
|
\ \ \ \ \-\ "drive.readonly"
|
|
\ \ \ \ \ \ \ \ \-\ Read\-only\ access\ to\ file\ metadata\ and\ file\ contents.
|
|
\ \ \ \ \-\ "drive.file"
|
|
\ \ \ \ \ \ \ \ \-\ Access\ to\ files\ created\ by\ rclone\ only.
|
|
\ \ \ \ \ \ \ \ \-\ These\ are\ visible\ in\ the\ drive\ website.
|
|
\ \ \ \ \ \ \ \ \-\ File\ authorization\ is\ revoked\ when\ the\ user\ deauthorizes\ the\ app.
|
|
\ \ \ \ \-\ "drive.appfolder"
|
|
\ \ \ \ \ \ \ \ \-\ Allows\ read\ and\ write\ access\ to\ the\ Application\ Data\ folder.
|
|
\ \ \ \ \ \ \ \ \-\ This\ is\ not\ visible\ in\ the\ drive\ website.
|
|
\ \ \ \ \-\ "drive.metadata.readonly"
|
|
\ \ \ \ \ \ \ \ \-\ Allows\ read\-only\ access\ to\ file\ metadata\ but
|
|
\ \ \ \ \ \ \ \ \-\ does\ not\ allow\ any\ access\ to\ read\ or\ download\ file\ content.
|
|
|
|
####\ \-\-drive\-root\-folder\-id
|
|
|
|
ID\ of\ the\ root\ folder
|
|
Leave\ blank\ normally.
|
|
|
|
Fill\ in\ to\ access\ "Computers"\ folders\ (see\ docs),\ or\ for\ rclone\ to\ use
|
|
a\ non\ root\ folder\ as\ its\ starting\ point.
|
|
|
|
Note\ that\ if\ this\ is\ blank,\ the\ first\ time\ rclone\ runs\ it\ will\ fill\ it
|
|
in\ with\ the\ ID\ of\ the\ root\ folder.
|
|
|
|
|
|
\-\ Config:\ \ \ \ \ \ root_folder_id
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_ROOT_FOLDER_ID
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-service\-account\-file
|
|
|
|
Service\ Account\ Credentials\ JSON\ file\ path\
|
|
Leave\ blank\ normally.
|
|
Needed\ only\ if\ you\ want\ use\ SA\ instead\ of\ interactive\ login.
|
|
|
|
\-\ Config:\ \ \ \ \ \ service_account_file
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
###\ Advanced\ Options
|
|
|
|
Here\ are\ the\ advanced\ options\ specific\ to\ drive\ (Google\ Drive).
|
|
|
|
####\ \-\-drive\-service\-account\-credentials
|
|
|
|
Service\ Account\ Credentials\ JSON\ blob
|
|
Leave\ blank\ normally.
|
|
Needed\ only\ if\ you\ want\ use\ SA\ instead\ of\ interactive\ login.
|
|
|
|
\-\ Config:\ \ \ \ \ \ service_account_credentials
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-team\-drive
|
|
|
|
ID\ of\ the\ Team\ Drive
|
|
|
|
\-\ Config:\ \ \ \ \ \ team_drive
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_TEAM_DRIVE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-auth\-owner\-only
|
|
|
|
Only\ consider\ files\ owned\ by\ the\ authenticated\ user.
|
|
|
|
\-\ Config:\ \ \ \ \ \ auth_owner_only
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_AUTH_OWNER_ONLY
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-use\-trash
|
|
|
|
Send\ files\ to\ the\ trash\ instead\ of\ deleting\ permanently.
|
|
Defaults\ to\ true,\ namely\ sending\ files\ to\ the\ trash.
|
|
Use\ `\-\-drive\-use\-trash=false`\ to\ delete\ files\ permanently\ instead.
|
|
|
|
\-\ Config:\ \ \ \ \ \ use_trash
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_USE_TRASH
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ true
|
|
|
|
####\ \-\-drive\-skip\-gdocs
|
|
|
|
Skip\ google\ documents\ in\ all\ listings.
|
|
If\ given,\ gdocs\ practically\ become\ invisible\ to\ rclone.
|
|
|
|
\-\ Config:\ \ \ \ \ \ skip_gdocs
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SKIP_GDOCS
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-skip\-checksum\-gphotos
|
|
|
|
Skip\ MD5\ checksum\ on\ Google\ photos\ and\ videos\ only.
|
|
|
|
Use\ this\ if\ you\ get\ checksum\ errors\ when\ transferring\ Google\ photos\ or
|
|
videos.
|
|
|
|
Setting\ this\ flag\ will\ cause\ Google\ photos\ and\ videos\ to\ return\ a
|
|
blank\ MD5\ checksum.
|
|
|
|
Google\ photos\ are\ identifed\ by\ being\ in\ the\ "photos"\ space.
|
|
|
|
Corrupted\ checksums\ are\ caused\ by\ Google\ modifying\ the\ image/video\ but
|
|
not\ updating\ the\ checksum.
|
|
|
|
\-\ Config:\ \ \ \ \ \ skip_checksum_gphotos
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-shared\-with\-me
|
|
|
|
Only\ show\ files\ that\ are\ shared\ with\ me.
|
|
|
|
Instructs\ rclone\ to\ operate\ on\ your\ "Shared\ with\ me"\ folder\ (where
|
|
Google\ Drive\ lets\ you\ access\ the\ files\ and\ folders\ others\ have\ shared
|
|
with\ you).
|
|
|
|
This\ works\ both\ with\ the\ "list"\ (lsd,\ lsl,\ etc)\ and\ the\ "copy"
|
|
commands\ (copy,\ sync,\ etc),\ and\ with\ all\ other\ commands\ too.
|
|
|
|
\-\ Config:\ \ \ \ \ \ shared_with_me
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SHARED_WITH_ME
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-trashed\-only
|
|
|
|
Only\ show\ files\ that\ are\ in\ the\ trash.
|
|
This\ will\ show\ trashed\ files\ in\ their\ original\ directory\ structure.
|
|
|
|
\-\ Config:\ \ \ \ \ \ trashed_only
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_TRASHED_ONLY
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-formats
|
|
|
|
Deprecated:\ see\ export_formats
|
|
|
|
\-\ Config:\ \ \ \ \ \ formats
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_FORMATS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-export\-formats
|
|
|
|
Comma\ separated\ list\ of\ preferred\ formats\ for\ downloading\ Google\ docs.
|
|
|
|
\-\ Config:\ \ \ \ \ \ export_formats
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_EXPORT_FORMATS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ "docx,xlsx,pptx,svg"
|
|
|
|
####\ \-\-drive\-import\-formats
|
|
|
|
Comma\ separated\ list\ of\ preferred\ formats\ for\ uploading\ Google\ docs.
|
|
|
|
\-\ Config:\ \ \ \ \ \ import_formats
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_IMPORT_FORMATS
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-allow\-import\-name\-change
|
|
|
|
Allow\ the\ filetype\ to\ change\ when\ uploading\ Google\ docs\ (e.g.\ file.doc\ to\ file.docx).\ This\ will\ confuse\ sync\ and\ reupload\ every\ time.
|
|
|
|
\-\ Config:\ \ \ \ \ \ allow_import_name_change
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-use\-created\-date
|
|
|
|
Use\ file\ created\ date\ instead\ of\ modified\ date.,
|
|
|
|
Useful\ when\ downloading\ data\ and\ you\ want\ the\ creation\ date\ used\ in
|
|
place\ of\ the\ last\ modified\ date.
|
|
|
|
**WARNING**:\ This\ flag\ may\ have\ some\ unexpected\ consequences.
|
|
|
|
When\ uploading\ to\ your\ drive\ all\ files\ will\ be\ overwritten\ unless\ they
|
|
haven\[aq]t\ been\ modified\ since\ their\ creation.\ And\ the\ inverse\ will\ occur
|
|
while\ downloading.\ \ This\ side\ effect\ can\ be\ avoided\ by\ using\ the
|
|
"\-\-checksum"\ flag.
|
|
|
|
This\ feature\ was\ implemented\ to\ retain\ photos\ capture\ date\ as\ recorded
|
|
by\ google\ photos.\ You\ will\ first\ need\ to\ check\ the\ "Create\ a\ Google
|
|
Photos\ folder"\ option\ in\ your\ google\ drive\ settings.\ You\ can\ then\ copy
|
|
or\ move\ the\ photos\ locally\ and\ use\ the\ date\ the\ image\ was\ taken
|
|
(created)\ set\ as\ the\ modification\ date.
|
|
|
|
\-\ Config:\ \ \ \ \ \ use_created_date
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_USE_CREATED_DATE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-list\-chunk
|
|
|
|
Size\ of\ listing\ chunk\ 100\-1000.\ 0\ to\ disable.
|
|
|
|
\-\ Config:\ \ \ \ \ \ list_chunk
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_LIST_CHUNK
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 1000
|
|
|
|
####\ \-\-drive\-impersonate
|
|
|
|
Impersonate\ this\ user\ when\ using\ a\ service\ account.
|
|
|
|
\-\ Config:\ \ \ \ \ \ impersonate
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_IMPERSONATE
|
|
\-\ Type:\ \ \ \ \ \ \ \ string
|
|
\-\ Default:\ \ \ \ \ ""
|
|
|
|
####\ \-\-drive\-alternate\-export
|
|
|
|
Use\ alternate\ export\ URLs\ for\ google\ documents\ export.,
|
|
|
|
If\ this\ option\ is\ set\ this\ instructs\ rclone\ to\ use\ an\ alternate\ set\ of
|
|
export\ URLs\ for\ drive\ documents.\ \ Users\ have\ reported\ that\ the
|
|
official\ export\ URLs\ can\[aq]t\ export\ large\ documents,\ whereas\ these
|
|
unofficial\ ones\ can.
|
|
|
|
See\ rclone\ issue\ [#2243](https://github.com/rclone/rclone/issues/2243)\ for\ background,
|
|
[this\ google\ drive\ issue](https://issuetracker.google.com/issues/36761333)\ and
|
|
[this\ helpful\ post](https://www.labnol.org/internet/direct\-links\-for\-google\-drive/28356/).
|
|
|
|
\-\ Config:\ \ \ \ \ \ alternate_export
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_ALTERNATE_EXPORT
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-upload\-cutoff
|
|
|
|
Cutoff\ for\ switching\ to\ chunked\ upload
|
|
|
|
\-\ Config:\ \ \ \ \ \ upload_cutoff
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_UPLOAD_CUTOFF
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 8M
|
|
|
|
####\ \-\-drive\-chunk\-size
|
|
|
|
Upload\ chunk\ size.\ Must\ a\ power\ of\ 2\ >=\ 256k.
|
|
|
|
Making\ this\ larger\ will\ improve\ performance,\ but\ note\ that\ each\ chunk
|
|
is\ buffered\ in\ memory\ one\ per\ transfer.
|
|
|
|
Reducing\ this\ will\ reduce\ memory\ usage\ but\ decrease\ performance.
|
|
|
|
\-\ Config:\ \ \ \ \ \ chunk_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_CHUNK_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ 8M
|
|
|
|
####\ \-\-drive\-acknowledge\-abuse
|
|
|
|
Set\ to\ allow\ files\ which\ return\ cannotDownloadAbusiveFile\ to\ be\ downloaded.
|
|
|
|
If\ downloading\ a\ file\ returns\ the\ error\ "This\ file\ has\ been\ identified
|
|
as\ malware\ or\ spam\ and\ cannot\ be\ downloaded"\ with\ the\ error\ code
|
|
"cannotDownloadAbusiveFile"\ then\ supply\ this\ flag\ to\ rclone\ to
|
|
indicate\ you\ acknowledge\ the\ risks\ of\ downloading\ the\ file\ and\ rclone
|
|
will\ download\ it\ anyway.
|
|
|
|
\-\ Config:\ \ \ \ \ \ acknowledge_abuse
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-keep\-revision\-forever
|
|
|
|
Keep\ new\ head\ revision\ of\ each\ file\ forever.
|
|
|
|
\-\ Config:\ \ \ \ \ \ keep_revision_forever
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_KEEP_REVISION_FOREVER
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-size\-as\-quota
|
|
|
|
Show\ storage\ quota\ usage\ for\ file\ size.
|
|
|
|
The\ storage\ used\ by\ a\ file\ is\ the\ size\ of\ the\ current\ version\ plus\ any
|
|
older\ versions\ that\ have\ been\ set\ to\ keep\ forever.
|
|
|
|
\-\ Config:\ \ \ \ \ \ size_as_quota
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SIZE_AS_QUOTA
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-v2\-download\-min\-size
|
|
|
|
If\ Object\[aq]s\ are\ greater,\ use\ drive\ v2\ API\ to\ download.
|
|
|
|
\-\ Config:\ \ \ \ \ \ v2_download_min_size
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
|
|
\-\ Type:\ \ \ \ \ \ \ \ SizeSuffix
|
|
\-\ Default:\ \ \ \ \ off
|
|
|
|
####\ \-\-drive\-pacer\-min\-sleep
|
|
|
|
Minimum\ time\ to\ sleep\ between\ API\ calls.
|
|
|
|
\-\ Config:\ \ \ \ \ \ pacer_min_sleep
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_PACER_MIN_SLEEP
|
|
\-\ Type:\ \ \ \ \ \ \ \ Duration
|
|
\-\ Default:\ \ \ \ \ 100ms
|
|
|
|
####\ \-\-drive\-pacer\-burst
|
|
|
|
Number\ of\ API\ calls\ to\ allow\ without\ sleeping.
|
|
|
|
\-\ Config:\ \ \ \ \ \ pacer_burst
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_PACER_BURST
|
|
\-\ Type:\ \ \ \ \ \ \ \ int
|
|
\-\ Default:\ \ \ \ \ 100
|
|
|
|
####\ \-\-drive\-server\-side\-across\-configs
|
|
|
|
Allow\ server\ side\ operations\ (eg\ copy)\ to\ work\ across\ different\ drive\ configs.
|
|
|
|
This\ can\ be\ useful\ if\ you\ wish\ to\ do\ a\ server\ side\ copy\ between\ two
|
|
different\ Google\ drives.\ \ Note\ that\ this\ isn\[aq]t\ enabled\ by\ default
|
|
because\ it\ isn\[aq]t\ easy\ to\ tell\ if\ it\ will\ work\ between\ any\ two
|
|
configurations.
|
|
|
|
\-\ Config:\ \ \ \ \ \ server_side_across_configs
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ false
|
|
|
|
####\ \-\-drive\-disable\-http2
|
|
|
|
Disable\ drive\ using\ http2
|
|
|
|
There\ is\ currently\ an\ unsolved\ issue\ with\ the\ google\ drive\ backend\ and
|
|
HTTP/2.\ \ HTTP/2\ is\ therefore\ disabled\ by\ default\ for\ the\ drive\ backend
|
|
but\ can\ be\ re\-enabled\ here.\ \ When\ the\ issue\ is\ solved\ this\ flag\ will
|
|
be\ removed.
|
|
|
|
See:\ https://github.com/rclone/rclone/issues/3631
|
|
|
|
|
|
|
|
\-\ Config:\ \ \ \ \ \ disable_http2
|
|
\-\ Env\ Var:\ \ \ \ \ RCLONE_DRIVE_DISABLE_HTTP2
|
|
\-\ Type:\ \ \ \ \ \ \ \ bool
|
|
\-\ Default:\ \ \ \ \ true
|
|
|
|
<!\-\-\-\ autogenerated\ options\ stop\ \-\->
|
|
|
|
###\ Limitations\ ###
|
|
|
|
Drive\ has\ quite\ a\ lot\ of\ rate\ limiting.\ \ This\ causes\ rclone\ to\ be
|
|
limited\ to\ transferring\ about\ 2\ files\ per\ second\ only.\ \ Individual
|
|
files\ may\ be\ transferred\ much\ faster\ at\ 100s\ of\ MBytes/s\ but\ lots\ of
|
|
small\ files\ can\ take\ a\ long\ time.
|
|
|
|
Server\ side\ copies\ are\ also\ subject\ to\ a\ separate\ rate\ limit.\ If\ you
|
|
see\ User\ rate\ limit\ exceeded\ errors,\ wait\ at\ least\ 24\ hours\ and\ retry.
|
|
You\ can\ disable\ server\ side\ copies\ with\ `\-\-disable\ copy`\ to\ download
|
|
and\ upload\ the\ files\ if\ you\ prefer.
|
|
|
|
####\ Limitations\ of\ Google\ Docs\ ####
|
|
|
|
Google\ docs\ will\ appear\ as\ size\ \-1\ in\ `rclone\ ls`\ and\ as\ size\ 0\ in
|
|
anything\ which\ uses\ the\ VFS\ layer,\ eg\ `rclone\ mount`,\ `rclone\ serve`.
|
|
|
|
This\ is\ because\ rclone\ can\[aq]t\ find\ out\ the\ size\ of\ the\ Google\ docs
|
|
without\ downloading\ them.
|
|
|
|
Google\ docs\ will\ transfer\ correctly\ with\ `rclone\ sync`,\ `rclone\ copy`
|
|
etc\ as\ rclone\ knows\ to\ ignore\ the\ size\ when\ doing\ the\ transfer.
|
|
|
|
However\ an\ unfortunate\ consequence\ of\ this\ is\ that\ you\ may\ not\ be\ able
|
|
to\ download\ Google\ docs\ using\ `rclone\ mount`.\ If\ it\ doesn\[aq]t\ work\ you
|
|
will\ get\ a\ 0\ sized\ file.\ \ If\ you\ try\ again\ the\ doc\ may\ gain\ its
|
|
correct\ size\ and\ be\ downloadable.\ Whther\ it\ will\ work\ on\ not\ depends
|
|
on\ the\ application\ accessing\ the\ mount\ and\ the\ OS\ you\ are\ running\ \-
|
|
experiment\ to\ find\ out\ if\ it\ does\ work\ for\ you!
|
|
|
|
###\ Duplicated\ files\ ###
|
|
|
|
Sometimes,\ for\ no\ reason\ I\[aq]ve\ been\ able\ to\ track\ down,\ drive\ will
|
|
duplicate\ a\ file\ that\ rclone\ uploads.\ \ Drive\ unlike\ all\ the\ other
|
|
remotes\ can\ have\ duplicated\ files.
|
|
|
|
Duplicated\ files\ cause\ problems\ with\ the\ syncing\ and\ you\ will\ see
|
|
messages\ in\ the\ log\ about\ duplicates.
|
|
|
|
Use\ `rclone\ dedupe`\ to\ fix\ duplicated\ files.
|
|
|
|
Note\ that\ this\ isn\[aq]t\ just\ a\ problem\ with\ rclone,\ even\ Google\ Photos\ on
|
|
Android\ duplicates\ files\ on\ drive\ sometimes.
|
|
|
|
###\ Rclone\ appears\ to\ be\ re\-copying\ files\ it\ shouldn\[aq]t\ ###
|
|
|
|
The\ most\ likely\ cause\ of\ this\ is\ the\ duplicated\ file\ issue\ above\ \-\ run
|
|
`rclone\ dedupe`\ and\ check\ your\ logs\ for\ duplicate\ object\ or\ directory
|
|
messages.
|
|
|
|
This\ can\ also\ be\ caused\ by\ a\ delay/caching\ on\ google\ drive\[aq]s\ end\ when
|
|
comparing\ directory\ listings.\ Specifically\ with\ team\ drives\ used\ in
|
|
combination\ with\ \-\-fast\-list.\ Files\ that\ were\ uploaded\ recently\ may
|
|
not\ appear\ on\ the\ directory\ list\ sent\ to\ rclone\ when\ using\ \-\-fast\-list.
|
|
|
|
Waiting\ a\ moderate\ period\ of\ time\ between\ attempts\ (estimated\ to\ be
|
|
approximately\ 1\ hour)\ and/or\ not\ using\ \-\-fast\-list\ both\ seem\ to\ be
|
|
effective\ in\ preventing\ the\ problem.
|
|
|
|
###\ Making\ your\ own\ client_id\ ###
|
|
|
|
When\ you\ use\ rclone\ with\ Google\ drive\ in\ its\ default\ configuration\ you
|
|
are\ using\ rclone\[aq]s\ client_id.\ \ This\ is\ shared\ between\ all\ the\ rclone
|
|
users.\ \ There\ is\ a\ global\ rate\ limit\ on\ the\ number\ of\ queries\ per
|
|
second\ that\ each\ client_id\ can\ do\ set\ by\ Google.\ \ rclone\ already\ has\ a
|
|
high\ quota\ and\ I\ will\ continue\ to\ make\ sure\ it\ is\ high\ enough\ by
|
|
contacting\ Google.
|
|
|
|
It\ is\ strongly\ recommended\ to\ use\ your\ own\ client\ ID\ as\ the\ default\ rclone\ ID\ is\ heavily\ used.\ If\ you\ have\ multiple\ services\ running,\ it\ is\ recommended\ to\ use\ an\ API\ key\ for\ each\ service.\ The\ default\ Google\ quota\ is\ 10\ transactions\ per\ second\ so\ it\ is\ recommended\ to\ stay\ under\ that\ number\ as\ if\ you\ use\ more\ than\ that,\ it\ will\ cause\ rclone\ to\ rate\ limit\ and\ make\ things\ slower.
|
|
|
|
Here\ is\ how\ to\ create\ your\ own\ Google\ Drive\ client\ ID\ for\ rclone:
|
|
|
|
1.\ Log\ into\ the\ [Google\ API
|
|
Console](https://console.developers.google.com/)\ with\ your\ Google
|
|
account.\ It\ doesn\[aq]t\ matter\ what\ Google\ account\ you\ use.\ (It\ need\ not
|
|
be\ the\ same\ account\ as\ the\ Google\ Drive\ you\ want\ to\ access)
|
|
|
|
2.\ Select\ a\ project\ or\ create\ a\ new\ project.
|
|
|
|
3.\ Under\ "ENABLE\ APIS\ AND\ SERVICES"\ search\ for\ "Drive",\ and\ enable\ the
|
|
"Google\ Drive\ API".
|
|
|
|
4.\ Click\ "Credentials"\ in\ the\ left\-side\ panel\ (not\ "Create
|
|
credentials",\ which\ opens\ the\ wizard),\ then\ "Create\ credentials",\ then
|
|
"OAuth\ client\ ID".\ \ It\ will\ prompt\ you\ to\ set\ the\ OAuth\ consent\ screen
|
|
product\ name,\ if\ you\ haven\[aq]t\ set\ one\ already.
|
|
|
|
5.\ Choose\ an\ application\ type\ of\ "other",\ and\ click\ "Create".\ (the
|
|
default\ name\ is\ fine)
|
|
|
|
6.\ It\ will\ show\ you\ a\ client\ ID\ and\ client\ secret.\ \ Use\ these\ values
|
|
in\ rclone\ config\ to\ add\ a\ new\ remote\ or\ edit\ an\ existing\ remote.
|
|
|
|
(Thanks\ to\ \@balazer\ on\ github\ for\ these\ instructions.)
|
|
|
|
Google\ Photos
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
|
|
The\ rclone\ backend\ for\ [Google\ Photos](https://www.google.com/photos/about/)\ is
|
|
a\ specialized\ backend\ for\ transferring\ photos\ and\ videos\ to\ and\ from
|
|
Google\ Photos.
|
|
|
|
**NB**\ The\ Google\ Photos\ API\ which\ rclone\ uses\ has\ quite\ a\ few
|
|
limitations,\ so\ please\ read\ the\ [limitations\ section](#limitations)
|
|
carefully\ to\ make\ sure\ it\ is\ suitable\ for\ your\ use.
|
|
|
|
##\ Configuring\ Google\ Photos
|
|
|
|
The\ initial\ setup\ for\ google\ cloud\ storage\ involves\ getting\ a\ token\ from\ Google\ Photos
|
|
which\ you\ need\ to\ do\ in\ your\ browser.\ \ `rclone\ config`\ walks\ you
|
|
through\ it.
|
|
|
|
Here\ is\ an\ example\ of\ how\ to\ make\ a\ remote\ called\ `remote`.\ \ First\ run:
|
|
|
|
\ \ \ \ \ rclone\ config
|
|
|
|
This\ will\ guide\ you\ through\ an\ interactive\ setup\ process:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
No remotes found \- make a new one n) New remote s) Set configuration
|
|
password q) Quit config n/s/q> n name> remote Type of storage to
|
|
configure.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). Choose a number from below, or
|
|
type in your own value [snip] XX / Google Photos \ \[rq]google photos"
|
|
[snip] Storage> google photos ** See help for google photos backend at:
|
|
https://rclone.org/googlephotos/ **
|
|
.PP
|
|
Google Application Client Id Leave blank normally.
|
|
Enter a string value.
|
|
Press Enter for the default ("\[lq]). client_id> Google Application
|
|
Client Secret Leave blank normally. Enter a string value. Press Enter
|
|
for the default (\[rq]").
|
|
client_secret> Set to make the Google Photos backend read only.
|
|
.PP
|
|
If you choose read only then rclone will only request read only access
|
|
to your photos, otherwise rclone will request full access.
|
|
Enter a boolean value (true or false).
|
|
Press Enter for the default (\[lq]false\[rq]).
|
|
read_only> Edit advanced config?
|
|
(y/n) y) Yes n) No y/n> n Remote config Use auto config?
|
|
* Say Y if not sure * Say N if you are working on a remote or headless
|
|
machine y) Yes n) No y/n> y If your browser doesn't open automatically
|
|
go to the following link: http://127.0.0.1:53682/auth Log in and
|
|
authorize rclone for access Waiting for code\&... Got code
|
|
.PP
|
|
*** IMPORTANT: All media items uploaded to Google Photos with rclone ***
|
|
are stored in full resolution at original quality.
|
|
These uploads *** will count towards storage in your Google Account.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(20.4n).
|
|
T{
|
|
[remote] type = google photos token =
|
|
{\[lq]access_token\[rq]:\[lq]XXX\[rq],\[lq]token_type\[rq]:\[lq]Bearer\[rq],\[lq]refresh_token\[rq]:\[lq]XXX\[rq],\[lq]expiry\[rq]:\[lq]2019\-06\-28T17:38:04.644930156+01:00\[rq]}
|
|
T}
|
|
_
|
|
T{
|
|
y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y ```
|
|
T}
|
|
T{
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this may require you to
|
|
unblock it temporarily if you are running a host firewall, or use manual
|
|
mode.
|
|
T}
|
|
T{
|
|
This remote is called \f[C]remote\f[] and can now be used like this
|
|
T}
|
|
T{
|
|
See all the albums in your photos
|
|
T}
|
|
T{
|
|
rclone lsd remote:album
|
|
T}
|
|
T{
|
|
Make a new album
|
|
T}
|
|
T{
|
|
rclone mkdir remote:album/newAlbum
|
|
T}
|
|
T{
|
|
List the contents of an album
|
|
T}
|
|
T{
|
|
rclone ls remote:album/newAlbum
|
|
T}
|
|
T{
|
|
Sync \f[C]/home/local/images\f[] to the Google Photos, removing any
|
|
excess files in the album.
|
|
T}
|
|
T{
|
|
rclone sync /home/local/image remote:album/newAlbum
|
|
T}
|
|
T{
|
|
## Layout
|
|
T}
|
|
T{
|
|
As Google Photos is not a general purpose cloud storage system the
|
|
backend is laid out to help you navigate it.
|
|
T}
|
|
T{
|
|
The directories under \f[C]media\f[] show different ways of categorizing
|
|
the media.
|
|
Each file will appear multiple times.
|
|
So if you want to make a backup of your google photos you might choose
|
|
to backup \f[C]remote:media/by\-month\f[].
|
|
(\f[B]NB\f[] \f[C]remote:media/by\-day\f[] is rather slow at the moment
|
|
so avoid for syncing.)
|
|
T}
|
|
T{
|
|
Note that all your photos and videos will appear somewhere under
|
|
\f[C]media\f[], but they may not appear under \f[C]album\f[] unless
|
|
you've put them into albums.
|
|
T}
|
|
T{
|
|
\f[C]/\ \-\ upload\ \-\ file1.jpg\ \-\ file2.jpg\ \-\ ...\ \-\ media\ \-\ all\ \-\ file1.jpg\ \-\ file2.jpg\ \-\ ...\ \-\ by\-year\ \-\ 2000\ \-\ file1.jpg\ \-\ ...\ \-\ 2001\ \-\ file2.jpg\ \-\ ...\ \-\ ...\ \-\ by\-month\ \-\ 2000\ \-\ 2000\-01\ \-\ file1.jpg\ \-\ ...\ \-\ 2000\-02\ \-\ file2.jpg\ \-\ ...\ \-\ ...\ \-\ by\-day\ \-\ 2000\ \-\ 2000\-01\-01\ \-\ file1.jpg\ \-\ ...\ \-\ 2000\-01\-02\ \-\ file2.jpg\ \-\ ...\ \-\ ...\ \-\ album\ \-\ album\ name\ \-\ album\ name/sub\ \-\ shared\-album\ \-\ album\ name\ \-\ album\ name/sub\f[]
|
|
T}
|
|
T{
|
|
There are two writable parts of the tree, the \f[C]upload\f[] directory
|
|
and sub directories of the the \f[C]album\f[] directory.
|
|
T}
|
|
T{
|
|
The \f[C]upload\f[] directory is for uploading files you don't want to
|
|
put into albums.
|
|
This will be empty to start with and will contain the files you've
|
|
uploaded for one rclone session only, becoming empty again when you
|
|
restart rclone.
|
|
The use case for this would be if you have a load of files you just want
|
|
to once off dump into Google Photos.
|
|
For repeated syncing, uploading to \f[C]album\f[] will work better.
|
|
T}
|
|
T{
|
|
Directories within the \f[C]album\f[] directory are also writeable and
|
|
you may create new directories (albums) under \f[C]album\f[].
|
|
If you copy files with a directory hierarchy in there then rclone will
|
|
create albums with the \f[C]/\f[] character in them.
|
|
For example if you do
|
|
T}
|
|
T{
|
|
rclone copy /path/to/images remote:album/images
|
|
T}
|
|
T{
|
|
and the images directory contains
|
|
T}
|
|
T{
|
|
\f[C]images\ \-\ file1.jpg\ dir\ file2.jpg\ dir2\ dir3\ file3.jpg\f[]
|
|
T}
|
|
T{
|
|
Then rclone will create the following albums with the following files in
|
|
T}
|
|
T{
|
|
\- images \- file1.jpg \- images/dir \- file2.jpg \- images/dir2/dir3 \-
|
|
file3.jpg
|
|
T}
|
|
T{
|
|
This means that you can use the \f[C]album\f[] path pretty much like a
|
|
normal filesystem and it is a good target for repeated syncing.
|
|
T}
|
|
T{
|
|
The \f[C]shared\-album\f[] directory shows albums shared with you or by
|
|
you.
|
|
This is similar to the Sharing tab in the Google Photos web interface.
|
|
T}
|
|
T{
|
|
## Limitations
|
|
T}
|
|
T{
|
|
Only images and videos can be uploaded.
|
|
If you attempt to upload non videos or images or formats that Google
|
|
Photos doesn't understand, rclone will upload the file, then Google
|
|
Photos will give an error when it is put turned into a media item.
|
|
T}
|
|
T{
|
|
Note that all media items uploaded to Google Photos through the API are
|
|
stored in full resolution at \[lq]original quality\[rq] and
|
|
\f[B]will\f[] count towards your storage quota in your Google Account.
|
|
The API does \f[B]not\f[] offer a way to upload in \[lq]high
|
|
quality\[rq] mode..
|
|
T}
|
|
T{
|
|
### Downloading Images
|
|
T}
|
|
T{
|
|
When Images are downloaded this strips EXIF location (according to the
|
|
docs and my tests).
|
|
This is a limitation of the Google Photos API and is covered by bug
|
|
#112096115 (https://issuetracker.google.com/issues/112096115).
|
|
T}
|
|
T{
|
|
\f[B]The current google API does not allow photos to be downloaded at
|
|
original resolution. This is very important if you are, for example,
|
|
relying on \[lq]Google Photos\[rq] as a backup of your photos. You will
|
|
not be able to use rclone to redownload original images. You could use
|
|
`google takeout' to recover the original photos as a last resort\f[]
|
|
T}
|
|
T{
|
|
### Downloading Videos
|
|
T}
|
|
T{
|
|
When videos are downloaded they are downloaded in a really compressed
|
|
version of the video compared to downloading it via the Google Photos
|
|
web interface.
|
|
This is covered by bug
|
|
#113672044 (https://issuetracker.google.com/issues/113672044).
|
|
T}
|
|
T{
|
|
### Duplicates
|
|
T}
|
|
T{
|
|
If a file name is duplicated in a directory then rclone will add the
|
|
file ID into its name.
|
|
So two files called \f[C]file.jpg\f[] would then appear as
|
|
\f[C]file\ {123456}.jpg\f[] and \f[C]file\ {ABCDEF}.jpg\f[] (the actual
|
|
IDs are a lot longer alas!).
|
|
T}
|
|
T{
|
|
If you upload the same image (with the same binary data) twice then
|
|
Google Photos will deduplicate it.
|
|
However it will retain the filename from the first upload which may
|
|
confuse rclone.
|
|
For example if you uploaded an image to \f[C]upload\f[] then uploaded
|
|
the same image to \f[C]album/my_album\f[] the filename of the image in
|
|
\f[C]album/my_album\f[] will be what it was uploaded with initially, not
|
|
what you uploaded it with to \f[C]album\f[].
|
|
In practise this shouldn't cause too many problems.
|
|
T}
|
|
T{
|
|
### Modified time
|
|
T}
|
|
T{
|
|
The date shown of media in Google Photos is the creation date as
|
|
determined by the EXIF information, or the upload date if that is not
|
|
known.
|
|
T}
|
|
T{
|
|
This is not changeable by rclone and is not the modification date of the
|
|
media on local disk.
|
|
This means that rclone cannot use the dates from Google Photos for
|
|
syncing purposes.
|
|
T}
|
|
T{
|
|
### Size
|
|
T}
|
|
T{
|
|
The Google Photos API does not return the size of media.
|
|
This means that when syncing to Google Photos, rclone can only do a file
|
|
existence check.
|
|
T}
|
|
T{
|
|
It is possible to read the size of the media, but this needs an extra
|
|
HTTP HEAD request per media item so is \f[B]very slow\f[] and uses up a
|
|
lot of transactions.
|
|
This can be enabled with the \f[C]\-\-gphotos\-read\-size\f[] option or
|
|
the \f[C]read_size\ =\ true\f[] config parameter.
|
|
T}
|
|
T{
|
|
If you want to use the backend with \f[C]rclone\ mount\f[] you may need
|
|
to enable this flag (depending on your OS and application using the
|
|
photos) otherwise you may not be able to read media off the mount.
|
|
You'll need to experiment to see if it works for you without the flag.
|
|
T}
|
|
T{
|
|
### Albums
|
|
T}
|
|
T{
|
|
Rclone can only upload files to albums it created.
|
|
This is a limitation of the Google Photos
|
|
API (https://developers.google.com/photos/library/guides/manage-albums).
|
|
T}
|
|
T{
|
|
Rclone can remove files it uploaded from albums it created only.
|
|
T}
|
|
T{
|
|
### Deleting files
|
|
T}
|
|
T{
|
|
Rclone can remove files from albums it created, but note that the Google
|
|
Photos API does not allow media to be deleted permanently so this media
|
|
will still remain.
|
|
See bug #109759781 (https://issuetracker.google.com/issues/109759781).
|
|
T}
|
|
T{
|
|
Rclone cannot delete files anywhere except under \f[C]album\f[].
|
|
T}
|
|
T{
|
|
### Deleting albums
|
|
T}
|
|
T{
|
|
The Google Photos API does not support deleting albums \- see bug
|
|
#135714733 (https://issuetracker.google.com/issues/135714733).
|
|
T}
|
|
T{
|
|
### Standard Options
|
|
T}
|
|
T{
|
|
Here are the standard options specific to google photos (Google Photos).
|
|
T}
|
|
T{
|
|
#### \[en]gphotos\-client\-id
|
|
T}
|
|
T{
|
|
Google Application Client Id Leave blank normally.
|
|
T}
|
|
T{
|
|
\- Config: client_id \- Env Var: RCLONE_GPHOTOS_CLIENT_ID \- Type:
|
|
string \- Default: ""
|
|
T}
|
|
T{
|
|
#### \[en]gphotos\-client\-secret
|
|
T}
|
|
T{
|
|
Google Application Client Secret Leave blank normally.
|
|
T}
|
|
T{
|
|
\- Config: client_secret \- Env Var: RCLONE_GPHOTOS_CLIENT_SECRET \-
|
|
Type: string \- Default: ""
|
|
T}
|
|
T{
|
|
#### \[en]gphotos\-read\-only
|
|
T}
|
|
T{
|
|
Set to make the Google Photos backend read only.
|
|
T}
|
|
T{
|
|
If you choose read only then rclone will only request read only access
|
|
to your photos, otherwise rclone will request full access.
|
|
T}
|
|
T{
|
|
\- Config: read_only \- Env Var: RCLONE_GPHOTOS_READ_ONLY \- Type: bool
|
|
\- Default: false
|
|
T}
|
|
T{
|
|
### Advanced Options
|
|
T}
|
|
T{
|
|
Here are the advanced options specific to google photos (Google Photos).
|
|
T}
|
|
T{
|
|
#### \[en]gphotos\-read\-size
|
|
T}
|
|
T{
|
|
Set to read the size of media items.
|
|
T}
|
|
T{
|
|
Normally rclone does not read the size of media items since this takes
|
|
another transaction.
|
|
This isn't necessary for syncing.
|
|
However rclone mount needs to know the size of files in advance of
|
|
reading them, so setting this flag when using rclone mount is
|
|
recommended if you want to read the media.
|
|
T}
|
|
T{
|
|
\- Config: read_size \- Env Var: RCLONE_GPHOTOS_READ_SIZE \- Type: bool
|
|
\- Default: false
|
|
T}
|
|
T{
|
|
T}
|
|
T{
|
|
HTTP
|
|
T}
|
|
.TE
|
|
.PP
|
|
The HTTP remote is a read only remote for reading files of a webserver.
|
|
The webserver should provide file listings which rclone will read and
|
|
turn into a remote.
|
|
This has been tested with common webservers such as Apache/Nginx/Caddy
|
|
and will likely work with file listings from most web servers.
|
|
(If it doesn't then please file an issue, or send a pull request!)
|
|
.PP
|
|
Paths are specified as \f[C]remote:\f[] or \f[C]remote:path/to/dir\f[].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ http\ Connection
|
|
\ \ \ \\\ "http"
|
|
[snip]
|
|
Storage>\ http
|
|
URL\ of\ http\ host\ to\ connect\ to
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Connect\ to\ example.com
|
|
\ \ \ \\\ "https://example.com"
|
|
url>\ https://beta.rclone.org
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
url\ =\ https://beta.rclone.org
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
Current\ remotes:
|
|
|
|
Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Type
|
|
====\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ====
|
|
remote\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ http
|
|
|
|
e)\ Edit\ existing\ remote
|
|
n)\ New\ remote
|
|
d)\ Delete\ remote
|
|
r)\ Rename\ remote
|
|
c)\ Copy\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
e/n/d/r/c/s/q>\ q
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[] and can now be used like this
|
|
.PP
|
|
See all the top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync the remote \f[C]directory\f[] to \f[C]/home/local/directory\f[],
|
|
deleting any excess files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ remote:directory\ /home/local/directory
|
|
\f[]
|
|
.fi
|
|
.SS Read only
|
|
.PP
|
|
This remote is read only \- you can't upload files to an HTTP server.
|
|
.SS Modified time
|
|
.PP
|
|
Most HTTP servers store time accurate to 1 second.
|
|
.SS Checksum
|
|
.PP
|
|
No checksums are stored.
|
|
.SS Usage without a config file
|
|
.PP
|
|
Since the http remote only has one config parameter it is easy to use
|
|
without a config file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ \-\-http\-url\ https://beta.rclone.org\ :http:
|
|
\f[]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to http (http Connection).
|
|
.SS \[en]http\-url
|
|
.PP
|
|
URL of http host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://user:pass\@example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com using a username and password
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to http (http Connection).
|
|
.SS \[en]http\-headers
|
|
.PP
|
|
Set HTTP headers for all transactions
|
|
.PP
|
|
Use this to set additional HTTP headers for all transactions
|
|
.PP
|
|
The input format is comma separated list of key,value pairs.
|
|
Standard CSV encoding (https://godoc.org/encoding/csv) may be used.
|
|
.PP
|
|
For example to set a Cookie use `Cookie,name=value', or
|
|
`\[lq]Cookie\[rq],\[lq]name=value\[rq]'.
|
|
.PP
|
|
You can set multiple headers, eg
|
|
`\[lq]Cookie\[rq],\[lq]name=value\[rq],\[lq]Authorization\[rq],\[lq]xxx\[rq]'.
|
|
.IP \[bu] 2
|
|
Config: headers
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_HEADERS
|
|
.IP \[bu] 2
|
|
Type: CommaSepList
|
|
.IP \[bu] 2
|
|
Default:
|
|
.SS \[en]http\-no\-slash
|
|
.PP
|
|
Set this if the site doesn't end directories with /
|
|
.PP
|
|
Use this if your target website does not use / on the end of
|
|
directories.
|
|
.PP
|
|
A / on the end of a path is how rclone normally tells the difference
|
|
between files and directories.
|
|
If this flag is set, then rclone will treat all files with
|
|
Content\-Type: text/html as directories and read URLs from them rather
|
|
than downloading them.
|
|
.PP
|
|
Note that this may cause rclone to confuse genuine HTML files with
|
|
directories.
|
|
.IP \[bu] 2
|
|
Config: no_slash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_NO_SLASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]http\-no\-head
|
|
.PP
|
|
Don't use HEAD requests to find file sizes in dir listing
|
|
.PP
|
|
If your site is being very slow to load then you can try this option.
|
|
Normally rclone does a HEAD request for each potential file in a
|
|
directory listing to:
|
|
.IP \[bu] 2
|
|
find its size
|
|
.IP \[bu] 2
|
|
check it really exists
|
|
.IP \[bu] 2
|
|
check to see if it is a directory
|
|
.PP
|
|
If you set this option, rclone will not do the HEAD request.
|
|
This will mean
|
|
.IP \[bu] 2
|
|
directory listings are much quicker
|
|
.IP \[bu] 2
|
|
rclone won't have the times or sizes of any files
|
|
.IP \[bu] 2
|
|
some files that don't exist may be in the listing
|
|
.IP \[bu] 2
|
|
Config: no_head
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_NO_HEAD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Hubic
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[] (or \f[C]remote:\f[]
|
|
for the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:container/path/to/dir\f[].
|
|
.PP
|
|
The initial setup for Hubic involves getting a token from Hubic which
|
|
you need to do in your browser.
|
|
\f[C]rclone\ config\f[] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
n/s>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Hubic
|
|
\ \ \ \\\ "hubic"
|
|
[snip]
|
|
Storage>\ hubic
|
|
Hubic\ Client\ Id\ \-\ leave\ blank\ normally.
|
|
client_id>
|
|
Hubic\ Client\ Secret\ \-\ leave\ blank\ normally.
|
|
client_secret>
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id\ =
|
|
client_secret\ =
|
|
token\ =\ {"access_token":"XXXXXX"}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Hubic.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List containers in the top level of your Hubic
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your Hubic
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Hubic directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you want the directory to be visible in the official \f[I]Hubic
|
|
browser\f[], you need to copy your files to the \f[C]default\f[]
|
|
directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:default/backup
|
|
\f[]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (/docs/#fast-list) for more details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Object\-Meta\-Mtime\f[] as floating point since the epoch
|
|
accurate to 1 ns.
|
|
.PP
|
|
This is a de facto standard (used in the official python\-swiftclient
|
|
amongst others) for storing the modification time for an object.
|
|
.PP
|
|
Note that Hubic wraps the Swift backend, so most of the properties of
|
|
are the same.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to hubic (Hubic).
|
|
.SS \[en]hubic\-client\-id
|
|
.PP
|
|
Hubic Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]hubic\-client\-secret
|
|
.PP
|
|
Hubic Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to hubic (Hubic).
|
|
.SS \[en]hubic\-chunk\-size
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
The default for this is 5GB which is its maximum value.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \[en]hubic\-no\-chunk
|
|
.PP
|
|
Don't chunk files during streaming upload.
|
|
.PP
|
|
When doing streaming uploads (eg using rcat or mount) setting this flag
|
|
will cause the swift backend to not upload chunked files.
|
|
.PP
|
|
This will limit the maximum upload size to 5GB.
|
|
However non chunked files are easier to deal with and have an MD5SUM.
|
|
.PP
|
|
Rclone will still chunk files bigger than chunk_size when doing normal
|
|
copy operations.
|
|
.IP \[bu] 2
|
|
Config: no_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_NO_CHUNK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Limitations
|
|
.PP
|
|
This uses the normal OpenStack Swift mechanism to refresh the Swift API
|
|
credentials and ignores the expires field returned by the Hubic API.
|
|
.PP
|
|
The Swift API doesn't return a correct MD5SUM for segmented files
|
|
(Dynamic or Static Large Objects) so rclone won't check or use the
|
|
MD5SUM for these.
|
|
.SS Jottacloud
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
To configure Jottacloud you will need to enter your username and
|
|
password and select a mountpoint.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ jotta
|
|
Type\ of\ storage\ to\ configure.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ JottaCloud
|
|
\ \ \ \\\ "jottacloud"
|
|
[snip]
|
|
Storage>\ jottacloud
|
|
**\ See\ help\ for\ jottacloud\ backend\ at:\ https://rclone.org/jottacloud/\ **
|
|
|
|
Edit\ advanced\ config?\ (y/n)
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ n
|
|
Remote\ config
|
|
|
|
Do\ you\ want\ to\ create\ a\ machine\ specific\ API\ key?
|
|
|
|
Rclone\ has\ it\[aq]s\ own\ Jottacloud\ API\ KEY\ which\ works\ fine\ as\ long\ as\ one\ only\ uses\ rclone\ on\ a\ single\ machine.\ When\ you\ want\ to\ use\ rclone\ with\ this\ account\ on\ more\ than\ one\ machine\ it\[aq]s\ recommended\ to\ create\ a\ machine\ specific\ API\ key.\ These\ keys\ can\ NOT\ be\ shared\ between\ machines.
|
|
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
Username>\ 0xC4KE\@gmail.com
|
|
Your\ Jottacloud\ password\ is\ only\ required\ during\ setup\ and\ will\ not\ be\ stored.
|
|
password:
|
|
|
|
Do\ you\ want\ to\ use\ a\ non\ standard\ device/mountpoint\ e.g.\ for\ accessing\ files\ uploaded\ using\ the\ official\ Jottacloud\ client?
|
|
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
Please\ select\ the\ device\ to\ use.\ Normally\ this\ will\ be\ Jotta
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ an\ existing\ value
|
|
\ 1\ >\ DESKTOP\-3H31129
|
|
\ 2\ >\ fla1
|
|
\ 3\ >\ Jotta
|
|
Devices>\ 3
|
|
Please\ select\ the\ mountpoint\ to\ user.\ Normally\ this\ will\ be\ Archive
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ an\ existing\ value
|
|
\ 1\ >\ Archive
|
|
\ 2\ >\ Shared
|
|
\ 3\ >\ Sync
|
|
Mountpoints>\ 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[jotta]
|
|
type\ =\ jottacloud
|
|
user\ =\ 0xC4KE\@gmail.com
|
|
client_id\ =\ .....
|
|
client_secret\ =\ ........
|
|
token\ =\ {........}
|
|
device\ =\ Jotta
|
|
mountpoint\ =\ Archive
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your Jottacloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your Jottacloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Jottacloud directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Devices and Mountpoints
|
|
.PP
|
|
The official Jottacloud client registers a device for each computer you
|
|
install it on and then creates a mountpoint for each folder you select
|
|
for Backup.
|
|
The web interface uses a special device called Jotta for the Archive,
|
|
Sync and Shared mountpoints.
|
|
In most cases you'll want to use the Jotta/Archive device/mounpoint
|
|
however if you want to access files uploaded by the official rclone
|
|
provides the option to select other devices and mountpoints during
|
|
config.
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (/docs/#fast-list) for more details.
|
|
.PP
|
|
Note that the implementation in Jottacloud always uses only a single API
|
|
request to get the entire list, so for large folders this could lead to
|
|
long wait time before the first results are shown.
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Jottacloud allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
Jottacloud supports MD5 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[] flag.
|
|
.PP
|
|
Note that Jottacloud requires the MD5 hash before upload so if the
|
|
source does not have an MD5 checksum then the file will be cached
|
|
temporarily on disk (wherever the \f[C]TMPDIR\f[] environment variable
|
|
points to) before it is uploaded.
|
|
Small files will be cached in memory \- see the
|
|
\f[C]\-\-jottacloud\-md5\-memory\-limit\f[] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
"
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
＂
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
＊
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
：
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
＜
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
＞
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
？
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
｜
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in XML strings.
|
|
.SS Deleting files
|
|
.PP
|
|
By default rclone will send all files to the trash when deleting files.
|
|
Due to a lack of API documentation emptying the trash is currently only
|
|
possible via the Jottacloud website.
|
|
If deleting permanently is required then use the
|
|
\f[C]\-\-jottacloud\-hard\-delete\f[] flag, or set the equivalent
|
|
environment variable.
|
|
.SS Versions
|
|
.PP
|
|
Jottacloud supports file versioning.
|
|
When rclone uploads a new version of a file it creates a new version of
|
|
it.
|
|
Currently rclone only supports retrieving the current version but older
|
|
versions can be accessed via the Jottacloud Website.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone\ about\ remote:\f[] command which will display your usage
|
|
limit (unless it is unlimited) and the current usage.
|
|
.SS Device IDs
|
|
.PP
|
|
Jottacloud requires each `device' to be registered.
|
|
Rclone brings such a registration to easily access your account but if
|
|
you want to use Jottacloud together with rclone on multiple machines you
|
|
NEED to create a seperate deviceID/deviceSecrect on each machine.
|
|
You will asked during setting up the remote.
|
|
Please be aware that this also means that copying the rclone config from
|
|
one machine to another does NOT work with Jottacloud accounts.
|
|
You have to create it on each machine.
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to jottacloud (JottaCloud).
|
|
.SS \[en]jottacloud\-md5\-memory\-limit
|
|
.PP
|
|
Files bigger than this will be cached on disk to calculate the MD5 if
|
|
required.
|
|
.IP \[bu] 2
|
|
Config: md5_memory_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \[en]jottacloud\-hard\-delete
|
|
.PP
|
|
Delete files permanently rather than putting them into the trash.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]jottacloud\-unlink
|
|
.PP
|
|
Remove existing public link to file/folder with link command rather than
|
|
creating.
|
|
Default is false, meaning link command will create or retrieve public
|
|
link.
|
|
.IP \[bu] 2
|
|
Config: unlink
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_UNLINK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]jottacloud\-upload\-resume\-limit
|
|
.PP
|
|
Files bigger than this can be resumed if the upload fail's.
|
|
.IP \[bu] 2
|
|
Config: upload_resume_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS Limitations
|
|
.PP
|
|
Note that Jottacloud is case insensitive so you can't have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are quite a few characters that can't be in Jottacloud file names.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a ?
|
|
in it will be mapped to ？ instead.
|
|
.PP
|
|
Jottacloud only supports filenames up to 255 characters in length.
|
|
.SS Troubleshooting
|
|
.PP
|
|
Jottacloud exhibits some inconsistent behaviours regarding deleted files
|
|
and folders which may cause Copy, Move and DirMove operations to
|
|
previously deleted paths to fail.
|
|
Emptying the trash should help in such cases.
|
|
.SS Koofr
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
The initial setup for Koofr involves creating an application password
|
|
for rclone.
|
|
You can do that by opening the Koofr web
|
|
application (https://app.koofr.net/app/admin/preferences/password),
|
|
giving the password a nice name like \f[C]rclone\f[] and clicking on
|
|
generate.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]koofr\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ koofr\
|
|
Type\ of\ storage\ to\ configure.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Koofr
|
|
\ \ \ \\\ "koofr"
|
|
[snip]
|
|
Storage>\ koofr
|
|
**\ See\ help\ for\ koofr\ backend\ at:\ https://rclone.org/koofr/\ **
|
|
|
|
Your\ Koofr\ user\ name
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
user>\ USER\@NAME
|
|
Your\ Koofr\ password\ for\ rclone\ (generate\ one\ at\ https://app.koofr.net/app/admin/preferences/password)
|
|
y)\ Yes\ type\ in\ my\ own\ password
|
|
g)\ Generate\ random\ password
|
|
y/g>\ y
|
|
Enter\ the\ password:
|
|
password:
|
|
Confirm\ the\ password:
|
|
password:
|
|
Edit\ advanced\ config?\ (y/n)
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ n
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[koofr]
|
|
type\ =\ koofr
|
|
baseurl\ =\ https://app.koofr.net
|
|
user\ =\ USER\@NAME
|
|
password\ =\ ***\ ENCRYPTED\ ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
You can choose to edit advanced config in order to enter your own
|
|
service URL if you use an on\-premise or white label Koofr instance, or
|
|
choose an alternative mount instead of your primary storage.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your Koofr
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ koofr:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your Koofr
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ koofr:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Koofr directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in XML strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to koofr (Koofr).
|
|
.SS \[en]koofr\-user
|
|
.PP
|
|
Your Koofr user name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]koofr\-password
|
|
.PP
|
|
Your Koofr password for rclone (generate one at
|
|
https://app.koofr.net/app/admin/preferences/password)
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to koofr (Koofr).
|
|
.SS \[en]koofr\-endpoint
|
|
.PP
|
|
The Koofr API endpoint to use
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]https://app.koofr.net\[rq]
|
|
.SS \[en]koofr\-mountid
|
|
.PP
|
|
Mount ID of the mount to use.
|
|
If omitted, the primary mount is used.
|
|
.IP \[bu] 2
|
|
Config: mountid
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_MOUNTID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]koofr\-setmtime
|
|
.PP
|
|
Does the backend support setting modification time.
|
|
Set this to false if you use a mount ID that points to a Dropbox or
|
|
Amazon Drive backend.
|
|
.IP \[bu] 2
|
|
Config: setmtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_SETMTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS Limitations
|
|
.PP
|
|
Note that Koofr is case insensitive so you can't have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.SS Mail.ru Cloud
|
|
.PP
|
|
Mail.ru Cloud (https://cloud.mail.ru/) is a cloud storage provided by a
|
|
Russian internet company Mail.Ru Group (https://mail.ru).
|
|
The official desktop client is Disk\-O: (https://disk-o.cloud/),
|
|
available only on Windows.
|
|
(Please note that official sites are in Russian)
|
|
.SS Features highlights
|
|
.IP \[bu] 2
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[]
|
|
.IP \[bu] 2
|
|
Files have a \f[C]last\ modified\ time\f[] property, directories don't
|
|
.IP \[bu] 2
|
|
Deleted files are by default moved to the trash
|
|
.IP \[bu] 2
|
|
Files and directories can be shared via public links
|
|
.IP \[bu] 2
|
|
Partial uploads or streaming are not supported, file size must be known
|
|
before upload
|
|
.IP \[bu] 2
|
|
Maximum file size is limited to 2G for a free acount, unlimited for paid
|
|
accounts
|
|
.IP \[bu] 2
|
|
Storage keeps hash for all files and performs transparent deduplication,
|
|
the hash algorithm is a modified SHA1
|
|
.IP \[bu] 2
|
|
If a particular file is already present in storage, one can quickly
|
|
submit file hash instead of long file upload (this optimization is
|
|
supported by rclone)
|
|
.SS Configuration
|
|
.PP
|
|
Here is an example of making a mailru configuration.
|
|
First create a Mail.ru Cloud account and choose a tariff, then run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Type\ of\ storage\ to\ configure.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Mail.ru\ Cloud
|
|
\ \ \ \\\ "mailru"
|
|
[snip]
|
|
Storage>\ mailru
|
|
User\ name\ (usually\ email)
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
user>\ username\@mail.ru
|
|
Password
|
|
y)\ Yes\ type\ in\ my\ own\ password
|
|
g)\ Generate\ random\ password
|
|
y/g>\ y
|
|
Enter\ the\ password:
|
|
password:
|
|
Confirm\ the\ password:
|
|
password:
|
|
Skip\ full\ upload\ if\ there\ is\ another\ file\ with\ same\ data\ hash.
|
|
This\ feature\ is\ called\ "speedup"\ or\ "put\ by\ hash".\ It\ is\ especially\ efficient
|
|
in\ case\ of\ generally\ available\ files\ like\ popular\ books,\ video\ or\ audio\ clips
|
|
[snip]
|
|
Enter\ a\ boolean\ value\ (true\ or\ false).\ Press\ Enter\ for\ the\ default\ ("true").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Enable
|
|
\ \ \ \\\ "true"
|
|
\ 2\ /\ Disable
|
|
\ \ \ \\\ "false"
|
|
speedup_enable>\ 1
|
|
Edit\ advanced\ config?\ (y/n)
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ n
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type\ =\ mailru
|
|
user\ =\ username\@mail.ru
|
|
pass\ =\ ***\ ENCRYPTED\ ***
|
|
speedup_enable\ =\ true
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Configuration of this backend does not require a local web browser.
|
|
You can use the configured backend as shown below:
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.SS Modified time
|
|
.PP
|
|
Files support a modification time attribute with up to 1 second
|
|
precision.
|
|
Directories do not have a modification time, which is shown as \[lq]Jan
|
|
1 1970\[rq].
|
|
.SS Hash checksums
|
|
.PP
|
|
Hash sums use a custom Mail.ru algorithm based on SHA1.
|
|
If file size is less than or equal to the SHA1 block size (20 bytes),
|
|
its hash is simply its data right\-padded with zero bytes.
|
|
Hash sum of a larger file is computed as a SHA1 sum of the file data
|
|
bytes concatenated with a decimal representation of the data length.
|
|
.SS Emptying Trash
|
|
.PP
|
|
Removing a file or directory actually moves it to the trash, which is
|
|
not visible to rclone but can be seen in a web browser.
|
|
The trashed file still occupies part of total quota.
|
|
If you wish to empty your trash and free some quota, you can use the
|
|
\f[C]rclone\ cleanup\ remote:\f[] command, which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone\ about\ remote:\f[] command which will display your usage
|
|
limit (quota) and the current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
"
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
＂
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
＊
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
：
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
＜
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
＞
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
？
|
|
T}
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
｜
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
File size limits depend on your account.
|
|
A single file size is limited by 2G for a free account and unlimited for
|
|
paid tariffs.
|
|
Please refer to the Mail.ru site for the total uploaded size limits.
|
|
.PP
|
|
Note that Mailru is case insensitive so you can't have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to mailru (Mail.ru Cloud).
|
|
.SS \[en]mailru\-user
|
|
.PP
|
|
User name (usually email)
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]mailru\-pass
|
|
.PP
|
|
Password
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]mailru\-speedup\-enable
|
|
.PP
|
|
Skip full upload if there is another file with same data hash.
|
|
This feature is called \[lq]speedup\[rq] or \[lq]put by hash\[rq].
|
|
It is especially efficient in case of generally available files like
|
|
popular books, video or audio clips, because files are searched by hash
|
|
in all accounts of all mailru users.
|
|
Please note that rclone may need local memory and disk space to
|
|
calculate content hash in advance and decide whether full upload is
|
|
required.
|
|
Also, if rclone does not know file size in advance (e.g.\ in case of
|
|
streaming or partial uploads), it will not even try this optimization.
|
|
.IP \[bu] 2
|
|
Config: speedup_enable
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to mailru (Mail.ru Cloud).
|
|
.SS \[en]mailru\-speedup\-file\-patterns
|
|
.PP
|
|
Comma separated list of file name patterns eligible for speedup (put by
|
|
hash).
|
|
Patterns are case insensitive and can contain '*' or `?' meta
|
|
characters.
|
|
.IP \[bu] 2
|
|
Config: speedup_file_patterns
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default:
|
|
\[lq]\f[I].mkv,\f[].avi,\f[I].mp4,\f[].mp3,\f[I].zip,\f[].gz,\f[I].rar,\f[].pdf\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
""
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty list completely disables speedup (put by hash).
|
|
.RE
|
|
.IP \[bu] 2
|
|
"*"
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All files will be attempted for speedup.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]\f[I].mkv,\f[].avi,\f[I].mp4,\f[].mp3\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only common audio/video files will be tried for put by hash.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]\f[I].zip,\f[].gz,\f[I].rar,\f[].pdf\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only common archives or PDF books will be tried for speedup.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-speedup\-max\-disk
|
|
.PP
|
|
This option allows you to disable speedup (put by hash) for large files
|
|
(because preliminary hashing can exhaust you RAM or disk space)
|
|
.IP \[bu] 2
|
|
Config: speedup_max_disk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 3G
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Completely disable speedup (put by hash).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]1G\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Files larger than 1Gb will be uploaded directly.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]3G\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Choose this option if you have less than 3Gb free on local disk.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-speedup\-max\-memory
|
|
.PP
|
|
Files larger than the size given below will always be hashed on disk.
|
|
.IP \[bu] 2
|
|
Config: speedup_max_memory
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 32M
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preliminary hashing will always be done in a temporary disk location.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]32M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not dedicate more than 32Mb RAM for preliminary hashing.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]256M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
You have at most 256Mb RAM free for hash calculations.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-check\-hash
|
|
.PP
|
|
What should copy do if file checksum is mismatched or invalid
|
|
.IP \[bu] 2
|
|
Config: check_hash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_CHECK_HASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fail with error.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore and continue.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-user\-agent
|
|
.PP
|
|
HTTP user agent used internally by client.
|
|
Defaults to \[lq]rclone/VERSION\[rq] or \[lq]\[en]user\-agent\[rq]
|
|
provided on command line.
|
|
.IP \[bu] 2
|
|
Config: user_agent
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_USER_AGENT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]mailru\-quirks
|
|
.PP
|
|
Comma separated list of internal maintenance flags.
|
|
This option must not be used by an ordinary user.
|
|
It is intended only to facilitate remote troubleshooting of backend
|
|
issues.
|
|
Strict meaning of flags is not documented and not guaranteed to persist
|
|
between releases.
|
|
Quirks will be removed when the backend grows stable.
|
|
Supported quirks: atomicmkdir binlist gzip insecure retry400
|
|
.IP \[bu] 2
|
|
Config: quirks
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_QUIRKS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Mega
|
|
.PP
|
|
Mega (https://mega.nz/) is a cloud storage and file hosting service
|
|
known for its security feature where all files are encrypted locally
|
|
before they are uploaded.
|
|
This prevents anyone (including employees of Mega) from accessing the
|
|
files without knowledge of the key used for encryption.
|
|
.PP
|
|
This is an rclone backend for Mega which supports the file transfer
|
|
features of Mega using the same client side encryption.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Mega
|
|
\ \ \ \\\ "mega"
|
|
[snip]
|
|
Storage>\ mega
|
|
User\ name
|
|
user>\ you\@example.com
|
|
Password.
|
|
y)\ Yes\ type\ in\ my\ own\ password
|
|
g)\ Generate\ random\ password
|
|
n)\ No\ leave\ this\ optional\ password\ blank
|
|
y/g/n>\ y
|
|
Enter\ the\ password:
|
|
password:
|
|
Confirm\ the\ password:
|
|
password:
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type\ =\ mega
|
|
user\ =\ you\@example.com
|
|
pass\ =\ ***\ ENCRYPTED\ ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\f[B]NOTE:\f[] The encryption keys need to have been already generated
|
|
after a regular login via the browser, otherwise attempting to use the
|
|
credentials in \f[C]rclone\f[] will fail.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your Mega
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your Mega
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Mega directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Mega does not support modification times or hashes yet.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
␀
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
／
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Duplicated files
|
|
.PP
|
|
Mega can have two files with exactly the same name and path (unlike a
|
|
normal file system).
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.PP
|
|
Use \f[C]rclone\ dedupe\f[] to fix duplicated files.
|
|
.SS Failure to log\-in
|
|
.PP
|
|
Mega remotes seem to get blocked (reject logins) under \[lq]heavy
|
|
use\[rq].
|
|
We haven't worked out the exact blocking rules but it seems to be
|
|
related to fast paced, sucessive rclone commands.
|
|
.PP
|
|
For example, executing this command 90 times in a row
|
|
\f[C]rclone\ link\ remote:file\f[] will cause the remote to become
|
|
\[lq]blocked\[rq].
|
|
This is not an abnormal situation, for example if you wish to get the
|
|
public links of a directory with hundred of files\&... After more or
|
|
less a week, the remote will remote accept rclone logins normally again.
|
|
.PP
|
|
You can mitigate this issue by mounting the remote it with
|
|
\f[C]rclone\ mount\f[].
|
|
This will log\-in when mounting and a log\-out when unmounting only.
|
|
You can also run \f[C]rclone\ rcd\f[] and then use \f[C]rclone\ rc\f[]
|
|
to run the commands over the API to avoid logging in each time.
|
|
.PP
|
|
Rclone does not currently close mega sessions (you can see them in the
|
|
web interface), however closing the sessions does not solve the issue.
|
|
.PP
|
|
If you space rclone commands by 3 seconds it will avoid blocking the
|
|
remote.
|
|
We haven't identified the exact blocking rules, so perhaps one could
|
|
execute the command 80 times without waiting and avoid blocking by
|
|
waiting 3 seconds, then continuing\&...
|
|
.PP
|
|
Note that this has been observed by trial and error and might not be set
|
|
in stone.
|
|
.PP
|
|
Other tools seem not to produce this blocking effect, as they use a
|
|
different working approach (state\-based, using sessionIDs instead of
|
|
log\-in) which isn't compatible with the current stateless rclone
|
|
approach.
|
|
.PP
|
|
Note that once blocked, the use of other tools (such as megacmd) is not
|
|
a sure workaround: following megacmd login times have been observed in
|
|
sucession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min.
|
|
Web access looks unaffected though.
|
|
.PP
|
|
Investigation is continuing in relation to workarounds based on
|
|
timeouts, pacers, retrials and tpslimits \- if you discover something
|
|
relevant, please post on the forum.
|
|
.PP
|
|
So, if rclone was working nicely and suddenly you are unable to log\-in
|
|
and you are sure the user and the password are correct, likely you have
|
|
got the remote blocked for a while.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to mega (Mega).
|
|
.SS \[en]mega\-user
|
|
.PP
|
|
User name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]mega\-pass
|
|
.PP
|
|
Password.
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to mega (Mega).
|
|
.SS \[en]mega\-debug
|
|
.PP
|
|
Output more debug from Mega.
|
|
.PP
|
|
If this flag is set (along with \-vv) it will print further debugging
|
|
information from the mega backend.
|
|
.IP \[bu] 2
|
|
Config: debug
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_DEBUG
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]mega\-hard\-delete
|
|
.PP
|
|
Delete files permanently rather than putting them into the trash.
|
|
.PP
|
|
Normally the mega backend will put all deletions into the trash rather
|
|
than permanently deleting them.
|
|
If you specify this then rclone will permanently delete objects instead.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Limitations
|
|
.PP
|
|
This backend uses the go\-mega go
|
|
library (https://github.com/t3rm1n4l/go-mega) which is an opensource go
|
|
library implementing the Mega API.
|
|
There doesn't appear to be any documentation for the mega protocol
|
|
beyond the mega C++ SDK (https://github.com/meganz/sdk) source code so
|
|
there are likely quite a few errors still remaining in this library.
|
|
.PP
|
|
Mega allows duplicate files which may confuse rclone.
|
|
.SS Microsoft Azure Blob Storage
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[] (or \f[C]remote:\f[]
|
|
for the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:container/path/to/dir\f[].
|
|
.PP
|
|
Here is an example of making a Microsoft Azure Blob Storage
|
|
configuration.
|
|
For a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Microsoft\ Azure\ Blob\ Storage
|
|
\ \ \ \\\ "azureblob"
|
|
[snip]
|
|
Storage>\ azureblob
|
|
Storage\ Account\ Name
|
|
account>\ account_name
|
|
Storage\ Account\ Key
|
|
key>\ base64encodedkey==
|
|
Endpoint\ for\ the\ service\ \-\ leave\ blank\ normally.
|
|
endpoint>\
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
account\ =\ account_name
|
|
key\ =\ base64encodedkey==
|
|
endpoint\ =\
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See all containers
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Make a new container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:container
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:container
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[] to the remote container, deleting
|
|
any excess files in the container.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:container
|
|
\f[]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (/docs/#fast-list) for more details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object with the
|
|
\f[C]mtime\f[] key.
|
|
It is stored using RFC3339 Format time with nanosecond precision.
|
|
The metadata is supplied during directory listings so there is no
|
|
overhead to using it.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
／
|
|
T}
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
．
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Hashes
|
|
.PP
|
|
MD5 hashes are stored with blobs.
|
|
However blobs that were uploaded in chunks only have an MD5 if the
|
|
source remote was capable of MD5 hashes, eg the local disk.
|
|
.SS Authenticating with Azure Blob Storage
|
|
.PP
|
|
Rclone has 3 ways of authenticating with Azure Blob Storage:
|
|
.SS Account and Key
|
|
.PP
|
|
This is the most straight forward and least flexible way.
|
|
Just fill in the \f[C]account\f[] and \f[C]key\f[] lines and leave the
|
|
rest blank.
|
|
.SS SAS URL
|
|
.PP
|
|
This can be an account level SAS URL or container level SAS URL
|
|
.PP
|
|
To use it leave \f[C]account\f[], \f[C]key\f[] blank and fill in
|
|
\f[C]sas_url\f[].
|
|
.PP
|
|
Account level SAS URL or container level SAS URL can be obtained from
|
|
Azure portal or Azure Storage Explorer.
|
|
To get a container level SAS URL right click on a container in the Azure
|
|
Blob explorer in the Azure portal.
|
|
.PP
|
|
If You use container level SAS URL, rclone operations are permitted only
|
|
on particular container, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ azureblob:container\ or\ rclone\ ls\ azureblob:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Since container name already exists in SAS URL, you can leave it empty
|
|
as well.
|
|
.PP
|
|
However these will not work
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ azureblob:
|
|
rclone\ ls\ azureblob:othercontainer
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This would be useful for temporarily allowing third parties access to a
|
|
single container or putting credentials into an untrusted environment.
|
|
.SS Multipart uploads
|
|
.PP
|
|
Rclone supports multipart uploads with Azure Blob storage.
|
|
Files bigger than 256MB will be uploaded using chunked upload by
|
|
default.
|
|
.PP
|
|
The files will be uploaded in parallel in 4MB chunks (by default).
|
|
Note that these chunks are buffered in memory and there may be up to
|
|
\f[C]\-\-transfers\f[] of them being uploaded at once.
|
|
.PP
|
|
Files can't be split into more than 50,000 chunks so by default, so the
|
|
largest file that can be uploaded with 4MB chunk size is 195GB.
|
|
Above this rclone will double the chunk size until it creates less than
|
|
50,000 chunks.
|
|
By default this will mean a maximum file size of 3.2TB can be uploaded.
|
|
This can be raised to 5TB using
|
|
\f[C]\-\-azureblob\-chunk\-size\ 100M\f[].
|
|
.PP
|
|
Note that rclone doesn't commit the block list until the end of the
|
|
upload which means that there is a limit of 9.5TB of multipart uploads
|
|
in progress as Azure won't allow more than that amount of uncommitted
|
|
blocks.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to azureblob (Microsoft Azure
|
|
Blob Storage).
|
|
.SS \[en]azureblob\-account
|
|
.PP
|
|
Storage Account Name (leave blank to use SAS URL or Emulator)
|
|
.IP \[bu] 2
|
|
Config: account
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ACCOUNT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]azureblob\-key
|
|
.PP
|
|
Storage Account Key (leave blank to use SAS URL or Emulator)
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]azureblob\-sas\-url
|
|
.PP
|
|
SAS URL for container level access only (leave blank if using
|
|
account/key or Emulator)
|
|
.IP \[bu] 2
|
|
Config: sas_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_SAS_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]azureblob\-use\-emulator
|
|
.PP
|
|
Uses local storage emulator if provided as `true' (leave blank if using
|
|
real azure storage endpoint)
|
|
.IP \[bu] 2
|
|
Config: use_emulator
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to azureblob (Microsoft Azure
|
|
Blob Storage).
|
|
.SS \[en]azureblob\-endpoint
|
|
.PP
|
|
Endpoint for the service Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]azureblob\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload (<= 256MB).
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 256M
|
|
.SS \[en]azureblob\-chunk\-size
|
|
.PP
|
|
Upload chunk size (<= 100MB).
|
|
.PP
|
|
Note that this is stored in memory and there may be up to
|
|
\[lq]\[en]transfers\[rq] chunks stored at once in memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4M
|
|
.SS \[en]azureblob\-list\-chunk
|
|
.PP
|
|
Size of blob list.
|
|
.PP
|
|
This sets the number of blobs requested in each listing chunk.
|
|
Default is the maximum, 5000.
|
|
\[lq]List blobs\[rq] requests are permitted 2 minutes per megabyte to
|
|
complete.
|
|
If an operation is taking longer than 2 minutes per megabyte on average,
|
|
it will time out (
|
|
source (https://docs.microsoft.com/en-us/rest/api/storageservices/setting-timeouts-for-blob-service-operations#exceptions-to-default-timeout-interval)
|
|
).
|
|
This can be used to limit the number of blobs items to return, to avoid
|
|
the time out.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 5000
|
|
.SS \[en]azureblob\-access\-tier
|
|
.PP
|
|
Access tier of blob: hot, cool or archive.
|
|
.PP
|
|
Archived blobs can be restored by setting access tier to hot or cool.
|
|
Leave blank if you intend to use default access tier, which is set at
|
|
account level
|
|
.PP
|
|
If there is no \[lq]access tier\[rq] specified, rclone doesn't apply any
|
|
tier.
|
|
rclone performs \[lq]Set Tier\[rq] operation on blobs while uploading,
|
|
if objects are not modified, specifying \[lq]access tier\[rq] to new one
|
|
will have no effect.
|
|
If blobs are in \[lq]archive tier\[rq] at remote, trying to perform data
|
|
transfer operations from remote will not be allowed.
|
|
User should first restore by tiering blob to \[lq]Hot\[rq] or
|
|
\[lq]Cool\[rq].
|
|
.IP \[bu] 2
|
|
Config: access_tier
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Limitations
|
|
.PP
|
|
MD5 sums are only uploaded with chunked files if the source has an MD5
|
|
sum.
|
|
This will always be the case for a local to azure copy.
|
|
.SS Azure Storage Emulator Support
|
|
.PP
|
|
You can test rlcone with storage emulator locally, to do this make sure
|
|
azure storage emulator installed locally and set up a new remote with
|
|
\f[C]rclone\ config\f[] follow instructions described in introduction,
|
|
set \f[C]use_emulator\f[] config as \f[C]true\f[], you do not need to
|
|
provide default account name or key if using emulator.
|
|
.SS Microsoft OneDrive
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
The initial setup for OneDrive involves getting a token from Microsoft
|
|
which you need to do in your browser.
|
|
\f[C]rclone\ config\f[] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
e)\ Edit\ existing\ remote
|
|
n)\ New\ remote
|
|
d)\ Delete\ remote
|
|
r)\ Rename\ remote
|
|
c)\ Copy\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
e/n/d/r/c/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Microsoft\ OneDrive
|
|
\ \ \ \\\ "onedrive"
|
|
[snip]
|
|
Storage>\ onedrive
|
|
Microsoft\ App\ Client\ Id
|
|
Leave\ blank\ normally.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
client_id>
|
|
Microsoft\ App\ Client\ Secret
|
|
Leave\ blank\ normally.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
client_secret>
|
|
Edit\ advanced\ config?\ (y/n)
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ n
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ an\ existing\ value
|
|
\ 1\ /\ OneDrive\ Personal\ or\ Business
|
|
\ \ \ \\\ "onedrive"
|
|
\ 2\ /\ Sharepoint\ site
|
|
\ \ \ \\\ "sharepoint"
|
|
\ 3\ /\ Type\ in\ driveID
|
|
\ \ \ \\\ "driveid"
|
|
\ 4\ /\ Type\ in\ SiteID
|
|
\ \ \ \\\ "siteid"
|
|
\ 5\ /\ Search\ a\ Sharepoint\ site
|
|
\ \ \ \\\ "search"
|
|
Your\ choice>\ 1
|
|
Found\ 1\ drives,\ please\ select\ the\ one\ you\ want\ to\ use:
|
|
0:\ OneDrive\ (business)\ id=b!Eqwertyuiopasdfghjklzxcvbnm\-7mnbvcxzlkjhgfdsapoiuytrewqk
|
|
Chose\ drive\ to\ use:>\ 0
|
|
Found\ drive\ \[aq]root\[aq]\ of\ type\ \[aq]business\[aq],\ URL:\ https://org\-my.sharepoint.com/personal/you/Documents
|
|
Is\ that\ okay?
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type\ =\ onedrive
|
|
token\ =\ {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018\-08\-26T22:39:52.486512262+08:00"}
|
|
drive_id\ =\ b!Eqwertyuiopasdfghjklzxcvbnm\-7mnbvcxzlkjhgfdsapoiuytrewqk
|
|
drive_type\ =\ business
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Microsoft.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your OneDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your OneDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an OneDrive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Getting your own Client ID and Key
|
|
.PP
|
|
rclone uses a pair of Client ID and Key shared by all rclone users when
|
|
performing requests by default.
|
|
If you are having problems with them (E.g., seeing a lot of throttling),
|
|
you can get your own Client ID and Key by following the steps below:
|
|
.IP "1." 3
|
|
Open
|
|
https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade,
|
|
then click \f[C]New\ registration\f[].
|
|
.IP "2." 3
|
|
Enter a name for your app, choose your account type, select \f[C]Web\f[]
|
|
in \f[C]Redirect\ URI\f[] Enter \f[C]http://localhost:53682/\f[] and
|
|
click Register.
|
|
Copy and keep the \f[C]Application\ (client)\ ID\f[] under the app name
|
|
for later use.
|
|
.IP "3." 3
|
|
Under \f[C]manage\f[] select \f[C]Certificates\ &\ secrets\f[], click
|
|
\f[C]New\ client\ secret\f[].
|
|
Copy and keep that secret for later use.
|
|
.IP "4." 3
|
|
Under \f[C]manage\f[] select \f[C]API\ permissions\f[], click
|
|
\f[C]Add\ a\ permission\f[] and select \f[C]Microsoft\ Graph\f[] then
|
|
select \f[C]delegated\ permissions\f[].
|
|
.IP "5." 3
|
|
Search and select the follwing permssions: \f[C]Files.Read\f[],
|
|
\f[C]Files.ReadWrite\f[], \f[C]Files.Read.All\f[],
|
|
\f[C]Files.ReadWrite.All\f[], \f[C]offline_access\f[],
|
|
\f[C]User.Read\f[].
|
|
Once selected click \f[C]Add\ permissions\f[] at the bottom.
|
|
.PP
|
|
Now the application is complete.
|
|
Run \f[C]rclone\ config\f[] to create or edit a OneDrive remote.
|
|
Supply the app ID and password as Client ID and Secret, respectively.
|
|
rclone will walk you through the remaining steps.
|
|
.SS Modification time and hashes
|
|
.PP
|
|
OneDrive allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
OneDrive personal supports SHA1 type hashes.
|
|
OneDrive for business and Sharepoint Server support
|
|
QuickXorHash (https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash).
|
|
.PP
|
|
For all types of OneDrive you can use the \f[C]\-\-checksum\f[] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
"
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
＂
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
＊
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
：
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
＜
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
＞
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
？
|
|
T}
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
｜
|
|
T}
|
|
T{
|
|
#
|
|
T}@T{
|
|
0x23
|
|
T}@T{
|
|
＃
|
|
T}
|
|
T{
|
|
%
|
|
T}@T{
|
|
0x25
|
|
T}@T{
|
|
％
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
␠
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
．
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not begin with the following characters.
|
|
These only get replaced if they are first character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
␠
|
|
T}
|
|
T{
|
|
~
|
|
T}@T{
|
|
0x7E
|
|
T}@T{
|
|
～
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Any files you delete with rclone will end up in the trash.
|
|
Microsoft doesn't provide an API to permanently delete files, nor to
|
|
empty the trash, so you will have to do that with one of Microsoft's
|
|
apps or via the OneDrive website.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to onedrive (Microsoft OneDrive).
|
|
.SS \[en]onedrive\-client\-id
|
|
.PP
|
|
Microsoft App Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]onedrive\-client\-secret
|
|
.PP
|
|
Microsoft App Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to onedrive (Microsoft OneDrive).
|
|
.SS \[en]onedrive\-chunk\-size
|
|
.PP
|
|
Chunk size to upload files with \- must be multiple of 320k (327,680
|
|
bytes).
|
|
.PP
|
|
Above this size files will be chunked \- must be multiple of 320k
|
|
(327,680 bytes).
|
|
Note that the chunks will be buffered into memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \[en]onedrive\-drive\-id
|
|
.PP
|
|
The ID of the drive to use
|
|
.IP \[bu] 2
|
|
Config: drive_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_DRIVE_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]onedrive\-drive\-type
|
|
.PP
|
|
The type of the drive ( personal | business | documentLibrary )
|
|
.IP \[bu] 2
|
|
Config: drive_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]onedrive\-expose\-onenote\-files
|
|
.PP
|
|
Set to make OneNote files show up in directory listings.
|
|
.PP
|
|
By default rclone will hide OneNote files in directory listings because
|
|
operations like \[lq]Open\[rq] and \[lq]Update\[rq] won't work on them.
|
|
But this behaviour may also prevent you from deleting them.
|
|
If you want to delete OneNote files or otherwise want them to show up in
|
|
directory listing, set this option.
|
|
.IP \[bu] 2
|
|
Config: expose_onenote_files
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Limitations
|
|
.SS Naming
|
|
.PP
|
|
Note that OneDrive is case insensitive so you can't have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are quite a few characters that can't be in OneDrive file names.
|
|
These can't occur on Windows platforms, but on non\-Windows platforms
|
|
they are common.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a \f[C]?\f[] in it will be mapped to
|
|
\f[C]？\f[] instead.
|
|
.SS File sizes
|
|
.PP
|
|
The largest allowed file sizes are 15GB for OneDrive for Business and
|
|
35GB for OneDrive Personal (Updated 4 Jan 2019).
|
|
.SS Path length
|
|
.PP
|
|
The entire path, including the file name, must contain fewer than 400
|
|
characters for OneDrive, OneDrive for Business and SharePoint Online.
|
|
If you are encrypting file and folder names with rclone, you may want to
|
|
pay attention to this limitation because the encrypted names are
|
|
typically longer than the original ones.
|
|
.SS Number of files
|
|
.PP
|
|
OneDrive seems to be OK with at least 50,000 files in a folder, but at
|
|
100,000 rclone will get errors listing the directory like
|
|
\f[C]couldn't\ list\ files:\ UnknownError:\f[].
|
|
See #2707 (https://github.com/rclone/rclone/issues/2707) for more info.
|
|
.PP
|
|
An official document about the limitations for different types of
|
|
OneDrive can be found
|
|
here (https://support.office.com/en-us/article/invalid-file-names-and-file-types-in-onedrive-onedrive-for-business-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
|
|
.SS Versioning issue
|
|
.PP
|
|
Every change in OneDrive causes the service to create a new version.
|
|
This counts against a users quota.
|
|
For example changing the modification time of a file creates a second
|
|
version, so the file is using twice the space.
|
|
.PP
|
|
The \f[C]copy\f[] is the only rclone command affected by this as we copy
|
|
the file and then afterwards set the modification time to match the
|
|
source file.
|
|
.PP
|
|
\f[B]Note\f[]: Starting October 2018, users will no longer be able to
|
|
disable versioning by default.
|
|
This is because Microsoft has brought an
|
|
update (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Updates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390)
|
|
to the mechanism.
|
|
To change this new default setting, a PowerShell command is required to
|
|
be run by a SharePoint admin.
|
|
If you are an admin, you can run these commands in PowerShell to change
|
|
that setting:
|
|
.IP "1." 3
|
|
\f[C]Install\-Module\ \-Name\ Microsoft.Online.SharePoint.PowerShell\f[]
|
|
(in case you haven't installed this already)
|
|
.IP "2." 3
|
|
\f[C]Import\-Module\ Microsoft.Online.SharePoint.PowerShell\ \-DisableNameChecking\f[]
|
|
.IP "3." 3
|
|
\f[C]Connect\-SPOService\ \-Url\ https://YOURSITE\-admin.sharepoint.com\ \-Credential\ YOU\@YOURSITE.COM\f[]
|
|
(replacing \f[C]YOURSITE\f[], \f[C]YOU\f[], \f[C]YOURSITE.COM\f[] with
|
|
the actual values; this will prompt for your credentials)
|
|
.IP "4." 3
|
|
\f[C]Set\-SPOTenant\ \-EnableMinimumVersionRequirement\ $False\f[]
|
|
.IP "5." 3
|
|
\f[C]Disconnect\-SPOService\f[] (to disconnect from the server)
|
|
.PP
|
|
\f[I]Below are the steps for normal users to disable versioning. If you
|
|
don't see the \[lq]No Versioning\[rq] option, make sure the above
|
|
requirements are met.\f[]
|
|
.PP
|
|
User Weropol (https://github.com/Weropol) has found a method to disable
|
|
versioning on OneDrive
|
|
.IP "1." 3
|
|
Open the settings menu by clicking on the gear symbol at the top of the
|
|
OneDrive Business page.
|
|
.IP "2." 3
|
|
Click Site settings.
|
|
.IP "3." 3
|
|
Once on the Site settings page, navigate to Site Administration > Site
|
|
libraries and lists.
|
|
.IP "4." 3
|
|
Click Customize \[lq]Documents\[rq].
|
|
.IP "5." 3
|
|
Click General Settings > Versioning Settings.
|
|
.IP "6." 3
|
|
Under Document Version History select the option No versioning.
|
|
Note: This will disable the creation of new file versions, but will not
|
|
remove any previous versions.
|
|
Your documents are safe.
|
|
.IP "7." 3
|
|
Apply the changes by clicking OK.
|
|
.IP "8." 3
|
|
Use rclone to upload or modify files.
|
|
(I also use the \[en]no\-update\-modtime flag)
|
|
.IP "9." 3
|
|
Restore the versioning settings after using rclone.
|
|
(Optional)
|
|
.SS Troubleshooting
|
|
.SS Unexpected file size/hash differences on Sharepoint
|
|
.PP
|
|
It is a
|
|
known (https://github.com/OneDrive/onedrive-api-docs/issues/935#issuecomment-441741631)
|
|
issue that Sharepoint (not OneDrive or OneDrive for Business) silently
|
|
modifies uploaded files, mainly Office files (.docx, .xlsx, etc.),
|
|
causing file size and hash checks to fail.
|
|
To use rclone with such affected files on Sharepoint, you may disable
|
|
these checks with the following command line arguments:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ignore\-checksum\ \-\-ignore\-size
|
|
\f[]
|
|
.fi
|
|
.SS Replacing/deleting existing files on Sharepoint gets \[lq]item not
|
|
found\[rq]
|
|
.PP
|
|
It is a
|
|
known (https://github.com/OneDrive/onedrive-api-docs/issues/1068) issue
|
|
that Sharepoint (not OneDrive or OneDrive for Business) may return
|
|
\[lq]item not found\[rq] errors when users try to replace or delete
|
|
uploaded files; this seems to mainly affect Office files (.docx, .xlsx,
|
|
etc.).
|
|
As a workaround, you may use the \f[C]\-\-backup\-dir\ <BACKUP_DIR>\f[]
|
|
command line argument so rclone moves the files to be replaced/deleted
|
|
into a given backup directory (instead of directly replacing/deleting
|
|
them).
|
|
For example, to instruct rclone to move the files into the directory
|
|
\f[C]rclone\-backup\-dir\f[] on backend \f[C]mysharepoint\f[], you may
|
|
use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-backup\-dir\ mysharepoint:rclone\-backup\-dir
|
|
\f[]
|
|
.fi
|
|
.SS access_denied (AADSTS65005)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Error:\ access_denied
|
|
Code:\ AADSTS65005
|
|
Description:\ Using\ application\ \[aq]rclone\[aq]\ is\ currently\ not\ supported\ for\ your\ organization\ [YOUR_ORGANIZATION]\ because\ it\ is\ in\ an\ unmanaged\ state.\ An\ administrator\ needs\ to\ claim\ ownership\ of\ the\ company\ by\ DNS\ validation\ of\ [YOUR_ORGANIZATION]\ before\ the\ application\ rclone\ can\ be\ provisioned.
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This means that rclone can't use the OneDrive for Business API with your
|
|
account.
|
|
You can't do much about it, maybe write an email to your admins.
|
|
.PP
|
|
However, there are other ways to interact with your OneDrive account.
|
|
Have a look at the webdav backend: https://rclone.org/webdav/#sharepoint
|
|
.SS invalid_grant (AADSTS50076)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Error:\ invalid_grant
|
|
Code:\ AADSTS50076
|
|
Description:\ Due\ to\ a\ configuration\ change\ made\ by\ your\ administrator,\ or\ because\ you\ moved\ to\ a\ new\ location,\ you\ must\ use\ multi\-factor\ authentication\ to\ access\ \[aq]...\[aq].
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you see the error above after enabling multi\-factor authentication
|
|
for your account, you can fix it by refreshing your OAuth refresh token.
|
|
To do that, run \f[C]rclone\ config\f[], and choose to edit your
|
|
OneDrive backend.
|
|
Then, you don't need to actually make any changes until you reach this
|
|
question: \f[C]Already\ have\ a\ token\ \-\ refresh?\f[].
|
|
For this question, answer \f[C]y\f[] and go through the process to
|
|
refresh your token, just like the first time the backend is configured.
|
|
After this, rclone should work again for this backend.
|
|
.SS OpenDrive
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n)\ New\ remote
|
|
d)\ Delete\ remote
|
|
q)\ Quit\ config
|
|
e/n/d/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ OpenDrive
|
|
\ \ \ \\\ "opendrive"
|
|
[snip]
|
|
Storage>\ opendrive
|
|
Username
|
|
username>
|
|
Password
|
|
y)\ Yes\ type\ in\ my\ own\ password
|
|
g)\ Generate\ random\ password
|
|
y/g>\ y
|
|
Enter\ the\ password:
|
|
password:
|
|
Confirm\ the\ password:
|
|
password:
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
username\ =
|
|
password\ =\ ***\ ENCRYPTED\ ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List directories in top level of your OpenDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your OpenDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an OpenDrive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Modified time and MD5SUMs
|
|
.PP
|
|
OpenDrive allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
␀
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
／
|
|
T}
|
|
T{
|
|
"
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
＂
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
＊
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
：
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
＜
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
＞
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
？
|
|
T}
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
｜
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not begin or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
␠
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
␉
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
␊
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
␋
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
␍
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to opendrive (OpenDrive).
|
|
.SS \[en]opendrive\-username
|
|
.PP
|
|
Username
|
|
.IP \[bu] 2
|
|
Config: username
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_USERNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]opendrive\-password
|
|
.PP
|
|
Password.
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Limitations
|
|
.PP
|
|
Note that OpenDrive is case insensitive so you can't have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are quite a few characters that can't be in OpenDrive file names.
|
|
These can't occur on Windows platforms, but on non\-Windows platforms
|
|
they are common.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a \f[C]?\f[] in it will be mapped to
|
|
\f[C]？\f[] instead.
|
|
.SS QingStor
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[] (or \f[C]remote:\f[] for
|
|
the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:bucket/path/to/dir\f[].
|
|
.PP
|
|
Here is an example of making an QingStor configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
r)\ Rename\ remote
|
|
c)\ Copy\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/r/c/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ QingStor\ Object\ Storage
|
|
\ \ \ \\\ "qingstor"
|
|
[snip]
|
|
Storage>\ qingstor
|
|
Get\ QingStor\ credentials\ from\ runtime.\ Only\ applies\ if\ access_key_id\ and\ secret_access_key\ is\ blank.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Enter\ QingStor\ credentials\ in\ the\ next\ step
|
|
\ \ \ \\\ "false"
|
|
\ 2\ /\ Get\ QingStor\ credentials\ from\ the\ environment\ (env\ vars\ or\ IAM)
|
|
\ \ \ \\\ "true"
|
|
env_auth>\ 1
|
|
QingStor\ Access\ Key\ ID\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
|
access_key_id>\ access_key
|
|
QingStor\ Secret\ Access\ Key\ (password)\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
|
secret_access_key>\ secret_key
|
|
Enter\ a\ endpoint\ URL\ to\ connection\ QingStor\ API.
|
|
Leave\ blank\ will\ use\ the\ default\ value\ "https://qingstor.com:443"
|
|
endpoint>
|
|
Zone\ connect\ to.\ Default\ is\ "pek3a".
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ \ \ /\ The\ Beijing\ (China)\ Three\ Zone
|
|
\ 1\ |\ Needs\ location\ constraint\ pek3a.
|
|
\ \ \ \\\ "pek3a"
|
|
\ \ \ /\ The\ Shanghai\ (China)\ First\ Zone
|
|
\ 2\ |\ Needs\ location\ constraint\ sh1a.
|
|
\ \ \ \\\ "sh1a"
|
|
zone>\ 1
|
|
Number\ of\ connnection\ retry.
|
|
Leave\ blank\ will\ use\ the\ default\ value\ "3".
|
|
connection_retries>
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
env_auth\ =\ false
|
|
access_key_id\ =\ access_key
|
|
secret_access_key\ =\ secret_key
|
|
endpoint\ =
|
|
zone\ =\ pek3a
|
|
connection_retries\ =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[] and can now be used like this
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:bucket
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:bucket
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:bucket
|
|
\f[]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (/docs/#fast-list) for more details.
|
|
.SS Multipart uploads
|
|
.PP
|
|
rclone supports multipart uploads with QingStor which means that it can
|
|
upload files bigger than 5GB.
|
|
Note that files uploaded with multipart upload don't have an MD5SUM.
|
|
.SS Buckets and Zone
|
|
.PP
|
|
With QingStor you can list buckets (\f[C]rclone\ lsd\f[]) using any
|
|
zone, but you can only access the content of a bucket from the zone it
|
|
was created in.
|
|
If you attempt to access a bucket from the wrong zone, you will get an
|
|
error,
|
|
\f[C]incorrect\ zone,\ the\ bucket\ is\ not\ in\ \[aq]XXX\[aq]\ zone\f[].
|
|
.SS Authentication
|
|
.PP
|
|
There are two ways to supply \f[C]rclone\f[] with a set of QingStor
|
|
credentials.
|
|
In order of precedence:
|
|
.IP \[bu] 2
|
|
Directly in the rclone configuration file (as configured by
|
|
\f[C]rclone\ config\f[])
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
set \f[C]access_key_id\f[] and \f[C]secret_access_key\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Runtime configuration:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
set \f[C]env_auth\f[] to \f[C]true\f[] in the config file
|
|
.IP \[bu] 2
|
|
Exporting the following environment variables before running
|
|
\f[C]rclone\f[]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access Key ID: \f[C]QS_ACCESS_KEY_ID\f[] or \f[C]QS_ACCESS_KEY\f[]
|
|
.IP \[bu] 2
|
|
Secret Access Key: \f[C]QS_SECRET_ACCESS_KEY\f[] or
|
|
\f[C]QS_SECRET_KEY\f[]
|
|
.RE
|
|
.RE
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The control characters 0x00\-0x1F and / are replaced as in the default
|
|
restricted characters set (/overview/#restricted-characters).
|
|
Note that 0x7F is not replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to qingstor (QingCloud Object
|
|
Storage).
|
|
.SS \[en]qingstor\-env\-auth
|
|
.PP
|
|
Get QingStor credentials from runtime.
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter QingStor credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get QingStor credentials from the environment (env vars or IAM)
|
|
.RE
|
|
.RE
|
|
.SS \[en]qingstor\-access\-key\-id
|
|
.PP
|
|
QingStor Access Key ID Leave blank for anonymous access or runtime
|
|
credentials.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]qingstor\-secret\-access\-key
|
|
.PP
|
|
QingStor Secret Access Key (password) Leave blank for anonymous access
|
|
or runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: secret_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]qingstor\-endpoint
|
|
.PP
|
|
Enter a endpoint URL to connection QingStor API.
|
|
Leave blank will use the default value
|
|
\[lq]https://qingstor.com:443\[rq]
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]qingstor\-zone
|
|
.PP
|
|
Zone to connect to.
|
|
Default is \[lq]pek3a\[rq].
|
|
.IP \[bu] 2
|
|
Config: zone
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ZONE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]pek3a\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Beijing (China) Three Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint pek3a.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sh1a\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Shanghai (China) First Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint sh1a.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]gd2a\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Guangdong (China) Second Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint gd2a.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to qingstor (QingCloud Object
|
|
Storage).
|
|
.SS \[en]qingstor\-connection\-retries
|
|
.PP
|
|
Number of connection retries.
|
|
.IP \[bu] 2
|
|
Config: connection_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 3
|
|
.SS \[en]qingstor\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of chunk_size.
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \[en]qingstor\-chunk\-size
|
|
.PP
|
|
Chunk size to use for uploading.
|
|
.PP
|
|
When uploading files larger than upload_cutoff they will be uploaded as
|
|
multipart uploads using this chunk size.
|
|
.PP
|
|
Note that \[lq]\[en]qingstor\-upload\-concurrency\[rq] chunks of this
|
|
size are buffered in memory per transfer.
|
|
.PP
|
|
If you are transferring large files over high speed links and you have
|
|
enough memory, then increasing this will speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4M
|
|
.SS \[en]qingstor\-upload\-concurrency
|
|
.PP
|
|
Concurrency for multipart uploads.
|
|
.PP
|
|
This is the number of chunks of the same file that are uploaded
|
|
concurrently.
|
|
.PP
|
|
NB if you set this to > 1 then the checksums of multpart uploads become
|
|
corrupted (the uploads themselves are not corrupted though).
|
|
.PP
|
|
If you are uploading small numbers of large file over high speed link
|
|
and these uploads do not fully utilize your bandwidth, then increasing
|
|
this may help to speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: upload_concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1
|
|
.SS Swift
|
|
.PP
|
|
Swift refers to Openstack Object
|
|
Storage (https://docs.openstack.org/swift/latest/).
|
|
Commercial implementations of that being:
|
|
.IP \[bu] 2
|
|
Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
|
|
.IP \[bu] 2
|
|
Memset Memstore (https://www.memset.com/cloud/storage/)
|
|
.IP \[bu] 2
|
|
OVH Object
|
|
Storage (https://www.ovh.co.uk/public-cloud/storage/object-storage/)
|
|
.IP \[bu] 2
|
|
Oracle Cloud Storage (https://cloud.oracle.com/storage-opc)
|
|
.IP \[bu] 2
|
|
IBM Bluemix Cloud ObjectStorage
|
|
Swift (https://console.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[] (or \f[C]remote:\f[]
|
|
for the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:container/path/to/dir\f[].
|
|
.PP
|
|
Here is an example of making a swift configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
|
\ \ \ \\\ "swift"
|
|
[snip]
|
|
Storage>\ swift
|
|
Get\ swift\ credentials\ from\ environment\ variables\ in\ standard\ OpenStack\ form.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Enter\ swift\ credentials\ in\ the\ next\ step
|
|
\ \ \ \\\ "false"
|
|
\ 2\ /\ Get\ swift\ credentials\ from\ environment\ vars.\ Leave\ other\ fields\ blank\ if\ using\ this.
|
|
\ \ \ \\\ "true"
|
|
env_auth>\ true
|
|
User\ name\ to\ log\ in\ (OS_USERNAME).
|
|
user>\
|
|
API\ key\ or\ password\ (OS_PASSWORD).
|
|
key>\
|
|
Authentication\ URL\ for\ server\ (OS_AUTH_URL).
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Rackspace\ US
|
|
\ \ \ \\\ "https://auth.api.rackspacecloud.com/v1.0"
|
|
\ 2\ /\ Rackspace\ UK
|
|
\ \ \ \\\ "https://lon.auth.api.rackspacecloud.com/v1.0"
|
|
\ 3\ /\ Rackspace\ v2
|
|
\ \ \ \\\ "https://identity.api.rackspacecloud.com/v2.0"
|
|
\ 4\ /\ Memset\ Memstore\ UK
|
|
\ \ \ \\\ "https://auth.storage.memset.com/v1.0"
|
|
\ 5\ /\ Memset\ Memstore\ UK\ v2
|
|
\ \ \ \\\ "https://auth.storage.memset.com/v2.0"
|
|
\ 6\ /\ OVH
|
|
\ \ \ \\\ "https://auth.cloud.ovh.net/v2.0"
|
|
auth>\
|
|
User\ ID\ to\ log\ in\ \-\ optional\ \-\ most\ swift\ systems\ use\ user\ and\ leave\ this\ blank\ (v3\ auth)\ (OS_USER_ID).
|
|
user_id>\
|
|
User\ domain\ \-\ optional\ (v3\ auth)\ (OS_USER_DOMAIN_NAME)
|
|
domain>\
|
|
Tenant\ name\ \-\ optional\ for\ v1\ auth,\ this\ or\ tenant_id\ required\ otherwise\ (OS_TENANT_NAME\ or\ OS_PROJECT_NAME)
|
|
tenant>\
|
|
Tenant\ ID\ \-\ optional\ for\ v1\ auth,\ this\ or\ tenant\ required\ otherwise\ (OS_TENANT_ID)
|
|
tenant_id>\
|
|
Tenant\ domain\ \-\ optional\ (v3\ auth)\ (OS_PROJECT_DOMAIN_NAME)
|
|
tenant_domain>\
|
|
Region\ name\ \-\ optional\ (OS_REGION_NAME)
|
|
region>\
|
|
Storage\ URL\ \-\ optional\ (OS_STORAGE_URL)
|
|
storage_url>\
|
|
Auth\ Token\ from\ alternate\ authentication\ \-\ optional\ (OS_AUTH_TOKEN)
|
|
auth_token>\
|
|
AuthVersion\ \-\ optional\ \-\ set\ to\ (1,2,3)\ if\ your\ auth\ URL\ has\ no\ version\ (ST_AUTH_VERSION)
|
|
auth_version>\
|
|
Endpoint\ type\ to\ choose\ from\ the\ service\ catalogue\ (OS_ENDPOINT_TYPE)
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Public\ (default,\ choose\ this\ if\ not\ sure)
|
|
\ \ \ \\\ "public"
|
|
\ 2\ /\ Internal\ (use\ internal\ service\ net)
|
|
\ \ \ \\\ "internal"
|
|
\ 3\ /\ Admin
|
|
\ \ \ \\\ "admin"
|
|
endpoint_type>\
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[test]
|
|
env_auth\ =\ true
|
|
user\ =\
|
|
key\ =\
|
|
auth\ =\
|
|
user_id\ =\
|
|
domain\ =\
|
|
tenant\ =\
|
|
tenant_id\ =\
|
|
tenant_domain\ =\
|
|
region\ =\
|
|
storage_url\ =\
|
|
auth_token\ =\
|
|
auth_version\ =\
|
|
endpoint_type\ =\
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[] and can now be used like this
|
|
.PP
|
|
See all containers
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Make a new container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:container
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:container
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[] to the remote container, deleting
|
|
any excess files in the container.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:container
|
|
\f[]
|
|
.fi
|
|
.SS Configuration from an OpenStack credentials file
|
|
.PP
|
|
An OpenStack credentials file typically looks something something like
|
|
this (without the comments)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export\ OS_AUTH_URL=https://a.provider.net/v2.0
|
|
export\ OS_TENANT_ID=ffffffffffffffffffffffffffffffff
|
|
export\ OS_TENANT_NAME="1234567890123456"
|
|
export\ OS_USERNAME="123abc567xy"
|
|
echo\ "Please\ enter\ your\ OpenStack\ Password:\ "
|
|
read\ \-sr\ OS_PASSWORD_INPUT
|
|
export\ OS_PASSWORD=$OS_PASSWORD_INPUT
|
|
export\ OS_REGION_NAME="SBG1"
|
|
if\ [\ \-z\ "$OS_REGION_NAME"\ ];\ then\ unset\ OS_REGION_NAME;\ fi
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The config file needs to look something like this where
|
|
\f[C]$OS_USERNAME\f[] represents the value of the \f[C]OS_USERNAME\f[]
|
|
variable \- \f[C]123abc567xy\f[] in the example above.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[remote]
|
|
type\ =\ swift
|
|
user\ =\ $OS_USERNAME
|
|
key\ =\ $OS_PASSWORD
|
|
auth\ =\ $OS_AUTH_URL
|
|
tenant\ =\ $OS_TENANT_NAME
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that you may (or may not) need to set \f[C]region\f[] too \- try
|
|
without first.
|
|
.SS Configuration from the environment
|
|
.PP
|
|
If you prefer you can configure rclone to use swift using a standard set
|
|
of OpenStack environment variables.
|
|
.PP
|
|
When you run through the config, make sure you choose \f[C]true\f[] for
|
|
\f[C]env_auth\f[] and leave everything else blank.
|
|
.PP
|
|
rclone will then set any empty config parameters from the environment
|
|
using standard OpenStack environment variables.
|
|
There is a list of the
|
|
variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvironment)
|
|
in the docs for the swift library.
|
|
.SS Using an alternate authentication method
|
|
.PP
|
|
If your OpenStack installation uses a non\-standard authentication
|
|
method that might not be yet supported by rclone or the underlying swift
|
|
library, you can authenticate externally (e.g.\ calling manually the
|
|
\f[C]openstack\f[] commands to get a token).
|
|
Then, you just need to pass the two configuration variables
|
|
\f[C]auth_token\f[] and \f[C]storage_url\f[].
|
|
If they are both provided, the other variables are ignored.
|
|
rclone will not try to authenticate but instead assume it is already
|
|
authenticated and use these two variables to access the OpenStack
|
|
installation.
|
|
.SS Using rclone without a config file
|
|
.PP
|
|
You can use rclone with swift without a config file, if desired, like
|
|
this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
source\ openstack\-credentials\-file
|
|
export\ RCLONE_CONFIG_MYREMOTE_TYPE=swift
|
|
export\ RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
|
|
rclone\ lsd\ myremote:
|
|
\f[]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (/docs/#fast-list) for more details.
|
|
.SS \[en]update and \[en]use\-server\-modtime
|
|
.PP
|
|
As noted below, the modified time is stored on metadata on the object.
|
|
It is used by default for all operations that require checking the time
|
|
a file was last updated.
|
|
It allows rclone to treat the remote more like a true filesystem, but it
|
|
is inefficient because it requires an extra API call to retrieve the
|
|
metadata.
|
|
.PP
|
|
For many operations, the time the object was last uploaded to the remote
|
|
is sufficient to determine if it is \[lq]dirty\[rq].
|
|
By using \f[C]\-\-update\f[] along with
|
|
\f[C]\-\-use\-server\-modtime\f[], you can avoid the extra API call and
|
|
simply upload files whose local modtime is newer than the time it was
|
|
last uploaded.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to swift (Openstack Swift
|
|
(Rackspace Cloud Files, Memset Memstore, OVH)).
|
|
.SS \[en]swift\-env\-auth
|
|
.PP
|
|
Get swift credentials from environment variables in standard OpenStack
|
|
form.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter swift credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get swift credentials from environment vars.
|
|
Leave other fields blank if using this.
|
|
.RE
|
|
.RE
|
|
.SS \[en]swift\-user
|
|
.PP
|
|
User name to log in (OS_USERNAME).
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-key
|
|
.PP
|
|
API key or password (OS_PASSWORD).
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-auth
|
|
.PP
|
|
Authentication URL for server (OS_AUTH_URL).
|
|
.IP \[bu] 2
|
|
Config: auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.api.rackspacecloud.com/v1.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace US
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://lon.auth.api.rackspacecloud.com/v1.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace UK
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://identity.api.rackspacecloud.com/v2.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace v2
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.storage.memset.com/v1.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memset Memstore UK
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.storage.memset.com/v2.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memset Memstore UK v2
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.cloud.ovh.net/v2.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH
|
|
.RE
|
|
.RE
|
|
.SS \[en]swift\-user\-id
|
|
.PP
|
|
User ID to log in \- optional \- most swift systems use user and leave
|
|
this blank (v3 auth) (OS_USER_ID).
|
|
.IP \[bu] 2
|
|
Config: user_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_USER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-domain
|
|
.PP
|
|
User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
.IP \[bu] 2
|
|
Config: domain
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_DOMAIN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-tenant
|
|
.PP
|
|
Tenant name \- optional for v1 auth, this or tenant_id required
|
|
otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
.IP \[bu] 2
|
|
Config: tenant
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-tenant\-id
|
|
.PP
|
|
Tenant ID \- optional for v1 auth, this or tenant required otherwise
|
|
(OS_TENANT_ID)
|
|
.IP \[bu] 2
|
|
Config: tenant_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-tenant\-domain
|
|
.PP
|
|
Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
.IP \[bu] 2
|
|
Config: tenant_domain
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT_DOMAIN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-region
|
|
.PP
|
|
Region name \- optional (OS_REGION_NAME)
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-storage\-url
|
|
.PP
|
|
Storage URL \- optional (OS_STORAGE_URL)
|
|
.IP \[bu] 2
|
|
Config: storage_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_STORAGE_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-auth\-token
|
|
.PP
|
|
Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
.IP \[bu] 2
|
|
Config: auth_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-application\-credential\-id
|
|
.PP
|
|
Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
|
|
.IP \[bu] 2
|
|
Config: application_credential_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-application\-credential\-name
|
|
.PP
|
|
Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
|
|
.IP \[bu] 2
|
|
Config: application_credential_name
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-application\-credential\-secret
|
|
.PP
|
|
Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
|
|
.IP \[bu] 2
|
|
Config: application_credential_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]swift\-auth\-version
|
|
.PP
|
|
AuthVersion \- optional \- set to (1,2,3) if your auth URL has no
|
|
version (ST_AUTH_VERSION)
|
|
.IP \[bu] 2
|
|
Config: auth_version
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH_VERSION
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 0
|
|
.SS \[en]swift\-endpoint\-type
|
|
.PP
|
|
Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
|
|
.IP \[bu] 2
|
|
Config: endpoint_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]public\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]public\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Public (default, choose this if not sure)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]internal\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Internal (use internal service net)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]admin\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Admin
|
|
.RE
|
|
.RE
|
|
.SS \[en]swift\-storage\-policy
|
|
.PP
|
|
The storage policy to use when creating a new container
|
|
.PP
|
|
This applies the specified storage policy when creating a new container.
|
|
The policy cannot be changed afterwards.
|
|
The allowed configuration values and their meaning depend on your Swift
|
|
storage provider.
|
|
.IP \[bu] 2
|
|
Config: storage_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_STORAGE_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
""
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]pcs\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH Public Cloud Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]pca\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH Public Cloud Archive
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to swift (Openstack Swift
|
|
(Rackspace Cloud Files, Memset Memstore, OVH)).
|
|
.SS \[en]swift\-chunk\-size
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
The default for this is 5GB which is its maximum value.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \[en]swift\-no\-chunk
|
|
.PP
|
|
Don't chunk files during streaming upload.
|
|
.PP
|
|
When doing streaming uploads (eg using rcat or mount) setting this flag
|
|
will cause the swift backend to not upload chunked files.
|
|
.PP
|
|
This will limit the maximum upload size to 5GB.
|
|
However non chunked files are easier to deal with and have an MD5SUM.
|
|
.PP
|
|
Rclone will still chunk files bigger than chunk_size when doing normal
|
|
copy operations.
|
|
.IP \[bu] 2
|
|
Config: no_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_NO_CHUNK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Object\-Meta\-Mtime\f[] as floating point since the epoch
|
|
accurate to 1 ns.
|
|
.PP
|
|
This is a defacto standard (used in the official python\-swiftclient
|
|
amongst others) for storing the modification time for an object.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
␀
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
／
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
The Swift API doesn't return a correct MD5SUM for segmented files
|
|
(Dynamic or Static Large Objects) so rclone won't check or use the
|
|
MD5SUM for these.
|
|
.SS Troubleshooting
|
|
.SS Rclone gives Failed to create file system for \[lq]remote:\[rq]: Bad
|
|
Request
|
|
.PP
|
|
Due to an oddity of the underlying swift library, it gives a \[lq]Bad
|
|
Request\[rq] error rather than a more sensible error when the
|
|
authentication fails for Swift.
|
|
.PP
|
|
So this most likely means your username / password is wrong.
|
|
You can investigate further with the \f[C]\-\-dump\-bodies\f[] flag.
|
|
.PP
|
|
This may also be caused by specifying the region when you shouldn't have
|
|
(eg OVH).
|
|
.SS Rclone gives Failed to create file system: Response didn't have
|
|
storage storage url and auth token
|
|
.PP
|
|
This is most likely caused by forgetting to specify your tenant when
|
|
setting up a swift remote.
|
|
.SS pCloud
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
The initial setup for pCloud involves getting a token from pCloud which
|
|
you need to do in your browser.
|
|
\f[C]rclone\ config\f[] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Pcloud
|
|
\ \ \ \\\ "pcloud"
|
|
[snip]
|
|
Storage>\ pcloud
|
|
Pcloud\ App\ Client\ Id\ \-\ leave\ blank\ normally.
|
|
client_id>\
|
|
Pcloud\ App\ Client\ Secret\ \-\ leave\ blank\ normally.
|
|
client_secret>\
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id\ =\
|
|
client_secret\ =\
|
|
token\ =\ {"access_token":"XXX","token_type":"bearer","expiry":"0001\-01\-01T00:00:00Z"}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from pCloud.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your pCloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your pCloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an pCloud directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
pCloud allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
In order to set a Modification time pCloud requires the object be
|
|
re\-uploaded.
|
|
.PP
|
|
pCloud supports MD5 and SHA1 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Deleted files will be moved to the trash.
|
|
Your subscription level will determine how long items stay in the trash.
|
|
\f[C]rclone\ cleanup\f[] can be used to empty the trash.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to pcloud (Pcloud).
|
|
.SS \[en]pcloud\-client\-id
|
|
.PP
|
|
Pcloud App Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]pcloud\-client\-secret
|
|
.PP
|
|
Pcloud App Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS premiumize.me
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
The initial setup for premiumize.me (https://premiumize.me/) involves
|
|
getting a token from premiumize.me which you need to do in your browser.
|
|
\f[C]rclone\ config\f[] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ premiumize.me
|
|
\ \ \ \\\ "premiumizeme"
|
|
[snip]
|
|
Storage>\ premiumizeme
|
|
**\ See\ help\ for\ premiumizeme\ backend\ at:\ https://rclone.org/premiumizeme/\ **
|
|
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type\ =\ premiumizeme
|
|
token\ =\ {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2029\-08\-07T18:44:15.548915378+01:00"}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from premiumize.me.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your premiumize.me
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your premiumize.me
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an premiumize.me directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
premiumize.me does not support modification times or hashes, therefore
|
|
syncing will default to \f[C]\-\-size\-only\f[] checking.
|
|
Note that using \f[C]\-\-update\f[] will work.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
T{
|
|
"
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
＂
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to premiumizeme (premiumize.me).
|
|
.SS \[en]premiumizeme\-api\-key
|
|
.PP
|
|
API Key.
|
|
.PP
|
|
This is not normally used \- use oauth instead.
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PREMIUMIZEME_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Limitations
|
|
.PP
|
|
Note that premiumize.me is case insensitive so you can't have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
premiumize.me file names can't have the \f[C]\\\f[] or \f[C]"\f[]
|
|
characters in.
|
|
rclone maps these to and from an identical looking unicode equivalents
|
|
\f[C]＼\f[] and \f[C]＂\f[]
|
|
.PP
|
|
premiumize.me only supports filenames up to 255 characters in length.
|
|
.SS put.io
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
put.io paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
The initial setup for put.io involves getting a token from put.io which
|
|
you need to do in your browser.
|
|
\f[C]rclone\ config\f[] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ putio
|
|
Type\ of\ storage\ to\ configure.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Put.io
|
|
\ \ \ \\\ "putio"
|
|
[snip]
|
|
Storage>\ putio
|
|
**\ See\ help\ for\ putio\ backend\ at:\ https://rclone.org/putio/\ **
|
|
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[putio]
|
|
type\ =\ putio
|
|
token\ =\ {"access_token":"XXXXXXXX","expiry":"0001\-01\-01T00:00:00Z"}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
Current\ remotes:
|
|
|
|
Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Type
|
|
====\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ====
|
|
putio\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ putio
|
|
|
|
e)\ Edit\ existing\ remote
|
|
n)\ New\ remote
|
|
d)\ Delete\ remote
|
|
r)\ Rename\ remote
|
|
c)\ Copy\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
e/n/d/r/c/s/q>\ q
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your put.io
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your put.io
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a put.io directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (/overview/#restricted-characters) the following characters are also
|
|
replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS SFTP
|
|
.PP
|
|
SFTP is the Secure (or SSH) File Transfer
|
|
Protocol (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
|
|
.PP
|
|
The SFTP backend can be used with a number of different providers:
|
|
.IP \[bu] 2
|
|
C14
|
|
.IP \[bu] 2
|
|
rsync.net
|
|
.PP
|
|
SFTP runs over SSH v2 and is installed as standard with most modern SSH
|
|
installations.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[].
|
|
If the path does not begin with a \f[C]/\f[] it is relative to the home
|
|
directory of the user.
|
|
An empty path \f[C]remote:\f[] refers to the user's home directory.
|
|
.PP
|
|
"Note that some SFTP servers will need the leading / \- Synology is a
|
|
good example of this.
|
|
rsync.net, on the other hand, requires users to OMIT the leading /.
|
|
.PP
|
|
Here is an example of making an SFTP configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ SSH/SFTP\ Connection
|
|
\ \ \ \\\ "sftp"
|
|
[snip]
|
|
Storage>\ sftp
|
|
SSH\ host\ to\ connect\ to
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Connect\ to\ example.com
|
|
\ \ \ \\\ "example.com"
|
|
host>\ example.com
|
|
SSH\ username,\ leave\ blank\ for\ current\ username,\ ncw
|
|
user>\ sftpuser
|
|
SSH\ port,\ leave\ blank\ to\ use\ default\ (22)
|
|
port>
|
|
SSH\ password,\ leave\ blank\ to\ use\ ssh\-agent.
|
|
y)\ Yes\ type\ in\ my\ own\ password
|
|
g)\ Generate\ random\ password
|
|
n)\ No\ leave\ this\ optional\ password\ blank
|
|
y/g/n>\ n
|
|
Path\ to\ unencrypted\ PEM\-encoded\ private\ key\ file,\ leave\ blank\ to\ use\ ssh\-agent.
|
|
key_file>
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
host\ =\ example.com
|
|
user\ =\ sftpuser
|
|
port\ =
|
|
pass\ =
|
|
key_file\ =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[] and can now be used like this:
|
|
.PP
|
|
See all directories in the home directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:path/to/directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:path/to/directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[] to the remote directory, deleting
|
|
any excess files in the directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.SS SSH Authentication
|
|
.PP
|
|
The SFTP remote supports three authentication methods:
|
|
.IP \[bu] 2
|
|
Password
|
|
.IP \[bu] 2
|
|
Key file
|
|
.IP \[bu] 2
|
|
ssh\-agent
|
|
.PP
|
|
Key files should be PEM\-encoded private key files.
|
|
For instance \f[C]/home/$USER/.ssh/id_rsa\f[].
|
|
Only unencrypted OpenSSH or PEM encrypted files are supported.
|
|
.PP
|
|
If you don't specify \f[C]pass\f[] or \f[C]key_file\f[] then rclone will
|
|
attempt to contact an ssh\-agent.
|
|
.PP
|
|
You can also specify \f[C]key_use_agent\f[] to force the usage of an
|
|
ssh\-agent.
|
|
In this case \f[C]key_file\f[] can also be specified to force the usage
|
|
of a specific key in the ssh\-agent.
|
|
.PP
|
|
Using an ssh\-agent is the only way to load encrypted OpenSSH keys at
|
|
the moment.
|
|
.PP
|
|
If you set the \f[C]\-\-sftp\-ask\-password\f[] option, rclone will
|
|
prompt for a password when needed and no password has been configured.
|
|
.SS ssh\-agent on macOS
|
|
.PP
|
|
Note that there seem to be various problems with using an ssh\-agent on
|
|
macOS due to recent changes in the OS.
|
|
The most effective work\-around seems to be to start an ssh\-agent in
|
|
each session, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
eval\ `ssh\-agent\ \-s`\ &&\ ssh\-add\ \-A
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
And then at the end of the session
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
eval\ `ssh\-agent\ \-k`
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
These commands can be used in scripts of course.
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are stored on the server to 1 second precision.
|
|
.PP
|
|
Modified times are used in syncing and are fully supported.
|
|
.PP
|
|
Some SFTP servers disable setting/modifying the file modification time
|
|
after upload (for example, certain configurations of ProFTPd with
|
|
mod_sftp).
|
|
If you are using one of these servers, you can set the option
|
|
\f[C]set_modtime\ =\ false\f[] in your RClone backend configuration to
|
|
disable this behaviour.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sftp (SSH/SFTP Connection).
|
|
.SS \[en]sftp\-host
|
|
.PP
|
|
SSH host to connect to
|
|
.IP \[bu] 2
|
|
Config: host
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_HOST
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.RE
|
|
.SS \[en]sftp\-user
|
|
.PP
|
|
SSH username, leave blank for current username, ncw
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-port
|
|
.PP
|
|
SSH port, leave blank to use default (22)
|
|
.IP \[bu] 2
|
|
Config: port
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PORT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-pass
|
|
.PP
|
|
SSH password, leave blank to use ssh\-agent.
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-key\-file
|
|
.PP
|
|
Path to PEM\-encoded private key file, leave blank or set
|
|
key\-use\-agent to use ssh\-agent.
|
|
.IP \[bu] 2
|
|
Config: key_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-key\-file\-pass
|
|
.PP
|
|
The passphrase to decrypt the PEM\-encoded private key file.
|
|
.PP
|
|
Only PEM encrypted key files (old OpenSSH format) are supported.
|
|
Encrypted keys in the new OpenSSH format can't be used.
|
|
.IP \[bu] 2
|
|
Config: key_file_pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_FILE_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-key\-use\-agent
|
|
.PP
|
|
When set forces the usage of the ssh\-agent.
|
|
.PP
|
|
When key\-file is also set, the \[lq].pub\[rq] file of the specified
|
|
key\-file is read and only the associated key is requested from the
|
|
ssh\-agent.
|
|
This allows to avoid
|
|
\f[C]Too\ many\ authentication\ failures\ for\ *username*\f[] errors
|
|
when the ssh\-agent contains many keys.
|
|
.IP \[bu] 2
|
|
Config: key_use_agent
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_USE_AGENT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]sftp\-use\-insecure\-cipher
|
|
.PP
|
|
Enable the use of insecure ciphers and key exchange methods.
|
|
.PP
|
|
This enables the use of the the following insecure ciphers and key
|
|
exchange methods:
|
|
.IP \[bu] 2
|
|
aes128\-cbc
|
|
.IP \[bu] 2
|
|
aes192\-cbc
|
|
.IP \[bu] 2
|
|
aes256\-cbc
|
|
.IP \[bu] 2
|
|
3des\-cbc
|
|
.IP \[bu] 2
|
|
diffie\-hellman\-group\-exchange\-sha256
|
|
.IP \[bu] 2
|
|
diffie\-hellman\-group\-exchange\-sha1
|
|
.PP
|
|
Those algorithms are insecure and may allow plaintext data to be
|
|
recovered by an attacker.
|
|
.IP \[bu] 2
|
|
Config: use_insecure_cipher
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use default Cipher list.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enables the use of the aes128\-cbc cipher and
|
|
diffie\-hellman\-group\-exchange\-sha256,
|
|
diffie\-hellman\-group\-exchange\-sha1 key exchange.
|
|
.RE
|
|
.RE
|
|
.SS \[en]sftp\-disable\-hashcheck
|
|
.PP
|
|
Disable the execution of SSH commands to determine if remote file
|
|
hashing is available.
|
|
Leave blank or set to false to enable hashing (recommended), set to true
|
|
to disable hashing.
|
|
.IP \[bu] 2
|
|
Config: disable_hashcheck
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sftp (SSH/SFTP Connection).
|
|
.SS \[en]sftp\-ask\-password
|
|
.PP
|
|
Allow asking for SFTP password when needed.
|
|
.PP
|
|
If this is set and no password is supplied then rclone will: \- ask for
|
|
a password \- not contact the ssh agent
|
|
.IP \[bu] 2
|
|
Config: ask_password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_ASK_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]sftp\-path\-override
|
|
.PP
|
|
Override path used by SSH connection.
|
|
.PP
|
|
This allows checksum calculation when SFTP and SSH paths are different.
|
|
This issue affects among others Synology NAS boxes.
|
|
.PP
|
|
Shared folders can be found in directories representing volumes
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:/directory\ \-\-ssh\-path\-override\ /volume2/directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Home directory can be found in a shared folder called \[lq]home\[rq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:/home/directory\ \-\-ssh\-path\-override\ /volume1/homes/USER/directory
|
|
\f[]
|
|
.fi
|
|
.IP \[bu] 2
|
|
Config: path_override
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PATH_OVERRIDE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-set\-modtime
|
|
.PP
|
|
Set the modified time on the remote if set.
|
|
.IP \[bu] 2
|
|
Config: set_modtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SET_MODTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \[en]sftp\-md5sum\-command
|
|
.PP
|
|
The command used to read md5 hashes.
|
|
Leave blank for autodetect.
|
|
.IP \[bu] 2
|
|
Config: md5sum_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_MD5SUM_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]sftp\-sha1sum\-command
|
|
.PP
|
|
The command used to read sha1 hashes.
|
|
Leave blank for autodetect.
|
|
.IP \[bu] 2
|
|
Config: sha1sum_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Limitations
|
|
.PP
|
|
SFTP supports checksums if the same login has shell access and
|
|
\f[C]md5sum\f[] or \f[C]sha1sum\f[] as well as \f[C]echo\f[] are in the
|
|
remote's PATH.
|
|
This remote checksumming (file hashing) is recommended and enabled by
|
|
default.
|
|
Disabling the checksumming may be required if you are connecting to SFTP
|
|
servers which are not under your control, and to which the execution of
|
|
remote commands is prohibited.
|
|
Set the configuration option \f[C]disable_hashcheck\f[] to \f[C]true\f[]
|
|
to disable checksumming.
|
|
.PP
|
|
SFTP also supports \f[C]about\f[] if the same login has shell access and
|
|
\f[C]df\f[] are in the remote's PATH.
|
|
\f[C]about\f[] will return the total space, free space, and used space
|
|
on the remote for the disk of the specified path on the remote or, if
|
|
not set, the disk of the root on the remote.
|
|
\f[C]about\f[] will fail if it does not have shell access or if
|
|
\f[C]df\f[] is not in the remote's PATH.
|
|
.PP
|
|
Note that some SFTP servers (eg Synology) the paths are different for
|
|
SSH and SFTP so the hashes can't be calculated properly.
|
|
For them using \f[C]disable_hashcheck\f[] is a good idea.
|
|
.PP
|
|
The only ssh agent supported under Windows is Putty's pageant.
|
|
.PP
|
|
The Go SSH library disables the use of the aes128\-cbc cipher by
|
|
default, due to security concerns.
|
|
This can be re\-enabled on a per\-connection basis by setting the
|
|
\f[C]use_insecure_cipher\f[] setting in the configuration file to
|
|
\f[C]true\f[].
|
|
Further details on the insecurity of this cipher can be found [in this
|
|
paper] (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
|
|
.PP
|
|
SFTP isn't supported under plan9 until this
|
|
issue (https://github.com/pkg/sftp/issues/156) is fixed.
|
|
.PP
|
|
Note that since SFTP isn't HTTP based the following flags don't work
|
|
with it: \f[C]\-\-dump\-headers\f[], \f[C]\-\-dump\-bodies\f[],
|
|
\f[C]\-\-dump\-auth\f[]
|
|
.PP
|
|
Note that \f[C]\-\-timeout\f[] isn't supported (but
|
|
\f[C]\-\-contimeout\f[] is).
|
|
.SS C14
|
|
.PP
|
|
C14 is supported through the SFTP backend.
|
|
.PP
|
|
See C14's
|
|
documentation (https://www.online.net/en/storage/c14-cold-storage)
|
|
.SS rsync.net
|
|
.PP
|
|
rsync.net is supported through the SFTP backend.
|
|
.PP
|
|
See rsync.net's documentation of rclone
|
|
examples (https://www.rsync.net/products/rclone.html).
|
|
.SS Union
|
|
.PP
|
|
The \f[C]union\f[] remote provides a unification similar to UnionFS
|
|
using other remotes.
|
|
.PP
|
|
Paths may be as deep as required or a local path, eg
|
|
\f[C]remote:directory/subdirectory\f[] or
|
|
\f[C]/directory/subdirectory\f[].
|
|
.PP
|
|
During the initial setup with \f[C]rclone\ config\f[] you will specify
|
|
the target remotes as a space separated list.
|
|
The target remotes can either be a local paths or other remotes.
|
|
.PP
|
|
The order of the remotes is important as it defines which remotes take
|
|
precedence over others if there are files with the same name in the same
|
|
logical path.
|
|
The last remote is the topmost remote and replaces files with the same
|
|
name from previous remotes.
|
|
.PP
|
|
Only the last remote is used to write to and delete from, all other
|
|
remotes are read\-only.
|
|
.PP
|
|
Subfolders can be used in target remote.
|
|
Assume a union remote named \f[C]backup\f[] with the remotes
|
|
\f[C]mydrive:private/backup\ mydrive2:/backup\f[].
|
|
Invoking \f[C]rclone\ mkdir\ backup:desktop\f[] is exactly the same as
|
|
invoking \f[C]rclone\ mkdir\ mydrive2:/backup/desktop\f[].
|
|
.PP
|
|
There will be no special handling of paths containing \f[C]\&..\f[]
|
|
segments.
|
|
Invoking \f[C]rclone\ mkdir\ backup:../desktop\f[] is exactly the same
|
|
as invoking \f[C]rclone\ mkdir\ mydrive2:/backup/../desktop\f[].
|
|
.PP
|
|
Here is an example of how to make a union called \f[C]remote\f[] for
|
|
local folders.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Union\ merges\ the\ contents\ of\ several\ remotes
|
|
\ \ \ \\\ "union"
|
|
[snip]
|
|
Storage>\ union
|
|
List\ of\ space\ separated\ remotes.
|
|
Can\ be\ \[aq]remotea:test/dir\ remoteb:\[aq],\ \[aq]"remotea:test/space\ dir"\ remoteb:\[aq],\ etc.
|
|
The\ last\ remote\ is\ used\ to\ write\ to.
|
|
Enter\ a\ string\ value.\ Press\ Enter\ for\ the\ default\ ("").
|
|
remotes>
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type\ =\ union
|
|
remotes\ =\ C:\\dir1\ C:\\dir2\ C:\\dir3
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
Current\ remotes:
|
|
|
|
Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Type
|
|
====\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ====
|
|
remote\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ union
|
|
|
|
e)\ Edit\ existing\ remote
|
|
n)\ New\ remote
|
|
d)\ Delete\ remote
|
|
r)\ Rename\ remote
|
|
c)\ Copy\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
e/n/d/r/c/s/q>\ q
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level in \f[C]C:\\dir1\f[], \f[C]C:\\dir2\f[]
|
|
and \f[C]C:\\dir3\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in \f[C]C:\\dir1\f[], \f[C]C:\\dir2\f[] and
|
|
\f[C]C:\\dir3\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Copy another local directory to the union directory called source, which
|
|
will be placed into \f[C]C:\\dir3\f[]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ C:\\source\ remote:source
|
|
\f[]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to union (Union merges the
|
|
contents of several remotes).
|
|
.SS \[en]union\-remotes
|
|
.PP
|
|
List of space separated remotes.
|
|
Can be `remotea:test/dir remoteb:', `\[lq]remotea:test/space dir\[rq]
|
|
remoteb:', etc.
|
|
The last remote is used to write to.
|
|
.IP \[bu] 2
|
|
Config: remotes
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_REMOTES
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS WebDAV
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.PP
|
|
To configure the WebDAV remote you will need to have a URL for it, and a
|
|
username and password.
|
|
If you know what kind of system you are connecting to then rclone can
|
|
enable extra features.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\ rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
q)\ Quit\ config
|
|
n/s/q>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Webdav
|
|
\ \ \ \\\ "webdav"
|
|
[snip]
|
|
Storage>\ webdav
|
|
URL\ of\ http\ host\ to\ connect\ to
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Connect\ to\ example.com
|
|
\ \ \ \\\ "https://example.com"
|
|
url>\ https://example.com/remote.php/webdav/
|
|
Name\ of\ the\ Webdav\ site/service/software\ you\ are\ using
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
\ 1\ /\ Nextcloud
|
|
\ \ \ \\\ "nextcloud"
|
|
\ 2\ /\ Owncloud
|
|
\ \ \ \\\ "owncloud"
|
|
\ 3\ /\ Sharepoint
|
|
\ \ \ \\\ "sharepoint"
|
|
\ 4\ /\ Other\ site/service\ or\ software
|
|
\ \ \ \\\ "other"
|
|
vendor>\ 1
|
|
User\ name
|
|
user>\ user
|
|
Password.
|
|
y)\ Yes\ type\ in\ my\ own\ password
|
|
g)\ Generate\ random\ password
|
|
n)\ No\ leave\ this\ optional\ password\ blank
|
|
y/g/n>\ y
|
|
Enter\ the\ password:
|
|
password:
|
|
Confirm\ the\ password:
|
|
password:
|
|
Bearer\ token\ instead\ of\ user/pass\ (eg\ a\ Macaroon)
|
|
bearer_token>
|
|
Remote\ config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type\ =\ webdav
|
|
url\ =\ https://example.com/remote.php/webdav/
|
|
vendor\ =\ nextcloud
|
|
user\ =\ user
|
|
pass\ =\ ***\ ENCRYPTED\ ***
|
|
bearer_token\ =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
List directories in top level of your WebDAV
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List all the files in your WebDAV
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an WebDAV directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ copy\ /home/source\ remote:backup
|
|
\f[]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Plain WebDAV does not support modified times.
|
|
However when used with Owncloud or Nextcloud rclone will support
|
|
modified times.
|
|
.PP
|
|
Likewise plain WebDAV does not support hashes, however when used with
|
|
Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes.
|
|
Depending on the exact version of Owncloud or Nextcloud hashes may
|
|
appear on all objects, or only on objects which had a hash uploaded with
|
|
them.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to webdav (Webdav).
|
|
.SS \[en]webdav\-url
|
|
.PP
|
|
URL of http host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.RE
|
|
.SS \[en]webdav\-vendor
|
|
.PP
|
|
Name of the Webdav site/service/software you are using
|
|
.IP \[bu] 2
|
|
Config: vendor
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_VENDOR
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]nextcloud\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nextcloud
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]owncloud\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owncloud
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sharepoint\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Sharepoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]other\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Other site/service or software
|
|
.RE
|
|
.RE
|
|
.SS \[en]webdav\-user
|
|
.PP
|
|
User name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]webdav\-pass
|
|
.PP
|
|
Password.
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]webdav\-bearer\-token
|
|
.PP
|
|
Bearer token instead of user/pass (eg a Macaroon)
|
|
.IP \[bu] 2
|
|
Config: bearer_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_BEARER_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to webdav (Webdav).
|
|
.SS \[en]webdav\-bearer\-token\-command
|
|
.PP
|
|
Command to run to get a bearer token
|
|
.IP \[bu] 2
|
|
Config: bearer_token_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Provider notes
|
|
.PP
|
|
See below for notes on specific providers.
|
|
.SS Owncloud
|
|
.PP
|
|
Click on the settings cog in the bottom right of the page and this will
|
|
show the WebDAV URL that rclone needs in the config step.
|
|
It will look something like
|
|
\f[C]https://example.com/remote.php/webdav/\f[].
|
|
.PP
|
|
Owncloud supports modified times using the \f[C]X\-OC\-Mtime\f[] header.
|
|
.SS Nextcloud
|
|
.PP
|
|
This is configured in an identical way to Owncloud.
|
|
Note that Nextcloud does not support streaming of files (\f[C]rcat\f[])
|
|
whereas Owncloud does.
|
|
This may be
|
|
fixed (https://github.com/nextcloud/nextcloud-snap/issues/365) in the
|
|
future.
|
|
.SS Sharepoint
|
|
.PP
|
|
Rclone can be used with Sharepoint provided by OneDrive for Business or
|
|
Office365 Education Accounts.
|
|
This feature is only needed for a few of these Accounts, mostly
|
|
Office365 Education ones.
|
|
These accounts are sometimes not verified by the domain owner
|
|
github#1975 (https://github.com/rclone/rclone/issues/1975)
|
|
.PP
|
|
This means that these accounts can't be added using the official API
|
|
(other Accounts should work with the \[lq]onedrive\[rq] option).
|
|
However, it is possible to access them using webdav.
|
|
.PP
|
|
To use a sharepoint remote with rclone, add it like this: First, you
|
|
need to get your remote's URL:
|
|
.IP \[bu] 2
|
|
Go here (https://onedrive.live.com/about/en-us/signin/) to open your
|
|
OneDrive or to sign in
|
|
.IP \[bu] 2
|
|
Now take a look at your address bar, the URL should look like this:
|
|
\f[C]https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/_layouts/15/onedrive.aspx\f[]
|
|
.PP
|
|
You'll only need this URL upto the email address.
|
|
After that, you'll most likely want to add \[lq]/Documents\[rq].
|
|
That subdirectory contains the actual data stored on your OneDrive.
|
|
.PP
|
|
Add the remote to rclone like this: Configure the \f[C]url\f[] as
|
|
\f[C]https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/Documents\f[]
|
|
and use your normal account email and password for \f[C]user\f[] and
|
|
\f[C]pass\f[].
|
|
If you have 2FA enabled, you have to generate an app password.
|
|
Set the \f[C]vendor\f[] to \f[C]sharepoint\f[].
|
|
.PP
|
|
Your config file should look like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[sharepoint]
|
|
type\ =\ webdav
|
|
url\ =\ https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/Documents
|
|
vendor\ =\ other
|
|
user\ =\ YourEmailAddress
|
|
pass\ =\ encryptedpassword
|
|
\f[]
|
|
.fi
|
|
.SS Required Flags for SharePoint
|
|
.PP
|
|
As SharePoint does some special things with uploaded documents, you
|
|
won't be able to use the documents size or the documents hash to compare
|
|
if a file has been changed since the upload / which file is newer.
|
|
.PP
|
|
For Rclone calls copying files (especially Office files such as .docx,
|
|
\&.xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should
|
|
append these flags to ensure Rclone uses the \[lq]Last Modified\[rq]
|
|
datetime property to compare your documents:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ignore\-size\ \-\-ignore\-checksum\ \-\-update
|
|
\f[]
|
|
.fi
|
|
.SS dCache
|
|
.PP
|
|
dCache is a storage system that supports many protocols and
|
|
authentication/authorisation schemes.
|
|
For WebDAV clients, it allows users to authenticate with username and
|
|
password (BASIC), X.509, Kerberos, and various bearer tokens, including
|
|
Macaroons (https://www.dcache.org/manuals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf)
|
|
and OpenID\-Connect (https://en.wikipedia.org/wiki/OpenID_Connect)
|
|
access tokens.
|
|
.PP
|
|
Configure as normal using the \f[C]other\f[] type.
|
|
Don't enter a username or password, instead enter your Macaroon as the
|
|
\f[C]bearer_token\f[].
|
|
.PP
|
|
The config will end up looking something like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dcache]
|
|
type\ =\ webdav
|
|
url\ =\ https://dcache...
|
|
vendor\ =\ other
|
|
user\ =
|
|
pass\ =
|
|
bearer_token\ =\ your\-macaroon
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
There is a
|
|
script (https://github.com/sara-nl/GridScripts/blob/master/get-macaroon)
|
|
that obtains a Macaroon from a dCache WebDAV endpoint, and creates an
|
|
rclone config file.
|
|
.PP
|
|
Macaroons may also be obtained from the dCacheView
|
|
web\-browser/JavaScript client that comes with dCache.
|
|
.SS OpenID\-Connect
|
|
.PP
|
|
dCache also supports authenticating with OpenID\-Connect access tokens.
|
|
OpenID\-Connect is a protocol (based on OAuth 2.0) that allows services
|
|
to identify users who have authenticated with some central service.
|
|
.PP
|
|
Support for OpenID\-Connect in rclone is currently achieved using
|
|
another software package called
|
|
oidc\-agent (https://github.com/indigo-dc/oidc-agent).
|
|
This is a command\-line tool that facilitates obtaining an access token.
|
|
Once installed and configured, an access token is obtained by running
|
|
the \f[C]oidc\-token\f[] command.
|
|
The following example shows a (shortened) access token obtained from the
|
|
\f[I]XDC\f[] OIDC Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
paul\@celebrimbor:~$\ oidc\-token\ XDC
|
|
eyJraWQ[...]QFXDt0
|
|
paul\@celebrimbor:~$
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\f[B]Note\f[] Before the \f[C]oidc\-token\f[] command will work, the
|
|
refresh token must be loaded into the oidc agent.
|
|
This is done with the \f[C]oidc\-add\f[] command (e.g.,
|
|
\f[C]oidc\-add\ XDC\f[]).
|
|
This is typically done once per login session.
|
|
Full details on this and how to register oidc\-agent with your OIDC
|
|
Provider are provided in the oidc\-agent
|
|
documentation (https://indigo-dc.gitbooks.io/oidc-agent/).
|
|
.PP
|
|
The rclone \f[C]bearer_token_command\f[] configuration option is used to
|
|
fetch the access token from oidc\-agent.
|
|
.PP
|
|
Configure as a normal WebDAV endpoint, using the `other' vendor, leaving
|
|
the username and password empty.
|
|
When prompted, choose to edit the advanced config and enter the command
|
|
to get a bearer token (e.g., \f[C]oidc\-agent\ XDC\f[]).
|
|
.PP
|
|
The following example config shows a WebDAV endpoint that uses
|
|
oidc\-agent to supply an access token from the \f[I]XDC\f[] OIDC
|
|
Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dcache]
|
|
type\ =\ webdav
|
|
url\ =\ https://dcache.example.org/
|
|
vendor\ =\ other
|
|
bearer_token_command\ =\ oidc\-token\ XDC
|
|
\f[]
|
|
.fi
|
|
.SS Yandex Disk
|
|
.PP
|
|
Yandex Disk (https://disk.yandex.com) is a cloud storage solution
|
|
created by Yandex (https://yandex.com).
|
|
.PP
|
|
Here is an example of making a yandex configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ config
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No\ remotes\ found\ \-\ make\ a\ new\ one
|
|
n)\ New\ remote
|
|
s)\ Set\ configuration\ password
|
|
n/s>\ n
|
|
name>\ remote
|
|
Type\ of\ storage\ to\ configure.
|
|
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
|
[snip]
|
|
XX\ /\ Yandex\ Disk
|
|
\ \ \ \\\ "yandex"
|
|
[snip]
|
|
Storage>\ yandex
|
|
Yandex\ Client\ Id\ \-\ leave\ blank\ normally.
|
|
client_id>
|
|
Yandex\ Client\ Secret\ \-\ leave\ blank\ normally.
|
|
client_secret>
|
|
Remote\ config
|
|
Use\ auto\ config?
|
|
\ *\ Say\ Y\ if\ not\ sure
|
|
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
|
y)\ Yes
|
|
n)\ No
|
|
y/n>\ y
|
|
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
|
Log\ in\ and\ authorize\ rclone\ for\ access
|
|
Waiting\ for\ code...
|
|
Got\ code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id\ =
|
|
client_secret\ =
|
|
token\ =\ {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","expiry":"2016\-12\-29T12:27:11.362788025Z"}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y)\ Yes\ this\ is\ OK
|
|
e)\ Edit\ this\ remote
|
|
d)\ Delete\ this\ remote
|
|
y/e/d>\ y
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Yandex Disk.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[] like this,
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ lsd\ remote:
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ mkdir\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ ls\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/local/directory\ remote:directory
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Yandex paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[].
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are supported and are stored accurate to 1 ns in custom
|
|
metadata called \f[C]rclone_modified\f[] in RFC3339 with nanoseconds
|
|
format.
|
|
.SS MD5 checksums
|
|
.PP
|
|
MD5 checksums are natively supported by Yandex Disk.
|
|
.SS Emptying Trash
|
|
.PP
|
|
If you wish to empty your trash you can use the
|
|
\f[C]rclone\ cleanup\ remote:\f[] command which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone\ about\ remote:\f[] command which will display your usage
|
|
limit (quota) and the current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The default restricted characters set (/overview/#restricted-characters)
|
|
are replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
When uploading very large files (bigger than about 5GB) you will need to
|
|
increase the \f[C]\-\-timeout\f[] parameter.
|
|
This is because Yandex pauses (perhaps to calculate the MD5SUM for the
|
|
entire file) before returning confirmation that the file has been
|
|
uploaded.
|
|
The default handling of timeouts in rclone is to assume a 5 minute pause
|
|
is an error and close the connection \- you'll see
|
|
\f[C]net/http:\ timeout\ awaiting\ response\ headers\f[] errors in the
|
|
logs if this is happening.
|
|
Setting the timeout to twice the max size of file in GB should be
|
|
enough, so if you want to upload a 30GB file set a timeout of
|
|
\f[C]2\ *\ 30\ =\ 60m\f[], that is \f[C]\-\-timeout\ 60m\f[].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to yandex (Yandex Disk).
|
|
.SS \[en]yandex\-client\-id
|
|
.PP
|
|
Yandex Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS \[en]yandex\-client\-secret
|
|
.PP
|
|
Yandex Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to yandex (Yandex Disk).
|
|
.SS \[en]yandex\-unlink
|
|
.PP
|
|
Remove existing public link to file/folder with link command rather than
|
|
creating.
|
|
Default is false, meaning link command will create or retrieve public
|
|
link.
|
|
.IP \[bu] 2
|
|
Config: unlink
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_UNLINK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Local Filesystem
|
|
.PP
|
|
Local paths are specified as normal filesystem paths, eg
|
|
\f[C]/path/to/wherever\f[], so
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ /home/source\ /tmp/destination
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Will sync \f[C]/home/source\f[] to \f[C]/tmp/destination\f[]
|
|
.PP
|
|
These can be configured into the config file for consistencies sake, but
|
|
it is probably easier not to.
|
|
.SS Modified time
|
|
.PP
|
|
Rclone reads and writes the modified time using an accuracy determined
|
|
by the OS.
|
|
Typically this is 1ns on Linux, 10 ns on Windows and 1 Second on OS X.
|
|
.SS Filenames
|
|
.PP
|
|
Filenames should be encoded in UTF\-8 on disk.
|
|
This is the normal case for Windows and OS X.
|
|
.PP
|
|
There is a bit more uncertainty in the Linux world, but new
|
|
distributions will have UTF\-8 encoded files names.
|
|
If you are using an old Linux filesystem with non UTF\-8 file names (eg
|
|
latin1) then you can use the \f[C]convmv\f[] tool to convert the
|
|
filesystem to UTF\-8.
|
|
This tool is available in most distributions' package managers.
|
|
.PP
|
|
If an invalid (non\-UTF8) filename is read, the invalid characters will
|
|
be replaced with a quoted representation of the invalid bytes.
|
|
The name \f[C]gro\\xdf\f[] will be transferred as \f[C]gro‛DF\f[].
|
|
\f[C]rclone\f[] will emit a debug message in this case (use \f[C]\-v\f[]
|
|
to see), eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Local\ file\ system\ at\ .:\ Replacing\ invalid\ UTF\-8\ characters\ in\ "gro\\xdf"
|
|
\f[]
|
|
.fi
|
|
.SS Restricted characters
|
|
.PP
|
|
On non Windows platforms the following characters are replaced when
|
|
handling file names.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
␀
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
／
|
|
T}
|
|
.TE
|
|
.PP
|
|
When running on Windows the following characters are replaced.
|
|
This list is based on the Windows file naming
|
|
conventions (https://docs.microsoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conventions).
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
␀
|
|
T}
|
|
T{
|
|
SOH
|
|
T}@T{
|
|
0x01
|
|
T}@T{
|
|
␁
|
|
T}
|
|
T{
|
|
STX
|
|
T}@T{
|
|
0x02
|
|
T}@T{
|
|
␂
|
|
T}
|
|
T{
|
|
ETX
|
|
T}@T{
|
|
0x03
|
|
T}@T{
|
|
␃
|
|
T}
|
|
T{
|
|
EOT
|
|
T}@T{
|
|
0x04
|
|
T}@T{
|
|
␄
|
|
T}
|
|
T{
|
|
ENQ
|
|
T}@T{
|
|
0x05
|
|
T}@T{
|
|
␅
|
|
T}
|
|
T{
|
|
ACK
|
|
T}@T{
|
|
0x06
|
|
T}@T{
|
|
␆
|
|
T}
|
|
T{
|
|
BEL
|
|
T}@T{
|
|
0x07
|
|
T}@T{
|
|
␇
|
|
T}
|
|
T{
|
|
BS
|
|
T}@T{
|
|
0x08
|
|
T}@T{
|
|
␈
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
␉
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
␊
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
␋
|
|
T}
|
|
T{
|
|
FF
|
|
T}@T{
|
|
0x0C
|
|
T}@T{
|
|
␌
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
␍
|
|
T}
|
|
T{
|
|
SO
|
|
T}@T{
|
|
0x0E
|
|
T}@T{
|
|
␎
|
|
T}
|
|
T{
|
|
SI
|
|
T}@T{
|
|
0x0F
|
|
T}@T{
|
|
␏
|
|
T}
|
|
T{
|
|
DLE
|
|
T}@T{
|
|
0x10
|
|
T}@T{
|
|
␐
|
|
T}
|
|
T{
|
|
DC1
|
|
T}@T{
|
|
0x11
|
|
T}@T{
|
|
␑
|
|
T}
|
|
T{
|
|
DC2
|
|
T}@T{
|
|
0x12
|
|
T}@T{
|
|
␒
|
|
T}
|
|
T{
|
|
DC3
|
|
T}@T{
|
|
0x13
|
|
T}@T{
|
|
␓
|
|
T}
|
|
T{
|
|
DC4
|
|
T}@T{
|
|
0x14
|
|
T}@T{
|
|
␔
|
|
T}
|
|
T{
|
|
NAK
|
|
T}@T{
|
|
0x15
|
|
T}@T{
|
|
␕
|
|
T}
|
|
T{
|
|
SYN
|
|
T}@T{
|
|
0x16
|
|
T}@T{
|
|
␖
|
|
T}
|
|
T{
|
|
ETB
|
|
T}@T{
|
|
0x17
|
|
T}@T{
|
|
␗
|
|
T}
|
|
T{
|
|
CAN
|
|
T}@T{
|
|
0x18
|
|
T}@T{
|
|
␘
|
|
T}
|
|
T{
|
|
EM
|
|
T}@T{
|
|
0x19
|
|
T}@T{
|
|
␙
|
|
T}
|
|
T{
|
|
SUB
|
|
T}@T{
|
|
0x1A
|
|
T}@T{
|
|
␚
|
|
T}
|
|
T{
|
|
ESC
|
|
T}@T{
|
|
0x1B
|
|
T}@T{
|
|
␛
|
|
T}
|
|
T{
|
|
FS
|
|
T}@T{
|
|
0x1C
|
|
T}@T{
|
|
␜
|
|
T}
|
|
T{
|
|
GS
|
|
T}@T{
|
|
0x1D
|
|
T}@T{
|
|
␝
|
|
T}
|
|
T{
|
|
RS
|
|
T}@T{
|
|
0x1E
|
|
T}@T{
|
|
␞
|
|
T}
|
|
T{
|
|
US
|
|
T}@T{
|
|
0x1F
|
|
T}@T{
|
|
␟
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
／
|
|
T}
|
|
T{
|
|
"
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
＂
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
＊
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
：
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
＜
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
＞
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
？
|
|
T}
|
|
T{
|
|
\\
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
＼
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
｜
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names on Windows can also not end with the following characters.
|
|
These only get replaced if they are last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
␠
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
．
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be replaced (/overview/#invalid-utf8), as
|
|
they can't be converted to UTF\-16.
|
|
.SS Long paths on Windows
|
|
.PP
|
|
Rclone handles long paths automatically, by converting all paths to long
|
|
UNC
|
|
paths (https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath)
|
|
which allows paths up to 32,767 characters.
|
|
.PP
|
|
This is why you will see that your paths, for instance
|
|
\f[C]c:\\files\f[] is converted to the UNC path
|
|
\f[C]\\\\?\\c:\\files\f[] in the output, and \f[C]\\\\server\\share\f[]
|
|
is converted to \f[C]\\\\?\\UNC\\server\\share\f[].
|
|
.PP
|
|
However, in rare cases this may cause problems with buggy file system
|
|
drivers like EncFS (https://github.com/rclone/rclone/issues/261).
|
|
To disable UNC conversion globally, add this to your
|
|
\f[C]\&.rclone.conf\f[] file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[local]
|
|
nounc\ =\ true
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you want to selectively disable UNC, you can add it to a separate
|
|
entry like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[nounc]
|
|
type\ =\ local
|
|
nounc\ =\ true
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
And use rclone like this:
|
|
.PP
|
|
\f[C]rclone\ copy\ c:\\src\ nounc:z:\\dst\f[]
|
|
.PP
|
|
This will use UNC paths on \f[C]c:\\src\f[] but not on \f[C]z:\\dst\f[].
|
|
Of course this will cause problems if the absolute path length of a file
|
|
exceeds 258 characters on z, so only use this option if you have to.
|
|
.SS Symlinks / Junction points
|
|
.PP
|
|
Normally rclone will ignore symlinks or junction points (which behave
|
|
like symlinks under Windows).
|
|
.PP
|
|
If you supply \f[C]\-\-copy\-links\f[] or \f[C]\-L\f[] then rclone will
|
|
follow the symlink and copy the pointed to file or directory.
|
|
Note that this flag is incompatible with \f[C]\-links\f[] /
|
|
\f[C]\-l\f[].
|
|
.PP
|
|
This flag applies to all commands.
|
|
.PP
|
|
For example, supposing you have a directory structure like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ tree\ /tmp/a
|
|
/tmp/a
|
|
├──\ b\ \->\ ../b
|
|
├──\ expected\ \->\ ../expected
|
|
├──\ one
|
|
└──\ two
|
|
\ \ \ \ └──\ three
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Then you can see the difference with and without the flag like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ ls\ /tmp/a
|
|
\ \ \ \ \ \ \ \ 6\ one
|
|
\ \ \ \ \ \ \ \ 6\ two/three
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
and
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ \-L\ ls\ /tmp/a
|
|
\ \ \ \ \ 4174\ expected
|
|
\ \ \ \ \ \ \ \ 6\ one
|
|
\ \ \ \ \ \ \ \ 6\ two/three
|
|
\ \ \ \ \ \ \ \ 6\ b/two
|
|
\ \ \ \ \ \ \ \ 6\ b/one
|
|
\f[]
|
|
.fi
|
|
.SS \[en]links, \-l
|
|
.PP
|
|
Normally rclone will ignore symlinks or junction points (which behave
|
|
like symlinks under Windows).
|
|
.PP
|
|
If you supply this flag then rclone will copy symbolic links from the
|
|
local storage, and store them as text files, with a `.rclonelink' suffix
|
|
in the remote storage.
|
|
.PP
|
|
The text file will contain the target of the symbolic link (see
|
|
example).
|
|
.PP
|
|
This flag applies to all commands.
|
|
.PP
|
|
For example, supposing you have a directory structure like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ tree\ /tmp/a
|
|
/tmp/a
|
|
├──\ file1\ \->\ ./file4
|
|
└──\ file2\ \->\ /home/user/file3
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Copying the entire directory with `\-l'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ copyto\ \-l\ /tmp/a/file1\ remote:/tmp/a/
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The remote files are created with a `.rclonelink' suffix
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ ls\ remote:/tmp/a
|
|
\ \ \ \ \ \ \ 5\ file1.rclonelink
|
|
\ \ \ \ \ \ 14\ file2.rclonelink
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The remote files will contain the target of the symbolic links
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ cat\ remote:/tmp/a/file1.rclonelink
|
|
\&./file4
|
|
|
|
$\ rclone\ cat\ remote:/tmp/a/file2.rclonelink
|
|
/home/user/file3
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Copying them back with `\-l'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ copyto\ \-l\ remote:/tmp/a/\ /tmp/b/
|
|
|
|
$\ tree\ /tmp/b
|
|
/tmp/b
|
|
├──\ file1\ \->\ ./file4
|
|
└──\ file2\ \->\ /home/user/file3
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
However, if copied back without `\-l'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ copyto\ remote:/tmp/a/\ /tmp/b/
|
|
|
|
$\ tree\ /tmp/b
|
|
/tmp/b
|
|
├──\ file1.rclonelink
|
|
└──\ file2.rclonelink
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that this flag is incompatible with \f[C]\-copy\-links\f[] /
|
|
\f[C]\-L\f[].
|
|
.SS Restricting filesystems with \[en]one\-file\-system
|
|
.PP
|
|
Normally rclone will recurse through filesystems as mounted.
|
|
.PP
|
|
However if you set \f[C]\-\-one\-file\-system\f[] or \f[C]\-x\f[] this
|
|
tells rclone to stay in the filesystem specified by the root and not to
|
|
recurse into different file systems.
|
|
.PP
|
|
For example if you have a directory hierarchy like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
root
|
|
├──\ disk1\ \ \ \ \ \-\ disk1\ mounted\ on\ the\ root
|
|
│\ \ \ └──\ file3\ \-\ stored\ on\ disk1
|
|
├──\ disk2\ \ \ \ \ \-\ disk2\ mounted\ on\ the\ root
|
|
│\ \ \ └──\ file4\ \-\ stored\ on\ disk12
|
|
├──\ file1\ \ \ \ \ \-\ stored\ on\ the\ root\ disk
|
|
└──\ file2\ \ \ \ \ \-\ stored\ on\ the\ root\ disk
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Using \f[C]rclone\ \-\-one\-file\-system\ copy\ root\ remote:\f[] will
|
|
only copy \f[C]file1\f[] and \f[C]file2\f[].
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ \-q\ \-\-one\-file\-system\ ls\ root
|
|
\ \ \ \ \ \ \ \ 0\ file1
|
|
\ \ \ \ \ \ \ \ 0\ file2
|
|
\f[]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$\ rclone\ \-q\ ls\ root
|
|
\ \ \ \ \ \ \ \ 0\ disk1/file3
|
|
\ \ \ \ \ \ \ \ 0\ disk2/file4
|
|
\ \ \ \ \ \ \ \ 0\ file1
|
|
\ \ \ \ \ \ \ \ 0\ file2
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
\f[B]NB\f[] Rclone (like most unix tools such as \f[C]du\f[],
|
|
\f[C]rsync\f[] and \f[C]tar\f[]) treats a bind mount to the same device
|
|
as being on the same filesystem.
|
|
.PP
|
|
\f[B]NB\f[] This flag is only available on Unix based systems.
|
|
On systems where it isn't supported (eg Windows) it will be ignored.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to local (Local Disk).
|
|
.SS \[en]local\-nounc
|
|
.PP
|
|
Disable UNC (long path names) conversion on Windows
|
|
.IP \[bu] 2
|
|
Config: nounc
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NOUNC
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: ""
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disables long file names
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to local (Local Disk).
|
|
.SS \[en]copy\-links / \-L
|
|
.PP
|
|
Follow symlinks and copy the pointed to item.
|
|
.IP \[bu] 2
|
|
Config: copy_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_COPY_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]links / \-l
|
|
.PP
|
|
Translate symlinks to/from regular files with a `.rclonelink' extension
|
|
.IP \[bu] 2
|
|
Config: links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]skip\-links
|
|
.PP
|
|
Don't warn about skipped symlinks.
|
|
This flag disables warning messages on skipped symlinks or junction
|
|
points, as you explicitly acknowledge that they should be skipped.
|
|
.IP \[bu] 2
|
|
Config: skip_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_SKIP_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-no\-unicode\-normalization
|
|
.PP
|
|
Don't apply unicode normalization to paths and filenames (Deprecated)
|
|
.PP
|
|
This flag is deprecated now.
|
|
Rclone no longer normalizes unicode file names, but it compares them
|
|
with unicode normalization in the sync routine instead.
|
|
.IP \[bu] 2
|
|
Config: no_unicode_normalization
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_UNICODE_NORMALIZATION
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-no\-check\-updated
|
|
.PP
|
|
Don't check to see if the files change during upload
|
|
.PP
|
|
Normally rclone checks the size and modification time of files as they
|
|
are being uploaded and aborts with a message which starts \[lq]can't
|
|
copy \- source file is being updated\[rq] if the file changes during
|
|
upload.
|
|
.PP
|
|
However on some file systems this modification time check may fail (eg
|
|
Glusterfs #2206 (https://github.com/rclone/rclone/issues/2206)) so this
|
|
check can be disabled with this flag.
|
|
.IP \[bu] 2
|
|
Config: no_check_updated
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]one\-file\-system / \-x
|
|
.PP
|
|
Don't cross filesystem boundaries (unix/macOS only).
|
|
.IP \[bu] 2
|
|
Config: one_file_system
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-case\-sensitive
|
|
.PP
|
|
Force the filesystem to report itself as case sensitive.
|
|
.PP
|
|
Normally the local backend declares itself as case insensitive on
|
|
Windows/macOS and case sensitive for everything else.
|
|
Use this flag to override the default choice.
|
|
.IP \[bu] 2
|
|
Config: case_sensitive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_CASE_SENSITIVE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-case\-insensitive
|
|
.PP
|
|
Force the filesystem to report itself as case insensitive
|
|
.PP
|
|
Normally the local backend declares itself as case insensitive on
|
|
Windows/macOS and case sensitive for everything else.
|
|
Use this flag to override the default choice.
|
|
.IP \[bu] 2
|
|
Config: case_insensitive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_CASE_INSENSITIVE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SH Changelog
|
|
.SS v1.50.0 \- 2019\-10\-26
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Citrix Sharefile (/sharefile) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Chunker (/chunker) \- an overlay backend to split files into smaller
|
|
parts (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud (/mailru) (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
encodings (Fabian Möller & Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All backends now use file name encoding to ensure any file name can be
|
|
written to any backend.
|
|
.IP \[bu] 2
|
|
See the restricted file name docs (/overview/#restricted-filenames) for
|
|
more info and the local backend docs (/local/#filenames).
|
|
.IP \[bu] 2
|
|
Some file names may look different in rclone if you are using any
|
|
control characters in names or unicode FULLWIDTH
|
|
symbols (https://en.wikipedia.org/wiki/Halfwidth_and_Fullwidth_Forms_(Unicode_block)).
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to use go1.13 for the build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Drop support for go1.9 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Build rclone with GitHub actions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Convert python scripts to python3 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Swap Azure/go\-ansiterm for mattn/go\-colorable (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Dockerfile fixes (Matei David)
|
|
.IP \[bu] 2
|
|
Add plugin
|
|
support (https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#writing-a-plugin)
|
|
for backends and commands (Richard Patel)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use alternating Red/Green in config to make more obvious (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
contrib
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add sample DLNA server Docker Compose manifest.
|
|
(pataquets)
|
|
.IP \[bu] 2
|
|
Add sample WebDAV server Docker Compose manifest.
|
|
(pataquets)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-auto\-filename\f[] flag for using file name from URL in
|
|
destination path (Denis)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many compatability improvements (Dan Walters)
|
|
.IP \[bu] 2
|
|
Support for external srt subtitles (Dan Walters)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added command core/quit (Saksham Khanna)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
sync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-update\f[]/\f[C]\-u\f[] not transfer files that haven't
|
|
changed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Free objects after they come out of the transfer pipe to save memory
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-files\-from\ without\ \-\-no\-traverse\f[] doing a
|
|
recursive scan (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accounting for server side copies (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Display `All duplicates removed' only if dedupe successful (Sezal
|
|
Agrawal)
|
|
.IP \[bu] 2
|
|
Display `Deleted X extra copies' only if dedupe successful (Sezal
|
|
Agrawal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only allow up to 100 completed transfers in the accounting list to save
|
|
memory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Cull the old time ranges when possible to save memory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic due to server\-side copy fallback (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix memory leak noticeable for transfers of large numbers of objects
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix total duration calculation (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix environment variables not setting command line flags (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make autocomplete compatible with bash's posix mode for macOS (Danil
|
|
Semelenov)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-progress\f[] work in git bash on Windows (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix `compopt: command not found' on autocomplete on macOS (Danil
|
|
Semelenov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting of non top level flags from environment variables (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check config names more carefully and report errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove error: can't use \f[C]\-\-size\-only\f[] and
|
|
\f[C]\-\-ignore\-size\f[] together.
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
filter: Prevent mixing options when \f[C]\-\-files\-from\f[] is in use
|
|
(Michele Caci)
|
|
.IP \[bu] 2
|
|
serve sftp: Fix crash on unsupported operations (eg Readlink) (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow files of unkown size to be read properly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Skip tests on <= 2 CPUs to avoid lockup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic on File.Open (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[lq]mount_fusefs: \-o timeout=: option not supported\[rq] on
|
|
FreeBSD (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don't pass huge filenames (>4k) to FUSE as it can't cope (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add flag \f[C]\-\-vfs\-case\-insensitive\f[] for windows/macOS mounts
|
|
(Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Make objects of unknown size readable through the VFS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move writeback of dirty data out of close() method into its own method
|
|
(FlushWrites) and remove close() call from Flush() (Brett Dutro)
|
|
.IP \[bu] 2
|
|
Stop empty dirs disappearing when renamed on bucket based remotes (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop change notify polling clearing so much of the directory cache (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable logging to the Windows event log (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove \f[C]unverified:\f[] prefix on sha1 to improve interop (eg with
|
|
CyberDuck) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add options to get access token via JWT auth (David)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable HTTP/2 by default to work around INTERNAL_ERROR problems (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure that drive root ID is always canonical (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-drive\-shared\-with\-me\f[] from the root with lsand
|
|
\f[C]\-\-fast\-list\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify polling for shared drives (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix change notify polling when using appDataFolder (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make disallowed filenames errors not retry (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix nil pointer exception on restricted files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accessing files > 2GB on 32 bit systems (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow disabling EPSV mode (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
HEAD directory entries in parallel to speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-no\-head\f[] to stop rclone doing HEAD in listings
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Putio
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to resume uploads (Cenk Alti)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix signature v2_auth headers (Anthony Rusdi)
|
|
.IP \[bu] 2
|
|
Fix encoding for control characters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only ask for URL encoded directory listings if we need them on Ceph
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add option for multipart failiure behaviour (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Support for multipart copy (庄天翼)
|
|
.IP \[bu] 2
|
|
Fix nil pointer reference if no metadata returned for object (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-sftp\-ask\-password\f[] trying to contact the ssh agent
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix hashes of files with backslashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Include more ciphers with \f[C]\-\-sftp\-use\-insecure\-cipher\f[]
|
|
(Carlos Ferreyra)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Parse and return Sharepoint error response (Henning Surmeier)
|
|
.RE
|
|
.SS v1.49.5 \- 2019\-10\-05
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Revert back to go1.12.x for the v1.49.x builds as go1.13.x was causing
|
|
issues (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rpm packages by using master builds of nfpm (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix macOS build after brew changes (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.4 \- 2019\-09\-29
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cmd/rcd: Address ZipSlip vulnerability (Richard Patel)
|
|
.IP \[bu] 2
|
|
accounting: Fix file handle leak on errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
oauthutil: Fix security problem when running with two users on the same
|
|
machine (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing of an empty root returning: error dir not found (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.3 \- 2019\-09\-15
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix total duration calculation (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Fix \[lq]file already closed\[rq] on transfer retries (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.SS v1.49.2 \- 2019\-09\-08
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
build: Add Docker workflow support (Alfonso Montero)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix locking in Transfer to avoid deadlock with
|
|
\f[C]\-\-progress\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs: Fix template argument for mktemp in install.sh (Cnly)
|
|
.IP \[bu] 2
|
|
operations: Fix \f[C]\-u\f[]/\f[C]\-\-update\f[] with google photos /
|
|
files of unknown size (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Fix docs for config/create /update /password (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix need for elevated permissions on SetModTime (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.1 \- 2019\-08\-28
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
config: Fix generated passwords being stored as empty password (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Added missing parameter for web\-gui info logs.
|
|
(Chaitanya)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Googlephotos
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on error response (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on error response (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.0 \- 2019\-08\-26
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1fichier (https://rclone.org/fichier/) (Laura Hausmann)
|
|
.IP \[bu] 2
|
|
Google Photos (/googlephotos) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Putio (https://rclone.org/putio/) (Cenk Alti)
|
|
.IP \[bu] 2
|
|
premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-compare\-dest\f[] & \f[C]\-\-copy\-dest\f[]
|
|
(yparitcher)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-suffix\f[] without \f[C]\-\-backup\-dir\f[] for
|
|
backup to current dir (yparitcher)
|
|
.IP \[bu] 2
|
|
\f[C]config\ reconnect\f[] to re\-login (re\-run the oauth login) for
|
|
the backend.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]config\ userinfo\f[] to discover which user you are logged in as.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]config\ disconnect\f[] to disconnect you (log out) from the
|
|
backend.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-use\-json\-log\f[] for JSON logging (justinalin)
|
|
.IP \[bu] 2
|
|
Add context propagation to rclone (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Reworking internal statistics interfaces so they work with rc jobs
|
|
(Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Add Higher units for ETA (AbelThar)
|
|
.IP \[bu] 2
|
|
Update rclone logos to new design (Andreas Chlupka)
|
|
.IP \[bu] 2
|
|
hash: Add CRC\-32 support (Cenk Alti)
|
|
.IP \[bu] 2
|
|
help showbackend: Fixed advanced option category when there are no
|
|
standard options (buengese)
|
|
.IP \[bu] 2
|
|
ncdu: Display/Copy to Clipboard Current Path (Gary Kim)
|
|
.IP \[bu] 2
|
|
operations:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Run hashing operations in parallel (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don't calculate checksums when using \f[C]\-\-ignore\-checksum\f[] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check transfer hashes when using \f[C]\-\-size\-only\f[] mode (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Disable multi thread copy for local to local copies (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Debug successful hashes as well as failures (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to stop async jobs (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Return current settings if core/bwlimit called without parameters (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rclone\-WebUI integration with rclone (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Added command line parameter to control the cross origin resource
|
|
sharing (CORS) in the rcd.
|
|
(Security Improvement) (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Add anchor tags to the docs so links are consistent (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove _async key from input parameters after parsing so later
|
|
operations won't get confused (buengese)
|
|
.IP \[bu] 2
|
|
Add call to clear stats (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Auto\-login for web\-gui (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-baseurl\f[] for rcd and web\-gui (Chaitanya
|
|
Bankanhal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only select interfaces which can multicast for SSDP (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add more builtin mime types to cover standard audio/video (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix missing mime types on Android causing missing videos (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve ftp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refactor to bring into line with other serve commands (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-auth\-proxy\f[] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http: Implement \f[C]\-\-baseurl\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve restic: Implement \f[C]\-\-baseurl\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve sftp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement auth proxy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix detection of whether server is authorized (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-baseurl\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support \f[C]\-\-auth\-proxy\f[] (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \[lq]bad record MAC\[rq] a retriable error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
copyurl: Fix copying files that return HTTP errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
march: Fix checking sub\-directories when using
|
|
\f[C]\-\-no\-traverse\f[] (buengese)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix unmarshalable http.AuthFn in options and put in test for
|
|
marshalability (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move job expire flags to rc to fix initalization problem (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-loopback\f[] with rc/list and others (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcat: Fix slowdown on systems with multiple hashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Fix permissions problems on cache directory with web gui download
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default \f[C]\-\-deamon\-timout\f[] to 15 minutes on macOS and FreeBSD
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update docs to show mounting from root OK for bucket based (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove nonseekable flag from write files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make write without cache more efficient (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-vfs\-cache\-mode\ minimal\f[] and \f[C]writes\f[] ignoring
|
|
cached files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-local\-case\-sensitive\f[] and
|
|
\f[C]\-\-local\-case\-insensitive\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Avoid polluting page cache when uploading local files to remote backends
|
|
(Michał Matczuk)
|
|
.IP \[bu] 2
|
|
Don't calculate any hashes by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fadvise run syscall on a dedicated go routine (Michał Matczuk)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure Storage Emulator support (Sandeep)
|
|
.IP \[bu] 2
|
|
Updated config help details to remove connection string references
|
|
(Sandeep)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement link sharing (yparitcher)
|
|
.IP \[bu] 2
|
|
Enable server side copy to copy between buckets (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server side copy of big files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update API for teamdrive use (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add error for purge with \f[C]\-\-drive\-trashed\-only\f[] (ginvine)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make FolderID int and adjust related code (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce oauth scope requested as suggested by Google (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-headers\f[] flag for setting arbitrary headers (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new api for retrieving internal username (buengese)
|
|
.IP \[bu] 2
|
|
Refactor configuration and minor cleanup (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support setting modification times on Koofr backend.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refactor to use existing lib/rest facilities for uploads (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Upgrade to v3 SDK and fix listing loop (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add INTELLIGENT_TIERING storage class (Matti Niemenmaa)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add missing interface check and fix About (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Completely ignore all modtime checks if SetModTime=false (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Support md5/sha1 with rsync.net (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Save the md5/sha1 command in use to the config file for efficiency (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Opt\-in support for diffie\-hellman\-group\-exchange\-sha256
|
|
diffie\-hellman\-group\-exchange\-sha1 (Yi FU)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use FixRangeOption to fix 0 length files via the VFS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix upload when using no_chunk to return the correct size (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix segments leak during failed large file uploads.
|
|
(nguyenhuuluan434)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-webdav\-bearer\-token\-command\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Refresh token when it expires with
|
|
\f[C]\-\-webdav\-bearer\-token\-command\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add docs for using bearer_token_command with oidc\-agent (Paul Millar)
|
|
.RE
|
|
.SS v1.48.0 \- 2019\-06\-15
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve sftp: Serve an rclone remote over SFTP (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi threaded downloads to local storage (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
controlled with \f[C]\-\-multi\-thread\-cutoff\f[] and
|
|
\f[C]\-\-multi\-thread\-streams\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use rclone.conf from rclone executable directory to enable portable use
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Allow sync of a file and a directory with the same name (forgems)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is common on bucket based remotes, eg s3, gcs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ignore\-case\-sync\f[] for forced case insensitivity
|
|
(garry415)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-stats\-one\-line\-date\f[] and
|
|
\f[C]\-\-stats\-one\-line\-date\-format\f[] (Peter Berbec)
|
|
.IP \[bu] 2
|
|
Log an ERROR for all commands which exit with non\-zero status (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use go\-homedir to read the home directory more reliably (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable creating encrypted config through external script invocation
|
|
(Wojciech Smigielski)
|
|
.IP \[bu] 2
|
|
build: Drop support for go1.8 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: Make config create/update encrypt passwords where necessary
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
copyurl: Honor \f[C]\-\-no\-check\-certificate\f[] (Stefan Breunig)
|
|
.IP \[bu] 2
|
|
install: Linux skip man pages if no mandb (didil)
|
|
.IP \[bu] 2
|
|
lsf: Support showing the Tier of the object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added EncryptedPath to output (calisro)
|
|
.IP \[bu] 2
|
|
Support showing the Tier of the object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add IsBucket field for bucket based remote listing of the root (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-loopback\f[] flag to run commands directly without a server
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add operations/fsinfo: Return information about the remote (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Skip auth for OPTIONS request (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
jobs: Make job expiry timeouts configurable (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna reworked and improved (Dan Walters)
|
|
.IP \[bu] 2
|
|
serve ftp: add \f[C]\-\-ftp\-public\-ip\f[] flag to specify public IP
|
|
(calistri)
|
|
.IP \[bu] 2
|
|
serve restic: Add support for \f[C]\-\-private\-repos\f[] in
|
|
\f[C]serve\ restic\f[] (Florian Apolloner)
|
|
.IP \[bu] 2
|
|
serve webdav: Combine serve webdav and serve http (Gary Kim)
|
|
.IP \[bu] 2
|
|
size: Ignore negative sizes when calculating total (Garry McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make move and copy individual files obey \f[C]\-\-backup\-dir\f[] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
If \f[C]\-\-ignore\-checksum\f[] is in effect, don't calculate checksum
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
moveto: Fix case\-insensitive same remote move (Gary Kim)
|
|
.IP \[bu] 2
|
|
rc: Fix serving bucket based objects with \f[C]\-\-rc\-serve\f[] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve webdav: Fix serveDir not being updated with changes from webdav
|
|
(Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix poll interval documentation (Animosity022)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make WriteAt for non cached files work with non\-sequential writes (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only calculate the required hashes for big speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log errors when listing instead of returning an error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix preallocate warning on Linux with ZFS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make rclone dedupe work through crypt (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix wrapping of ChangeNotify to decrypt directories properly (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support PublicLink (rclone link) of underlying backend (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement Optional methods SetTier, GetTier (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement server side copy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement SetModTime (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix move and copy from TeamDrive to GDrive (Fionera)
|
|
.IP \[bu] 2
|
|
Add notes that cleanup works in the background on drive (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-server\-side\-across\-configs\f[] to default back to
|
|
old server side copy semantics by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-size\-as\-quota\f[] to show storage quota usage for
|
|
file size (Garry McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add FTP List timeout (Jeff Quinn)
|
|
.IP \[bu] 2
|
|
Add FTP over TLS support (Gary Kim)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-no\-check\-certificate\f[] option for FTPS (Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload errors when uploading pre 1970 files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for selecting device and mountpoint.
|
|
(buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add cleanup support (Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
More accurately check if root is found (Cnly)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Suppport S3 Accelerated endpoints with
|
|
\f[C]\-\-s3\-use\-accelerate\-endpoint\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add config info for Wasabi's EU Central endpoint (Robert Marko)
|
|
.IP \[bu] 2
|
|
Make SetModTime work for GLACIER while syncing (Philip Harvey)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add About support (Gary Kim)
|
|
.IP \[bu] 2
|
|
Fix about parsing of \f[C]df\f[] results so it can cope with \-ve
|
|
results (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Send custom client version and debug server version (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry on 423 Locked errors (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.47.0 \- 2019\-04\-13
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Backend for Koofr cloud storage service.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Resume downloads if the reader fails in copy (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this means rclone will restart transfers if the source has an error
|
|
.IP \[bu] 2
|
|
this is most useful for downloads or cloud to cloud copies
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-fast\-list\f[] for listing operations where it won't use
|
|
more memory (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this should speed up the following operations on remotes which support
|
|
\f[C]ListR\f[]
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[], \f[C]serve\ restic\f[] \f[C]lsf\f[], \f[C]ls\f[],
|
|
\f[C]lsl\f[], \f[C]lsjson\f[], \f[C]lsd\f[], \f[C]md5sum\f[],
|
|
\f[C]sha1sum\f[], \f[C]hashsum\f[], \f[C]size\f[], \f[C]delete\f[],
|
|
\f[C]cat\f[], \f[C]settier\f[]
|
|
.IP \[bu] 2
|
|
use \f[C]\-\-disable\ ListR\f[] to get old behaviour if required
|
|
.RE
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-files\-from\f[] traverse the destination unless
|
|
\f[C]\-\-no\-traverse\f[] is set (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes \f[C]\-\-files\-from\f[] with Google drive and excessive API
|
|
use in general.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Make server side copy account bytes and obey \f[C]\-\-max\-transfer\f[]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-create\-empty\-src\-dirs\f[] flag and default to not
|
|
creating empty dirs (ishuah)
|
|
.IP \[bu] 2
|
|
Add client side TLS/SSL flags
|
|
\f[C]\-\-ca\-cert\f[]/\f[C]\-\-client\-cert\f[]/\f[C]\-\-client\-key\f[]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-suffix\-keep\-extension\f[] for use with
|
|
\f[C]\-\-suffix\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Switch to semvar compliant version tags to be go modules compliant (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update to use go1.12.x for the build (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna: Add connection manager service description to improve
|
|
compatibility (Dan Walters)
|
|
.IP \[bu] 2
|
|
lsf: Add `e' format to show encrypted names and `o' for original IDs
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Added \f[C]\-\-files\-only\f[] and \f[C]\-\-dirs\-only\f[] flags
|
|
(calistri)
|
|
.IP \[bu] 2
|
|
rc: Implement operations/publiclink the equivalent of
|
|
\f[C]rclone\ link\f[] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix total ETA when \f[C]\-\-stats\-unit\ bits\f[] is in
|
|
effect (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Bash TAB completion
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use private custom func to fix clash between rclone and kubectl (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix for remotes with underscores in their names (Six)
|
|
.IP \[bu] 2
|
|
Fix completion of remotes (Florian Gamböck)
|
|
.IP \[bu] 2
|
|
Fix autocompletion of remote paths with spaces (Danil Semelenov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna: Fix root XML service descriptor (Dan Walters)
|
|
.IP \[bu] 2
|
|
ncdu: Fix display corruption with Chinese characters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add SIGTERM to signals which run the exit handlers on unix (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Reload filter when the options are set via the rc (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS / Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix FreeBSD: Ignore Truncate if called with no readers and already the
|
|
correct size (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Read directory and check for a file before mkdir (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Shorten the locking window for vfs/refresh (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable MD5 checksums when uploading files bigger than the
|
|
\[lq]Cutoff\[rq] (Dr.Rx)
|
|
.IP \[bu] 2
|
|
Fix SAS URL support (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow manual configuration of backblaze downloadUrl (Vince)
|
|
.IP \[bu] 2
|
|
Ignore already_hidden error on remove (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore malformed \f[C]src_last_modified_millis\f[] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-skip\-checksum\-gphotos\f[] to ignore incorrect checksums
|
|
on Google Photos (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow server side move/copy between different remotes.
|
|
(Fionera)
|
|
.IP \[bu] 2
|
|
Add docs on team drives and \f[C]\-\-fast\-list\f[] eventual consistency
|
|
(Nestar47)
|
|
.IP \[bu] 2
|
|
Fix imports of text files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix range requests on 0 length files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix creation of duplicates with server side copy (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry blank errors to fix long listings (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-concurrency\f[] to limit maximum number of connections
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fall back to default application credentials (marcintustin)
|
|
.IP \[bu] 2
|
|
Allow bucket policy only buckets (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-no\-slash\f[] for websites with directories with no
|
|
slashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove duplicates from listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix socket leak on 404 errors (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token refresh (Sebastian Bünger)
|
|
.IP \[bu] 2
|
|
Add device registration (Oliver Heyme)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement graceful cancel of multipart uploads if rclone is interrupted
|
|
(Cnly)
|
|
.IP \[bu] 2
|
|
Always add trailing colon to path when addressing items, (Cnly)
|
|
.IP \[bu] 2
|
|
Return errors instead of panic for invalid uploads (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \[lq]Glacier Deep Archive\[rq] storage class (Manu)
|
|
.IP \[bu] 2
|
|
Update Dreamhost endpoint (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Note incompatibility with CEPH Jewel (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow custom ssh client config (Alexandru Bumbacea)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obey Retry\-After to enable OVH restore from cold storage (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Work around token expiry on CEPH (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow IsCollection property to be integer or boolean (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix race when creating directories (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix About/df when reading the available/total returns 0 (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.46 \- 2019\-02\-09
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve dlna: serves a remove via DLNA for the local network (nicolov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copy, move: Restore deprecated \f[C]\-\-no\-traverse\f[] flag (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is useful for when transferring a small number of files into a
|
|
large destination
|
|
.RE
|
|
.IP \[bu] 2
|
|
genautocomplete: Add remote path completion for bash completion
|
|
(Christopher Peterson & Danil Semelenov)
|
|
.IP \[bu] 2
|
|
Buffer memory handling reworked to return memory to the OS better (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Buffer recycling library to replace sync.Pool
|
|
.IP \[bu] 2
|
|
Optionally use memory mapped memory for better memory shrinking
|
|
.IP \[bu] 2
|
|
Enable with \f[C]\-\-use\-mmap\f[] if having memory problems \- not
|
|
default yet
|
|
.RE
|
|
.IP \[bu] 2
|
|
Parallelise reading of files specified by \f[C]\-\-files\-from\f[] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
check: Add stats showing total files matched.
|
|
(Dario Guzik)
|
|
.IP \[bu] 2
|
|
Allow rename/delete open files under Windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Use exactly the correct number of decimal places in the seconds
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add cookie support with cmdline switch \f[C]\-\-use\-cookies\f[] for all
|
|
HTTP based remotes (qip)
|
|
.IP \[bu] 2
|
|
Warn if \f[C]\-\-checksum\f[] is set but there are no hashes available
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rework rate limiting (pacer) to be more accurate and allow bursting
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Improve error reporting for too many/few arguments in commands (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
listremotes: Remove \f[C]\-l\f[] short flag as it conflicts with the new
|
|
global flag (weetmuts)
|
|
.IP \[bu] 2
|
|
Make http serving with auth generate INFO messages on auth fail (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix layout of stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-progress\f[] crash under Windows Jenkins (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix transfer of google/onedrive docs by calling Rcat in Copy when size
|
|
is \-1 (Cnly)
|
|
.IP \[bu] 2
|
|
copyurl: Fix checking of \f[C]\-\-dry\-run\f[] (Denis Skovpen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check that mountpoint and local directory to mount don't overlap (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mount size under 32 bit Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement renaming of directories for backends without DirMove (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
now all backends except b2 support renaming directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-vfs\-cache\-max\-size\f[] to limit the total size of
|
|
the cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dir\-perms\f[] and \f[C]\-\-file\-perms\f[] flags to set
|
|
default permissions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock on concurrent operations on a directory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock between RWFileHandle.close and File.Remove (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix renaming/deleting open files with cache mode \[lq]writes\[rq] under
|
|
Windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic on rename with \f[C]\-\-dry\-run\f[] set (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix vfs/refresh with recurse=true needing the \f[C]\-\-fast\-list\f[]
|
|
flag
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-l\f[]/\f[C]\-\-links\f[] (symbolic link
|
|
translation) (yair\@unicorn)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this works by showing links as \f[C]link.rclonelink\f[] \- see local
|
|
backend docs for more info
|
|
.IP \[bu] 2
|
|
this errors if used with \f[C]\-L\f[]/\f[C]\-\-copy\-links\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix renaming/deleting open files on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check for maximum length before decrypting filename to fix panic (Garry
|
|
McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow building azureblob backend on *BSD (themylogin)
|
|
.IP \[bu] 2
|
|
Use the rclone HTTP client to support \f[C]\-\-dump\ headers\f[],
|
|
\f[C]\-\-tpslimit\f[] etc (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use the s3 pacer for 0 delay in non error conditions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore directory markers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop Mkdir attempting to create existing containers (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cleanup: will remove unfinished large files >24hrs old (Garry McNulty)
|
|
.IP \[bu] 2
|
|
For a bucket limited application key check the bucket name (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
before this, rclone would use the authorised bucket regardless of what
|
|
you put on the command line
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-b2\-disable\-checksum\f[] flag (Wojciech Smigielski)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this enables large files to be uploaded without a SHA\-1 hash for speed
|
|
reasons
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set default pacer to 100ms for 10 tps (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This fits the Google defaults much better and reduces the 403 errors
|
|
massively
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-pacer\-min\-sleep\f[] and
|
|
\f[C]\-\-drive\-pacer\-burst\f[] to control the pacer
|
|
.RE
|
|
.IP \[bu] 2
|
|
Improve ChangeNotify support for items with multiple parents (Fabian
|
|
Möller)
|
|
.IP \[bu] 2
|
|
Fix ListR for items with multiple parents \- this fixes oddities with
|
|
\f[C]vfs/refresh\f[] (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix using \f[C]\-\-drive\-impersonate\f[] and appfolders (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix google docs in rclone mount for some (not all) applications (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry\-After support for Dropbox backend (Mathieu Carbou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wait for 60 seconds for a connection to Close then declare it dead (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
helps with indefinite hangs on some FTP servers
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update google cloud storage endpoints (weetmuts)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add an example with username and password which is supported but wasn't
|
|
documented (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix backend with \f[C]\-\-files\-from\f[] and non\-existent files (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make error message more informative if authentication fails (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Resume and deduplication support (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Use token auth for all API requests Don't store password anymore
|
|
(Sebastian Bünger)
|
|
.IP \[bu] 2
|
|
Add support for 2\-factor authentification (Sebastian Bünger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement v2 account login which fixes logins for newer Mega accounts
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Return error if an unknown length file is attempted to be uploaded (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add new error codes for better error reporting (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix broken support for \[lq]shared with me\[rq] folders (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix root ID not normalised (Cnly)
|
|
.IP \[bu] 2
|
|
Return err instead of panic on unknown\-sized uploads (Cnly)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix go routine leak on multipart upload errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add upload chunk size/concurrency/cutoff control (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Default \f[C]\-\-qingstor\-upload\-concurrency\f[] to 1 to work around
|
|
bug (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-s3\-upload\-cutoff\f[] for single part uploads below
|
|
this (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change \f[C]\-\-s3\-upload\-concurrency\f[] default to 4 to increase
|
|
perfomance (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-bucket\-acl\f[] to control bucket ACL (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Auto detect region for buckets on operation failure (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add GLACIER storage class (William Cocker)
|
|
.IP \[bu] 2
|
|
Add Scaleway to s3 documentation (Rémy Léone)
|
|
.IP \[bu] 2
|
|
Add AWS endpoint eu\-north\-1 (weetmuts)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for PEM encrypted private keys (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add option to force the usage of an ssh\-agent (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Perform environment variable expansion on key\-file (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix rmdir on Windows based servers (eg CrushFTP) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rmdir deleting directory contents on some SFTP servers (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix error on dangling symlinks (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-swift\-no\-chunk\f[] to disable segmented uploads in
|
|
rcat/mount (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Introduce application credential auth support (kayrus)
|
|
.IP \[bu] 2
|
|
Fix memory usage by slimming Object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix extra requests on upload (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix reauth on big files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix poll\-interval not working (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support About which means rclone mount will show the correct disk size
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fail soft on time parsing errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix infinite loop on failed directory creation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix identification of directories for Bitrix Site Manager (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix upload of 0 length files on some servers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix if MKCOL fails with 423 Locked assume the directory exists (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.45 \- 2018\-11\-24
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Yandex backend was re\-written \- see below for details (Sebastian
|
|
Bünger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rcd: New command just to serve the remote control API (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The remote control API (rc) was greatly expanded to allow full control
|
|
over rclone (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
sensitive operations require authorization or the
|
|
\f[C]\-\-rc\-no\-auth\f[] flag
|
|
.IP \[bu] 2
|
|
config/* operations to configure rclone
|
|
.IP \[bu] 2
|
|
options/* for reading/setting command line flags
|
|
.IP \[bu] 2
|
|
operations/* for all low level operations, eg copy file, list directory
|
|
.IP \[bu] 2
|
|
sync/* for sync, copy and move
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-files\f[] flag to serve files on the rc http server
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is for building web native GUIs for rclone
|
|
.RE
|
|
.IP \[bu] 2
|
|
Optionally serving objects on the rc http server
|
|
.IP \[bu] 2
|
|
Ensure rclone fails to start up if the \f[C]\-\-rc\f[] port is in use
|
|
already
|
|
.IP \[bu] 2
|
|
See the rc docs (https://rclone.org/rc/) for more info
|
|
.RE
|
|
.IP \[bu] 2
|
|
sync/copy/move
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-files\-from\f[] only read the objects specified and don't
|
|
scan directories (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is a huge speed improvement for destinations with lots of files
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
filter: Add \f[C]\-\-ignore\-case\f[] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
ncdu: Add remove function (`d' key) (Henning Surmeier)
|
|
.IP \[bu] 2
|
|
rc command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-json\f[] flag for structured JSON input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-user\f[] and \f[C]\-\-pass\f[] flags and interpret
|
|
\f[C]\-\-rc\-user\f[], \f[C]\-\-rc\-pass\f[], \f[C]\-\-rc\-addr\f[]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Require go1.8 or later for compilation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable softfloat on MIPS arch (Scott Edlund)
|
|
.IP \[bu] 2
|
|
Integration test framework revamped with a better report and better
|
|
retries (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cmd: Make \f[C]\-\-progress\f[] update the stats correctly at the end
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: Create config directory on save if it is missing (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Check for existing filename before renaming a dupe file (ssaqua)
|
|
.IP \[bu] 2
|
|
move: Don't create directories with \f[C]\-\-dry\-run\f[] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations: Fix Purge and Rmdirs when dir is not the root (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve http/webdav/restic: Ensure rclone exits if the port is in use
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-volname\f[] work for Windows and macOS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Avoid context deadline exceeded error by setting a large TryTimeout
|
|
value (brused27)
|
|
.IP \[bu] 2
|
|
Fix erroneous Rmdir error \[lq]directory not empty\[rq] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Wait for up to 60s to create a just deleted container (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add dropbox impersonate support (Jake Coggiano)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bug in \f[C]\-\-fast\-list\f[] handing of empty folders (albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix transfer of files with \f[C]+\f[] and \f[C]&\f[] in (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix retries of upload chunks (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set ACL for server side copies to that provided by the user (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix role_arn, credential_source, \&... (Erik Swanson)
|
|
.IP \[bu] 2
|
|
Add config info for Wasabi's US\-West endpoint (Henry Ptasinski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure file hash checking is really disabled (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add pacer for retries to make swift more reliable (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add Content\-Type to PUT requests (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix config parsing so \f[C]\-\-webdav\-user\f[] and
|
|
\f[C]\-\-webdav\-pass\f[] flags work (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add RFC3339 date format (Ralf Hemberger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The yandex backend was re\-written (Sebastian Bünger)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This implements low level retries (Sebastian Bünger)
|
|
.IP \[bu] 2
|
|
Copy, Move, DirMove, PublicLink and About optional interfaces (Sebastian
|
|
Bünger)
|
|
.IP \[bu] 2
|
|
Improved general error handling (Sebastian Bünger)
|
|
.IP \[bu] 2
|
|
Removed ListR for now due to inconsistent behaviour (Sebastian Bünger)
|
|
.RE
|
|
.RE
|
|
.SS v1.44 \- 2018\-10\-15
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve ftp: Add ftp server (Antoine GIRARD)
|
|
.IP \[bu] 2
|
|
settier: perform storage tier changes on supported remotes (sandeepkru)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reworked command line help
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make default help less verbose (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Split flags up into global and backend flags (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement specialised help for flags and backends (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Show URL of backend help page when starting config (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
stats: Long names now split in center (Joanna Marek)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-log\-format\f[] flag for more control over log output
|
|
(dcpu)
|
|
.IP \[bu] 2
|
|
rc: Add support for OPTIONS and basic CORS (frenos)
|
|
.IP \[bu] 2
|
|
stats: show FatalErrors and NoRetryErrors in stats (Cédric Connes)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \-P not ending with a new line (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: don't create default config dir when user supplies
|
|
\f[C]\-\-config\f[] (albertony)
|
|
.IP \[bu] 2
|
|
Don't print non\-ASCII characters with \f[C]\-\-progress\f[] on windows
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Correct logs for excluded items (ssaqua)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove EXPERIMENTAL tags (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix race condition detected by serve ftp tests (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add vfs/poll\-interval rc command (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Enable rename for nearly all remotes using server side Move or Copy
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Reduce directory cache cleared by poll\-interval (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Remove EXPERIMENTAL tags (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Skip bad symlinks in dir listing with \-L enabled (Cédric Connes)
|
|
.IP \[bu] 2
|
|
Preallocate files on Windows to reduce fragmentation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Preallocate files on linux with fallocate(2) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add cache/fetch rc function (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix worker scale down (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Improve performance by not sending info requests for cached chunks
|
|
(dcpu)
|
|
.IP \[bu] 2
|
|
Fix error return value of cache/fetch rc method (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Documentation fix for cache\-chunk\-total\-size (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Preserve leading / in wrapped remote path (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add plex_insecure option to skip certificate validation (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Remove entries that no longer exist in the source (dcpu)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preserve leading / in wrapped remote path (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Alias
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix handling of Windows network paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-azureblob\-list\-chunk\f[] parameter (Santiago Rodríguez)
|
|
.IP \[bu] 2
|
|
Implemented settier command support on azureblob remote.
|
|
(sandeepkru)
|
|
.IP \[bu] 2
|
|
Work around SDK bug which causes errors for chunk\-sized files (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement link sharing.
|
|
(Sebastian Bünger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-import\-formats\f[] \- google docs can now be
|
|
imported (Fabian Möller)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rewrite mime type and extension handling (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add document links (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add support for multipart document extensions (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add support for apps\-script to json export (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix escaped chars in documents during list (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-v2\-download\-min\-size\f[] a workaround for slow
|
|
downloads (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Improve directory notifications in ChangeNotify (Fabian Möller)
|
|
.IP \[bu] 2
|
|
When listing team drives in config, continue on failure (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add a small pause after failed upload before deleting file (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix service_account_file being ignored (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Minor improvement in quota info (omit if unlimited) (albertony)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-fast\-list\f[] support (albertony)
|
|
.IP \[bu] 2
|
|
Add permanent delete support: \f[C]\-\-jottacloud\-hard\-delete\f[]
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Add link sharing support (albertony)
|
|
.IP \[bu] 2
|
|
Fix handling of reserved characters.
|
|
(Sebastian Bünger)
|
|
.IP \[bu] 2
|
|
Fix socket leak on Object.Remove (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework to support Microsoft Graph (Cnly)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[] this will require re\-authenticating the remote
|
|
.RE
|
|
.IP \[bu] 2
|
|
Removed upload cutoff and always do session uploads (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Use single\-part upload for empty files (Cnly)
|
|
.IP \[bu] 2
|
|
Fix new fields not saved when editing old config (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix sometimes special chars in filenames not replaced (Alex Chen)
|
|
.IP \[bu] 2
|
|
Ignore OneNote files by default (Alex Chen)
|
|
.IP \[bu] 2
|
|
Add link sharing support (jackyzy823)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use custom pacer, to retry operations when reasonable (Craig Miskell)
|
|
.IP \[bu] 2
|
|
Use configured server\-side\-encryption and storace class options when
|
|
calling CopyObject() (Paul Kohout)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-s3\-v2\-auth\f[] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix v2 auth on files with spaces (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement union backend which reads from multiple backends (Felix
|
|
Brucker)
|
|
.IP \[bu] 2
|
|
Implement optional interfaces (Move, DirMove, Copy etc) (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify to support multiple remotes (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-backup\-dir\f[] on union backend (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add another time format (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add a small pause after failed upload before deleting file (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add workaround for missing mtime (buergi)
|
|
.IP \[bu] 2
|
|
Sharepoint: Renew cookies after 12hrs (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove redundant nil checks (teresy)
|
|
.RE
|
|
.SS v1.43.1 \- 2018\-09\-07
|
|
.PP
|
|
Point release to fix hubic and azureblob backends.
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
ncdu: Return error instead of log.Fatal in Show (Fabian Möller)
|
|
.IP \[bu] 2
|
|
cmd: Fix crash with \f[C]\-\-progress\f[] and \f[C]\-\-stats\ 0\f[]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs: Tidy website display (Anagh Kumar Baranwal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix multi\-part uploads.
|
|
(sandeepkru)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry auth fetching if it fails to make hubic more reliable (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.43 \- 2018\-09\-01
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Jottacloud (Sebastian Bünger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copyurl: copies a URL to a remote (Denis)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reworked config for backends (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All backend config can now be supplied by command line, env var or
|
|
config file
|
|
.IP \[bu] 2
|
|
Advanced section in the config wizard for the optional items
|
|
.IP \[bu] 2
|
|
A large step towards rclone backends being usable in other go software
|
|
.IP \[bu] 2
|
|
Allow on the fly remotes with :backend: syntax
|
|
.RE
|
|
.IP \[bu] 2
|
|
Stats revamp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-progress\f[]/\f[C]\-P\f[] flag to show interactive progress
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Show the total progress of the sync in the stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-stats\-one\-line\f[] flag for single line stats (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added weekday schedule into \f[C]\-\-bwlimit\f[] (Mateusz)
|
|
.IP \[bu] 2
|
|
lsjson: Add option to show the original object IDs (Fabian Möller)
|
|
.IP \[bu] 2
|
|
serve webdav: Make Content\-Type without reading the file and add
|
|
\f[C]\-\-etag\-hash\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Build macOS with native compiler (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update to use go1.11 for the build (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added core/stats to return the stats (reddi1)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]version\ \-\-check\f[]: Prints the current release and beta
|
|
versions (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix time to completion estimates (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix moving average speed for file stats (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config: Fix error reading password from piped input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
move: Fix \f[C]\-\-delete\-empty\-src\-dirs\f[] flag to delete all empty
|
|
dirs on move (ishuah)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-daemon\-timeout\f[] flag for OSXFUSE (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mount \f[C]\-\-daemon\f[] not working with encrypted config (Alex
|
|
Chen)
|
|
.IP \[bu] 2
|
|
Clip the number of blocks to 2^32\-1 on macOS \- fixes borg backup (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable vfs\-read\-chunk\-size by default (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add the vfs/refresh rc command (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add non recursive mode to vfs/refresh rc command (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Try to seek buffer on read only files (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash when deprecated \f[C]\-\-local\-no\-unicode\-normalization\f[]
|
|
is supplied (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mkdir error when trying to copy files to the root of a drive on
|
|
windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref when using lsjson on cached directory (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref for occasional crash on playback (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accounting when checking hashes on upload (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Cloud Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make very clear in the docs that rclone has no ACD keys (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add connection string and SAS URL auth (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
List the container to see if it exists (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Port new Azure Blob Storage SDK (sandeepkru)
|
|
.IP \[bu] 2
|
|
Added blob tier, tier between Hot, Cool and Archive.
|
|
(sandeepkru)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support Application Keys (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of > 2GB files on 32 bit platforms (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-box\-commit\-retries\f[] flag defaulting to 100 to fix
|
|
large uploads (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-keep\-revision\-forever\f[] flag (lewapm)
|
|
.IP \[bu] 2
|
|
Handle gdocs when filtering file names in list (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Support using \f[C]\-\-fast\-list\f[] for large speedups (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix index out of range error with \f[C]\-\-fast\-list\f[] (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix MD5 error check (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Handle empty time values (Martin Polden)
|
|
.IP \[bu] 2
|
|
Calculate missing MD5s (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Docs, fixes and tests for MD5 calculation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add optional MimeTyper interface.
|
|
(Sebastian Bünger)
|
|
.IP \[bu] 2
|
|
Implement optional About interface (for \f[C]df\f[] support).
|
|
(Sebastian Bünger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wait for events instead of arbitrary sleeping (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-mega\-hard\-delete\f[] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix failed logins with upper case chars in email (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Shared folder support (Yoni Jah)
|
|
.IP \[bu] 2
|
|
Implement DirMove (Cnly)
|
|
.IP \[bu] 2
|
|
Fix rmdir sometimes deleting directories with contents (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Delete half uploaded files on upload error (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix index out of range error with \f[C]\-\-fast\-list\f[] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-force\-path\-style\f[] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for KMS Key ID (bsteiss)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]storage_policy\f[] (Ruben Vandamme)
|
|
.IP \[bu] 2
|
|
Make it so just \f[C]storage_url\f[] or \f[C]auth_token\f[] can be
|
|
overidden (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix server side copy bug for unusal file names (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure we call MKCOL with a URL with a trailing / for QNAP interop (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
If root ends with / then don't check if it is a file (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don't accept redirects when reading metadata (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add bearer token (Macaroon) support for dCache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Document dCache and Macaroons (Onno Zweers)
|
|
.IP \[bu] 2
|
|
Sharepoint recursion with different depth (Henning)
|
|
.IP \[bu] 2
|
|
Attempt to remove failed uploads (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing/deleting files in the root (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.42 \- 2018\-06\-16
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
deletefile command (Filip Bartodziej)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copy, move: Copy single files directly, don't use
|
|
\f[C]\-\-files\-from\f[] work\-around
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes them much more efficient
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-max\-transfer\f[] flag to quit transferring at a
|
|
limit
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
make exit code 8 for \f[C]\-\-max\-transfer\f[] exceeded
|
|
.RE
|
|
.IP \[bu] 2
|
|
copy: copy empty source directories to destination (Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
check: Add \f[C]\-\-one\-way\f[] flag (Kasper Byrdal Nielsen)
|
|
.IP \[bu] 2
|
|
Add siginfo handler for macOS for ctrl\-T stats (kubatasiemski)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
add core/gc to run a garbage collection on demand
|
|
.IP \[bu] 2
|
|
enable go profiling by default on the \f[C]\-\-rc\f[] port
|
|
.IP \[bu] 2
|
|
return error from remote on failure
|
|
.RE
|
|
.IP \[bu] 2
|
|
lsf
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-absolute\f[] flag to add a leading / onto path names
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-csv\f[] flag for compliant CSV output
|
|
.IP \[bu] 2
|
|
Add `m' format specifier to show the MimeType
|
|
.IP \[bu] 2
|
|
Implement `i' format for showing object ID
|
|
.RE
|
|
.IP \[bu] 2
|
|
lsjson
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add MimeType to the output
|
|
.IP \[bu] 2
|
|
Add ID field to output to show Object ID
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-retries\-sleep\f[] flag (Benjamin Joseph Dag)
|
|
.IP \[bu] 2
|
|
Oauth tidy up web page and error handling (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Password prompt output with \f[C]\-\-log\-file\f[] fixed for unix (Filip
|
|
Bartodziej)
|
|
.IP \[bu] 2
|
|
Calculate ModifyWindow each time on the fly to fix various problems
|
|
(Stefan Breunig)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only print \[lq]File.rename error\[rq] if there actually is an error
|
|
(Stefan Breunig)
|
|
.IP \[bu] 2
|
|
Delay rename if file has open writers instead of failing outright
|
|
(Stefan Breunig)
|
|
.IP \[bu] 2
|
|
Ensure atexit gets run on interrupt
|
|
.IP \[bu] 2
|
|
macOS enhancements
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-noappledouble\f[] \f[C]\-\-noapplexattr\f[]
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-volname\f[] flag and remove special chars from it
|
|
.IP \[bu] 2
|
|
Make Get/List/Set/Remove xattr return ENOSYS for efficiency
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-daemon\f[] work for macOS without CGO
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-read\-chunk\-size\f[] and
|
|
\f[C]\-\-vfs\-read\-chunk\-size\-limit\f[] (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify for new or changed folders (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix symlink/junction point directory handling under Windows
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[] you will need to add \f[C]\-L\f[] to your command line to
|
|
copy files with reparse points
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add non cached dirs on notifications (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Allow root to be expired from rc (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Clean remaining empty folders from temp upload path (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Cache lists using batch writes (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Use secure websockets for HTTPS Plex addresses (John Clayton)
|
|
.IP \[bu] 2
|
|
Reconnect plex websocket on failures (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Fix panic when running without plex configs (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Fix root folder caching (Remus Bunduc)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check the crypted hash of files when uploading for extra data security
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make Dropbox for business folders accessible using an initial \f[C]/\f[]
|
|
in the path
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Low level retry all operations if necessary
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-acknowledge\-abuse\f[] to download flagged files
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-alternate\-export\f[] to fix large doc export
|
|
.IP \[bu] 2
|
|
Don't attempt to choose Team Drives when using rclone config create
|
|
.IP \[bu] 2
|
|
Fix change list polling with team drives
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify for folders (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Fix about (and df on a mount) for team drives
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Errorhandler for onedrive for business requests (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adjust upload concurrency with \f[C]\-\-s3\-upload\-concurrency\f[]
|
|
(themylogin)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-s3\-chunk\-size\f[] which was always using the minimum
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ssh\-path\-override\f[] flag (Piotr Oleszczyk)
|
|
.IP \[bu] 2
|
|
Fix slow downloads for long latency connections
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add workarounds for biz.mail.ru
|
|
.IP \[bu] 2
|
|
Ignore Reason\-Phrase in status line to fix 4shared (Rodrigo)
|
|
.IP \[bu] 2
|
|
Better error message generation
|
|
.RE
|
|
.SS v1.41 \- 2018\-04\-28
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mega support added
|
|
.IP \[bu] 2
|
|
Webdav now supports SharePoint cookie authentication (hensur)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
link: create public link to files and folders (Stefan Breunig)
|
|
.IP \[bu] 2
|
|
about: gets quota info from a remote (a\-roussos, ncw)
|
|
.IP \[bu] 2
|
|
hashsum: a generic tool for any hash to produce md5sum like output
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
lsd: Add \-R flag and fix and update docs for all ls commands
|
|
.IP \[bu] 2
|
|
ncdu: added a \[lq]refresh\[rq] key \- CTRL\-L (Keith Goldfarb)
|
|
.IP \[bu] 2
|
|
serve restic: Add append\-only mode (Steve Kriss)
|
|
.IP \[bu] 2
|
|
serve restic: Disallow overwriting files in append\-only mode (Alexander
|
|
Neumann)
|
|
.IP \[bu] 2
|
|
serve restic: Print actual listener address (Matt Holt)
|
|
.IP \[bu] 2
|
|
size: Add \[en]json flag (Matthew Holt)
|
|
.IP \[bu] 2
|
|
sync: implement \[en]ignore\-errors (Mateusz Pabian)
|
|
.IP \[bu] 2
|
|
dedupe: Add dedupe largest functionality (Richard Yang)
|
|
.IP \[bu] 2
|
|
fs: Extend SizeSuffix to include TB and PB for rclone about
|
|
.IP \[bu] 2
|
|
fs: add \[en]dump goroutines and \[en]dump openfiles for debugging
|
|
.IP \[bu] 2
|
|
rc: implement core/memstats to print internal memory usage info
|
|
.IP \[bu] 2
|
|
rc: new call rc/pid (Michael P.
|
|
Dubner)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Drop support for go1.6
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]make\ tarball\f[] (Chih\-Hsuan Yen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
filter: fix \[en]min\-age and \[en]max\-age together check
|
|
.IP \[bu] 2
|
|
fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport
|
|
.IP \[bu] 2
|
|
lsd,lsf: make sure all times we output are in local time
|
|
.IP \[bu] 2
|
|
rc: fix setting bwlimit to unlimited
|
|
.IP \[bu] 2
|
|
rc: take note of the \[en]rc\-addr flag too as per the docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use About to return the correct disk total/used/free (eg in \f[C]df\f[])
|
|
.IP \[bu] 2
|
|
Set \f[C]\-\-attr\-timeout\ default\f[] to \f[C]1s\f[] \- fixes:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone using too much memory
|
|
.IP \[bu] 2
|
|
rclone not serving files to samba
|
|
.IP \[bu] 2
|
|
excessive time listing directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix \f[C]df\ \-i\f[] (upstream fix)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Filter files \f[C]\&.\f[] and \f[C]\&..\f[] from directory listing
|
|
.IP \[bu] 2
|
|
Only make the VFS cache if \[en]vfs\-cache\-mode > Off
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]local\-no\-check\-updated to disable updated file checks
|
|
.IP \[bu] 2
|
|
Retry remove on Windows sharing violation error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Flush the memory cache after close
|
|
.IP \[bu] 2
|
|
Purge file data on notification
|
|
.IP \[bu] 2
|
|
Always forget parent dir for notifications
|
|
.IP \[bu] 2
|
|
Integrate with Plex websocket
|
|
.IP \[bu] 2
|
|
Add rc cache/stats (seuffert)
|
|
.IP \[bu] 2
|
|
Add info log on notification
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix failure reading large directories \- parse file/directory size as
|
|
float
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crypt+obfuscate on dropbox
|
|
.IP \[bu] 2
|
|
Fix repeatedly uploading the same files
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around strange response from box FTP server
|
|
.IP \[bu] 2
|
|
More workarounds for FTP servers to fix mkParentDir error
|
|
.IP \[bu] 2
|
|
Fix no error on listing non\-existent directory
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service_account_credentials (Matt Holt)
|
|
.IP \[bu] 2
|
|
Detect bucket presence by listing it \- minimises permissions needed
|
|
.IP \[bu] 2
|
|
Ignore zero length directory markers
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service_account_credentials (Matt Holt)
|
|
.IP \[bu] 2
|
|
Fix directory move leaving a hardlinked directory behind
|
|
.IP \[bu] 2
|
|
Return proper google errors when Opening files
|
|
.IP \[bu] 2
|
|
When initialized with a filepath, optional features used incorrect root
|
|
path (Stefan Breunig)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sync for servers which don't return Content\-Length in HEAD
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add QuickXorHash support for OneDrive for business
|
|
.IP \[bu] 2
|
|
Fix socket leak in multipart session upload
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Look in S3 named profile files for credentials
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-disable\-checksum\f[] to disable checksum uploading
|
|
(Chris Redekop)
|
|
.IP \[bu] 2
|
|
Hierarchical configuration support (Giri Badanahatti)
|
|
.IP \[bu] 2
|
|
Add in config for all the supported S3 providers
|
|
.IP \[bu] 2
|
|
Add One Zone Infrequent Access storage class (Craig Rachel)
|
|
.IP \[bu] 2
|
|
Add \[en]use\-server\-modtime support (Peter Baumgartner)
|
|
.IP \[bu] 2
|
|
Add \[en]s3\-chunk\-size option to control multipart uploads
|
|
.IP \[bu] 2
|
|
Ignore zero length directory markers
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update docs to match code, fix typos and clarify disable_hashcheck
|
|
prompt (Michael G.
|
|
Noll)
|
|
.IP \[bu] 2
|
|
Update docs with Synology quirks
|
|
.IP \[bu] 2
|
|
Fail soft with a debug on hash failure
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]use\-server\-modtime support (Peter Baumgartner)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support SharePoint cookie authentication (hensur)
|
|
.IP \[bu] 2
|
|
Strip leading and trailing / off root
|
|
.RE
|
|
.SS v1.40 \- 2018\-03\-19
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Alias backend to create aliases for existing remote names (Fabian
|
|
Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[]: list for parsing purposes (Jakub Tasiemski)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
by default this is a simple non recursive list of files and directories
|
|
.IP \[bu] 2
|
|
it can be configured to add more info in an easy to parse way
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]serve\ restic\f[]: for serving a remote as a Restic REST endpoint
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This enables restic to use any backends that rclone can access
|
|
.IP \[bu] 2
|
|
Thanks Alexander Neumann for help, patches and review
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rc\f[]: enable the remote control of a running rclone
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The running rclone must be started with \[en]rc and related flags.
|
|
.IP \[bu] 2
|
|
Currently there is support for bwlimit, and flushing for mount and
|
|
cache.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-delete\f[] flag to add a delete threshold (Bjørn Erik
|
|
Pedersen)
|
|
.IP \[bu] 2
|
|
All backends now support RangeOption for ranged Open
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]cat\f[]: Use RangeOption for limited fetches to make more efficient
|
|
.IP \[bu] 2
|
|
\f[C]cryptcheck\f[]: make reading of nonce more efficient with
|
|
RangeOption
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http/webdav/restic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
support SSL/TLS
|
|
.IP \[bu] 2
|
|
add \f[C]\-\-user\f[] \f[C]\-\-pass\f[] and \f[C]\-\-htpasswd\f[] for
|
|
authentication
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]copy\f[]/\f[C]move\f[]: detect file size change during copy/move
|
|
and abort transfer (ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]cryptdecode\f[]: added option to return encrypted file names.
|
|
(ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[]: add \f[C]\-\-encrypted\f[] to show encrypted name
|
|
(Jakub Tasiemski)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-stats\-file\-name\-length\f[] to specify the printed file
|
|
name length for stats (Will Gunn)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Code base was shuffled and factored
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
backends moved into a backend directory
|
|
.IP \[bu] 2
|
|
large packages split up
|
|
.IP \[bu] 2
|
|
See the CONTRIBUTING.md doc for info as to what lives where now
|
|
.RE
|
|
.IP \[bu] 2
|
|
Update to using go1.10 as the default go version
|
|
.IP \[bu] 2
|
|
Implement daily full integration
|
|
tests (https://pub.rclone.org/integration-tests/)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Include a source tarball and sign it and the binaries
|
|
.IP \[bu] 2
|
|
Sign the git tags as part of the release process
|
|
.IP \[bu] 2
|
|
Add .deb and .rpm packages as part of the build
|
|
.IP \[bu] 2
|
|
Make a beta release for all branches on the main repo (but not pull
|
|
requests)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
config: fixes errors on non existing config by loading config file only
|
|
on first access
|
|
.IP \[bu] 2
|
|
config: retry saving the config after failure (Mateusz)
|
|
.IP \[bu] 2
|
|
sync: when using \f[C]\-\-backup\-dir\f[] don't delete files if we can't
|
|
set their modtime
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes odd behaviour with Dropbox and \f[C]\-\-backup\-dir\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
fshttp: fix idle timeouts for HTTP connections
|
|
.IP \[bu] 2
|
|
\f[C]serve\ http\f[]: fix serving files with : in \- fixes
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-exclude\-if\-present\f[] to ignore directories which it
|
|
doesn't have permission for (Iakov Davydov)
|
|
.IP \[bu] 2
|
|
Make accounting work properly with crypt and b2
|
|
.IP \[bu] 2
|
|
remove \f[C]\-\-no\-traverse\f[] flag because it is obsolete
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-attr\-timeout\f[] flag to control attribute caching in
|
|
kernel
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this now defaults to 0 which is correct but less efficient
|
|
.IP \[bu] 2
|
|
see the mount docs (/commands/rclone_mount/#attribute-caching) for more
|
|
info
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-daemon\f[] flag to allow mount to run in the background
|
|
(ishuah)
|
|
.IP \[bu] 2
|
|
Fix: Return ENOSYS rather than EIO on attempted link
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This fixes FileZilla accessing an rclone mount served over sftp.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix setting modtime twice
|
|
.IP \[bu] 2
|
|
Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
|
|
.IP \[bu] 2
|
|
Many bugs fixed in the VFS layer \- see below
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many fixes for \f[C]\-\-vfs\-cache\-mode\f[] writes and above
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update cached copy if we know it has changed (fixes stale data)
|
|
.IP \[bu] 2
|
|
Clean path names before using them in the cache
|
|
.IP \[bu] 2
|
|
Disable cache cleaner if \f[C]\-\-vfs\-cache\-poll\-interval=0\f[]
|
|
.IP \[bu] 2
|
|
Fill and clean the cache immediately on startup
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix Windows opening every file when it stats the file
|
|
.IP \[bu] 2
|
|
Fix applying modtime for an open Write Handle
|
|
.IP \[bu] 2
|
|
Fix creation of files when truncating
|
|
.IP \[bu] 2
|
|
Write 0 bytes when flushing unwritten handles to avoid race conditions
|
|
in FUSE
|
|
.IP \[bu] 2
|
|
Downgrade \[lq]poll\-interval is not supported\[rq] message to Info
|
|
.IP \[bu] 2
|
|
Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Downgrade \[lq]invalid cross\-device link: trying copy\[rq] to debug
|
|
.IP \[bu] 2
|
|
Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy for
|
|
cross device
|
|
.IP \[bu] 2
|
|
Fix race conditions updating the hashes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for polling \- cache will update when remote changes on
|
|
supported backends
|
|
.IP \[bu] 2
|
|
Reduce log level for Plex api
|
|
.IP \[bu] 2
|
|
Fix dir cache issue
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-cache\-db\-wait\-time\f[] flag
|
|
.IP \[bu] 2
|
|
Improve efficiency with RangeOption and RangeSeek
|
|
.IP \[bu] 2
|
|
Fix dirmove with temp fs enabled
|
|
.IP \[bu] 2
|
|
Notify vfs when using temp fs
|
|
.IP \[bu] 2
|
|
Offline uploading
|
|
.IP \[bu] 2
|
|
Remote control support for path flushing
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone no longer has any working keys \- disable integration tests
|
|
.IP \[bu] 2
|
|
Implement DirChangeNotify to notify cache/vfs/mount of changes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azureblob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't check for bucket/container presense if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.IP \[bu] 2
|
|
Improve accounting for chunked uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't check for bucket/container presense if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve accounting for chunked uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix custom oauth client parameters
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't check for bucket/container presense if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Migrate to api v3 (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Add scope configuration and root folder selection
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-impersonate\f[] for service accounts
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
thanks to everyone who tested, explored and contributed docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-created\-date\f[] to use created date as
|
|
modified date (nbuchanan)
|
|
.IP \[bu] 2
|
|
Request the export formats only when required
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This makes rclone quicker when there are no google docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix finding paths with latin1 chars (a workaround for a drive bug)
|
|
.IP \[bu] 2
|
|
Fix copying of a single Google doc file
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-drive\-auth\-owner\-only\f[] to look in all directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix handling of directories with & in
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Removed upload cutoff and always do session uploads
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this stops the creation of multiple versions on business onedrive
|
|
.RE
|
|
.IP \[bu] 2
|
|
Overwrite object size value with real size when reading file.
|
|
(Victor)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes oddities when onedrive misreports the size of images
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove unused chunked upload flag and code
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't check for bucket/container presense if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support hashes for multipart files (Chris Redekop)
|
|
.IP \[bu] 2
|
|
Initial support for IBM COS (S3) (Giri Badanahatti)
|
|
.IP \[bu] 2
|
|
Update docs to discourage use of v2 auth with CEPH and others
|
|
.IP \[bu] 2
|
|
Don't check for bucket/container presense if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix server side copy and set modtime on files with + in
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to disable remote hash check command execution (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-sftp\-ask\-password\f[] flag to prompt for password when
|
|
needed (Leo R.
|
|
Lundgren)
|
|
.IP \[bu] 2
|
|
Add \f[C]set_modtime\f[] configuration option
|
|
.IP \[bu] 2
|
|
Fix following of symlinks
|
|
.IP \[bu] 2
|
|
Fix reading config file outside of Fs setup
|
|
.IP \[bu] 2
|
|
Fix reading $USER in username fallback not $HOME
|
|
.IP \[bu] 2
|
|
Fix running under crontab \- Use correct OS way of reading username
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix refresh of authentication token
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
in v1.39 a bug was introduced which ignored new tokens \- this fixes it
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix extra HEAD transaction when uploading a new file
|
|
.IP \[bu] 2
|
|
Don't check for bucket/container presense if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add new time formats to support mydrive.ch and others
|
|
.RE
|
|
.SS v1.39 \- 2017\-12\-23
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
tested with nextcloud, owncloud, put.io and others!
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.IP \[bu] 2
|
|
cache \- wraps a cache around other backends (Remus Bunduc)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
useful in combination with mount
|
|
.IP \[bu] 2
|
|
NB this feature is in beta so use with care
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve command with subcommands:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve webdav: this implements a webdav server for any rclone remote.
|
|
.IP \[bu] 2
|
|
serve http: command to serve a remote over HTTP
|
|
.RE
|
|
.IP \[bu] 2
|
|
config: add sub commands for full config file management
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
create/delete/dump/edit/file/password/providers/show/update
|
|
.RE
|
|
.IP \[bu] 2
|
|
touch: to create or update the timestamp of a file (Jakub Tasiemski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
curl install for rclone (Filip Bartodziej)
|
|
.IP \[bu] 2
|
|
\[en]stats now shows percentage, size, rate and ETA in condensed form
|
|
(Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
\[en]exclude\-if\-present to exclude a directory if a file is present
|
|
(Iakov Davydov)
|
|
.IP \[bu] 2
|
|
rmdirs: add \[en]leave\-root flag (lewpam)
|
|
.IP \[bu] 2
|
|
move: add \[en]delete\-empty\-src\-dirs flag to remove dirs after move
|
|
(Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
Add \[en]dump flag, introduce \[en]dump requests, responses and remove
|
|
\[en]dump\-auth, \[en]dump\-filters
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obscure X\-Auth\-Token: from headers when dumping too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Document and implement exit codes for different failure modes (Ishuah
|
|
Kariuki)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry lots more different types of errors to make multipart transfers
|
|
more reliable
|
|
.IP \[bu] 2
|
|
Save the config before asking for a token, fixes disappearing oauth
|
|
config
|
|
.IP \[bu] 2
|
|
Warn the user if \[en]include and \[en]exclude are used together (Ernest
|
|
Borowski)
|
|
.IP \[bu] 2
|
|
Fix duplicate files (eg on Google drive) causing spurious copies
|
|
.IP \[bu] 2
|
|
Allow trailing and leading whitespace for passwords (Jason Rose)
|
|
.IP \[bu] 2
|
|
ncdu: fix crashes on empty directories
|
|
.IP \[bu] 2
|
|
rcat: fix goroutine leak
|
|
.IP \[bu] 2
|
|
moveto/copyto: Fix to allow copying to the same name
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[en]vfs\-cache mode to make writes into mounts more reliable.
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this requires caching files on the disk (see \[en]cache\-dir)
|
|
.IP \[bu] 2
|
|
As this is a new feature, use with care
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use sdnotify to signal systemd the mount is ready (Fabian Möller)
|
|
.IP \[bu] 2
|
|
Check if directory is not empty before mounting (Ernest Borowski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add error message for cross file system moves
|
|
.IP \[bu] 2
|
|
Fix equality check for times
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework multipart upload
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
buffer the chunks when uploading large files so they can be retried
|
|
.IP \[bu] 2
|
|
change default chunk size to 48MB now we are buffering them in memory
|
|
.IP \[bu] 2
|
|
retry every error after the first chunk is done successfully
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix error when renaming directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on bad authentication
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service account support (Tim Cooijmans)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make it work properly with Digital Ocean Spaces (Andrew
|
|
Starr\-Bochicchio)
|
|
.IP \[bu] 2
|
|
Fix crash if a bad listing is received
|
|
.IP \[bu] 2
|
|
Add support for ECS task IAM roles (David Minor)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix multipart upload retries
|
|
.IP \[bu] 2
|
|
Fix \[en]hard\-delete to make it work 100% of the time
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow authentication with storage URL and auth key (Giovanni Pizzi)
|
|
.IP \[bu] 2
|
|
Add new fields for swift configuration to support IBM Bluemix Swift
|
|
(Pierre Carlson)
|
|
.IP \[bu] 2
|
|
Add OS_TENANT_ID and OS_USER_ID to config
|
|
.IP \[bu] 2
|
|
Allow configs with user id instead of user name
|
|
.IP \[bu] 2
|
|
Check if swift segments container exists before creating (John Leach)
|
|
.IP \[bu] 2
|
|
Fix memory leak in swift transfers (upstream fix)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to enable the use of aes128\-cbc cipher (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix download of large files failing with \[lq]Only one auth mechanism
|
|
allowed\[rq]
|
|
.RE
|
|
.IP \[bu] 2
|
|
crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Option to encrypt directory names or leave them intact
|
|
.IP \[bu] 2
|
|
Implement DirChangeNotify (Fabian Möller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to choose resourceURL during setup of OneDrive Business
|
|
account if more than one is available for user
|
|
.RE
|
|
.SS v1.38 \- 2017\-09\-30
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure Blob Storage (thanks Andrei Dragomir)
|
|
.IP \[bu] 2
|
|
Box
|
|
.IP \[bu] 2
|
|
Onedrive for Business (thanks Oliver Heyme)
|
|
.IP \[bu] 2
|
|
QingStor from QingCloud (thanks wuyu)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]rcat\f[] \- read from standard input and stream upload
|
|
.IP \[bu] 2
|
|
\f[C]tree\f[] \- shows a nicely formatted recursive listing
|
|
.IP \[bu] 2
|
|
\f[C]cryptdecode\f[] \- decode crypted file names (thanks ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]config\ show\f[] \- print the config file
|
|
.IP \[bu] 2
|
|
\f[C]config\ file\f[] \- print the config file location
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty directories are deleted on \f[C]sync\f[]
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[] \- implement merging of duplicate directories
|
|
.IP \[bu] 2
|
|
\f[C]check\f[] and \f[C]cryptcheck\f[] made more consistent and use less
|
|
memory
|
|
.IP \[bu] 2
|
|
\f[C]cleanup\f[] for remaining remotes (thanks ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-immutable\f[] for ensuring that files don't change (thanks
|
|
Jacob McNamee)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-user\-agent\f[] option (thanks Alex McGrath Kraak)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-disable\f[] flag to disable optional features
|
|
.IP \[bu] 2
|
|
\f[C]\-\-bind\f[] flag for choosing the local addr on outgoing
|
|
connections
|
|
.IP \[bu] 2
|
|
Support for zsh auto\-completion (thanks bpicode)
|
|
.IP \[bu] 2
|
|
Stop normalizing file names but do a normalized compare in \f[C]sync\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to using go1.9 as the default go version
|
|
.IP \[bu] 2
|
|
Remove snapd build due to maintenance problems
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve retriable error detection which makes multipart uploads better
|
|
.IP \[bu] 2
|
|
Make \f[C]check\f[] obey \f[C]\-\-ignore\-size\f[]
|
|
.IP \[bu] 2
|
|
Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
|
|
.IP \[bu] 2
|
|
\f[C]config\f[] ensures newly written config is on the same mount
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Revert to copy when moving file across file system boundaries
|
|
.IP \[bu] 2
|
|
\f[C]\-\-skip\-links\f[] to suppress symlink warnings (thanks Zhiming
|
|
Wang)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-use \f[C]rcat\f[] internals to support uploads from all remotes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \[lq]entry doesn't belong in directory\[rq] error
|
|
.IP \[bu] 2
|
|
Stop using deprecated API methods
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server side copy to empty container with \f[C]\-\-fast\-list\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Change the default for \f[C]\-\-drive\-use\-trash\f[] to \f[C]true\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set session token when using STS (thanks Girish Ramakrishnan)
|
|
.IP \[bu] 2
|
|
Glacier docs and error messages (thanks Jan Varho)
|
|
.IP \[bu] 2
|
|
Read 1000 (not 1024) items in dir listings to fix Wasabi
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SHA1 mismatch when downloading files with no SHA1
|
|
.IP \[bu] 2
|
|
Calculate missing hashes on the fly instead of spooling
|
|
.IP \[bu] 2
|
|
\f[C]\-\-b2\-hard\-delete\f[] to permanently delete (not hide) files
|
|
(thanks John Papandriopoulos)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix creating containers \- no longer have to use the \f[C]default\f[]
|
|
container
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Optionally configure from a standard set of OpenStack environment vars
|
|
.IP \[bu] 2
|
|
Add \f[C]endpoint_type\f[] config
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bucket creation to work with limited permission users
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement connection pooling for multiple ssh connections
|
|
.IP \[bu] 2
|
|
Limit new connections per second
|
|
.IP \[bu] 2
|
|
Add support for MD5 and SHA1 hashes where available (thanks Christian
|
|
Brüggemann)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix URL encoding issues
|
|
.IP \[bu] 2
|
|
Fix directories with \f[C]:\f[] in
|
|
.IP \[bu] 2
|
|
Fix panic with URL encoded content
|
|
.RE
|
|
.SS v1.37 \- 2017\-07\-22
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
FTP \- thanks to Antonio Messina
|
|
.IP \[bu] 2
|
|
HTTP \- thanks to Vasiliy Tolstov
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone ncdu \- for exploring a remote with a text based user interface.
|
|
.IP \[bu] 2
|
|
rclone lsjson \- for listing with a machine readable output
|
|
.IP \[bu] 2
|
|
rclone dbhashsum \- to show Dropbox style hashes of files (local or
|
|
Dropbox)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \[en]fast\-list flag
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This allows remotes to list recursively if they can
|
|
.IP \[bu] 2
|
|
This uses less transactions (important if you pay for them)
|
|
.IP \[bu] 2
|
|
This may or may not be quicker
|
|
.IP \[bu] 2
|
|
This will use more memory as it has to hold the listing in memory
|
|
.IP \[bu] 2
|
|
\[en]old\-sync\-method deprecated \- the remaining uses are covered by
|
|
\[en]fast\-list
|
|
.IP \[bu] 2
|
|
This involved a major re\-write of all the listing code
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \[en]tpslimit and \[en]tpslimit\-burst to limit transactions per
|
|
second
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is useful in conjuction with \f[C]rclone\ mount\f[] to limit
|
|
external apps
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \[en]stats\-log\-level so can see \[en]stats without \-v
|
|
.IP \[bu] 2
|
|
Print password prompts to stderr \- Hraban Luyat
|
|
.IP \[bu] 2
|
|
Warn about duplicate files when syncing
|
|
.IP \[bu] 2
|
|
Oauth improvements
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
allow auth_url and token_url to be set in the config file
|
|
.IP \[bu] 2
|
|
Print redirection URI if using own credentials.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Don't Mkdir at the start of sync to save transactions
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update build to go1.8.3
|
|
.IP \[bu] 2
|
|
Require go1.6 for building rclone
|
|
.IP \[bu] 2
|
|
Compile 386 builds with \[lq]GO386=387\[rq] for maximum compatibility
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix menu selection when no remotes
|
|
.IP \[bu] 2
|
|
Config saving reworked to not kill the file if disk gets full
|
|
.IP \[bu] 2
|
|
Don't delete remote if name does not change while renaming
|
|
.IP \[bu] 2
|
|
moveto, copyto: report transfers and checks as per move and copy
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]local\-no\-unicode\-normalization flag \- Bob Potter
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now supported on Windows using cgofuse and WinFsp \- thanks to Bill
|
|
Zissimopoulos for much help
|
|
.IP \[bu] 2
|
|
Compare checksums on upload/download via FUSE
|
|
.IP \[bu] 2
|
|
Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM \- Jérôme
|
|
Vizcaino
|
|
.IP \[bu] 2
|
|
On read only open of file, make open pending until first read
|
|
.IP \[bu] 2
|
|
Make \[en]read\-only reject modify operations
|
|
.IP \[bu] 2
|
|
Implement ModTime via FUSE for remotes that support it
|
|
.IP \[bu] 2
|
|
Allow modTime to be changed even before all writers are closed
|
|
.IP \[bu] 2
|
|
Fix panic on renames
|
|
.IP \[bu] 2
|
|
Fix hang on errored upload
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Report the name:root as specified by the user
|
|
.IP \[bu] 2
|
|
Add an \[lq]obfuscate\[rq] option for filename encryption \- Stephen
|
|
Harris
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix initialization order for token renewer
|
|
.IP \[bu] 2
|
|
Remove revoked credentials, allow oauth proxy config and update docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce minimum chunk size to 5MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add team drive support
|
|
.IP \[bu] 2
|
|
Reduce bandwidth by adding fields for partial responses \- Martin
|
|
Kristensen
|
|
.IP \[bu] 2
|
|
Implement \[en]drive\-shared\-with\-me flag to view shared with me files
|
|
\- Danny Tsai
|
|
.IP \[bu] 2
|
|
Add \[en]drive\-trashed\-only to read only the files in the trash
|
|
.IP \[bu] 2
|
|
Remove obsolete \[en]drive\-full\-list
|
|
.IP \[bu] 2
|
|
Add missing seek to start on retries of chunked uploads
|
|
.IP \[bu] 2
|
|
Fix stats accounting for upload
|
|
.IP \[bu] 2
|
|
Convert / in names to a unicode equivalent (／)
|
|
.IP \[bu] 2
|
|
Poll for Google Drive changes when mounted
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the uploading of files with spaces
|
|
.IP \[bu] 2
|
|
Fix initialization order for token renewer
|
|
.IP \[bu] 2
|
|
Display speeds accurately when uploading \- Yoni Jah
|
|
.IP \[bu] 2
|
|
Swap to using http://localhost:53682/ as redirect URL \- Michael Ledin
|
|
.IP \[bu] 2
|
|
Retry on token expired error, reset upload body on retry \- Yoni Jah
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to specify location and storage class via config and command
|
|
line \- thanks gdm85
|
|
.IP \[bu] 2
|
|
Create container if necessary on server side copy
|
|
.IP \[bu] 2
|
|
Increase directory listing chunk to 1000 to increase performance
|
|
.IP \[bu] 2
|
|
Obtain a refresh token for GCS \- Steven Lu
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the name reported in log messages (was empty)
|
|
.IP \[bu] 2
|
|
Correct error return for listing empty directory
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rewritten to use the v2 API
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now supports ModTime
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Can only set by uploading the file again
|
|
.IP \[bu] 2
|
|
If you uploaded with an old rclone, rclone may upload everything again
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-size\-only\f[] or \f[C]\-\-checksum\f[] to avoid this
|
|
.RE
|
|
.IP \[bu] 2
|
|
Now supports the Dropbox content hashing scheme
|
|
.IP \[bu] 2
|
|
Now supports low level retries
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around eventual consistency in bucket creation
|
|
.IP \[bu] 2
|
|
Create container if necessary on server side copy
|
|
.IP \[bu] 2
|
|
Add us\-east\-2 (Ohio) and eu\-west\-2 (London) S3 regions \- Zahiar
|
|
Ahmed
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift, Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix zero length directory markers showing in the subdirectory listing
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this caused lots of duplicate transfers
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix paged directory listings
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this caused duplicate directory errors
|
|
.RE
|
|
.IP \[bu] 2
|
|
Create container if necessary on server side copy
|
|
.IP \[bu] 2
|
|
Increase directory listing chunk to 1000 to increase performance
|
|
.IP \[bu] 2
|
|
Make sensible error if the user forgets the container
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for using ssh key files
|
|
.IP \[bu] 2
|
|
Fix under Windows
|
|
.IP \[bu] 2
|
|
Fix ssh agent on Windows
|
|
.IP \[bu] 2
|
|
Adapt to latest version of library \- Igor Kharin
|
|
.RE
|
|
.SS v1.36 \- 2017\-03\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SFTP remote (Jack Schmidt)
|
|
.IP \[bu] 2
|
|
Re\-implement sync routine to work a directory at a time reducing memory
|
|
usage
|
|
.IP \[bu] 2
|
|
Logging revamped to be more inline with rsync \- now much quieter * \-v
|
|
only shows transfers * \-vv is for full debug * \[en]syslog to log to
|
|
syslog on capable platforms
|
|
.IP \[bu] 2
|
|
Implement \[en]backup\-dir and \[en]suffix
|
|
.IP \[bu] 2
|
|
Implement \[en]track\-renames (initial implementation by Bjørn Erik
|
|
Pedersen)
|
|
.IP \[bu] 2
|
|
Add time\-based bandwidth limits (Lukas Loesche)
|
|
.IP \[bu] 2
|
|
rclone cryptcheck: checks integrity of crypt remotes
|
|
.IP \[bu] 2
|
|
Allow all config file variables and options to be set from environment
|
|
variables
|
|
.IP \[bu] 2
|
|
Add \[en]buffer\-size parameter to control buffer size for copy
|
|
.IP \[bu] 2
|
|
Make \[en]delete\-after the default
|
|
.IP \[bu] 2
|
|
Add \[en]ignore\-checksum flag (fixed by Hisham Zarka)
|
|
.IP \[bu] 2
|
|
rclone check: Add \[en]download flag to check all the data, not just
|
|
hashes
|
|
.IP \[bu] 2
|
|
rclone cat: add \[en]head, \[en]tail, \[en]offset, \[en]count and
|
|
\[en]discard
|
|
.IP \[bu] 2
|
|
rclone config: when choosing from a list, allow the value to be entered
|
|
too
|
|
.IP \[bu] 2
|
|
rclone config: allow rename and copy of remotes
|
|
.IP \[bu] 2
|
|
rclone obscure: for generating encrypted passwords for rclone's config
|
|
(T.C.
|
|
Ferguson)
|
|
.IP \[bu] 2
|
|
Comply with XDG Base Directory specification (Dario Giovannetti)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this moves the default location of the config file in a backwards
|
|
compatible way
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release changes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ubuntu snap support (Dedsec1)
|
|
.IP \[bu] 2
|
|
Compile with go 1.8
|
|
.IP \[bu] 2
|
|
MIPS/Linux big and little endian support
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix copyto copying things to the wrong place if the destination dir
|
|
didn't exist
|
|
.IP \[bu] 2
|
|
Fix parsing of remotes in moveto and copyto
|
|
.IP \[bu] 2
|
|
Fix \[en]delete\-before deleting files on copy
|
|
.IP \[bu] 2
|
|
Fix \[en]files\-from with an empty file copying everything
|
|
.IP \[bu] 2
|
|
Fix sync: don't update mod times if \[en]dry\-run set
|
|
.IP \[bu] 2
|
|
Fix MimeType propagation
|
|
.IP \[bu] 2
|
|
Fix filters to add ** rules to directory rules
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \-L, \[en]copy\-links flag to allow rclone to follow symlinks
|
|
.IP \[bu] 2
|
|
Open files in write only mode so rclone can write to an rclone mount
|
|
.IP \[bu] 2
|
|
Fix unnormalised unicode causing problems reading directories
|
|
.IP \[bu] 2
|
|
Fix interaction between \-x flag and \[en]max\-depth
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement proper directory handling (mkdir, rmdir, renaming)
|
|
.IP \[bu] 2
|
|
Make include and exclude filters apply to mount
|
|
.IP \[bu] 2
|
|
Implement read and write async buffers \- control with \[en]buffer\-size
|
|
.IP \[bu] 2
|
|
Fix fsync on for directories
|
|
.IP \[bu] 2
|
|
Fix retry on network failure when reading off crypt
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]crypt\-show\-mapping to show encrypted file mapping
|
|
.IP \[bu] 2
|
|
Fix crypt writer getting stuck in a loop
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]IMPORTANT\f[] this bug had the potential to cause data corruption
|
|
when
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
reading data from a network based remote and
|
|
.IP \[bu] 2
|
|
writing to a crypt on Google Drive
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use the cryptcheck command to validate your data if you are concerned
|
|
.IP \[bu] 2
|
|
If syncing two crypt remotes, sync the unencrypted remote
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix panics on Move (rename)
|
|
.IP \[bu] 2
|
|
Fix panic on token expiry
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix inconsistent listings and rclone check
|
|
.IP \[bu] 2
|
|
Fix uploading empty files with go1.8
|
|
.IP \[bu] 2
|
|
Constrain memory usage when doing multipart uploads
|
|
.IP \[bu] 2
|
|
Fix upload url not being refreshed properly
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Rmdir on directories with trashed files
|
|
.IP \[bu] 2
|
|
Fix \[lq]Ignoring unknown object\[rq] when downloading
|
|
.IP \[bu] 2
|
|
Add \[en]drive\-list\-chunk
|
|
.IP \[bu] 2
|
|
Add \[en]drive\-skip\-gdocs (Károly Oláh)
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Move
|
|
.IP \[bu] 2
|
|
Fix Copy
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix overwrite detection in Copy
|
|
.IP \[bu] 2
|
|
Fix waitForJob to parse errors correctly
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use token renewer to stop auth errors on long uploads
|
|
.IP \[bu] 2
|
|
Fix uploading empty files with go1.8
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix depth 1 directory listings
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix single level directory listing
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Normalise the case for single level directory listings
|
|
.IP \[bu] 2
|
|
Fix depth 1 listing
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added ca\-central\-1 region (Jon Yergatian)
|
|
.RE
|
|
.SS v1.35 \- 2017\-01\-02
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
moveto and copyto commands for choosing a destination name on copy/move
|
|
.IP \[bu] 2
|
|
rmdirs command to recursively delete empty directories
|
|
.IP \[bu] 2
|
|
Allow repeated \[en]include/\[en]exclude/\[en]filter options
|
|
.IP \[bu] 2
|
|
Only show transfer stats on commands which transfer stuff
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
show stats on any command using the \f[C]\-\-stats\f[] flag
|
|
.RE
|
|
.IP \[bu] 2
|
|
Allow overlapping directories in move when server side dir move is
|
|
supported
|
|
.IP \[bu] 2
|
|
Add \[en]stats\-unit option \- thanks Scott McGillivray
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the config file being overwritten when two rclones are running
|
|
.IP \[bu] 2
|
|
Make rclone lsd obey the filters properly
|
|
.IP \[bu] 2
|
|
Fix compilation on mips
|
|
.IP \[bu] 2
|
|
Fix not transferring files that don't differ in size
|
|
.IP \[bu] 2
|
|
Fix panic on nil retry/fatal error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry reads on error \- should help with reliability a lot
|
|
.IP \[bu] 2
|
|
Report the modification times for directories from the remote
|
|
.IP \[bu] 2
|
|
Add bandwidth accounting and limiting (fixes \[en]bwlimit)
|
|
.IP \[bu] 2
|
|
If \[en]stats provided will show stats and which files are transferring
|
|
.IP \[bu] 2
|
|
Support R/W files if truncate is set.
|
|
.IP \[bu] 2
|
|
Implement statfs interface so df works
|
|
.IP \[bu] 2
|
|
Note that write is now supported on Amazon Drive
|
|
.IP \[bu] 2
|
|
Report number of blocks in a file \- thanks Stefan Breunig
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Prevent the user pointing crypt at itself
|
|
.IP \[bu] 2
|
|
Fix failed to authenticate decrypted block errors
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
these will now return the underlying unexpected EOF instead
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for server side move and directory move \- thanks Stefan
|
|
Breunig
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref on size attribute
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new prefix and delimiter parameters in directory listings
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This makes \[en]max\-depth 1 dir listings as used in mount much faster
|
|
.RE
|
|
.IP \[bu] 2
|
|
Reauth the account while doing uploads too \- should help with token
|
|
expiry
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make DirMove more efficient and complain about moving the root
|
|
.IP \[bu] 2
|
|
Create destination directory on Move()
|
|
.RE
|
|
.SS v1.34 \- 2016\-11\-06
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop single file and \f[C]\-\-files\-from\f[] operations iterating
|
|
through the source bucket.
|
|
.IP \[bu] 2
|
|
Stop removing failed upload to cloud storage remotes
|
|
.IP \[bu] 2
|
|
Make ContentType be preserved for cloud to cloud copies
|
|
.IP \[bu] 2
|
|
Add support to toggle bandwidth limits via SIGUSR2 \- thanks Marco
|
|
Paganini
|
|
.IP \[bu] 2
|
|
\f[C]rclone\ check\f[] shows count of hashes that couldn't be checked
|
|
.IP \[bu] 2
|
|
\f[C]rclone\ listremotes\f[] command
|
|
.IP \[bu] 2
|
|
Support linux/arm64 build \- thanks Fredrik Fornwall
|
|
.IP \[bu] 2
|
|
Remove \f[C]Authorization:\f[] lines from \f[C]\-\-dump\-headers\f[]
|
|
output
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore files with control characters in the names
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone\ move\f[] command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Delete src files which already existed in dst
|
|
.IP \[bu] 2
|
|
Fix deletion of src file when dst file older
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone\ check\f[] on crypted file systems
|
|
.IP \[bu] 2
|
|
Make failed uploads not count as \[lq]Transferred\[rq]
|
|
.IP \[bu] 2
|
|
Make sure high level retries show with \f[C]\-q\f[]
|
|
.IP \[bu] 2
|
|
Use a vendor directory with godep for repeatable builds
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone\ mount\f[] \- FUSE
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement FUSE mount options
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-no\-modtime\f[], \f[C]\-\-debug\-fuse\f[],
|
|
\f[C]\-\-read\-only\f[], \f[C]\-\-allow\-non\-empty\f[],
|
|
\f[C]\-\-allow\-root\f[], \f[C]\-\-allow\-other\f[]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-default\-permissions\f[], \f[C]\-\-write\-back\-cache\f[],
|
|
\f[C]\-\-max\-read\-ahead\f[], \f[C]\-\-umask\f[], \f[C]\-\-uid\f[],
|
|
\f[C]\-\-gid\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dir\-cache\-time\f[] to control caching of directory
|
|
entries
|
|
.IP \[bu] 2
|
|
Implement seek for files opened for read (useful for video players)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
with \f[C]\-no\-seek\f[] flag to disable
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix crash on 32 bit ARM (alignment of 64 bit counter)
|
|
.IP \[bu] 2
|
|
\&...and many more internal fixes and improvements!
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't show encrypted password in configurator to stop confusion
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
New wait for upload option \f[C]\-\-acd\-upload\-wait\-per\-gb\f[]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
upload timeouts scale by file size and can be disabled
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add 502 Bad Gateway to list of errors we retry
|
|
.IP \[bu] 2
|
|
Fix overwriting a file with a zero length file
|
|
.IP \[bu] 2
|
|
Fix ACD file size warning limit \- thanks Felix Bünemann
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Unix: implement \f[C]\-x\f[]/\f[C]\-\-one\-file\-system\f[] to stay on a
|
|
single file system
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
|
|
.RE
|
|
.IP \[bu] 2
|
|
Windows: ignore the symlink bit on files
|
|
.IP \[bu] 2
|
|
Windows: Ignore directory based junction points
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make sure each upload has at least one upload slot \- fixes strange
|
|
upload stats
|
|
.IP \[bu] 2
|
|
Fix uploads when using crypt
|
|
.IP \[bu] 2
|
|
Fix download of large files (sha1 mismatch)
|
|
.IP \[bu] 2
|
|
Return error when we try to create a bucket which someone else owns
|
|
.IP \[bu] 2
|
|
Update B2 docs with Data usage, and Crypt section \- thanks Tomasz Mazur
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Command line and config file support for
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Setting/overriding ACL \- thanks Radek Senfeld
|
|
.IP \[bu] 2
|
|
Setting storage class \- thanks Asko Tamm
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make exponential backoff work exactly as per Google specification
|
|
.IP \[bu] 2
|
|
add \f[C]\&.epub\f[], \f[C]\&.odp\f[] and \f[C]\&.tsv\f[] as export
|
|
formats.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't read metadata for directory marker objects
|
|
.RE
|
|
.SS v1.33 \- 2016\-08\-24
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement encryption
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
data encrypted in NACL secretbox format
|
|
.IP \[bu] 2
|
|
with optional file name encryption
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone mount \- implements FUSE mounting of remotes (EXPERIMENTAL)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
works on Linux, FreeBSD and OS X (need testers for the last 2!)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rclone cat \- outputs remote file or files to the terminal
|
|
.IP \[bu] 2
|
|
rclone genautocomplete \- command to make a bash completion script for
|
|
rclone
|
|
.RE
|
|
.IP \[bu] 2
|
|
Editing a remote using \f[C]rclone\ config\f[] now goes through the
|
|
wizard
|
|
.IP \[bu] 2
|
|
Compile with go 1.7 \- this fixes rclone on macOS Sierra and on 386
|
|
processors
|
|
.IP \[bu] 2
|
|
Use cobra for sub commands and docs generation
|
|
.RE
|
|
.IP \[bu] 2
|
|
drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Document how to make your own client_id
|
|
.RE
|
|
.IP \[bu] 2
|
|
s3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
User\-configurable Amazon S3 ACL (thanks Radek Šenfeld)
|
|
.RE
|
|
.IP \[bu] 2
|
|
b2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix stats accounting for upload \- no more jumping to 100% done
|
|
.IP \[bu] 2
|
|
On cleanup delete hide marker if it is the current file
|
|
.IP \[bu] 2
|
|
New B2 API endpoint (thanks Per Cederberg)
|
|
.IP \[bu] 2
|
|
Set maximum backoff to 5 Minutes
|
|
.RE
|
|
.IP \[bu] 2
|
|
onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix URL escaping in file names \- eg uploading files with \f[C]+\f[] in
|
|
them.
|
|
.RE
|
|
.IP \[bu] 2
|
|
amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token expiry during large uploads
|
|
.IP \[bu] 2
|
|
Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
|
|
.RE
|
|
.IP \[bu] 2
|
|
local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix filenames with invalid UTF\-8 not being uploaded
|
|
.IP \[bu] 2
|
|
Fix problem with some UTF\-8 characters on OS X
|
|
.RE
|
|
.SS v1.32 \- 2016\-07\-13
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of files large files not in root
|
|
.RE
|
|
.SS v1.31 \- 2016\-07\-13
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce memory on sync by about 50%
|
|
.IP \[bu] 2
|
|
Implement \[en]no\-traverse flag to stop copy traversing the destination
|
|
remote.
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This can be used to reduce memory usage down to the smallest possible.
|
|
.IP \[bu] 2
|
|
Useful to copy a small number of files into a large destination folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement cleanup command for emptying trash / removing old versions of
|
|
files
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Currently B2 only
|
|
.RE
|
|
.IP \[bu] 2
|
|
Single file handling improved
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now copied with \[en]files\-from
|
|
.IP \[bu] 2
|
|
Automatically sets \[en]no\-traverse when copying a single file
|
|
.RE
|
|
.IP \[bu] 2
|
|
Info on using installing with ansible \- thanks Stefan Weichinger
|
|
.IP \[bu] 2
|
|
Implement \[en]no\-update\-modtime flag to stop rclone fixing the remote
|
|
modified times.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix move command \- stop it running for overlapping Fses \- this was
|
|
causing data loss.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix incomplete hashes \- this was causing problems for B2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rename Amazon Cloud Drive to Amazon Drive \- no changes to config file
|
|
needed.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for non\-default project domain \- thanks Antonio Messina.
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add instructions on how to use rclone with minio.
|
|
.IP \[bu] 2
|
|
Add ap\-northeast\-2 (Seoul) and ap\-south\-1 (Mumbai) regions.
|
|
.IP \[bu] 2
|
|
Skip setting the modified time for objects > 5GB as it isn't possible.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]b2\-versions flag so old versions can be listed and retreived.
|
|
.IP \[bu] 2
|
|
Treat 403 errors (eg cap exceeded) as fatal.
|
|
.IP \[bu] 2
|
|
Implement cleanup command for deleting old file versions.
|
|
.IP \[bu] 2
|
|
Make error handling compliant with B2 integrations notes.
|
|
.IP \[bu] 2
|
|
Fix handling of token expiry.
|
|
.IP \[bu] 2
|
|
Implement \[en]b2\-test\-mode to set \f[C]X\-Bz\-Test\-Mode\f[] header.
|
|
.IP \[bu] 2
|
|
Set cutoff for chunked upload to 200MB as per B2 guidelines.
|
|
.IP \[bu] 2
|
|
Make upload multi\-threaded.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't retry 461 errors.
|
|
.RE
|
|
.SS v1.30 \- 2016\-06\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Directory listing code reworked for more features and better error
|
|
reporting (thanks to Klaus Post for help).
|
|
This enables
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Directory include filtering for efficiency
|
|
.IP \[bu] 2
|
|
\[en]max\-depth parameter
|
|
.IP \[bu] 2
|
|
Better error reporting
|
|
.IP \[bu] 2
|
|
More to come
|
|
.RE
|
|
.IP \[bu] 2
|
|
Retry more errors
|
|
.IP \[bu] 2
|
|
Add \[en]ignore\-size flag \- for uploading images to onedrive
|
|
.IP \[bu] 2
|
|
Log \-v output to stdout by default
|
|
.IP \[bu] 2
|
|
Display the transfer stats in more human readable form
|
|
.IP \[bu] 2
|
|
Make 0 size files specifiable with \f[C]\-\-max\-size\ 0b\f[]
|
|
.IP \[bu] 2
|
|
Add \f[C]b\f[] suffix so we can specify bytes in \[en]bwlimit,
|
|
\[en]min\-size etc
|
|
.IP \[bu] 2
|
|
Use \[lq]password:\[rq] instead of \[lq]password>\[rq] prompt \- thanks
|
|
Klaus Post and Leigh Klotz
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix retry doing one too many retries
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix problems with OS X and UTF\-8 characters
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check a file exists before uploading to help with 408 Conflict errors
|
|
.IP \[bu] 2
|
|
Reauth on 401 errors \- this has been causing a lot of problems
|
|
.IP \[bu] 2
|
|
Work around spurious 403 errors
|
|
.IP \[bu] 2
|
|
Restart directory listings on error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check a file exists before uploading to help with duplicates
|
|
.IP \[bu] 2
|
|
Fix retry of multipart uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement large file uploading
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add AES256 server\-side encryption for \- thanks Justin R.
|
|
Wilson
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make sure we don't use conflicting content types on upload
|
|
.IP \[bu] 2
|
|
Add service account support \- thanks Michal Witkowski
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add auth version parameter
|
|
.IP \[bu] 2
|
|
Add domain option for openstack (v3 auth) \- thanks Fabian Ruff
|
|
.RE
|
|
.SS v1.29 \- 2016\-04\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-I,\ \-\-ignore\-times\f[] for unconditional upload
|
|
.IP \[bu] 2
|
|
Improve \f[C]dedupe\f[]command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now removes identical copies without asking
|
|
.IP \[bu] 2
|
|
Now obeys \f[C]\-\-dry\-run\f[]
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-dedupe\-mode\f[] for non interactive running
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ interactive\f[] \- interactive the default.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ skip\f[] \- removes identical files then skips
|
|
anything left.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ first\f[] \- removes identical files then keeps
|
|
the first one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ newest\f[] \- removes identical files then keeps
|
|
the newest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ oldest\f[] \- removes identical files then keeps
|
|
the oldest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode\ rename\f[] \- removes identical files then
|
|
renames the rest to be different.
|
|
.RE
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make rclone check obey the \f[C]\-\-size\-only\f[] flag.
|
|
.IP \[bu] 2
|
|
Use \[lq]application/octet\-stream\[rq] if discovered mime type is
|
|
invalid.
|
|
.IP \[bu] 2
|
|
Fix missing \[lq]quit\[rq] option when there are no remotes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase default chunk size to 8 MB \- increases upload speed of big
|
|
files
|
|
.IP \[bu] 2
|
|
Speed up directory listings and make more reliable
|
|
.IP \[bu] 2
|
|
Add missing retries for Move and DirMove \- increases reliability
|
|
.IP \[bu] 2
|
|
Preserve mime type on file update
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable mod time syncing
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This means that B2 will now check modification times
|
|
.IP \[bu] 2
|
|
It will upload new files to update the modification times
|
|
.IP \[bu] 2
|
|
(there isn't an API to just set the mod time.)
|
|
.IP \[bu] 2
|
|
If you want the old behaviour use \f[C]\-\-size\-only\f[].
|
|
.RE
|
|
.IP \[bu] 2
|
|
Update API to new version
|
|
.IP \[bu] 2
|
|
Fix parsing of mod time when not in metadata
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift/Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't return an MD5SUM for static large objects
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploading files bigger than 50GB
|
|
.RE
|
|
.SS v1.28 \- 2016\-03\-01
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Configuration file encryption \- thanks Klaus Post
|
|
.IP \[bu] 2
|
|
Improve \f[C]rclone\ config\f[] adding more help and making it easier to
|
|
understand
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-u\f[]/\f[C]\-\-update\f[] so creation times can be used
|
|
on all remotes
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-low\-level\-retries\f[] flag
|
|
.IP \[bu] 2
|
|
Optionally disable gzip compression on downloads with
|
|
\f[C]\-\-no\-gzip\-encoding\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't make directories if \f[C]\-\-dry\-run\f[] set
|
|
.IP \[bu] 2
|
|
Fix and document the \f[C]move\f[] command
|
|
.IP \[bu] 2
|
|
Fix redirecting stderr on unix\-like OSes when using
|
|
\f[C]\-\-log\-file\f[]
|
|
.IP \[bu] 2
|
|
Fix \f[C]delete\f[] command to wait until all finished \- fixes missing
|
|
deletes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use one upload URL per go routine fixes
|
|
\f[C]more\ than\ one\ upload\ using\ auth\ token\f[]
|
|
.IP \[bu] 2
|
|
Add pacing, retries and reauthentication \- fixes token expiry problems
|
|
.IP \[bu] 2
|
|
Upload without using a temporary file from local (and remotes which
|
|
support SHA1)
|
|
.IP \[bu] 2
|
|
Fix reading metadata for all files when it shouldn't have been
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing drive documents at root
|
|
.IP \[bu] 2
|
|
Disable copy and move for Google docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploading of chunked files with non ASCII characters
|
|
.IP \[bu] 2
|
|
Allow setting of \f[C]storage_url\f[] in the config \- thanks Xavier
|
|
Lucas
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow IAM role and credentials from environment variables \- thanks
|
|
Brian Stengaard
|
|
.IP \[bu] 2
|
|
Allow low privilege users to use S3 (check if directory exists during
|
|
Mkdir) \- thanks Jakub Gedeon
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry on more things to make directory listings more reliable
|
|
.RE
|
|
.SS v1.27 \- 2016\-01\-31
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Easier headless configuration with \f[C]rclone\ authorize\f[]
|
|
.IP \[bu] 2
|
|
Add support for multiple hash types \- we now check SHA1 as well as MD5
|
|
hashes.
|
|
.IP \[bu] 2
|
|
\f[C]delete\f[] command which does obey the filters (unlike
|
|
\f[C]purge\f[])
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[] command to deduplicate a remote.
|
|
Useful with Google Drive.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ignore\-existing\f[] flag to skip all files that exist on
|
|
destination.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-delete\-before\f[], \f[C]\-\-delete\-during\f[],
|
|
\f[C]\-\-delete\-after\f[] flags.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-memprofile\f[] flag to debug memory use.
|
|
.IP \[bu] 2
|
|
Warn the user about files with same name but different case
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-include\f[] rules add their implict exclude * at the end
|
|
of the filter list
|
|
.IP \[bu] 2
|
|
Deprecate compiling with go1.3
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix download of files > 10 GB
|
|
.IP \[bu] 2
|
|
Fix directory traversal (\[lq]Next token is expired\[rq]) for large
|
|
directory listings
|
|
.IP \[bu] 2
|
|
Remove 409 conflict from error codes we will retry \- stops very long
|
|
pauses
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 hashes now checked by rclone core
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-auth\-owner\-only\f[] to only consider files owned
|
|
by the user \- thanks Björn Harrtell
|
|
.IP \[bu] 2
|
|
Export Google documents
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make file exclusion error controllable with \-q
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload from unprivileged user.
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix updating of mod times of files with \f[C]+\f[] in.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add local file system option to disable UNC on Windows.
|
|
.RE
|
|
.SS v1.26 \- 2016\-01\-02
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Yandex storage backend \- thank you Dmitry Burdeev (\[lq]dibu\[rq])
|
|
.IP \[bu] 2
|
|
Implement Backblaze B2 storage backend
|
|
.IP \[bu] 2
|
|
Add \[en]min\-age and \[en]max\-age flags \- thank you Adriano Aurélio
|
|
Meirelles
|
|
.IP \[bu] 2
|
|
Make ls/lsl/md5sum/size/check obey includes and excludes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash in http logging
|
|
.IP \[bu] 2
|
|
Upload releases to github too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sync for chunked files
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-enable server side copy
|
|
.IP \[bu] 2
|
|
Don't mask HTTP error codes with JSON decode error
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix corrupting Content\-Type on mod time update (thanks Joseph Spurrier)
|
|
.RE
|
|
.SS v1.25 \- 2015\-11\-14
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Hubic storage system
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix deletion of some excluded files without \[en]delete\-excluded
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This could have deleted files unexpectedly on sync
|
|
.IP \[bu] 2
|
|
Always check first with \f[C]\-\-dry\-run\f[]!
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop SetModTime losing metadata (eg X\-Object\-Manifest)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This could have caused data loss for files > 5GB in size
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use ContentType from Object to avoid lookups in listings
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
disable server side copy as it seems to be broken at Microsoft
|
|
.RE
|
|
.SS v1.24 \- 2015\-11\-07
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for Microsoft OneDrive
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-no\-check\-certificate\f[] option to disable server
|
|
certificate verification
|
|
.IP \[bu] 2
|
|
Add async readahead buffer for faster transfer of big files
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow spaces in remotes and check remote names for validity at creation
|
|
time
|
|
.IP \[bu] 2
|
|
Allow `&' and disallow `:' in Windows filenames.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore directory marker objects where appropriate \- allows working with
|
|
Hubic
|
|
.IP \[bu] 2
|
|
Don't delete the container if fs wasn't at root
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't delete the bucket if fs wasn't at root
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don't delete the bucket if fs wasn't at root
|
|
.RE
|
|
.SS v1.23 \- 2015\-10\-03
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone\ size\f[] for measuring remotes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix headless config for drive and gcs
|
|
.IP \[bu] 2
|
|
Tell the user they should try again if the webserver method failed
|
|
.IP \[bu] 2
|
|
Improve output of \f[C]\-\-dump\-headers\f[]
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow anonymous access to public buckets
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop chunked operations logging \[lq]Failed to read info: Object Not
|
|
Found\[rq]
|
|
.IP \[bu] 2
|
|
Use Content\-Length on uploads for extra reliability
|
|
.RE
|
|
.SS v1.22 \- 2015\-09\-28
|
|
.IP \[bu] 2
|
|
Implement rsync like include and exclude flags
|
|
.IP \[bu] 2
|
|
swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support files > 5GB \- thanks Sergey Tolmachev
|
|
.RE
|
|
.SS v1.21 \- 2015\-09\-22
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Display individual transfer progress
|
|
.IP \[bu] 2
|
|
Make lsl output times in localtime
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix allowing user to override credentials again in Drive, GCS and ACD
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement compliant pacing scheme
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make directory reads concurrent for increased speed.
|
|
.RE
|
|
.SS v1.20 \- 2015\-09\-15
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amazon Drive support
|
|
.IP \[bu] 2
|
|
Oauth support redone \- fix many bugs and improve usability
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use \[lq]golang.org/x/oauth2\[rq] as oauth libary of choice
|
|
.IP \[bu] 2
|
|
Improve oauth usability for smoother initial signup
|
|
.IP \[bu] 2
|
|
drive, googlecloudstorage: optionally use auto config for the oauth
|
|
token
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \[en]dump\-headers and \[en]dump\-bodies debug flags
|
|
.IP \[bu] 2
|
|
Show multiple matched commands if abbreviation too short
|
|
.IP \[bu] 2
|
|
Implement server side move where possible
|
|
.RE
|
|
.IP \[bu] 2
|
|
local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Always use UNC paths internally on Windows \- fixes a lot of bugs
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
force use of our custom transport which makes timeouts work
|
|
.RE
|
|
.IP \[bu] 2
|
|
Thanks to Klaus Post for lots of help with this release
|
|
.SS v1.19 \- 2015\-08\-28
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Server side copies for s3/swift/drive/dropbox/gcs
|
|
.IP \[bu] 2
|
|
Move command \- uses server side copies if it can
|
|
.IP \[bu] 2
|
|
Implement \[en]retries flag \- tries 3 times by default
|
|
.IP \[bu] 2
|
|
Build for plan9/amd64 and solaris/amd64 too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make a current version download with a fixed URL for scripting
|
|
.IP \[bu] 2
|
|
Ignore rmdir in limited fs rather than throwing error
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase chunk size to improve upload speeds massively
|
|
.IP \[bu] 2
|
|
Issue an error message when trying to upload bad file name
|
|
.RE
|
|
.SS v1.18 \- 2015\-08\-17
|
|
.IP \[bu] 2
|
|
drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-trash\f[] flag so rclone trashes instead of
|
|
deletes
|
|
.IP \[bu] 2
|
|
Add \[lq]Forbidden to download\[rq] message for files with no
|
|
downloadURL
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove datastore
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This was deprecated and it caused a lot of problems
|
|
.IP \[bu] 2
|
|
Modification times and MD5SUMs no longer stored
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix uploading files > 2GB
|
|
.RE
|
|
.IP \[bu] 2
|
|
s3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
use official AWS SDK from github.com/aws/aws\-sdk\-go
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[] will most likely require you to delete and recreate remote
|
|
.IP \[bu] 2
|
|
enable multipart upload which enables files > 5GB
|
|
.IP \[bu] 2
|
|
tested with Ceph / RadosGW / S3 emulation
|
|
.IP \[bu] 2
|
|
many thanks to Sam Liston and Brian Haymore at the Utah Center for High
|
|
Performance Computing (https://www.chpc.utah.edu/) for a Ceph test
|
|
account
|
|
.RE
|
|
.IP \[bu] 2
|
|
misc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Show errors when reading the config file
|
|
.IP \[bu] 2
|
|
Do not print stats in quiet mode \- thanks Leonid Shalupov
|
|
.IP \[bu] 2
|
|
Add FAQ
|
|
.IP \[bu] 2
|
|
Fix created directories not obeying umask
|
|
.IP \[bu] 2
|
|
Linux installation instructions \- thanks Shimon Doodkin
|
|
.RE
|
|
.SS v1.17 \- 2015\-06\-14
|
|
.IP \[bu] 2
|
|
dropbox: fix case insensitivity issues \- thanks Leonid Shalupov
|
|
.SS v1.16 \- 2015\-06\-09
|
|
.IP \[bu] 2
|
|
Fix uploading big files which was causing timeouts or panics
|
|
.IP \[bu] 2
|
|
Don't check md5sum after download with \[en]size\-only
|
|
.SS v1.15 \- 2015\-06\-06
|
|
.IP \[bu] 2
|
|
Add \[en]checksum flag to only discard transfers by MD5SUM \- thanks
|
|
Alex Couper
|
|
.IP \[bu] 2
|
|
Implement \[en]size\-only flag to sync on size not checksum & modtime
|
|
.IP \[bu] 2
|
|
Expand docs and remove duplicated information
|
|
.IP \[bu] 2
|
|
Document rclone's limitations with directories
|
|
.IP \[bu] 2
|
|
dropbox: update docs about case insensitivity
|
|
.SS v1.14 \- 2015\-05\-21
|
|
.IP \[bu] 2
|
|
local: fix encoding of non utf\-8 file names \- fixes a duplicate file
|
|
problem
|
|
.IP \[bu] 2
|
|
drive: docs about rate limiting
|
|
.IP \[bu] 2
|
|
google cloud storage: Fix compile after API change in
|
|
\[lq]google.golang.org/api/storage/v1\[rq]
|
|
.SS v1.13 \- 2015\-05\-10
|
|
.IP \[bu] 2
|
|
Revise documentation (especially sync)
|
|
.IP \[bu] 2
|
|
Implement \[en]timeout and \[en]conntimeout
|
|
.IP \[bu] 2
|
|
s3: ignore etags from multipart uploads which aren't md5sums
|
|
.SS v1.12 \- 2015\-03\-15
|
|
.IP \[bu] 2
|
|
drive: Use chunked upload for files above a certain size
|
|
.IP \[bu] 2
|
|
drive: add \[en]drive\-chunk\-size and \[en]drive\-upload\-cutoff
|
|
parameters
|
|
.IP \[bu] 2
|
|
drive: switch to insert from update when a failed copy deletes the
|
|
upload
|
|
.IP \[bu] 2
|
|
core: Log duplicate files if they are detected
|
|
.SS v1.11 \- 2015\-03\-04
|
|
.IP \[bu] 2
|
|
swift: add region parameter
|
|
.IP \[bu] 2
|
|
drive: fix crash on failed to update remote mtime
|
|
.IP \[bu] 2
|
|
In remote paths, change native directory separators to /
|
|
.IP \[bu] 2
|
|
Add synchronization to ls/lsl/lsd output to stop corruptions
|
|
.IP \[bu] 2
|
|
Ensure all stats/log messages to go stderr
|
|
.IP \[bu] 2
|
|
Add \[en]log\-file flag to log everything (including panics) to file
|
|
.IP \[bu] 2
|
|
Make it possible to disable stats printing with \[en]stats=0
|
|
.IP \[bu] 2
|
|
Implement \[en]bwlimit to limit data transfer bandwidth
|
|
.SS v1.10 \- 2015\-02\-12
|
|
.IP \[bu] 2
|
|
s3: list an unlimited number of items
|
|
.IP \[bu] 2
|
|
Fix getting stuck in the configurator
|
|
.SS v1.09 \- 2015\-02\-07
|
|
.IP \[bu] 2
|
|
windows: Stop drive letters (eg C:) getting mixed up with remotes (eg
|
|
drive:)
|
|
.IP \[bu] 2
|
|
local: Fix directory separators on Windows
|
|
.IP \[bu] 2
|
|
drive: fix rate limit exceeded errors
|
|
.SS v1.08 \- 2015\-02\-04
|
|
.IP \[bu] 2
|
|
drive: fix subdirectory listing to not list entire drive
|
|
.IP \[bu] 2
|
|
drive: Fix SetModTime
|
|
.IP \[bu] 2
|
|
dropbox: adapt code to recent library changes
|
|
.SS v1.07 \- 2014\-12\-23
|
|
.IP \[bu] 2
|
|
google cloud storage: fix memory leak
|
|
.SS v1.06 \- 2014\-12\-12
|
|
.IP \[bu] 2
|
|
Fix \[lq]Couldn't find home directory\[rq] on OSX
|
|
.IP \[bu] 2
|
|
swift: Add tenant parameter
|
|
.IP \[bu] 2
|
|
Use new location of Google API packages
|
|
.SS v1.05 \- 2014\-08\-09
|
|
.IP \[bu] 2
|
|
Improved tests and consequently lots of minor fixes
|
|
.IP \[bu] 2
|
|
core: Fix race detected by go race detector
|
|
.IP \[bu] 2
|
|
core: Fixes after running errcheck
|
|
.IP \[bu] 2
|
|
drive: reset root directory on Rmdir and Purge
|
|
.IP \[bu] 2
|
|
fs: Document that Purger returns error on empty directory, test and fix
|
|
.IP \[bu] 2
|
|
google cloud storage: fix ListDir on subdirectory
|
|
.IP \[bu] 2
|
|
google cloud storage: re\-read metadata in SetModTime
|
|
.IP \[bu] 2
|
|
s3: make reading metadata more reliable to work around eventual
|
|
consistency problems
|
|
.IP \[bu] 2
|
|
s3: strip trailing / from ListDir()
|
|
.IP \[bu] 2
|
|
swift: return directories without / in ListDir
|
|
.SS v1.04 \- 2014\-07\-21
|
|
.IP \[bu] 2
|
|
google cloud storage: Fix crash on Update
|
|
.SS v1.03 \- 2014\-07\-20
|
|
.IP \[bu] 2
|
|
swift, s3, dropbox: fix updated files being marked as corrupted
|
|
.IP \[bu] 2
|
|
Make compile with go 1.1 again
|
|
.SS v1.02 \- 2014\-07\-19
|
|
.IP \[bu] 2
|
|
Implement Dropbox remote
|
|
.IP \[bu] 2
|
|
Implement Google Cloud Storage remote
|
|
.IP \[bu] 2
|
|
Verify Md5sums and Sizes after copies
|
|
.IP \[bu] 2
|
|
Remove times from \[lq]ls\[rq] command \- lists sizes only
|
|
.IP \[bu] 2
|
|
Add add \[lq]lsl\[rq] \- lists times and sizes
|
|
.IP \[bu] 2
|
|
Add \[lq]md5sum\[rq] command
|
|
.SS v1.01 \- 2014\-07\-04
|
|
.IP \[bu] 2
|
|
drive: fix transfer of big files using up lots of memory
|
|
.SS v1.00 \- 2014\-07\-03
|
|
.IP \[bu] 2
|
|
drive: fix whole second dates
|
|
.SS v0.99 \- 2014\-06\-26
|
|
.IP \[bu] 2
|
|
Fix \[en]dry\-run not working
|
|
.IP \[bu] 2
|
|
Make compatible with go 1.1
|
|
.SS v0.98 \- 2014\-05\-30
|
|
.IP \[bu] 2
|
|
s3: Treat missing Content\-Length as 0 for some ceph installations
|
|
.IP \[bu] 2
|
|
rclonetest: add file with a space in
|
|
.SS v0.97 \- 2014\-05\-05
|
|
.IP \[bu] 2
|
|
Implement copying of single files
|
|
.IP \[bu] 2
|
|
s3 & swift: support paths inside containers/buckets
|
|
.SS v0.96 \- 2014\-04\-24
|
|
.IP \[bu] 2
|
|
drive: Fix multiple files of same name being created
|
|
.IP \[bu] 2
|
|
drive: Use o.Update and fs.Put to optimise transfers
|
|
.IP \[bu] 2
|
|
Add version number, \-V and \[en]version
|
|
.SS v0.95 \- 2014\-03\-28
|
|
.IP \[bu] 2
|
|
rclone.org: website, docs and graphics
|
|
.IP \[bu] 2
|
|
drive: fix path parsing
|
|
.SS v0.94 \- 2014\-03\-27
|
|
.IP \[bu] 2
|
|
Change remote format one last time
|
|
.IP \[bu] 2
|
|
GNU style flags
|
|
.SS v0.93 \- 2014\-03\-16
|
|
.IP \[bu] 2
|
|
drive: store token in config file
|
|
.IP \[bu] 2
|
|
cross compile other versions
|
|
.IP \[bu] 2
|
|
set strict permissions on config file
|
|
.SS v0.92 \- 2014\-03\-15
|
|
.IP \[bu] 2
|
|
Config fixes and \[en]config option
|
|
.SS v0.91 \- 2014\-03\-15
|
|
.IP \[bu] 2
|
|
Make config file
|
|
.SS v0.90 \- 2013\-06\-27
|
|
.IP \[bu] 2
|
|
Project named rclone
|
|
.SS v0.00 \- 2012\-11\-18
|
|
.IP \[bu] 2
|
|
Project started
|
|
.SH Bugs and Limitations
|
|
.SS Limitations
|
|
.SS Directory timestamps aren't preserved
|
|
.PP
|
|
Rclone doesn't currently preserve the timestamps of directories.
|
|
This is because rclone only really considers objects when syncing.
|
|
.SS Rclone struggles with millions of files in a directory
|
|
.PP
|
|
Currently rclone loads each directory entirely into memory before using
|
|
it.
|
|
Since each Rclone object takes 0.5k\-1k of memory this can take a very
|
|
long time and use an extremely large amount of memory.
|
|
.PP
|
|
Millions of files in a directory tend caused by software writing cloud
|
|
storage (eg S3 buckets).
|
|
.SS Bucket based remotes and folders
|
|
.PP
|
|
Bucket based remotes (eg S3/GCS/Swift/B2) do not have a concept of
|
|
directories.
|
|
Rclone therefore cannot create directories in them which means that
|
|
empty directories on a bucket based remote will tend to disappear.
|
|
.PP
|
|
Some software creates empty keys ending in \f[C]/\f[] as directory
|
|
markers.
|
|
Rclone doesn't do this as it potentially creates more objects and costs
|
|
more.
|
|
It may do in future (probably with a flag).
|
|
.SS Bugs
|
|
.PP
|
|
Bugs are stored in rclone's GitHub project:
|
|
.IP \[bu] 2
|
|
Reported
|
|
bugs (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+label%3Abug)
|
|
.IP \[bu] 2
|
|
Known
|
|
issues (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22)
|
|
.SS Frequently Asked Questions
|
|
.SS Do all cloud storage systems support all rclone commands
|
|
.PP
|
|
Yes they do.
|
|
All the rclone commands (eg \f[C]sync\f[], \f[C]copy\f[] etc) will work
|
|
on all the remote storage systems.
|
|
.SS Can I copy the config from one machine to another
|
|
.PP
|
|
Sure! Rclone stores all of its config in a single file.
|
|
If you want to find this file, run \f[C]rclone\ config\ file\f[] which
|
|
will tell you where it is.
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for more
|
|
info.
|
|
.SS How do I configure rclone on a remote / headless box with no
|
|
browser?
|
|
.PP
|
|
This has now been documented in its own remote setup
|
|
page (https://rclone.org/remote_setup/).
|
|
.SS Can rclone sync directly from drive to s3
|
|
.PP
|
|
Rclone can sync between two remote cloud storage systems just fine.
|
|
.PP
|
|
Note that it effectively downloads the file and uploads it again, so the
|
|
node running rclone would need to have lots of bandwidth.
|
|
.PP
|
|
The syncs would be incremental (on a file by file basis).
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone\ sync\ drive:Folder\ s3:bucket
|
|
\f[]
|
|
.fi
|
|
.SS Using rclone from multiple locations at the same time
|
|
.PP
|
|
You can use rclone from multiple places at the same time if you choose
|
|
different subdirectory for the output, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Server\ A>\ rclone\ sync\ /tmp/whatever\ remote:ServerA
|
|
Server\ B>\ rclone\ sync\ /tmp/whatever\ remote:ServerB
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you sync to the same directory then you should use rclone copy
|
|
otherwise the two rclones may delete each others files, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Server\ A>\ rclone\ copy\ /tmp/whatever\ remote:Backup
|
|
Server\ B>\ rclone\ copy\ /tmp/whatever\ remote:Backup
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The file names you upload from Server A and Server B should be different
|
|
in this case, otherwise some file systems (eg Drive) may make
|
|
duplicates.
|
|
.SS Why doesn't rclone support partial transfers / binary diffs like
|
|
rsync?
|
|
.PP
|
|
Rclone stores each file you transfer as a native object on the remote
|
|
cloud storage system.
|
|
This means that you can see the files you upload as expected using
|
|
alternative access methods (eg using the Google Drive web interface).
|
|
There is a 1:1 mapping between files on your hard disk and objects
|
|
created in the cloud storage system.
|
|
.PP
|
|
Cloud storage systems (at least none I've come across yet) don't support
|
|
partially uploading an object.
|
|
You can't take an existing object, and change some bytes in the middle
|
|
of it.
|
|
.PP
|
|
It would be possible to make a sync system which stored binary diffs
|
|
instead of whole objects like rclone does, but that would break the 1:1
|
|
mapping of files on your hard disk to objects in the remote cloud
|
|
storage system.
|
|
.PP
|
|
All the cloud storage systems support partial downloads of content, so
|
|
it would be possible to make partial downloads work.
|
|
However to make this work efficiently this would require storing a
|
|
significant amount of metadata, which breaks the desired 1:1 mapping of
|
|
files to objects.
|
|
.SS Can rclone do bi\-directional sync?
|
|
.PP
|
|
No, not at present.
|
|
rclone only does uni\-directional sync from A \-> B.
|
|
It may do in the future though since it has all the primitives \- it
|
|
just requires writing the algorithm to do it.
|
|
.SS Can I use rclone with an HTTP proxy?
|
|
.PP
|
|
Yes.
|
|
rclone will follow the standard environment variables for proxies,
|
|
similar to cURL and other programs.
|
|
.PP
|
|
In general the variables are called \f[C]http_proxy\f[] (for services
|
|
reached over \f[C]http\f[]) and \f[C]https_proxy\f[] (for services
|
|
reached over \f[C]https\f[]).
|
|
Most public services will be using \f[C]https\f[], but you may wish to
|
|
set both.
|
|
.PP
|
|
The content of the variable is \f[C]protocol://server:port\f[].
|
|
The protocol value is the one used to talk to the proxy server, itself,
|
|
and is commonly either \f[C]http\f[] or \f[C]socks5\f[].
|
|
.PP
|
|
Slightly annoyingly, there is no \f[I]standard\f[] for the name; some
|
|
applications may use \f[C]http_proxy\f[] but another one
|
|
\f[C]HTTP_PROXY\f[].
|
|
The \f[C]Go\f[] libraries used by \f[C]rclone\f[] will try both
|
|
variations, but you may wish to set all possibilities.
|
|
So, on Linux, you may end up with code similar to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export\ http_proxy=http://proxyserver:12345
|
|
export\ https_proxy=$http_proxy
|
|
export\ HTTP_PROXY=$http_proxy
|
|
export\ HTTPS_PROXY=$http_proxy
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The \f[C]NO_PROXY\f[] allows you to disable the proxy for specific
|
|
hosts.
|
|
Hosts must be comma separated, and can contain domains or parts.
|
|
For instance \[lq]foo.com\[rq] also matches \[lq]bar.foo.com\[rq].
|
|
.PP
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export\ no_proxy=localhost,127.0.0.0/8,my.host.name
|
|
export\ NO_PROXY=$no_proxy
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
Note that the ftp backend does not support \f[C]ftp_proxy\f[] yet.
|
|
.SS Rclone gives x509: failed to load system roots and no roots provided
|
|
error
|
|
.PP
|
|
This means that \f[C]rclone\f[] can't file the SSL root certificates.
|
|
Likely you are running \f[C]rclone\f[] on a NAS with a cut\-down Linux
|
|
OS, or possibly on Solaris.
|
|
.PP
|
|
Rclone (via the Go runtime) tries to load the root certificates from
|
|
these places on Linux.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
"/etc/ssl/certs/ca\-certificates.crt",\ //\ Debian/Ubuntu/Gentoo\ etc.
|
|
"/etc/pki/tls/certs/ca\-bundle.crt",\ \ \ //\ Fedora/RHEL
|
|
"/etc/ssl/ca\-bundle.pem",\ \ \ \ \ \ \ \ \ \ \ \ \ //\ OpenSUSE
|
|
"/etc/pki/tls/cacert.pem",\ \ \ \ \ \ \ \ \ \ \ \ //\ OpenELEC
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
So doing something like this should fix the problem.
|
|
It also sets the time which is important for SSL to work properly.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
mkdir\ \-p\ /etc/ssl/certs/
|
|
curl\ \-o\ /etc/ssl/certs/ca\-certificates.crt\ https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
|
ntpclient\ \-s\ \-h\ pool.ntp.org
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
The two environment variables \f[C]SSL_CERT_FILE\f[] and
|
|
\f[C]SSL_CERT_DIR\f[], mentioned in the x509
|
|
package (https://godoc.org/crypto/x509), provide an additional way to
|
|
provide the SSL root certificates.
|
|
.PP
|
|
Note that you may need to add the \f[C]\-\-insecure\f[] option to the
|
|
\f[C]curl\f[] command line if it doesn't work without.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl\ \-\-insecure\ \-o\ /etc/ssl/certs/ca\-certificates.crt\ https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
|
\f[]
|
|
.fi
|
|
.SS Rclone gives Failed to load config file: function not implemented
|
|
error
|
|
.PP
|
|
Likely this means that you are running rclone on Linux version not
|
|
supported by the go runtime, ie earlier than version 2.6.23.
|
|
.PP
|
|
See the system requirements section in the go install
|
|
docs (https://golang.org/doc/install) for full details.
|
|
.SS All my uploaded docx/xlsx/pptx files appear as archive/zip
|
|
.PP
|
|
This is caused by uploading these files from a Windows computer which
|
|
hasn't got the Microsoft Office suite installed.
|
|
The easiest way to fix is to install the Word viewer and the Microsoft
|
|
Office Compatibility Pack for Word, Excel, and PowerPoint 2007 and later
|
|
versions' file formats
|
|
.SS tcp lookup some.domain.com no such host
|
|
.PP
|
|
This happens when rclone cannot resolve a domain.
|
|
Please check that your DNS setup is generally working, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#\ both\ should\ print\ a\ long\ list\ of\ possible\ IP\ addresses
|
|
dig\ www.googleapis.com\ \ \ \ \ \ \ \ \ \ #\ resolve\ using\ your\ default\ DNS
|
|
dig\ www.googleapis.com\ \@8.8.8.8\ #\ resolve\ with\ Google\[aq]s\ DNS\ server
|
|
\f[]
|
|
.fi
|
|
.PP
|
|
If you are using \f[C]systemd\-resolved\f[] (default on Arch Linux),
|
|
ensure it is at version 233 or higher.
|
|
Previous releases contain a bug which causes not all domains to be
|
|
resolved properly.
|
|
.PP
|
|
Additionally with the \f[C]GODEBUG=netdns=\f[] environment variable the
|
|
Go resolver decision can be influenced.
|
|
This also allows to resolve certain issues with DNS resolution.
|
|
See the name resolution section in the go
|
|
docs (https://golang.org/pkg/net/#hdr-Name_Resolution).
|
|
.SS The total size reported in the stats for a sync is wrong and keeps
|
|
changing
|
|
.PP
|
|
It is likely you have more than 10,000 files that need to be synced.
|
|
By default rclone only gets 10,000 files ahead in a sync so as not to
|
|
use up too much memory.
|
|
You can change this default with the
|
|
\[en]max\-backlog (/docs/#max-backlog-n) flag.
|
|
.SS Rclone is using too much memory or appears to have a memory leak
|
|
.PP
|
|
Rclone is written in Go which uses a garbage collector.
|
|
The default settings for the garbage collector mean that it runs when
|
|
the heap size has doubled.
|
|
.PP
|
|
However it is possible to tune the garbage collector to use less memory
|
|
by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value, say
|
|
\f[C]export\ GOGC=20\f[].
|
|
This will make the garbage collector work harder, reducing memory size
|
|
at the expense of CPU usage.
|
|
.PP
|
|
The most common cause of rclone using lots of memory is a single
|
|
directory with thousands or millions of files in.
|
|
Rclone has to load this entirely into memory as rclone objects.
|
|
Each rclone object takes 0.5k\-1k of memory.
|
|
.SS License
|
|
.PP
|
|
This is free software under the terms of MIT the license (check the
|
|
COPYING file included with the source code).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Copyright\ (C)\ 2019\ by\ Nick\ Craig\-Wood\ https://www.craig\-wood.com/nick/
|
|
|
|
Permission\ is\ hereby\ granted,\ free\ of\ charge,\ to\ any\ person\ obtaining\ a\ copy
|
|
of\ this\ software\ and\ associated\ documentation\ files\ (the\ "Software"),\ to\ deal
|
|
in\ the\ Software\ without\ restriction,\ including\ without\ limitation\ the\ rights
|
|
to\ use,\ copy,\ modify,\ merge,\ publish,\ distribute,\ sublicense,\ and/or\ sell
|
|
copies\ of\ the\ Software,\ and\ to\ permit\ persons\ to\ whom\ the\ Software\ is
|
|
furnished\ to\ do\ so,\ subject\ to\ the\ following\ conditions:
|
|
|
|
The\ above\ copyright\ notice\ and\ this\ permission\ notice\ shall\ be\ included\ in
|
|
all\ copies\ or\ substantial\ portions\ of\ the\ Software.
|
|
|
|
THE\ SOFTWARE\ IS\ PROVIDED\ "AS\ IS",\ WITHOUT\ WARRANTY\ OF\ ANY\ KIND,\ EXPRESS\ OR
|
|
IMPLIED,\ INCLUDING\ BUT\ NOT\ LIMITED\ TO\ THE\ WARRANTIES\ OF\ MERCHANTABILITY,
|
|
FITNESS\ FOR\ A\ PARTICULAR\ PURPOSE\ AND\ NONINFRINGEMENT.\ IN\ NO\ EVENT\ SHALL\ THE
|
|
AUTHORS\ OR\ COPYRIGHT\ HOLDERS\ BE\ LIABLE\ FOR\ ANY\ CLAIM,\ DAMAGES\ OR\ OTHER
|
|
LIABILITY,\ WHETHER\ IN\ AN\ ACTION\ OF\ CONTRACT,\ TORT\ OR\ OTHERWISE,\ ARISING\ FROM,
|
|
OUT\ OF\ OR\ IN\ CONNECTION\ WITH\ THE\ SOFTWARE\ OR\ THE\ USE\ OR\ OTHER\ DEALINGS\ IN
|
|
THE\ SOFTWARE.
|
|
\f[]
|
|
.fi
|
|
.SS Authors
|
|
.IP \[bu] 2
|
|
Nick Craig\-Wood <nick@craig-wood.com>
|
|
.SS Contributors
|
|
.IP \[bu] 2
|
|
Alex Couper <amcouper@gmail.com>
|
|
.IP \[bu] 2
|
|
Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
|
|
.IP \[bu] 2
|
|
Shimon Doodkin <helpmepro1@gmail.com>
|
|
.IP \[bu] 2
|
|
Colin Nicholson <colin@colinn.com>
|
|
.IP \[bu] 2
|
|
Klaus Post <klauspost@gmail.com>
|
|
.IP \[bu] 2
|
|
Sergey Tolmachev <tolsi.ru@gmail.com>
|
|
.IP \[bu] 2
|
|
Adriano Aurélio Meirelles <adriano@atinge.com>
|
|
.IP \[bu] 2
|
|
C.
|
|
Bess <cbess@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dmitry Burdeev <dibu28@gmail.com>
|
|
.IP \[bu] 2
|
|
Joseph Spurrier <github@josephspurrier.com>
|
|
.IP \[bu] 2
|
|
Björn Harrtell <bjorn@wololo.org>
|
|
.IP \[bu] 2
|
|
Xavier Lucas <xavier.lucas@corp.ovh.com>
|
|
.IP \[bu] 2
|
|
Werner Beroux <werner@beroux.com>
|
|
.IP \[bu] 2
|
|
Brian Stengaard <brian@stengaard.eu>
|
|
.IP \[bu] 2
|
|
Jakub Gedeon <jgedeon@sofi.com>
|
|
.IP \[bu] 2
|
|
Jim Tittsler <jwt@onjapan.net>
|
|
.IP \[bu] 2
|
|
Michal Witkowski <michal@improbable.io>
|
|
.IP \[bu] 2
|
|
Fabian Ruff <fabian.ruff@sap.com>
|
|
.IP \[bu] 2
|
|
Leigh Klotz <klotz@quixey.com>
|
|
.IP \[bu] 2
|
|
Romain Lapray <lapray.romain@gmail.com>
|
|
.IP \[bu] 2
|
|
Justin R.
|
|
Wilson <jrw972@gmail.com>
|
|
.IP \[bu] 2
|
|
Antonio Messina <antonio.s.messina@gmail.com>
|
|
.IP \[bu] 2
|
|
Stefan G.
|
|
Weichinger <office@oops.co.at>
|
|
.IP \[bu] 2
|
|
Per Cederberg <cederberg@gmail.com>
|
|
.IP \[bu] 2
|
|
Radek Šenfeld <rush@logic.cz>
|
|
.IP \[bu] 2
|
|
Fredrik Fornwall <fredrik@fornwall.net>
|
|
.IP \[bu] 2
|
|
Asko Tamm <asko@deekit.net>
|
|
.IP \[bu] 2
|
|
xor\-zz <xor@gstocco.com>
|
|
.IP \[bu] 2
|
|
Tomasz Mazur <tmazur90@gmail.com>
|
|
.IP \[bu] 2
|
|
Marco Paganini <paganini@paganini.net>
|
|
.IP \[bu] 2
|
|
Felix Bünemann <buenemann@louis.info>
|
|
.IP \[bu] 2
|
|
Durval Menezes <jmrclone@durval.com>
|
|
.IP \[bu] 2
|
|
Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
|
|
.IP \[bu] 2
|
|
Stefan Breunig <stefan-github@yrden.de>
|
|
.IP \[bu] 2
|
|
Alishan Ladhani <ali-l@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
0xJAKE <0xJAKE@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Thibault Molleman <thibaultmol@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Scott McGillivray <scott.mcgillivray@gmail.com>
|
|
.IP \[bu] 2
|
|
Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
|
|
.IP \[bu] 2
|
|
Lukas Loesche <lukas@mesosphere.io>
|
|
.IP \[bu] 2
|
|
emyarod <allllaboutyou@gmail.com>
|
|
.IP \[bu] 2
|
|
T.C.
|
|
Ferguson <tcf909@gmail.com>
|
|
.IP \[bu] 2
|
|
Brandur <brandur@mutelight.org>
|
|
.IP \[bu] 2
|
|
Dario Giovannetti <dev@dariogiovannetti.net>
|
|
.IP \[bu] 2
|
|
Károly Oláh <okaresz@aol.com>
|
|
.IP \[bu] 2
|
|
Jon Yergatian <jon@macfanatic.ca>
|
|
.IP \[bu] 2
|
|
Jack Schmidt <github@mowsey.org>
|
|
.IP \[bu] 2
|
|
Dedsec1 <Dedsec1@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Hisham Zarka <hzarka@gmail.com>
|
|
.IP \[bu] 2
|
|
Jérôme Vizcaino <jerome.vizcaino@gmail.com>
|
|
.IP \[bu] 2
|
|
Mike Tesch <mjt6129@rit.edu>
|
|
.IP \[bu] 2
|
|
Marvin Watson <marvwatson@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Danny Tsai <danny8376@gmail.com>
|
|
.IP \[bu] 2
|
|
Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
|
|
.IP \[bu] 2
|
|
Stephen Harris <github@spuddy.org> <sweharris@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Craton <jncraton@gmail.com>
|
|
.IP \[bu] 2
|
|
Hraban Luyat <hraban@0brg.net>
|
|
.IP \[bu] 2
|
|
Michael Ledin <mledin89@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Kristensen <me@azgul.com>
|
|
.IP \[bu] 2
|
|
Too Much IO <toomuchio@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Anisse Astier <anisse@astier.eu>
|
|
.IP \[bu] 2
|
|
Zahiar Ahmed <zahiar@live.com>
|
|
.IP \[bu] 2
|
|
Igor Kharin <igorkharin@gmail.com>
|
|
.IP \[bu] 2
|
|
Bill Zissimopoulos <billziss@navimatics.com>
|
|
.IP \[bu] 2
|
|
Bob Potter <bobby.potter@gmail.com>
|
|
.IP \[bu] 2
|
|
Steven Lu <tacticalazn@gmail.com>
|
|
.IP \[bu] 2
|
|
Sjur Fredriksen <sjurtf@ifi.uio.no>
|
|
.IP \[bu] 2
|
|
Ruwbin <hubus12345@gmail.com>
|
|
.IP \[bu] 2
|
|
Fabian Möller <fabianm88@gmail.com> <f.moeller@nynex.de>
|
|
.IP \[bu] 2
|
|
Edward Q.
|
|
Bridges <github@eqbridges.com>
|
|
.IP \[bu] 2
|
|
Vasiliy Tolstov <v.tolstov@selfip.ru>
|
|
.IP \[bu] 2
|
|
Harshavardhana <harsha@minio.io>
|
|
.IP \[bu] 2
|
|
sainaen <sainaen@gmail.com>
|
|
.IP \[bu] 2
|
|
gdm85 <gdm85@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yaroslav Halchenko <debian@onerussian.com>
|
|
.IP \[bu] 2
|
|
John Papandriopoulos <jpap@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Zhiming Wang <zmwangx@gmail.com>
|
|
.IP \[bu] 2
|
|
Andy Pilate <cubox@cubox.me>
|
|
.IP \[bu] 2
|
|
Oliver Heyme <olihey@googlemail.com> <olihey@users.noreply.github.com>
|
|
<de8olihe@lego.com>
|
|
.IP \[bu] 2
|
|
wuyu <wuyu@yunify.com>
|
|
.IP \[bu] 2
|
|
Andrei Dragomir <adragomi@adobe.com>
|
|
.IP \[bu] 2
|
|
Christian Brüggemann <mail@cbruegg.com>
|
|
.IP \[bu] 2
|
|
Alex McGrath Kraak <amkdude@gmail.com>
|
|
.IP \[bu] 2
|
|
bpicode <bjoern.pirnay@googlemail.com>
|
|
.IP \[bu] 2
|
|
Daniel Jagszent <daniel@jagszent.de>
|
|
.IP \[bu] 2
|
|
Josiah White <thegenius2009@gmail.com>
|
|
.IP \[bu] 2
|
|
Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
|
|
.IP \[bu] 2
|
|
Jan Varho <jan@varho.org>
|
|
.IP \[bu] 2
|
|
Girish Ramakrishnan <girish@cloudron.io>
|
|
.IP \[bu] 2
|
|
LingMan <LingMan@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Jacob McNamee <jacobmcnamee@gmail.com>
|
|
.IP \[bu] 2
|
|
jersou <jertux@gmail.com>
|
|
.IP \[bu] 2
|
|
thierry <thierry@substantiel.fr>
|
|
.IP \[bu] 2
|
|
Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
|
|
.IP \[bu] 2
|
|
Dan Dascalescu <ddascalescu+github@gmail.com>
|
|
.IP \[bu] 2
|
|
Jason Rose <jason@jro.io>
|
|
.IP \[bu] 2
|
|
Andrew Starr\-Bochicchio <a.starr.b@gmail.com>
|
|
.IP \[bu] 2
|
|
John Leach <john@johnleach.co.uk>
|
|
.IP \[bu] 2
|
|
Corban Raun <craun@instructure.com>
|
|
.IP \[bu] 2
|
|
Pierre Carlson <mpcarl@us.ibm.com>
|
|
.IP \[bu] 2
|
|
Ernest Borowski <er.borowski@gmail.com>
|
|
.IP \[bu] 2
|
|
Remus Bunduc <remus.bunduc@gmail.com>
|
|
.IP \[bu] 2
|
|
Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru>
|
|
.IP \[bu] 2
|
|
Jakub Tasiemski <tasiemski@gmail.com>
|
|
.IP \[bu] 2
|
|
David Minor <dminor@saymedia.com>
|
|
.IP \[bu] 2
|
|
Tim Cooijmans <cooijmans.tim@gmail.com>
|
|
.IP \[bu] 2
|
|
Laurence <liuxy6@gmail.com>
|
|
.IP \[bu] 2
|
|
Giovanni Pizzi <gio.piz@gmail.com>
|
|
.IP \[bu] 2
|
|
Filip Bartodziej <filipbartodziej@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Fautley <jon@dead.li>
|
|
.IP \[bu] 2
|
|
lewapm <32110057+lewapm@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yassine Imounachen <yassine256@gmail.com>
|
|
.IP \[bu] 2
|
|
Chris Redekop <chris-redekop@users.noreply.github.com>
|
|
<chris.redekop@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Fautley <jon@adenoid.appstal.co.uk>
|
|
.IP \[bu] 2
|
|
Will Gunn <WillGunn@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Lucas Bremgartner <lucas@bremis.ch>
|
|
.IP \[bu] 2
|
|
Jody Frankowski <jody.frankowski@gmail.com>
|
|
.IP \[bu] 2
|
|
Andreas Roussos <arouss1980@gmail.com>
|
|
.IP \[bu] 2
|
|
nbuchanan <nbuchanan@utah.gov>
|
|
.IP \[bu] 2
|
|
Durval Menezes <rclone@durval.com>
|
|
.IP \[bu] 2
|
|
Victor <vb-github@viblo.se>
|
|
.IP \[bu] 2
|
|
Mateusz <pabian.mateusz@gmail.com>
|
|
.IP \[bu] 2
|
|
Daniel Loader <spicypixel@gmail.com>
|
|
.IP \[bu] 2
|
|
David0rk <davidork@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexander Neumann <alexander@bumpern.de>
|
|
.IP \[bu] 2
|
|
Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local>
|
|
.IP \[bu] 2
|
|
Leo R.
|
|
Lundgren <leo@finalresort.org>
|
|
.IP \[bu] 2
|
|
wolfv <wolfv6@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dave Pedu <dave@davepedu.com>
|
|
.IP \[bu] 2
|
|
Stefan Lindblom <lindblom@spotify.com>
|
|
.IP \[bu] 2
|
|
seuffert <oliver@seuffert.biz>
|
|
.IP \[bu] 2
|
|
gbadanahatti <37121690+gbadanahatti@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Keith Goldfarb <barkofdelight@gmail.com>
|
|
.IP \[bu] 2
|
|
Steve Kriss <steve@heptio.com>
|
|
.IP \[bu] 2
|
|
Chih\-Hsuan Yen <yan12125@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexander Neumann <fd0@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Matt Holt <mholt@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Eri Bastos <bastos.eri@gmail.com>
|
|
.IP \[bu] 2
|
|
Michael P.
|
|
Dubner <pywebmail@list.ru>
|
|
.IP \[bu] 2
|
|
Antoine GIRARD <sapk@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Mateusz Piotrowski <mpp302@gmail.com>
|
|
.IP \[bu] 2
|
|
Animosity022 <animosity22@users.noreply.github.com>
|
|
<earl.texter@gmail.com>
|
|
.IP \[bu] 2
|
|
Peter Baumgartner <pete@lincolnloop.com>
|
|
.IP \[bu] 2
|
|
Craig Rachel <craig@craigrachel.com>
|
|
.IP \[bu] 2
|
|
Michael G.
|
|
Noll <miguno@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
hensur <me@hensur.de>
|
|
.IP \[bu] 2
|
|
Oliver Heyme <de8olihe@lego.com>
|
|
.IP \[bu] 2
|
|
Richard Yang <richard@yenforyang.com>
|
|
.IP \[bu] 2
|
|
Piotr Oleszczyk <piotr.oleszczyk@gmail.com>
|
|
.IP \[bu] 2
|
|
Rodrigo <rodarima@gmail.com>
|
|
.IP \[bu] 2
|
|
NoLooseEnds <NoLooseEnds@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Jakub Karlicek <jakub@karlicek.me>
|
|
.IP \[bu] 2
|
|
John Clayton <john@codemonkeylabs.com>
|
|
.IP \[bu] 2
|
|
Kasper Byrdal Nielsen <byrdal76@gmail.com>
|
|
.IP \[bu] 2
|
|
Benjamin Joseph Dag <bjdag1234@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
themylogin <themylogin@gmail.com>
|
|
.IP \[bu] 2
|
|
Onno Zweers <onno.zweers@surfsara.nl>
|
|
.IP \[bu] 2
|
|
Jasper Lievisse Adriaanse <jasper@humppa.nl>
|
|
.IP \[bu] 2
|
|
sandeepkru <sandeep.ummadi@gmail.com>
|
|
<sandeepkru@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
HerrH <atomtigerzoo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Andrew <4030760+sparkyman215@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
dan smith <XX1011@gmail.com>
|
|
.IP \[bu] 2
|
|
Oleg Kovalov <iamolegkovalov@gmail.com>
|
|
.IP \[bu] 2
|
|
Ruben Vandamme <github-com-00ff86@vandamme.email>
|
|
.IP \[bu] 2
|
|
Cnly <minecnly@gmail.com>
|
|
.IP \[bu] 2
|
|
Andres Alvarez <1671935+kir4h@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
reddi1 <xreddi@gmail.com>
|
|
.IP \[bu] 2
|
|
Matt Tucker <matthewtckr@gmail.com>
|
|
.IP \[bu] 2
|
|
Sebastian Bünger <buengese@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Polden <mpolden@mpolden.no>
|
|
.IP \[bu] 2
|
|
Alex Chen <Cnly@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Denis <deniskovpen@gmail.com>
|
|
.IP \[bu] 2
|
|
bsteiss <35940619+bsteiss@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Cédric Connes <cedric.connes@gmail.com>
|
|
.IP \[bu] 2
|
|
Dr.\ Tobias Quathamer <toddy15@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
dcpu <42736967+dcpu@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sheldon Rupp <me@shel.io>
|
|
.IP \[bu] 2
|
|
albertony <12441419+albertony@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
cron410 <cron410@gmail.com>
|
|
.IP \[bu] 2
|
|
Anagh Kumar Baranwal <anaghk.dos@gmail.com>
|
|
.IP \[bu] 2
|
|
Felix Brucker <felix@felixbrucker.com>
|
|
.IP \[bu] 2
|
|
Santiago Rodríguez <scollazo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Craig Miskell <craig.miskell@fluxfederation.com>
|
|
.IP \[bu] 2
|
|
Antoine GIRARD <sapk@sapk.fr>
|
|
.IP \[bu] 2
|
|
Joanna Marek <joanna.marek@u2i.com>
|
|
.IP \[bu] 2
|
|
frenos <frenos@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ssaqua <ssaqua@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
xnaas <me@xnaas.info>
|
|
.IP \[bu] 2
|
|
Frantisek Fuka <fuka@fuxoft.cz>
|
|
.IP \[bu] 2
|
|
Paul Kohout <pauljkohout@yahoo.com>
|
|
.IP \[bu] 2
|
|
dcpu <43330287+dcpu@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
jackyzy823 <jackyzy823@gmail.com>
|
|
.IP \[bu] 2
|
|
David Haguenauer <ml@kurokatta.org>
|
|
.IP \[bu] 2
|
|
teresy <hi.teresy@gmail.com>
|
|
.IP \[bu] 2
|
|
buergi <patbuergi@gmx.de>
|
|
.IP \[bu] 2
|
|
Florian Gamboeck <mail@floga.de>
|
|
.IP \[bu] 2
|
|
Ralf Hemberger <10364191+rhemberger@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Scott Edlund <sedlund@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Erik Swanson <erik@retailnext.net>
|
|
.IP \[bu] 2
|
|
Jake Coggiano <jake@stripe.com>
|
|
.IP \[bu] 2
|
|
brused27 <brused27@noemailaddress>
|
|
.IP \[bu] 2
|
|
Peter Kaminski <kaminski@istori.com>
|
|
.IP \[bu] 2
|
|
Henry Ptasinski <henry@logout.com>
|
|
.IP \[bu] 2
|
|
Alexander <kharkovalexander@gmail.com>
|
|
.IP \[bu] 2
|
|
Garry McNulty <garrmcnu@gmail.com>
|
|
.IP \[bu] 2
|
|
Mathieu Carbou <mathieu.carbou@gmail.com>
|
|
.IP \[bu] 2
|
|
Mark Otway <mark@otway.com>
|
|
.IP \[bu] 2
|
|
William Cocker <37018962+WilliamCocker@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
François Leurent <131.js@cloudyks.org>
|
|
.IP \[bu] 2
|
|
Arkadius Stefanski <arkste@gmail.com>
|
|
.IP \[bu] 2
|
|
Jay <dev@jaygoel.com>
|
|
.IP \[bu] 2
|
|
andrea rota <a@xelera.eu>
|
|
.IP \[bu] 2
|
|
nicolov <nicolov@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dario Guzik <dario@guzik.com.ar>
|
|
.IP \[bu] 2
|
|
qip <qip@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
yair\@unicorn <yair@unicorn>
|
|
.IP \[bu] 2
|
|
Matt Robinson <brimstone@the.narro.ws>
|
|
.IP \[bu] 2
|
|
kayrus <kay.diam@gmail.com>
|
|
.IP \[bu] 2
|
|
Rémy Léone <remy.leone@gmail.com>
|
|
.IP \[bu] 2
|
|
Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com>
|
|
.IP \[bu] 2
|
|
weetmuts <oehrstroem@gmail.com>
|
|
.IP \[bu] 2
|
|
Jonathan <vanillajonathan@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
James Carpenter <orbsmiv@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Vince <vince0villamora@gmail.com>
|
|
.IP \[bu] 2
|
|
Nestar47 <47841759+Nestar47@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Six <brbsix@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexandru Bumbacea <alexandru.bumbacea@booking.com>
|
|
.IP \[bu] 2
|
|
calisro <robert.calistri@gmail.com>
|
|
.IP \[bu] 2
|
|
Dr.Rx <david.rey@nventive.com>
|
|
.IP \[bu] 2
|
|
marcintustin <marcintustin@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
jaKa Močnik <jaka@koofr.net>
|
|
.IP \[bu] 2
|
|
Fionera <fionera@fionera.de>
|
|
.IP \[bu] 2
|
|
Dan Walters <dan@walters.io>
|
|
.IP \[bu] 2
|
|
Danil Semelenov <sgtpep@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
xopez <28950736+xopez@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ben Boeckel <mathstuf@gmail.com>
|
|
.IP \[bu] 2
|
|
Manu <manu@snapdragon.cc>
|
|
.IP \[bu] 2
|
|
Kyle E.
|
|
Mitchell <kyle@kemitchell.com>
|
|
.IP \[bu] 2
|
|
Gary Kim <gary@garykim.dev>
|
|
.IP \[bu] 2
|
|
Jon <jonathn@github.com>
|
|
.IP \[bu] 2
|
|
Jeff Quinn <jeffrey.quinn@bluevoyant.com>
|
|
.IP \[bu] 2
|
|
Peter Berbec <peter@berbec.com>
|
|
.IP \[bu] 2
|
|
didil <1284255+didil@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
id01 <gaviniboom@gmail.com>
|
|
.IP \[bu] 2
|
|
Robert Marko <robimarko@gmail.com>
|
|
.IP \[bu] 2
|
|
Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
JorisE <JorisE@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
garry415 <garry.415@gmail.com>
|
|
.IP \[bu] 2
|
|
forgems <forgems@gmail.com>
|
|
.IP \[bu] 2
|
|
Florian Apolloner <florian@apolloner.eu>
|
|
.IP \[bu] 2
|
|
Aleksandar Jankovic <office@ajankovic.com>
|
|
.IP \[bu] 2
|
|
Maran <maran@protonmail.com>
|
|
.IP \[bu] 2
|
|
nguyenhuuluan434 <nguyenhuuluan434@gmail.com>
|
|
.IP \[bu] 2
|
|
Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev>
|
|
.IP \[bu] 2
|
|
yparitcher <y@paritcher.com>
|
|
.IP \[bu] 2
|
|
AbelThar <abela.tharen@gmail.com>
|
|
.IP \[bu] 2
|
|
Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
|
|
.IP \[bu] 2
|
|
Russell Davis <russelldavis@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yi FU <yi.fu@tink.se>
|
|
.IP \[bu] 2
|
|
Paul Millar <paul.millar@desy.de>
|
|
.IP \[bu] 2
|
|
justinalin <justinalin@qnap.com>
|
|
.IP \[bu] 2
|
|
EliEron <subanimehd@gmail.com>
|
|
.IP \[bu] 2
|
|
justina777 <chiahuei.lin@gmail.com>
|
|
.IP \[bu] 2
|
|
Chaitanya Bankanhal <bchaitanya15@gmail.com>
|
|
.IP \[bu] 2
|
|
Michał Matczuk <michal@scylladb.com>
|
|
.IP \[bu] 2
|
|
Macavirus <macavirus@zoho.com>
|
|
.IP \[bu] 2
|
|
Abhinav Sharma <abhi18av@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ginvine <34869051+ginvine@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Patrick Wang <mail6543210@yahoo.com.tw>
|
|
.IP \[bu] 2
|
|
Cenk Alti <cenkalti@gmail.com>
|
|
.IP \[bu] 2
|
|
Andreas Chlupka <andy@chlupka.com>
|
|
.IP \[bu] 2
|
|
Alfonso Montero <amontero@tinet.org>
|
|
.IP \[bu] 2
|
|
Ivan Andreev <ivandeex@gmail.com>
|
|
.IP \[bu] 2
|
|
David Baumgold <david@davidbaumgold.com>
|
|
.IP \[bu] 2
|
|
Lars Lehtonen <lars.lehtonen@gmail.com>
|
|
.IP \[bu] 2
|
|
Matei David <matei.david@gmail.com>
|
|
.IP \[bu] 2
|
|
David <david.bramwell@endemolshine.com>
|
|
.IP \[bu] 2
|
|
Anthony Rusdi <33247310+antrusd@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Richard Patel <me@terorie.dev>
|
|
.IP \[bu] 2
|
|
庄天翼 <zty0826@gmail.com>
|
|
.IP \[bu] 2
|
|
SwitchJS <dev@switchjs.com>
|
|
.IP \[bu] 2
|
|
Raphael <PowershellNinja@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sezal Agrawal <sezalagrawal@gmail.com>
|
|
.IP \[bu] 2
|
|
Tyler <TylerNakamura@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Brett Dutro <brett.dutro@gmail.com>
|
|
.IP \[bu] 2
|
|
Vighnesh SK <booterror99@gmail.com>
|
|
.IP \[bu] 2
|
|
Arijit Biswas <dibbyo456@gmail.com>
|
|
.IP \[bu] 2
|
|
Michele Caci <michele.caci@gmail.com>
|
|
.IP \[bu] 2
|
|
AlexandrBoltris <ua2fgb@gmail.com>
|
|
.IP \[bu] 2
|
|
Bryce Larson <blarson@saltstack.com>
|
|
.IP \[bu] 2
|
|
Carlos Ferreyra <crypticmind@gmail.com>
|
|
.IP \[bu] 2
|
|
Saksham Khanna <sakshamkhanna@outlook.com>
|
|
.IP \[bu] 2
|
|
dausruddin <5763466+dausruddin@users.noreply.github.com>
|
|
.SH Contact the rclone project
|
|
.SS Forum
|
|
.PP
|
|
Forum for questions and general discussion:
|
|
.IP \[bu] 2
|
|
https://forum.rclone.org
|
|
.SS Gitub project
|
|
.PP
|
|
The project website is at:
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone
|
|
.PP
|
|
There you can file bug reports or contribute pull requests.
|
|
.SS Twitter
|
|
.PP
|
|
You can also follow me on twitter for rclone announcements:
|
|
.IP \[bu] 2
|
|
[\@njcw](https://twitter.com/njcw)
|
|
.SS Email
|
|
.PP
|
|
Or if all else fails or you want to ask something private or
|
|
confidential email Nick Craig\-Wood (mailto:nick@craig-wood.com).
|
|
Please don't email me requests for help \- those are better directed to
|
|
the forum \- thanks!
|
|
.SH AUTHORS
|
|
Nick Craig\-Wood.
|