diff --git a/MANUAL.html b/MANUAL.html
index d4b683fec..1327ff595 100644
--- a/MANUAL.html
+++ b/MANUAL.html
@@ -81,7 +81,7 @@
Rclone syncs your files to cloud storage

@@ -2964,7 +2964,9 @@ rclone mount remote:path/to/files \\cloud\remote
When running in background mode the user will have to stop the mount manually:
# Linux
fusermount -u /path/to/local/mount
-# OS X
+#... or on some systems
+fusermount3 -u /path/to/local/mount
+# OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy. When that happens, it is the user's responsibility to stop the mount manually.
The size of the mounted file system will be set according to information retrieved from the remote, the same as returned by the rclone about command. Remotes with unlimited storage may report the used size only, then an additional 1 PiB of free space is assumed. If the remote does not support the about feature at all, then 1 PiB is set as both the total and the free size.
@@ -3048,7 +3050,7 @@ sudo ln -s /opt/local/lib/libfuse.2.dylib
Note that all the rclone filters can be used to select a subset of the files to be visible in the mount.
systemd
When running rclone mount as a systemd service, it is possible to use Type=notify. In this case the service will enter the started state after the mountpoint has been successfully set up. Units having the rclone mount service specified as a requirement will see all files and folders immediately in this mode.
-Note that systemd runs mount units without any environment variables including PATH
or HOME
. This means that tilde (~
) expansion will not work and you should provide --config
and --cache-dir
explicitly as absolute paths via rclone arguments. Since mounting requires the fusermount
program, rclone will use the fallback PATH of /bin:/usr/bin
in this scenario. Please ensure that fusermount
is present on this PATH.
+Note that systemd runs mount units without any environment variables including PATH
or HOME
. This means that tilde (~
) expansion will not work and you should provide --config
and --cache-dir
explicitly as absolute paths via rclone arguments. Since mounting requires the fusermount
or fusermount3
program, rclone will use the fallback PATH of /bin:/usr/bin
in this scenario. Please ensure that fusermount
/fusermount3
is present on this PATH.
Rclone as Unix mount helper
The core Unix program /bin/mount
normally takes the -t FSTYPE
argument then runs the /sbin/mount.FSTYPE
helper program passing it mount options as -o key=val,...
or --opt=...
. Automount (classic or systemd) behaves in a similar way.
rclone by default expects GNU-style flags --key val
. To run it as a mount helper you should symlink rclone binary to /sbin/mount.rclone
and optionally /usr/bin/rclonefs
, e.g. ln -s /usr/bin/rclone /sbin/mount.rclone
. rclone will detect it and translate command-line arguments appropriately.
@@ -3482,7 +3484,9 @@ rclone nfsmount remote:path/to/files \\cloud\remote
When running in background mode the user will have to stop the mount manually:
# Linux
fusermount -u /path/to/local/mount
-# OS X
+#... or on some systems
+fusermount3 -u /path/to/local/mount
+# OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy. When that happens, it is the user's responsibility to stop the mount manually.
The size of the mounted file system will be set according to information retrieved from the remote, the same as returned by the rclone about command. Remotes with unlimited storage may report the used size only, then an additional 1 PiB of free space is assumed. If the remote does not support the about feature at all, then 1 PiB is set as both the total and the free size.
@@ -3566,7 +3570,7 @@ sudo ln -s /opt/local/lib/libfuse.2.dylib
Note that all the rclone filters can be used to select a subset of the files to be visible in the mount.
systemd
When running rclone nfsmount as a systemd service, it is possible to use Type=notify. In this case the service will enter the started state after the mountpoint has been successfully set up. Units having the rclone nfsmount service specified as a requirement will see all files and folders immediately in this mode.
-Note that systemd runs mount units without any environment variables including PATH
or HOME
. This means that tilde (~
) expansion will not work and you should provide --config
and --cache-dir
explicitly as absolute paths via rclone arguments. Since mounting requires the fusermount
program, rclone will use the fallback PATH of /bin:/usr/bin
in this scenario. Please ensure that fusermount
is present on this PATH.
+Note that systemd runs mount units without any environment variables including PATH
or HOME
. This means that tilde (~
) expansion will not work and you should provide --config
and --cache-dir
explicitly as absolute paths via rclone arguments. Since mounting requires the fusermount
or fusermount3
program, rclone will use the fallback PATH of /bin:/usr/bin
in this scenario. Please ensure that fusermount
/fusermount3
is present on this PATH.
Rclone as Unix mount helper
The core Unix program /bin/mount
normally takes the -t FSTYPE
argument then runs the /sbin/mount.FSTYPE
helper program passing it mount options as -o key=val,...
or --opt=...
. Automount (classic or systemd) behaves in a similar way.
rclone by default expects GNU-style flags --key val
. To run it as a mount helper you should symlink rclone binary to /sbin/mount.rclone
and optionally /usr/bin/rclonefs
, e.g. ln -s /usr/bin/rclone /sbin/mount.rclone
. rclone will detect it and translate command-line arguments appropriately.
@@ -4056,7 +4060,7 @@ htpasswd -B htpasswd anotherUser
RC Options
Flags to control the Remote Control API
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -7849,6 +7853,38 @@ export RCLONE_CONFIG_PASS
Verbosity is slightly different, the environment variable equivalent of --verbose
or -v
is RCLONE_VERBOSE=1
, or for -vv
, RCLONE_VERBOSE=2
.
The same parser is used for the options and the environment variables so they take exactly the same form.
The options set by environment variables can be seen with the -vv
flag, e.g. rclone version -vv
.
+Options that can appear multiple times (type stringArray
) are treated slighly differently as environment variables can only be defined once. In order to allow a simple mechanism for adding one or many items, the input is treated as a CSV encoded string. For example
+
+
+
+
+
+
+
+
+
+
+RCLONE_EXCLUDE="*.jpg" |
+--exclude "*.jpg" |
+
+
+RCLONE_EXCLUDE="*.jpg,*.png" |
+--exclude "*.jpg" --exclude "*.png" |
+
+
+RCLONE_EXCLUDE='"*.jpg","*.png"' |
+--exclude "*.jpg" --exclude "*.png" |
+
+
+RCLONE_EXCLUDE='"/directory with comma , in it /**"' |
+`--exclude "/directory with comma , in it /**" |
+
+
+
+If stringArray
options are defined as environment variables and options on the command line then all the values will be used.
Config file
You can set defaults for values in the config file on an individual remote basis. The names of the config items are documented in the page for each backend.
To find the name of the environment variable, you need to set, take RCLONE_CONFIG_
+ name of remote + _
+ name of config file option and make it all uppercase. Note one implication here is the remote's name must be convertible into a valid environment variable name, so it can only contain letters, digits, or the _
(underscore) character.
@@ -11932,7 +11968,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.1")
Flags helpful for increasing performance.
--buffer-size SizeSuffix In memory buffer size when reading files for each --transfer (default 16Mi)
@@ -12033,7 +12069,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
RC
Flags to control the Remote Control API.
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -12063,7 +12099,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--rc-web-gui-update Check and update to latest version of web gui
Metrics
Flags to control the Metrics HTTP endpoint..
- --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to (default [""])
+ --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to
--metrics-allow-origin string Origin which cross-domain request (CORS) can be executed from
--metrics-baseurl string Prefix for URLs - leave blank for root
--metrics-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -12551,21 +12587,18 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--pcloud-token string OAuth Access Token as a JSON blob
--pcloud-token-url string Token server url
--pcloud-username string Your pcloud username
- --pikpak-auth-url string Auth server URL
--pikpak-chunk-size SizeSuffix Chunk size for multipart uploads (default 5Mi)
- --pikpak-client-id string OAuth Client Id
- --pikpak-client-secret string OAuth Client Secret
--pikpak-description string Description of the remote
+ --pikpak-device-id string Device ID used for authorization
--pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot)
--pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi)
--pikpak-pass string Pikpak password (obscured)
--pikpak-root-folder-id string ID of the root folder
- --pikpak-token string OAuth Access Token as a JSON blob
- --pikpak-token-url string Token server url
--pikpak-trashed-only Only show files that are in the trash
--pikpak-upload-concurrency int Concurrency for multipart uploads (default 5)
--pikpak-use-trash Send files to the trash instead of deleting permanently (default true)
--pikpak-user string Pikpak username
+ --pikpak-user-agent string HTTP user agent for pikpak (default "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0")
--pixeldrain-api-key string API key for your pixeldrain account
--pixeldrain-api-url string The API endpoint to connect to. In the vast majority of cases it's fine to leave (default "https://pixeldrain.com/api")
--pixeldrain-description string Description of the remote
@@ -14529,6 +14562,18 @@ y/e/d>
By default, rclone will HEAD every object it uploads. It does this to check the object got uploaded correctly.
You can disable this with the --s3-no-head option - see there for more details.
Setting this flag increases the chance for undetected upload failures.
+
+Using server-side copy
+If you are copying objects between S3 buckets in the same region, you should use server-side copy. This is much faster than downloading and re-uploading the objects, as no data is transferred.
+For rclone to use server-side copy, you must use the same remote for the source and destination.
+rclone copy s3:source-bucket s3:destination-bucket
+When using server-side copy, the performance is limited by the rate at which rclone issues API requests to S3. See below for how to increase the number of API requests rclone makes.
+Increasing the rate of API requests
+You can increase the rate of API requests to S3 by increasing the parallelism using --transfers
and --checkers
options.
+Rclone uses a very conservative defaults for these settings, as not all providers support high rates of requests. Depending on your provider, you can increase significantly the number of transfers and checkers.
+For example, with AWS S3, if you can increase the number of checkers to values like 200. If you are doing a server-side copy, you can also increase the number of transfers to 200.
+rclone sync --transfers 200 --checkers 200 --checksum s3:source-bucket s3:destination-bucket
+You will need to experiment with these values to find the optimal settings for your setup.
Versions
When bucket versioning is enabled (this can be done with rclone with the rclone backend versioning
command) when rclone uploads a new version of a file it creates a new version of it Likewise when you delete a file, the old version will be marked hidden and still be available.
Old versions of files, where available, are visible using the --s3-versions
flag.
@@ -18322,7 +18367,7 @@ cos s3
For Netease NOS configure as per the configurator rclone config
setting the provider Netease
. This will automatically set force_path_style = false
which is necessary for it to run properly.
Petabox
Here is an example of making a Petabox configuration. First run:
-
+
This will guide you through an interactive setup process.
No remotes found, make a new one?
n) New remote
@@ -29660,75 +29705,75 @@ rclone rc vfs/refresh recursive=true
Permissions are also supported, if --onedrive-metadata-permissions
is set. The accepted values for --onedrive-metadata-permissions
are "read
", "write
", "read,write
", and "off
" (the default). "write
" supports adding new permissions, updating the "role" of existing permissions, and removing permissions. Updating and removing require the Permission ID to be known, so it is recommended to use "read,write
" instead of "write
" if you wish to update/remove permissions.
Permissions are read/written in JSON format using the same schema as the OneDrive API, which differs slightly between OneDrive Personal and Business.
Example for OneDrive Personal:
-[
- {
- "id": "1234567890ABC!123",
- "grantedTo": {
- "user": {
- "id": "ryan@contoso.com"
- },
- "application": {},
- "device": {}
- },
- "invitation": {
- "email": "ryan@contoso.com"
- },
- "link": {
- "webUrl": "https://1drv.ms/t/s!1234567890ABC"
- },
- "roles": [
- "read"
- ],
- "shareId": "s!1234567890ABC"
- }
-]
+[
+ {
+ "id": "1234567890ABC!123",
+ "grantedTo": {
+ "user": {
+ "id": "ryan@contoso.com"
+ },
+ "application": {},
+ "device": {}
+ },
+ "invitation": {
+ "email": "ryan@contoso.com"
+ },
+ "link": {
+ "webUrl": "https://1drv.ms/t/s!1234567890ABC"
+ },
+ "roles": [
+ "read"
+ ],
+ "shareId": "s!1234567890ABC"
+ }
+]
Example for OneDrive Business:
-[
- {
- "id": "48d31887-5fad-4d73-a9f5-3c356e68a038",
- "grantedToIdentities": [
- {
- "user": {
- "displayName": "ryan@contoso.com"
- },
- "application": {},
- "device": {}
- }
- ],
- "link": {
- "type": "view",
- "scope": "users",
- "webUrl": "https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s"
- },
- "roles": [
- "read"
- ],
- "shareId": "u!LKj1lkdlals90j1nlkascl"
- },
- {
- "id": "5D33DD65C6932946",
- "grantedTo": {
- "user": {
- "displayName": "John Doe",
- "id": "efee1b77-fb3b-4f65-99d6-274c11914d12"
- },
- "application": {},
- "device": {}
- },
- "roles": [
- "owner"
- ],
- "shareId": "FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U"
- }
-]
+[
+ {
+ "id": "48d31887-5fad-4d73-a9f5-3c356e68a038",
+ "grantedToIdentities": [
+ {
+ "user": {
+ "displayName": "ryan@contoso.com"
+ },
+ "application": {},
+ "device": {}
+ }
+ ],
+ "link": {
+ "type": "view",
+ "scope": "users",
+ "webUrl": "https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s"
+ },
+ "roles": [
+ "read"
+ ],
+ "shareId": "u!LKj1lkdlals90j1nlkascl"
+ },
+ {
+ "id": "5D33DD65C6932946",
+ "grantedTo": {
+ "user": {
+ "displayName": "John Doe",
+ "id": "efee1b77-fb3b-4f65-99d6-274c11914d12"
+ },
+ "application": {},
+ "device": {}
+ },
+ "roles": [
+ "owner"
+ ],
+ "shareId": "FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U"
+ }
+]
To write permissions, pass in a "permissions" metadata key using this same format. The --metadata-mapper
tool can be very helpful for this.
When adding permissions, an email address can be provided in the User.ID
or DisplayName
properties of grantedTo
or grantedToIdentities
. Alternatively, an ObjectID can be provided in User.ID
. At least one valid recipient must be provided in order to add a permission for a user. Creating a Public Link is also supported, if Link.Scope
is set to "anonymous"
.
Example request to add a "read" permission with --metadata-mapper
:
-{
- "Metadata": {
- "permissions": "[{\"grantedToIdentities\":[{\"user\":{\"id\":\"ryan@contoso.com\"}}],\"roles\":[\"read\"]}]"
- }
-}
+{
+ "Metadata": {
+ "permissions": "[{\"grantedToIdentities\":[{\"user\":{\"id\":\"ryan@contoso.com\"}}],\"roles\":[\"read\"]}]"
+ }
+}
Note that adding a permission can fail if a conflicting permission already exists for the file/folder.
To update an existing permission, include both the Permission ID and the new roles
to be assigned. roles
is the only property that can be changed.
To remove permissions, pass in a blob containing only the permissions you wish to keep (which can be empty, to remove all.) Note that the owner
role will be ignored, as it cannot be removed.
@@ -32200,54 +32245,24 @@ y/e/d> y
Advanced options
Here are the Advanced options specific to pikpak (PikPak).
---pikpak-client-id
-OAuth Client Id.
-Leave blank normally.
+--pikpak-device-id
+Device ID used for authorization.
Properties:
-- Config: client_id
-- Env Var: RCLONE_PIKPAK_CLIENT_ID
+- Config: device_id
+- Env Var: RCLONE_PIKPAK_DEVICE_ID
- Type: string
- Required: false
---pikpak-client-secret
-OAuth Client Secret.
-Leave blank normally.
+--pikpak-user-agent
+HTTP user agent for pikpak.
+Defaults to "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0" or "--pikpak-user-agent" provided on command line.
Properties:
-- Config: client_secret
-- Env Var: RCLONE_PIKPAK_CLIENT_SECRET
+- Config: user_agent
+- Env Var: RCLONE_PIKPAK_USER_AGENT
- Type: string
-- Required: false
-
---pikpak-token
-OAuth Access Token as a JSON blob.
-Properties:
-
-- Config: token
-- Env Var: RCLONE_PIKPAK_TOKEN
-- Type: string
-- Required: false
-
---pikpak-auth-url
-Auth server URL.
-Leave blank to use the provider defaults.
-Properties:
-
-- Config: auth_url
-- Env Var: RCLONE_PIKPAK_AUTH_URL
-- Type: string
-- Required: false
-
---pikpak-token-url
-Token server url.
-Leave blank to use the provider defaults.
-Properties:
-
-- Config: token_url
-- Env Var: RCLONE_PIKPAK_TOKEN_URL
-- Type: string
-- Required: false
+- Default: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0"
--pikpak-root-folder-id
ID of the root folder. Leave blank normally.
@@ -36949,6 +36964,34 @@ $ tree /tmp/c
"error": return an error based on option value
Changelog
+v1.68.1 - 2024-09-24
+See commits
+
+- Bug Fixes
+
+- build: Fix docker release build (ttionya)
+- doc fixes (Nick Craig-Wood, Pawel Palucha)
+- fs
+
+- Fix
--dump filters
not always appearing (Nick Craig-Wood)
+- Fix setting
stringArray
config values from environment variables (Nick Craig-Wood)
+
+- rc: Fix default value of
--metrics-addr
(Nick Craig-Wood)
+- serve docker: Add missing
vfs-read-chunk-streams
option in docker volume driver (Divyam)
+
+- Onedrive
+
+- Fix spurious "Couldn't decode error response: EOF" DEBUG (Nick Craig-Wood)
+
+- Pikpak
+
+- Fix login issue where token retrieval fails (wiserain)
+
+- S3
+
+- Fix rclone ignoring static credentials when
env_auth=true
(Nick Craig-Wood)
+
+
v1.68.0 - 2024-09-08
See commits
diff --git a/MANUAL.md b/MANUAL.md
index 11e520ba8..cf128a775 100644
--- a/MANUAL.md
+++ b/MANUAL.md
@@ -1,6 +1,6 @@
% rclone(1) User Manual
% Nick Craig-Wood
-% Sep 08, 2024
+% Sep 24, 2024
# Rclone syncs your files to cloud storage
@@ -5259,7 +5259,9 @@ When running in background mode the user will have to stop the mount manually:
# Linux
fusermount -u /path/to/local/mount
- # OS X
+ #... or on some systems
+ fusermount3 -u /path/to/local/mount
+ # OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy.
@@ -5603,9 +5605,9 @@ Note that systemd runs mount units without any environment variables including
`PATH` or `HOME`. This means that tilde (`~`) expansion will not work
and you should provide `--config` and `--cache-dir` explicitly as absolute
paths via rclone arguments.
-Since mounting requires the `fusermount` program, rclone will use the fallback
-PATH of `/bin:/usr/bin` in this scenario. Please ensure that `fusermount`
-is present on this PATH.
+Since mounting requires the `fusermount` or `fusermount3` program,
+rclone will use the fallback PATH of `/bin:/usr/bin` in this scenario.
+Please ensure that `fusermount`/`fusermount3` is present on this PATH.
## Rclone as Unix mount helper
@@ -6472,7 +6474,9 @@ When running in background mode the user will have to stop the mount manually:
# Linux
fusermount -u /path/to/local/mount
- # OS X
+ #... or on some systems
+ fusermount3 -u /path/to/local/mount
+ # OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy.
@@ -6816,9 +6820,9 @@ Note that systemd runs mount units without any environment variables including
`PATH` or `HOME`. This means that tilde (`~`) expansion will not work
and you should provide `--config` and `--cache-dir` explicitly as absolute
paths via rclone arguments.
-Since mounting requires the `fusermount` program, rclone will use the fallback
-PATH of `/bin:/usr/bin` in this scenario. Please ensure that `fusermount`
-is present on this PATH.
+Since mounting requires the `fusermount` or `fusermount3` program,
+rclone will use the fallback PATH of `/bin:/usr/bin` in this scenario.
+Please ensure that `fusermount`/`fusermount3` is present on this PATH.
## Rclone as Unix mount helper
@@ -7734,7 +7738,7 @@ Flags to control the Remote Control API
```
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -16254,6 +16258,22 @@ so they take exactly the same form.
The options set by environment variables can be seen with the `-vv` flag, e.g. `rclone version -vv`.
+Options that can appear multiple times (type `stringArray`) are
+treated slighly differently as environment variables can only be
+defined once. In order to allow a simple mechanism for adding one or
+many items, the input is treated as a [CSV encoded](https://godoc.org/encoding/csv)
+string. For example
+
+| Environment Variable | Equivalent options |
+|----------------------|--------------------|
+| `RCLONE_EXCLUDE="*.jpg"` | `--exclude "*.jpg"` |
+| `RCLONE_EXCLUDE="*.jpg,*.png"` | `--exclude "*.jpg"` `--exclude "*.png"` |
+| `RCLONE_EXCLUDE='"*.jpg","*.png"'` | `--exclude "*.jpg"` `--exclude "*.png"` |
+| `RCLONE_EXCLUDE='"/directory with comma , in it /**"'` | `--exclude "/directory with comma , in it /**" |
+
+If `stringArray` options are defined as environment variables **and**
+options on the command line then all the values will be used.
+
### Config file ###
You can set defaults for values in the config file on an individual
@@ -20474,7 +20494,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.1")
```
@@ -20623,7 +20643,7 @@ Flags to control the Remote Control API.
```
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -20659,7 +20679,7 @@ Flags to control the Remote Control API.
Flags to control the Metrics HTTP endpoint..
```
- --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to (default [""])
+ --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to
--metrics-allow-origin string Origin which cross-domain request (CORS) can be executed from
--metrics-baseurl string Prefix for URLs - leave blank for root
--metrics-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -21153,21 +21173,18 @@ Backend-only flags (these can be set in the config file also).
--pcloud-token string OAuth Access Token as a JSON blob
--pcloud-token-url string Token server url
--pcloud-username string Your pcloud username
- --pikpak-auth-url string Auth server URL
--pikpak-chunk-size SizeSuffix Chunk size for multipart uploads (default 5Mi)
- --pikpak-client-id string OAuth Client Id
- --pikpak-client-secret string OAuth Client Secret
--pikpak-description string Description of the remote
+ --pikpak-device-id string Device ID used for authorization
--pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot)
--pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi)
--pikpak-pass string Pikpak password (obscured)
--pikpak-root-folder-id string ID of the root folder
- --pikpak-token string OAuth Access Token as a JSON blob
- --pikpak-token-url string Token server url
--pikpak-trashed-only Only show files that are in the trash
--pikpak-upload-concurrency int Concurrency for multipart uploads (default 5)
--pikpak-use-trash Send files to the trash instead of deleting permanently (default true)
--pikpak-user string Pikpak username
+ --pikpak-user-agent string HTTP user agent for pikpak (default "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0")
--pixeldrain-api-key string API key for your pixeldrain account
--pixeldrain-api-url string The API endpoint to connect to. In the vast majority of cases it's fine to leave (default "https://pixeldrain.com/api")
--pixeldrain-description string Description of the remote
@@ -24741,6 +24758,38 @@ there for more details.
Setting this flag increases the chance for undetected upload failures.
+### Increasing performance
+
+#### Using server-side copy
+
+If you are copying objects between S3 buckets in the same region, you should
+use server-side copy.
+This is much faster than downloading and re-uploading the objects, as no data is transferred.
+
+For rclone to use server-side copy, you must use the same remote for the source and destination.
+
+ rclone copy s3:source-bucket s3:destination-bucket
+
+When using server-side copy, the performance is limited by the rate at which rclone issues
+API requests to S3.
+See below for how to increase the number of API requests rclone makes.
+
+#### Increasing the rate of API requests
+
+You can increase the rate of API requests to S3 by increasing the parallelism using `--transfers` and `--checkers`
+options.
+
+Rclone uses a very conservative defaults for these settings, as not all providers support high rates of requests.
+Depending on your provider, you can increase significantly the number of transfers and checkers.
+
+For example, with AWS S3, if you can increase the number of checkers to values like 200.
+If you are doing a server-side copy, you can also increase the number of transfers to 200.
+
+ rclone sync --transfers 200 --checkers 200 --checksum s3:source-bucket s3:destination-bucket
+
+You will need to experiment with these values to find the optimal settings for your setup.
+
+
### Versions
When bucket versioning is enabled (this can be done with rclone with
@@ -48038,68 +48087,29 @@ Properties:
Here are the Advanced options specific to pikpak (PikPak).
-#### --pikpak-client-id
+#### --pikpak-device-id
-OAuth Client Id.
-
-Leave blank normally.
+Device ID used for authorization.
Properties:
-- Config: client_id
-- Env Var: RCLONE_PIKPAK_CLIENT_ID
+- Config: device_id
+- Env Var: RCLONE_PIKPAK_DEVICE_ID
- Type: string
- Required: false
-#### --pikpak-client-secret
+#### --pikpak-user-agent
-OAuth Client Secret.
+HTTP user agent for pikpak.
-Leave blank normally.
+Defaults to "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0" or "--pikpak-user-agent" provided on command line.
Properties:
-- Config: client_secret
-- Env Var: RCLONE_PIKPAK_CLIENT_SECRET
+- Config: user_agent
+- Env Var: RCLONE_PIKPAK_USER_AGENT
- Type: string
-- Required: false
-
-#### --pikpak-token
-
-OAuth Access Token as a JSON blob.
-
-Properties:
-
-- Config: token
-- Env Var: RCLONE_PIKPAK_TOKEN
-- Type: string
-- Required: false
-
-#### --pikpak-auth-url
-
-Auth server URL.
-
-Leave blank to use the provider defaults.
-
-Properties:
-
-- Config: auth_url
-- Env Var: RCLONE_PIKPAK_AUTH_URL
-- Type: string
-- Required: false
-
-#### --pikpak-token-url
-
-Token server url.
-
-Leave blank to use the provider defaults.
-
-Properties:
-
-- Config: token_url
-- Env Var: RCLONE_PIKPAK_TOKEN_URL
-- Type: string
-- Required: false
+- Default: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0"
#### --pikpak-root-folder-id
@@ -54602,6 +54612,25 @@ Options:
# Changelog
+## v1.68.1 - 2024-09-24
+
+[See commits](https://github.com/rclone/rclone/compare/v1.68.0...v1.68.1)
+
+* Bug Fixes
+ * build: Fix docker release build (ttionya)
+ * doc fixes (Nick Craig-Wood, Pawel Palucha)
+ * fs
+ * Fix `--dump filters` not always appearing (Nick Craig-Wood)
+ * Fix setting `stringArray` config values from environment variables (Nick Craig-Wood)
+ * rc: Fix default value of `--metrics-addr` (Nick Craig-Wood)
+ * serve docker: Add missing `vfs-read-chunk-streams` option in docker volume driver (Divyam)
+* Onedrive
+ * Fix spurious "Couldn't decode error response: EOF" DEBUG (Nick Craig-Wood)
+* Pikpak
+ * Fix login issue where token retrieval fails (wiserain)
+* S3
+ * Fix rclone ignoring static credentials when `env_auth=true` (Nick Craig-Wood)
+
## v1.68.0 - 2024-09-08
[See commits](https://github.com/rclone/rclone/compare/v1.67.0...v1.68.0)
diff --git a/MANUAL.txt b/MANUAL.txt
index 0c2d0d066..ff0b295a0 100644
--- a/MANUAL.txt
+++ b/MANUAL.txt
@@ -1,6 +1,6 @@
rclone(1) User Manual
Nick Craig-Wood
-Sep 08, 2024
+Sep 24, 2024
Rclone syncs your files to cloud storage
@@ -4843,7 +4843,9 @@ manually:
# Linux
fusermount -u /path/to/local/mount
- # OS X
+ #... or on some systems
+ fusermount3 -u /path/to/local/mount
+ # OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy.
@@ -5188,8 +5190,9 @@ Note that systemd runs mount units without any environment variables
including PATH or HOME. This means that tilde (~) expansion will not
work and you should provide --config and --cache-dir explicitly as
absolute paths via rclone arguments. Since mounting requires the
-fusermount program, rclone will use the fallback PATH of /bin:/usr/bin
-in this scenario. Please ensure that fusermount is present on this PATH.
+fusermount or fusermount3 program, rclone will use the fallback PATH of
+/bin:/usr/bin in this scenario. Please ensure that
+fusermount/fusermount3 is present on this PATH.
Rclone as Unix mount helper
@@ -6027,7 +6030,9 @@ manually:
# Linux
fusermount -u /path/to/local/mount
- # OS X
+ #... or on some systems
+ fusermount3 -u /path/to/local/mount
+ # OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy.
@@ -6372,8 +6377,9 @@ Note that systemd runs mount units without any environment variables
including PATH or HOME. This means that tilde (~) expansion will not
work and you should provide --config and --cache-dir explicitly as
absolute paths via rclone arguments. Since mounting requires the
-fusermount program, rclone will use the fallback PATH of /bin:/usr/bin
-in this scenario. Please ensure that fusermount is present on this PATH.
+fusermount or fusermount3 program, rclone will use the fallback PATH of
+/bin:/usr/bin in this scenario. Please ensure that
+fusermount/fusermount3 is present on this PATH.
Rclone as Unix mount helper
@@ -7298,7 +7304,7 @@ RC Options
Flags to control the Remote Control API
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -15704,6 +15710,29 @@ they take exactly the same form.
The options set by environment variables can be seen with the -vv flag,
e.g. rclone version -vv.
+Options that can appear multiple times (type stringArray) are treated
+slighly differently as environment variables can only be defined once.
+In order to allow a simple mechanism for adding one or many items, the
+input is treated as a CSV encoded string. For example
+
+ ----------------------------------------------------------------------------------------
+ Environment Variable Equivalent options
+ ------------------------------------------------------ ---------------------------------
+ RCLONE_EXCLUDE="*.jpg" --exclude "*.jpg"
+
+ RCLONE_EXCLUDE="*.jpg,*.png" --exclude "*.jpg"
+ --exclude "*.png"
+
+ RCLONE_EXCLUDE='"*.jpg","*.png"' --exclude "*.jpg"
+ --exclude "*.png"
+
+ RCLONE_EXCLUDE='"/directory with comma , in it /**"' `--exclude "/directory with comma
+ , in it /**"
+ ----------------------------------------------------------------------------------------
+
+If stringArray options are defined as environment variables and options
+on the command line then all the values will be used.
+
Config file
You can set defaults for values in the config file on an individual
@@ -20047,7 +20076,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.1")
Performance
@@ -20172,7 +20201,7 @@ RC
Flags to control the Remote Control API.
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -20205,7 +20234,7 @@ Metrics
Flags to control the Metrics HTTP endpoint..
- --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to (default [""])
+ --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to
--metrics-allow-origin string Origin which cross-domain request (CORS) can be executed from
--metrics-baseurl string Prefix for URLs - leave blank for root
--metrics-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -20696,21 +20725,18 @@ Backend-only flags (these can be set in the config file also).
--pcloud-token string OAuth Access Token as a JSON blob
--pcloud-token-url string Token server url
--pcloud-username string Your pcloud username
- --pikpak-auth-url string Auth server URL
--pikpak-chunk-size SizeSuffix Chunk size for multipart uploads (default 5Mi)
- --pikpak-client-id string OAuth Client Id
- --pikpak-client-secret string OAuth Client Secret
--pikpak-description string Description of the remote
+ --pikpak-device-id string Device ID used for authorization
--pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot)
--pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi)
--pikpak-pass string Pikpak password (obscured)
--pikpak-root-folder-id string ID of the root folder
- --pikpak-token string OAuth Access Token as a JSON blob
- --pikpak-token-url string Token server url
--pikpak-trashed-only Only show files that are in the trash
--pikpak-upload-concurrency int Concurrency for multipart uploads (default 5)
--pikpak-use-trash Send files to the trash instead of deleting permanently (default true)
--pikpak-user string Pikpak username
+ --pikpak-user-agent string HTTP user agent for pikpak (default "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0")
--pixeldrain-api-key string API key for your pixeldrain account
--pixeldrain-api-url string The API endpoint to connect to. In the vast majority of cases it's fine to leave (default "https://pixeldrain.com/api")
--pixeldrain-description string Description of the remote
@@ -24275,6 +24301,41 @@ details.
Setting this flag increases the chance for undetected upload failures.
+Increasing performance
+
+Using server-side copy
+
+If you are copying objects between S3 buckets in the same region, you
+should use server-side copy. This is much faster than downloading and
+re-uploading the objects, as no data is transferred.
+
+For rclone to use server-side copy, you must use the same remote for the
+source and destination.
+
+ rclone copy s3:source-bucket s3:destination-bucket
+
+When using server-side copy, the performance is limited by the rate at
+which rclone issues API requests to S3. See below for how to increase
+the number of API requests rclone makes.
+
+Increasing the rate of API requests
+
+You can increase the rate of API requests to S3 by increasing the
+parallelism using --transfers and --checkers options.
+
+Rclone uses a very conservative defaults for these settings, as not all
+providers support high rates of requests. Depending on your provider,
+you can increase significantly the number of transfers and checkers.
+
+For example, with AWS S3, if you can increase the number of checkers to
+values like 200. If you are doing a server-side copy, you can also
+increase the number of transfers to 200.
+
+ rclone sync --transfers 200 --checkers 200 --checksum s3:source-bucket s3:destination-bucket
+
+You will need to experiment with these values to find the optimal
+settings for your setup.
+
Versions
When bucket versioning is enabled (this can be done with rclone with the
@@ -47695,68 +47756,32 @@ Advanced options
Here are the Advanced options specific to pikpak (PikPak).
---pikpak-client-id
+--pikpak-device-id
-OAuth Client Id.
-
-Leave blank normally.
+Device ID used for authorization.
Properties:
-- Config: client_id
-- Env Var: RCLONE_PIKPAK_CLIENT_ID
+- Config: device_id
+- Env Var: RCLONE_PIKPAK_DEVICE_ID
- Type: string
- Required: false
---pikpak-client-secret
+--pikpak-user-agent
-OAuth Client Secret.
+HTTP user agent for pikpak.
-Leave blank normally.
+Defaults to "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0)
+Gecko/20100101 Firefox/129.0" or "--pikpak-user-agent" provided on
+command line.
Properties:
-- Config: client_secret
-- Env Var: RCLONE_PIKPAK_CLIENT_SECRET
+- Config: user_agent
+- Env Var: RCLONE_PIKPAK_USER_AGENT
- Type: string
-- Required: false
-
---pikpak-token
-
-OAuth Access Token as a JSON blob.
-
-Properties:
-
-- Config: token
-- Env Var: RCLONE_PIKPAK_TOKEN
-- Type: string
-- Required: false
-
---pikpak-auth-url
-
-Auth server URL.
-
-Leave blank to use the provider defaults.
-
-Properties:
-
-- Config: auth_url
-- Env Var: RCLONE_PIKPAK_AUTH_URL
-- Type: string
-- Required: false
-
---pikpak-token-url
-
-Token server url.
-
-Leave blank to use the provider defaults.
-
-Properties:
-
-- Config: token_url
-- Env Var: RCLONE_PIKPAK_TOKEN_URL
-- Type: string
-- Required: false
+- Default: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0)
+ Gecko/20100101 Firefox/129.0"
--pikpak-root-folder-id
@@ -54265,6 +54290,29 @@ Options:
Changelog
+v1.68.1 - 2024-09-24
+
+See commits
+
+- Bug Fixes
+ - build: Fix docker release build (ttionya)
+ - doc fixes (Nick Craig-Wood, Pawel Palucha)
+ - fs
+ - Fix --dump filters not always appearing (Nick Craig-Wood)
+ - Fix setting stringArray config values from environment
+ variables (Nick Craig-Wood)
+ - rc: Fix default value of --metrics-addr (Nick Craig-Wood)
+ - serve docker: Add missing vfs-read-chunk-streams option in
+ docker volume driver (Divyam)
+- Onedrive
+ - Fix spurious "Couldn't decode error response: EOF" DEBUG (Nick
+ Craig-Wood)
+- Pikpak
+ - Fix login issue where token retrieval fails (wiserain)
+- S3
+ - Fix rclone ignoring static credentials when env_auth=true (Nick
+ Craig-Wood)
+
v1.68.0 - 2024-09-08
See commits
diff --git a/docs/content/changelog.md b/docs/content/changelog.md
index 4deee4aa3..3f04c6746 100644
--- a/docs/content/changelog.md
+++ b/docs/content/changelog.md
@@ -5,6 +5,25 @@ description: "Rclone Changelog"
# Changelog
+## v1.68.1 - 2024-09-24
+
+[See commits](https://github.com/rclone/rclone/compare/v1.68.0...v1.68.1)
+
+* Bug Fixes
+ * build: Fix docker release build (ttionya)
+ * doc fixes (Nick Craig-Wood, Pawel Palucha)
+ * fs
+ * Fix `--dump filters` not always appearing (Nick Craig-Wood)
+ * Fix setting `stringArray` config values from environment variables (Nick Craig-Wood)
+ * rc: Fix default value of `--metrics-addr` (Nick Craig-Wood)
+ * serve docker: Add missing `vfs-read-chunk-streams` option in docker volume driver (Divyam)
+* Onedrive
+ * Fix spurious "Couldn't decode error response: EOF" DEBUG (Nick Craig-Wood)
+* Pikpak
+ * Fix login issue where token retrieval fails (wiserain)
+* S3
+ * Fix rclone ignoring static credentials when `env_auth=true` (Nick Craig-Wood)
+
## v1.68.0 - 2024-09-08
[See commits](https://github.com/rclone/rclone/compare/v1.67.0...v1.68.0)
diff --git a/docs/content/commands/rclone.md b/docs/content/commands/rclone.md
index 2a39a39f8..7651ff0e9 100644
--- a/docs/content/commands/rclone.md
+++ b/docs/content/commands/rclone.md
@@ -510,7 +510,7 @@ rclone [flags]
--metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin)
--metadata-mapper SpaceSepList Program to run to transforming metadata before upload
--metadata-set stringArray Add metadata key=value when uploading
- --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to (default [""])
+ --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to
--metrics-allow-origin string Origin which cross-domain request (CORS) can be executed from
--metrics-baseurl string Prefix for URLs - leave blank for root
--metrics-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -616,21 +616,18 @@ rclone [flags]
--pcloud-token string OAuth Access Token as a JSON blob
--pcloud-token-url string Token server url
--pcloud-username string Your pcloud username
- --pikpak-auth-url string Auth server URL
--pikpak-chunk-size SizeSuffix Chunk size for multipart uploads (default 5Mi)
- --pikpak-client-id string OAuth Client Id
- --pikpak-client-secret string OAuth Client Secret
--pikpak-description string Description of the remote
+ --pikpak-device-id string Device ID used for authorization
--pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot)
--pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi)
--pikpak-pass string Pikpak password (obscured)
--pikpak-root-folder-id string ID of the root folder
- --pikpak-token string OAuth Access Token as a JSON blob
- --pikpak-token-url string Token server url
--pikpak-trashed-only Only show files that are in the trash
--pikpak-upload-concurrency int Concurrency for multipart uploads (default 5)
--pikpak-use-trash Send files to the trash instead of deleting permanently (default true)
--pikpak-user string Pikpak username
+ --pikpak-user-agent string HTTP user agent for pikpak (default "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0")
--pixeldrain-api-key string API key for your pixeldrain account
--pixeldrain-api-url string The API endpoint to connect to. In the vast majority of cases it's fine to leave (default "https://pixeldrain.com/api")
--pixeldrain-description string Description of the remote
@@ -683,7 +680,7 @@ rclone [flags]
--quatrix-skip-project-folders Skip project folders in operations
-q, --quiet Print as little stuff as possible
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -932,7 +929,7 @@ rclone [flags]
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.1")
-v, --verbose count Print lots more stuff (repeat for more)
-V, --version Print the version number
--webdav-bearer-token string Bearer token instead of user/pass (e.g. a Macaroon)
diff --git a/docs/content/commands/rclone_mount.md b/docs/content/commands/rclone_mount.md
index 67bd27f55..44d423cf1 100644
--- a/docs/content/commands/rclone_mount.md
+++ b/docs/content/commands/rclone_mount.md
@@ -54,7 +54,9 @@ When running in background mode the user will have to stop the mount manually:
# Linux
fusermount -u /path/to/local/mount
- # OS X
+ #... or on some systems
+ fusermount3 -u /path/to/local/mount
+ # OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy.
@@ -398,9 +400,9 @@ Note that systemd runs mount units without any environment variables including
`PATH` or `HOME`. This means that tilde (`~`) expansion will not work
and you should provide `--config` and `--cache-dir` explicitly as absolute
paths via rclone arguments.
-Since mounting requires the `fusermount` program, rclone will use the fallback
-PATH of `/bin:/usr/bin` in this scenario. Please ensure that `fusermount`
-is present on this PATH.
+Since mounting requires the `fusermount` or `fusermount3` program,
+rclone will use the fallback PATH of `/bin:/usr/bin` in this scenario.
+Please ensure that `fusermount`/`fusermount3` is present on this PATH.
## Rclone as Unix mount helper
diff --git a/docs/content/commands/rclone_nfsmount.md b/docs/content/commands/rclone_nfsmount.md
index 5230b00d9..62e5b295c 100644
--- a/docs/content/commands/rclone_nfsmount.md
+++ b/docs/content/commands/rclone_nfsmount.md
@@ -55,7 +55,9 @@ When running in background mode the user will have to stop the mount manually:
# Linux
fusermount -u /path/to/local/mount
- # OS X
+ #... or on some systems
+ fusermount3 -u /path/to/local/mount
+ # OS X or Linux when using nfsmount
umount /path/to/local/mount
The umount operation can fail, for example when the mountpoint is busy.
@@ -399,9 +401,9 @@ Note that systemd runs mount units without any environment variables including
`PATH` or `HOME`. This means that tilde (`~`) expansion will not work
and you should provide `--config` and `--cache-dir` explicitly as absolute
paths via rclone arguments.
-Since mounting requires the `fusermount` program, rclone will use the fallback
-PATH of `/bin:/usr/bin` in this scenario. Please ensure that `fusermount`
-is present on this PATH.
+Since mounting requires the `fusermount` or `fusermount3` program,
+rclone will use the fallback PATH of `/bin:/usr/bin` in this scenario.
+Please ensure that `fusermount`/`fusermount3` is present on this PATH.
## Rclone as Unix mount helper
diff --git a/docs/content/commands/rclone_rcd.md b/docs/content/commands/rclone_rcd.md
index 3750bf477..c5a2c6e32 100644
--- a/docs/content/commands/rclone_rcd.md
+++ b/docs/content/commands/rclone_rcd.md
@@ -166,7 +166,7 @@ Flags to control the Remote Control API
```
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
diff --git a/docs/content/flags.md b/docs/content/flags.md
index c2908561a..ff4cd0955 100644
--- a/docs/content/flags.md
+++ b/docs/content/flags.md
@@ -115,7 +115,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.1")
```
@@ -264,7 +264,7 @@ Flags to control the Remote Control API.
```
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default ["localhost:5572"])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -300,7 +300,7 @@ Flags to control the Remote Control API.
Flags to control the Metrics HTTP endpoint..
```
- --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to (default [""])
+ --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to
--metrics-allow-origin string Origin which cross-domain request (CORS) can be executed from
--metrics-baseurl string Prefix for URLs - leave blank for root
--metrics-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -794,21 +794,18 @@ Backend-only flags (these can be set in the config file also).
--pcloud-token string OAuth Access Token as a JSON blob
--pcloud-token-url string Token server url
--pcloud-username string Your pcloud username
- --pikpak-auth-url string Auth server URL
--pikpak-chunk-size SizeSuffix Chunk size for multipart uploads (default 5Mi)
- --pikpak-client-id string OAuth Client Id
- --pikpak-client-secret string OAuth Client Secret
--pikpak-description string Description of the remote
+ --pikpak-device-id string Device ID used for authorization
--pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot)
--pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi)
--pikpak-pass string Pikpak password (obscured)
--pikpak-root-folder-id string ID of the root folder
- --pikpak-token string OAuth Access Token as a JSON blob
- --pikpak-token-url string Token server url
--pikpak-trashed-only Only show files that are in the trash
--pikpak-upload-concurrency int Concurrency for multipart uploads (default 5)
--pikpak-use-trash Send files to the trash instead of deleting permanently (default true)
--pikpak-user string Pikpak username
+ --pikpak-user-agent string HTTP user agent for pikpak (default "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0")
--pixeldrain-api-key string API key for your pixeldrain account
--pixeldrain-api-url string The API endpoint to connect to. In the vast majority of cases it's fine to leave (default "https://pixeldrain.com/api")
--pixeldrain-description string Description of the remote
diff --git a/docs/content/pikpak.md b/docs/content/pikpak.md
index 6c2fab232..9fcf925a9 100644
--- a/docs/content/pikpak.md
+++ b/docs/content/pikpak.md
@@ -111,68 +111,29 @@ Properties:
Here are the Advanced options specific to pikpak (PikPak).
-#### --pikpak-client-id
+#### --pikpak-device-id
-OAuth Client Id.
-
-Leave blank normally.
+Device ID used for authorization.
Properties:
-- Config: client_id
-- Env Var: RCLONE_PIKPAK_CLIENT_ID
+- Config: device_id
+- Env Var: RCLONE_PIKPAK_DEVICE_ID
- Type: string
- Required: false
-#### --pikpak-client-secret
+#### --pikpak-user-agent
-OAuth Client Secret.
+HTTP user agent for pikpak.
-Leave blank normally.
+Defaults to "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0" or "--pikpak-user-agent" provided on command line.
Properties:
-- Config: client_secret
-- Env Var: RCLONE_PIKPAK_CLIENT_SECRET
+- Config: user_agent
+- Env Var: RCLONE_PIKPAK_USER_AGENT
- Type: string
-- Required: false
-
-#### --pikpak-token
-
-OAuth Access Token as a JSON blob.
-
-Properties:
-
-- Config: token
-- Env Var: RCLONE_PIKPAK_TOKEN
-- Type: string
-- Required: false
-
-#### --pikpak-auth-url
-
-Auth server URL.
-
-Leave blank to use the provider defaults.
-
-Properties:
-
-- Config: auth_url
-- Env Var: RCLONE_PIKPAK_AUTH_URL
-- Type: string
-- Required: false
-
-#### --pikpak-token-url
-
-Token server url.
-
-Leave blank to use the provider defaults.
-
-Properties:
-
-- Config: token_url
-- Env Var: RCLONE_PIKPAK_TOKEN_URL
-- Type: string
-- Required: false
+- Default: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0"
#### --pikpak-root-folder-id
diff --git a/rclone.1 b/rclone.1
index 12a2f7604..1ec0f04cd 100644
--- a/rclone.1
+++ b/rclone.1
@@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 2.9.2.1
.\"
-.TH "rclone" "1" "Sep 08, 2024" "User Manual" ""
+.TH "rclone" "1" "Sep 24, 2024" "User Manual" ""
.hy
.SH Rclone syncs your files to cloud storage
.PP
@@ -6429,7 +6429,9 @@ manually:
\f[C]
# Linux
fusermount -u /path/to/local/mount
-# OS X
+#... or on some systems
+fusermount3 -u /path/to/local/mount
+# OS X or Linux when using nfsmount
umount /path/to/local/mount
\f[R]
.fi
@@ -6859,9 +6861,11 @@ including \f[C]PATH\f[R] or \f[C]HOME\f[R].
This means that tilde (\f[C]\[ti]\f[R]) expansion will not work and you
should provide \f[C]--config\f[R] and \f[C]--cache-dir\f[R] explicitly
as absolute paths via rclone arguments.
-Since mounting requires the \f[C]fusermount\f[R] program, rclone will
-use the fallback PATH of \f[C]/bin:/usr/bin\f[R] in this scenario.
-Please ensure that \f[C]fusermount\f[R] is present on this PATH.
+Since mounting requires the \f[C]fusermount\f[R] or
+\f[C]fusermount3\f[R] program, rclone will use the fallback PATH of
+\f[C]/bin:/usr/bin\f[R] in this scenario.
+Please ensure that \f[C]fusermount\f[R]/\f[C]fusermount3\f[R] is present
+on this PATH.
.SS Rclone as Unix mount helper
.PP
The core Unix program \f[C]/bin/mount\f[R] normally takes the
@@ -7886,7 +7890,9 @@ manually:
\f[C]
# Linux
fusermount -u /path/to/local/mount
-# OS X
+#... or on some systems
+fusermount3 -u /path/to/local/mount
+# OS X or Linux when using nfsmount
umount /path/to/local/mount
\f[R]
.fi
@@ -8317,9 +8323,11 @@ including \f[C]PATH\f[R] or \f[C]HOME\f[R].
This means that tilde (\f[C]\[ti]\f[R]) expansion will not work and you
should provide \f[C]--config\f[R] and \f[C]--cache-dir\f[R] explicitly
as absolute paths via rclone arguments.
-Since mounting requires the \f[C]fusermount\f[R] program, rclone will
-use the fallback PATH of \f[C]/bin:/usr/bin\f[R] in this scenario.
-Please ensure that \f[C]fusermount\f[R] is present on this PATH.
+Since mounting requires the \f[C]fusermount\f[R] or
+\f[C]fusermount3\f[R] program, rclone will use the fallback PATH of
+\f[C]/bin:/usr/bin\f[R] in this scenario.
+Please ensure that \f[C]fusermount\f[R]/\f[C]fusermount3\f[R] is present
+on this PATH.
.SS Rclone as Unix mount helper
.PP
The core Unix program \f[C]/bin/mount\f[R] normally takes the
@@ -9527,7 +9535,7 @@ Flags to control the Remote Control API
.nf
\f[C]
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default [\[dq]localhost:5572\[dq]])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -19836,6 +19844,49 @@ they take exactly the same form.
The options set by environment variables can be seen with the
\f[C]-vv\f[R] flag, e.g.
\f[C]rclone version -vv\f[R].
+.PP
+Options that can appear multiple times (type \f[C]stringArray\f[R]) are
+treated slighly differently as environment variables can only be defined
+once.
+In order to allow a simple mechanism for adding one or many items, the
+input is treated as a CSV encoded (https://godoc.org/encoding/csv)
+string.
+For example
+.PP
+.TS
+tab(@);
+lw(36.7n) lw(33.3n).
+T{
+Environment Variable
+T}@T{
+Equivalent options
+T}
+_
+T{
+\f[C]RCLONE_EXCLUDE=\[dq]*.jpg\[dq]\f[R]
+T}@T{
+\f[C]--exclude \[dq]*.jpg\[dq]\f[R]
+T}
+T{
+\f[C]RCLONE_EXCLUDE=\[dq]*.jpg,*.png\[dq]\f[R]
+T}@T{
+\f[C]--exclude \[dq]*.jpg\[dq]\f[R] \f[C]--exclude \[dq]*.png\[dq]\f[R]
+T}
+T{
+\f[C]RCLONE_EXCLUDE=\[aq]\[dq]*.jpg\[dq],\[dq]*.png\[dq]\[aq]\f[R]
+T}@T{
+\f[C]--exclude \[dq]*.jpg\[dq]\f[R] \f[C]--exclude \[dq]*.png\[dq]\f[R]
+T}
+T{
+\f[C]RCLONE_EXCLUDE=\[aq]\[dq]/directory with comma , in it /**\[dq]\[aq]\f[R]
+T}@T{
+\[ga]--exclude \[dq]/directory with comma , in it /**\[dq]
+T}
+.TE
+.PP
+If \f[C]stringArray\f[R] options are defined as environment variables
+\f[B]and\f[R] options on the command line then all the values will be
+used.
.SS Config file
.PP
You can set defaults for values in the config file on an individual
@@ -27668,7 +27719,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
- --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.68.0\[dq])
+ --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.68.1\[dq])
\f[R]
.fi
.SS Performance
@@ -27817,7 +27868,7 @@ Flags to control the Remote Control API.
.nf
\f[C]
--rc Enable the remote control server
- --rc-addr stringArray IPaddress:Port or :Port to bind server to (default [\[dq]localhost:5572\[dq]])
+ --rc-addr stringArray IPaddress:Port or :Port to bind server to (default localhost:5572)
--rc-allow-origin string Origin which cross-domain request (CORS) can be executed from
--rc-baseurl string Prefix for URLs - leave blank for root
--rc-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -27853,7 +27904,7 @@ Flags to control the Metrics HTTP endpoint..
.IP
.nf
\f[C]
- --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to (default [\[dq]\[dq]])
+ --metrics-addr stringArray IPaddress:Port or :Port to bind metrics server to
--metrics-allow-origin string Origin which cross-domain request (CORS) can be executed from
--metrics-baseurl string Prefix for URLs - leave blank for root
--metrics-cert string TLS PEM key (concatenation of certificate and CA certificate)
@@ -28347,21 +28398,18 @@ Backend-only flags (these can be set in the config file also).
--pcloud-token string OAuth Access Token as a JSON blob
--pcloud-token-url string Token server url
--pcloud-username string Your pcloud username
- --pikpak-auth-url string Auth server URL
--pikpak-chunk-size SizeSuffix Chunk size for multipart uploads (default 5Mi)
- --pikpak-client-id string OAuth Client Id
- --pikpak-client-secret string OAuth Client Secret
--pikpak-description string Description of the remote
+ --pikpak-device-id string Device ID used for authorization
--pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot)
--pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi)
--pikpak-pass string Pikpak password (obscured)
--pikpak-root-folder-id string ID of the root folder
- --pikpak-token string OAuth Access Token as a JSON blob
- --pikpak-token-url string Token server url
--pikpak-trashed-only Only show files that are in the trash
--pikpak-upload-concurrency int Concurrency for multipart uploads (default 5)
--pikpak-use-trash Send files to the trash instead of deleting permanently (default true)
--pikpak-user string Pikpak username
+ --pikpak-user-agent string HTTP user agent for pikpak (default \[dq]Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0\[dq])
--pixeldrain-api-key string API key for your pixeldrain account
--pixeldrain-api-url string The API endpoint to connect to. In the vast majority of cases it\[aq]s fine to leave (default \[dq]https://pixeldrain.com/api\[dq])
--pixeldrain-description string Description of the remote
@@ -33001,6 +33049,50 @@ You can disable this with the --s3-no-head option - see there for more
details.
.PP
Setting this flag increases the chance for undetected upload failures.
+.SS Increasing performance
+.SS Using server-side copy
+.PP
+If you are copying objects between S3 buckets in the same region, you
+should use server-side copy.
+This is much faster than downloading and re-uploading the objects, as no
+data is transferred.
+.PP
+For rclone to use server-side copy, you must use the same remote for the
+source and destination.
+.IP
+.nf
+\f[C]
+rclone copy s3:source-bucket s3:destination-bucket
+\f[R]
+.fi
+.PP
+When using server-side copy, the performance is limited by the rate at
+which rclone issues API requests to S3.
+See below for how to increase the number of API requests rclone makes.
+.SS Increasing the rate of API requests
+.PP
+You can increase the rate of API requests to S3 by increasing the
+parallelism using \f[C]--transfers\f[R] and \f[C]--checkers\f[R]
+options.
+.PP
+Rclone uses a very conservative defaults for these settings, as not all
+providers support high rates of requests.
+Depending on your provider, you can increase significantly the number of
+transfers and checkers.
+.PP
+For example, with AWS S3, if you can increase the number of checkers to
+values like 200.
+If you are doing a server-side copy, you can also increase the number of
+transfers to 200.
+.IP
+.nf
+\f[C]
+rclone sync --transfers 200 --checkers 200 --checksum s3:source-bucket s3:destination-bucket
+\f[R]
+.fi
+.PP
+You will need to experiment with these values to find the optimal
+settings for your setup.
.SS Versions
.PP
When bucket versioning is enabled (this can be done with rclone with the
@@ -63841,79 +63933,37 @@ Required: true
.SS Advanced options
.PP
Here are the Advanced options specific to pikpak (PikPak).
-.SS --pikpak-client-id
+.SS --pikpak-device-id
.PP
-OAuth Client Id.
-.PP
-Leave blank normally.
+Device ID used for authorization.
.PP
Properties:
.IP \[bu] 2
-Config: client_id
+Config: device_id
.IP \[bu] 2
-Env Var: RCLONE_PIKPAK_CLIENT_ID
+Env Var: RCLONE_PIKPAK_DEVICE_ID
.IP \[bu] 2
Type: string
.IP \[bu] 2
Required: false
-.SS --pikpak-client-secret
+.SS --pikpak-user-agent
.PP
-OAuth Client Secret.
+HTTP user agent for pikpak.
.PP
-Leave blank normally.
+Defaults to \[dq]Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0)
+Gecko/20100101 Firefox/129.0\[dq] or \[dq]--pikpak-user-agent\[dq]
+provided on command line.
.PP
Properties:
.IP \[bu] 2
-Config: client_secret
+Config: user_agent
.IP \[bu] 2
-Env Var: RCLONE_PIKPAK_CLIENT_SECRET
+Env Var: RCLONE_PIKPAK_USER_AGENT
.IP \[bu] 2
Type: string
.IP \[bu] 2
-Required: false
-.SS --pikpak-token
-.PP
-OAuth Access Token as a JSON blob.
-.PP
-Properties:
-.IP \[bu] 2
-Config: token
-.IP \[bu] 2
-Env Var: RCLONE_PIKPAK_TOKEN
-.IP \[bu] 2
-Type: string
-.IP \[bu] 2
-Required: false
-.SS --pikpak-auth-url
-.PP
-Auth server URL.
-.PP
-Leave blank to use the provider defaults.
-.PP
-Properties:
-.IP \[bu] 2
-Config: auth_url
-.IP \[bu] 2
-Env Var: RCLONE_PIKPAK_AUTH_URL
-.IP \[bu] 2
-Type: string
-.IP \[bu] 2
-Required: false
-.SS --pikpak-token-url
-.PP
-Token server url.
-.PP
-Leave blank to use the provider defaults.
-.PP
-Properties:
-.IP \[bu] 2
-Config: token_url
-.IP \[bu] 2
-Env Var: RCLONE_PIKPAK_TOKEN_URL
-.IP \[bu] 2
-Type: string
-.IP \[bu] 2
-Required: false
+Default: \[dq]Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0)
+Gecko/20100101 Firefox/129.0\[dq]
.SS --pikpak-root-folder-id
.PP
ID of the root folder.
@@ -72472,6 +72522,51 @@ Options:
.IP \[bu] 2
\[dq]error\[dq]: return an error based on option value
.SH Changelog
+.SS v1.68.1 - 2024-09-24
+.PP
+See commits (https://github.com/rclone/rclone/compare/v1.68.0...v1.68.1)
+.IP \[bu] 2
+Bug Fixes
+.RS 2
+.IP \[bu] 2
+build: Fix docker release build (ttionya)
+.IP \[bu] 2
+doc fixes (Nick Craig-Wood, Pawel Palucha)
+.IP \[bu] 2
+fs
+.RS 2
+.IP \[bu] 2
+Fix \f[C]--dump filters\f[R] not always appearing (Nick Craig-Wood)
+.IP \[bu] 2
+Fix setting \f[C]stringArray\f[R] config values from environment
+variables (Nick Craig-Wood)
+.RE
+.IP \[bu] 2
+rc: Fix default value of \f[C]--metrics-addr\f[R] (Nick Craig-Wood)
+.IP \[bu] 2
+serve docker: Add missing \f[C]vfs-read-chunk-streams\f[R] option in
+docker volume driver (Divyam)
+.RE
+.IP \[bu] 2
+Onedrive
+.RS 2
+.IP \[bu] 2
+Fix spurious \[dq]Couldn\[aq]t decode error response: EOF\[dq] DEBUG
+(Nick Craig-Wood)
+.RE
+.IP \[bu] 2
+Pikpak
+.RS 2
+.IP \[bu] 2
+Fix login issue where token retrieval fails (wiserain)
+.RE
+.IP \[bu] 2
+S3
+.RS 2
+.IP \[bu] 2
+Fix rclone ignoring static credentials when \f[C]env_auth=true\f[R]
+(Nick Craig-Wood)
+.RE
.SS v1.68.0 - 2024-09-08
.PP
See commits (https://github.com/rclone/rclone/compare/v1.67.0...v1.68.0)