From 12b572e62a10d9aa2e81449659d60fc98e4706f2 Mon Sep 17 00:00:00 2001 From: Aleksey Kravchenko Date: Wed, 22 Jan 2025 19:23:42 +0300 Subject: [PATCH] [#5] Update frostfs backend docs Signed-off-by: Aleksey Kravchenko --- MANUAL.html | 230 ++++++++++++++++++-------------- MANUAL.md | 37 ++++- MANUAL.txt | 41 +++++- docs/content/commands/rclone.md | 3 +- docs/content/flags.md | 3 +- docs/content/frostfs.md | 24 ++++ rclone.1 | 44 +++++- 7 files changed, 258 insertions(+), 124 deletions(-) diff --git a/MANUAL.html b/MANUAL.html index 3c480012e..eb4138e2b 100644 --- a/MANUAL.html +++ b/MANUAL.html @@ -233,7 +233,7 @@

rclone(1) User Manual

Nick Craig-Wood

-

Jan 10, 2025

+

Jan 23, 2025

Rclone syncs your files to cloud storage

@@ -19294,7 +19294,7 @@ split into groups.

--tpslimit float Limit HTTP transactions per second to this --tpslimit-burst int Max burst of transactions for --tpslimit (default 1) --use-cookies Enable session cookiejar - --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support") + --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8335.a85292ca0.feature/add-container-zones-support")

Performance

Flags helpful for increasing performance.

      --buffer-size SizeSuffix   In memory buffer size when reading files for each --transfer (default 16Mi)
@@ -19677,6 +19677,7 @@ split into groups.

--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms) --frostfs-connection-timeout Duration FrostFS connection timeout (default 4s) --frostfs-container-creation-policy string Container creation policy for new containers (default "private") + --frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container") --frostfs-description string Description of the remote --frostfs-endpoint string Endpoints to connect to FrostFS node --frostfs-password string Password to decrypt wallet @@ -32844,7 +32845,7 @@ connection_timeout> Option request_timeout. FrostFS request timeout -Enter a value of type Duration. Press Enter for the default (4s). +Enter a value of type Duration. Press Enter for the default (12s). request_timeout> Option rebalance_interval. @@ -32864,7 +32865,7 @@ ape_cache_invalidation_duration> Option ape_cache_invalidation_timeout. APE cache invalidation timeout -Enter a value of type Duration. Press Enter for the default (16s). +Enter a value of type Duration. Press Enter for the default (24s). ape_cache_invalidation_timeout> Option ape_chain_check_interval. @@ -32900,6 +32901,11 @@ Press Enter for the default (REP 3). \ (REP 3) placement_policy> REP 1 +Option default_container_zone. +The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty. +Enter a value of type string. Press Enter for the default (container). +default_container_zone> + Option container_creation_policy. Container creation policy for new containers Choose a number from below, or type in your own value of type string. @@ -32938,6 +32944,13 @@ identifier 23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf:

rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy
 rclone copy ~/test-copy remote:container-name/test-copy
+

Also, for user-friendly container names, you can explicitly specify +the name of the zone in which you want to create or search for a +container:

+
rclone copy ~/test-copy remote:container-name.container-zone/test-copy
+

If the zone is not explicitly specified, its name will be obtained +from the configuration parameter +default_container_zone.

Standard options

Here are the Standard options specific to frostfs (Distributed, decentralized object storage FrostFS).

@@ -33093,6 +33106,18 @@ FrostFS.

--frostfs-default-container-zone

+

The name of the zone in which containers will be created or resolved +if the zone name is not explicitly specified with the container +name.

+

Properties:

+
    +
  • Config: default_container_zone
  • +
  • Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE
  • +
  • Type: string
  • +
  • Default: "container"
  • +
+

--frostfs-container-creation-policy

Container creation policy for new containers

Properties:

@@ -33234,15 +33259,14 @@ password "dummy", and therefore works even in Windows Command Prompt:

rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM
 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:
-

Implicit TLSutil.go

+

Implicit TLS

Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be enabled in the FTP backend config for the remote, or with --ftp-tls. The default FTPS port is 990, not 21 and can be set with --ftp-port. util.go ### Restricted filename -characters

+href="#ftp-port">--ftp-port.

+

Restricted filename +characters

In addition to the default restricted characters set the following characters are also @@ -33653,7 +33677,7 @@ hashes

Gofile supports modification times with a resolution of 1 second.

Gofile supports MD5 hashes, so you can use the --checksum flag.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -34086,7 +34110,7 @@ gsutil only contain timestamps with one-second precision. If you use rclone to sync files previously uploaded by gsutil, rclone will attempt to update modification time for all these files. To avoid these possibly unnecessary updates, use --modify-window 1s.

-

Restricted filename +

Restricted filename characters

@@ -34952,7 +34976,7 @@ hashes

Hash algorithms MD5, SHA1 and SHA256 are supported. Note, however, that a small fraction of files uploaded may not have SHA1 or SHA256 hashes especially if they were uploaded before 2018.

-

Restricted filename +

Restricted filename characters

Only Invalid UTF-8 bytes will be replaced, as they @@ -37377,7 +37401,7 @@ be lost.)

Usage information

You can use the rclone about remote: command which will display filesystem size and current usage.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -37601,7 +37625,7 @@ second.

HiDrive supports its own hash type which is used to verify the integrity of file contents after successful transfers.

-

Restricted filename +

Restricted filename characters

HiDrive cannot store files or folders that include / (0x2F) or null-bytes (0x00) in their name. Any other characters can be @@ -38994,7 +39018,7 @@ available, so this does not apply. Starting with rclone version 1.52 the same is true for encrypted remotes (in older versions the crypt backend would not calculate hashes for uploads from local disk, so the Jottacloud backend had to do it as described above).

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -39346,7 +39370,7 @@ choose an alternative mount instead of your primary storage.

rclone ls koofr:

To copy a local directory to an Koofr directory called backup

rclone copy /home/source koofr:backup
-

Restricted filename +

Restricted filename characters

In addition to the default @@ -39808,7 +39832,7 @@ command does not take any path arguments.

To view your current quota you can use the rclone about remote: command which will display your usage limit (quota) and the current usage.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -40197,7 +40221,7 @@ use the credentials in rclone will fail.

Modification times and hashes

Mega does not support modification times or hashes yet.

-

Restricted filename +

Restricted filename characters

@@ -40436,7 +40460,7 @@ rclone serve sftp :memory: hashes

The memory backend supports MD5 hashes and modification times accurate to 1 nS.

-

Restricted filename +

Restricted filename characters

The memory backend replaces the default @@ -40821,7 +40845,7 @@ the cost of using more memory. The default of 16 is set quite conservatively to use less memory. It maybe be necessary raise it to 64 or higher to fully utilize a 1 GBit/s link with a single file transfer.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -41666,7 +41690,7 @@ at the cost of using more memory. The default of 16 is set quite conservatively to use less memory. It maybe be necessary raise it to 64 or higher to fully utilize a 1 GBit/s link with a single file transfer.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -42477,7 +42501,7 @@ using.

Some commands (like rclone lsf -R) will use ListR by default - you can turn this off with --disable ListR if you need to.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -43068,69 +43092,69 @@ href="https://learn.microsoft.com/en-us/onedrive/developer/rest-api/resources/pe API, which differs slightly between OneDrive Personal and Business.

Example for OneDrive Personal:

-
[
-    {
-        "id": "1234567890ABC!123",
-        "grantedTo": {
-            "user": {
-                "id": "ryan@contoso.com"
-            },
-            "application": {},
-            "device": {}
-        },
-        "invitation": {
-            "email": "ryan@contoso.com"
-        },
-        "link": {
-            "webUrl": "https://1drv.ms/t/s!1234567890ABC"
-        },
-        "roles": [
-            "read"
-        ],
-        "shareId": "s!1234567890ABC"
-    }
-]
-

Example for OneDrive Business:

[
     {
-        "id": "48d31887-5fad-4d73-a9f5-3c356e68a038",
-        "grantedToIdentities": [
-            {
-                "user": {
-                    "displayName": "ryan@contoso.com"
-                },
-                "application": {},
-                "device": {}
-            }
-        ],
-        "link": {
-            "type": "view",
-            "scope": "users",
-            "webUrl": "https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s"
-        },
-        "roles": [
-            "read"
-        ],
-        "shareId": "u!LKj1lkdlals90j1nlkascl"
-    },
-    {
-        "id": "5D33DD65C6932946",
-        "grantedTo": {
-            "user": {
-                "displayName": "John Doe",
-                "id": "efee1b77-fb3b-4f65-99d6-274c11914d12"
-            },
-            "application": {},
-            "device": {}
-        },
-        "roles": [
-            "owner"
-        ],
-        "shareId": "FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U"
-    }
-]
+ "id": "1234567890ABC!123", + "grantedTo": { + "user": { + "id": "ryan@contoso.com" + }, + "application": {}, + "device": {} + }, + "invitation": { + "email": "ryan@contoso.com" + }, + "link": { + "webUrl": "https://1drv.ms/t/s!1234567890ABC" + }, + "roles": [ + "read" + ], + "shareId": "s!1234567890ABC" + } +] +

Example for OneDrive Business:

+
[
+    {
+        "id": "48d31887-5fad-4d73-a9f5-3c356e68a038",
+        "grantedToIdentities": [
+            {
+                "user": {
+                    "displayName": "ryan@contoso.com"
+                },
+                "application": {},
+                "device": {}
+            }
+        ],
+        "link": {
+            "type": "view",
+            "scope": "users",
+            "webUrl": "https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s"
+        },
+        "roles": [
+            "read"
+        ],
+        "shareId": "u!LKj1lkdlals90j1nlkascl"
+    },
+    {
+        "id": "5D33DD65C6932946",
+        "grantedTo": {
+            "user": {
+                "displayName": "John Doe",
+                "id": "efee1b77-fb3b-4f65-99d6-274c11914d12"
+            },
+            "application": {},
+            "device": {}
+        },
+        "roles": [
+            "owner"
+        ],
+        "shareId": "FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U"
+    }
+]

To write permissions, pass in a "permissions" metadata key using this same format. The --metadata-mapper @@ -43144,12 +43168,12 @@ for a user. Creating a Public Link is also supported, if Link.Scope is set to "anonymous".

Example request to add a "read" permission with --metadata-mapper:

-
{
-    "Metadata": {
-        "permissions": "[{\"grantedToIdentities\":[{\"user\":{\"id\":\"ryan@contoso.com\"}}],\"roles\":[\"read\"]}]"
-    }
-}
+
{
+    "Metadata": {
+        "permissions": "[{\"grantedToIdentities\":[{\"user\":{\"id\":\"ryan@contoso.com\"}}],\"roles\":[\"read\"]}]"
+    }
+}

Note that adding a permission can fail if a conflicting permission already exists for the file/folder.

To update an existing permission, include both the Permission ID and @@ -43615,7 +43639,7 @@ hashes 1 second. These will be used to detect whether objects need syncing or not.

The MD5 hash algorithm is supported.

-

Restricted filename +

Restricted filename characters

@@ -44687,7 +44711,7 @@ file -

Restricted filename +

Restricted filename characters

The control characters 0x00-0x1F and / are replaced as in the default @@ -44972,7 +44996,7 @@ microsecond. These will be used to detect whether objects need syncing or not.

Quatrix does not support hashes, so you cannot use the --checksum flag.

-

Restricted filename +

Restricted filename characters

File names in Quatrix are case sensitive and have limitations like the maximum length of a filename is 255, and the minimum length is 1. A @@ -45501,7 +45525,7 @@ accurate to 1 ns.

This is a de facto standard (used in the official python-swiftclient amongst others) for storing the modification time for an object.

The MD5 hash algorithm is supported.

-

Restricted filename +

Restricted filename characters

@@ -46051,7 +46075,7 @@ re-uploaded.

pCloud supports MD5 and SHA1 hashes in the US region, and SHA1 and SHA256 hashes in the EU region, so you can use the --checksum flag.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -46748,7 +46772,7 @@ hashes

premiumize.me does not support modification times or hashes, therefore syncing will default to --size-only checking. Note that using --update will work.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -46949,7 +46973,7 @@ hashes

Proton Drive Bridge does not support updating modification times yet.

The SHA1 hash algorithm is supported.

-

Restricted filename +

Restricted filename characters

Invalid UTF-8 bytes will be replaced, also left @@ -47262,7 +47286,7 @@ mode.

rclone ls remote:

To copy a local directory to a put.io directory called backup

rclone copy /home/source remote:backup
-

Restricted filename +

Restricted filename characters

In addition to the default @@ -47445,7 +47469,7 @@ hashes

Proton Drive Bridge does not support updating modification times yet.

The SHA1 hash algorithm is supported.

-

Restricted filename +

Restricted filename characters

Invalid UTF-8 bytes will be replaced, also left @@ -47869,7 +47893,7 @@ to use fewer transactions in exchange for more memory. See the rclone docs for more details. Please note this is not supported on seafile server version 6.x

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -49679,7 +49703,7 @@ hashes syncing will default to --size-only checking. Note that using --update will work as rclone can read the time files were uploaded.

-

Restricted filename +

Restricted filename characters

SugarSync replaces the default @@ -49907,7 +49931,7 @@ and hashes. Timestamps are stored with microsecond precision.

Afterwards, the backend only serves the client-side calculated hashes. Hashes can also be retrieved upon creating a file download link, but it's impractical for list-like use cases.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -50122,7 +50146,7 @@ y/e/d> hashes

Uptobox supports neither modified times nor checksums. All timestamps will read as that set by --default-time.

-

Restricted filename +

Restricted filename characters

In addition to the default @@ -51129,7 +51153,7 @@ arguments.

To view your current quota you can use the rclone about remote: command which will display your usage limit (quota) and the current usage.

-

Restricted filename +

Restricted filename characters

The default @@ -51339,7 +51363,7 @@ hashes

To view your current quota you can use the rclone about remote: command which will display your current usage.

-

Restricted filename +

Restricted filename characters

Only control characters and invalid UTF-8 are replaced. In addition most Unicode full-width characters are not supported at all and will be diff --git a/MANUAL.md b/MANUAL.md index db8468c13..d3c9c94ab 100644 --- a/MANUAL.md +++ b/MANUAL.md @@ -1,6 +1,6 @@ % rclone(1) User Manual % Nick Craig-Wood -% Jan 10, 2025 +% Jan 23, 2025 # Rclone syncs your files to cloud storage @@ -20498,7 +20498,7 @@ Flags for general networking and HTTP stuff. --tpslimit float Limit HTTP transactions per second to this --tpslimit-burst int Max burst of transactions for --tpslimit (default 1) --use-cookies Enable session cookiejar - --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support") + --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8335.a85292ca0.feature/add-container-zones-support") ``` @@ -20941,6 +20941,7 @@ Backend-only flags (these can be set in the config file also). --frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms) --frostfs-connection-timeout Duration FrostFS connection timeout (default 4s) --frostfs-container-creation-policy string Container creation policy for new containers (default "private") + --frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container") --frostfs-description string Description of the remote --frostfs-endpoint string Endpoints to connect to FrostFS node --frostfs-password string Password to decrypt wallet @@ -34790,7 +34791,7 @@ connection_timeout> Option request_timeout. FrostFS request timeout -Enter a value of type Duration. Press Enter for the default (4s). +Enter a value of type Duration. Press Enter for the default (12s). request_timeout> Option rebalance_interval. @@ -34810,7 +34811,7 @@ ape_cache_invalidation_duration> Option ape_cache_invalidation_timeout. APE cache invalidation timeout -Enter a value of type Duration. Press Enter for the default (16s). +Enter a value of type Duration. Press Enter for the default (24s). ape_cache_invalidation_timeout> Option ape_chain_check_interval. @@ -34846,6 +34847,11 @@ Press Enter for the default (REP 3). \ (REP 3) placement_policy> REP 1 +Option default_container_zone. +The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty. +Enter a value of type string. Press Enter for the default (container). +default_container_zone> + Option container_creation_policy. Container creation policy for new containers Choose a number from below, or type in your own value of type string. @@ -34886,6 +34892,14 @@ the identifier `23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf`: rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy rclone copy ~/test-copy remote:container-name/test-copy +Also, for user-friendly container names, you can explicitly specify the name of the zone in which you want +to create or search for a container: + + rclone copy ~/test-copy remote:container-name.container-zone/test-copy + +If the zone is not explicitly specified, its name will be obtained from the configuration parameter +`default_container_zone`. + ### Standard options @@ -35046,6 +35060,17 @@ Properties: - "REP 3" - Container will have 3 replicas +#### --frostfs-default-container-zone + +The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. + +Properties: + +- Config: default_container_zone +- Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE +- Type: string +- Default: "container" + #### --frostfs-container-creation-policy Container creation policy for new containers @@ -35204,13 +35229,13 @@ therefore works even in Windows Command Prompt: rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM: -### Implicit TLS[util.go](../../backend/frostfs/util.go) +### Implicit TLS Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be enabled in the FTP backend config for the remote, or with [`--ftp-tls`](#ftp-tls). The default FTPS port is `990`, not `21` and can be set with [`--ftp-port`](#ftp-port). -[util.go](../../backend/frostfs/util.go) + ### Restricted filename characters In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) diff --git a/MANUAL.txt b/MANUAL.txt index 79efb97d1..1ea427688 100644 --- a/MANUAL.txt +++ b/MANUAL.txt @@ -1,6 +1,6 @@ rclone(1) User Manual Nick Craig-Wood -Jan 10, 2025 +Jan 23, 2025 Rclone syncs your files to cloud storage @@ -20166,7 +20166,7 @@ Flags for general networking and HTTP stuff. --tpslimit float Limit HTTP transactions per second to this --tpslimit-burst int Max burst of transactions for --tpslimit (default 1) --use-cookies Enable session cookiejar - --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support") + --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8335.a85292ca0.feature/add-container-zones-support") Performance @@ -20579,6 +20579,7 @@ Backend-only flags (these can be set in the config file also). --frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms) --frostfs-connection-timeout Duration FrostFS connection timeout (default 4s) --frostfs-container-creation-policy string Container creation policy for new containers (default "private") + --frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container") --frostfs-description string Description of the remote --frostfs-endpoint string Endpoints to connect to FrostFS node --frostfs-password string Password to decrypt wallet @@ -34252,7 +34253,7 @@ FrostFS user wallet. Option request_timeout. FrostFS request timeout - Enter a value of type Duration. Press Enter for the default (4s). + Enter a value of type Duration. Press Enter for the default (12s). request_timeout> Option rebalance_interval. @@ -34272,7 +34273,7 @@ FrostFS user wallet. Option ape_cache_invalidation_timeout. APE cache invalidation timeout - Enter a value of type Duration. Press Enter for the default (16s). + Enter a value of type Duration. Press Enter for the default (24s). ape_cache_invalidation_timeout> Option ape_chain_check_interval. @@ -34308,6 +34309,11 @@ FrostFS user wallet. \ (REP 3) placement_policy> REP 1 + Option default_container_zone. + The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty. + Enter a value of type string. Press Enter for the default (container). + default_container_zone> + Option container_creation_policy. Container creation policy for new containers Choose a number from below, or type in your own value of type string. @@ -34349,6 +34355,14 @@ a ~/test-copy directory and a container with the identifier rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy rclone copy ~/test-copy remote:container-name/test-copy +Also, for user-friendly container names, you can explicitly specify the +name of the zone in which you want to create or search for a container: + + rclone copy ~/test-copy remote:container-name.container-zone/test-copy + +If the zone is not explicitly specified, its name will be obtained from +the configuration parameter default_container_zone. + Standard options Here are the Standard options specific to frostfs (Distributed, @@ -34514,6 +34528,18 @@ Properties: - "REP 3" - Container will have 3 replicas +--frostfs-default-container-zone + +The name of the zone in which containers will be created or resolved if +the zone name is not explicitly specified with the container name. + +Properties: + +- Config: default_container_zone +- Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE +- Type: string +- Default: "container" + --frostfs-container-creation-policy Container creation policy for new containers @@ -34668,12 +34694,13 @@ even in Windows Command Prompt: rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM: -Implicit TLSutil.go +Implicit TLS Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be enabled in the FTP backend config for the remote, or with --ftp-tls. The -default FTPS port is 990, not 21 and can be set with --ftp-port. util.go -### Restricted filename characters +default FTPS port is 990, not 21 and can be set with --ftp-port. + +Restricted filename characters In addition to the default restricted characters set the following characters are also replaced: diff --git a/docs/content/commands/rclone.md b/docs/content/commands/rclone.md index f9956f61b..d43e14ba8 100644 --- a/docs/content/commands/rclone.md +++ b/docs/content/commands/rclone.md @@ -302,6 +302,7 @@ rclone [flags] --frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms) --frostfs-connection-timeout Duration FrostFS connection timeout (default 4s) --frostfs-container-creation-policy string Container creation policy for new containers (default "private") + --frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container") --frostfs-description string Description of the remote --frostfs-endpoint string Endpoints to connect to FrostFS node --frostfs-password string Password to decrypt wallet @@ -944,7 +945,7 @@ rclone [flags] --use-json-log Use json log format --use-mmap Use mmap allocator (see docs) --use-server-modtime Use server modified time instead of object metadata - --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support") + --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8335.a85292ca0.feature/add-container-zones-support") -v, --verbose count Print lots more stuff (repeat for more) -V, --version Print the version number --webdav-bearer-token string Bearer token instead of user/pass (e.g. a Macaroon) diff --git a/docs/content/flags.md b/docs/content/flags.md index a925fba6b..af8048265 100644 --- a/docs/content/flags.md +++ b/docs/content/flags.md @@ -115,7 +115,7 @@ Flags for general networking and HTTP stuff. --tpslimit float Limit HTTP transactions per second to this --tpslimit-burst int Max burst of transactions for --tpslimit (default 1) --use-cookies Enable session cookiejar - --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support") + --user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8335.a85292ca0.feature/add-container-zones-support") ``` @@ -558,6 +558,7 @@ Backend-only flags (these can be set in the config file also). --frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms) --frostfs-connection-timeout Duration FrostFS connection timeout (default 4s) --frostfs-container-creation-policy string Container creation policy for new containers (default "private") + --frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container") --frostfs-description string Description of the remote --frostfs-endpoint string Endpoints to connect to FrostFS node --frostfs-password string Password to decrypt wallet diff --git a/docs/content/frostfs.md b/docs/content/frostfs.md index d066433cc..13f6b0d27 100644 --- a/docs/content/frostfs.md +++ b/docs/content/frostfs.md @@ -230,6 +230,11 @@ Press Enter for the default (REP 3). \ (REP 3) placement_policy> REP 1 +Option default_container_zone. +The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty. +Enter a value of type string. Press Enter for the default (container). +default_container_zone> + Option container_creation_policy. Container creation policy for new containers Choose a number from below, or type in your own value of type string. @@ -270,6 +275,14 @@ the identifier `23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf`: rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy rclone copy ~/test-copy remote:container-name/test-copy +Also, for user-friendly container names, you can explicitly specify the name of the zone in which you want +to create or search for a container: + + rclone copy ~/test-copy remote:container-name.container-zone/test-copy + +If the zone is not explicitly specified, its name will be obtained from the configuration parameter +`default_container_zone`. + {{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/frostfs/frostfs.go then run make backenddocs" >}} ### Standard options @@ -430,6 +443,17 @@ Properties: - "REP 3" - Container will have 3 replicas +#### --frostfs-default-container-zone + +The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. + +Properties: + +- Config: default_container_zone +- Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE +- Type: string +- Default: "container" + #### --frostfs-container-creation-policy Container creation policy for new containers diff --git a/rclone.1 b/rclone.1 index 38306f409..a4c6ddc25 100644 --- a/rclone.1 +++ b/rclone.1 @@ -15,7 +15,7 @@ . ftr VB CB . ftr VBI CBI .\} -.TH "rclone" "1" "Jan 10, 2025" "User Manual" "" +.TH "rclone" "1" "Jan 23, 2025" "User Manual" "" .hy .SH Rclone syncs your files to cloud storage .PP @@ -27735,7 +27735,7 @@ Flags for general networking and HTTP stuff. --tpslimit float Limit HTTP transactions per second to this --tpslimit-burst int Max burst of transactions for --tpslimit (default 1) --use-cookies Enable session cookiejar - --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support\[dq]) + --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.68.2-beta.8335.a85292ca0.feature/add-container-zones-support\[dq]) \f[R] .fi .SS Performance @@ -28178,6 +28178,7 @@ Backend-only flags (these can be set in the config file also). --frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms) --frostfs-connection-timeout Duration FrostFS connection timeout (default 4s) --frostfs-container-creation-policy string Container creation policy for new containers (default \[dq]private\[dq]) + --frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default \[dq]container\[dq]) --frostfs-description string Description of the remote --frostfs-endpoint string Endpoints to connect to FrostFS node --frostfs-password string Password to decrypt wallet @@ -45689,7 +45690,7 @@ connection_timeout> Option request_timeout. FrostFS request timeout -Enter a value of type Duration. Press Enter for the default (4s). +Enter a value of type Duration. Press Enter for the default (12s). request_timeout> Option rebalance_interval. @@ -45709,7 +45710,7 @@ ape_cache_invalidation_duration> Option ape_cache_invalidation_timeout. APE cache invalidation timeout -Enter a value of type Duration. Press Enter for the default (16s). +Enter a value of type Duration. Press Enter for the default (24s). ape_cache_invalidation_timeout> Option ape_chain_check_interval. @@ -45745,6 +45746,11 @@ Press Enter for the default (REP 3). \[rs] (REP 3) placement_policy> REP 1 +Option default_container_zone. +The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty. +Enter a value of type string. Press Enter for the default (container). +default_container_zone> + Option container_creation_policy. Container creation policy for new containers Choose a number from below, or type in your own value of type string. @@ -45791,6 +45797,18 @@ rclone copy \[ti]/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/ rclone copy \[ti]/test-copy remote:container-name/test-copy \f[R] .fi +.PP +Also, for user-friendly container names, you can explicitly specify the +name of the zone in which you want to create or search for a container: +.IP +.nf +\f[C] +rclone copy \[ti]/test-copy remote:container-name.container-zone/test-copy +\f[R] +.fi +.PP +If the zone is not explicitly specified, its name will be obtained from +the configuration parameter \f[V]default_container_zone\f[R]. .SS Standard options .PP Here are the Standard options specific to frostfs (Distributed, @@ -46006,6 +46024,20 @@ Examples: Container will have 3 replicas .RE .RE +.SS --frostfs-default-container-zone +.PP +The name of the zone in which containers will be created or resolved if +the zone name is not explicitly specified with the container name. +.PP +Properties: +.IP \[bu] 2 +Config: default_container_zone +.IP \[bu] 2 +Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE +.IP \[bu] 2 +Type: string +.IP \[bu] 2 +Default: \[dq]container\[dq] .SS --frostfs-container-creation-policy .PP Container creation policy for new containers @@ -46215,14 +46247,14 @@ rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass= rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM: \f[R] .fi -.SS Implicit TLSutil.go +.SS Implicit TLS .PP Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be enabled in the FTP backend config for the remote, or with \f[V]--ftp-tls\f[R]. The default FTPS port is \f[V]990\f[R], not \f[V]21\f[R] and can be set with \f[V]--ftp-port\f[R]. -util.go ### Restricted filename characters +.SS Restricted filename characters .PP In addition to the default restricted characters set (https://rclone.org/overview/#restricted-characters) the following