Compare commits

..

No commits in common. "tcl/master" and "tcl/master" have entirely different histories.

10 changed files with 216 additions and 1125 deletions

305
MANUAL.html generated
View file

@ -233,7 +233,7 @@
<header id="title-block-header">
<h1 class="title">rclone(1) User Manual</h1>
<p class="author">Nick Craig-Wood</p>
<p class="date">Feb 22, 2025</p>
<p class="date">Jan 10, 2025</p>
</header>
<h1 id="rclone-syncs-your-files-to-cloud-storage">Rclone syncs your
files to cloud storage</h1>
@ -19294,7 +19294,7 @@ split into groups.</p>
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default &quot;rclone/v1.68.2-beta.8339.71f0cdbd4.feat/frostfs_info_cmd&quot;)</code></pre>
--user-agent string Set the user-agent to a specified string (default &quot;rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support&quot;)</code></pre>
<h2 id="performance">Performance</h2>
<p>Flags helpful for increasing performance.</p>
<pre><code> --buffer-size SizeSuffix In memory buffer size when reading files for each --transfer (default 16Mi)
@ -19677,7 +19677,6 @@ split into groups.</p>
--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms)
--frostfs-connection-timeout Duration FrostFS connection timeout (default 4s)
--frostfs-container-creation-policy string Container creation policy for new containers (default &quot;private&quot;)
--frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default &quot;container&quot;)
--frostfs-description string Description of the remote
--frostfs-endpoint string Endpoints to connect to FrostFS node
--frostfs-password string Password to decrypt wallet
@ -32845,7 +32844,7 @@ connection_timeout&gt;
Option request_timeout.
FrostFS request timeout
Enter a value of type Duration. Press Enter for the default (12s).
Enter a value of type Duration. Press Enter for the default (4s).
request_timeout&gt;
Option rebalance_interval.
@ -32865,7 +32864,7 @@ ape_cache_invalidation_duration&gt;
Option ape_cache_invalidation_timeout.
APE cache invalidation timeout
Enter a value of type Duration. Press Enter for the default (24s).
Enter a value of type Duration. Press Enter for the default (16s).
ape_cache_invalidation_timeout&gt;
Option ape_chain_check_interval.
@ -32901,11 +32900,6 @@ Press Enter for the default (REP 3).
\ (REP 3)
placement_policy&gt; REP 1
Option default_container_zone.
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty.
Enter a value of type string. Press Enter for the default (container).
default_container_zone&gt;
Option container_creation_policy.
Container creation policy for new containers
Choose a number from below, or type in your own value of type string.
@ -32944,13 +32938,6 @@ identifier
<code>23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf</code>:</p>
<pre><code>rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy
rclone copy ~/test-copy remote:container-name/test-copy</code></pre>
<p>Also, for user-friendly container names, you can explicitly specify
the name of the zone in which you want to create or search for a
container:</p>
<pre><code>rclone copy ~/test-copy remote:container-name.container-zone/test-copy</code></pre>
<p>If the zone is not explicitly specified, its name will be obtained
from the configuration parameter
<code>default_container_zone</code>.</p>
<h3 id="standard-options-14">Standard options</h3>
<p>Here are the Standard options specific to frostfs (Distributed,
decentralized object storage FrostFS).</p>
@ -33106,18 +33093,6 @@ FrostFS.</p>
</ul></li>
</ul>
<h4
id="frostfs-default-container-zone">--frostfs-default-container-zone</h4>
<p>The name of the zone in which containers will be created or resolved
if the zone name is not explicitly specified with the container
name.</p>
<p>Properties:</p>
<ul>
<li>Config: default_container_zone</li>
<li>Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE</li>
<li>Type: string</li>
<li>Default: "container"</li>
</ul>
<h4
id="frostfs-container-creation-policy">--frostfs-container-creation-policy</h4>
<p>Container creation policy for new containers</p>
<p>Properties:</p>
@ -33155,63 +33130,6 @@ decentralized object storage FrostFS).</p>
<li>Type: string</li>
<li>Required: false</li>
</ul>
<h2 id="backend-commands-4">Backend commands</h2>
<p>Here are the commands specific to the frostfs backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>
<p>The help below will explain what arguments each command takes.</p>
<p>See the <a
href="https://rclone.org/commands/rclone_backend/">backend</a> command
for more info on how to pass options and arguments.</p>
<p>These can be run on a running backend using the rc command <a
href="https://rclone.org/rc/#backend-command">backend/command</a>.</p>
<h3 id="info">info</h3>
<p>Show information about the FrostFS objects and containers</p>
<pre><code>rclone backend info remote: [options] [&lt;arguments&gt;+]</code></pre>
<p>This command can be used to get information about the FrostFS objects
and containers.</p>
<p>Usage Examples:</p>
<pre><code>rclone backend info frostfs:container/path/to/dir
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt -o &quot;format={cid}:{oid}&quot;</code></pre>
<p>The optional "format" flag overrides the information output. In this
example, if an object is stored in a container with the identifier
"9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1" and its own identifier is
"4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP", the output of this
command will be
"9mvN7hsucoGoHjxPqrWmDipnMaemGVDqrxxPyynn1:4VpcNFsZqsQt1Gnfw2utBnzn5Blgc7i4kvtuXyKyJp".</p>
<p>The default output format is the same as that of the frostfs-cli
utility, with the "container get" and "object head" options. Here is an
example of output:</p>
<pre><code> --- Container info ---
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner ID: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
Created: 2025-02-17 15:07:51 +0300 MSK
Attributes:
Timestamp=1739794071
Name=test
__SYSTEM__NAME=test
__SYSTEM__ZONE=container
__SYSTEM__DISABLE_HOMOMORPHIC_HASHING=true
Placement policy:
REP 3
--- Object info ---
ID: 4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
CreatedAt: 559
Size: 402905
HomoHash: &lt;empty&gt;
Checksum: 2a068fe24c53bc8bf7d6bbb997414f7938b080305dc45f9fd3ff684bc11fbb7b
Type: REGULAR
Attributes:
FileName=cat.png
FilePath=/dir1/dir2/dir3/cat.png
Timestamp=1733410524 (2024-12-05 17:55:24 +0300 MSK)
ID signature:
public key: 026b7c7a7a16225eb13a5a733495a1bcdd1f016dfa9193498821379b0de2ba6870
signature: 049f6712c8378d323269b605a282bcacd7565ce2eefe1f10a9739c48945f739d95102c478b9cb1d429cd3330b4b5262e725392e322de3bbfa4ce18a9c842289219</code></pre>
<h1 id="ftp">FTP</h1>
<p>FTP is the File Transfer Protocol. Rclone FTP support is provided
using the <a
@ -33316,14 +33234,15 @@ password "dummy", and therefore works even in Windows Command
Prompt:</p>
<pre><code>rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM
rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:</code></pre>
<h3 id="implicit-tls">Implicit TLS</h3>
<h3 id="implicit-tlsutil.go">Implicit TLS<a
href="../../backend/frostfs/util.go">util.go</a></h3>
<p>Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to
be enabled in the FTP backend config for the remote, or with <a
href="#ftp-tls"><code>--ftp-tls</code></a>. The default FTPS port is
<code>990</code>, not <code>21</code> and can be set with <a
href="#ftp-port"><code>--ftp-port</code></a>.</p>
<h3 id="restricted-filename-characters-7">Restricted filename
characters</h3>
href="#ftp-port"><code>--ftp-port</code></a>. <a
href="../../backend/frostfs/util.go">util.go</a> ### Restricted filename
characters</p>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
restricted characters set</a> the following characters are also
@ -33734,7 +33653,7 @@ hashes</h3>
<p>Gofile supports modification times with a resolution of 1 second.</p>
<p>Gofile supports MD5 hashes, so you can use the
<code>--checksum</code> flag.</p>
<h3 id="restricted-filename-characters-8">Restricted filename
<h3 id="restricted-filename-characters-7">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -34167,7 +34086,7 @@ gsutil only contain timestamps with one-second precision. If you use
rclone to sync files previously uploaded by gsutil, rclone will attempt
to update modification time for all these files. To avoid these possibly
unnecessary updates, use <code>--modify-window 1s</code>.</p>
<h3 id="restricted-filename-characters-9">Restricted filename
<h3 id="restricted-filename-characters-8">Restricted filename
characters</h3>
<table>
<thead>
@ -35033,7 +34952,7 @@ hashes</h3>
<p>Hash algorithms MD5, SHA1 and SHA256 are supported. Note, however,
that a small fraction of files uploaded may not have SHA1 or SHA256
hashes especially if they were uploaded before 2018.</p>
<h3 id="restricted-filename-characters-10">Restricted filename
<h3 id="restricted-filename-characters-9">Restricted filename
characters</h3>
<p>Only Invalid UTF-8 bytes will be <a
href="https://rclone.org/overview/#invalid-utf8">replaced</a>, as they
@ -36279,7 +36198,7 @@ drives.</td>
</table>
<p>See the <a href="https://rclone.org/docs/#metadata">metadata</a> docs
for more info.</p>
<h2 id="backend-commands-5">Backend commands</h2>
<h2 id="backend-commands-4">Backend commands</h2>
<p>Here are the commands specific to the drive backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>
@ -37266,7 +37185,7 @@ default).</p>
written.</p>
<p>See the <a href="https://rclone.org/docs/#metadata">metadata</a> docs
for more info.</p>
<h2 id="backend-commands-6">Backend commands</h2>
<h2 id="backend-commands-5">Backend commands</h2>
<p>Here are the commands specific to the hasher backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>
@ -37458,7 +37377,7 @@ be lost.)</p>
<h3 id="usage-information">Usage information</h3>
<p>You can use the <code>rclone about remote:</code> command which will
display filesystem size and current usage.</p>
<h3 id="restricted-filename-characters-11">Restricted filename
<h3 id="restricted-filename-characters-10">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -37682,7 +37601,7 @@ second.</p>
<p>HiDrive supports <a href="https://static.hidrive.com/dev/0001">its
own hash type</a> which is used to verify the integrity of file contents
after successful transfers.</p>
<h3 id="restricted-filename-characters-12">Restricted filename
<h3 id="restricted-filename-characters-11">Restricted filename
characters</h3>
<p>HiDrive cannot store files or folders that include <code>/</code>
(0x2F) or null-bytes (0x00) in their name. Any other characters can be
@ -38168,7 +38087,7 @@ may be in the listing.</p>
<li>Type: string</li>
<li>Required: false</li>
</ul>
<h2 id="backend-commands-7">Backend commands</h2>
<h2 id="backend-commands-6">Backend commands</h2>
<p>Here are the commands specific to the http backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>
@ -39075,7 +38994,7 @@ available, so this does not apply. Starting with rclone version 1.52 the
same is true for encrypted remotes (in older versions the crypt backend
would not calculate hashes for uploads from local disk, so the
Jottacloud backend had to do it as described above).</p>
<h3 id="restricted-filename-characters-13">Restricted filename
<h3 id="restricted-filename-characters-12">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -39427,7 +39346,7 @@ choose an alternative mount instead of your primary storage.</p>
<pre><code>rclone ls koofr:</code></pre>
<p>To copy a local directory to an Koofr directory called backup</p>
<pre><code>rclone copy /home/source koofr:backup</code></pre>
<h3 id="restricted-filename-characters-14">Restricted filename
<h3 id="restricted-filename-characters-13">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -39889,7 +39808,7 @@ command does not take any path arguments.</p>
<p>To view your current quota you can use the
<code>rclone about remote:</code> command which will display your usage
limit (quota) and the current usage.</p>
<h3 id="restricted-filename-characters-15">Restricted filename
<h3 id="restricted-filename-characters-14">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -40278,7 +40197,7 @@ use the credentials in <code>rclone</code> will fail.</p>
<h3 id="modification-times-and-hashes-12">Modification times and
hashes</h3>
<p>Mega does not support modification times or hashes yet.</p>
<h3 id="restricted-filename-characters-16">Restricted filename
<h3 id="restricted-filename-characters-15">Restricted filename
characters</h3>
<table>
<thead>
@ -40517,7 +40436,7 @@ rclone serve sftp :memory:</code></pre>
hashes</h3>
<p>The memory backend supports MD5 hashes and modification times
accurate to 1 nS.</p>
<h3 id="restricted-filename-characters-17">Restricted filename
<h3 id="restricted-filename-characters-16">Restricted filename
characters</h3>
<p>The memory backend replaces the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -40802,7 +40721,7 @@ provided primarily for debugging purposes.</p>
<li>Type: string</li>
<li>Required: false</li>
</ul>
<h2 id="backend-commands-8">Backend commands</h2>
<h2 id="backend-commands-7">Backend commands</h2>
<p>Here are the commands specific to the netstorage backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>
@ -40902,7 +40821,7 @@ the cost of using more memory. The default of 16 is set quite
conservatively to use less memory. It maybe be necessary raise it to 64
or higher to fully utilize a 1 GBit/s link with a single file
transfer.</p>
<h3 id="restricted-filename-characters-18">Restricted filename
<h3 id="restricted-filename-characters-17">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -41747,7 +41666,7 @@ at the cost of using more memory. The default of 16 is set quite
conservatively to use less memory. It maybe be necessary raise it to 64
or higher to fully utilize a 1 GBit/s link with a single file
transfer.</p>
<h3 id="restricted-filename-characters-19">Restricted filename
<h3 id="restricted-filename-characters-18">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -42558,7 +42477,7 @@ using.</p>
<p>Some commands (like <code>rclone lsf -R</code>) will use
<code>ListR</code> by default - you can turn this off with
<code>--disable ListR</code> if you need to.</p>
<h3 id="restricted-filename-characters-20">Restricted filename
<h3 id="restricted-filename-characters-19">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -43149,69 +43068,69 @@ href="https://learn.microsoft.com/en-us/onedrive/developer/rest-api/resources/pe
API</a>, which differs slightly between OneDrive Personal and
Business.</p>
<p>Example for OneDrive Personal:</p>
<div class="sourceCode" id="cb1268"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb1268-1"><a href="#cb1268-1" aria-hidden="true" tabindex="-1"></a><span class="ot">[</span></span>
<span id="cb1268-2"><a href="#cb1268-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1268-3"><a href="#cb1268-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;1234567890ABC!123&quot;</span><span class="fu">,</span></span>
<span id="cb1268-4"><a href="#cb1268-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;grantedTo&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1268-5"><a href="#cb1268-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;user&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1268-6"><a href="#cb1268-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;ryan@contoso.com&quot;</span></span>
<span id="cb1268-7"><a href="#cb1268-7" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1268-8"><a href="#cb1268-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;application&quot;</span><span class="fu">:</span> <span class="fu">{},</span></span>
<span id="cb1268-9"><a href="#cb1268-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;device&quot;</span><span class="fu">:</span> <span class="fu">{}</span></span>
<span id="cb1268-10"><a href="#cb1268-10" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1268-11"><a href="#cb1268-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;invitation&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1268-12"><a href="#cb1268-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;email&quot;</span><span class="fu">:</span> <span class="st">&quot;ryan@contoso.com&quot;</span></span>
<span id="cb1268-13"><a href="#cb1268-13" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1268-14"><a href="#cb1268-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;link&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1268-15"><a href="#cb1268-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;webUrl&quot;</span><span class="fu">:</span> <span class="st">&quot;https://1drv.ms/t/s!1234567890ABC&quot;</span></span>
<span id="cb1268-16"><a href="#cb1268-16" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1268-17"><a href="#cb1268-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;roles&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1268-18"><a href="#cb1268-18" aria-hidden="true" tabindex="-1"></a> <span class="st">&quot;read&quot;</span></span>
<span id="cb1268-19"><a href="#cb1268-19" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1268-20"><a href="#cb1268-20" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;shareId&quot;</span><span class="fu">:</span> <span class="st">&quot;s!1234567890ABC&quot;</span></span>
<span id="cb1268-21"><a href="#cb1268-21" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1268-22"><a href="#cb1268-22" aria-hidden="true" tabindex="-1"></a><span class="ot">]</span></span></code></pre></div>
<div class="sourceCode" id="cb1263"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb1263-1"><a href="#cb1263-1" aria-hidden="true" tabindex="-1"></a><span class="ot">[</span></span>
<span id="cb1263-2"><a href="#cb1263-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1263-3"><a href="#cb1263-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;1234567890ABC!123&quot;</span><span class="fu">,</span></span>
<span id="cb1263-4"><a href="#cb1263-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;grantedTo&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1263-5"><a href="#cb1263-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;user&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1263-6"><a href="#cb1263-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;ryan@contoso.com&quot;</span></span>
<span id="cb1263-7"><a href="#cb1263-7" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1263-8"><a href="#cb1263-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;application&quot;</span><span class="fu">:</span> <span class="fu">{},</span></span>
<span id="cb1263-9"><a href="#cb1263-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;device&quot;</span><span class="fu">:</span> <span class="fu">{}</span></span>
<span id="cb1263-10"><a href="#cb1263-10" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1263-11"><a href="#cb1263-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;invitation&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1263-12"><a href="#cb1263-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;email&quot;</span><span class="fu">:</span> <span class="st">&quot;ryan@contoso.com&quot;</span></span>
<span id="cb1263-13"><a href="#cb1263-13" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1263-14"><a href="#cb1263-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;link&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1263-15"><a href="#cb1263-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;webUrl&quot;</span><span class="fu">:</span> <span class="st">&quot;https://1drv.ms/t/s!1234567890ABC&quot;</span></span>
<span id="cb1263-16"><a href="#cb1263-16" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1263-17"><a href="#cb1263-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;roles&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1263-18"><a href="#cb1263-18" aria-hidden="true" tabindex="-1"></a> <span class="st">&quot;read&quot;</span></span>
<span id="cb1263-19"><a href="#cb1263-19" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1263-20"><a href="#cb1263-20" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;shareId&quot;</span><span class="fu">:</span> <span class="st">&quot;s!1234567890ABC&quot;</span></span>
<span id="cb1263-21"><a href="#cb1263-21" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1263-22"><a href="#cb1263-22" aria-hidden="true" tabindex="-1"></a><span class="ot">]</span></span></code></pre></div>
<p>Example for OneDrive Business:</p>
<div class="sourceCode" id="cb1269"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb1269-1"><a href="#cb1269-1" aria-hidden="true" tabindex="-1"></a><span class="ot">[</span></span>
<span id="cb1269-2"><a href="#cb1269-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1269-3"><a href="#cb1269-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;48d31887-5fad-4d73-a9f5-3c356e68a038&quot;</span><span class="fu">,</span></span>
<span id="cb1269-4"><a href="#cb1269-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;grantedToIdentities&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1269-5"><a href="#cb1269-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1269-6"><a href="#cb1269-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;user&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1269-7"><a href="#cb1269-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;displayName&quot;</span><span class="fu">:</span> <span class="st">&quot;ryan@contoso.com&quot;</span></span>
<span id="cb1269-8"><a href="#cb1269-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1269-9"><a href="#cb1269-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;application&quot;</span><span class="fu">:</span> <span class="fu">{},</span></span>
<span id="cb1269-10"><a href="#cb1269-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;device&quot;</span><span class="fu">:</span> <span class="fu">{}</span></span>
<span id="cb1269-11"><a href="#cb1269-11" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1269-12"><a href="#cb1269-12" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1269-13"><a href="#cb1269-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;link&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1269-14"><a href="#cb1269-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;view&quot;</span><span class="fu">,</span></span>
<span id="cb1269-15"><a href="#cb1269-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;scope&quot;</span><span class="fu">:</span> <span class="st">&quot;users&quot;</span><span class="fu">,</span></span>
<span id="cb1269-16"><a href="#cb1269-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;webUrl&quot;</span><span class="fu">:</span> <span class="st">&quot;https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s&quot;</span></span>
<span id="cb1269-17"><a href="#cb1269-17" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1269-18"><a href="#cb1269-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;roles&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1269-19"><a href="#cb1269-19" aria-hidden="true" tabindex="-1"></a> <span class="st">&quot;read&quot;</span></span>
<span id="cb1269-20"><a href="#cb1269-20" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1269-21"><a href="#cb1269-21" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;shareId&quot;</span><span class="fu">:</span> <span class="st">&quot;u!LKj1lkdlals90j1nlkascl&quot;</span></span>
<span id="cb1269-22"><a href="#cb1269-22" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span><span class="ot">,</span></span>
<span id="cb1269-23"><a href="#cb1269-23" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1269-24"><a href="#cb1269-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;5D33DD65C6932946&quot;</span><span class="fu">,</span></span>
<span id="cb1269-25"><a href="#cb1269-25" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;grantedTo&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1269-26"><a href="#cb1269-26" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;user&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1269-27"><a href="#cb1269-27" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;displayName&quot;</span><span class="fu">:</span> <span class="st">&quot;John Doe&quot;</span><span class="fu">,</span></span>
<span id="cb1269-28"><a href="#cb1269-28" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;efee1b77-fb3b-4f65-99d6-274c11914d12&quot;</span></span>
<span id="cb1269-29"><a href="#cb1269-29" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1269-30"><a href="#cb1269-30" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;application&quot;</span><span class="fu">:</span> <span class="fu">{},</span></span>
<span id="cb1269-31"><a href="#cb1269-31" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;device&quot;</span><span class="fu">:</span> <span class="fu">{}</span></span>
<span id="cb1269-32"><a href="#cb1269-32" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1269-33"><a href="#cb1269-33" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;roles&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1269-34"><a href="#cb1269-34" aria-hidden="true" tabindex="-1"></a> <span class="st">&quot;owner&quot;</span></span>
<span id="cb1269-35"><a href="#cb1269-35" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1269-36"><a href="#cb1269-36" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;shareId&quot;</span><span class="fu">:</span> <span class="st">&quot;FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U&quot;</span></span>
<span id="cb1269-37"><a href="#cb1269-37" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1269-38"><a href="#cb1269-38" aria-hidden="true" tabindex="-1"></a><span class="ot">]</span></span></code></pre></div>
<div class="sourceCode" id="cb1264"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb1264-1"><a href="#cb1264-1" aria-hidden="true" tabindex="-1"></a><span class="ot">[</span></span>
<span id="cb1264-2"><a href="#cb1264-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1264-3"><a href="#cb1264-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;48d31887-5fad-4d73-a9f5-3c356e68a038&quot;</span><span class="fu">,</span></span>
<span id="cb1264-4"><a href="#cb1264-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;grantedToIdentities&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1264-5"><a href="#cb1264-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1264-6"><a href="#cb1264-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;user&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1264-7"><a href="#cb1264-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;displayName&quot;</span><span class="fu">:</span> <span class="st">&quot;ryan@contoso.com&quot;</span></span>
<span id="cb1264-8"><a href="#cb1264-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1264-9"><a href="#cb1264-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;application&quot;</span><span class="fu">:</span> <span class="fu">{},</span></span>
<span id="cb1264-10"><a href="#cb1264-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;device&quot;</span><span class="fu">:</span> <span class="fu">{}</span></span>
<span id="cb1264-11"><a href="#cb1264-11" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1264-12"><a href="#cb1264-12" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1264-13"><a href="#cb1264-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;link&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1264-14"><a href="#cb1264-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;view&quot;</span><span class="fu">,</span></span>
<span id="cb1264-15"><a href="#cb1264-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;scope&quot;</span><span class="fu">:</span> <span class="st">&quot;users&quot;</span><span class="fu">,</span></span>
<span id="cb1264-16"><a href="#cb1264-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;webUrl&quot;</span><span class="fu">:</span> <span class="st">&quot;https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s&quot;</span></span>
<span id="cb1264-17"><a href="#cb1264-17" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1264-18"><a href="#cb1264-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;roles&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1264-19"><a href="#cb1264-19" aria-hidden="true" tabindex="-1"></a> <span class="st">&quot;read&quot;</span></span>
<span id="cb1264-20"><a href="#cb1264-20" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1264-21"><a href="#cb1264-21" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;shareId&quot;</span><span class="fu">:</span> <span class="st">&quot;u!LKj1lkdlals90j1nlkascl&quot;</span></span>
<span id="cb1264-22"><a href="#cb1264-22" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span><span class="ot">,</span></span>
<span id="cb1264-23"><a href="#cb1264-23" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb1264-24"><a href="#cb1264-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;5D33DD65C6932946&quot;</span><span class="fu">,</span></span>
<span id="cb1264-25"><a href="#cb1264-25" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;grantedTo&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1264-26"><a href="#cb1264-26" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;user&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1264-27"><a href="#cb1264-27" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;displayName&quot;</span><span class="fu">:</span> <span class="st">&quot;John Doe&quot;</span><span class="fu">,</span></span>
<span id="cb1264-28"><a href="#cb1264-28" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;id&quot;</span><span class="fu">:</span> <span class="st">&quot;efee1b77-fb3b-4f65-99d6-274c11914d12&quot;</span></span>
<span id="cb1264-29"><a href="#cb1264-29" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1264-30"><a href="#cb1264-30" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;application&quot;</span><span class="fu">:</span> <span class="fu">{},</span></span>
<span id="cb1264-31"><a href="#cb1264-31" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;device&quot;</span><span class="fu">:</span> <span class="fu">{}</span></span>
<span id="cb1264-32"><a href="#cb1264-32" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb1264-33"><a href="#cb1264-33" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;roles&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb1264-34"><a href="#cb1264-34" aria-hidden="true" tabindex="-1"></a> <span class="st">&quot;owner&quot;</span></span>
<span id="cb1264-35"><a href="#cb1264-35" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb1264-36"><a href="#cb1264-36" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;shareId&quot;</span><span class="fu">:</span> <span class="st">&quot;FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U&quot;</span></span>
<span id="cb1264-37"><a href="#cb1264-37" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1264-38"><a href="#cb1264-38" aria-hidden="true" tabindex="-1"></a><span class="ot">]</span></span></code></pre></div>
<p>To write permissions, pass in a "permissions" metadata key using this
same format. The <a
href="https://rclone.org/docs/#metadata-mapper"><code>--metadata-mapper</code></a>
@ -43225,12 +43144,12 @@ for a user. Creating a Public Link is also supported, if
<code>Link.Scope</code> is set to <code>"anonymous"</code>.</p>
<p>Example request to add a "read" permission with
<code>--metadata-mapper</code>:</p>
<div class="sourceCode" id="cb1270"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb1270-1"><a href="#cb1270-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb1270-2"><a href="#cb1270-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;Metadata&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1270-3"><a href="#cb1270-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;permissions&quot;</span><span class="fu">:</span> <span class="st">&quot;[{</span><span class="ch">\&quot;</span><span class="st">grantedToIdentities</span><span class="ch">\&quot;</span><span class="st">:[{</span><span class="ch">\&quot;</span><span class="st">user</span><span class="ch">\&quot;</span><span class="st">:{</span><span class="ch">\&quot;</span><span class="st">id</span><span class="ch">\&quot;</span><span class="st">:</span><span class="ch">\&quot;</span><span class="st">ryan@contoso.com</span><span class="ch">\&quot;</span><span class="st">}}],</span><span class="ch">\&quot;</span><span class="st">roles</span><span class="ch">\&quot;</span><span class="st">:[</span><span class="ch">\&quot;</span><span class="st">read</span><span class="ch">\&quot;</span><span class="st">]}]&quot;</span></span>
<span id="cb1270-4"><a href="#cb1270-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1270-5"><a href="#cb1270-5" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<div class="sourceCode" id="cb1265"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb1265-1"><a href="#cb1265-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb1265-2"><a href="#cb1265-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;Metadata&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb1265-3"><a href="#cb1265-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">&quot;permissions&quot;</span><span class="fu">:</span> <span class="st">&quot;[{</span><span class="ch">\&quot;</span><span class="st">grantedToIdentities</span><span class="ch">\&quot;</span><span class="st">:[{</span><span class="ch">\&quot;</span><span class="st">user</span><span class="ch">\&quot;</span><span class="st">:{</span><span class="ch">\&quot;</span><span class="st">id</span><span class="ch">\&quot;</span><span class="st">:</span><span class="ch">\&quot;</span><span class="st">ryan@contoso.com</span><span class="ch">\&quot;</span><span class="st">}}],</span><span class="ch">\&quot;</span><span class="st">roles</span><span class="ch">\&quot;</span><span class="st">:[</span><span class="ch">\&quot;</span><span class="st">read</span><span class="ch">\&quot;</span><span class="st">]}]&quot;</span></span>
<span id="cb1265-4"><a href="#cb1265-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb1265-5"><a href="#cb1265-5" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<p>Note that adding a permission can fail if a conflicting permission
already exists for the file/folder.</p>
<p>To update an existing permission, include both the Permission ID and
@ -43696,7 +43615,7 @@ hashes</h3>
1 second. These will be used to detect whether objects need syncing or
not.</p>
<p>The MD5 hash algorithm is supported.</p>
<h3 id="restricted-filename-characters-21">Restricted filename
<h3 id="restricted-filename-characters-20">Restricted filename
characters</h3>
<table>
<thead>
@ -44559,7 +44478,7 @@ Encryption
<li>Type: string</li>
<li>Required: false</li>
</ul>
<h2 id="backend-commands-9">Backend commands</h2>
<h2 id="backend-commands-8">Backend commands</h2>
<p>Here are the commands specific to the oracleobjectstorage
backend.</p>
<p>Run them with</p>
@ -44768,7 +44687,7 @@ file</li>
</ul></li>
</ul></li>
</ul>
<h3 id="restricted-filename-characters-22">Restricted filename
<h3 id="restricted-filename-characters-21">Restricted filename
characters</h3>
<p>The control characters 0x00-0x1F and / are replaced as in the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -45053,7 +44972,7 @@ microsecond. These will be used to detect whether objects need syncing
or not.</p>
<p>Quatrix does not support hashes, so you cannot use the
<code>--checksum</code> flag.</p>
<h3 id="restricted-filename-characters-23">Restricted filename
<h3 id="restricted-filename-characters-22">Restricted filename
characters</h3>
<p>File names in Quatrix are case sensitive and have limitations like
the maximum length of a filename is 255, and the minimum length is 1. A
@ -45582,7 +45501,7 @@ accurate to 1 ns.</p>
<p>This is a de facto standard (used in the official python-swiftclient
amongst others) for storing the modification time for an object.</p>
<p>The MD5 hash algorithm is supported.</p>
<h3 id="restricted-filename-characters-24">Restricted filename
<h3 id="restricted-filename-characters-23">Restricted filename
characters</h3>
<table>
<thead>
@ -46132,7 +46051,7 @@ re-uploaded.</p>
<p>pCloud supports MD5 and SHA1 hashes in the US region, and SHA1 and
SHA256 hashes in the EU region, so you can use the
<code>--checksum</code> flag.</p>
<h3 id="restricted-filename-characters-25">Restricted filename
<h3 id="restricted-filename-characters-24">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -46527,7 +46446,7 @@ Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,Righ
<li>Type: string</li>
<li>Required: false</li>
</ul>
<h2 id="backend-commands-10">Backend commands</h2>
<h2 id="backend-commands-9">Backend commands</h2>
<p>Here are the commands specific to the pikpak backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>
@ -46829,7 +46748,7 @@ hashes</h3>
<p>premiumize.me does not support modification times or hashes,
therefore syncing will default to <code>--size-only</code> checking.
Note that using <code>--update</code> will work.</p>
<h3 id="restricted-filename-characters-26">Restricted filename
<h3 id="restricted-filename-characters-25">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -47030,7 +46949,7 @@ hashes</h3>
<p>Proton Drive Bridge does not support updating modification times
yet.</p>
<p>The SHA1 hash algorithm is supported.</p>
<h3 id="restricted-filename-characters-27">Restricted filename
<h3 id="restricted-filename-characters-26">Restricted filename
characters</h3>
<p>Invalid UTF-8 bytes will be <a
href="https://rclone.org/overview/#invalid-utf8">replaced</a>, also left
@ -47343,7 +47262,7 @@ mode.</p>
<pre><code>rclone ls remote:</code></pre>
<p>To copy a local directory to a put.io directory called backup</p>
<pre><code>rclone copy /home/source remote:backup</code></pre>
<h3 id="restricted-filename-characters-28">Restricted filename
<h3 id="restricted-filename-characters-27">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -47526,7 +47445,7 @@ hashes</h3>
<p>Proton Drive Bridge does not support updating modification times
yet.</p>
<p>The SHA1 hash algorithm is supported.</p>
<h3 id="restricted-filename-characters-29">Restricted filename
<h3 id="restricted-filename-characters-28">Restricted filename
characters</h3>
<p>Invalid UTF-8 bytes will be <a
href="https://rclone.org/overview/#invalid-utf8">replaced</a>, also left
@ -47950,7 +47869,7 @@ to use fewer transactions in exchange for more memory. See the <a
href="https://rclone.org/docs/#fast-list">rclone docs</a> for more
details. Please note this is not supported on seafile server version
6.x</p>
<h3 id="restricted-filename-characters-30">Restricted filename
<h3 id="restricted-filename-characters-29">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -49760,7 +49679,7 @@ hashes</h3>
syncing will default to <code>--size-only</code> checking. Note that
using <code>--update</code> will work as rclone can read the time files
were uploaded.</p>
<h3 id="restricted-filename-characters-31">Restricted filename
<h3 id="restricted-filename-characters-30">Restricted filename
characters</h3>
<p>SugarSync replaces the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -49988,7 +49907,7 @@ and hashes. Timestamps are stored with microsecond precision.</p>
Afterwards, the backend only serves the client-side calculated hashes.
Hashes can also be retrieved upon creating a file download link, but
it's impractical for <code>list</code>-like use cases.</p>
<h3 id="restricted-filename-characters-32">Restricted filename
<h3 id="restricted-filename-characters-31">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -50203,7 +50122,7 @@ y/e/d&gt; </code></pre>
hashes</h3>
<p>Uptobox supports neither modified times nor checksums. All timestamps
will read as that set by <code>--default-time</code>.</p>
<h3 id="restricted-filename-characters-33">Restricted filename
<h3 id="restricted-filename-characters-32">Restricted filename
characters</h3>
<p>In addition to the <a
href="https://rclone.org/overview/#restricted-characters">default
@ -51210,7 +51129,7 @@ arguments.</p>
<p>To view your current quota you can use the
<code>rclone about remote:</code> command which will display your usage
limit (quota) and the current usage.</p>
<h3 id="restricted-filename-characters-34">Restricted filename
<h3 id="restricted-filename-characters-33">Restricted filename
characters</h3>
<p>The <a
href="https://rclone.org/overview/#restricted-characters">default
@ -51420,7 +51339,7 @@ hashes</h3>
<p>To view your current quota you can use the
<code>rclone about remote:</code> command which will display your
current usage.</p>
<h3 id="restricted-filename-characters-35">Restricted filename
<h3 id="restricted-filename-characters-34">Restricted filename
characters</h3>
<p>Only control characters and invalid UTF-8 are replaced. In addition
most Unicode full-width characters are not supported at all and will be
@ -52372,7 +52291,7 @@ backend.</p>
</table>
<p>See the <a href="https://rclone.org/docs/#metadata">metadata</a> docs
for more info.</p>
<h2 id="backend-commands-11">Backend commands</h2>
<h2 id="backend-commands-10">Backend commands</h2>
<p>Here are the commands specific to the local backend.</p>
<p>Run them with</p>
<pre><code>rclone backend COMMAND remote:</code></pre>

106
MANUAL.md generated
View file

@ -1,6 +1,6 @@
% rclone(1) User Manual
% Nick Craig-Wood
% Feb 22, 2025
% Jan 10, 2025
# Rclone syncs your files to cloud storage
@ -20498,7 +20498,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8339.71f0cdbd4.feat/frostfs_info_cmd")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support")
```
@ -20941,7 +20941,6 @@ Backend-only flags (these can be set in the config file also).
--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms)
--frostfs-connection-timeout Duration FrostFS connection timeout (default 4s)
--frostfs-container-creation-policy string Container creation policy for new containers (default "private")
--frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container")
--frostfs-description string Description of the remote
--frostfs-endpoint string Endpoints to connect to FrostFS node
--frostfs-password string Password to decrypt wallet
@ -34791,7 +34790,7 @@ connection_timeout>
Option request_timeout.
FrostFS request timeout
Enter a value of type Duration. Press Enter for the default (12s).
Enter a value of type Duration. Press Enter for the default (4s).
request_timeout>
Option rebalance_interval.
@ -34811,7 +34810,7 @@ ape_cache_invalidation_duration>
Option ape_cache_invalidation_timeout.
APE cache invalidation timeout
Enter a value of type Duration. Press Enter for the default (24s).
Enter a value of type Duration. Press Enter for the default (16s).
ape_cache_invalidation_timeout>
Option ape_chain_check_interval.
@ -34847,11 +34846,6 @@ Press Enter for the default (REP 3).
\ (REP 3)
placement_policy> REP 1
Option default_container_zone.
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty.
Enter a value of type string. Press Enter for the default (container).
default_container_zone>
Option container_creation_policy.
Container creation policy for new containers
Choose a number from below, or type in your own value of type string.
@ -34892,14 +34886,6 @@ the identifier `23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf`:
rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy
rclone copy ~/test-copy remote:container-name/test-copy
Also, for user-friendly container names, you can explicitly specify the name of the zone in which you want
to create or search for a container:
rclone copy ~/test-copy remote:container-name.container-zone/test-copy
If the zone is not explicitly specified, its name will be obtained from the configuration parameter
`default_container_zone`.
### Standard options
@ -35060,17 +35046,6 @@ Properties:
- "REP 3"
- Container will have 3 replicas
#### --frostfs-default-container-zone
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name.
Properties:
- Config: default_container_zone
- Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE
- Type: string
- Default: "container"
#### --frostfs-container-creation-policy
Container creation policy for new containers
@ -35104,75 +35079,6 @@ Properties:
- Type: string
- Required: false
## Backend commands
Here are the commands specific to the frostfs backend.
Run them with
rclone backend COMMAND remote:
The help below will explain what arguments each command takes.
See the [backend](https://rclone.org/commands/rclone_backend/) command for more
info on how to pass options and arguments.
These can be run on a running backend using the rc command
[backend/command](https://rclone.org/rc/#backend-command).
### info
Show information about the FrostFS objects and containers
rclone backend info remote: [options] [<arguments>+]
This command can be used to get information about the FrostFS objects and containers.
Usage Examples:
rclone backend info frostfs:container/path/to/dir
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt -o "format={cid}:{oid}"
The optional "format" flag overrides the information output. In this example, if an object is stored in
a container with the identifier "9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1" and
its own identifier is "4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP", the output of this command will be
"9mvN7hsucoGoHjxPqrWmDipnMaemGVDqrxxPyynn1:4VpcNFsZqsQt1Gnfw2utBnzn5Blgc7i4kvtuXyKyJp".
The default output format is the same as that of the frostfs-cli utility,
with the "container get" and "object head" options. Here is an example of output:
--- Container info ---
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner ID: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
Created: 2025-02-17 15:07:51 +0300 MSK
Attributes:
Timestamp=1739794071
Name=test
__SYSTEM__NAME=test
__SYSTEM__ZONE=container
__SYSTEM__DISABLE_HOMOMORPHIC_HASHING=true
Placement policy:
REP 3
--- Object info ---
ID: 4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
CreatedAt: 559
Size: 402905
HomoHash: <empty>
Checksum: 2a068fe24c53bc8bf7d6bbb997414f7938b080305dc45f9fd3ff684bc11fbb7b
Type: REGULAR
Attributes:
FileName=cat.png
FilePath=/dir1/dir2/dir3/cat.png
Timestamp=1733410524 (2024-12-05 17:55:24 +0300 MSK)
ID signature:
public key: 026b7c7a7a16225eb13a5a733495a1bcdd1f016dfa9193498821379b0de2ba6870
signature: 049f6712c8378d323269b605a282bcacd7565ce2eefe1f10a9739c48945f739d95102c478b9cb1d429cd3330b4b5262e725392e322de3bbfa4ce18a9c842289219
# FTP
@ -35298,13 +35204,13 @@ therefore works even in Windows Command Prompt:
rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM
rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:
### Implicit TLS
### Implicit TLS[util.go](../../backend/frostfs/util.go)
Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to
be enabled in the FTP backend config for the remote, or with
[`--ftp-tls`](#ftp-tls). The default FTPS port is `990`, not `21` and
can be set with [`--ftp-port`](#ftp-port).
[util.go](../../backend/frostfs/util.go)
### Restricted filename characters
In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters)

113
MANUAL.txt generated
View file

@ -1,6 +1,6 @@
rclone(1) User Manual
Nick Craig-Wood
Feb 22, 2025
Jan 10, 2025
Rclone syncs your files to cloud storage
@ -20166,7 +20166,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8339.71f0cdbd4.feat/frostfs_info_cmd")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support")
Performance
@ -20579,7 +20579,6 @@ Backend-only flags (these can be set in the config file also).
--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms)
--frostfs-connection-timeout Duration FrostFS connection timeout (default 4s)
--frostfs-container-creation-policy string Container creation policy for new containers (default "private")
--frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container")
--frostfs-description string Description of the remote
--frostfs-endpoint string Endpoints to connect to FrostFS node
--frostfs-password string Password to decrypt wallet
@ -34253,7 +34252,7 @@ FrostFS user wallet.
Option request_timeout.
FrostFS request timeout
Enter a value of type Duration. Press Enter for the default (12s).
Enter a value of type Duration. Press Enter for the default (4s).
request_timeout>
Option rebalance_interval.
@ -34273,7 +34272,7 @@ FrostFS user wallet.
Option ape_cache_invalidation_timeout.
APE cache invalidation timeout
Enter a value of type Duration. Press Enter for the default (24s).
Enter a value of type Duration. Press Enter for the default (16s).
ape_cache_invalidation_timeout>
Option ape_chain_check_interval.
@ -34309,11 +34308,6 @@ FrostFS user wallet.
\ (REP 3)
placement_policy> REP 1
Option default_container_zone.
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty.
Enter a value of type string. Press Enter for the default (container).
default_container_zone>
Option container_creation_policy.
Container creation policy for new containers
Choose a number from below, or type in your own value of type string.
@ -34355,14 +34349,6 @@ a ~/test-copy directory and a container with the identifier
rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy
rclone copy ~/test-copy remote:container-name/test-copy
Also, for user-friendly container names, you can explicitly specify the
name of the zone in which you want to create or search for a container:
rclone copy ~/test-copy remote:container-name.container-zone/test-copy
If the zone is not explicitly specified, its name will be obtained from
the configuration parameter default_container_zone.
Standard options
Here are the Standard options specific to frostfs (Distributed,
@ -34528,18 +34514,6 @@ Properties:
- "REP 3"
- Container will have 3 replicas
--frostfs-default-container-zone
The name of the zone in which containers will be created or resolved if
the zone name is not explicitly specified with the container name.
Properties:
- Config: default_container_zone
- Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE
- Type: string
- Default: "container"
--frostfs-container-creation-policy
Container creation policy for new containers
@ -34575,78 +34549,6 @@ Properties:
- Type: string
- Required: false
Backend commands
Here are the commands specific to the frostfs backend.
Run them with
rclone backend COMMAND remote:
The help below will explain what arguments each command takes.
See the backend command for more info on how to pass options and
arguments.
These can be run on a running backend using the rc command
backend/command.
info
Show information about the FrostFS objects and containers
rclone backend info remote: [options] [<arguments>+]
This command can be used to get information about the FrostFS objects
and containers.
Usage Examples:
rclone backend info frostfs:container/path/to/dir
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt -o "format={cid}:{oid}"
The optional "format" flag overrides the information output. In this
example, if an object is stored in a container with the identifier
"9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1" and its own identifier is
"4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP", the output of this
command will be
"9mvN7hsucoGoHjxPqrWmDipnMaemGVDqrxxPyynn1:4VpcNFsZqsQt1Gnfw2utBnzn5Blgc7i4kvtuXyKyJp".
The default output format is the same as that of the frostfs-cli
utility, with the "container get" and "object head" options. Here is an
example of output:
--- Container info ---
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner ID: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
Created: 2025-02-17 15:07:51 +0300 MSK
Attributes:
Timestamp=1739794071
Name=test
__SYSTEM__NAME=test
__SYSTEM__ZONE=container
__SYSTEM__DISABLE_HOMOMORPHIC_HASHING=true
Placement policy:
REP 3
--- Object info ---
ID: 4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
CreatedAt: 559
Size: 402905
HomoHash: <empty>
Checksum: 2a068fe24c53bc8bf7d6bbb997414f7938b080305dc45f9fd3ff684bc11fbb7b
Type: REGULAR
Attributes:
FileName=cat.png
FilePath=/dir1/dir2/dir3/cat.png
Timestamp=1733410524 (2024-12-05 17:55:24 +0300 MSK)
ID signature:
public key: 026b7c7a7a16225eb13a5a733495a1bcdd1f016dfa9193498821379b0de2ba6870
signature: 049f6712c8378d323269b605a282bcacd7565ce2eefe1f10a9739c48945f739d95102c478b9cb1d429cd3330b4b5262e725392e322de3bbfa4ce18a9c842289219
FTP
FTP is the File Transfer Protocol. Rclone FTP support is provided using
@ -34766,13 +34668,12 @@ even in Windows Command Prompt:
rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM
rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:
Implicit TLS
Implicit TLSutil.go
Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be
enabled in the FTP backend config for the remote, or with --ftp-tls. The
default FTPS port is 990, not 21 and can be set with --ftp-port.
Restricted filename characters
default FTPS port is 990, not 21 and can be set with --ftp-port. util.go
### Restricted filename characters
In addition to the default restricted characters set the following
characters are also replaced:

View file

@ -5,7 +5,6 @@ import (
"bytes"
"context"
"encoding/hex"
"errors"
"fmt"
"io"
"math"
@ -17,8 +16,6 @@ import (
"time"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/api/refs"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
sdkClient "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
@ -43,7 +40,6 @@ func init() {
Name: "frostfs",
Description: "Distributed, decentralized object storage FrostFS",
NewFs: NewFs,
CommandHelp: commandHelp,
Options: []fs.Option{
{
Name: "endpoint",
@ -131,11 +127,6 @@ func init() {
},
},
},
{
Name: "default_container_zone",
Default: "container",
Help: "The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name.",
},
{
Name: "container_creation_policy",
Default: "private",
@ -159,61 +150,6 @@ func init() {
})
}
var commandHelp = []fs.CommandHelp{
{
Name: "info",
Short: "Show information about the FrostFS objects and containers",
Long: `This command can be used to get information about the FrostFS objects and containers.
Usage Examples:
rclone backend info frostfs:container/path/to/dir
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt -o "format={cid}:{oid}"
The optional "format" flag overrides the information output. In this example, if an object is stored in
a container with the identifier "9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1" and
its own identifier is "4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP", the output of this command will be
"9mvN7hsucoGoHjxPqrWmDipnMaemGVDqrxxPyynn1:4VpcNFsZqsQt1Gnfw2utBnzn5Blgc7i4kvtuXyKyJp".
The default output format is the same as that of the frostfs-cli utility,
with the "container get" and "object head" options. Here is an example of output:
--- Container info ---
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner ID: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
Created: 2025-02-17 15:07:51 +0300 MSK
Attributes:
Timestamp=1739794071
Name=test
__SYSTEM__NAME=test
__SYSTEM__ZONE=container
__SYSTEM__DISABLE_HOMOMORPHIC_HASHING=true
Placement policy:
REP 3
--- Object info ---
ID: 4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
CreatedAt: 559
Size: 402905
HomoHash: <empty>
Checksum: 2a068fe24c53bc8bf7d6bbb997414f7938b080305dc45f9fd3ff684bc11fbb7b
Type: REGULAR
Attributes:
FileName=cat.png
FilePath=/dir1/dir2/dir3/cat.png
Timestamp=1733410524 (2024-12-05 17:55:24 +0300 MSK)
ID signature:
public key: 026b7c7a7a16225eb13a5a733495a1bcdd1f016dfa9193498821379b0de2ba6870
signature: 049f6712c8378d323269b605a282bcacd7565ce2eefe1f10a9739c48945f739d95102c478b9cb1d429cd3330b4b5262e725392e322de3bbfa4ce18a9c842289219
`,
},
}
var errMalformedObject = errors.New("malformed object")
// Options defines the configuration for this backend
type Options struct {
FrostfsEndpoint string `config:"endpoint"`
@ -230,7 +166,6 @@ type Options struct {
Address string `config:"address"`
Password string `config:"password"`
PlacementPolicy string `config:"placement_policy"`
DefaultContainerZone string `config:"default_container_zone"`
ContainerCreationPolicy string `config:"container_creation_policy"`
APERules []chain.Rule `config:"-"`
}
@ -269,272 +204,6 @@ type Object struct {
timestamp time.Time
}
// Command the backend to run a named command
//
// The command run is name
// args may be used to read arguments from
// opts may be used to read optional arguments from
func (f *Fs) Command(ctx context.Context, name string, arg []string, opt map[string]string) (out interface{}, err error) {
switch name {
case "info":
return f.infoCmd(ctx, arg, opt)
default:
return nil, fs.ErrorCommandNotFound
}
}
func (f *Fs) containerInfo(ctx context.Context, cnrID cid.ID) (container.Container, error) {
prm := pool.PrmContainerGet{
ContainerID: cnrID,
}
var cnr container.Container
cnr, err := f.pool.GetContainer(ctx, prm)
if err != nil {
return container.Container{}, fmt.Errorf("couldn't get container '%s': %w", cnrID, err)
}
return cnr, err
}
func (f *Fs) getObjectsHead(ctx context.Context, cnrID cid.ID, objIDs []oid.ID) ([]object.Object, error) {
var res []object.Object
for _, objID := range objIDs {
var prmHead pool.PrmObjectHead
prmHead.SetAddress(newAddress(cnrID, objID))
obj, err := f.pool.HeadObject(ctx, prmHead)
if err != nil {
return nil, err
}
res = append(res, obj)
}
return res, nil
}
type printer struct {
w io.Writer
err error
}
func newPrinter(w io.Writer) *printer {
return &printer{w: w}
}
func (p *printer) printf(format string, a ...interface{}) {
if p.err != nil {
return
}
if _, err := fmt.Fprintf(p.w, format, a...); err != nil {
p.err = err
}
}
func (p *printer) lastError() error {
return p.err
}
func (p *printer) printContainerInfo(cnrID cid.ID, cnr container.Container) {
p.printf("CID: %v\nOwner ID: %v", cnrID, cnr.Owner())
var timestamp time.Time
var attrs []string
cnr.IterateAttributes(func(key string, value string) {
attrs = append(attrs, fmt.Sprintf(" %v=%v", key, value))
if key == object.AttributeTimestamp {
val, err := strconv.ParseInt(value, 10, 64)
if err == nil {
timestamp = time.Unix(val, 0)
}
}
})
if !timestamp.IsZero() {
p.printf("\nCreated: %v", timestamp)
}
if len(attrs) > 0 {
p.printf("\nAttributes:\n%s", strings.Join(attrs, "\n"))
}
s := bytes.NewBufferString("")
if err := cnr.PlacementPolicy().WriteStringTo(s); err != nil {
return
}
p.printf("\nPlacement policy:\n%s", s.String())
}
func (p *printer) printChecksum(name string, recv func() (checksum.Checksum, bool)) {
var strVal string
cs, csSet := recv()
if csSet {
strVal = hex.EncodeToString(cs.Value())
} else {
strVal = "<empty>"
}
p.printf("\n%s: %s", name, strVal)
}
func (p *printer) printObject(obj *object.Object) {
objIDStr := "<empty>"
cnrIDStr := objIDStr
if objID, ok := obj.ID(); ok {
objIDStr = objID.String()
}
if cnrID, ok := obj.ContainerID(); ok {
cnrIDStr = cnrID.String()
}
p.printf("\nID: %v", objIDStr)
p.printf("\nCID: %v", cnrIDStr)
p.printf("\nOwner: %s", obj.OwnerID())
p.printf("\nCreatedAt: %d", obj.CreationEpoch())
p.printf("\nSize: %d", obj.PayloadSize())
p.printChecksum("HomoHash", obj.PayloadHomomorphicHash)
p.printChecksum("Checksum", obj.PayloadChecksum)
p.printf("\nType: %s", obj.Type())
p.printf("\nAttributes:")
for _, attr := range obj.Attributes() {
if attr.Key() == object.AttributeTimestamp {
var strVal string
val, err := strconv.ParseInt(attr.Value(), 10, 64)
if err == nil {
strVal = time.Unix(val, 0).String()
} else {
strVal = "malformed"
}
p.printf("\n %s=%s (%s)",
attr.Key(),
attr.Value(),
strVal)
continue
}
p.printf("\n %s=%s", attr.Key(), attr.Value())
}
if signature := obj.Signature(); signature != nil {
p.printf("\nID signature:")
var sigV2 refs.Signature
signature.WriteToV2(&sigV2)
p.printf("\n public key: %s", hex.EncodeToString(sigV2.GetKey()))
p.printf("\n signature: %s", hex.EncodeToString(sigV2.GetSign()))
}
if ecHeader := obj.ECHeader(); ecHeader != nil {
p.printf("\nEC header:")
p.printf("\n parent object ID: %s", ecHeader.Parent().EncodeToString())
p.printf("\n index: %d", ecHeader.Index())
p.printf("\n total: %d", ecHeader.Total())
p.printf("\n header length: %d", ecHeader.HeaderLength())
}
p.printSplitHeader(obj)
}
func (p *printer) printSplitHeader(obj *object.Object) {
if splitID := obj.SplitID(); splitID != nil {
p.printf("Split ID: %s\n", splitID)
}
if objID, ok := obj.ParentID(); ok {
p.printf("Split ParentID: %s\n", objID)
}
if prev, ok := obj.PreviousID(); ok {
p.printf("\nSplit PreviousID: %s", prev)
}
for _, child := range obj.Children() {
p.printf("\nSplit ChildID: %s", child.String())
}
parent := obj.Parent()
if parent != nil {
p.printf("\n\nSplit Parent Header:")
p.printObject(parent)
}
}
func formattedInfoOutput(format string, cnrID cid.ID, objHeads []object.Object) (string, error) {
format = strings.ReplaceAll(format, "{cid}", cnrID.String())
objIDStr := "<empty>"
if len(objHeads) > 0 {
objID, ok := objHeads[0].ID()
if ok {
objIDStr = objID.String()
}
}
return strings.ReplaceAll(format, "{oid}", objIDStr), nil
}
func (f *Fs) infoCmd(ctx context.Context, arg []string, opt map[string]string) (out interface{}, err error) {
var cnrID cid.ID
if cnrID, err = f.resolveContainerID(ctx, f.rootContainer); err != nil {
return nil, err
}
var format string
for k, v := range opt {
switch k {
case "format":
format = v
default:
return nil, fmt.Errorf("unknown option \"%s\"", k)
}
}
var objIDs []oid.ID
var filePath string
if len(arg) > 0 {
filePath = strings.TrimPrefix(arg[0], "/")
if f.rootDirectory != "" {
filePath = f.rootDirectory + "/" + filePath
}
if objIDs, err = f.findObjectsFilePath(ctx, cnrID, filePath); err != nil {
return
}
}
cnr, err := f.containerInfo(ctx, cnrID)
if err != nil {
return
}
var objHeads []object.Object
if objHeads, err = f.getObjectsHead(ctx, cnrID, objIDs); err != nil {
return
}
if format != "" {
return formattedInfoOutput(format, cnrID, objHeads)
}
w := bytes.NewBufferString("")
p := newPrinter(w)
p.printf(" --- Container info ---\n")
p.printContainerInfo(cnrID, cnr)
if len(arg) > 0 {
p.printf("\n\n --- Object info ---")
if len(objHeads) > 0 {
// Print info about the first object only
p.printObject(&objHeads[0])
} else {
p.printf("\nNo object with \"%s\" file path was found", filePath)
}
}
if err := p.lastError(); err != nil {
return nil, err
}
return w.String(), nil
}
// Shutdown the backend, closing any background tasks and any
// cached connections.
func (f *Fs) Shutdown(_ context.Context) error {
@ -617,7 +286,7 @@ func NewFs(ctx context.Context, name string, root string, m configmap.Mapper) (f
return f, nil
}
func newObject(f *Fs, obj object.Object, container string) (*Object, error) {
func newObject(f *Fs, obj object.Object, container string) *Object {
// we should not include rootDirectory into remote name
prefix := f.rootDirectory
if prefix != "" {
@ -651,12 +320,9 @@ func newObject(f *Fs, obj object.Object, container string) (*Object, error) {
}
}
// We expect that the FilePath attribute is present in the object and that it starts with a leading slash
if objInfo.filePath == "" || objInfo.filePath[0] != '/' {
return nil, errMalformedObject
if objInfo.filePath == "" {
objInfo.filePath = objInfo.name
}
// Don't include a leading slash in the resulting object's file path.
objInfo.filePath = objInfo.filePath[1:]
objInfo.remote = objInfo.filePath
if strings.Contains(objInfo.remote, prefix) {
@ -667,7 +333,7 @@ func newObject(f *Fs, obj object.Object, container string) (*Object, error) {
}
}
return objInfo, nil
return objInfo
}
// MimeType of an Object if known, "" otherwise
@ -835,26 +501,26 @@ func (f *Fs) Features() *fs.Features {
// List the objects and directories in dir into entries.
func (f *Fs) List(ctx context.Context, dir string) (fs.DirEntries, error) {
rootDirName, containerPath := bucket.Split(path.Join(f.root, dir))
containerStr, containerPath := bucket.Split(path.Join(f.root, dir))
if rootDirName == "" {
if containerStr == "" {
if containerPath != "" {
return nil, fs.ErrorListBucketRequired
}
return f.listContainers(ctx)
}
return f.listEntries(ctx, rootDirName, containerPath, dir, false)
return f.listEntries(ctx, containerStr, containerPath, dir, false)
}
// ListR lists the objects and directories of the Fs starting
// from dir recursively into out.
func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) error {
rootDirName, containerPath := bucket.Split(path.Join(f.root, dir))
containerStr, containerPath := bucket.Split(path.Join(f.root, dir))
list := walk.NewListRHelper(callback)
if rootDirName == "" {
if containerStr == "" {
if containerPath != "" {
return fs.ErrorListBucketRequired
}
@ -870,15 +536,15 @@ func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) e
return list.Flush()
}
if err := f.listR(ctx, list, rootDirName, containerPath, dir); err != nil {
if err := f.listR(ctx, list, containerStr, containerPath, dir); err != nil {
return err
}
return list.Flush()
}
func (f *Fs) listR(ctx context.Context, list *walk.ListRHelper, rootDirName, containerPath, dir string) error {
entries, err := f.listEntries(ctx, rootDirName, containerPath, dir, true)
func (f *Fs) listR(ctx context.Context, list *walk.ListRHelper, containerStr, containerPath, dir string) error {
entries, err := f.listEntries(ctx, containerStr, containerPath, dir, true)
if err != nil {
return err
}
@ -891,37 +557,31 @@ func (f *Fs) listR(ctx context.Context, list *walk.ListRHelper, rootDirName, con
return nil
}
func (f *Fs) resolveOrCreateContainer(ctx context.Context, rootDirName string) (cid.ID, error) {
// Due to the fact that this method is called when performing "put" operations,
// which can be run in parallel in several goroutines,
// we need to use a global lock here so that if a requested container is missing,
// multiple goroutines do not attempt to create a container with the same name simultaneously,
// which could cause unexpected behavior.
func (f *Fs) resolveOrCreateContainer(ctx context.Context, containerStr string) (cid.ID, error) {
f.m.Lock()
defer f.m.Unlock()
cnrIDStr, ok := f.containerIDCache[rootDirName]
if ok {
return parseContainerID(cnrIDStr)
cnrID, err := f.resolveContainerIDHelper(ctx, containerStr)
if err == nil {
return cnrID, err
}
cnrID, err := f.resolveCIDByRootDirName(ctx, rootDirName)
if err != nil {
if cnrID, err = f.createContainer(ctx, rootDirName); err != nil {
return cid.ID{}, fmt.Errorf("createContainer: %w", err)
}
if cnrID, err = f.createContainer(ctx, containerStr); err != nil {
delete(f.containerIDCache, containerStr)
return cid.ID{}, fmt.Errorf("createContainer: %w", err)
}
f.containerIDCache[rootDirName] = cnrID.String()
f.containerIDCache[containerStr] = cnrID.String()
return cnrID, nil
}
// Put the Object into the container
func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error) {
rootDirName, containerPath := bucket.Split(filepath.Join(f.root, src.Remote()))
containerStr, containerPath := bucket.Split(filepath.Join(f.root, src.Remote()))
cnrID, err := parseContainerID(rootDirName)
cnrID, err := parseContainerID(containerStr)
if err != nil {
if cnrID, err = f.resolveOrCreateContainer(ctx, rootDirName); err != nil {
if cnrID, err = f.resolveOrCreateContainer(ctx, containerStr); err != nil {
return nil, err
}
}
@ -959,7 +619,7 @@ func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options .
_ = f.pool.DeleteObject(ctx, prmDelete)
}
return newObject(f, obj, "")
return newObject(f, obj, ""), nil
}
func fillHeaders(ctx context.Context, filePath string, src fs.ObjectInfo, options ...fs.OpenOption) map[string]string {
@ -970,7 +630,7 @@ func fillHeaders(ctx context.Context, filePath string, src fs.ObjectInfo, option
})
}
headers := map[string]string{object.AttributeFilePath: "/" + filePath}
headers := map[string]string{object.AttributeFilePath: filePath}
for _, option := range options {
key, value := option.Header()
@ -1003,10 +663,10 @@ func fillHeaders(ctx context.Context, filePath string, src fs.ObjectInfo, option
func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) error {
// When updating an object, the path to it should not change.
src = robject.NewStaticObjectInfo(o.Remote(), src.ModTime(ctx), src.Size(), src.Storable(), nil, src.Fs())
rootDirName, containerPath := bucket.Split(filepath.Join(o.fs.root, src.Remote()))
containerStr, containerPath := bucket.Split(filepath.Join(o.fs.root, src.Remote()))
var cnrID cid.ID
var err error
if cnrID, err = o.fs.parseContainer(ctx, rootDirName); err != nil {
if cnrID, err = o.fs.parseContainer(ctx, containerStr); err != nil {
return fmt.Errorf("parse container: %w", err)
}
@ -1046,10 +706,8 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
if err != nil {
return fmt.Errorf("fetch head object: %w", err)
}
var objInfo *Object
if objInfo, err = newObject(o.fs, obj, ""); err != nil {
return err
}
objInfo := newObject(o.fs, obj, "")
o.filePath = objInfo.filePath
o.remote = objInfo.remote
@ -1060,10 +718,6 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
return nil
}
func (f *Fs) getContainerNameAndZone(containerStr string) (string, string) {
return getContainerNameAndZone(containerStr, f.opt.DefaultContainerZone)
}
// Remove an object
func (o *Object) Remove(ctx context.Context) error {
cnrID, _ := o.ContainerID()
@ -1099,7 +753,7 @@ func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error) {
return nil, fmt.Errorf("head object: %w", err)
}
return newObject(f, obj, "")
return newObject(f, obj, ""), nil
}
func (f *Fs) waitForAPECacheInvalidated(ctx context.Context, expectedCh chain.Chain, cnrID cid.ID) error {
@ -1151,7 +805,7 @@ func (f *Fs) waitForAPECacheInvalidated(ctx context.Context, expectedCh chain.Ch
}
}
func (f *Fs) createContainer(ctx context.Context, rootDirName string) (cid.ID, error) {
func (f *Fs) createContainer(ctx context.Context, containerName string) (cid.ID, error) {
var policy netmap.PlacementPolicy
if err := policy.DecodeString(f.opt.PlacementPolicy); err != nil {
return cid.ID{}, fmt.Errorf("parse placement policy: %w", err)
@ -1163,12 +817,10 @@ func (f *Fs) createContainer(ctx context.Context, rootDirName string) (cid.ID, e
cnr.SetOwner(*f.owner)
container.SetCreationTime(&cnr, time.Now())
container.SetName(&cnr, rootDirName)
container.SetName(&cnr, containerName)
cnrName, cnrZone := f.getContainerNameAndZone(rootDirName)
var domain container.Domain
domain.SetZone(cnrZone)
domain.SetName(cnrName)
domain.SetName(containerName)
container.WriteDomain(&cnr, domain)
if err := pool.SyncContainerWithNetwork(ctx, &cnr, f.pool); err != nil {
@ -1214,14 +866,14 @@ func (f *Fs) createContainer(ctx context.Context, rootDirName string) (cid.ID, e
// Mkdir creates the container if it doesn't exist
func (f *Fs) Mkdir(ctx context.Context, dir string) error {
rootDirName, _ := bucket.Split(path.Join(f.root, dir))
if rootDirName == "" {
containerStr, _ := bucket.Split(path.Join(f.root, dir))
if containerStr == "" {
return nil
}
_, err := parseContainerID(rootDirName)
_, err := parseContainerID(containerStr)
if err != nil {
if _, err = f.resolveOrCreateContainer(ctx, rootDirName); err != nil {
if _, err = f.resolveOrCreateContainer(ctx, containerStr); err != nil {
return err
}
}
@ -1231,12 +883,12 @@ func (f *Fs) Mkdir(ctx context.Context, dir string) error {
// Rmdir deletes the bucket if the fs is at the root
func (f *Fs) Rmdir(ctx context.Context, dir string) error {
rootDirName, containerPath := bucket.Split(path.Join(f.root, dir))
if rootDirName == "" || containerPath != "" {
containerStr, containerPath := bucket.Split(path.Join(f.root, dir))
if containerStr == "" || containerPath != "" {
return nil
}
cnrID, err := f.parseContainer(ctx, rootDirName)
cnrID, err := f.parseContainer(ctx, containerStr)
if err != nil {
return fs.ErrorDirNotFound
}
@ -1256,18 +908,18 @@ func (f *Fs) Rmdir(ctx context.Context, dir string) error {
f.m.Lock()
defer f.m.Unlock()
if err = f.pool.DeleteContainer(ctx, prm); err != nil {
return fmt.Errorf("couldn't delete container %s '%s': %w", cnrID, rootDirName, err)
return fmt.Errorf("couldn't delete container %s '%s': %w", cnrID, containerStr, err)
}
delete(f.containerIDCache, rootDirName)
delete(f.containerIDCache, containerStr)
return nil
}
// Purge deletes all the files and directories including the old versions.
func (f *Fs) Purge(ctx context.Context, dir string) error {
rootDirName, containerPath := bucket.Split(path.Join(f.root, dir))
containerStr, containerPath := bucket.Split(path.Join(f.root, dir))
cnrID, err := f.parseContainer(ctx, rootDirName)
cnrID, err := f.parseContainer(ctx, containerStr)
if err != nil {
return nil
}
@ -1292,9 +944,8 @@ func parseContainerID(containerStr string) (cid.ID, error) {
return cnrID, err
}
func getContainerIDByNameAndZone(dirEntry fs.DirEntry, cnrName, cnrZone, defaultZone string) (cnrID cid.ID, ok bool, err error) {
actualName, actualZone := getContainerNameAndZone(dirEntry.Remote(), defaultZone)
if cnrName != actualName || cnrZone != actualZone {
func getContainerIDByName(dirEntry fs.DirEntry, containerName string) (ok bool, cnrID cid.ID, err error) {
if dirEntry.Remote() != containerName {
return
}
var idEr fs.IDer
@ -1305,62 +956,61 @@ func getContainerIDByNameAndZone(dirEntry fs.DirEntry, cnrName, cnrZone, default
return
}
func resolveContainerIDWithNNS(resolver *resolver.NNS, cnrName, cnrZone string) (cid.ID, error) {
func resolveContainerIDWithNNS(resolver *resolver.NNS, containerName string) (cid.ID, error) {
var d container.Domain
d.SetZone(cnrZone)
d.SetName(cnrName)
d.SetName(containerName)
if cnrID, err := resolver.ResolveContainerDomain(d); err == nil {
return cnrID, err
}
return cid.ID{}, fmt.Errorf("couldn't resolve container with name '%s' and zone '%s'", cnrName, cnrZone)
return cid.ID{}, fmt.Errorf("couldn't resolve container '%s'", containerName)
}
func (f *Fs) resolveCIDByRootDirName(ctx context.Context, rootDirName string) (cid.ID, error) {
cnrName, cnrZone := f.getContainerNameAndZone(rootDirName)
if cnrName == "" {
return cid.ID{}, fmt.Errorf("couldn't resolve container '%s'", rootDirName)
func (f *Fs) resolveContainerIDHelper(ctx context.Context, containerName string) (cid.ID, error) {
cnrIDStr, ok := f.containerIDCache[containerName]
if ok {
return parseContainerID(cnrIDStr)
}
if f.resolver != nil {
return resolveContainerIDWithNNS(f.resolver, cnrName, cnrZone)
var err error
var cnrID cid.ID
if cnrID, err = resolveContainerIDWithNNS(f.resolver, containerName); err == nil {
f.containerIDCache[containerName] = cnrID.String()
}
return cnrID, err
}
if dirEntries, err := f.listContainers(ctx); err == nil {
for _, dirEntry := range dirEntries {
if cnrID, ok, err := getContainerIDByNameAndZone(dirEntry, cnrName, cnrZone, f.opt.DefaultContainerZone); ok {
if ok, cnrID, err := getContainerIDByName(dirEntry, containerName); ok {
if err == nil {
f.containerIDCache[containerName] = cnrID.String()
}
return cnrID, err
}
}
}
return cid.ID{}, fmt.Errorf("couldn't resolve container '%s'", rootDirName)
return cid.ID{}, fmt.Errorf("couldn't resolve container '%s'", containerName)
}
func (f *Fs) resolveContainerID(ctx context.Context, rootDirName string) (cid.ID, error) {
func (f *Fs) resolveContainerID(ctx context.Context, containerName string) (cid.ID, error) {
f.m.Lock()
defer f.m.Unlock()
cnrIDStr, ok := f.containerIDCache[rootDirName]
if ok {
return parseContainerID(cnrIDStr)
}
cnrID, err := f.resolveCIDByRootDirName(ctx, rootDirName)
if err != nil {
return cid.ID{}, err
}
f.containerIDCache[rootDirName] = cnrID.String()
return cnrID, nil
return f.resolveContainerIDHelper(ctx, containerName)
}
func (f *Fs) parseContainer(ctx context.Context, rootDirName string) (cid.ID, error) {
cnrID, err := parseContainerID(rootDirName)
if err != nil {
return f.resolveContainerID(ctx, rootDirName)
func (f *Fs) parseContainer(ctx context.Context, containerName string) (cid.ID, error) {
cnrID, err := parseContainerID(containerName)
if err == nil {
return cnrID, err
}
return cnrID, nil
return f.resolveContainerID(ctx, containerName)
}
func (f *Fs) listEntries(ctx context.Context, rootDirName, containerPath, directory string, recursive bool) (fs.DirEntries, error) {
cnrID, err := f.parseContainer(ctx, rootDirName)
func (f *Fs) listEntries(ctx context.Context, containerStr, containerPath, directory string, recursive bool) (fs.DirEntries, error) {
cnrID, err := f.parseContainer(ctx, containerStr)
if err != nil {
return nil, fs.ErrorDirNotFound
}
@ -1383,11 +1033,7 @@ func (f *Fs) listEntries(ctx context.Context, rootDirName, containerPath, direct
return nil, err
}
objInf, err := newObject(f, obj, rootDirName)
if err != nil {
// skip an erroneous object
continue
}
objInf := newObject(f, obj, containerStr)
if !recursive {
withoutPath := strings.TrimPrefix(objInf.filePath, containerPath)
@ -1437,7 +1083,7 @@ func (f *Fs) listContainers(ctx context.Context) (fs.DirEntries, error) {
return nil, fmt.Errorf("couldn't get container '%s': %w", containerID, err)
}
res[i] = newDir(containerID, cnr, f.opt.DefaultContainerZone)
res[i] = newDir(containerID, cnr)
}
return res, nil
@ -1446,7 +1092,7 @@ func (f *Fs) listContainers(ctx context.Context) (fs.DirEntries, error) {
func (f *Fs) findObjectsFilePath(ctx context.Context, cnrID cid.ID, filePath string) ([]oid.ID, error) {
return f.findObjects(ctx, cnrID, searchFilter{
Header: object.AttributeFilePath,
Value: "/" + filePath,
Value: filePath,
MatchType: object.MatchStringEqual,
})
}
@ -1454,7 +1100,7 @@ func (f *Fs) findObjectsFilePath(ctx context.Context, cnrID cid.ID, filePath str
func (f *Fs) findObjectsPrefix(ctx context.Context, cnrID cid.ID, prefix string) ([]oid.ID, error) {
return f.findObjects(ctx, cnrID, searchFilter{
Header: object.AttributeFilePath,
Value: "/" + prefix,
Value: prefix,
MatchType: object.MatchCommonPrefix,
})
}
@ -1473,7 +1119,7 @@ func (f *Fs) findObjects(ctx context.Context, cnrID cid.ID, filters ...searchFil
func (f *Fs) deleteByPrefix(ctx context.Context, cnrID cid.ID, prefix string) error {
filters := object.NewSearchFilters()
filters.AddRootFilter()
filters.AddFilter(object.AttributeFilePath, "/"+prefix, object.MatchCommonPrefix)
filters.AddFilter(object.AttributeFilePath, prefix, object.MatchCommonPrefix)
var prmSearch pool.PrmObjectSearch
prmSearch.SetContainerID(cnrID)

View file

@ -296,31 +296,16 @@ func formObject(own *user.ID, cnrID cid.ID, name string, header map[string]strin
return obj
}
func newDir(cnrID cid.ID, cnr container.Container, defaultZone string) *fs.Dir {
func newDir(cnrID cid.ID, cnr container.Container) *fs.Dir {
remote := cnrID.EncodeToString()
timestamp := container.CreatedAt(cnr)
if domain := container.ReadDomain(cnr); domain.Name() != "" {
if defaultZone != domain.Zone() {
remote = domain.Name() + "." + domain.Zone()
} else {
remote = domain.Name()
}
remote = domain.Name()
}
dir := fs.NewDir(remote, timestamp)
dir.SetID(cnrID.String())
return dir
}
func getContainerNameAndZone(containerStr, defaultZone string) (cnrName string, cnrZone string) {
defer func() {
if len(cnrZone) == 0 {
cnrZone = defaultZone
}
}()
if idx := strings.Index(containerStr, "."); idx >= 0 {
return containerStr[:idx], containerStr[idx+1:]
}
return containerStr, defaultZone
}

View file

@ -7,57 +7,6 @@ import (
"github.com/stretchr/testify/require"
)
func TestGetZoneAndContainerNames(t *testing.T) {
for i, tc := range []struct {
cnrStr string
defZone string
expectedName string
expectedZone string
}{
{
cnrStr: "",
defZone: "def_zone",
expectedName: "",
expectedZone: "def_zone",
},
{
cnrStr: "",
defZone: "def_zone",
expectedName: "",
expectedZone: "def_zone",
},
{
cnrStr: "cnr_name",
defZone: "def_zone",
expectedName: "cnr_name",
expectedZone: "def_zone",
},
{
cnrStr: "cnr_name.",
defZone: "def_zone",
expectedName: "cnr_name",
expectedZone: "def_zone",
},
{
cnrStr: ".cnr_zone",
defZone: "def_zone",
expectedName: "",
expectedZone: "cnr_zone",
}, {
cnrStr: ".cnr_zone",
defZone: "def_zone",
expectedName: "",
expectedZone: "cnr_zone",
},
} {
t.Run(strconv.Itoa(i), func(t *testing.T) {
actualName, actualZone := getContainerNameAndZone(tc.cnrStr, tc.defZone)
require.Equal(t, tc.expectedZone, actualZone)
require.Equal(t, tc.expectedName, actualName)
})
}
}
func TestParseContainerCreationPolicy(t *testing.T) {
for i, tc := range []struct {
ACLString string

View file

@ -302,7 +302,6 @@ rclone [flags]
--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms)
--frostfs-connection-timeout Duration FrostFS connection timeout (default 4s)
--frostfs-container-creation-policy string Container creation policy for new containers (default "private")
--frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container")
--frostfs-description string Description of the remote
--frostfs-endpoint string Endpoints to connect to FrostFS node
--frostfs-password string Password to decrypt wallet
@ -945,7 +944,7 @@ rclone [flags]
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8339.71f0cdbd4.feat/frostfs_info_cmd")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support")
-v, --verbose count Print lots more stuff (repeat for more)
-V, --version Print the version number
--webdav-bearer-token string Bearer token instead of user/pass (e.g. a Macaroon)

View file

@ -115,7 +115,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8339.71f0cdbd4.feat/frostfs_info_cmd")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support")
```
@ -558,7 +558,6 @@ Backend-only flags (these can be set in the config file also).
--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms)
--frostfs-connection-timeout Duration FrostFS connection timeout (default 4s)
--frostfs-container-creation-policy string Container creation policy for new containers (default "private")
--frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default "container")
--frostfs-description string Description of the remote
--frostfs-endpoint string Endpoints to connect to FrostFS node
--frostfs-password string Password to decrypt wallet

View file

@ -230,11 +230,6 @@ Press Enter for the default (REP 3).
\ (REP 3)
placement_policy> REP 1
Option default_container_zone.
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty.
Enter a value of type string. Press Enter for the default (container).
default_container_zone>
Option container_creation_policy.
Container creation policy for new containers
Choose a number from below, or type in your own value of type string.
@ -275,14 +270,6 @@ the identifier `23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf`:
rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy
rclone copy ~/test-copy remote:container-name/test-copy
Also, for user-friendly container names, you can explicitly specify the name of the zone in which you want
to create or search for a container:
rclone copy ~/test-copy remote:container-name.container-zone/test-copy
If the zone is not explicitly specified, its name will be obtained from the configuration parameter
`default_container_zone`.
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/frostfs/frostfs.go then run make backenddocs" >}}
### Standard options
@ -443,17 +430,6 @@ Properties:
- "REP 3"
- Container will have 3 replicas
#### --frostfs-default-container-zone
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name.
Properties:
- Config: default_container_zone
- Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE
- Type: string
- Default: "container"
#### --frostfs-container-creation-policy
Container creation policy for new containers
@ -487,73 +463,4 @@ Properties:
- Type: string
- Required: false
## Backend commands
Here are the commands specific to the frostfs backend.
Run them with
rclone backend COMMAND remote:
The help below will explain what arguments each command takes.
See the [backend](/commands/rclone_backend/) command for more
info on how to pass options and arguments.
These can be run on a running backend using the rc command
[backend/command](/rc/#backend-command).
### info
Show information about the FrostFS objects and containers
rclone backend info remote: [options] [<arguments>+]
This command can be used to get information about the FrostFS objects and containers.
Usage Examples:
rclone backend info frostfs:container/path/to/dir
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt -o "format={cid}:{oid}"
The optional "format" flag overrides the information output. In this example, if an object is stored in
a container with the identifier "9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1" and
its own identifier is "4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP", the output of this command will be
"9mvN7hsucoGoHjxPqrWmDipnMaemGVDqrxxPyynn1:4VpcNFsZqsQt1Gnfw2utBnzn5Blgc7i4kvtuXyKyJp".
The default output format is the same as that of the frostfs-cli utility,
with the "container get" and "object head" options. Here is an example of output:
--- Container info ---
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner ID: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
Created: 2025-02-17 15:07:51 +0300 MSK
Attributes:
Timestamp=1739794071
Name=test
__SYSTEM__NAME=test
__SYSTEM__ZONE=container
__SYSTEM__DISABLE_HOMOMORPHIC_HASHING=true
Placement policy:
REP 3
--- Object info ---
ID: 4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
CreatedAt: 559
Size: 402905
HomoHash: <empty>
Checksum: 2a068fe24c53bc8bf7d6bbb997414f7938b080305dc45f9fd3ff684bc11fbb7b
Type: REGULAR
Attributes:
FileName=cat.png
FilePath=/dir1/dir2/dir3/cat.png
Timestamp=1733410524 (2024-12-05 17:55:24 +0300 MSK)
ID signature:
public key: 026b7c7a7a16225eb13a5a733495a1bcdd1f016dfa9193498821379b0de2ba6870
signature: 049f6712c8378d323269b605a282bcacd7565ce2eefe1f10a9739c48945f739d95102c478b9cb1d429cd3330b4b5262e725392e322de3bbfa4ce18a9c842289219
{{< rem autogenerated options stop >}}

132
rclone.1 generated
View file

@ -15,7 +15,7 @@
. ftr VB CB
. ftr VBI CBI
.\}
.TH "rclone" "1" "Feb 22, 2025" "User Manual" ""
.TH "rclone" "1" "Jan 10, 2025" "User Manual" ""
.hy
.SH Rclone syncs your files to cloud storage
.PP
@ -27735,7 +27735,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.68.2-beta.8339.71f0cdbd4.feat/frostfs_info_cmd\[dq])
--user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.68.2-beta.8331.25cf42493.feature/add-frostfs-support\[dq])
\f[R]
.fi
.SS Performance
@ -28178,7 +28178,6 @@ Backend-only flags (these can be set in the config file also).
--frostfs-ape-chain-check-interval Duration The interval for verifying that the APE chain is saved in FrostFS (default 500ms)
--frostfs-connection-timeout Duration FrostFS connection timeout (default 4s)
--frostfs-container-creation-policy string Container creation policy for new containers (default \[dq]private\[dq])
--frostfs-default-container-zone string The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name (default \[dq]container\[dq])
--frostfs-description string Description of the remote
--frostfs-endpoint string Endpoints to connect to FrostFS node
--frostfs-password string Password to decrypt wallet
@ -45690,7 +45689,7 @@ connection_timeout>
Option request_timeout.
FrostFS request timeout
Enter a value of type Duration. Press Enter for the default (12s).
Enter a value of type Duration. Press Enter for the default (4s).
request_timeout>
Option rebalance_interval.
@ -45710,7 +45709,7 @@ ape_cache_invalidation_duration>
Option ape_cache_invalidation_timeout.
APE cache invalidation timeout
Enter a value of type Duration. Press Enter for the default (24s).
Enter a value of type Duration. Press Enter for the default (16s).
ape_cache_invalidation_timeout>
Option ape_chain_check_interval.
@ -45746,11 +45745,6 @@ Press Enter for the default (REP 3).
\[rs] (REP 3)
placement_policy> REP 1
Option default_container_zone.
The name of the zone in which containers will be created or resolved if the zone name is not explicitly specified with the container name. Can be empty.
Enter a value of type string. Press Enter for the default (container).
default_container_zone>
Option container_creation_policy.
Container creation policy for new containers
Choose a number from below, or type in your own value of type string.
@ -45797,18 +45791,6 @@ rclone copy \[ti]/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/
rclone copy \[ti]/test-copy remote:container-name/test-copy
\f[R]
.fi
.PP
Also, for user-friendly container names, you can explicitly specify the
name of the zone in which you want to create or search for a container:
.IP
.nf
\f[C]
rclone copy \[ti]/test-copy remote:container-name.container-zone/test-copy
\f[R]
.fi
.PP
If the zone is not explicitly specified, its name will be obtained from
the configuration parameter \f[V]default_container_zone\f[R].
.SS Standard options
.PP
Here are the Standard options specific to frostfs (Distributed,
@ -46024,20 +46006,6 @@ Examples:
Container will have 3 replicas
.RE
.RE
.SS --frostfs-default-container-zone
.PP
The name of the zone in which containers will be created or resolved if
the zone name is not explicitly specified with the container name.
.PP
Properties:
.IP \[bu] 2
Config: default_container_zone
.IP \[bu] 2
Env Var: RCLONE_FROSTFS_DEFAULT_CONTAINER_ZONE
.IP \[bu] 2
Type: string
.IP \[bu] 2
Default: \[dq]container\[dq]
.SS --frostfs-container-creation-policy
.PP
Container creation policy for new containers
@ -46090,94 +46058,6 @@ Env Var: RCLONE_FROSTFS_DESCRIPTION
Type: string
.IP \[bu] 2
Required: false
.SS Backend commands
.PP
Here are the commands specific to the frostfs backend.
.PP
Run them with
.IP
.nf
\f[C]
rclone backend COMMAND remote:
\f[R]
.fi
.PP
The help below will explain what arguments each command takes.
.PP
See the backend (https://rclone.org/commands/rclone_backend/) command
for more info on how to pass options and arguments.
.PP
These can be run on a running backend using the rc command
backend/command (https://rclone.org/rc/#backend-command).
.SS info
.PP
Show information about the FrostFS objects and containers
.IP
.nf
\f[C]
rclone backend info remote: [options] [<arguments>+]
\f[R]
.fi
.PP
This command can be used to get information about the FrostFS objects
and containers.
.PP
Usage Examples:
.IP
.nf
\f[C]
rclone backend info frostfs:container/path/to/dir
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt
rclone backend info frostfs:container/path/to/dir path/to/file/in/dir.txt -o \[dq]format={cid}:{oid}\[dq]
\f[R]
.fi
.PP
The optional \[dq]format\[dq] flag overrides the information output.
In this example, if an object is stored in a container with the
identifier \[dq]9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1\[dq] and
its own identifier is
\[dq]4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP\[dq], the output of
this command will be
\[dq]9mvN7hsucoGoHjxPqrWmDipnMaemGVDqrxxPyynn1:4VpcNFsZqsQt1Gnfw2utBnzn5Blgc7i4kvtuXyKyJp\[dq].
.PP
The default output format is the same as that of the frostfs-cli
utility, with the \[dq]container get\[dq] and \[dq]object head\[dq]
options.
Here is an example of output:
.IP
.nf
\f[C]
--- Container info ---
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner ID: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
Created: 2025-02-17 15:07:51 +0300 MSK
Attributes:
Timestamp=1739794071
Name=test
__SYSTEM__NAME=test
__SYSTEM__ZONE=container
__SYSTEM__DISABLE_HOMOMORPHIC_HASHING=true
Placement policy:
REP 3
--- Object info ---
ID: 4VPCNFsZ2SQt1GNfYw2uTBNnz5bLgC7i4k4ovtuXKyJP
CID: 9mvN7hsUcYoGoHjxpRWtqmDipnmaeRmGVDqRxxPyy2n1
Owner: NQL7q6PvPaisWNwdWfoR1LsEsAyje8P3jX
CreatedAt: 559
Size: 402905
HomoHash: <empty>
Checksum: 2a068fe24c53bc8bf7d6bbb997414f7938b080305dc45f9fd3ff684bc11fbb7b
Type: REGULAR
Attributes:
FileName=cat.png
FilePath=/dir1/dir2/dir3/cat.png
Timestamp=1733410524 (2024-12-05 17:55:24 +0300 MSK)
ID signature:
public key: 026b7c7a7a16225eb13a5a733495a1bcdd1f016dfa9193498821379b0de2ba6870
signature: 049f6712c8378d323269b605a282bcacd7565ce2eefe1f10a9739c48945f739d95102c478b9cb1d429cd3330b4b5262e725392e322de3bbfa4ce18a9c842289219
\f[R]
.fi
.SH FTP
.PP
FTP is the File Transfer Protocol.
@ -46335,14 +46215,14 @@ rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=
rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:
\f[R]
.fi
.SS Implicit TLS
.SS Implicit TLSutil.go
.PP
Rlone FTP supports implicit FTP over TLS servers (FTPS).
This has to be enabled in the FTP backend config for the remote, or with
\f[V]--ftp-tls\f[R].
The default FTPS port is \f[V]990\f[R], not \f[V]21\f[R] and can be set
with \f[V]--ftp-port\f[R].
.SS Restricted filename characters
util.go ### Restricted filename characters
.PP
In addition to the default restricted characters
set (https://rclone.org/overview/#restricted-characters) the following