Compare commits

...
Sign in to create a new pull request.

23 commits

Author SHA1 Message Date
Nick Craig-Wood
ab8be2663f Version v1.60.1 2022-11-17 13:56:19 +00:00
asdffdsazqqq
9e2aaa0f67 docs: faq: how to use a proxy server that requires a username and password - fixes #6565 2022-11-17 13:44:24 +00:00
Nick Craig-Wood
f8997b8336 webdav: fix Move/Copy/DirMove when using -server-side-across-configs
Before this change, when using -server-side-across-configs rclone
would direct Move/Copy/DirMove to the destination server.

However this should be directed to the source server. This is a little
unclear in the RFC, but the name of the parameter "Destination:" seems
clear and this is how dCache and Rucio have implemented it.

See: https://forum.rclone.org/t/webdav-copy-request-implemented-incorrectly/34072/
2022-11-17 13:44:02 +00:00
Nick Craig-Wood
567d8fc8d1 smb: fix Failed to sync: context canceled at the end of syncs
Before this change we were putting connections into the connection
pool which had a local context in.

This meant that when the operation had finished the context was
cancelled and the connection became unusable.

See: https://forum.rclone.org/t/failed-to-sync-context-canceled/34017/
2022-11-16 10:56:09 +00:00
Nick Craig-Wood
4ffe9dcfef vfs: fix deadlock caused by cache cleaner and upload finishing
Before this patch a deadlock could occur if the cache cleaner was
running when an object upload finished.

This fixes the problem by delaying marking the object as clean until
we have notified the VFS layer. This means that the cache cleaner
won't consider the object until **after** the VFS layer has been
notified, thus avoiding the deadlock.

See: https://forum.rclone.org/t/rclone-mount-deadlock-when-dir-cache-time-strikes/33486/
2022-11-15 18:01:46 +00:00
techknowlogick
6d0824954b s3: Add additional Wasabi locations 2022-11-15 09:02:48 +00:00
Aaron Gokaslan
1a5160c7aa s3: fix for unchecked err value in s3 listv2 2022-11-15 09:02:25 +00:00
Nick Craig-Wood
5012f78c45 local: fix -L/--copy-links with filters missing directories
In this commit

8d1fff9a82 local: obey file filters in listing to fix errors on excluded files

We introduced the concept of local backend filters.

Unfortunately the filters were being applied before we had resolved
the symlink to point to a directory. This meant that symlinks pointing
to directories were filtered out when they shouldn't have been.

This was fixed by moving the filter check until after the symlink had
been resolved.

See: https://forum.rclone.org/t/copy-links-not-following-symlinks-on-1-60-0/34073/7
2022-11-15 09:00:08 +00:00
Nick Craig-Wood
8f3741419f mailru: allow timestamps to be before the epoch 1970-01-01
Fixes #6547
2022-11-10 11:27:50 +00:00
Nick Craig-Wood
91afa14d28 lib/cache: fix alias backend shutting down too soon
Before this patch, when an alias backend was created it would be
renamed to be canonical and in the process Shutdown would be called on
it. This was particularly noticeable with the dropbox backend which
gave this error when uploading files after the backend was Shutdown.

    Failed to copy: upload failed: batcher is shutting down

This patch fixes the cache Rename code not to finalize objects if the
object that is being overwritten is the same as the existing object.

See: https://forum.rclone.org/t/upload-failed-batcher-is-shutting-down/33900
2022-11-09 16:30:33 +00:00
Nick Craig-Wood
4038c2c316 mailru: note that an app password is now needed - fixes #6398 2022-11-08 20:33:33 +00:00
Nick Craig-Wood
ed7c286ef6 docs: remove link to rclone slack as it is no longer supported 2022-11-08 16:12:02 +00:00
Nick Craig-Wood
0866a7b6e2 s3: allow Storj to server side copy since it seems to work now - fixes #6550 2022-11-08 16:08:29 +00:00
João Henrique Franco
ac38744e69 wasm: fix walltime link error by adding up-to-date wasm_exec.js
Solves link error while running rclone's wasm version. Go's `walltime1` function was renamed to `walltime`. This commit updates wasm_exec.js with the new name.
2022-11-08 16:08:29 +00:00
Clément Notin
dfc21a5b9e docs: fix character that was incorrectly interpreted as markdown 2022-11-08 16:08:29 +00:00
Joda Stößer
ff408f5309 docs: fix typo remove in rclone_serve_restic command 2022-11-08 16:08:29 +00:00
Nick Craig-Wood
e2d2a05361 s3: add provider quirk --s3-might-gzip to fix corrupted on transfer: sizes differ
Before this change, some files were giving this error when downloaded
from Cloudflare and other providers.

    ERROR corrupted on transfer: sizes differ NNN vs MMM

This is because these providers auto gzips the object when rclone
wasn't expecting it to. (AWS does not gzip objects without their being
uploaded gzipped).

This patch adds a quirk to for fix the problem and a flag to control
it. The quirk `might_gzip` is set to `true` for all providers except
AWS.

See: https://forum.rclone.org/t/s3-error-corrupted-on-transfer-sizes-differ-nnn-vs-mmm/33694/
Fixes: #6533
2022-11-04 17:01:03 +00:00
albertony
e8002c66a9 docs: add direct download link for windows arm64 2022-11-04 17:01:03 +00:00
coultonluke
6f207668c2 docs: corrected download links in windows install docs 2022-11-04 17:01:03 +00:00
Manoj Ghosh
a2491719a1 oracle-object-storage: doc fix
See #6521
2022-11-04 17:01:03 +00:00
albertony
29e1c68ee0 local: clean absolute paths - fixes #6493 2022-11-04 17:01:03 +00:00
Samuel Johnson
05040cc504 docs: Update faq.md with bisync
Updated FAQ to clarify that experimental bi-sync is now available.
2022-11-04 17:01:03 +00:00
Nick Craig-Wood
6944f28dc2 Start v1.60.1-DEV development 2022-11-04 16:57:08 +00:00
26 changed files with 1473 additions and 641 deletions

141
MANUAL.html generated
View file

@ -19,7 +19,7 @@
<header id="title-block-header">
<h1 class="title">rclone(1) User Manual</h1>
<p class="author">Nick Craig-Wood</p>
<p class="date">Oct 21, 2022</p>
<p class="date">Nov 17, 2022</p>
</header>
<h1 id="rclone-syncs-your-files-to-cloud-storage">Rclone syncs your files to cloud storage</h1>
<p><img width="50%" src="https://rclone.org/img/logo_on_light__horizontal_color.svg" alt="rclone logo" style="float:right; padding: 5px;" ></p>
@ -227,9 +227,9 @@ macOS cannot verify that this app is free from malware.</code></pre>
<h3 id="windows-precompiled">Precompiled binary</h3>
<p>Fetch the correct binary for your processor type by clicking on these links. If not sure, use the first link.</p>
<ul>
<li><a href="https://downloads.rclone.org/rclone-current-linux-amd64.zip">Intel/AMD - 64 Bit</a></li>
<li><a href="https://downloads.rclone.org/rclone-current-linux-386.zip">Intel/AMD - 32 Bit</a></li>
<li><a href="https://downloads.rclone.org/rclone-current-linux-arm64.zip">ARM - 64 Bit</a></li>
<li><a href="https://downloads.rclone.org/rclone-current-windows-amd64.zip">Intel/AMD - 64 Bit</a></li>
<li><a href="https://downloads.rclone.org/rclone-current-windows-386.zip">Intel/AMD - 32 Bit</a></li>
<li><a href="https://downloads.rclone.org/rclone-current-windows-arm64.zip">ARM - 64 Bit</a></li>
</ul>
<p>Open this file in the Explorer and extract <code>rclone.exe</code>. Rclone is a portable executable so you can place it wherever is convenient.</p>
<p>Open a CMD window (or powershell) and run the binary. Note that rclone does not launch a GUI by default, it runs in the CMD Window.</p>
@ -8081,13 +8081,13 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
</tr>
<tr class="odd">
<td>Oracle Object Storage</td>
<td style="text-align: center;">Yes</td>
<td style="text-align: center;">No</td>
<td style="text-align: center;">Yes</td>
<td style="text-align: center;">No</td>
<td style="text-align: center;">No</td>
<td style="text-align: center;">Yes</td>
<td style="text-align: center;">Yes</td>
<td style="text-align: center;">No</td>
<td style="text-align: center;">Yes</td>
<td style="text-align: center;">No</td>
<td style="text-align: center;">No</td>
<td style="text-align: center;">No</td>
@ -8471,7 +8471,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default &quot;rclone/v1.60.0&quot;)
--user-agent string Set the user-agent to a specified string (default &quot;rclone/v1.60.1&quot;)
-v, --verbose count Print lots more stuff (repeat for more)</code></pre>
<h2 id="backend-flags">Backend Flags</h2>
<p>These flags are available for every command. They control the backends and may be set in the config file.</p>
@ -8848,6 +8848,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
--s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don&#39;t attempt to check the bucket exists or create it
--s3-no-head If set, don&#39;t HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@ -12315,15 +12316,39 @@ $ rclone -q --s3-versions ls s3:cleanup-test
</ul></li>
<li>"s3.wasabisys.com"
<ul>
<li>Wasabi US East endpoint</li>
<li>Wasabi US East 1 (N. Virginia)</li>
</ul></li>
<li>"s3.us-east-2.wasabisys.com"
<ul>
<li>Wasabi US East 2 (N. Virginia)</li>
</ul></li>
<li>"s3.us-central-1.wasabisys.com"
<ul>
<li>Wasabi US Central 1 (Texas)</li>
</ul></li>
<li>"s3.us-west-1.wasabisys.com"
<ul>
<li>Wasabi US West endpoint</li>
<li>Wasabi US West 1 (Oregon)</li>
</ul></li>
<li>"s3.ca-central-1.wasabisys.com"
<ul>
<li>Wasabi CA Central 1 (Toronto)</li>
</ul></li>
<li>"s3.eu-central-1.wasabisys.com"
<ul>
<li>Wasabi EU Central endpoint</li>
<li>Wasabi EU Central 1 (Amsterdam)</li>
</ul></li>
<li>"s3.eu-central-2.wasabisys.com"
<ul>
<li>Wasabi EU Central 2 (Frankfurt)</li>
</ul></li>
<li>"s3.eu-west-1.wasabisys.com"
<ul>
<li>Wasabi EU West 1 (London)</li>
</ul></li>
<li>"s3.eu-west-2.wasabisys.com"
<ul>
<li>Wasabi EU West 2 (Paris)</li>
</ul></li>
<li>"s3.ap-northeast-1.wasabisys.com"
<ul>
@ -12333,6 +12358,14 @@ $ rclone -q --s3-versions ls s3:cleanup-test
<ul>
<li>Wasabi AP Northeast 2 (Osaka) endpoint</li>
</ul></li>
<li>"s3.ap-southeast-1.wasabisys.com"
<ul>
<li>Wasabi AP Southeast 1 (Singapore)</li>
</ul></li>
<li>"s3.ap-southeast-2.wasabisys.com"
<ul>
<li>Wasabi AP Southeast 2 (Sydney)</li>
</ul></li>
<li>"s3.ir-thr-at1.arvanstorage.com"
<ul>
<li>ArvanCloud Tehran Iran (Asiatech) endpoint</li>
@ -13673,6 +13706,21 @@ Windows: &quot;%USERPROFILE%\.aws\credentials&quot;</code></pre>
<li>Type: bool</li>
<li>Default: false</li>
</ul>
<h4 id="s3-might-gzip">--s3-might-gzip</h4>
<p>Set this if the backend might gzip objects.</p>
<p>Normally providers will not alter objects when they are downloaded. If an object was not uploaded with <code>Content-Encoding: gzip</code> then it won't be set on download.</p>
<p>However some providers may gzip objects even if they weren't uploaded with <code>Content-Encoding: gzip</code> (eg Cloudflare).</p>
<p>A symptom of this would be receiving errors like</p>
<pre><code>ERROR corrupted on transfer: sizes differ NNN vs MMM</code></pre>
<p>If you set this flag and rclone downloads an object with Content-Encoding: gzip set and chunked transfer encoding, then rclone will decompress the object on the fly.</p>
<p>If this is set to unset (the default) then rclone will choose according to the provider setting what to apply, but you can override rclone's choice here.</p>
<p>Properties:</p>
<ul>
<li>Config: might_gzip</li>
<li>Env Var: RCLONE_S3_MIGHT_GZIP</li>
<li>Type: Tristate</li>
<li>Default: unset</li>
</ul>
<h4 id="s3-no-system-metadata">--s3-no-system-metadata</h4>
<p>Suppress setting and reading of system metadata</p>
<p>Properties:</p>
@ -22458,7 +22506,17 @@ y/e/d&gt; y</code></pre>
<li>If a particular file is already present in storage, one can quickly submit file hash instead of long file upload (this optimization is supported by rclone)</li>
</ul>
<h2 id="configuration-24">Configuration</h2>
<p>Here is an example of making a mailru configuration. First create a Mail.ru Cloud account and choose a tariff, then run</p>
<p>Here is an example of making a mailru configuration.</p>
<p>First create a Mail.ru Cloud account and choose a tariff.</p>
<p>You will need to log in and create an app password for rclone. Rclone <strong>will not work</strong> with your normal username and password - it will give an error like <code>oauth2: server response missing access_token</code>.</p>
<ul>
<li>Click on your user icon in the top right</li>
<li>Go to Security / "Пароль и безопасность"</li>
<li>Click password for apps / "Пароли для внешних приложений"</li>
<li>Add the password - give it a name - eg "rclone"</li>
<li>Copy the password and use this password below - your normal login password won't work.</li>
</ul>
<p>Now run</p>
<pre><code>rclone config</code></pre>
<p>This will guide you through an interactive setup process:</p>
<pre><code>No remotes found, make a new one?
@ -22480,6 +22538,10 @@ User name (usually email)
Enter a string value. Press Enter for the default (&quot;&quot;).
user&gt; username@mail.ru
Password
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
y) Yes type in my own password
g) Generate random password
y/g&gt; y
@ -22598,6 +22660,7 @@ y/e/d&gt; y</code></pre>
</ul>
<h4 id="mailru-pass">--mailru-pass</h4>
<p>Password.</p>
<p>This must be an app password - rclone will not work with your normal password. See the Configuration section in the docs for how to make an app password.</p>
<p><strong>NB</strong> Input to this must be obscured - see <a href="https://rclone.org/commands/rclone_obscure/">rclone obscure</a>.</p>
<p>Properties:</p>
<ul>
@ -28967,6 +29030,55 @@ $ tree /tmp/b
<li>"error": return an error based on option value</li>
</ul>
<h1 id="changelog">Changelog</h1>
<h2 id="v1.60.1---2022-11-17">v1.60.1 - 2022-11-17</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1">See commits</a></p>
<ul>
<li>Bug Fixes
<ul>
<li>lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)</li>
<li>wasm: Fix walltime link error by adding up-to-date wasm_exec.js (João Henrique Franco)</li>
<li>docs
<ul>
<li>Update faq.md with bisync (Samuel Johnson)</li>
<li>Corrected download links in windows install docs (coultonluke)</li>
<li>Add direct download link for windows arm64 (albertony)</li>
<li>Remove link to rclone slack as it is no longer supported (Nick Craig-Wood)</li>
<li>Faq: how to use a proxy server that requires a username and password (asdffdsazqqq)</li>
<li>Oracle-object-storage: doc fix (Manoj Ghosh)</li>
<li>Fix typo <code>remove</code> in rclone_serve_restic command (Joda Stößer)</li>
<li>Fix character that was incorrectly interpreted as markdown (Clément Notin)</li>
</ul></li>
</ul></li>
<li>VFS
<ul>
<li>Fix deadlock caused by cache cleaner and upload finishing (Nick Craig-Wood)</li>
</ul></li>
<li>Local
<ul>
<li>Clean absolute paths (albertony)</li>
<li>Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)</li>
</ul></li>
<li>Mailru
<ul>
<li>Note that an app password is now needed (Nick Craig-Wood)</li>
<li>Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)</li>
</ul></li>
<li>S3
<ul>
<li>Add provider quirk <code>--s3-might-gzip</code> to fix corrupted on transfer: sizes differ (Nick Craig-Wood)</li>
<li>Allow Storj to server side copy since it seems to work now (Nick Craig-Wood)</li>
<li>Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)</li>
<li>Add additional Wasabi locations (techknowlogick)</li>
</ul></li>
<li>Smb
<ul>
<li>Fix <code>Failed to sync: context canceled</code> at the end of syncs (Nick Craig-Wood)</li>
</ul></li>
<li>WebDAV
<ul>
<li>Fix Move/Copy/DirMove when using -server-side-across-configs (Nick Craig-Wood)</li>
</ul></li>
</ul>
<h2 id="v1.60.0---2022-10-21">v1.60.0 - 2022-10-21</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0">See commits</a></p>
<ul>
@ -35170,7 +35282,7 @@ Server B&gt; rclone copy /tmp/whatever remote:Backup</code></pre>
<p>It would be possible to make a sync system which stored binary diffs instead of whole objects like rclone does, but that would break the 1:1 mapping of files on your hard disk to objects in the remote cloud storage system.</p>
<p>All the cloud storage systems support partial downloads of content, so it would be possible to make partial downloads work. However to make this work efficiently this would require storing a significant amount of metadata, which breaks the desired 1:1 mapping of files to objects.</p>
<h3 id="can-rclone-do-bi-directional-sync">Can rclone do bi-directional sync?</h3>
<p>No, not at present. rclone only does uni-directional sync from A -&gt; B. It may do in the future though since it has all the primitives - it just requires writing the algorithm to do it.</p>
<p>Yes, since rclone v1.58.0, <a href="https://rclone.org/bisync/">bidirectional cloud sync</a> is available.</p>
<h3 id="can-i-use-rclone-with-an-http-proxy">Can I use rclone with an HTTP proxy?</h3>
<p>Yes. rclone will follow the standard environment variables for proxies, similar to cURL and other programs.</p>
<p>In general the variables are called <code>http_proxy</code> (for services reached over <code>http</code>) and <code>https_proxy</code> (for services reached over <code>https</code>). Most public services will be using <code>https</code>, but you may wish to set both.</p>
@ -35180,6 +35292,11 @@ Server B&gt; rclone copy /tmp/whatever remote:Backup</code></pre>
export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy</code></pre>
<p>Note: If the proxy server requires a username and password, then use</p>
<pre><code>export http_proxy=http://username:password@proxyserver:12345
export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy</code></pre>
<p>The <code>NO_PROXY</code> allows you to disable the proxy for specific hosts. Hosts must be comma separated, and can contain domains or parts. For instance "foo.com" also matches "bar.foo.com".</p>
<p>e.g.</p>
<pre><code>export no_proxy=localhost,127.0.0.0/8,my.host.name

139
MANUAL.md generated
View file

@ -1,6 +1,6 @@
% rclone(1) User Manual
% Nick Craig-Wood
% Oct 21, 2022
% Nov 17, 2022
# Rclone syncs your files to cloud storage
@ -315,9 +315,9 @@ The simplest fix is to run
Fetch the correct binary for your processor type by clicking on these
links. If not sure, use the first link.
- [Intel/AMD - 64 Bit](https://downloads.rclone.org/rclone-current-linux-amd64.zip)
- [Intel/AMD - 32 Bit](https://downloads.rclone.org/rclone-current-linux-386.zip)
- [ARM - 64 Bit](https://downloads.rclone.org/rclone-current-linux-arm64.zip)
- [Intel/AMD - 64 Bit](https://downloads.rclone.org/rclone-current-windows-amd64.zip)
- [Intel/AMD - 32 Bit](https://downloads.rclone.org/rclone-current-windows-386.zip)
- [ARM - 64 Bit](https://downloads.rclone.org/rclone-current-windows-arm64.zip)
Open this file in the Explorer and extract `rclone.exe`. Rclone is a
portable executable so you can place it wherever is convenient.
@ -14241,7 +14241,7 @@ upon backend-specific capabilities.
| Microsoft OneDrive | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes |
| OpenDrive | Yes | Yes | Yes | Yes | No | No | No | No | No | Yes |
| OpenStack Swift | Yes † | Yes | No | No | No | Yes | Yes | No | Yes | No |
| Oracle Object Storage | Yes | Yes | No | No | Yes | Yes | No | No | No | No |
| Oracle Object Storage | No | Yes | No | No | Yes | Yes | Yes | No | No | No |
| pCloud | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes |
| premiumize.me | Yes | No | Yes | Yes | No | No | No | Yes | Yes | Yes |
| put.io | Yes | No | Yes | Yes | Yes | No | Yes | No | Yes | Yes |
@ -14501,7 +14501,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default "rclone/v1.60.0")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
-v, --verbose count Print lots more stuff (repeat for more)
```
@ -14884,6 +14884,7 @@ and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
--s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
--s3-no-head If set, don't HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@ -18940,15 +18941,31 @@ Properties:
- "s3.ap-southeast-1.lyvecloud.seagate.com"
- Seagate Lyve Cloud AP Southeast 1 (Singapore)
- "s3.wasabisys.com"
- Wasabi US East endpoint
- Wasabi US East 1 (N. Virginia)
- "s3.us-east-2.wasabisys.com"
- Wasabi US East 2 (N. Virginia)
- "s3.us-central-1.wasabisys.com"
- Wasabi US Central 1 (Texas)
- "s3.us-west-1.wasabisys.com"
- Wasabi US West endpoint
- Wasabi US West 1 (Oregon)
- "s3.ca-central-1.wasabisys.com"
- Wasabi CA Central 1 (Toronto)
- "s3.eu-central-1.wasabisys.com"
- Wasabi EU Central endpoint
- Wasabi EU Central 1 (Amsterdam)
- "s3.eu-central-2.wasabisys.com"
- Wasabi EU Central 2 (Frankfurt)
- "s3.eu-west-1.wasabisys.com"
- Wasabi EU West 1 (London)
- "s3.eu-west-2.wasabisys.com"
- Wasabi EU West 2 (Paris)
- "s3.ap-northeast-1.wasabisys.com"
- Wasabi AP Northeast 1 (Tokyo) endpoint
- "s3.ap-northeast-2.wasabisys.com"
- Wasabi AP Northeast 2 (Osaka) endpoint
- "s3.ap-southeast-1.wasabisys.com"
- Wasabi AP Southeast 1 (Singapore)
- "s3.ap-southeast-2.wasabisys.com"
- Wasabi AP Southeast 2 (Sydney)
- "s3.ir-thr-at1.arvanstorage.com"
- ArvanCloud Tehran Iran (Asiatech) endpoint
@ -20167,6 +20184,37 @@ Properties:
- Type: bool
- Default: false
#### --s3-might-gzip
Set this if the backend might gzip objects.
Normally providers will not alter objects when they are downloaded. If
an object was not uploaded with `Content-Encoding: gzip` then it won't
be set on download.
However some providers may gzip objects even if they weren't uploaded
with `Content-Encoding: gzip` (eg Cloudflare).
A symptom of this would be receiving errors like
ERROR corrupted on transfer: sizes differ NNN vs MMM
If you set this flag and rclone downloads an object with
Content-Encoding: gzip set and chunked transfer encoding, then rclone
will decompress the object on the fly.
If this is set to unset (the default) then rclone will choose
according to the provider setting what to apply, but you can override
rclone's choice here.
Properties:
- Config: might_gzip
- Env Var: RCLONE_S3_MIGHT_GZIP
- Type: Tristate
- Default: unset
#### --s3-no-system-metadata
Suppress setting and reading of system metadata
@ -32271,8 +32319,21 @@ Currently it is recommended to disable 2FA on Mail.ru accounts intended for rclo
## Configuration
Here is an example of making a mailru configuration. First create a Mail.ru Cloud
account and choose a tariff, then run
Here is an example of making a mailru configuration.
First create a Mail.ru Cloud account and choose a tariff.
You will need to log in and create an app password for rclone. Rclone
**will not work** with your normal username and password - it will
give an error like `oauth2: server response missing access_token`.
- Click on your user icon in the top right
- Go to Security / "Пароль и безопасность"
- Click password for apps / "Пароли для внешних приложений"
- Add the password - give it a name - eg "rclone"
- Copy the password and use this password below - your normal login password won't work.
Now run
rclone config
@ -32298,6 +32359,10 @@ User name (usually email)
Enter a string value. Press Enter for the default ("").
user> username@mail.ru
Password
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@ -32420,6 +32485,11 @@ Properties:
Password.
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
**NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/).
Properties:
@ -40954,6 +41024,40 @@ Options:
# Changelog
## v1.60.1 - 2022-11-17
[See commits](https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1)
* Bug Fixes
* lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)
* wasm: Fix walltime link error by adding up-to-date wasm_exec.js (João Henrique Franco)
* docs
* Update faq.md with bisync (Samuel Johnson)
* Corrected download links in windows install docs (coultonluke)
* Add direct download link for windows arm64 (albertony)
* Remove link to rclone slack as it is no longer supported (Nick Craig-Wood)
* Faq: how to use a proxy server that requires a username and password (asdffdsazqqq)
* Oracle-object-storage: doc fix (Manoj Ghosh)
* Fix typo `remove` in rclone_serve_restic command (Joda Stößer)
* Fix character that was incorrectly interpreted as markdown (Clément Notin)
* VFS
* Fix deadlock caused by cache cleaner and upload finishing (Nick Craig-Wood)
* Local
* Clean absolute paths (albertony)
* Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)
* Mailru
* Note that an app password is now needed (Nick Craig-Wood)
* Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)
* S3
* Add provider quirk `--s3-might-gzip` to fix corrupted on transfer: sizes differ (Nick Craig-Wood)
* Allow Storj to server side copy since it seems to work now (Nick Craig-Wood)
* Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
* Add additional Wasabi locations (techknowlogick)
* Smb
* Fix `Failed to sync: context canceled` at the end of syncs (Nick Craig-Wood)
* WebDAV
* Fix Move/Copy/DirMove when using -server-side-across-configs (Nick Craig-Wood)
## v1.60.0 - 2022-10-21
[See commits](https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0)
@ -45237,9 +45341,8 @@ of metadata, which breaks the desired 1:1 mapping of files to objects.
### Can rclone do bi-directional sync? ###
No, not at present. rclone only does uni-directional sync from A ->
B. It may do in the future though since it has all the primitives - it
just requires writing the algorithm to do it.
Yes, since rclone v1.58.0, [bidirectional cloud sync](https://rclone.org/bisync/) is
available.
### Can I use rclone with an HTTP proxy? ###
@ -45264,6 +45367,14 @@ possibilities. So, on Linux, you may end up with code similar to
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
Note: If the proxy server requires a username and password, then use
export http_proxy=http://username:password@proxyserver:12345
export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
The `NO_PROXY` allows you to disable the proxy for specific hosts.
Hosts must be comma separated, and can contain domains or parts.
For instance "foo.com" also matches "bar.foo.com".

143
MANUAL.txt generated
View file

@ -1,6 +1,6 @@
rclone(1) User Manual
Nick Craig-Wood
Oct 21, 2022
Nov 17, 2022
Rclone syncs your files to cloud storage
@ -13781,7 +13781,7 @@ upon backend-specific capabilities.
Microsoft OneDrive Yes Yes Yes Yes Yes No No Yes Yes Yes
OpenDrive Yes Yes Yes Yes No No No No No Yes
OpenStack Swift Yes † Yes No No No Yes Yes No Yes No
Oracle Object Storage Yes Yes No No Yes Yes No No No No
Oracle Object Storage No Yes No No Yes Yes Yes No No No
pCloud Yes Yes Yes Yes Yes No No Yes Yes Yes
premiumize.me Yes No Yes Yes No No No Yes Yes Yes
put.io Yes No Yes Yes Yes No Yes No Yes Yes
@ -14038,7 +14038,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default "rclone/v1.60.0")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
-v, --verbose count Print lots more stuff (repeat for more)
Backend Flags
@ -14419,6 +14419,7 @@ and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
--s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
--s3-no-head If set, don't HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@ -18425,15 +18426,31 @@ Properties:
- "s3.ap-southeast-1.lyvecloud.seagate.com"
- Seagate Lyve Cloud AP Southeast 1 (Singapore)
- "s3.wasabisys.com"
- Wasabi US East endpoint
- Wasabi US East 1 (N. Virginia)
- "s3.us-east-2.wasabisys.com"
- Wasabi US East 2 (N. Virginia)
- "s3.us-central-1.wasabisys.com"
- Wasabi US Central 1 (Texas)
- "s3.us-west-1.wasabisys.com"
- Wasabi US West endpoint
- Wasabi US West 1 (Oregon)
- "s3.ca-central-1.wasabisys.com"
- Wasabi CA Central 1 (Toronto)
- "s3.eu-central-1.wasabisys.com"
- Wasabi EU Central endpoint
- Wasabi EU Central 1 (Amsterdam)
- "s3.eu-central-2.wasabisys.com"
- Wasabi EU Central 2 (Frankfurt)
- "s3.eu-west-1.wasabisys.com"
- Wasabi EU West 1 (London)
- "s3.eu-west-2.wasabisys.com"
- Wasabi EU West 2 (Paris)
- "s3.ap-northeast-1.wasabisys.com"
- Wasabi AP Northeast 1 (Tokyo) endpoint
- "s3.ap-northeast-2.wasabisys.com"
- Wasabi AP Northeast 2 (Osaka) endpoint
- "s3.ap-southeast-1.wasabisys.com"
- Wasabi AP Southeast 1 (Singapore)
- "s3.ap-southeast-2.wasabisys.com"
- Wasabi AP Southeast 2 (Sydney)
- "s3.ir-thr-at1.arvanstorage.com"
- ArvanCloud Tehran Iran (Asiatech) endpoint
@ -19665,6 +19682,36 @@ Properties:
- Type: bool
- Default: false
--s3-might-gzip
Set this if the backend might gzip objects.
Normally providers will not alter objects when they are downloaded. If
an object was not uploaded with Content-Encoding: gzip then it won't be
set on download.
However some providers may gzip objects even if they weren't uploaded
with Content-Encoding: gzip (eg Cloudflare).
A symptom of this would be receiving errors like
ERROR corrupted on transfer: sizes differ NNN vs MMM
If you set this flag and rclone downloads an object with
Content-Encoding: gzip set and chunked transfer encoding, then rclone
will decompress the object on the fly.
If this is set to unset (the default) then rclone will choose according
to the provider setting what to apply, but you can override rclone's
choice here.
Properties:
- Config: might_gzip
- Env Var: RCLONE_S3_MIGHT_GZIP
- Type: Tristate
- Default: unset
--s3-no-system-metadata
Suppress setting and reading of system metadata
@ -31679,8 +31726,22 @@ Features highlights
Configuration
Here is an example of making a mailru configuration. First create a
Mail.ru Cloud account and choose a tariff, then run
Here is an example of making a mailru configuration.
First create a Mail.ru Cloud account and choose a tariff.
You will need to log in and create an app password for rclone. Rclone
will not work with your normal username and password - it will give an
error like oauth2: server response missing access_token.
- Click on your user icon in the top right
- Go to Security / "Пароль и безопасность"
- Click password for apps / "Пароли для внешних приложений"
- Add the password - give it a name - eg "rclone"
- Copy the password and use this password below - your normal login
password won't work.
Now run
rclone config
@ -31705,6 +31766,10 @@ This will guide you through an interactive setup process:
Enter a string value. Press Enter for the default ("").
user> username@mail.ru
Password
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@ -31826,6 +31891,10 @@ Properties:
Password.
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
NB Input to this must be obscured - see rclone obscure.
Properties:
@ -40428,6 +40497,53 @@ Options:
Changelog
v1.60.1 - 2022-11-17
See commits
- Bug Fixes
- lib/cache: Fix alias backend shutting down too soon (Nick
Craig-Wood)
- wasm: Fix walltime link error by adding up-to-date wasm_exec.js
(João Henrique Franco)
- docs
- Update faq.md with bisync (Samuel Johnson)
- Corrected download links in windows install docs
(coultonluke)
- Add direct download link for windows arm64 (albertony)
- Remove link to rclone slack as it is no longer supported
(Nick Craig-Wood)
- Faq: how to use a proxy server that requires a username and
password (asdffdsazqqq)
- Oracle-object-storage: doc fix (Manoj Ghosh)
- Fix typo remove in rclone_serve_restic command (Joda Stößer)
- Fix character that was incorrectly interpreted as markdown
(Clément Notin)
- VFS
- Fix deadlock caused by cache cleaner and upload finishing (Nick
Craig-Wood)
- Local
- Clean absolute paths (albertony)
- Fix -L/--copy-links with filters missing directories (Nick
Craig-Wood)
- Mailru
- Note that an app password is now needed (Nick Craig-Wood)
- Allow timestamps to be before the epoch 1970-01-01 (Nick
Craig-Wood)
- S3
- Add provider quirk --s3-might-gzip to fix corrupted on transfer:
sizes differ (Nick Craig-Wood)
- Allow Storj to server side copy since it seems to work now (Nick
Craig-Wood)
- Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
- Add additional Wasabi locations (techknowlogick)
- Smb
- Fix Failed to sync: context canceled at the end of syncs (Nick
Craig-Wood)
- WebDAV
- Fix Move/Copy/DirMove when using -server-side-across-configs
(Nick Craig-Wood)
v1.60.0 - 2022-10-21
See commits
@ -45966,9 +46082,7 @@ metadata, which breaks the desired 1:1 mapping of files to objects.
Can rclone do bi-directional sync?
No, not at present. rclone only does uni-directional sync from A -> B.
It may do in the future though since it has all the primitives - it just
requires writing the algorithm to do it.
Yes, since rclone v1.58.0, bidirectional cloud sync is available.
Can I use rclone with an HTTP proxy?
@ -45993,6 +46107,13 @@ set all possibilities. So, on Linux, you may end up with code similar to
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
Note: If the proxy server requires a username and password, then use
export http_proxy=http://username:password@proxyserver:12345
export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
The NO_PROXY allows you to disable the proxy for specific hosts. Hosts
must be comma separated, and can contain domains or parts. For instance
"foo.com" also matches "bar.foo.com".

View file

@ -1 +1 @@
v1.60.0
v1.60.1

View file

@ -521,11 +521,6 @@ func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err e
name := fi.Name()
mode := fi.Mode()
newRemote := f.cleanRemote(dir, name)
// Don't include non directory if not included
// we leave directory filtering to the layer above
if useFilter && !fi.IsDir() && !filter.IncludeRemote(newRemote) {
continue
}
// Follow symlinks if required
if f.opt.FollowSymlinks && (mode&os.ModeSymlink) != 0 {
localPath := filepath.Join(fsDirPath, name)
@ -542,6 +537,11 @@ func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err e
}
mode = fi.Mode()
}
// Don't include non directory if not included
// we leave directory filtering to the layer above
if useFilter && !fi.IsDir() && !filter.IncludeRemote(newRemote) {
continue
}
if fi.IsDir() {
// Ignore directories which are symlinks. These are junction points under windows which
// are kind of a souped up symlink. Unix doesn't have directories which are symlinks.
@ -1400,30 +1400,27 @@ func (o *Object) writeMetadata(metadata fs.Metadata) (err error) {
}
func cleanRootPath(s string, noUNC bool, enc encoder.MultiEncoder) string {
if runtime.GOOS == "windows" {
if !filepath.IsAbs(s) && !strings.HasPrefix(s, "\\") {
if runtime.GOOS != "windows" || !strings.HasPrefix(s, "\\") {
if !filepath.IsAbs(s) {
s2, err := filepath.Abs(s)
if err == nil {
s = s2
}
} else {
s = filepath.Clean(s)
}
}
if runtime.GOOS == "windows" {
s = filepath.ToSlash(s)
vol := filepath.VolumeName(s)
s = vol + enc.FromStandardPath(s[len(vol):])
s = filepath.FromSlash(s)
if !noUNC {
// Convert to UNC
s = file.UNCPath(s)
}
return s
}
if !filepath.IsAbs(s) {
s2, err := filepath.Abs(s)
if err == nil {
s = s2
}
}
s = enc.FromStandardPath(s)
return s
}

View file

@ -69,6 +69,11 @@ func (w *BinWriter) WritePu64(val int64) {
w.b.Write(w.a[:binary.PutUvarint(w.a, uint64(val))])
}
// WriteP64 writes an signed long as unsigned varint
func (w *BinWriter) WriteP64(val int64) {
w.b.Write(w.a[:binary.PutUvarint(w.a, uint64(val))])
}
// WriteString writes a zero-terminated string
func (w *BinWriter) WriteString(str string) {
buf := []byte(str)

View file

@ -91,8 +91,13 @@ func init() {
Help: "User name (usually email).",
Required: true,
}, {
Name: "pass",
Help: "Password.",
Name: "pass",
Help: `Password.
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
`,
Required: true,
IsPassword: true,
}, {
@ -641,12 +646,7 @@ func (f *Fs) itemToDirEntry(ctx context.Context, item *api.ListItem) (entry fs.D
return nil, -1, err
}
mTime := int64(item.Mtime)
if mTime < 0 {
fs.Debugf(f, "Fixing invalid timestamp %d on mailru file %q", mTime, remote)
mTime = 0
}
modTime := time.Unix(mTime, 0)
modTime := time.Unix(int64(item.Mtime), 0)
isDir, err := f.isDir(item.Kind, remote)
if err != nil {
@ -2058,7 +2058,7 @@ func (o *Object) addFileMetaData(ctx context.Context, overwrite bool) error {
req.WritePu16(0) // revision
req.WriteString(o.fs.opt.Enc.FromStandardPath(o.absPath()))
req.WritePu64(o.size)
req.WritePu64(o.modTime.Unix())
req.WriteP64(o.modTime.Unix())
req.WritePu32(0)
req.Write(o.mrHash)

View file

@ -686,6 +686,7 @@ var (
_ fs.PutStreamer = &Fs{}
_ fs.ListRer = &Fs{}
_ fs.Commander = &Fs{}
_ fs.CleanUpper = &Fs{}
_ fs.Object = &Object{}
_ fs.MimeTyper = &Object{}

View file

@ -1126,15 +1126,39 @@ func init() {
Provider: "LyveCloud",
}, {
Value: "s3.wasabisys.com",
Help: "Wasabi US East endpoint",
Help: "Wasabi US East 1 (N. Virginia)",
Provider: "Wasabi",
}, {
Value: "s3.us-east-2.wasabisys.com",
Help: "Wasabi US East 2 (N. Virginia)",
Provider: "Wasabi",
}, {
Value: "s3.us-central-1.wasabisys.com",
Help: "Wasabi US Central 1 (Texas)",
Provider: "Wasabi",
}, {
Value: "s3.us-west-1.wasabisys.com",
Help: "Wasabi US West endpoint",
Help: "Wasabi US West 1 (Oregon)",
Provider: "Wasabi",
}, {
Value: "s3.ca-central-1.wasabisys.com",
Help: "Wasabi CA Central 1 (Toronto)",
Provider: "Wasabi",
}, {
Value: "s3.eu-central-1.wasabisys.com",
Help: "Wasabi EU Central endpoint",
Help: "Wasabi EU Central 1 (Amsterdam)",
Provider: "Wasabi",
}, {
Value: "s3.eu-central-2.wasabisys.com",
Help: "Wasabi EU Central 2 (Frankfurt)",
Provider: "Wasabi",
}, {
Value: "s3.eu-west-1.wasabisys.com",
Help: "Wasabi EU West 1 (London)",
Provider: "Wasabi",
}, {
Value: "s3.eu-west-2.wasabisys.com",
Help: "Wasabi EU West 2 (Paris)",
Provider: "Wasabi",
}, {
Value: "s3.ap-northeast-1.wasabisys.com",
@ -1144,6 +1168,14 @@ func init() {
Value: "s3.ap-northeast-2.wasabisys.com",
Help: "Wasabi AP Northeast 2 (Osaka) endpoint",
Provider: "Wasabi",
}, {
Value: "s3.ap-southeast-1.wasabisys.com",
Help: "Wasabi AP Southeast 1 (Singapore)",
Provider: "Wasabi",
}, {
Value: "s3.ap-southeast-2.wasabisys.com",
Help: "Wasabi AP Southeast 2 (Sydney)",
Provider: "Wasabi",
}, {
Value: "s3.ir-thr-at1.arvanstorage.com",
Help: "ArvanCloud Tehran Iran (Asiatech) endpoint",
@ -2164,6 +2196,31 @@ can't check the size and hash but the file contents will be decompressed.
`,
Advanced: true,
Default: false,
}, {
Name: "might_gzip",
Help: strings.ReplaceAll(`Set this if the backend might gzip objects.
Normally providers will not alter objects when they are downloaded. If
an object was not uploaded with |Content-Encoding: gzip| then it won't
be set on download.
However some providers may gzip objects even if they weren't uploaded
with |Content-Encoding: gzip| (eg Cloudflare).
A symptom of this would be receiving errors like
ERROR corrupted on transfer: sizes differ NNN vs MMM
If you set this flag and rclone downloads an object with
Content-Encoding: gzip set and chunked transfer encoding, then rclone
will decompress the object on the fly.
If this is set to unset (the default) then rclone will choose
according to the provider setting what to apply, but you can override
rclone's choice here.
`, "|", "`"),
Default: fs.Tristate{},
Advanced: true,
}, {
Name: "no_system_metadata",
Help: `Suppress setting and reading of system metadata`,
@ -2295,6 +2352,7 @@ type Options struct {
Versions bool `config:"versions"`
VersionAt fs.Time `config:"version_at"`
Decompress bool `config:"decompress"`
MightGzip fs.Tristate `config:"might_gzip"`
NoSystemMetadata bool `config:"no_system_metadata"`
}
@ -2655,10 +2713,12 @@ func setQuirks(opt *Options) {
virtualHostStyle = true
urlEncodeListings = true
useMultipartEtag = true
mightGzip = true // assume all providers might gzip until proven otherwise
)
switch opt.Provider {
case "AWS":
// No quirks
mightGzip = false // Never auto gzips objects
case "Alibaba":
useMultipartEtag = false // Alibaba seems to calculate multipart Etags differently from AWS
case "HuaweiOBS":
@ -2772,6 +2832,12 @@ func setQuirks(opt *Options) {
opt.UseMultipartEtag.Valid = true
opt.UseMultipartEtag.Value = useMultipartEtag
}
// set MightGzip if not manually set
if !opt.MightGzip.Valid {
opt.MightGzip.Valid = true
opt.MightGzip.Value = mightGzip
}
}
// setRoot changes the root of the Fs
@ -2889,7 +2955,6 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
return f, fs.ErrorIsFile
}
if opt.Provider == "Storj" {
f.features.Copy = nil
f.features.SetTier = false
f.features.GetTier = false
}
@ -3118,8 +3183,11 @@ func (f *Fs) newV2List(req *s3.ListObjectsV2Input) bucketLister {
// Do a V2 listing
func (ls *v2List) List(ctx context.Context) (resp *s3.ListObjectsV2Output, versionIDs []*string, err error) {
resp, err = ls.f.c.ListObjectsV2WithContext(ctx, &ls.req)
if err != nil {
return nil, nil, err
}
ls.req.ContinuationToken = resp.NextContinuationToken
return resp, nil, err
return resp, nil, nil
}
// URL Encode the listings
@ -4858,7 +4926,7 @@ func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.Read
// Decompress body if necessary
if aws.StringValue(resp.ContentEncoding) == "gzip" {
if o.fs.opt.Decompress {
if o.fs.opt.Decompress || (resp.ContentLength == nil && o.fs.opt.MightGzip.Value) {
return readers.NewGzipReader(resp.Body)
}
o.fs.warnCompressed.Do(func() {

View file

@ -103,6 +103,10 @@ func (f *Fs) getSessions() int32 {
// Open a new connection to the SMB server.
func (f *Fs) newConnection(ctx context.Context, share string) (c *conn, err error) {
// As we are pooling these connections we need to decouple
// them from the current context
ctx = context.Background()
c, err = f.dial(ctx, "tcp", f.opt.Host+":"+f.opt.Port)
if err != nil {
return nil, fmt.Errorf("couldn't connect SMB: %w", err)

View file

@ -991,6 +991,7 @@ func (f *Fs) copyOrMove(ctx context.Context, src fs.Object, remote string, metho
}
return nil, fs.ErrorCantMove
}
srcFs := srcObj.fs
dstPath := f.filePath(remote)
err := f.mkParentDir(ctx, dstPath)
if err != nil {
@ -1013,9 +1014,10 @@ func (f *Fs) copyOrMove(ctx context.Context, src fs.Object, remote string, metho
if f.useOCMtime {
opts.ExtraHeaders["X-OC-Mtime"] = fmt.Sprintf("%d", src.ModTime(ctx).Unix())
}
err = f.pacer.Call(func() (bool, error) {
resp, err = f.srv.Call(ctx, &opts)
return f.shouldRetry(ctx, resp, err)
// Direct the MOVE/COPY to the source server
err = srcFs.pacer.Call(func() (bool, error) {
resp, err = srcFs.srv.Call(ctx, &opts)
return srcFs.shouldRetry(ctx, resp, err)
})
if err != nil {
return nil, fmt.Errorf("Copy call failed: %w", err)
@ -1109,9 +1111,10 @@ func (f *Fs) DirMove(ctx context.Context, src fs.Fs, srcRemote, dstRemote string
"Overwrite": "F",
},
}
err = f.pacer.Call(func() (bool, error) {
resp, err = f.srv.Call(ctx, &opts)
return f.shouldRetry(ctx, resp, err)
// Direct the MOVE/COPY to the source server
err = srcFs.pacer.Call(func() (bool, error) {
resp, err = srcFs.srv.Call(ctx, &opts)
return srcFs.shouldRetry(ctx, resp, err)
})
if err != nil {
return fmt.Errorf("DirMove MOVE call failed: %w", err)

View file

@ -5,6 +5,40 @@ description: "Rclone Changelog"
# Changelog
## v1.60.1 - 2022-11-17
[See commits](https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1)
* Bug Fixes
* lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)
* wasm: Fix walltime link error by adding up-to-date wasm_exec.js (João Henrique Franco)
* docs
* Update faq.md with bisync (Samuel Johnson)
* Corrected download links in windows install docs (coultonluke)
* Add direct download link for windows arm64 (albertony)
* Remove link to rclone slack as it is no longer supported (Nick Craig-Wood)
* Faq: how to use a proxy server that requires a username and password (asdffdsazqqq)
* Oracle-object-storage: doc fix (Manoj Ghosh)
* Fix typo `remove` in rclone_serve_restic command (Joda Stößer)
* Fix character that was incorrectly interpreted as markdown (Clément Notin)
* VFS
* Fix deadlock caused by cache cleaner and upload finishing (Nick Craig-Wood)
* Local
* Clean absolute paths (albertony)
* Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)
* Mailru
* Note that an app password is now needed (Nick Craig-Wood)
* Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)
* S3
* Add provider quirk `--s3-might-gzip` to fix corrupted on transfer: sizes differ (Nick Craig-Wood)
* Allow Storj to server side copy since it seems to work now (Nick Craig-Wood)
* Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
* Add additional Wasabi locations (techknowlogick)
* Smb
* Fix `Failed to sync: context canceled` at the end of syncs (Nick Craig-Wood)
* WebDAV
* Fix Move/Copy/DirMove when using -server-side-across-configs (Nick Craig-Wood)
## v1.60.0 - 2022-10-21
[See commits](https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0)

View file

@ -85,7 +85,7 @@ script) from a URL which doesn't change then you can use these links.
| Intel/AMD - 32 Bit | {{< cdownload windows 386 >}} | - | {{< cdownload linux 386 >}} | {{< cdownload linux 386 deb >}} | {{< cdownload linux 386 rpm >}} | {{< cdownload freebsd 386 >}} | {{< cdownload netbsd 386 >}} | {{< cdownload openbsd 386 >}} | {{< cdownload plan9 386 >}} | - |
| ARMv6 - 32 Bit | - | - | {{< cdownload linux arm >}} | {{< cdownload linux arm deb >}} | {{< cdownload linux arm rpm >}} | {{< cdownload freebsd arm >}} | {{< cdownload netbsd arm >}} | - | - | - |
| ARMv7 - 32 Bit | - | - | {{< cdownload linux arm-v7 >}} | {{< cdownload linux arm-v7 deb >}} | {{< cdownload linux arm-v7 rpm >}} | {{< cdownload freebsd arm-v7 >}} | {{< cdownload netbsd arm-v7 >}} | - | - | - |
| ARM - 64 Bit | - | {{< cdownload osx arm64 >}} | {{< cdownload linux arm64 >}} | {{< cdownload linux arm64 deb >}} | {{< cdownload linux arm64 rpm >}} | - | - | - | - | - |
| ARM - 64 Bit | {{< cdownload windows arm64 >}} | {{< cdownload osx arm64 >}} | {{< cdownload linux arm64 >}} | {{< cdownload linux arm64 deb >}} | {{< cdownload linux arm64 rpm >}} | - | - | - | - | - |
| MIPS - Big Endian | - | - | {{< cdownload linux mips >}} | {{< cdownload linux mips deb >}} | {{< cdownload linux mips rpm >}} | - | - | - | - | - |
| MIPS - Little Endian | - | - | {{< cdownload linux mipsle >}} | {{< cdownload linux mipsle deb >}} | {{< cdownload linux mipsle rpm >}} | - | - | - | - | - |

View file

@ -82,9 +82,8 @@ of metadata, which breaks the desired 1:1 mapping of files to objects.
### Can rclone do bi-directional sync? ###
No, not at present. rclone only does uni-directional sync from A ->
B. It may do in the future though since it has all the primitives - it
just requires writing the algorithm to do it.
Yes, since rclone v1.58.0, [bidirectional cloud sync](/bisync/) is
available.
### Can I use rclone with an HTTP proxy? ###
@ -109,6 +108,14 @@ possibilities. So, on Linux, you may end up with code similar to
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
Note: If the proxy server requires a username and password, then use
export http_proxy=http://username:password@proxyserver:12345
export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
The `NO_PROXY` allows you to disable the proxy for specific hosts.
Hosts must be comma separated, and can contain domains or parts.
For instance "foo.com" also matches "bar.foo.com".

View file

@ -162,7 +162,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default "rclone/v1.60.0")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
-v, --verbose count Print lots more stuff (repeat for more)
```
@ -545,6 +545,7 @@ and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
--s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
--s3-no-head If set, don't HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects

View file

@ -125,9 +125,9 @@ The simplest fix is to run
Fetch the correct binary for your processor type by clicking on these
links. If not sure, use the first link.
- [Intel/AMD - 64 Bit](https://downloads.rclone.org/rclone-current-linux-amd64.zip)
- [Intel/AMD - 32 Bit](https://downloads.rclone.org/rclone-current-linux-386.zip)
- [ARM - 64 Bit](https://downloads.rclone.org/rclone-current-linux-arm64.zip)
- [Intel/AMD - 64 Bit](https://downloads.rclone.org/rclone-current-windows-amd64.zip)
- [Intel/AMD - 32 Bit](https://downloads.rclone.org/rclone-current-windows-386.zip)
- [ARM - 64 Bit](https://downloads.rclone.org/rclone-current-windows-arm64.zip)
Open this file in the Explorer and extract `rclone.exe`. Rclone is a
portable executable so you can place it wherever is convenient.

View file

@ -24,8 +24,21 @@ Currently it is recommended to disable 2FA on Mail.ru accounts intended for rclo
## Configuration
Here is an example of making a mailru configuration. First create a Mail.ru Cloud
account and choose a tariff, then run
Here is an example of making a mailru configuration.
First create a Mail.ru Cloud account and choose a tariff.
You will need to log in and create an app password for rclone. Rclone
**will not work** with your normal username and password - it will
give an error like `oauth2: server response missing access_token`.
- Click on your user icon in the top right
- Go to Security / "Пароль и безопасность"
- Click password for apps / "Пароли для внешних приложений"
- Add the password - give it a name - eg "rclone"
- Copy the password and use this password below - your normal login password won't work.
Now run
rclone config
@ -51,6 +64,10 @@ User name (usually email)
Enter a string value. Press Enter for the default ("").
user> username@mail.ru
Password
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@ -173,6 +190,11 @@ Properties:
Password.
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
**NB** Input to this must be obscured - see [rclone obscure](/commands/rclone_obscure/).
Properties:

View file

@ -494,7 +494,7 @@ upon backend-specific capabilities.
| Microsoft OneDrive | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes |
| OpenDrive | Yes | Yes | Yes | Yes | No | No | No | No | No | Yes |
| OpenStack Swift | Yes † | Yes | No | No | No | Yes | Yes | No | Yes | No |
| Oracle Object Storage | Yes | Yes | No | No | Yes | Yes | No | No | No | No |
| Oracle Object Storage | No | Yes | No | No | Yes | Yes | Yes | No | No | No |
| pCloud | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes |
| premiumize.me | Yes | No | Yes | Yes | No | No | No | Yes | Yes | Yes |
| put.io | Yes | No | Yes | Yes | Yes | No | Yes | No | Yes | Yes |

View file

@ -1631,15 +1631,31 @@ Properties:
- "s3.ap-southeast-1.lyvecloud.seagate.com"
- Seagate Lyve Cloud AP Southeast 1 (Singapore)
- "s3.wasabisys.com"
- Wasabi US East endpoint
- Wasabi US East 1 (N. Virginia)
- "s3.us-east-2.wasabisys.com"
- Wasabi US East 2 (N. Virginia)
- "s3.us-central-1.wasabisys.com"
- Wasabi US Central 1 (Texas)
- "s3.us-west-1.wasabisys.com"
- Wasabi US West endpoint
- Wasabi US West 1 (Oregon)
- "s3.ca-central-1.wasabisys.com"
- Wasabi CA Central 1 (Toronto)
- "s3.eu-central-1.wasabisys.com"
- Wasabi EU Central endpoint
- Wasabi EU Central 1 (Amsterdam)
- "s3.eu-central-2.wasabisys.com"
- Wasabi EU Central 2 (Frankfurt)
- "s3.eu-west-1.wasabisys.com"
- Wasabi EU West 1 (London)
- "s3.eu-west-2.wasabisys.com"
- Wasabi EU West 2 (Paris)
- "s3.ap-northeast-1.wasabisys.com"
- Wasabi AP Northeast 1 (Tokyo) endpoint
- "s3.ap-northeast-2.wasabisys.com"
- Wasabi AP Northeast 2 (Osaka) endpoint
- "s3.ap-southeast-1.wasabisys.com"
- Wasabi AP Southeast 1 (Singapore)
- "s3.ap-southeast-2.wasabisys.com"
- Wasabi AP Southeast 2 (Sydney)
- "s3.ir-thr-at1.arvanstorage.com"
- ArvanCloud Tehran Iran (Asiatech) endpoint
@ -2858,6 +2874,37 @@ Properties:
- Type: bool
- Default: false
#### --s3-might-gzip
Set this if the backend might gzip objects.
Normally providers will not alter objects when they are downloaded. If
an object was not uploaded with `Content-Encoding: gzip` then it won't
be set on download.
However some providers may gzip objects even if they weren't uploaded
with `Content-Encoding: gzip` (eg Cloudflare).
A symptom of this would be receiving errors like
ERROR corrupted on transfer: sizes differ NNN vs MMM
If you set this flag and rclone downloads an object with
Content-Encoding: gzip set and chunked transfer encoding, then rclone
will decompress the object on the fly.
If this is set to unset (the default) then rclone will choose
according to the provider setting what to apply, but you can override
rclone's choice here.
Properties:
- Config: might_gzip
- Env Var: RCLONE_S3_MIGHT_GZIP
- Type: Tristate
- Default: unset
#### --s3-no-system-metadata
Suppress setting and reading of system metadata

View file

@ -33,7 +33,6 @@
<p class="menu">
<i class="fa fa-comments fa-fw" aria-hidden="true"></i> <a href="https://forum.rclone.org">Rclone forum</a><br />
<i class="fab fa-github fa-fw" aria-hidden="true"></i> <a href="https://github.com/rclone/rclone">GitHub project</a><br />
<i class="fab fa-slack fa-fw" aria-hidden="true"></i> <a href="https://slack-invite.rclone.org/">Rclone slack</a><br />
<i class="fa fa-book fa-fw" aria-hidden="true"></i> <a href="https://github.com/rclone/rclone/wiki">Rclone Wiki</a><br />
<i class="fa fa-heart heart fa-fw" aria-hidden="true"></i> <a href="/donate/">Donate</a><br />
<i class="fab fa-twitter fa-fw" aria-hidden="true"></i> <a href="https://twitter.com/njcw">@njcw</a>

View file

@ -1 +1 @@
v1.60.0
v1.60.1

File diff suppressed because it is too large Load diff

View file

@ -1,4 +1,4 @@
package fs
// VersionTag of rclone
var VersionTag = "v1.60.0"
var VersionTag = "v1.60.1"

8
lib/cache/cache.go vendored
View file

@ -190,9 +190,11 @@ func (c *Cache) Rename(oldKey, newKey string) (value interface{}, found bool) {
c.mu.Lock()
if newEntry, newFound := c.cache[newKey]; newFound {
// If new entry is found use that
if _, oldFound := c.cache[oldKey]; oldFound {
// If there's an old entry, we drop it and also try shutdown.
c.finalize(c.cache[oldKey].value)
if oldEntry, oldFound := c.cache[oldKey]; oldFound {
// If there's an old entry that is different we must finalize it
if newEntry.value != oldEntry.value {
c.finalize(c.cache[oldKey].value)
}
}
delete(c.cache, oldKey)
value, found = newEntry.value, newFound

247
rclone.1 generated
View file

@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 2.9.2.1
.\"
.TH "rclone" "1" "Oct 21, 2022" "User Manual" ""
.TH "rclone" "1" "Nov 17, 2022" "User Manual" ""
.hy
.SH Rclone syncs your files to cloud storage
.PP
@ -490,13 +490,13 @@ links.
If not sure, use the first link.
.IP \[bu] 2
Intel/AMD - 64
Bit (https://downloads.rclone.org/rclone-current-linux-amd64.zip)
Bit (https://downloads.rclone.org/rclone-current-windows-amd64.zip)
.IP \[bu] 2
Intel/AMD - 32
Bit (https://downloads.rclone.org/rclone-current-linux-386.zip)
Bit (https://downloads.rclone.org/rclone-current-windows-386.zip)
.IP \[bu] 2
ARM - 64
Bit (https://downloads.rclone.org/rclone-current-linux-arm64.zip)
Bit (https://downloads.rclone.org/rclone-current-windows-arm64.zip)
.PP
Open this file in the Explorer and extract \f[C]rclone.exe\f[R].
Rclone is a portable executable so you can place it wherever is
@ -19313,7 +19313,7 @@ T}
T{
Oracle Object Storage
T}@T{
Yes
No
T}@T{
Yes
T}@T{
@ -19325,7 +19325,7 @@ Yes
T}@T{
Yes
T}@T{
No
Yes
T}@T{
No
T}@T{
@ -19924,7 +19924,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.60.0\[dq])
--user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.60.1\[dq])
-v, --verbose count Print lots more stuff (repeat for more)
\f[R]
.fi
@ -20308,6 +20308,7 @@ They control the backends and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
--s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don\[aq]t attempt to check the bucket exists or create it
--s3-no-head If set, don\[aq]t HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@ -26648,19 +26649,57 @@ Seagate Lyve Cloud AP Southeast 1 (Singapore)
\[dq]s3.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi US East endpoint
Wasabi US East 1 (N.
Virginia)
.RE
.IP \[bu] 2
\[dq]s3.us-east-2.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi US East 2 (N.
Virginia)
.RE
.IP \[bu] 2
\[dq]s3.us-central-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi US Central 1 (Texas)
.RE
.IP \[bu] 2
\[dq]s3.us-west-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi US West endpoint
Wasabi US West 1 (Oregon)
.RE
.IP \[bu] 2
\[dq]s3.ca-central-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi CA Central 1 (Toronto)
.RE
.IP \[bu] 2
\[dq]s3.eu-central-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi EU Central endpoint
Wasabi EU Central 1 (Amsterdam)
.RE
.IP \[bu] 2
\[dq]s3.eu-central-2.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi EU Central 2 (Frankfurt)
.RE
.IP \[bu] 2
\[dq]s3.eu-west-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi EU West 1 (London)
.RE
.IP \[bu] 2
\[dq]s3.eu-west-2.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi EU West 2 (Paris)
.RE
.IP \[bu] 2
\[dq]s3.ap-northeast-1.wasabisys.com\[dq]
@ -26675,6 +26714,18 @@ Wasabi AP Northeast 1 (Tokyo) endpoint
Wasabi AP Northeast 2 (Osaka) endpoint
.RE
.IP \[bu] 2
\[dq]s3.ap-southeast-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi AP Southeast 1 (Singapore)
.RE
.IP \[bu] 2
\[dq]s3.ap-southeast-2.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
Wasabi AP Southeast 2 (Sydney)
.RE
.IP \[bu] 2
\[dq]s3.ir-thr-at1.arvanstorage.com\[dq]
.RS 2
.IP \[bu] 2
@ -28823,6 +28874,42 @@ Env Var: RCLONE_S3_DECOMPRESS
Type: bool
.IP \[bu] 2
Default: false
.SS --s3-might-gzip
.PP
Set this if the backend might gzip objects.
.PP
Normally providers will not alter objects when they are downloaded.
If an object was not uploaded with \f[C]Content-Encoding: gzip\f[R] then
it won\[aq]t be set on download.
.PP
However some providers may gzip objects even if they weren\[aq]t
uploaded with \f[C]Content-Encoding: gzip\f[R] (eg Cloudflare).
.PP
A symptom of this would be receiving errors like
.IP
.nf
\f[C]
ERROR corrupted on transfer: sizes differ NNN vs MMM
\f[R]
.fi
.PP
If you set this flag and rclone downloads an object with
Content-Encoding: gzip set and chunked transfer encoding, then rclone
will decompress the object on the fly.
.PP
If this is set to unset (the default) then rclone will choose according
to the provider setting what to apply, but you can override rclone\[aq]s
choice here.
.PP
Properties:
.IP \[bu] 2
Config: might_gzip
.IP \[bu] 2
Env Var: RCLONE_S3_MIGHT_GZIP
.IP \[bu] 2
Type: Tristate
.IP \[bu] 2
Default: unset
.SS --s3-no-system-metadata
.PP
Suppress setting and reading of system metadata
@ -44213,7 +44300,32 @@ supported by rclone)
.SS Configuration
.PP
Here is an example of making a mailru configuration.
First create a Mail.ru Cloud account and choose a tariff, then run
.PP
First create a Mail.ru Cloud account and choose a tariff.
.PP
You will need to log in and create an app password for rclone.
Rclone \f[B]will not work\f[R] with your normal username and password -
it will give an error like
\f[C]oauth2: server response missing access_token\f[R].
.IP \[bu] 2
Click on your user icon in the top right
.IP \[bu] 2
Go to Security / \[dq]\[u041F]\[u0430]\[u0440]\[u043E]\[u043B]\[u044C]
\[u0438]
\[u0431]\[u0435]\[u0437]\[u043E]\[u043F]\[u0430]\[u0441]\[u043D]\[u043E]\[u0441]\[u0442]\[u044C]\[dq]
.IP \[bu] 2
Click password for apps /
\[dq]\[u041F]\[u0430]\[u0440]\[u043E]\[u043B]\[u0438]
\[u0434]\[u043B]\[u044F]
\[u0432]\[u043D]\[u0435]\[u0448]\[u043D]\[u0438]\[u0445]
\[u043F]\[u0440]\[u0438]\[u043B]\[u043E]\[u0436]\[u0435]\[u043D]\[u0438]\[u0439]\[dq]
.IP \[bu] 2
Add the password - give it a name - eg \[dq]rclone\[dq]
.IP \[bu] 2
Copy the password and use this password below - your normal login
password won\[aq]t work.
.PP
Now run
.IP
.nf
\f[C]
@ -44244,6 +44356,10 @@ User name (usually email)
Enter a string value. Press Enter for the default (\[dq]\[dq]).
user> username\[at]mail.ru
Password
This must be an app password - rclone will not work with your normal
password. See the Configuration section in the docs for how to make an
app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@ -44441,6 +44557,11 @@ Required: true
.PP
Password.
.PP
This must be an app password - rclone will not work with your normal
password.
See the Configuration section in the docs for how to make an app
password.
.PP
\f[B]NB\f[R] Input to this must be obscured - see rclone
obscure (https://rclone.org/commands/rclone_obscure/).
.PP
@ -56024,6 +56145,93 @@ Options:
.IP \[bu] 2
\[dq]error\[dq]: return an error based on option value
.SH Changelog
.SS v1.60.1 - 2022-11-17
.PP
See commits (https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1)
.IP \[bu] 2
Bug Fixes
.RS 2
.IP \[bu] 2
lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)
.IP \[bu] 2
wasm: Fix walltime link error by adding up-to-date wasm_exec.js
(Jo\[~a]o Henrique Franco)
.IP \[bu] 2
docs
.RS 2
.IP \[bu] 2
Update faq.md with bisync (Samuel Johnson)
.IP \[bu] 2
Corrected download links in windows install docs (coultonluke)
.IP \[bu] 2
Add direct download link for windows arm64 (albertony)
.IP \[bu] 2
Remove link to rclone slack as it is no longer supported (Nick
Craig-Wood)
.IP \[bu] 2
Faq: how to use a proxy server that requires a username and password
(asdffdsazqqq)
.IP \[bu] 2
Oracle-object-storage: doc fix (Manoj Ghosh)
.IP \[bu] 2
Fix typo \f[C]remove\f[R] in rclone_serve_restic command (Joda
St\[:o]\[ss]er)
.IP \[bu] 2
Fix character that was incorrectly interpreted as markdown (Cl\['e]ment
Notin)
.RE
.RE
.IP \[bu] 2
VFS
.RS 2
.IP \[bu] 2
Fix deadlock caused by cache cleaner and upload finishing (Nick
Craig-Wood)
.RE
.IP \[bu] 2
Local
.RS 2
.IP \[bu] 2
Clean absolute paths (albertony)
.IP \[bu] 2
Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)
.RE
.IP \[bu] 2
Mailru
.RS 2
.IP \[bu] 2
Note that an app password is now needed (Nick Craig-Wood)
.IP \[bu] 2
Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)
.RE
.IP \[bu] 2
S3
.RS 2
.IP \[bu] 2
Add provider quirk \f[C]--s3-might-gzip\f[R] to fix corrupted on
transfer: sizes differ (Nick Craig-Wood)
.IP \[bu] 2
Allow Storj to server side copy since it seems to work now (Nick
Craig-Wood)
.IP \[bu] 2
Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
.IP \[bu] 2
Add additional Wasabi locations (techknowlogick)
.RE
.IP \[bu] 2
Smb
.RS 2
.IP \[bu] 2
Fix \f[C]Failed to sync: context canceled\f[R] at the end of syncs (Nick
Craig-Wood)
.RE
.IP \[bu] 2
WebDAV
.RS 2
.IP \[bu] 2
Fix Move/Copy/DirMove when using -server-side-across-configs (Nick
Craig-Wood)
.RE
.SS v1.60.0 - 2022-10-21
.PP
See commits (https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0)
@ -67037,10 +67245,8 @@ significant amount of metadata, which breaks the desired 1:1 mapping of
files to objects.
.SS Can rclone do bi-directional sync?
.PP
No, not at present.
rclone only does uni-directional sync from A -> B.
It may do in the future though since it has all the primitives - it just
requires writing the algorithm to do it.
Yes, since rclone v1.58.0, bidirectional cloud
sync (https://rclone.org/bisync/) is available.
.SS Can I use rclone with an HTTP proxy?
.PP
Yes.
@ -67073,6 +67279,17 @@ export HTTPS_PROXY=$http_proxy
\f[R]
.fi
.PP
Note: If the proxy server requires a username and password, then use
.IP
.nf
\f[C]
export http_proxy=http://username:password\[at]proxyserver:12345
export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
\f[R]
.fi
.PP
The \f[C]NO_PROXY\f[R] allows you to disable the proxy for specific
hosts.
Hosts must be comma separated, and can contain domains or parts.

View file

@ -590,20 +590,25 @@ func (item *Item) _store(ctx context.Context, storeFn StoreFn) (err error) {
item._updateFingerprint()
}
item.info.Dirty = false
err = item._save()
if err != nil {
fs.Errorf(item.name, "vfs cache: failed to write metadata file: %v", err)
}
// Write the object back to the VFS layer before we mark it as
// clean, otherwise it will become eligible for removal which
// can cause a deadlock
if storeFn != nil && item.o != nil {
fs.Debugf(item.name, "vfs cache: writeback object to VFS layer")
// Write the object back to the VFS layer as last
// thing we do with mutex unlocked
// Write the object back to the VFS layer last with mutex unlocked
o := item.o
item.mu.Unlock()
storeFn(o)
item.mu.Lock()
}
// Show item is clean and is elegible for cache removal
item.info.Dirty = false
err = item._save()
if err != nil {
fs.Errorf(item.name, "vfs cache: failed to write metadata file: %v", err)
}
return nil
}