3ffa47ea16
This adds a new optional parameter to the backend, to specify a path to a unix domain socket to connect to, instead the specified URL. The URL itself is still used for the rest of the HTTP client, allowing host and subpath to stay intact. This allows using rclone with the webdav backend to connect to a WebDAV server provided at a Unix Domain socket: rclone serve webdav --addr unix:///tmp/my.socket remote:path rclone --webdav-unix-socket /tmp/my.socket --webdav-url http://localhost lsf :webdav:
513 lines
14 KiB
Markdown
513 lines
14 KiB
Markdown
---
|
|
title: "WebDAV"
|
|
description: "Rclone docs for WebDAV"
|
|
versionIntroduced: "v1.39"
|
|
---
|
|
|
|
# {{< icon "fa fa-globe" >}} WebDAV
|
|
|
|
Paths are specified as `remote:path`
|
|
|
|
Paths may be as deep as required, e.g. `remote:directory/subdirectory`.
|
|
|
|
## Configuration
|
|
|
|
To configure the WebDAV remote you will need to have a URL for it, and
|
|
a username and password. If you know what kind of system you are
|
|
connecting to then rclone can enable extra features.
|
|
|
|
Here is an example of how to make a remote called `remote`. First run:
|
|
|
|
rclone config
|
|
|
|
This will guide you through an interactive setup process:
|
|
|
|
```
|
|
No remotes found, make a new one?
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / WebDAV
|
|
\ "webdav"
|
|
[snip]
|
|
Storage> webdav
|
|
URL of http host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\ "https://example.com"
|
|
url> https://example.com/remote.php/webdav/
|
|
Name of the WebDAV site/service/software you are using
|
|
Choose a number from below, or type in your own value
|
|
1 / Fastmail Files
|
|
\ (fastmail)
|
|
2 / Nextcloud
|
|
\ (nextcloud)
|
|
3 / Owncloud
|
|
\ (owncloud)
|
|
4 / Sharepoint Online, authenticated by Microsoft account
|
|
\ (sharepoint)
|
|
5 / Sharepoint with NTLM authentication, usually self-hosted or on-premises
|
|
\ (sharepoint-ntlm)
|
|
6 / rclone WebDAV server to serve a remote over HTTP via the WebDAV protocol
|
|
\ (rclone)
|
|
7 / Other site/service or software
|
|
\ (other)
|
|
vendor> 2
|
|
User name
|
|
user> user
|
|
Password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Bearer token instead of user/pass (e.g. a Macaroon)
|
|
bearer_token>
|
|
Remote config
|
|
--------------------
|
|
[remote]
|
|
type = webdav
|
|
url = https://example.com/remote.php/webdav/
|
|
vendor = nextcloud
|
|
user = user
|
|
pass = *** ENCRYPTED ***
|
|
bearer_token =
|
|
--------------------
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
```
|
|
|
|
Once configured you can then use `rclone` like this,
|
|
|
|
List directories in top level of your WebDAV
|
|
|
|
rclone lsd remote:
|
|
|
|
List all the files in your WebDAV
|
|
|
|
rclone ls remote:
|
|
|
|
To copy a local directory to an WebDAV directory called backup
|
|
|
|
rclone copy /home/source remote:backup
|
|
|
|
### Modification times and hashes
|
|
|
|
Plain WebDAV does not support modified times. However when used with
|
|
Fastmail Files, Owncloud or Nextcloud rclone will support modified times.
|
|
|
|
Likewise plain WebDAV does not support hashes, however when used with
|
|
Fastmail Files, Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes.
|
|
Depending on the exact version of Owncloud or Nextcloud hashes may
|
|
appear on all objects, or only on objects which had a hash uploaded
|
|
with them.
|
|
|
|
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/webdav/webdav.go then run make backenddocs" >}}
|
|
### Standard options
|
|
|
|
Here are the Standard options specific to webdav (WebDAV).
|
|
|
|
#### --webdav-url
|
|
|
|
URL of http host to connect to.
|
|
|
|
E.g. https://example.com.
|
|
|
|
Properties:
|
|
|
|
- Config: url
|
|
- Env Var: RCLONE_WEBDAV_URL
|
|
- Type: string
|
|
- Required: true
|
|
|
|
#### --webdav-vendor
|
|
|
|
Name of the WebDAV site/service/software you are using.
|
|
|
|
Properties:
|
|
|
|
- Config: vendor
|
|
- Env Var: RCLONE_WEBDAV_VENDOR
|
|
- Type: string
|
|
- Required: false
|
|
- Examples:
|
|
- "fastmail"
|
|
- Fastmail Files
|
|
- "nextcloud"
|
|
- Nextcloud
|
|
- "owncloud"
|
|
- Owncloud
|
|
- "sharepoint"
|
|
- Sharepoint Online, authenticated by Microsoft account
|
|
- "sharepoint-ntlm"
|
|
- Sharepoint with NTLM authentication, usually self-hosted or on-premises
|
|
- "rclone"
|
|
- rclone WebDAV server to serve a remote over HTTP via the WebDAV protocol
|
|
- "other"
|
|
- Other site/service or software
|
|
|
|
#### --webdav-user
|
|
|
|
User name.
|
|
|
|
In case NTLM authentication is used, the username should be in the format 'Domain\User'.
|
|
|
|
Properties:
|
|
|
|
- Config: user
|
|
- Env Var: RCLONE_WEBDAV_USER
|
|
- Type: string
|
|
- Required: false
|
|
|
|
#### --webdav-pass
|
|
|
|
Password.
|
|
|
|
**NB** Input to this must be obscured - see [rclone obscure](/commands/rclone_obscure/).
|
|
|
|
Properties:
|
|
|
|
- Config: pass
|
|
- Env Var: RCLONE_WEBDAV_PASS
|
|
- Type: string
|
|
- Required: false
|
|
|
|
#### --webdav-bearer-token
|
|
|
|
Bearer token instead of user/pass (e.g. a Macaroon).
|
|
|
|
Properties:
|
|
|
|
- Config: bearer_token
|
|
- Env Var: RCLONE_WEBDAV_BEARER_TOKEN
|
|
- Type: string
|
|
- Required: false
|
|
|
|
### Advanced options
|
|
|
|
Here are the Advanced options specific to webdav (WebDAV).
|
|
|
|
#### --webdav-bearer-token-command
|
|
|
|
Command to run to get a bearer token.
|
|
|
|
Properties:
|
|
|
|
- Config: bearer_token_command
|
|
- Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
|
|
- Type: string
|
|
- Required: false
|
|
|
|
#### --webdav-encoding
|
|
|
|
The encoding for the backend.
|
|
|
|
See the [encoding section in the overview](/overview/#encoding) for more info.
|
|
|
|
Default encoding is Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8 for sharepoint-ntlm or identity otherwise.
|
|
|
|
Properties:
|
|
|
|
- Config: encoding
|
|
- Env Var: RCLONE_WEBDAV_ENCODING
|
|
- Type: string
|
|
- Required: false
|
|
|
|
#### --webdav-headers
|
|
|
|
Set HTTP headers for all transactions.
|
|
|
|
Use this to set additional HTTP headers for all transactions
|
|
|
|
The input format is comma separated list of key,value pairs. Standard
|
|
[CSV encoding](https://godoc.org/encoding/csv) may be used.
|
|
|
|
For example, to set a Cookie use 'Cookie,name=value', or '"Cookie","name=value"'.
|
|
|
|
You can set multiple headers, e.g. '"Cookie","name=value","Authorization","xxx"'.
|
|
|
|
|
|
Properties:
|
|
|
|
- Config: headers
|
|
- Env Var: RCLONE_WEBDAV_HEADERS
|
|
- Type: CommaSepList
|
|
- Default:
|
|
|
|
#### --webdav-pacer-min-sleep
|
|
|
|
Minimum time to sleep between API calls.
|
|
|
|
Properties:
|
|
|
|
- Config: pacer_min_sleep
|
|
- Env Var: RCLONE_WEBDAV_PACER_MIN_SLEEP
|
|
- Type: Duration
|
|
- Default: 10ms
|
|
|
|
#### --webdav-nextcloud-chunk-size
|
|
|
|
Nextcloud upload chunk size.
|
|
|
|
We recommend configuring your NextCloud instance to increase the max chunk size to 1 GB for better upload performances.
|
|
See https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html#adjust-chunk-size-on-nextcloud-side
|
|
|
|
Set to 0 to disable chunked uploading.
|
|
|
|
|
|
Properties:
|
|
|
|
- Config: nextcloud_chunk_size
|
|
- Env Var: RCLONE_WEBDAV_NEXTCLOUD_CHUNK_SIZE
|
|
- Type: SizeSuffix
|
|
- Default: 10Mi
|
|
|
|
#### --webdav-owncloud-exclude-shares
|
|
|
|
Exclude ownCloud shares
|
|
|
|
Properties:
|
|
|
|
- Config: owncloud_exclude_shares
|
|
- Env Var: RCLONE_WEBDAV_OWNCLOUD_EXCLUDE_SHARES
|
|
- Type: bool
|
|
- Default: false
|
|
|
|
#### --webdav-owncloud-exclude-mounts
|
|
|
|
Exclude ownCloud mounted storages
|
|
|
|
Properties:
|
|
|
|
- Config: owncloud_exclude_mounts
|
|
- Env Var: RCLONE_WEBDAV_OWNCLOUD_EXCLUDE_MOUNTS
|
|
- Type: bool
|
|
- Default: false
|
|
|
|
#### --webdav-unix-socket
|
|
|
|
Path to a unix domain socket to dial to, instead of opening a TCP connection directly
|
|
|
|
Properties:
|
|
|
|
- Config: unix_socket
|
|
- Env Var: RCLONE_WEBDAV_UNIX_SOCKET
|
|
- Type: string
|
|
- Required: false
|
|
|
|
#### --webdav-description
|
|
|
|
Description of the remote.
|
|
|
|
Properties:
|
|
|
|
- Config: description
|
|
- Env Var: RCLONE_WEBDAV_DESCRIPTION
|
|
- Type: string
|
|
- Required: false
|
|
|
|
{{< rem autogenerated options stop >}}
|
|
|
|
## Provider notes
|
|
|
|
See below for notes on specific providers.
|
|
|
|
### Fastmail Files
|
|
|
|
Use `https://webdav.fastmail.com/` or a subdirectory as the URL,
|
|
and your Fastmail email `username@domain.tld` as the username.
|
|
Follow [this documentation](https://www.fastmail.help/hc/en-us/articles/360058752854-App-passwords)
|
|
to create an app password with access to `Files (WebDAV)` and use
|
|
this as the password.
|
|
|
|
Fastmail supports modified times using the `X-OC-Mtime` header.
|
|
|
|
### Owncloud
|
|
|
|
Click on the settings cog in the bottom right of the page and this
|
|
will show the WebDAV URL that rclone needs in the config step. It
|
|
will look something like `https://example.com/remote.php/webdav/`.
|
|
|
|
Owncloud supports modified times using the `X-OC-Mtime` header.
|
|
|
|
### Nextcloud
|
|
|
|
This is configured in an identical way to Owncloud. Note that
|
|
Nextcloud initially did not support streaming of files (`rcat`) whereas
|
|
Owncloud did, but [this](https://github.com/nextcloud/nextcloud-snap/issues/365) seems to be fixed as of 2020-11-27 (tested with rclone v1.53.1 and Nextcloud Server v19).
|
|
|
|
### Sharepoint Online
|
|
|
|
Rclone can be used with Sharepoint provided by OneDrive for Business
|
|
or Office365 Education Accounts.
|
|
This feature is only needed for a few of these Accounts,
|
|
mostly Office365 Education ones. These accounts are sometimes not
|
|
verified by the domain owner [github#1975](https://github.com/rclone/rclone/issues/1975)
|
|
|
|
This means that these accounts can't be added using the official
|
|
API (other Accounts should work with the "onedrive" option). However,
|
|
it is possible to access them using webdav.
|
|
|
|
To use a sharepoint remote with rclone, add it like this:
|
|
First, you need to get your remote's URL:
|
|
|
|
- Go [here](https://onedrive.live.com/about/en-us/signin/)
|
|
to open your OneDrive or to sign in
|
|
- Now take a look at your address bar, the URL should look like this:
|
|
`https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/_layouts/15/onedrive.aspx`
|
|
|
|
You'll only need this URL up to the email address. After that, you'll
|
|
most likely want to add "/Documents". That subdirectory contains
|
|
the actual data stored on your OneDrive.
|
|
|
|
Add the remote to rclone like this:
|
|
Configure the `url` as `https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents`
|
|
and use your normal account email and password for `user` and `pass`.
|
|
If you have 2FA enabled, you have to generate an app password.
|
|
Set the `vendor` to `sharepoint`.
|
|
|
|
Your config file should look like this:
|
|
|
|
```
|
|
[sharepoint]
|
|
type = webdav
|
|
url = https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents
|
|
vendor = sharepoint
|
|
user = YourEmailAddress
|
|
pass = encryptedpassword
|
|
```
|
|
|
|
### Sharepoint with NTLM Authentication
|
|
|
|
Use this option in case your (hosted) Sharepoint is not tied to OneDrive accounts and uses NTLM authentication.
|
|
|
|
To get the `url` configuration, similarly to the above, first navigate to the desired directory in your browser to get the URL,
|
|
then strip everything after the name of the opened directory.
|
|
|
|
Example:
|
|
If the URL is:
|
|
https://example.sharepoint.com/sites/12345/Documents/Forms/AllItems.aspx
|
|
|
|
The configuration to use would be:
|
|
https://example.sharepoint.com/sites/12345/Documents
|
|
|
|
Set the `vendor` to `sharepoint-ntlm`.
|
|
|
|
NTLM uses domain and user name combination for authentication,
|
|
set `user` to `DOMAIN\username`.
|
|
|
|
Your config file should look like this:
|
|
|
|
```
|
|
[sharepoint]
|
|
type = webdav
|
|
url = https://[YOUR-DOMAIN]/some-path-to/Documents
|
|
vendor = sharepoint-ntlm
|
|
user = DOMAIN\user
|
|
pass = encryptedpassword
|
|
```
|
|
|
|
#### Required Flags for SharePoint
|
|
|
|
As SharePoint does some special things with uploaded documents, you won't be able to use the documents size or the documents hash to compare if a file has been changed since the upload / which file is newer.
|
|
|
|
For Rclone calls copying files (especially Office files such as .docx, .xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should append these flags to ensure Rclone uses the "Last Modified" datetime property to compare your documents:
|
|
|
|
```
|
|
--ignore-size --ignore-checksum --update
|
|
```
|
|
|
|
## Rclone
|
|
|
|
Use this option if you are hosting remotes over WebDAV provided by rclone.
|
|
Read [rclone serve webdav](commands/rclone_serve_webdav/) for more details.
|
|
|
|
rclone serve supports modified times using the `X-OC-Mtime` header.
|
|
|
|
|
|
### dCache
|
|
|
|
dCache is a storage system that supports many protocols and
|
|
authentication/authorisation schemes. For WebDAV clients, it allows
|
|
users to authenticate with username and password (BASIC), X.509,
|
|
Kerberos, and various bearer tokens, including
|
|
[Macaroons](https://www.dcache.org/manuals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf)
|
|
and [OpenID-Connect](https://en.wikipedia.org/wiki/OpenID_Connect)
|
|
access tokens.
|
|
|
|
Configure as normal using the `other` type. Don't enter a username or
|
|
password, instead enter your Macaroon as the `bearer_token`.
|
|
|
|
The config will end up looking something like this.
|
|
|
|
```
|
|
[dcache]
|
|
type = webdav
|
|
url = https://dcache...
|
|
vendor = other
|
|
user =
|
|
pass =
|
|
bearer_token = your-macaroon
|
|
```
|
|
|
|
There is a [script](https://github.com/sara-nl/GridScripts/blob/master/get-macaroon) that
|
|
obtains a Macaroon from a dCache WebDAV endpoint, and creates an rclone config file.
|
|
|
|
Macaroons may also be obtained from the dCacheView
|
|
web-browser/JavaScript client that comes with dCache.
|
|
|
|
### OpenID-Connect
|
|
|
|
dCache also supports authenticating with OpenID-Connect access tokens.
|
|
OpenID-Connect is a protocol (based on OAuth 2.0) that allows services
|
|
to identify users who have authenticated with some central service.
|
|
|
|
Support for OpenID-Connect in rclone is currently achieved using
|
|
another software package called
|
|
[oidc-agent](https://github.com/indigo-dc/oidc-agent). This is a
|
|
command-line tool that facilitates obtaining an access token. Once
|
|
installed and configured, an access token is obtained by running the
|
|
`oidc-token` command. The following example shows a (shortened)
|
|
access token obtained from the *XDC* OIDC Provider.
|
|
|
|
```
|
|
paul@celebrimbor:~$ oidc-token XDC
|
|
eyJraWQ[...]QFXDt0
|
|
paul@celebrimbor:~$
|
|
```
|
|
|
|
**Note** Before the `oidc-token` command will work, the refresh token
|
|
must be loaded into the oidc agent. This is done with the `oidc-add`
|
|
command (e.g., `oidc-add XDC`). This is typically done once per login
|
|
session. Full details on this and how to register oidc-agent with
|
|
your OIDC Provider are provided in the [oidc-agent
|
|
documentation](https://indigo-dc.gitbooks.io/oidc-agent/).
|
|
|
|
The rclone `bearer_token_command` configuration option is used to
|
|
fetch the access token from oidc-agent.
|
|
|
|
Configure as a normal WebDAV endpoint, using the 'other' vendor,
|
|
leaving the username and password empty. When prompted, choose to
|
|
edit the advanced config and enter the command to get a bearer token
|
|
(e.g., `oidc-agent XDC`).
|
|
|
|
The following example config shows a WebDAV endpoint that uses
|
|
oidc-agent to supply an access token from the *XDC* OIDC Provider.
|
|
|
|
```
|
|
[dcache]
|
|
type = webdav
|
|
url = https://dcache.example.org/
|
|
vendor = other
|
|
bearer_token_command = oidc-token XDC
|
|
```
|