265 lines
7.3 KiB
Markdown
265 lines
7.3 KiB
Markdown
---
|
|
title: "HTTP Remote"
|
|
description: "Read only remote for HTTP servers"
|
|
versionIntroduced: "v1.37"
|
|
---
|
|
|
|
# {{< icon "fa fa-globe" >}} HTTP
|
|
|
|
The HTTP remote is a read only remote for reading files of a
|
|
webserver. The webserver should provide file listings which rclone
|
|
will read and turn into a remote. This has been tested with common
|
|
webservers such as Apache/Nginx/Caddy and will likely work with file
|
|
listings from most web servers. (If it doesn't then please file an
|
|
issue, or send a pull request!)
|
|
|
|
Paths are specified as `remote:` or `remote:path`.
|
|
|
|
The `remote:` represents the configured [url](#http-url), and any path following
|
|
it will be resolved relative to this url, according to the URL standard. This
|
|
means with remote url `https://beta.rclone.org/branch` and path `fix`, the
|
|
resolved URL will be `https://beta.rclone.org/branch/fix`, while with path
|
|
`/fix` the resolved URL will be `https://beta.rclone.org/fix` as the absolute
|
|
path is resolved from the root of the domain.
|
|
|
|
If the path following the `remote:` ends with `/` it will be assumed to point
|
|
to a directory. If the path does not end with `/`, then a HEAD request is sent
|
|
and the response used to decide if it it is treated as a file or a directory
|
|
(run with `-vv` to see details). When [--http-no-head](#http-no-head) is
|
|
specified, a path without ending `/` is always assumed to be a file. If rclone
|
|
incorrectly assumes the path is a file, the solution is to specify the path with
|
|
ending `/`. When you know the path is a directory, ending it with `/` is always
|
|
better as it avoids the initial HEAD request.
|
|
|
|
To just download a single file it is easier to use
|
|
[copyurl](/commands/rclone_copyurl/).
|
|
|
|
## Configuration
|
|
|
|
Here is an example of how to make a remote called `remote`. First
|
|
run:
|
|
|
|
rclone config
|
|
|
|
This will guide you through an interactive setup process:
|
|
|
|
```
|
|
No remotes found, make a new one?
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / HTTP
|
|
\ "http"
|
|
[snip]
|
|
Storage> http
|
|
URL of http host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\ "https://example.com"
|
|
url> https://beta.rclone.org
|
|
Remote config
|
|
--------------------
|
|
[remote]
|
|
url = https://beta.rclone.org
|
|
--------------------
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote http
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
```
|
|
|
|
This remote is called `remote` and can now be used like this
|
|
|
|
See all the top level directories
|
|
|
|
rclone lsd remote:
|
|
|
|
List the contents of a directory
|
|
|
|
rclone ls remote:directory
|
|
|
|
Sync the remote `directory` to `/home/local/directory`, deleting any excess files.
|
|
|
|
rclone sync --interactive remote:directory /home/local/directory
|
|
|
|
### Read only
|
|
|
|
This remote is read only - you can't upload files to an HTTP server.
|
|
|
|
### Modification times
|
|
|
|
Most HTTP servers store time accurate to 1 second.
|
|
|
|
### Checksum
|
|
|
|
No checksums are stored.
|
|
|
|
### Usage without a config file
|
|
|
|
Since the http remote only has one config parameter it is easy to use
|
|
without a config file:
|
|
|
|
rclone lsd --http-url https://beta.rclone.org :http:
|
|
|
|
or:
|
|
|
|
rclone lsd :http,url='https://beta.rclone.org':
|
|
|
|
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/http/http.go then run make backenddocs" >}}
|
|
### Standard options
|
|
|
|
Here are the Standard options specific to http (HTTP).
|
|
|
|
#### --http-url
|
|
|
|
URL of HTTP host to connect to.
|
|
|
|
E.g. "https://example.com", or "https://user:pass@example.com" to use a username and password.
|
|
|
|
Properties:
|
|
|
|
- Config: url
|
|
- Env Var: RCLONE_HTTP_URL
|
|
- Type: string
|
|
- Required: true
|
|
|
|
### Advanced options
|
|
|
|
Here are the Advanced options specific to http (HTTP).
|
|
|
|
#### --http-headers
|
|
|
|
Set HTTP headers for all transactions.
|
|
|
|
Use this to set additional HTTP headers for all transactions.
|
|
|
|
The input format is comma separated list of key,value pairs. Standard
|
|
[CSV encoding](https://godoc.org/encoding/csv) may be used.
|
|
|
|
For example, to set a Cookie use 'Cookie,name=value', or '"Cookie","name=value"'.
|
|
|
|
You can set multiple headers, e.g. '"Cookie","name=value","Authorization","xxx"'.
|
|
|
|
Properties:
|
|
|
|
- Config: headers
|
|
- Env Var: RCLONE_HTTP_HEADERS
|
|
- Type: CommaSepList
|
|
- Default:
|
|
|
|
#### --http-no-slash
|
|
|
|
Set this if the site doesn't end directories with /.
|
|
|
|
Use this if your target website does not use / on the end of
|
|
directories.
|
|
|
|
A / on the end of a path is how rclone normally tells the difference
|
|
between files and directories. If this flag is set, then rclone will
|
|
treat all files with Content-Type: text/html as directories and read
|
|
URLs from them rather than downloading them.
|
|
|
|
Note that this may cause rclone to confuse genuine HTML files with
|
|
directories.
|
|
|
|
Properties:
|
|
|
|
- Config: no_slash
|
|
- Env Var: RCLONE_HTTP_NO_SLASH
|
|
- Type: bool
|
|
- Default: false
|
|
|
|
#### --http-no-head
|
|
|
|
Don't use HEAD requests.
|
|
|
|
HEAD requests are mainly used to find file sizes in dir listing.
|
|
If your site is being very slow to load then you can try this option.
|
|
Normally rclone does a HEAD request for each potential file in a
|
|
directory listing to:
|
|
|
|
- find its size
|
|
- check it really exists
|
|
- check to see if it is a directory
|
|
|
|
If you set this option, rclone will not do the HEAD request. This will mean
|
|
that directory listings are much quicker, but rclone won't have the times or
|
|
sizes of any files, and some files that don't exist may be in the listing.
|
|
|
|
Properties:
|
|
|
|
- Config: no_head
|
|
- Env Var: RCLONE_HTTP_NO_HEAD
|
|
- Type: bool
|
|
- Default: false
|
|
|
|
## Backend commands
|
|
|
|
Here are the commands specific to the http backend.
|
|
|
|
Run them with
|
|
|
|
rclone backend COMMAND remote:
|
|
|
|
The help below will explain what arguments each command takes.
|
|
|
|
See the [backend](/commands/rclone_backend/) command for more
|
|
info on how to pass options and arguments.
|
|
|
|
These can be run on a running backend using the rc command
|
|
[backend/command](/rc/#backend-command).
|
|
|
|
### set
|
|
|
|
Set command for updating the config parameters.
|
|
|
|
rclone backend set remote: [options] [<arguments>+]
|
|
|
|
This set command can be used to update the config parameters
|
|
for a running http backend.
|
|
|
|
Usage Examples:
|
|
|
|
rclone backend set remote: [-o opt_name=opt_value] [-o opt_name2=opt_value2]
|
|
rclone rc backend/command command=set fs=remote: [-o opt_name=opt_value] [-o opt_name2=opt_value2]
|
|
rclone rc backend/command command=set fs=remote: -o url=https://example.com
|
|
|
|
The option keys are named as they are in the config file.
|
|
|
|
This rebuilds the connection to the http backend when it is called with
|
|
the new parameters. Only new parameters need be passed as the values
|
|
will default to those currently in use.
|
|
|
|
It doesn't return anything.
|
|
|
|
|
|
{{< rem autogenerated options stop >}}
|
|
|
|
## Limitations
|
|
|
|
`rclone about` is not supported by the HTTP backend. Backends without
|
|
this capability cannot determine free space for an rclone mount or
|
|
use policy `mfs` (most free space) as a member of an rclone union
|
|
remote.
|
|
|
|
See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/)
|
|
|