299 lines
18 KiB
Markdown
299 lines
18 KiB
Markdown
|
---
|
||
|
|
date: 2018-03-19T10:05:30Z
|
||
|
|
title: "rclone serve restic"
|
||
|
|
slug: rclone_serve_restic
|
||
|
|
url: /commands/rclone_serve_restic/
|
||
|
|
---
|
||
|
|
## rclone serve restic
|
||
|
|
|
||
|
|
Serve the remote for restic's REST API.
|
||
|
|
|
||
|
|
### Synopsis
|
||
|
|
|
||
|
|
rclone serve restic implements restic's REST backend API
|
||
|
|
over HTTP. This allows restic to use rclone as a data storage
|
||
|
|
mechanism for cloud providers that restic does not support directly.
|
||
|
|
|
||
|
|
[Restic](https://restic.net/) is a command line program for doing
|
||
|
|
backups.
|
||
|
|
|
||
|
|
The server will log errors. Use -v to see access logs.
|
||
|
|
|
||
|
|
--bwlimit will be respected for file transfers. Use --stats to
|
||
|
|
control the stats printing.
|
||
|
|
|
||
|
|
### Setting up rclone for use by restic ###
|
||
|
|
|
||
|
|
First [set up a remote for your chosen cloud provider](/docs/#configure).
|
||
|
|
|
||
|
|
Once you have set up the remote, check it is working with, for example
|
||
|
|
"rclone lsd remote:". You may have called the remote something other
|
||
|
|
than "remote:" - just substitute whatever you called it in the
|
||
|
|
following instructions.
|
||
|
|
|
||
|
|
Now start the rclone restic server
|
||
|
|
|
||
|
|
rclone serve restic -v remote:backup
|
||
|
|
|
||
|
|
Where you can replace "backup" in the above by whatever path in the
|
||
|
|
remote you wish to use.
|
||
|
|
|
||
|
|
By default this will serve on "localhost:8080" you can change this
|
||
|
|
with use of the "--addr" flag.
|
||
|
|
|
||
|
|
You might wish to start this server on boot.
|
||
|
|
|
||
|
|
### Setting up restic to use rclone ###
|
||
|
|
|
||
|
|
Now you can [follow the restic
|
||
|
|
instructions](http://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server)
|
||
|
|
on setting up restic.
|
||
|
|
|
||
|
|
Note that you will need restic 0.8.2 or later to interoperate with
|
||
|
|
rclone.
|
||
|
|
|
||
|
|
For the example above you will want to use "http://localhost:8080/" as
|
||
|
|
the URL for the REST server.
|
||
|
|
|
||
|
|
For example:
|
||
|
|
|
||
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/
|
||
|
|
$ export RESTIC_PASSWORD=yourpassword
|
||
|
|
$ restic init
|
||
|
|
created restic backend 8b1a4b56ae at rest:http://localhost:8080/
|
||
|
|
|
||
|
|
Please note that knowledge of your password is required to access
|
||
|
|
the repository. Losing your password means that your data is
|
||
|
|
irrecoverably lost.
|
||
|
|
$ restic backup /path/to/files/to/backup
|
||
|
|
scan [/path/to/files/to/backup]
|
||
|
|
scanned 189 directories, 312 files in 0:00
|
||
|
|
[0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00
|
||
|
|
duration: 0:00
|
||
|
|
snapshot 45c8fdd8 saved
|
||
|
|
|
||
|
|
#### Multiple repositories ####
|
||
|
|
|
||
|
|
Note that you can use the endpoint to host multiple repositories. Do
|
||
|
|
this by adding a directory name or path after the URL. Note that
|
||
|
|
these **must** end with /. Eg
|
||
|
|
|
||
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
|
||
|
|
# backup user1 stuff
|
||
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
|
||
|
|
# backup user2 stuff
|
||
|
|
|
||
|
|
|
||
|
|
### Server options
|
||
|
|
|
||
|
|
Use --addr to specify which IP address and port the server should
|
||
|
|
listen on, eg --addr 1.2.3.4:8000 or --addr :8080 to listen to all
|
||
|
|
IPs. By default it only listens on localhost.
|
||
|
|
|
||
|
|
If you set --addr to listen on a public or LAN accessible IP address
|
||
|
|
then using Authentication if advised - see the next section for info.
|
||
|
|
|
||
|
|
--server-read-timeout and --server-write-timeout can be used to
|
||
|
|
control the timeouts on the server. Note that this is the total time
|
||
|
|
for a transfer.
|
||
|
|
|
||
|
|
--max-header-bytes controls the maximum number of bytes the server will
|
||
|
|
accept in the HTTP header.
|
||
|
|
|
||
|
|
#### Authentication
|
||
|
|
|
||
|
|
By default this will serve files without needing a login.
|
||
|
|
|
||
|
|
You can either use an htpasswd file which can take lots of users, or
|
||
|
|
set a single username and password with the --user and --pass flags.
|
||
|
|
|
||
|
|
Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
|
||
|
|
in standard apache format and supports MD5, SHA1 and BCrypt for basic
|
||
|
|
authentication. Bcrypt is recommended.
|
||
|
|
|
||
|
|
To create an htpasswd file:
|
||
|
|
|
||
|
|
touch htpasswd
|
||
|
|
htpasswd -B htpasswd user
|
||
|
|
htpasswd -B htpasswd anotherUser
|
||
|
|
|
||
|
|
The password file can be updated while rclone is running.
|
||
|
|
|
||
|
|
Use --realm to set the authentication realm.
|
||
|
|
|
||
|
|
#### SSL/TLS
|
||
|
|
|
||
|
|
By default this will serve over http. If you want you can serve over
|
||
|
|
https. You will need to supply the --cert and --key flags. If you
|
||
|
|
wish to do client side certificate validation then you will need to
|
||
|
|
supply --client-ca also.
|
||
|
|
|
||
|
|
--cert should be a either a PEM encoded certificate or a concatenation
|
||
|
|
of that with the CA certificate. --key should be the PEM encoded
|
||
|
|
private key and --client-ca should be the PEM encoded client
|
||
|
|
certificate authority certificate.
|
||
|
|
|
||
|
|
|
||
|
|
```
|
||
|
|
rclone serve restic remote:path [flags]
|
||
|
|
```
|
||
|
|
|
||
|
|
### Options
|
||
|
|
|
||
|
|
```
|
||
|
|
--addr string IPaddress:Port or :Port to bind server to. (default "localhost:8080")
|
||
|
|
--cert string SSL PEM key (concatenation of certificate and CA certificate)
|
||
|
|
--client-ca string Client certificate authority to verify clients with
|
||
|
|
-h, --help help for restic
|
||
|
|
--htpasswd string htpasswd file - if not provided no authentication is done
|
||
|
|
--key string SSL PEM Private key
|
||
|
|
--max-header-bytes int Maximum size of request header (default 4096)
|
||
|
|
--pass string Password for authentication.
|
||
|
|
--realm string realm for authentication (default "rclone")
|
||
|
|
--server-read-timeout duration Timeout for server reading data (default 1h0m0s)
|
||
|
|
--server-write-timeout duration Timeout for server writing data (default 1h0m0s)
|
||
|
|
--stdio run an HTTP2 server on stdin/stdout
|
||
|
|
--user string User name for authentication.
|
||
|
|
```
|
||
|
|
|
||
|
|
### Options inherited from parent commands
|
||
|
|
|
||
|
|
```
|
||
|
|
--acd-templink-threshold int Files >= this size will be downloaded via their tempLink. (default 9G)
|
||
|
|
--acd-upload-wait-per-gb duration Additional time per GB to wait after a failed complete upload to see if it appears. (default 3m0s)
|
||
|
|
--ask-password Allow prompt for password for encrypted configuration. (default true)
|
||
|
|
--auto-confirm If enabled, do not request console confirmation.
|
||
|
|
--azureblob-chunk-size int Upload chunk size. Must fit in memory. (default 4M)
|
||
|
|
--azureblob-upload-cutoff int Cutoff for switching to chunked upload (default 256M)
|
||
|
|
--b2-chunk-size int Upload chunk size. Must fit in memory. (default 96M)
|
||
|
|
--b2-hard-delete Permanently delete files on remote removal, otherwise hide files.
|
||
|
|
--b2-test-mode string A flag string for X-Bz-Test-Mode header.
|
||
|
|
--b2-upload-cutoff int Cutoff for switching to chunked upload (default 190.735M)
|
||
|
|
--b2-versions Include old versions in directory listings.
|
||
|
|
--backup-dir string Make backups into hierarchy based in DIR.
|
||
|
|
--bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name.
|
||
|
|
--box-upload-cutoff int Cutoff for switching to multipart upload (default 50M)
|
||
|
|
--buffer-size int Buffer size when copying files. (default 16M)
|
||
|
|
--bwlimit BwTimetable Bandwidth limit in kBytes/s, or use suffix b|k|M|G or a full timetable.
|
||
|
|
--cache-chunk-clean-interval string Interval at which chunk cleanup runs (default "1m")
|
||
|
|
--cache-chunk-no-memory Disable the in-memory cache for storing chunks during streaming
|
||
|
|
--cache-chunk-path string Directory to cached chunk files (default "/home/ncw/.cache/rclone/cache-backend")
|
||
|
|
--cache-chunk-size string The size of a chunk (default "5M")
|
||
|
|
--cache-db-path string Directory to cache DB (default "/home/ncw/.cache/rclone/cache-backend")
|
||
|
|
--cache-db-purge Purge the cache DB before
|
||
|
|
--cache-db-wait-time duration How long to wait for the DB to be available - 0 is unlimited (default 1s)
|
||
|
|
--cache-dir string Directory rclone will use for caching. (default "/home/ncw/.cache/rclone")
|
||
|
|
--cache-info-age string How much time should object info be stored in cache (default "6h")
|
||
|
|
--cache-read-retries int How many times to retry a read from a cache storage (default 10)
|
||
|
|
--cache-rps int Limits the number of requests per second to the source FS. -1 disables the rate limiter (default -1)
|
||
|
|
--cache-tmp-upload-path string Directory to keep temporary files until they are uploaded to the cloud storage
|
||
|
|
--cache-tmp-wait-time string How long should files be stored in local cache before being uploaded (default "15m")
|
||
|
|
--cache-total-chunk-size string The total size which the chunks can take up from the disk (default "10G")
|
||
|
|
--cache-workers int How many workers should run in parallel to download chunks (default 4)
|
||
|
|
--cache-writes Will cache file data on writes through the FS
|
||
|
|
--checkers int Number of checkers to run in parallel. (default 8)
|
||
|
|
-c, --checksum Skip based on checksum & size, not mod-time & size
|
||
|
|
--config string Config file. (default "/home/ncw/.rclone.conf")
|
||
|
|
--contimeout duration Connect timeout (default 1m0s)
|
||
|
|
-L, --copy-links Follow symlinks and copy the pointed to item.
|
||
|
|
--cpuprofile string Write cpu profile to file
|
||
|
|
--crypt-show-mapping For all files listed show how the names encrypt.
|
||
|
|
--delete-after When synchronizing, delete files on destination after transfering
|
||
|
|
--delete-before When synchronizing, delete files on destination before transfering
|
||
|
|
--delete-during When synchronizing, delete files during transfer (default)
|
||
|
|
--delete-excluded Delete files on dest excluded from sync
|
||
|
|
--disable string Disable a comma separated list of features. Use help to see a list.
|
||
|
|
--drive-auth-owner-only Only consider files owned by the authenticated user.
|
||
|
|
--drive-chunk-size int Upload chunk size. Must a power of 2 >= 256k. (default 8M)
|
||
|
|
--drive-formats string Comma separated list of preferred formats for downloading Google docs. (default "docx,xlsx,pptx,svg")
|
||
|
|
--drive-impersonate string Impersonate this user when using a service account.
|
||
|
|
--drive-list-chunk int Size of listing chunk 100-1000. 0 to disable. (default 1000)
|
||
|
|
--drive-shared-with-me Only show files that are shared with me
|
||
|
|
--drive-skip-gdocs Skip google documents in all listings.
|
||
|
|
--drive-trashed-only Only show files that are in the trash
|
||
|
|
--drive-upload-cutoff int Cutoff for switching to chunked upload (default 8M)
|
||
|
|
--drive-use-created-date Use created date instead of modified date.
|
||
|
|
--drive-use-trash Send files to the trash instead of deleting permanently. (default true)
|
||
|
|
--dropbox-chunk-size int Upload chunk size. Max 150M. (default 48M)
|
||
|
|
-n, --dry-run Do a trial run with no permanent changes
|
||
|
|
--dump string List of items to dump from: headers,bodies,requests,responses,auth,filters
|
||
|
|
--dump-bodies Dump HTTP headers and bodies - may contain sensitive info
|
||
|
|
--dump-headers Dump HTTP bodies - may contain sensitive info
|
||
|
|
--exclude stringArray Exclude files matching pattern
|
||
|
|
--exclude-from stringArray Read exclude patterns from file
|
||
|
|
--exclude-if-present string Exclude directories if filename is present
|
||
|
|
--fast-list Use recursive list if available. Uses more memory but fewer transactions.
|
||
|
|
--files-from stringArray Read list of source-file names from file
|
||
|
|
-f, --filter stringArray Add a file-filtering rule
|
||
|
|
--filter-from stringArray Read filtering patterns from a file
|
||
|
|
--gcs-location string Default location for buckets (us|eu|asia|us-central1|us-east1|us-east4|us-west1|asia-east1|asia-noetheast1|asia-southeast1|australia-southeast1|europe-west1|europe-west2).
|
||
|
|
--gcs-storage-class string Default storage class for buckets (MULTI_REGIONAL|REGIONAL|STANDARD|NEARLINE|COLDLINE|DURABLE_REDUCED_AVAILABILITY).
|
||
|
|
--ignore-checksum Skip post copy check of checksums.
|
||
|
|
--ignore-existing Skip all files that exist on destination
|
||
|
|
--ignore-size Ignore size when skipping use mod-time or checksum.
|
||
|
|
-I, --ignore-times Don't skip files that match size and time - transfer all files
|
||
|
|
--immutable Do not modify files. Fail if existing files have been modified.
|
||
|
|
--include stringArray Include files matching pattern
|
||
|
|
--include-from stringArray Read include patterns from file
|
||
|
|
--local-no-unicode-normalization Don't apply unicode normalization to paths and filenames
|
||
|
|
--log-file string Log everything to this file
|
||
|
|
--log-level string Log level DEBUG|INFO|NOTICE|ERROR (default "NOTICE")
|
||
|
|
--low-level-retries int Number of low level retries to do. (default 10)
|
||
|
|
--max-age duration Don't transfer any file older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
|
||
|
|
--max-delete int When synchronizing, limit the number of deletes (default -1)
|
||
|
|
--max-depth int If set limits the recursion depth to this. (default -1)
|
||
|
|
--max-size int Don't transfer any file larger than this in k or suffix b|k|M|G (default off)
|
||
|
|
--memprofile string Write memory profile to file
|
||
|
|
--min-age duration Don't transfer any file younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
|
||
|
|
--min-size int Don't transfer any file smaller than this in k or suffix b|k|M|G (default off)
|
||
|
|
--modify-window duration Max time diff to be considered the same (default 1ns)
|
||
|
|
--no-check-certificate Do not verify the server SSL certificate. Insecure.
|
||
|
|
--no-gzip-encoding Don't set Accept-Encoding: gzip.
|
||
|
|
--no-traverse Obsolete - does nothing.
|
||
|
|
--no-update-modtime Don't update destination mod-time if files identical.
|
||
|
|
-x, --one-file-system Don't cross filesystem boundaries.
|
||
|
|
--onedrive-chunk-size int Above this size files will be chunked - must be multiple of 320k. (default 10M)
|
||
|
|
-q, --quiet Print as little stuff as possible
|
||
|
|
--rc Enable the remote control server.
|
||
|
|
--rc-addr string IPaddress:Port or :Port to bind server to. (default "localhost:5572")
|
||
|
|
--rc-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
||
|
|
--rc-client-ca string Client certificate authority to verify clients with
|
||
|
|
--rc-htpasswd string htpasswd file - if not provided no authentication is done
|
||
|
|
--rc-key string SSL PEM Private key
|
||
|
|
--rc-max-header-bytes int Maximum size of request header (default 4096)
|
||
|
|
--rc-pass string Password for authentication.
|
||
|
|
--rc-realm string realm for authentication (default "rclone")
|
||
|
|
--rc-server-read-timeout duration Timeout for server reading data (default 1h0m0s)
|
||
|
|
--rc-server-write-timeout duration Timeout for server writing data (default 1h0m0s)
|
||
|
|
--rc-user string User name for authentication.
|
||
|
|
--retries int Retry operations this many times if they fail (default 3)
|
||
|
|
--s3-acl string Canned ACL used when creating buckets and/or storing objects in S3
|
||
|
|
--s3-storage-class string Storage class to use when uploading S3 objects (STANDARD|REDUCED_REDUNDANCY|STANDARD_IA)
|
||
|
|
--sftp-ask-password Allow asking for SFTP password when needed.
|
||
|
|
--size-only Skip based on size only, not mod-time or checksum
|
||
|
|
--skip-links Don't warn about skipped symlinks.
|
||
|
|
--stats duration Interval between printing stats, e.g 500ms, 60s, 5m. (0 to disable) (default 1m0s)
|
||
|
|
--stats-file-name-length int Max file name length in stats. 0 for no limit (default 40)
|
||
|
|
--stats-log-level string Log level to show --stats output DEBUG|INFO|NOTICE|ERROR (default "INFO")
|
||
|
|
--stats-unit string Show data rate in stats as either 'bits' or 'bytes'/s (default "bytes")
|
||
|
|
--streaming-upload-cutoff int Cutoff for switching to chunked upload if file size is unknown. Upload starts after reaching cutoff or when file ends. (default 100k)
|
||
|
|
--suffix string Suffix for use with --backup-dir.
|
||
|
|
--swift-chunk-size int Above this size files will be chunked into a _segments container. (default 5G)
|
||
|
|
--syslog Use Syslog for logging
|
||
|
|
--syslog-facility string Facility for syslog, eg KERN,USER,... (default "DAEMON")
|
||
|
|
--timeout duration IO idle timeout (default 5m0s)
|
||
|
|
--tpslimit float Limit HTTP transactions per second to this.
|
||
|
|
--tpslimit-burst int Max burst of transactions for --tpslimit. (default 1)
|
||
|
|
--track-renames When synchronizing, track file renames and do a server side move if possible
|
||
|
|
--transfers int Number of file transfers to run in parallel. (default 4)
|
||
|
|
-u, --update Skip files that are newer on the destination.
|
||
|
|
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.40")
|
||
|
|
-v, --verbose count Print lots more stuff (repeat for more)
|
||
|
|
```
|
||
|
|
|
||
|
|
### SEE ALSO
|
||
|
|
|
||
|
|
* [rclone serve](/commands/rclone_serve/) - Serve a remote over a protocol.
|
||
|
|
|
||
|
|
###### Auto generated by spf13/cobra on 19-Mar-2018
|