alexvanin/restic
1
0
Fork 0
forked from TrueCloudLab/restic
Code Pull requests Activity

Merge pull request #5016 from MichaelEischer/s3-doc-rework

Browse source 
Rework documentation for s3-compatible storages
This commit is contained in:
Michael Eischer 2024-08-29 16:55:40 +02:00 committed by GitHub
commit 5c238ea359
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 49 additions and 81 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

126
doc/030_preparing_a_new_repo.rst
View file

@ -249,28 +249,22 @@ while creating the bucket.
$ export AWS_ACCESS_KEY_ID=<MY_ACCESS_KEY> $ export AWS_ACCESS_KEY_ID=<MY_ACCESS_KEY>
$ export AWS_SECRET_ACCESS_KEY=<MY_SECRET_ACCESS_KEY> $ export AWS_SECRET_ACCESS_KEY=<MY_SECRET_ACCESS_KEY>
When using temporary credentials make sure to include the session token via
the environment variable ``AWS_SESSION_TOKEN``.
You can then easily initialize a repository that uses your Amazon S3 as You can then easily initialize a repository that uses your Amazon S3 as
a backend. If the bucket does not exist it will be created in the a backend. Make sure to use the endpoint for the correct region. The example
default location: uses ``us-east-1``. If the bucket does not exist it will be created in that region:
.. code-block:: console .. code-block:: console
$ restic -r s3:s3.amazonaws.com/bucket_name init $ restic -r s3:s3.us-east-1.amazonaws.com/bucket_name init
enter password for new repository: enter password for new repository:
enter password again: enter password again:
created restic repository eefee03bbd at s3:s3.amazonaws.com/bucket_name created restic repository eefee03bbd at s3:s3.us-east-1.amazonaws.com/bucket_name
Please note that knowledge of your password is required to access the repository. Please note that knowledge of your password is required to access the repository.
Losing your password means that your data is irrecoverably lost. Losing your password means that your data is irrecoverably lost.
If needed, you can manually specify the region to use by either setting the
environment variable ``AWS_DEFAULT_REGION`` or calling restic with an option
parameter like ``-o s3.region="us-east-1"``. If the region is not specified,
the default region is used. Afterwards, the S3 server (at least for AWS,
``s3.amazonaws.com``) will redirect restic to the correct endpoint.
When using temporary credentials make sure to include the session token via
then environment variable ``AWS_SESSION_TOKEN``.
Until version 0.8.0, restic used a default prefix of ``restic``, so the files Until version 0.8.0, restic used a default prefix of ``restic``, so the files
in the bucket were placed in a directory named ``restic``. If you want to in the bucket were placed in a directory named ``restic``. If you want to
access a repository created with an older version of restic, specify the path access a repository created with an older version of restic, specify the path
@ -278,25 +272,14 @@ after the bucket name like this:
.. code-block:: console .. code-block:: console
$ restic -r s3:s3.amazonaws.com/bucket_name/restic [...] $ restic -r s3:s3.us-east-1.amazonaws.com/bucket_name/restic [...]
For an S3-compatible server that is not Amazon (like Minio, see below),
or is only available via HTTP, you can specify the URL to the server
like this: ``s3:http://server:port/bucket_name``.
.. note:: restic expects `path-style URLs <https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html>`__ .. note:: restic expects `path-style URLs <https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html>`__
like for example ``s3.us-west-2.amazonaws.com/bucket_name``. like for example ``s3.us-west-2.amazonaws.com/bucket_name`` for Amazon S3.
Virtual-hosted–style URLs like ``bucket_name.s3.us-west-2.amazonaws.com``, Virtual-hosted–style URLs like ``bucket_name.s3.us-west-2.amazonaws.com``,
where the bucket name is part of the hostname are not supported. These must where the bucket name is part of the hostname are not supported. These must
be converted to path-style URLs instead, for example ``s3.us-west-2.amazonaws.com/bucket_name``. be converted to path-style URLs instead, for example ``s3.us-west-2.amazonaws.com/bucket_name``.
See below for configuration options for S3-compatible storage from other providers.
.. note:: Certain S3-compatible servers do not properly implement the
``ListObjectsV2`` API, most notably Ceph versions before v14.2.5. On these
backends, as a temporary workaround, you can provide the
``-o s3.list-objects-v1=true`` option to use the older
``ListObjects`` API instead. This option may be removed in future
versions of restic.
Minio Server Minio Server
************ ************
@ -321,81 +304,66 @@ this command.
.. code-block:: console .. code-block:: console
$ ./restic -r s3:http://localhost:9000/restic init $ restic -r s3:http://localhost:9000/restic init
enter password for new repository: enter password for new repository:
enter password again: enter password again:
created restic repository 6ad29560f5 at s3:http://localhost:9000/restic1 created restic repository 6ad29560f5 at s3:http://localhost:9000/restic
Please note that knowledge of your password is required to access Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is irrecoverably lost. the repository. Losing your password means that your data is irrecoverably lost.
S3-compatible Storage
*********************
For an S3-compatible server that is not Amazon, you can specify the URL to the server
like this: ``s3:https://server:port/bucket_name``.
If needed, you can manually specify the region to use by either setting the
environment variable ``AWS_DEFAULT_REGION`` or calling restic with an option
parameter like ``-o s3.region="us-east-1"``. If the region is not specified,
the default region ``us-east-1`` is used.
To select between path-style and virtual-hosted access, the extended option
``-o s3.bucket-lookup=auto`` can be used. It supports the following values:
- ``auto``: Default behavior. Uses ``dns`` for Amazon and Google endpoints. Uses
``path`` for all other endpoints
- ``dns``: Use virtual-hosted-style bucket access
- ``path``: Use path-style bucket access
Certain S3-compatible servers do not properly implement the ``ListObjectsV2`` API,
most notably Ceph versions before v14.2.5. On these backends, as a temporary
workaround, you can provide the ``-o s3.list-objects-v1=true`` option to use the
older ``ListObjects`` API instead. This option may be removed in future versions
of restic.
Wasabi Wasabi
************ ******
`Wasabi <https://wasabi.com>`__ is a low cost Amazon S3 conformant object storage provider. S3 storage from `Wasabi <https://wasabi.com>`__ can be used as follows.
Due to its S3 conformance, Wasabi can be used as a storage provider for a restic repository.
- Create a Wasabi bucket using the `Wasabi Console <https://console.wasabisys.com>`__. - Determine the correct Wasabi service URL for your bucket `here <https://wasabi-support.zendesk.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions->`__.
- Determine the correct Wasabi service URL for your bucket `here <https://wasabi-support.zendesk.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions->`__. - Set environment variables with the necessary account credentials
You must first setup the following environment variables with the
credentials of your Wasabi account.
.. code-block:: console .. code-block:: console
$ export AWS_ACCESS_KEY_ID=<YOUR-WASABI-ACCESS-KEY-ID> $ export AWS_ACCESS_KEY_ID=<YOUR-WASABI-ACCESS-KEY-ID>
$ export AWS_SECRET_ACCESS_KEY=<YOUR-WASABI-SECRET-ACCESS-KEY> $ export AWS_SECRET_ACCESS_KEY=<YOUR-WASABI-SECRET-ACCESS-KEY>
$ restic -r s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME> init
Now you can easily initialize restic to use Wasabi as a backend with
this command.
.. code-block:: console
$ ./restic -r s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME> init
enter password for new repository:
enter password again:
created restic repository xxxxxxxxxx at s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME>
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is irrecoverably lost.
Alibaba Cloud (Aliyun) Object Storage System (OSS) Alibaba Cloud (Aliyun) Object Storage System (OSS)
************************************************** **************************************************
`Alibaba OSS <https://www.alibabacloud.com/product/object-storage-service>`__ is an S3 storage from `Alibaba OSS <https://www.alibabacloud.com/product/object-storage-service>`__ can be used as follows.
encrypted, secure, cost-effective, and easy-to-use object storage
service that enables you to store, back up, and archive large amounts
of data in the cloud.
Alibaba OSS is S3 compatible so it can be used as a storage provider - Determine the correct `Alibaba OSS region endpoint <https://www.alibabacloud.com/help/en/object-storage-service/latest/regions-and-endpoints>`__ - this will be something like ``oss-eu-west-1.aliyuncs.com``
for a restic repository with a couple of extra parameters. - You will need the region name too - this will be something like ``oss-eu-west-1``
- Set environment variables with the necessary account credentials
- Determine the correct `Alibaba OSS region endpoint <https://www.alibabacloud.com/help/en/object-storage-service/latest/regions-and-endpoints>`__ - this will be something like ``oss-eu-west-1.aliyuncs.com``
- You'll need the region name too - this will be something like ``oss-eu-west-1``
You must first setup the following environment variables with the
credentials of your Alibaba OSS account.
.. code-block:: console .. code-block:: console
$ export AWS_ACCESS_KEY_ID=<YOUR-OSS-ACCESS-KEY-ID> $ export AWS_ACCESS_KEY_ID=<YOUR-OSS-ACCESS-KEY-ID>
$ export AWS_SECRET_ACCESS_KEY=<YOUR-OSS-SECRET-ACCESS-KEY> $ export AWS_SECRET_ACCESS_KEY=<YOUR-OSS-SECRET-ACCESS-KEY>
$ restic -o s3.bucket-lookup=dns -o s3.region=<OSS-REGION> -r s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME> init
Now you can easily initialize restic to use Alibaba OSS as a backend with
this command.
.. code-block:: console
$ ./restic -o s3.bucket-lookup=dns -o s3.region=<OSS-REGION> -r s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME> init
enter password for new backend:
enter password again:
created restic backend xxxxxxxxxx at s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME>
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is irrecoverably lost.
For example with an actual endpoint:
.. code-block:: console
$ restic -o s3.bucket-lookup=dns -o s3.region=oss-eu-west-1 -r s3:https://oss-eu-west-1.aliyuncs.com/bucketname init
OpenStack Swift OpenStack Swift
*************** ***************

4
doc/manual_rest.rst
View file

@ -8,7 +8,7 @@ Usage help is available:
.. code-block:: console .. code-block:: console
$ ./restic --help $ restic --help
restic is a backup program which allows saving multiple revisions of files and restic is a backup program which allows saving multiple revisions of files and
directories in an encrypted repository stored on different backends. directories in an encrypted repository stored on different backends.
@ -91,7 +91,7 @@ command:
.. code-block:: console .. code-block:: console
$ ./restic backup --help $ restic backup --help
The "backup" command creates a new snapshot and saves the files and directories The "backup" command creates a new snapshot and saves the files and directories
given as the arguments. given as the arguments.