Merge pull request #5016 from MichaelEischer/s3-doc-rework

Rework documentation for s3-compatible storages

doc/030_preparing_a_new_repo.rst

@@ -249,28 +249,22 @@ while creating the bucket.
     $ export AWS_ACCESS_KEY_ID=<MY_ACCESS_KEY>
     $ export AWS_SECRET_ACCESS_KEY=<MY_SECRET_ACCESS_KEY>
 
+When using temporary credentials make sure to include the session token via
+the environment variable ``AWS_SESSION_TOKEN``.
+
 You can then easily initialize a repository that uses your Amazon S3 as
-a backend. If the bucket does not exist it will be created in the
+a backend. Make sure to use the endpoint for the correct region. The example
-default location:
+uses ``us-east-1``. If the bucket does not exist it will be created in that region:
 
 .. code-block:: console
 
-    $ restic -r s3:s3.amazonaws.com/bucket_name init
+    $ restic -r s3:s3.us-east-1.amazonaws.com/bucket_name init
     enter password for new repository:
     enter password again:
-    created restic repository eefee03bbd at s3:s3.amazonaws.com/bucket_name
+    created restic repository eefee03bbd at s3:s3.us-east-1.amazonaws.com/bucket_name
     Please note that knowledge of your password is required to access the repository.
     Losing your password means that your data is irrecoverably lost.
 
-If needed, you can manually specify the region to use by either setting the
-environment variable ``AWS_DEFAULT_REGION`` or calling restic with an option
-parameter like ``-o s3.region="us-east-1"``. If the region is not specified,
-the default region is used. Afterwards, the S3 server (at least for AWS,
-``s3.amazonaws.com``) will redirect restic to the correct endpoint.
-
-When using temporary credentials make sure to include the session token via
-then environment variable ``AWS_SESSION_TOKEN``.
-
 Until version 0.8.0, restic used a default prefix of ``restic``, so the files
 in the bucket were placed in a directory named ``restic``. If you want to
 access a repository created with an older version of restic, specify the path
@@ -278,25 +272,14 @@ after the bucket name like this:
 
 .. code-block:: console
 
-    $ restic -r s3:s3.amazonaws.com/bucket_name/restic [...]
+    $ restic -r s3:s3.us-east-1.amazonaws.com/bucket_name/restic [...]
-
-For an S3-compatible server that is not Amazon (like Minio, see below),
-or is only available via HTTP, you can specify the URL to the server
-like this: ``s3:http://server:port/bucket_name``.
 
 .. note:: restic expects `path-style URLs <https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html>`__
-          like for example ``s3.us-west-2.amazonaws.com/bucket_name``.
+          like for example ``s3.us-west-2.amazonaws.com/bucket_name`` for Amazon S3.
           Virtual-hosted–style URLs like ``bucket_name.s3.us-west-2.amazonaws.com``,
           where the bucket name is part of the hostname are not supported. These must
           be converted to path-style URLs instead, for example ``s3.us-west-2.amazonaws.com/bucket_name``.
-
+          See below for configuration options for S3-compatible storage from other providers.
-.. note:: Certain S3-compatible servers do not properly implement the
-          ``ListObjectsV2`` API, most notably Ceph versions before v14.2.5. On these
-          backends, as a temporary workaround, you can provide the
-          ``-o s3.list-objects-v1=true`` option to use the older
-          ``ListObjects`` API instead. This option may be removed in future
-          versions of restic.
-
 
 Minio Server
 ************
@@ -321,81 +304,66 @@ this command.
 
 .. code-block:: console
 
-    $ ./restic -r s3:http://localhost:9000/restic init
+    $ restic -r s3:http://localhost:9000/restic init
     enter password for new repository:
     enter password again:
-    created restic repository 6ad29560f5 at s3:http://localhost:9000/restic1
+    created restic repository 6ad29560f5 at s3:http://localhost:9000/restic
     Please note that knowledge of your password is required to access
     the repository. Losing your password means that your data is irrecoverably lost.
 
+S3-compatible Storage
+*********************
+
+For an S3-compatible server that is not Amazon, you can specify the URL to the server
+like this: ``s3:https://server:port/bucket_name``.
+
+If needed, you can manually specify the region to use by either setting the
+environment variable ``AWS_DEFAULT_REGION`` or calling restic with an option
+parameter like ``-o s3.region="us-east-1"``. If the region is not specified,
+the default region ``us-east-1`` is used.
+
+To select between path-style and virtual-hosted access, the extended option
+``-o s3.bucket-lookup=auto`` can be used. It supports the following values:
+
+- ``auto``: Default behavior. Uses ``dns`` for Amazon and Google endpoints. Uses
+  ``path`` for all other endpoints
+- ``dns``: Use virtual-hosted-style bucket access
+- ``path``: Use path-style bucket access
+
+Certain S3-compatible servers do not properly implement the ``ListObjectsV2`` API,
+most notably Ceph versions before v14.2.5. On these backends, as a temporary
+workaround, you can provide the ``-o s3.list-objects-v1=true`` option to use the
+older ``ListObjects`` API instead. This option may be removed in future versions
+of restic.
+
 Wasabi
-************
+******
 
-`Wasabi <https://wasabi.com>`__ is a low cost Amazon S3 conformant object storage provider.
+S3 storage from `Wasabi <https://wasabi.com>`__ can be used as follows.
-Due to its S3 conformance, Wasabi can be used as a storage provider for a restic repository.
 
--  Create a Wasabi bucket using the `Wasabi Console <https://console.wasabisys.com>`__.
 - Determine the correct Wasabi service URL for your bucket `here <https://wasabi-support.zendesk.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions->`__.
-
+- Set environment variables with the necessary account credentials
-You must first setup the following environment variables with the
-credentials of your Wasabi account.
 
 .. code-block:: console
 
     $ export AWS_ACCESS_KEY_ID=<YOUR-WASABI-ACCESS-KEY-ID>
     $ export AWS_SECRET_ACCESS_KEY=<YOUR-WASABI-SECRET-ACCESS-KEY>
-
+    $ restic -r s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME> init
-Now you can easily initialize restic to use Wasabi as a backend with
-this command.
-
-.. code-block:: console
-
-    $ ./restic -r s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME> init
-    enter password for new repository:
-    enter password again:
-    created restic repository xxxxxxxxxx at s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME>
-    Please note that knowledge of your password is required to access
-    the repository. Losing your password means that your data is irrecoverably lost.
 
 Alibaba Cloud (Aliyun) Object Storage System (OSS)
 **************************************************
 
-`Alibaba OSS <https://www.alibabacloud.com/product/object-storage-service>`__ is an
+S3 storage from `Alibaba OSS <https://www.alibabacloud.com/product/object-storage-service>`__ can be used as follows.
-encrypted, secure, cost-effective, and easy-to-use object storage
-service that enables you to store, back up, and archive large amounts
-of data in the cloud.
-
-Alibaba OSS is S3 compatible so it can be used as a storage provider
-for a restic repository with a couple of extra parameters.
 
 - Determine the correct `Alibaba OSS region endpoint <https://www.alibabacloud.com/help/en/object-storage-service/latest/regions-and-endpoints>`__ - this will be something like ``oss-eu-west-1.aliyuncs.com``
--  You'll need the region name too - this will be something like ``oss-eu-west-1``
+- You will need the region name too - this will be something like ``oss-eu-west-1``
-
+- Set environment variables with the necessary account credentials
-You must first setup the following environment variables with the
-credentials of your Alibaba OSS account.
 
 .. code-block:: console
 
     $ export AWS_ACCESS_KEY_ID=<YOUR-OSS-ACCESS-KEY-ID>
     $ export AWS_SECRET_ACCESS_KEY=<YOUR-OSS-SECRET-ACCESS-KEY>
-
+    $ restic -o s3.bucket-lookup=dns -o s3.region=<OSS-REGION> -r s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME> init
-Now you can easily initialize restic to use Alibaba OSS as a backend with
-this command.
-
-.. code-block:: console
-
-    $ ./restic -o s3.bucket-lookup=dns -o s3.region=<OSS-REGION> -r s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME> init
-    enter password for new backend:
-    enter password again:
-    created restic backend xxxxxxxxxx at s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME>
-    Please note that knowledge of your password is required to access
-    the repository. Losing your password means that your data is irrecoverably lost.
-
-For example with an actual endpoint:
-
-.. code-block:: console
-
-    $ restic -o s3.bucket-lookup=dns -o s3.region=oss-eu-west-1 -r s3:https://oss-eu-west-1.aliyuncs.com/bucketname init
 
 OpenStack Swift
 ***************

doc/manual_rest.rst

@@ -8,7 +8,7 @@ Usage help is available:
 
 .. code-block:: console
 
-    $ ./restic --help
+    $ restic --help
 
     restic is a backup program which allows saving multiple revisions of files and
     directories in an encrypted repository stored on different backends.
@@ -91,7 +91,7 @@ command:
 
 .. code-block:: console
 
-    $ ./restic backup --help
+    $ restic backup --help
 
     The "backup" command creates a new snapshot and saves the files and directories
     given as the arguments.