diff --git a/docs/compatibility.md b/docs/compatibility.md index cba7e378..d162d8d4 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -1,84 +1,80 @@ - +--- +description: describes get by digest pitfall +keywords: registry, manifest, images, tags, repository, distribution, digest +title: Registry compatibility +--- -# Registry Compatibility +{% include registry.md %} ## Synopsis -*If a manifest is pulled by _digest_ from a registry 2.3 with Docker Engine 1.9 +If a manifest is pulled by _digest_ from a registry 2.3 with Docker Engine 1.9 and older, and the manifest was pushed with Docker Engine 1.10, a security check -will cause the Engine to receive a manifest it cannot use and the pull will fail.* +causes the Engine to receive a manifest it cannot use and the pull fails. -## Registry Manifest Support +## Registry manifest support Historically, the registry has supported a [single manifest type](./spec/manifest-v2-1.md) known as _Schema 1_. -With the move toward multiple architecture images the distribution project -introduced two new manifest types: Schema 2 manifests and manifest lists. The -registry 2.3 supports all three manifest types and in order to be compatible -with older Docker engines will, in certain cases, do an on-the-fly -transformation of a manifest before serving the JSON in the response. +With the move toward multiple architecture images, the distribution project +introduced two new manifest types: Schema 2 manifests and manifest lists. Registry +2.3 supports all three manifest types and sometimes performs an on-the-fly +transformation of a manifest before serving the JSON in the response, to +preserve compatibility with older versions of Docker Engine. This conversion has some implications for pulling manifests by digest and this -document enumerate these implications. +document enumerates these implications. ## Content Addressable Storage (CAS) Manifests are stored and retrieved in the registry by keying off a digest -representing a hash of the contents. One of the advantages provided by CAS is -security: if the contents are changed, then the digest will no longer match. +representing a hash of the contents. One of the advantages provided by CAS is +security: if the contents are changed, then the digest no longer matches. This prevents any modification of the manifest by a MITM attack or an untrusted third party. When a manifest is stored by the registry, this digest is returned in the HTTP -response headers and, if events are configured, delivered within the event. The +response headers and, if events are configured, delivered within the event. The manifest can either be retrieved by the tag, or this digest. -For registry versions 2.2.1 and below, the registry will always store and -serve _Schema 1_ manifests. The Docker Engine 1.10 will first -attempt to send a _Schema 2_ manifest, falling back to sending a +For registry versions 2.2.1 and below, the registry always stores and +serves _Schema 1_ manifests. Engine 1.10 first +attempts to send a _Schema 2_ manifest, falling back to sending a Schema 1 type manifest when it detects that the registry does not support the new version. ## Registry v2.3 -### Manifest Push with Docker 1.9 and Older +### Manifest push with Docker 1.10 -The Docker Engine will construct a _Schema 1_ manifest which the -registry will persist to disk. - -When the manifest is pulled by digest or tag with any docker version, a -_Schema 1_ manifest will be returned. - -### Manifest Push with Docker 1.10 - -The docker engine will construct a _Schema 2_ manifest which the -registry will persist to disk. +The Engine constructs a _Schema 2_ manifest which the +registry persists to disk. When the manifest is pulled by digest or tag with Docker Engine 1.10, a -_Schema 2_ manifest will be returned. The Docker Engine 1.10 +_Schema 2_ manifest is returned. Docker Engine 1.10 understands the new manifest format. When the manifest is pulled by *tag* with Docker Engine 1.9 and older, the manifest is converted on-the-fly to _Schema 1_ and sent in the -response. The Docker Engine 1.9 is compatible with this older format. +response. The Docker Engine 1.9 is compatible with this older format. -*When the manifest is pulled by _digest_ with Docker Engine 1.9 and older, the -same rewriting process will not happen in the registry. If this were to happen +When the manifest is pulled by _digest_ with Docker Engine 1.9 and older, the +same rewriting process does not happen in the registry. If it did, the digest would no longer match the hash of the manifest and would violate the -constraints of CAS.* +constraints of CAS. For this reason if a manifest is pulled by _digest_ from a registry 2.3 with Docker Engine 1.9 and older, and the manifest was pushed with Docker Engine 1.10, a -security check will cause the Engine to receive a manifest it cannot use and the -pull will fail. +security check causes the Engine to receive a manifest it cannot use and the +pull fails. + +### Manifest push with Docker 1.9 and older + +The Docker Engine constructs a _Schema 1_ manifest which the +registry persists to disk. + +When the manifest is pulled by digest or tag with any Docker version, a +_Schema 1_ manifest is returned. + diff --git a/docs/deploying.md b/docs/deploying.md index 2e8ce69e..f9e77d4f 100644 --- a/docs/deploying.md +++ b/docs/deploying.md @@ -1,103 +1,339 @@ - +--- +description: Explains how to deploy a registry +keywords: registry, on-prem, images, tags, repository, distribution, deployment +title: Deploy a registry server +--- -# Deploying a registry server +{% include registry.md %} -You need to [install Docker version 1.6.0 or newer](/engine/installation/index.md). +Before you can deploy a registry, you need to install Docker on the host. +A registry is an instance of the `registry` image, and runs within Docker. -## Running on localhost +This topic provides basic information about deploying and configuring a +registry. For an exhaustive list of configuration options, see the +[configuration reference](configuration.md). -Start your registry: +If you have an air-gapped datacenter, see +[Considerations for air-gapped registries](#considerations-for-air-gapped-registries). - docker run -d -p 5000:5000 --restart=always --name registry registry:2 +## Run a local registry -You can now use it with docker. +Use a command like the following to start the registry container: -Get any image from the hub and tag it to point to your registry: +```console +$ docker run -d -p 5000:5000 --restart=always --name registry registry:2 +``` - docker pull ubuntu && docker tag ubuntu localhost:5000/ubuntu +The registry is now ready to use. -... then push it to your registry: +> **Warning**: These first few examples show registry configurations that are +> only appropriate for testing. A production-ready registry must be protected by +> TLS and should ideally use an access-control mechanism. Keep reading and then +> continue to the [configuration guide](configuration.md) to deploy a +> production-ready registry. - docker push localhost:5000/ubuntu +## Copy an image from Docker Hub to your registry -... then pull it back from your registry: +You can pull an image from Docker Hub and push it to your registry. The +following example pulls the `ubuntu:16.04` image from Docker Hub and re-tags it +as `my-ubuntu`, then pushes it to the local registry. Finally, the +`ubuntu:16.04` and `my-ubuntu` images are deleted locally and the +`my-ubuntu` image is pulled from the local registry. - docker pull localhost:5000/ubuntu +1. Pull the `ubuntu:16.04` image from Docker Hub. -To stop your registry, you would: + ```console + $ docker pull ubuntu:16.04 + ``` - docker stop registry && docker rm -v registry +2. Tag the image as `localhost:5000/my-ubuntu`. This creates an additional tag + for the existing image. When the first part of the tag is a hostname and + port, Docker interprets this as the location of a registry, when pushing. -## Storage + ```console + $ docker tag ubuntu:16.04 localhost:5000/my-ubuntu + ``` -By default, your registry data is persisted as a [docker volume](/engine/tutorials/dockervolumes.md) on the host filesystem. Properly understanding volumes is essential if you want to stick with a local filesystem storage. +3. Push the image to the local registry running at `localhost:5000`: -Specifically, you might want to point your volume location to a specific place in order to more easily access your registry data. To do so you can: + ```console + $ docker push localhost:5000/my-ubuntu + ``` - docker run -d -p 5000:5000 --restart=always --name registry \ - -v `pwd`/data:/var/lib/registry \ - registry:2 +4. Remove the locally-cached `ubuntu:16.04` and `localhost:5000/my-ubuntu` + images, so that you can test pulling the image from your registry. This + does not remove the `localhost:5000/my-ubuntu` image from your registry. -### Alternatives + ```console + $ docker image remove ubuntu:16.04 + $ docker image remove localhost:5000/my-ubuntu + ``` -You should usually consider using [another storage backend](./storage-drivers/index.md) instead of the local filesystem. Use the [storage configuration options](./configuration.md#storage) to configure an alternate storage backend. +5. Pull the `localhost:5000/my-ubuntu` image from your local registry. -Using one of these will allow you to more easily scale your registry, and leverage your storage redundancy and availability features. + ```console + $ docker pull localhost:5000/my-ubuntu + ``` -## Running a domain registry +## Stop a local registry -While running on `localhost` has its uses, most people want their registry to be more widely available. To do so, the Docker engine requires you to secure it using TLS, which is conceptually very similar to configuring your web server with SSL. +To stop the registry, use the same `docker container stop` command as with any other +container. + +```console +$ docker container stop registry +``` + +To remove the container, use `docker container rm`. + +```console +$ docker container stop registry && docker container rm -v registry +``` + +## Basic configuration + +To configure the container, you can pass additional or modified options to the +`docker run` command. + +The following sections provide basic guidelines for configuring your registry. +For more details, see the [registry configuration reference](configuration.md). + +### Start the registry automatically + +If you want to use the registry as part of your permanent infrastructure, you +should set it to restart automatically when Docker restarts or if it exits. +This example uses the `--restart always` flag to set a restart policy for the +registry. + +```console +$ docker run -d \ + -p 5000:5000 \ + --restart=always \ + --name registry \ + registry:2 +``` + +### Customize the published port + +If you are already using port 5000, or you want to run multiple local +registries to separate areas of concern, you can customize the registry's +port settings. This example runs the registry on port 5001 and also names it +`registry-test`. Remember, the first part of the `-p` value is the host port +and the second part is the port within the container. Within the container, the +registry listens on port `5000` by default. + +```console +$ docker run -d \ + -p 5001:5000 \ + --name registry-test \ + registry:2 +``` + +If you want to change the port the registry listens on within the container, you +can use the environment variable `REGISTRY_HTTP_ADDR` to change it. This command +causes the registry to listen on port 5001 within the container: + +```console +$ docker run -d \ + -e REGISTRY_HTTP_ADDR=0.0.0.0:5001 \ + -p 5001:5001 \ + --name registry-test \ + registry:2 +``` + + +## Storage customization + +### Customize the storage location + +By default, your registry data is persisted as a [docker volume](../storage/volumes.md) +on the host filesystem. If you want to store your registry contents at a specific +location on your host filesystem, such as if you have an SSD or SAN mounted into +a particular directory, you might decide to use a bind mount instead. A bind mount +is more dependent on the filesystem layout of the Docker host, but more performant +in many situations. The following example bind-mounts the host directory +`/mnt/registry` into the registry container at `/var/lib/registry/`. + +```console +$ docker run -d \ + -p 5000:5000 \ + --restart=always \ + --name registry \ + -v /mnt/registry:/var/lib/registry \ + registry:2 +``` + +### Customize the storage back-end + +By default, the registry stores its data on the local filesystem, whether you +use a bind mount or a volume. You can store the registry data in an Amazon S3 +bucket, Google Cloud Platform, or on another storage back-end by using +[storage drivers](./storage-drivers/index.md). For more information, see +[storage configuration options](./configuration.md#storage). + +## Run an externally-accessible registry + +Running a registry only accessible on `localhost` has limited usefulness. In +order to make your registry accessible to external hosts, you must first secure +it using TLS. + +This example is extended in [Run the registry as a +service](#run-the-registry-as-a-service) below. ### Get a certificate -Assuming that you own the domain `myregistrydomain.com`, and that its DNS record points to the host where you are running your registry, you first need to get a certificate from a CA. +These examples assume the following: -Create a `certs` directory: +- Your registry URL is `https://myregistry.domain.com/`. +- Your DNS, routing, and firewall settings allow access to the registry's host + on port 443. +- You have already obtained a certificate from a certificate authority (CA). - mkdir -p certs +If you have been issued an _intermediate_ certificate instead, see +[use an intermediate certificate](#use-an-intermediate-certificate). -Then move and/or rename your crt file to: `certs/domain.crt`, and your key file to: `certs/domain.key`. +1. Create a `certs` directory. -Make sure you stopped your registry from the previous steps, then start your registry again with TLS enabled: + ```console + $ mkdir -p certs + ``` - docker run -d -p 5000:5000 --restart=always --name registry \ - -v `pwd`/certs:/certs \ + Copy the `.crt` and `.key` files from the CA into the `certs` directory. + The following steps assume that the files are named `domain.crt` and + `domain.key`. + +2. Stop the registry if it is currently running. + + ```console + $ docker container stop registry + ``` + +3. Restart the registry, directing it to use the TLS certificate. This command + bind-mounts the `certs/` directory into the container at `/certs/`, and sets + environment variables that tell the container where to find the `domain.crt` + and `domain.key` file. The registry runs on port 443, the default HTTPS port. + + ```console + $ docker run -d \ + --restart=always \ + --name registry \ + -v "$(pwd)"/certs:/certs \ + -e REGISTRY_HTTP_ADDR=0.0.0.0:443 \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ + -p 443:443 \ registry:2 + ``` -You should now be able to access your registry from another docker host: +4. Docker clients can now pull from and push to your registry using its + external address. The following commands demonstrate this: - docker pull ubuntu - docker tag ubuntu myregistrydomain.com:5000/ubuntu - docker push myregistrydomain.com:5000/ubuntu - docker pull myregistrydomain.com:5000/ubuntu + ```console + $ docker pull ubuntu:16.04 + $ docker tag ubuntu:16.04 myregistry.domain.com/my-ubuntu + $ docker push myregistry.domain.com/my-ubuntu + $ docker pull myregistry.domain.com/my-ubuntu + ``` -#### Gotcha +#### Use an intermediate certificate -A certificate issuer may supply you with an *intermediate* certificate. In this case, you must combine your certificate with the intermediate's to form a *certificate bundle*. You can do this using the `cat` command: +A certificate issuer may supply you with an *intermediate* certificate. In this +case, you must concatenate your certificate with the intermediate certificate to +form a *certificate bundle*. You can do this using the `cat` command: - cat domain.crt intermediate-certificates.pem > certs/domain.crt +```console +cat domain.crt intermediate-certificates.pem > certs/domain.crt +``` -### Let's Encrypt +You can use the certificate bundle just as you use the `domain.crt` file in +the previous example. -The registry supports using Let's Encrypt to automatically obtain a browser-trusted certificate. For more -information on Let's Encrypt, see [https://letsencrypt.org/how-it-works/](https://letsencrypt.org/how-it-works/) and the relevant section of the [registry configuration](configuration.md#letsencrypt). +### Support for Let's Encrypt -### Alternatives +The registry supports using Let's Encrypt to automatically obtain a +browser-trusted certificate. For more information on Let's Encrypt, see +[https://letsencrypt.org/how-it-works/](https://letsencrypt.org/how-it-works/) +and the relevant section of the +[registry configuration](configuration.md#letsencrypt). -While rarely advisable, you may want to use self-signed certificates instead, or use your registry in an insecure fashion. You will find instructions [here](insecure.md). +### Use an insecure registry (testing only) -## Load Balancing Considerations +It is possible to use a self-signed certificate, or to use our registry +insecurely. Unless you have set up verification for your self-signed +certificate, this is for testing only. See [run an insecure registry](insecure.md). + +## Run the registry as a service + +[Swarm services](../engine/swarm/services.md) provide several advantages over +standalone containers. They use a declarative model, which means that you define +the desired state and Docker works to keep your service in that state. Services +provide automatic load balancing scaling, and the ability to control the +distribution of your service, among other advantages. Services also allow you to +store sensitive data such as TLS certificates in +[secrets](../engine/swarm/secrets.md). + +The storage back-end you use determines whether you use a fully scaled service +or a service with either only a single node or a node constraint. + +- If you use a distributed storage driver, such as Amazon S3, you can use a + fully replicated service. Each worker can write to the storage back-end + without causing write conflicts. + +- If you use a local bind mount or volume, each worker node writes to its + own storage location, which means that each registry contains a different + data set. You can solve this problem by using a single-replica service and a + node constraint to ensure that only a single worker is writing to the bind + mount. + +The following example starts a registry as a single-replica service, which is +accessible on any swarm node on port 80. It assumes you are using the same +TLS certificates as in the previous examples. + +First, save the TLS certificate and key as secrets: + +```console +$ docker secret create domain.crt certs/domain.crt + +$ docker secret create domain.key certs/domain.key +``` + +Next, add a label to the node where you want to run the registry. +To get the node's name, use `docker node ls`. Substitute your node's name for +`node1` below. + +```console +$ docker node update --label-add registry=true node1 +``` + +Next, create the service, granting it access to the two secrets and constraining +it to only run on nodes with the label `registry=true`. Besides the constraint, +you are also specifying that only a single replica should run at a time. The +example bind-mounts `/mnt/registry` on the swarm node to `/var/lib/registry/` +within the container. Bind mounts rely on the pre-existing source directory, +so be sure `/mnt/registry` exists on `node1`. You might need to create it before +running the following `docker service create` command. + +By default, secrets are mounted into a service at `/run/secrets/`. + +```console +$ docker service create \ + --name registry \ + --secret domain.crt \ + --secret domain.key \ + --constraint 'node.labels.registry==true' \ + --mount type=bind,src=/mnt/registry,dst=/var/lib/registry \ + -e REGISTRY_HTTP_ADDR=0.0.0.0:443 \ + -e REGISTRY_HTTP_TLS_CERTIFICATE=/run/secrets/domain.crt \ + -e REGISTRY_HTTP_TLS_KEY=/run/secrets/domain.key \ + --publish published=443,target=443 \ + --replicas 1 \ + registry:2 +``` + +You can access the service on port 443 of any swarm node. Docker sends the +requests to the node which is running the service. + +## Load balancing considerations One may want to use a load balancer to distribute load, terminate TLS or provide high availability. While a full load balancing setup is outside the @@ -112,17 +348,18 @@ the following must be the same: - HTTP Secret - Redis Cache (if configured) -If any of these are different, the registry will have trouble serving requests. +Differences in any of the above cause problems serving requests. As an example, if you're using the filesystem driver, all registry instances -must have access to the same filesystem root, which means they should be in -the same machine. For other drivers, such as s3 or azure, they should be -accessing the same resource, and will likely share an identical configuration. +must have access to the same filesystem root, on +the same machine. For other drivers, such as S3 or Azure, they should be +accessing the same resource and share an identical configuration. The _HTTP Secret_ coordinates uploads, so also must be the same across -instances. Configuring different redis instances will work (at the time -of writing), but will not be optimal if the instances are not shared, causing -more requests to be directed to the backend. +instances. Configuring different redis instances works (at the time +of writing), but is not optimal if the instances are not shared, because +more requests are directed to the backend. + +### Important/Required HTTP-Headers -#### Important/Required HTTP-Headers Getting the headers correct is very important. For all responses to any request under the "/v2/" url space, the `Docker-Distribution-API-Version` header should be set to the value "registry/2.0", even for a 4xx response. @@ -131,7 +368,7 @@ and fallback to version 1 registries, if necessary. Confirming this is setup correctly can help avoid problems with fallback. In the same train of thought, you must make sure you are properly sending the -`X-Forwarded-Proto`, `X-Forwarded-For` and `Host` headers to their "client-side" +`X-Forwarded-Proto`, `X-Forwarded-For`, and `Host` headers to their "client-side" values. Failure to do so usually makes the registry issue redirects to internal hostnames or downgrading from https to http. @@ -140,68 +377,110 @@ without credentials. The response should include a `WWW-Authenticate` challenge, providing guidance on how to authenticate, such as with basic auth or a token service. If the load balancer has health checks, it is recommended to configure it to consider a 401 response as healthy and any other as down. -This will secure your registry by ensuring that configuration problems with +This secures your registry by ensuring that configuration problems with authentication don't accidentally expose an unprotected registry. If you're using a less sophisticated load balancer, such as Amazon's Elastic Load Balancer, that doesn't allow one to change the healthy response code, health -checks can be directed at "/", which will always return a `200 OK` response. +checks can be directed at "/", which always returns a `200 OK` response. ## Restricting access -Except for registries running on secure local networks, registries should always implement access restrictions. +Except for registries running on secure local networks, registries should always +implement access restrictions. ### Native basic auth -The simplest way to achieve access restriction is through basic authentication (this is very similar to other web servers' basic authentication mechanism). +The simplest way to achieve access restriction is through basic authentication +(this is very similar to other web servers' basic authentication mechanism). +This example uses native basic authentication using `htpasswd` to store the +secrets. -> **Warning**: You **cannot** use authentication with an insecure registry. You have to [configure TLS first](#running-a-domain-registry) for this to work. +> **Warning**: +> You **cannot** use authentication with authentication schemes that send +> credentials as clear text. You must +> [configure TLS first](deploying.md#run-an-externally-accessible-registry) for +> authentication to work. +{:.warning} -First create a password file with one entry for the user "testuser", with password "testpassword": +1. Create a password file with one entry for the user `testuser`, with password + `testpassword`: - mkdir auth - docker run --entrypoint htpasswd registry:2 -Bbn testuser testpassword > auth/htpasswd + ```console + $ mkdir auth + $ docker run \ + --entrypoint htpasswd \ + httpd:2 -Bbn testuser testpassword > auth/htpasswd + ``` + + On Windows, make sure the output file is correctly encoded: -Make sure you stopped your registry from the previous step, then start it again: + ```powershell + docker run --rm --entrypoint htpasswd httpd:2 -Bbn testuser testpassword | Set-Content -Encoding ASCII auth/htpasswd + ``` - docker run -d -p 5000:5000 --restart=always --name registry \ - -v `pwd`/auth:/auth \ +2. Stop the registry. + + ```console + $ docker container stop registry + ``` + +3. Start the registry with basic authentication. + + ```console + $ docker run -d \ + -p 5000:5000 \ + --restart=always \ + --name registry \ + -v "$(pwd)"/auth:/auth \ -e "REGISTRY_AUTH=htpasswd" \ -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \ -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \ - -v `pwd`/certs:/certs \ + -v "$(pwd)"/certs:/certs \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ registry:2 + ``` -You should now be able to: +4. Try to pull an image from the registry, or push an image to the registry. + These commands fail. - docker login myregistrydomain.com:5000 +5. Log in to the registry. -And then push and pull images as an authenticated user. + ```console + $ docker login myregistrydomain.com:5000 + ``` -#### Gotcha + Provide the username and password from the first step. -Seeing X509 errors is usually a sign you are trying to use self-signed certificates, and failed to [configure your docker daemon properly](insecure.md). + Test that you can now pull an image from the registry or push an image to + the registry. -### Alternatives +> **X509 errors**: X509 errors usually indicate that you are attempting to use +> a self-signed certificate without configuring the Docker daemon correctly. +> See [run an insecure registry](insecure.md). -1. You may want to leverage more advanced basic auth implementations through a proxy design, in front of the registry. You will find examples of such patterns in the [recipes list](recipes/index.md). +### More advanced authentication -2. Alternatively, the Registry also supports delegated authentication, redirecting users to a specific, trusted token server. That approach requires significantly more investment, and only makes sense if you want to fully configure ACLs and more control over the Registry integration into your global authorization and authentication systems. +You may want to leverage more advanced basic auth implementations by using a +proxy in front of the registry. See the [recipes list](recipes/index.md). -You will find [background information here](spec/auth/token.md), and [configuration information here](configuration.md#auth). +The registry also supports delegated authentication which redirects users to a +specific trusted token server. This approach is more complicated to set up, and +only makes sense if you need to fully configure ACLs and need more control over +the registry's integration into your global authorization and authentication +systems. Refer to the following [background information](spec/auth/token.md) and +[configuration information here](configuration.md#auth). -Beware that you will have to implement your own authentication service for this to work, or leverage a third-party implementation. +This approach requires you to implement your own authentication system or +leverage a third-party implementation. -## Managing with Compose +## Deploy your registry using a Compose file -As your registry configuration grows more complex, dealing with it can quickly become tedious. +If your registry invocation is advanced, it may be easier to use a Docker +compose file to deploy it, rather than relying on a specific `docker run` +invocation. Use the following example `docker-compose.yml` as a template. -It's highly recommended to use [Docker Compose](/compose/index.md) to facilitate operating your registry. - -Here is a simple `docker-compose.yml` example that condenses everything explained so far: - -``` +```yaml registry: restart: always image: registry:2 @@ -219,15 +498,72 @@ registry: - /path/auth:/auth ``` -> **Warning**: replace `/path` by whatever directory that holds your `certs` and `auth` folder from above. +Replace `/path` with the directory which contains the `certs/` and `auth/` +directories. +{:.warning} -You can then start your registry with a simple +Start your registry by issuing the following command in the directory containing +the `docker-compose.yml` file: - docker-compose up -d +```console +$ docker-compose up -d +``` -## Next +## Considerations for air-gapped registries -You will find more specific and advanced informations in the following sections: +You can run a registry in an environment with no internet connectivity. +However, if you rely on any images which are not local, you need to consider the +following: + +- You may need to build your local registry's data volume on a connected + host where you can run `docker pull` to get any images which are available + remotely, and then migrate the registry's data volume to the air-gapped + network. + +- Certain images, such as the official Microsoft Windows base images, are not + distributable. This means that when you push an image based on one of these + images to your private registry, the non-distributable layers are **not** + pushed, but are always fetched from their authorized location. This is fine + for internet-connected hosts, but not in an air-gapped set-up. + + You can configure the Docker daemon to allow pushing non-distributable layers + to private registries. + **This is only useful in air-gapped set-ups in the presence of + non-distributable images, or in extremely bandwidth-limited situations.** + You are responsible for ensuring that you are in compliance with the terms of + use for non-distributable layers. + + 1. Edit the `daemon.json` file, which is located in `/etc/docker/` on Linux + hosts and `C:\ProgramData\docker\config\daemon.json` on Windows Server. + Assuming the file was previously empty, add the following contents: + + ```json + { + "allow-nondistributable-artifacts": ["myregistrydomain.com:5000"] + } + ``` + + The value is an array of registry addresses, separated by commas. + + Save and exit the file. + + 2. Restart Docker. + + 3. Restart the registry if it does not start automatically. + + 4. When you push images to the registries in the list, their + non-distributable layers are pushed to the registry. + + > **Warning**: Non-distributable artifacts typically have restrictions on + > how and where they can be distributed and shared. Only use this feature + > to push artifacts to private registries and ensure that you are in + > compliance with any terms that cover redistributing non-distributable + > artifacts. + + +## Next steps + +More specific and advanced information is available in the following sections: - [Configuration reference](configuration.md) - [Working with notifications](notifications.md) diff --git a/docs/deprecated.md b/docs/deprecated.md index 73bde497..8c2c7aee 100644 --- a/docs/deprecated.md +++ b/docs/deprecated.md @@ -1,15 +1,10 @@ - +--- +description: describes deprecated functionality +keywords: registry, manifest, images, signatures, repository, distribution, digest +title: Docker Registry deprecation +--- -# Docker Registry Deprecation +{% include registry.md %} This document details functionality or components which are deprecated within the registry. diff --git a/docs/garbage-collection.md b/docs/garbage-collection.md index 2d03e787..688238be 100644 --- a/docs/garbage-collection.md +++ b/docs/garbage-collection.md @@ -1,64 +1,49 @@ - +--- +description: High level discussion of garbage collection +keywords: registry, garbage, images, tags, repository, distribution +title: Garbage collection +--- -# Garbage Collection +{% include registry.md %} As of v2.4.0 a garbage collector command is included within the registry binary. This document describes what this command does and how and why it should be used. -## What is Garbage Collection? - -From [wikipedia](https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)): - -"In computer science, garbage collection (GC) is a form of automatic memory management. The -garbage collector, or just collector, attempts to reclaim garbage, or memory occupied by -objects that are no longer in use by the program." +## About garbage collection In the context of the Docker registry, garbage collection is the process of -removing blobs from the filesystem which are no longer referenced by a -manifest. Blobs can include both layers and manifests. +removing blobs from the filesystem when they are no longer referenced by a +manifest. Blobs can include both layers and manifests. +Registry data can occupy considerable amounts of disk space. In addition, +garbage collection can be a security consideration, when it is desirable to ensure +that certain layers no longer exist on the filesystem. -## Why Garbage Collection? +## Garbage collection in practice -Registry data can occupy considerable amounts of disk space and freeing up -this disk space is an oft-requested feature. Additionally for reasons of security it -can be desirable to ensure that certain layers no longer exist on the filesystem. - - -## Garbage Collection in the Registry - -Filesystem layers are stored by their content address in the Registry. This +Filesystem layers are stored by their content address in the Registry. This has many advantages, one of which is that data is stored once and referred to by manifests. See [here](compatibility.md#content-addressable-storage-cas) for more details. Layers are therefore shared amongst manifests; each manifest maintains a reference -to the layer. As long as a layer is referenced by one manifest, it cannot be garbage +to the layer. As long as a layer is referenced by one manifest, it cannot be garbage collected. -Manifests and layers can be 'deleted` with the registry API (refer to the API +Manifests and layers can be `deleted` with the registry API (refer to the API documentation [here](spec/api.md#deleting-a-layer) and -[here](spec/api.md#deleting-an-image) for details). This API removes references -to the target and makes them eligible for garbage collection. It also makes them +[here](spec/api.md#deleting-an-image) for details). This API removes references +to the target and makes them eligible for garbage collection. It also makes them unable to be read via the API. -If a layer is deleted it will be removed from the filesystem when garbage collection -is run. If a manifest is deleted the layers to which it refers will be removed from +If a layer is deleted, it is removed from the filesystem when garbage collection +is run. If a manifest is deleted the layers to which it refers are removed from the filesystem if no other manifests refers to them. ### Example -In this example manifest A references two layers: `a` and `b`. Manifest `B` references -layers `a` and `c`. In this state, nothing is eligible for garbage collection: +In this example manifest A references two layers: `a` and `b`. Manifest `B` references +layers `a` and `c`. In this state, nothing is eligible for garbage collection: ``` A -----> a <----- B @@ -75,11 +60,11 @@ A -----> a B ``` In this state layer `c` no longer has a reference and is eligible for garbage -collection. Layer `a` had one reference removed but will not be garbage -collected as it is still referenced by manifest `A`. The blob representing -manifest `B` will also be eligible for garbage collection. +collection. Layer `a` had one reference removed but not garbage +collected as it is still referenced by manifest `A`. The blob representing +manifest `B` is eligible for garbage collection. -After garbage collection has been run manifest `A` and its blobs remain. +After garbage collection has been run, manifest `A` and its blobs remain. ``` A -----> a @@ -87,35 +72,40 @@ A -----> a ``` -## How Garbage Collection works +### More details about garbage collection Garbage collection runs in two phases. First, in the 'mark' phase, the process scans all the manifests in the registry. From these manifests, it constructs a set of content address digests. This set is the 'mark set' and denotes the set of blobs to *not* delete. Secondly, in the 'sweep' phase, the process scans all the blobs and if a blob's content address digest is not in the mark set, the -process will delete it. +process deletes it. -> **NOTE** You should ensure that the registry is in read-only mode or not running at +> **Note**: You should ensure that the registry is in read-only mode or not running at > all. If you were to upload an image while garbage collection is running, there is the -> risk that the image's layers will be mistakenly deleted, leading to a corrupted image. +> risk that the image's layers are mistakenly deleted leading to a corrupted image. -This type of garbage collection is known as stop-the-world garbage collection. In future -registry versions the intention is that garbage collection will be an automated background -action and this manual process will no longer apply. +This type of garbage collection is known as stop-the-world garbage collection. - - -# Running garbage collection +## Run garbage collection Garbage collection can be run as follows `bin/registry garbage-collect [--dry-run] /path/to/config.yml` -The garbage-collect command accepts a `--dry-run` parameter, which will print the progress -of the mark and sweep phases without removing any data. Running with a log leve of `info` -will give a clear indication of what will and will not be deleted. +The garbage-collect command accepts a `--dry-run` parameter, which prints the progress +of the mark and sweep phases without removing any data. Running with a log level of `info` +gives a clear indication of items eligible for deletion. + +The config.yml file should be in the following format: + +```yaml +version: 0.1 +storage: + filesystem: + rootdirectory: /registry/data +``` _Sample output from a dry run garbage collection with registry log level set to `info`_ @@ -134,4 +124,3 @@ blob eligible for deletion: sha256:87192bdbe00f8f2a62527f36bb4c7c7f4eaf9307e4b87 blob eligible for deletion: sha256:b549a9959a664038fc35c155a95742cf12297672ca0ae35735ec027d55bf4e97 blob eligible for deletion: sha256:f251d679a7c61455f06d793e43c06786d7766c88b8c24edf242b2c08e3c3f599 ``` - diff --git a/docs/glossary.md b/docs/glossary.md index 8159b520..b07cfc0c 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -1,8 +1,6 @@ - +--- +published: false +--- # Glossary @@ -21,7 +19,7 @@ This page contains definitions for distribution related terms.
An image is a named set of immutable data from which a Docker container can be created.

- An image is represented by a json file called a manifest, and is conceptually a set of layers. + An image is represented by a json file called a manifest, and is conceptually a set of layers. Image names indicate the location where they can be pulled from and pushed to, as they usually start with a registry domain name and port. @@ -63,8 +61,8 @@ This page contains definitions for distribution related terms.

Example: `docker pull myimage:latest` instructs docker to pull the image "myimage" in version "latest".

- +
- + diff --git a/docs/help.md b/docs/help.md index 77ec378f..65e4dc4d 100644 --- a/docs/help.md +++ b/docs/help.md @@ -1,24 +1,16 @@ - +--- +description: Getting help with the Registry +keywords: registry, on-prem, images, tags, repository, distribution, help, 101, TL;DR +title: Get help +--- -# Getting help +{% include registry.md %} -If you need help, or just want to chat, you can reach us: - -- on irc: `#docker-distribution` on freenode -- on the [mailing list](https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution) (mail at ) +If you need help, or just want to chat about development, you can reach us on the #distribution channel in the CNCF Slack. If you want to report a bug: -- be sure to first read about [how to contribute](https://github.com/docker/distribution/blob/master/CONTRIBUTING.md) -- you can then do so on the [GitHub project bugtracker](https://github.com/docker/distribution/issues) +- be sure to first read about [how to contribute](https://github.com/distribution/distribution/blob/master/CONTRIBUTING.md). +- you can then do so on the [GitHub project bugtracker](https://github.com/distribution/distribution/issues). -You can also find out more about the Docker's project [Getting Help resources](/opensource/get-help.md). +You can also find out more about the Docker's project [Getting Help resources](../opensource/ways.md). diff --git a/docs/images/notifications.png b/docs/images/notifications.png index 09de8d23..c7a78f37 100644 Binary files a/docs/images/notifications.png and b/docs/images/notifications.png differ diff --git a/docs/index.md b/docs/index.md index 21ec7a9a..54148c5a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,21 +1,18 @@ - +--- +description: High-level overview of the Registry +keywords: registry, on-prem, images, tags, repository, distribution +redirect_from: +- /registry/overview/ +title: Docker Registry +--- -# Docker Registry +{% include registry.md %} ## What it is -The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. -The Registry is open-source, under the permissive [Apache license](http://en.wikipedia.org/wiki/Apache_License). +The Registry is a stateless, highly scalable server side application that stores +and lets you distribute Docker images. The Registry is open-source, under the +permissive [Apache license](https://en.wikipedia.org/wiki/Apache_License). ## Why use it @@ -27,16 +24,16 @@ You should use the Registry if you want to: ## Alternatives -Users looking for a zero maintenance, ready-to-go solution are encouraged to head-over to the [Docker Hub](https://hub.docker.com), which provides a free-to-use, hosted Registry, plus additional features (organization accounts, automated builds, and more). - -Users looking for a commercially supported version of the Registry should look into [Docker Trusted Registry](https://docs.docker.com/docker-trusted-registry/overview/). +Users looking for a zero maintenance, ready-to-go solution are encouraged to +head-over to the [Docker Hub](https://hub.docker.com), which provides a +free-to-use, hosted Registry, plus additional features (organization accounts, +automated builds, and more). ## Requirements The Registry is compatible with Docker engine **version 1.6.0 or higher**. -If you really need to work with older Docker versions, you should look into the [old python registry](https://github.com/docker/docker-registry). -## TL;DR +## Basic commands Start your registry @@ -48,7 +45,7 @@ Pull (or build) some image from the hub Tag the image so that it points to your registry - docker tag ubuntu localhost:5000/myfirstimage + docker image tag ubuntu localhost:5000/myfirstimage Push it @@ -60,8 +57,9 @@ Pull it back Now stop your registry and remove all data - docker stop registry && docker rm -v registry + docker container stop registry && docker container rm -v registry ## Next -You should now read the [detailed introduction about the registry](introduction.md), or jump directly to [deployment instructions](deploying.md). +You should now read the [detailed introduction about the registry](introduction.md), +or jump directly to [deployment instructions](deploying.md). diff --git a/docs/insecure.md b/docs/insecure.md index 4b7917d2..461f9ba9 100644 --- a/docs/insecure.md +++ b/docs/insecure.md @@ -1,91 +1,123 @@ - +--- +description: Deploying a Registry in an insecure fashion +keywords: registry, on-prem, images, tags, repository, distribution, insecure +title: Test an insecure registry +--- -# Insecure Registry +{% include registry.md %} While it's highly recommended to secure your registry using a TLS certificate -issued by a known CA, you may alternatively decide to use self-signed -certificates, or even use your registry over plain http. +issued by a known CA, you can choose to use self-signed certificates, or use +your registry over an unencrypted HTTP connection. Either of these choices +involves security trade-offs and additional configuration steps. -You have to understand the downsides in doing so, and the extra burden in -configuration. +## Deploy a plain HTTP registry -## Deploying a plain HTTP registry +> **Warning**: +> It's not possible to use an insecure registry with basic authentication. +{:.warning} -> **Warning**: it's not possible to use an insecure registry with basic authentication. +This procedure configures Docker to entirely disregard security for your +registry. This is **very** insecure and is not recommended. It exposes your +registry to trivial man-in-the-middle (MITM) attacks. Only use this solution for +isolated testing or in a tightly controlled, air-gapped environment. -This basically tells Docker to entirely disregard security for your registry. -While this is relatively easy to configure the daemon in this way, it is -**very** insecure. It does expose your registry to trivial MITM. Only use this -solution for isolated testing or in a tightly controlled, air-gapped -environment. +1. Edit the `daemon.json` file, whose default location is + `/etc/docker/daemon.json` on Linux or + `C:\ProgramData\docker\config\daemon.json` on Windows Server. If you use + Docker Desktop for Mac or Docker Desktop for Windows, click the Docker icon, choose + **Preferences** (Mac) or **Settings** (Windows), and choose **Docker Engine**. -1. Open the `/etc/default/docker` file or `/etc/sysconfig/docker` for editing. + If the `daemon.json` file does not exist, create it. Assuming there are no + other settings in the file, it should have the following contents: - Depending on your operating system, your Engine daemon start options. + ```json + { + "insecure-registries" : ["myregistrydomain.com:5000"] + } + ``` -2. Edit (or add) the `DOCKER_OPTS` line and add the `--insecure-registry` flag. + Substitute the address of your insecure registry for the one in the example. - This flag takes the URL of your registry, for example. + With insecure registries enabled, Docker goes through the following steps: - `DOCKER_OPTS="--insecure-registry myregistrydomain.com:5000"` - -3. Close and save the configuration file. - -4. Restart your Docker daemon - - The command you use to restart the daemon depends on your operating system. - For example, on Ubuntu, this is usually the `service docker stop` and `service - docker start` command. - -5. Repeat this configuration on every Engine host that wants to access your registry. + - First, try using HTTPS. + - If HTTPS is available but the certificate is invalid, ignore the error + about the certificate. + - If HTTPS is not available, fall back to HTTP. -## Using self-signed certificates +2. Restart Docker for the changes to take effect. -> **Warning**: using this along with basic authentication requires to **also** trust the certificate into the OS cert store for some versions of docker (see below) -This is more secure than the insecure registry solution. You must configure every docker daemon that wants to access your registry +Repeat these steps on every Engine host that wants to access your registry. -1. Generate your own certificate: -``` - mkdir -p certs && openssl req \ +## Use self-signed certificates + +> **Warning**: +> Using this along with basic authentication requires to **also** trust the certificate into the OS cert store for some versions of docker (see below) +{:.warning} + +This is more secure than the insecure registry solution. + +1. Generate your own certificate: + + ```console + $ mkdir -p certs + + $ openssl req \ -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \ + -addext "subjectAltName = DNS:myregistry.domain.com" \ -x509 -days 365 -out certs/domain.crt -``` + ``` -2. Be sure to use the name `myregistrydomain.com` as a CN. + Be sure to use the name `myregistry.domain.com` as a CN. -3. Use the result to [start your registry with TLS enabled](./deploying.md#get-a-certificate) +2. Use the result to [start your registry with TLS enabled](./deploying.md#get-a-certificate). -4. Instruct every docker daemon to trust that certificate. +3. Instruct every Docker daemon to trust that certificate. The way to do this + depends on your OS. - This is done by copying the `domain.crt` file to `/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt`. + - **Linux**: Copy the `domain.crt` file to + `/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt` on every Docker + host. You do not need to restart Docker. -5. Don't forget to restart the Engine daemon. + - **Windows Server**: -## Troubleshooting insecure registry + 1. Open Windows Explorer, right-click the `domain.crt` + file, and choose Install certificate. When prompted, select the following + options: -This sections lists some common failures and how to recover from them. + | Store location | local machine | + | Place all certificates in the following store | selected | + + 2. Click **Browser** and select **Trusted Root Certificate Authorities**. + + 3. Click **Finish**. Restart Docker. + + - **Docker Desktop for Mac**: Follow the instructions in + [Adding custom CA certificates](../desktop/mac/index.md#add-tls-certificates){: target="_blank" rel="noopener" class="_"}. + Restart Docker. + + - **Docker Desktop for Windows**: Follow the instructions in + [Adding custom CA certificates](../desktop/windows/index.md#adding-tls-certificates){: target="_blank" rel="noopener" class="_"}. + Restart Docker. + + +## Troubleshoot insecure registry + +This section lists some common failures and how to recover from them. ### Failing... Failing to configure the Engine daemon and trying to pull from a registry that is not using -TLS will results in the following message: +TLS results in the following message: -``` +```none FATA[0000] Error response from daemon: v1 ping attempt failed with error: Get https://myregistrydomain.com:5000/v1/_ping: tls: oversized record received with length 20527. -If this private registry supports only HTTP or HTTPS with an unknown CA certificate,please add +If this private registry supports only HTTP or HTTPS with an unknown CA certificate, add `--insecure-registry myregistrydomain.com:5000` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt @@ -93,24 +125,43 @@ simply place the CA certificate at /etc/docker/certs.d/myregistrydomain.com:5000 ### Docker still complains about the certificate when using authentication? -When using authentication, some versions of docker also require you to trust the certificate at the OS level. Usually, on Ubuntu this is done with: +When using authentication, some versions of Docker also require you to trust the +certificate at the OS level. -```bash +#### Ubuntu + +```console $ cp certs/domain.crt /usr/local/share/ca-certificates/myregistrydomain.com.crt update-ca-certificates ``` -... and on Red Hat (and its derivatives) with: +#### Red Hat Enterprise Linux -```bash -cp certs/domain.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt +```console +$ cp certs/domain.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt update-ca-trust ``` -... On some distributions, e.g. Oracle Linux 6, the Shared System Certificates feature needs to be manually enabled: +#### Oracle Linux -```bash +```console $ update-ca-trust enable ``` -Now restart docker (`service docker stop && service docker start`, or any other way you use to restart docker). +Restart Docker for the changes to take effect. + +### Windows + +Open Windows Explorer, right-click the certificate, and choose +**Install certificate**. + +Then, select the following options: + +* Store location: local machine +* Check **place all certificates in the following store** +* Click **Browser**, and select **Trusted Root Certificate Authorities** +* Click **Finish** + +[Learn more about managing TLS certificates](https://technet.microsoft.com/en-us/library/cc754841(v=ws.11).aspx#BKMK_addlocal). + +After adding the CA certificate to Windows, restart Docker Desktop for Windows. diff --git a/docs/introduction.md b/docs/introduction.md index eceb5ffc..bb634861 100644 --- a/docs/introduction.md +++ b/docs/introduction.md @@ -1,17 +1,13 @@ - +--- +description: Explains what the Registry is, basic use cases and requirements +keywords: registry, on-prem, images, tags, repository, distribution, use cases, requirements +title: About Registry +--- -# Understanding the Registry +{% include registry.md %} -A registry is a storage and content delivery system, holding named Docker images, available in different tagged versions. +A registry is a storage and content delivery system, holding named Docker +images, available in different tagged versions. > Example: the image `distribution/registry`, with tags `2.0` and `2.1`. @@ -19,13 +15,23 @@ Users interact with a registry by using docker push and pull commands. > Example: `docker pull registry-1.docker.io/distribution/registry:2.1`. -Storage itself is delegated to drivers. The default storage driver is the local posix filesystem, which is suitable for development or small deployments. Additional cloud-based storage drivers like S3, Microsoft Azure, OpenStack Swift and Aliyun OSS are also supported. People looking into using other storage backends may do so by writing their own driver implementing the [Storage API](storage-drivers/index.md). +Storage itself is delegated to drivers. The default storage driver is the local +posix filesystem, which is suitable for development or small deployments. +Additional cloud-based storage drivers like S3, Microsoft Azure, OpenStack Swift, +and Aliyun OSS are also supported. People looking into using other storage +backends may do so by writing their own driver implementing the +[Storage API](storage-drivers/index.md). -Since securing access to your hosted images is paramount, the Registry natively supports TLS and basic authentication. +Since securing access to your hosted images is paramount, the Registry natively +supports TLS and basic authentication. -The Registry GitHub repository includes additional information about advanced authentication and authorization methods. Only very large or public deployments are expected to extend the Registry in this way. +The Registry GitHub repository includes additional information about advanced +authentication and authorization methods. Only very large or public deployments +are expected to extend the Registry in this way. -Finally, the Registry ships with a robust [notification system](notifications.md), calling webhooks in response to activity, and both extensive logging and reporting, mostly useful for large installations that want to collect metrics. +Finally, the Registry ships with a robust [notification system](notifications.md), +calling webhooks in response to activity, and both extensive logging and reporting, +mostly useful for large installations that want to collect metrics. ## Understanding image naming @@ -34,21 +40,35 @@ Image names as used in typical docker commands reflect their origin: * `docker pull ubuntu` instructs docker to pull an image named `ubuntu` from the official Docker Hub. This is simply a shortcut for the longer `docker pull docker.io/library/ubuntu` command * `docker pull myregistrydomain:port/foo/bar` instructs docker to contact the registry located at `myregistrydomain:port` to find the image `foo/bar` -You can find out more about the various Docker commands dealing with images in the [official Docker engine documentation](/engine/reference/commandline/cli.md). +You can find out more about the various Docker commands dealing with images in +the [official Docker engine documentation](../engine/reference/commandline/cli.md). ## Use cases -Running your own Registry is a great solution to integrate with and complement your CI/CD system. In a typical workflow, a commit to your source revision control system would trigger a build on your CI system, which would then push a new image to your Registry if the build is successful. A notification from the Registry would then trigger a deployment on a staging environment, or notify other systems that a new image is available. +Running your own Registry is a great solution to integrate with and complement +your CI/CD system. In a typical workflow, a commit to your source revision +control system would trigger a build on your CI system, which would then push a +new image to your Registry if the build is successful. A notification from the +Registry would then trigger a deployment on a staging environment, or notify +other systems that a new image is available. -It's also an essential component if you want to quickly deploy a new image over a large cluster of machines. +It's also an essential component if you want to quickly deploy a new image over +a large cluster of machines. Finally, it's the best way to distribute images inside an isolated network. ## Requirements -You absolutely need to be familiar with Docker, specifically with regard to pushing and pulling images. You must understand the difference between the daemon and the cli, and at least grasp basic concepts about networking. +You absolutely need to be familiar with Docker, specifically with regard to +pushing and pulling images. You must understand the difference between the +daemon and the cli, and at least grasp basic concepts about networking. -Also, while just starting a registry is fairly easy, operating it in a production environment requires operational skills, just like any other service. You are expected to be familiar with systems availability and scalability, logging and log processing, systems monitoring, and security 101. Strong understanding of http and overall network communications, plus familiarity with golang are certainly useful as well for advanced operations or hacking. +Also, while just starting a registry is fairly easy, operating it in a +production environment requires operational skills, just like any other service. +You are expected to be familiar with systems availability and scalability, +logging and log processing, systems monitoring, and security 101. Strong +understanding of http and overall network communications, plus familiarity with +golang are certainly useful as well for advanced operations or hacking. ## Next diff --git a/docs/migration.md b/docs/migration.md index da0aba91..e46441cb 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -1,8 +1,6 @@ - +--- +published: false +--- # Migrating a 1.0 registry to 2.0 diff --git a/docs/notifications.md b/docs/notifications.md index c511eb59..a17ba776 100644 --- a/docs/notifications.md +++ b/docs/notifications.md @@ -1,23 +1,18 @@ - +--- +description: Explains how to work with registry notifications +keywords: registry, on-prem, images, tags, repository, distribution, notifications, advanced +title: Work with notifications +--- -# Notifications +{% include registry.md %} The Registry supports sending webhook notifications in response to events happening within the registry. Notifications are sent in response to manifest pushes and pulls and layer pushes and pulls. These actions are serialized into events. The events are queued into a registry-internal broadcast system which -queues and dispatches events to [_Endpoints_](#endpoints). +queues and dispatches events to [_Endpoints_](notifications.md#endpoints). -![](images/notifications.png) +![Workflow of registry notifications](images/notifications.png) ## Endpoints @@ -34,23 +29,25 @@ order is not guaranteed. To setup a registry instance to send notifications to endpoints, one must add them to the configuration. A simple example follows: - notifications: - endpoints: - - name: alistener - url: https://mylistener.example.com/event - headers: - Authorization: [Bearer ] - timeout: 500ms - threshold: 5 - backoff: 1s +```yaml +notifications: + endpoints: + - name: alistener + url: https://mylistener.example.com/event + headers: + Authorization: [Bearer ] + timeout: 500ms + threshold: 5 + backoff: 1s +``` The above would configure the registry with an endpoint to send events to `https://mylistener.example.com/event`, with the header "Authorization: Bearer ". The request would timeout after 500 milliseconds. If -5 failures happen consecutively, the registry will backoff for 1 second before +5 failures happen consecutively, the registry backs off for 1 second before trying again. -For details on the fields, please see the [configuration documentation](configuration.md#notifications). +For details on the fields, see the [configuration documentation](configuration.md#notifications). A properly configured endpoint should lead to a log message from the registry upon startup: @@ -63,7 +60,7 @@ INFO[0000] configuring endpoint alistener (https://mylistener.example.com/event) Events have a well-defined JSON structure and are sent as the body of notification requests. One or more events are sent in a structure called an -envelope. Each event has a unique id that can be used to uniquely identify incoming +envelope. Each event has a unique ID that can be used to uniquely identify incoming requests, if required. Along with that, an _action_ is provided with a _target_, identifying the object mutated during the event. @@ -79,62 +76,64 @@ length | int | Length in bytes of content. Same as Size field in Descriptor. repository | string | Repository identifies the named repository. fromRepository | string | FromRepository identifies the named repository which a blob was mounted from if appropriate. url | string | URL provides a direct link to the content. -tag | string | Tag identifies a tag name in tag events +tag | string | Tag identifies a tag name in tag events. request | [RequestRecord](https://godoc.org/github.com/docker/distribution/notifications#RequestRecord) | Request covers the request that generated the event. actor | [ActorRecord](https://godoc.org/github.com/docker/distribution/notifications#ActorRecord). | Actor specifies the agent that initiated the event. For most situations, this could be from the authorization context of the request. source | [SourceRecord](https://godoc.org/github.com/docker/distribution/notifications#SourceRecord) | Source identifies the registry node that generated the event. Put differently, while the actor "initiates" the event, the source "generates" it. -The following is an example of a JSON event, sent in response to the push of a +The following is an example of a JSON event, sent in response to the pull of a manifest: ```json { - "events": [ - { - "id": "320678d8-ca14-430f-8bb6-4ca139cd83f7", - "timestamp": "2016-03-09T14:44:26.402973972-08:00", - "action": "pull", - "target": { - "mediaType": "application/vnd.docker.distribution.manifest.v2+json", - "size": 708, - "digest": "sha256:fea8895f450959fa676bcc1df0611ea93823a735a01205fd8622846041d0c7cf", - "length": 708, - "repository": "hello-world", - "url": "http://192.168.100.227:5000/v2/hello-world/manifests/sha256:fea8895f450959fa676bcc1df0611ea93823a735a01205fd8622846041d0c7cf", - "tag": "latest" - }, - "request": { - "id": "6df24a34-0959-4923-81ca-14f09767db19", - "addr": "192.168.64.11:42961", - "host": "192.168.100.227:5000", - "method": "GET", - "useragent": "curl/7.38.0" - }, - "actor": {}, - "source": { - "addr": "xtal.local:5000", - "instanceID": "a53db899-3b4b-4a62-a067-8dd013beaca4" - } + "events": [ + { + "id": "320678d8-ca14-430f-8bb6-4ca139cd83f7", + "timestamp": "2016-03-09T14:44:26.402973972-08:00", + "action": "pull", + "target": { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 708, + "digest": "sha256:fea8895f450959fa676bcc1df0611ea93823a735a01205fd8622846041d0c7cf", + "length": 708, + "repository": "hello-world", + "url": "http://192.168.100.227:5000/v2/hello-world/manifests/sha256:fea8895f450959fa676bcc1df0611ea93823a735a01205fd8622846041d0c7cf", + "tag": "latest" + }, + "request": { + "id": "6df24a34-0959-4923-81ca-14f09767db19", + "addr": "192.168.64.11:42961", + "host": "192.168.100.227:5000", + "method": "GET", + "useragent": "curl/7.38.0" + }, + "actor": {}, + "source": { + "addr": "xtal.local:5000", + "instanceID": "a53db899-3b4b-4a62-a067-8dd013beaca4" } - ] + } + ] } ``` The target struct of events which are sent when manifests and blobs are deleted -will contain a subset of the data contained in Get and Put events. Specifically, -only the digest and repository will be sent. +contains a subset of the data contained in Get and Put events. Specifically, +only the digest and repository are sent. ```json -"target": { - "digest": "sha256:d89e1bee20d9cb344674e213b581f14fbd8e70274ecf9d10c514bab78a307845", - "repository": "library/test" -}, +{ + "target": { + "digest": "sha256:d89e1bee20d9cb344674e213b581f14fbd8e70274ecf9d10c514bab78a307845", + "repository": "library/test" + } +} ``` -> __NOTE:__ As of version 2.1, the `length` field for event targets +> **Note**: As of version 2.1, the `length` field for event targets > is being deprecated for the `size` field, bringing the target in line with > common nomenclature. Both will continue to be set for the foreseeable > future. Newer code should favor `size` but accept either. @@ -145,7 +144,7 @@ The envelope contains one or more events, with the following json structure: ```json { - "events": [ ... ], + "events": [ "..." ] } ``` @@ -155,95 +154,95 @@ group unrelated events and send them in the same envelope to reduce the total number of requests. The full package has the mediatype -"application/vnd.docker.distribution.events.v1+json", which will be set on the +"application/vnd.docker.distribution.events.v1+json", which is set on the request coming to an endpoint. An example of a full event may look as follows: -```json -GET /callback +```http request +GET /callback HTTP/1.1 Host: application/vnd.docker.distribution.events.v1+json Authorization: Bearer Content-Type: application/vnd.docker.distribution.events.v1+json { - "events": [ - { - "id": "asdf-asdf-asdf-asdf-0", - "timestamp": "2006-01-02T15:04:05Z", - "action": "push", - "target": { - "mediaType": "application/vnd.docker.distribution.manifest.v1+json", - "length": 1, - "digest": "sha256:fea8895f450959fa676bcc1df0611ea93823a735a01205fd8622846041d0c7cf", - "repository": "library/test", - "url": "http://example.com/v2/library/test/manifests/sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5" - }, - "request": { - "id": "asdfasdf", - "addr": "client.local", - "host": "registrycluster.local", - "method": "PUT", - "useragent": "test/0.1" - }, - "actor": { - "name": "test-actor" - }, - "source": { - "addr": "hostname.local:port" - } + "events": [ + { + "id": "asdf-asdf-asdf-asdf-0", + "timestamp": "2006-01-02T15:04:05Z", + "action": "push", + "target": { + "mediaType": "application/vnd.docker.distribution.manifest.v1+json", + "length": 1, + "digest": "sha256:fea8895f450959fa676bcc1df0611ea93823a735a01205fd8622846041d0c7cf", + "repository": "library/test", + "url": "https://example.com/v2/library/test/manifests/sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5" }, - { - "id": "asdf-asdf-asdf-asdf-1", - "timestamp": "2006-01-02T15:04:05Z", - "action": "push", - "target": { - "mediaType": "application/vnd.docker.container.image.rootfs.diff+x-gtar", - "length": 2, - "digest": "sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5", - "repository": "library/test", - "url": "http://example.com/v2/library/test/blobs/sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5" - }, - "request": { - "id": "asdfasdf", - "addr": "client.local", - "host": "registrycluster.local", - "method": "PUT", - "useragent": "test/0.1" - }, - "actor": { - "name": "test-actor" - }, - "source": { - "addr": "hostname.local:port" - } + "request": { + "id": "asdfasdf", + "addr": "client.local", + "host": "registrycluster.local", + "method": "PUT", + "useragent": "test/0.1" }, - { - "id": "asdf-asdf-asdf-asdf-2", - "timestamp": "2006-01-02T15:04:05Z", - "action": "push", - "target": { - "mediaType": "application/vnd.docker.container.image.rootfs.diff+x-gtar", - "length": 3, - "digest": "sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5", - "repository": "library/test", - "url": "http://example.com/v2/library/test/blobs/sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5" - }, - "request": { - "id": "asdfasdf", - "addr": "client.local", - "host": "registrycluster.local", - "method": "PUT", - "useragent": "test/0.1" - }, - "actor": { - "name": "test-actor" - }, - "source": { - "addr": "hostname.local:port" - } + "actor": { + "name": "test-actor" + }, + "source": { + "addr": "hostname.local:port" } - ] + }, + { + "id": "asdf-asdf-asdf-asdf-1", + "timestamp": "2006-01-02T15:04:05Z", + "action": "push", + "target": { + "mediaType": "application/vnd.docker.container.image.rootfs.diff+x-gtar", + "length": 2, + "digest": "sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5", + "repository": "library/test", + "url": "https://example.com/v2/library/test/blobs/sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5" + }, + "request": { + "id": "asdfasdf", + "addr": "client.local", + "host": "registrycluster.local", + "method": "PUT", + "useragent": "test/0.1" + }, + "actor": { + "name": "test-actor" + }, + "source": { + "addr": "hostname.local:port" + } + }, + { + "id": "asdf-asdf-asdf-asdf-2", + "timestamp": "2006-01-02T15:04:05Z", + "action": "push", + "target": { + "mediaType": "application/vnd.docker.container.image.rootfs.diff+x-gtar", + "length": 3, + "digest": "sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5", + "repository": "library/test", + "url": "https://example.com/v2/library/test/blobs/sha256:c3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5" + }, + "request": { + "id": "asdfasdf", + "addr": "client.local", + "host": "registrycluster.local", + "method": "PUT", + "useragent": "test/0.1" + }, + "actor": { + "name": "test-actor" + }, + "source": { + "addr": "hostname.local:port" + } + } + ] } ``` @@ -251,7 +250,7 @@ Content-Type: application/vnd.docker.distribution.events.v1+json The registry is fairly accepting of the response codes from endpoints. If an endpoint responds with any 2xx or 3xx response code (after following -redirects), the message will be considered delivered and discarded. +redirects), the message is considered to have been delivered, and is discarded. In turn, it is recommended that endpoints are accepting of incoming responses, as well. While the format of event envelopes are standardized by media type, @@ -268,49 +267,50 @@ The following provides an example of a few endpoints that have experienced several failures and have since recovered: ```json -"notifications":{ - "endpoints":[ +{ + "notifications": { + "endpoints": [ { - "name":"local-5003", - "url":"http://localhost:5003/callback", - "Headers":{ - "Authorization":[ - "Bearer \u003can example token\u003e" - ] - }, - "Timeout":1000000000, - "Threshold":10, - "Backoff":1000000000, - "Metrics":{ - "Pending":76, - "Events":76, - "Successes":0, - "Failures":0, - "Errors":46, - "Statuses":{ - - } - } + "name": "local-5003", + "url": "http://localhost:5003/callback", + "Headers": { + "Authorization": [ + "Bearer \u003can example token\u003e" + ] + }, + "Timeout": 1000000000, + "Threshold": 10, + "Backoff": 1000000000, + "Metrics": { + "Pending": 76, + "Events": 76, + "Successes": 0, + "Failures": 0, + "Errors": 46, + "Statuses": { + } + } }, { - "name":"local-8083", - "url":"http://localhost:8083/callback", - "Headers":null, - "Timeout":1000000000, - "Threshold":10, - "Backoff":1000000000, - "Metrics":{ - "Pending":0, - "Events":76, - "Successes":76, - "Failures":0, - "Errors":28, - "Statuses":{ - "202 Accepted":76 - } - } + "name": "local-8083", + "url": "http://localhost:8083/callback", + "Headers": null, + "Timeout": 1000000000, + "Threshold": 10, + "Backoff": 1000000000, + "Metrics": { + "Pending": 0, + "Events": 76, + "Successes": 76, + "Failures": 0, + "Errors": 28, + "Statuses": { + "202 Accepted": 76 + } + } } - ] + ] + } } ``` @@ -319,15 +319,15 @@ monitor the size ("Pending" above) of the endpoint queues. If failures or queue sizes are increasing, it can indicate a larger problem. The logs are also a valuable resource for monitoring problems. A failing -endpoint will lead to messages similar to the following: +endpoint leads to messages similar to the following: -``` +```none ERRO[0340] retryingsink: error writing events: httpSink{http://localhost:5003/callback}: error posting: Post http://localhost:5003/callback: dial tcp 127.0.0.1:5003: connection refused, retrying WARN[0340] httpSink{http://localhost:5003/callback} encountered too many errors, backing off ``` -The above indicates that several errors have led to a backoff and the registry -will wait before retrying. +The above indicates that several errors caused a backoff and the registry +waits before retrying. ## Considerations @@ -335,7 +335,7 @@ Currently, the queues are inmemory, so endpoints should be _reasonably reliable_. They are designed to make a best-effort to send the messages but if an instance is lost, messages may be dropped. If an endpoint goes down, care should be taken to ensure that the registry instance is not terminated before -the endpoint comes back up or messages will be lost. +the endpoint comes back up or messages are lost. This can be mitigated by running endpoints in close proximity to the registry instances. One could run an endpoint that pages to disk and then forwards a @@ -345,6 +345,6 @@ The notification system is designed around a series of interchangeable _sinks_ which can be wired up to achieve interesting behavior. If this system doesn't provide acceptable guarantees, adding a transactional `Sink` to the registry is a possibility, although it may have an effect on request service time. -Please see the -[godoc](http://godoc.org/github.com/docker/distribution/notifications#Sink) +See the +[godoc](https://godoc.org/github.com/docker/distribution/notifications#Sink) for more information. diff --git a/docs/recipes/apache.md b/docs/recipes/apache.md index ac24113b..45a10c66 100644 --- a/docs/recipes/apache.md +++ b/docs/recipes/apache.md @@ -1,14 +1,10 @@ - +--- +description: Restricting access to your registry using an apache proxy +keywords: registry, on-prem, images, tags, repository, distribution, authentication, proxy, apache, httpd, TLS, recipe, advanced +title: Authenticate proxy with apache +--- -# Authenticating proxy with apache +{% include registry.md %} ## Use-case @@ -18,7 +14,7 @@ Usually, that includes enterprise setups using LDAP/AD on the backend and a SSO ### Alternatives -If you just want authentication for your registry, and are happy maintaining users access separately, you should really consider sticking with the native [basic auth registry feature](../deploying.md#native-basic-auth). +If you just want authentication for your registry, and are happy maintaining users access separately, you should really consider sticking with the native [basic auth registry feature](../deploying.md#native-basic-auth). ### Solution @@ -26,13 +22,13 @@ With the method presented here, you implement basic authentication for docker en While we use a simple htpasswd file as an example, any other apache authentication backend should be fairly easy to implement once you are done with the example. -We also implement push restriction (to a limited user group) for the sake of the example. Again, you should modify this to fit your mileage. +We also implement push restriction (to a limited user group) for the sake of the example. Again, you should modify this to fit your mileage. ### Gotchas While this model gives you the ability to use whatever authentication backend you want through the secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself. -Furthermore, introducing an extra http layer in your communication pipeline will make it more complex to deploy, maintain, and debug, and will possibly create issues. +Furthermore, introducing an extra http layer in your communication pipeline adds complexity when deploying, maintaining, and debugging. ## Setting things up @@ -46,7 +42,7 @@ Run the following script: mkdir -p auth mkdir -p data -# This is the main apache configuration you will use +# This is the main apache configuration cat < auth/httpd.conf LoadModule headers_module modules/mod_headers.so @@ -111,7 +107,7 @@ Listen 5043 SSLCertificateFile /usr/local/apache2/conf/domain.crt SSLCertificateKeyFile /usr/local/apache2/conf/domain.key - ## SSL settings recommandation from: https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html + ## SSL settings recommendation from: https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html # Anti CRIME SSLCompression off @@ -199,13 +195,13 @@ Now, start your stack: docker-compose up -d -Login with a "push" authorized user (using `testuserpush` and `testpasswordpush`), then tag and push your first image: +Log in with a "push" authorized user (using `testuserpush` and `testpasswordpush`), then tag and push your first image: docker login myregistrydomain.com:5043 docker tag ubuntu myregistrydomain.com:5043/test docker push myregistrydomain.com:5043/test -Now, login with a "pull-only" user (using `testuser` and `testpassword`), then pull back the image: +Now, log in with a "pull-only" user (using `testuser` and `testpassword`), then pull back the image: docker login myregistrydomain.com:5043 docker pull myregistrydomain.com:5043/test diff --git a/docs/recipes/index.md b/docs/recipes/index.md index b4dd6367..c7c0cae3 100644 --- a/docs/recipes/index.md +++ b/docs/recipes/index.md @@ -1,25 +1,17 @@ - +--- +description: Fun stuff to do with your registry +keywords: registry, on-prem, images, tags, repository, distribution, recipes, advanced +title: Recipes overview +--- -# Recipes +{% include registry.md %} -You will find here a list of "recipes", end-to-end scenarios for exotic or otherwise advanced use-cases. - -Most users are not expected to have a use for these. +This list of "recipes" provides end-to-end scenarios for exotic or otherwise advanced use-cases. +These recipes are not useful for most standard set-ups. ## Requirements -You should have followed entirely the basic [deployment guide](../deploying.md). - -If you have not, please take the time to do so. +Before following these steps, work through the [deployment guide](../deploying.md). At this point, it's assumed that: @@ -27,11 +19,11 @@ At this point, it's assumed that: * you have installed Docker Compose * it's HIGHLY recommended that you get a certificate from a known CA instead of self-signed certificates * inside the current directory, you have a X509 `domain.crt` and `domain.key`, for the CN `myregistrydomain.com` - * be sure you have stopped and removed any previously running registry (typically `docker stop registry && docker rm -v registry`) + * be sure you have stopped and removed any previously running registry (typically `docker container stop registry && docker container rm -v registry`) ## The List * [using Apache as an authenticating proxy](apache.md) * [using Nginx as an authenticating proxy](nginx.md) - * [running a Registry on OS X](osx-setup-guide.md) + * [running a Registry on macOS](osx-setup-guide.md) * [mirror the Docker Hub](mirror.md) diff --git a/docs/recipes/mirror.md b/docs/recipes/mirror.md index 241e41bd..33fa204d 100644 --- a/docs/recipes/mirror.md +++ b/docs/recipes/mirror.md @@ -1,74 +1,132 @@ - +--- +description: Setting-up a local mirror for Docker Hub images +keywords: registry, on-prem, images, tags, repository, distribution, mirror, Hub, recipe, advanced +title: Registry as a pull through cache +redirect_from: +- /engine/admin/registry_mirror/ +--- -# Registry as a pull through cache +{% include registry.md %} ## Use-case -If you have multiple instances of Docker running in your environment (e.g., multiple physical or virtual machines, all running the Docker daemon), each time one of them requires an image that it doesn’t have it will go out to the internet and fetch it from the public Docker registry. By running a local registry mirror, you can keep most of the redundant image fetch traffic on your local network. +If you have multiple instances of Docker running in your environment, such as +multiple physical or virtual machines all running Docker, each daemon goes out +to the internet and fetches an image it doesn't have locally, from the Docker +repository. You can run a local registry mirror and point all your daemons +there, to avoid this extra internet traffic. + +> **Note** +> +> Docker Official Images are an intellectual property of Docker. ### Alternatives -Alternatively, if the set of images you are using is well delimited, you can simply pull them manually and push them to a simple, local, private registry. +Alternatively, if the set of images you are using is well delimited, you can +simply pull them manually and push them to a simple, local, private registry. -Furthermore, if your images are all built in-house, not using the Hub at all and relying entirely on your local registry is the simplest scenario. +Furthermore, if your images are all built in-house, not using the Hub at all and +relying entirely on your local registry is the simplest scenario. ### Gotcha -It's currently not possible to mirror another private registry. Only the central Hub can be mirrored. +It's currently not possible to mirror another private registry. Only the central +Hub can be mirrored. + +> **Note** +> +> Mirrors of Docker Hub are still subject to Docker's [fair usage policy](https://www.docker.com/pricing/resource-consumption-updates){: target="blank" rel="noopener" class=“”}. ### Solution -The Registry can be configured as a pull through cache. In this mode a Registry responds to all normal docker pull requests but stores all content locally. +The Registry can be configured as a pull through cache. In this mode a Registry +responds to all normal docker pull requests but stores all content locally. ## How does it work? -The first time you request an image from your local registry mirror, it pulls the image from the public Docker registry and stores it locally before handing it back to you. On subsequent requests, the local registry mirror is able to serve the image from its own storage. +The first time you request an image from your local registry mirror, it pulls +the image from the public Docker registry and stores it locally before handing +it back to you. On subsequent requests, the local registry mirror is able to +serve the image from its own storage. ### What if the content changes on the Hub? -When a pull is attempted with a tag, the Registry will check the remote to ensure if it has the latest version of the requested content. If it doesn't it will fetch the latest content and cache it. +When a pull is attempted with a tag, the Registry checks the remote to +ensure if it has the latest version of the requested content. Otherwise, it +fetches and caches the latest content. ### What about my disk? -In environments with high churn rates, stale data can build up in the cache. When running as a pull through cache the Registry will periodically remove old content to save disk space. Subsequent requests for removed content will cause a remote fetch and local re-caching. +In environments with high churn rates, stale data can build up in the cache. +When running as a pull through cache the Registry periodically removes old +content to save disk space. Subsequent requests for removed content causes a +remote fetch and local re-caching. -To ensure best performance and guarantee correctness the Registry cache should be configured to use the `filesystem` driver for storage. +To ensure best performance and guarantee correctness the Registry cache should +be configured to use the `filesystem` driver for storage. -## Running a Registry as a pull through cache +## Run a Registry as a pull-through cache -The easiest way to run a registry as a pull through cache is to run the official Registry image. +The easiest way to run a registry as a pull through cache is to run the official +Registry image. +At least, you need to specify `proxy.remoteurl` within `/etc/docker/registry/config.yml` +as described in the following subsection. -Multiple registry caches can be deployed over the same back-end. A single registry cache will ensure that concurrent requests do not pull duplicate data, but this property will not hold true for a registry cache cluster. +Multiple registry caches can be deployed over the same back-end. A single +registry cache ensures that concurrent requests do not pull duplicate data, +but this property does not hold true for a registry cache cluster. -### Configuring the cache +> **Note** +> +> Service accounts included in the Team plan are limited to 5,000 pulls per day. See [Service Accounts](/docker-hub/service-accounts/) for more details. -To configure a Registry to run as a pull through cache, the addition of a `proxy` section is required to the config file. +### Configure the cache -In order to access private images on the Docker Hub, a username and password can be supplied. +To configure a Registry to run as a pull through cache, the addition of a +`proxy` section is required to the config file. - proxy: - remoteurl: https://registry-1.docker.io - username: [username] - password: [password] +To access private images on the Docker Hub, a username and password can +be supplied. -> :warn: if you specify a username and password, it's very important to understand that private resources that this user has access to on the Hub will be made available on your mirror. It's thus paramount that you secure your mirror by implementing authentication if you expect these resources to stay private! +```yaml +proxy: + remoteurl: https://registry-1.docker.io + username: [username] + password: [password] +``` -### Configuring the Docker daemon +> **Warning**: If you specify a username and password, it's very important to +> understand that private resources that this user has access to Docker Hub is +> made available on your mirror. **You must secure your mirror** by +> implementing authentication if you expect these resources to stay private! -You will need to pass the `--registry-mirror` option to your Docker daemon on startup: +> **Warning**: For the scheduler to clean up old entries, `delete` must +> be enabled in the registry configuration. See +> [Registry Configuration](../configuration.md) for more details. - docker --registry-mirror=https:// daemon +### Configure the Docker daemon -For example, if your mirror is serving on `http://10.0.0.2:5000`, you would run: +Either pass the `--registry-mirror` option when starting `dockerd` manually, +or edit [`/etc/docker/daemon.json`](../../engine/reference/commandline/dockerd.md#daemon-configuration-file) +and add the `registry-mirrors` key and value, to make the change persistent. - docker --registry-mirror=https://10.0.0.2:5000 daemon +```json +{ + "registry-mirrors": ["https://"] +} +``` -NOTE: Depending on your local host setup, you may be able to add the `--registry-mirror` option to the `DOCKER_OPTS` variable in `/etc/default/docker`. +Save the file and reload Docker for the change to take effect. + +> Some log messages that appear to be errors are actually informational messages. +> +> Check the `level` field to determine whether +> the message is warning you about an error or is giving you information. +> For example, this log message is informational: +> +> ```conf +> time="2017-06-02T15:47:37Z" level=info msg="error statting local store, serving from upstream: unknown blob" go.version=go1.7.4 +> ``` +> +> It's telling you that the file doesn't exist yet in the local cache and is +> being pulled from upstream. diff --git a/docs/recipes/nginx.md b/docs/recipes/nginx.md index f4a67679..b4ba138f 100644 --- a/docs/recipes/nginx.md +++ b/docs/recipes/nginx.md @@ -1,41 +1,58 @@ - - -# Authenticating proxy with nginx +--- +description: Restricting access to your registry using a nginx proxy +keywords: registry, on-prem, images, tags, repository, distribution, nginx, proxy, authentication, TLS, recipe, advanced +title: Authenticate proxy with nginx +redirect_from: +- /registry/nginx/ +--- +{% include registry.md %} ## Use-case -People already relying on a nginx proxy to authenticate their users to other services might want to leverage it and have Registry communications tunneled through the same pipeline. +People already relying on a nginx proxy to authenticate their users to other +services might want to leverage it and have Registry communications tunneled +through the same pipeline. -Usually, that includes enterprise setups using LDAP/AD on the backend and a SSO mechanism fronting their internal http portal. +Usually, that includes enterprise setups using LDAP/AD on the backend and a SSO +mechanism fronting their internal http portal. ### Alternatives -If you just want authentication for your registry, and are happy maintaining users access separately, you should really consider sticking with the native [basic auth registry feature](../deploying.md#native-basic-auth). +If you just want authentication for your registry, and are happy maintaining +users access separately, you should really consider sticking with the native +[basic auth registry feature](../deploying.md#native-basic-auth). ### Solution -With the method presented here, you implement basic authentication for docker engines in a reverse proxy that sits in front of your registry. +With the method presented here, you implement basic authentication for docker +engines in a reverse proxy that sits in front of your registry. -While we use a simple htpasswd file as an example, any other nginx authentication backend should be fairly easy to implement once you are done with the example. +While we use a simple htpasswd file as an example, any other nginx +authentication backend should be fairly easy to implement once you are done with +the example. -We also implement push restriction (to a limited user group) for the sake of the example. Again, you should modify this to fit your mileage. +We also implement push restriction (to a limited user group) for the sake of the +example. Again, you should modify this to fit your mileage. ### Gotchas -While this model gives you the ability to use whatever authentication backend you want through the secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself. +While this model gives you the ability to use whatever authentication backend +you want through the secondary authentication mechanism implemented inside your +proxy, it also requires that you move TLS termination from the Registry to the +proxy itself. -Furthermore, introducing an extra http layer in your communication pipeline will make it more complex to deploy, maintain, and debug, and will possibly create issues. Make sure the extra complexity is required. +> **Note**: Docker does not recommend binding your registry to `localhost:5000` without +> authentication. This creates a potential loophole in your Docker Registry security. +> As a result, anyone who can log on to the server where your Docker Registry is running +> can push images without authentication. -For instance, Amazon's Elastic Load Balancer (ELB) in HTTPS mode already sets the following client header: +Furthermore, introducing an extra http layer in your communication pipeline +makes it more complex to deploy, maintain, and debug. Make sure the extra +complexity is required. + +For instance, Amazon's Elastic Load Balancer (ELB) in HTTPS mode already sets +the following client header: ``` X-Real-IP @@ -43,138 +60,134 @@ X-Forwarded-For X-Forwarded-Proto ``` -So if you have an nginx sitting behind it, should remove these lines from the example config below: +So if you have an Nginx instance sitting behind it, remove these lines from the +example config below: -``` -X-Real-IP $remote_addr; # pass on real client's IP -X-Forwarded-For $proxy_add_x_forwarded_for; -X-Forwarded-Proto $scheme; +```none +proxy_set_header Host $http_host; # required for docker client's sake +proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP +proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; +proxy_set_header X-Forwarded-Proto $scheme; ``` -Otherwise nginx will reset the ELB's values, and the requests will not be routed properly. For more information, see [#970](https://github.com/docker/distribution/issues/970). +Otherwise Nginx resets the ELB's values, and the requests are not routed +properly. For more information, see +[#970](https://github.com/docker/distribution/issues/970). ## Setting things up -Read again [the requirements](index.md#requirements). +Review the [requirements](index.md#requirements), then follow these steps. -Ready? +1. Create the required directories --- + ```console + $ mkdir -p auth data + ``` -Create the required directories +2. Create the main nginx configuration. Paste this code block into a new file called `auth/nginx.conf`: -``` -mkdir -p auth -mkdir -p data -``` + ```conf + events { + worker_connections 1024; + } -Create the main nginx configuration you will use. + http { -``` - -cat < auth/nginx.conf -events { - worker_connections 1024; -} - -http { - - upstream docker-registry { - server registry:5000; - } - - ## Set a variable to help us decide if we need to add the - ## 'Docker-Distribution-Api-Version' header. - ## The registry always sets this header. - ## In the case of nginx performing auth, the header will be unset - ## since nginx is auth-ing before proxying. - map \$upstream_http_docker_distribution_api_version \$docker_distribution_api_version { - 'registry/2.0' ''; - default registry/2.0; - } - - server { - listen 443 ssl; - server_name myregistrydomain.com; - - # SSL - ssl_certificate /etc/nginx/conf.d/domain.crt; - ssl_certificate_key /etc/nginx/conf.d/domain.key; - - # Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html - ssl_protocols TLSv1.1 TLSv1.2; - ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH'; - ssl_prefer_server_ciphers on; - ssl_session_cache shared:SSL:10m; - - # disable any limits to avoid HTTP 413 for large image uploads - client_max_body_size 0; - - # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486) - chunked_transfer_encoding on; - - location /v2/ { - # Do not allow connections from docker 1.5 and earlier - # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents - if (\$http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*\$" ) { - return 404; + upstream docker-registry { + server registry:5000; } - # To add basic authentication to v2 use auth_basic setting. - auth_basic "Registry realm"; - auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd; + ## Set a variable to help us decide if we need to add the + ## 'Docker-Distribution-Api-Version' header. + ## The registry always sets this header. + ## In the case of nginx performing auth, the header is unset + ## since nginx is auth-ing before proxying. + map $upstream_http_docker_distribution_api_version $docker_distribution_api_version { + '' 'registry/2.0'; + } - ## If $docker_distribution_api_version is empty, the header will not be added. - ## See the map directive above where this variable is defined. - add_header 'Docker-Distribution-Api-Version' \$docker_distribution_api_version always; + server { + listen 443 ssl; + server_name myregistrydomain.com; - proxy_pass http://docker-registry; - proxy_set_header Host \$http_host; # required for docker client's sake - proxy_set_header X-Real-IP \$remote_addr; # pass on real client's IP - proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto \$scheme; - proxy_read_timeout 900; + # SSL + ssl_certificate /etc/nginx/conf.d/domain.crt; + ssl_certificate_key /etc/nginx/conf.d/domain.key; + + # Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html + ssl_protocols TLSv1.1 TLSv1.2; + ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH'; + ssl_prefer_server_ciphers on; + ssl_session_cache shared:SSL:10m; + + # disable any limits to avoid HTTP 413 for large image uploads + client_max_body_size 0; + + # required to avoid HTTP 411: see Issue #1486 (https://github.com/moby/moby/issues/1486) + chunked_transfer_encoding on; + + location /v2/ { + # Do not allow connections from docker 1.5 and earlier + # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents + if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) { + return 404; + } + + # To add basic authentication to v2 use auth_basic setting. + auth_basic "Registry realm"; + auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd; + + ## If $docker_distribution_api_version is empty, the header is not added. + ## See the map directive above where this variable is defined. + add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always; + + proxy_pass http://docker-registry; + proxy_set_header Host $http_host; # required for docker client's sake + proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_read_timeout 900; + } + } } - } -} -EOF -``` + ``` -Now create a password file for "testuser" and "testpassword" +3. Create a password file `auth/nginx.htpasswd` for "testuser" and "testpassword". -``` -docker run --rm --entrypoint htpasswd registry:2 -bn testuser testpassword > auth/nginx.htpasswd -``` + ```console + $ docker run --rm --entrypoint htpasswd registry:2 -Bbn testuser testpassword > auth/nginx.htpasswd + ``` -Copy over your certificate files + > **Note**: If you do not want to use `bcrypt`, you can omit the `-B` parameter. -``` -cp domain.crt auth -cp domain.key auth -``` +4. Copy your certificate files to the `auth/` directory. -Now create your compose file + ```console + $ cp domain.crt auth + $ cp domain.key auth + ``` -``` -cat < docker-compose.yml -nginx: - image: "nginx:1.9" - ports: - - 5043:443 - links: - - registry:registry - volumes: - - ./auth:/etc/nginx/conf.d - - ./auth/nginx.conf:/etc/nginx/nginx.conf:ro +5. Create the compose file. Paste the following YAML into a new file called `docker-compose.yml`. -registry: - image: registry:2 - ports: - - 127.0.0.1:5000:5000 - volumes: - - `pwd`./data:/var/lib/registry -EOF -``` + ```yaml + nginx: + # Note : Only nginx:alpine supports bcrypt. + # If you don't need to use bcrypt, you can use a different tag. + # Ref. https://github.com/nginxinc/docker-nginx/issues/29 + image: "nginx:alpine" + ports: + - 5043:443 + links: + - registry:registry + volumes: + - ./auth:/etc/nginx/conf.d + - ./auth/nginx.conf:/etc/nginx/nginx.conf:ro + + registry: + image: registry:2 + volumes: + - ./data:/var/lib/registry + ``` ## Starting and stopping @@ -182,7 +195,8 @@ Now, start your stack: docker-compose up -d -Login with a "push" authorized user (using `testuser` and `testpassword`), then tag and push your first image: +Login with a "push" authorized user (using `testuser` and `testpassword`), then +tag and push your first image: docker login -u=testuser -p=testpassword -e=root@example.ch myregistrydomain.com:5043 docker tag ubuntu myregistrydomain.com:5043/test diff --git a/docs/recipes/osx-setup-guide.md b/docs/recipes/osx-setup-guide.md index d47d31c1..894fc73a 100644 --- a/docs/recipes/osx-setup-guide.md +++ b/docs/recipes/osx-setup-guide.md @@ -1,32 +1,26 @@ - +--- +description: Explains how to run a registry on macOS +keywords: registry, on-prem, images, tags, repository, distribution, macOS, recipe, advanced +title: macOS setup guide +--- -# OS X Setup Guide +{% include registry.md %} ## Use-case -This is useful if you intend to run a registry server natively on OS X. +This is useful if you intend to run a registry server natively on macOS. ### Alternatives -You can start a VM on OS X, and deploy your registry normally as a container using Docker inside that VM. - -The simplest road to get there is traditionally to use the [docker Toolbox](https://www.docker.com/toolbox), or [docker-machine](/machine/index.md), which usually relies on the [boot2docker](http://boot2docker.io/) iso inside a VirtualBox VM. +You can start a VM on macOS, and deploy your registry normally as a container using Docker inside that VM. ### Solution -Using the method described here, you install and compile your own from the git repository and run it as an OS X agent. +Using the method described here, you install and compile your own from the git repository and run it as an macOS agent. ### Gotchas -Production services operation on OS X is out of scope of this document. Be sure you understand well these aspects before considering going to production with this. +Production services operation on macOS is out of scope of this document. Be sure you understand well these aspects before considering going to production with this. ## Setup golang on your machine @@ -50,6 +44,7 @@ If you want to understand, you should read [How to Write Go Code](https://golang ## Build the binary GOPATH=$(PWD)/Godeps/_workspace:$GOPATH make binaries + sudo mkdir -p /usr/local/libexec sudo cp bin/registry /usr/local/libexec/registry ## Setup @@ -59,23 +54,23 @@ Copy the registry configuration file in place: mkdir /Users/Shared/Registry cp docs/osx/config.yml /Users/Shared/Registry/config.yml -## Running the Docker Registry under launchd +## Run the Docker Registry under launchd Copy the Docker registry plist into place: - plutil -lint docs/osx/com.docker.registry.plist - cp docs/osx/com.docker.registry.plist ~/Library/LaunchAgents/ + plutil -lint docs/recipes/osx/com.docker.registry.plist + cp docs/recipes/osx/com.docker.registry.plist ~/Library/LaunchAgents/ chmod 644 ~/Library/LaunchAgents/com.docker.registry.plist Start the Docker registry: launchctl load ~/Library/LaunchAgents/com.docker.registry.plist -### Restarting the docker registry service +### Restart the docker registry service launchctl stop com.docker.registry launchctl start com.docker.registry -### Unloading the docker registry service +### Unload the docker registry service launchctl unload ~/Library/LaunchAgents/com.docker.registry.plist diff --git a/docs/recipes/osx/config.yml b/docs/recipes/osx/config.yml index 63b8f713..2677f824 100644 --- a/docs/recipes/osx/config.yml +++ b/docs/recipes/osx/config.yml @@ -11,6 +11,6 @@ storage: rootdirectory: /Users/Shared/Registry http: addr: 0.0.0.0:5000 - secret: mytokensecret + secret: mytokensecret debug: addr: localhost:5001 diff --git a/docs/storage-drivers/azure.md b/docs/storage-drivers/azure.md index a84888de..6043750a 100644 --- a/docs/storage-drivers/azure.md +++ b/docs/storage-drivers/azure.md @@ -1,78 +1,26 @@ - +--- +description: Explains how to use the Azure storage drivers +keywords: registry, service, driver, images, storage, azure +title: Microsoft Azure storage driver +--- +{% include registry.md %} -# Microsoft Azure storage driver - -An implementation of the `storagedriver.StorageDriver` interface which uses [Microsoft Azure Blob Storage](http://azure.microsoft.com/en-us/services/storage/) for object storage. +An implementation of the `storagedriver.StorageDriver` interface which uses [Microsoft Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/) for object storage. ## Parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ParameterRequiredDescription
- accountname - - yes - - Name of the Azure Storage Account. -
- accountkey - - yes - - Primary or Secondary Key for the Storage Account. -
- container - - yes - - Name of the Azure root storage container in which all registry data will be stored. Must comply the storage container name [requirements][create-container-api]. -
- realm - - no - - Domain name suffix for the Storage Service API endpoint. For example realm for "Azure in China" would be `core.chinacloudapi.cn` and realm for "Azure Government" would be `core.usgovcloudapi.net`. By default, this - is core.windows.net. -
+| Parameter | Required | Description | +|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `accountname` | yes | Name of the Azure Storage Account. | +| `accountkey` | yes | Primary or Secondary Key for the Storage Account. | +| `container` | yes | Name of the Azure root storage container in which all registry data is stored. Must comply the storage container name [requirements](https://docs.microsoft.com/rest/api/storageservices/fileservices/naming-and-referencing-containers--blobs--and-metadata). For example, if your url is `https://myaccount.blob.core.windows.net/myblob` use the container value of `myblob`.| +| `realm` | no | Domain name suffix for the Storage Service API endpoint. For example realm for "Azure in China" would be `core.chinacloudapi.cn` and realm for "Azure Government" would be `core.usgovcloudapi.net`. By default, this is `core.windows.net`. | -## Related Information +## Related information * To get information about -[azure-blob-storage](http://azure.microsoft.com/en-us/services/storage/) visit +[azure-blob-storage](https://azure.microsoft.com/en-us/services/storage/), visit the Microsoft website. -* You can use Microsoft's [Blob Service REST API](https://msdn.microsoft.com/en-us/library/azure/dd135733.aspx) to [create a container] (https://msdn.microsoft.com/en-us/library/azure/dd179468.aspx). +* You can use Microsoft's [Blob Service REST API](https://docs.microsoft.com/en-us/rest/api/storageservices/Blob-Service-REST-API) to [create a storage container](https://docs.microsoft.com/en-us/rest/api/storageservices/Create-Container). diff --git a/docs/storage-drivers/filesystem.md b/docs/storage-drivers/filesystem.md index 8e269cdb..f9216731 100644 --- a/docs/storage-drivers/filesystem.md +++ b/docs/storage-drivers/filesystem.md @@ -1,24 +1,21 @@ - +--- +description: Explains how to use the filesystem storage drivers +keywords: registry, service, driver, images, storage, filesystem +title: Filesystem storage driver +--- - -# Filesystem storage driver +{% include registry.md %} An implementation of the `storagedriver.StorageDriver` interface which uses the local filesystem. ## Parameters -`rootdirectory`: (optional) The absolute path to a root directory tree in which +* `rootdirectory`: (optional) The absolute path to a root directory tree in which to store all registry files. The registry stores all its data here so make sure -there is adequate space available. Defaults to `/var/lib/registry`. -`maxthreads`: (optional) The maximum number of simultaneous blocking filesystem +there is adequate space available. Defaults to `/var/lib/registry`. If the directory +does not exist, it will be created honoring [`umask`](https://man7.org/linux/man-pages/man2/umask.2.html) +bits. If `umask` bits are not set, the resulting permission will be `0777`. +* `maxthreads`: (optional) The maximum number of simultaneous blocking filesystem operations permitted within the registry. Each operation spawns a new thread and may cause thread exhaustion issues if many are done in parallel. Defaults to -`100`, and can be no lower than `25`. +`100`, and cannot be lower than `25`. diff --git a/docs/storage-drivers/gcs.md b/docs/storage-drivers/gcs.md index 1bc67f9e..d9184254 100644 --- a/docs/storage-drivers/gcs.md +++ b/docs/storage-drivers/gcs.md @@ -1,78 +1,21 @@ - +--- +description: Explains how to use the Google Cloud Storage drivers +keywords: registry, service, driver, images, storage, gcs, google, cloud +title: Google Cloud Storage driver +--- - -# Google Cloud Storage driver +{% include registry.md %} An implementation of the `storagedriver.StorageDriver` interface which uses Google Cloud for object storage. ## Parameters +| Parameter | Required | Description | +|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `bucket` | yes | The name of your Google Cloud Storage bucket where you wish to store objects (needs to already be created prior to driver initialization). | +| `keyfile` | no | A private service account key file in JSON format used for [Service Account Authentication](https://cloud.google.com/storage/docs/authentication#service_accounts). | +| `rootdirectory` | no | The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root). If a prefix is used, the path `bucketname/` has to be pre-created before starting the registry. The prefix is applied to all Google Cloud Storage keys to allow you to segment data in your bucket if necessary.| +| `chunksize` | no (default 5242880) | This is the chunk size used for uploading large blobs, must be a multiple of 256*1024. | - - - - - - - - - - - - - - - - - - - - - - - - - +**Note:** Instead of a key file you can use [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials). -
ParameterRequiredDescription
- bucket - - yes - - Storage bucket name. -
- keyfile - - no - - A private service account key file in JSON format. Instead of a key file Google Application Default Credentials can be used. -
- rootdirectory - - no - - This is a prefix that will be applied to all Google Cloud Storage keys to allow you to segment data in your bucket if necessary. -
- chunksize - - no (default 5242880) - - This is the chunk size used for uploading large blobs, must be a multiple of 256*1024. -
- - -`bucket`: The name of your Google Cloud Storage bucket where you wish to store objects (needs to already be created prior to driver initialization). - -`keyfile`: (optional) A private key file in JSON format, used for [Service Account Authentication](https://cloud.google.com/storage/docs/authentication#service_accounts). - -**Note** Instead of a key file you can use [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials). - -`rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root). diff --git a/docs/storage-drivers/index.md b/docs/storage-drivers/index.md index 89635bd3..750791bb 100644 --- a/docs/storage-drivers/index.md +++ b/docs/storage-drivers/index.md @@ -1,34 +1,28 @@ - +--- +description: Explains how to use storage drivers +keywords: registry, on-prem, images, tags, repository, distribution, storage drivers, advanced +redirect_from: +- /registry/storagedrivers/ +title: Docker Registry storage driver +--- - -# Docker Registry Storage Driver +{% include registry.md %} This document describes the registry storage driver model, implementation, and explains how to contribute new storage drivers. -## Provided Drivers +## Provided drivers This storage driver package comes bundled with several drivers: - [inmemory](inmemory.md): A temporary storage driver using a local inmemory map. This exists solely for reference and testing. - [filesystem](filesystem.md): A local storage driver configured to use a directory tree in the local filesystem. -- [s3](s3.md): A driver storing objects in an Amazon Simple Storage Solution (S3) bucket. -- [azure](azure.md): A driver storing objects in [Microsoft Azure Blob Storage](http://azure.microsoft.com/en-us/services/storage/). -- [swift](swift.md): A driver storing objects in [Openstack Swift](http://docs.openstack.org/developer/swift/). -- [oss](oss.md): A driver storing objects in [Aliyun OSS](http://www.aliyun.com/product/oss). +- [s3](s3.md): A driver storing objects in an Amazon Simple Storage Service (S3) bucket. +- [azure](azure.md): A driver storing objects in [Microsoft Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/). +- [swift](swift.md): A driver storing objects in [Openstack Swift](https://docs.openstack.org/swift/latest/). +- [oss](oss.md): A driver storing objects in [Aliyun OSS](https://www.aliyun.com/product/oss). - [gcs](gcs.md): A driver storing objects in a [Google Cloud Storage](https://cloud.google.com/storage/) bucket. -## Storage Driver API +## Storage driver API The storage driver API is designed to model a filesystem-like key/value storage in a manner abstract enough to support a range of drivers from the local filesystem to Amazon S3 or other distributed object storage systems. @@ -37,16 +31,16 @@ Storage drivers are required to implement the `storagedriver.StorageDriver` inte Storage drivers are intended to be written in Go, providing compile-time validation of the `storagedriver.StorageDriver` interface. -## Driver Selection and Configuration +## Driver selection and configuration -The preferred method of selecting a storage driver is using the `StorageDriverFactory` interface in the `storagedriver/factory` package. These factories provide a common interface for constructing storage drivers with a parameters map. The factory model is based off of the [Register](http://golang.org/pkg/database/sql/#Register) and [Open](http://golang.org/pkg/database/sql/#Open) methods in the builtin [database/sql](http://golang.org/pkg/database/sql) package. +The preferred method of selecting a storage driver is using the `StorageDriverFactory` interface in the `storagedriver/factory` package. These factories provide a common interface for constructing storage drivers with a parameters map. The factory model is based on the [Register](https://golang.org/pkg/database/sql/#Register) and [Open](https://golang.org/pkg/database/sql/#Open) methods in the builtin [database/sql](https://golang.org/pkg/database/sql) package. Storage driver factories may be registered by name using the `factory.Register` method, and then later invoked by calling `factory.Create` with a driver name and parameters map. If no such storage driver can be found, -`factory.Create` will return an `InvalidStorageDriverError`. +`factory.Create` returns an `InvalidStorageDriverError`. -## Driver Contribution +## Driver contribution ### Writing new storage drivers diff --git a/docs/storage-drivers/inmemory.md b/docs/storage-drivers/inmemory.md index 1a14e77a..6e913045 100644 --- a/docs/storage-drivers/inmemory.md +++ b/docs/storage-drivers/inmemory.md @@ -1,15 +1,10 @@ - +--- +description: Explains how to use the in-memory storage drivers +keywords: registry, service, driver, images, storage, in-memory +title: In-memory storage driver (testing only) +--- - -# In-memory storage driver (Testing Only) +{% include registry.md %} For purely tests purposes, you can use the `inmemory` storage driver. This driver is an implementation of the `storagedriver.StorageDriver` interface which diff --git a/docs/storage-drivers/oss.md b/docs/storage-drivers/oss.md index a85e315e..4a18c1f5 100644 --- a/docs/storage-drivers/oss.md +++ b/docs/storage-drivers/oss.md @@ -1,126 +1,25 @@ - +--- +description: Explains how to use the Aliyun OSS storage driver +keywords: registry, service, driver, images, storage, OSS, aliyun +title: Aliyun OSS storage driver +--- -# Aliyun OSS storage driver +{% include registry.md %} -An implementation of the `storagedriver.StorageDriver` interface which uses [Aliyun OSS](http://www.aliyun.com/product/oss) for object storage. +An implementation of the `storagedriver.StorageDriver` interface which uses +[Aliyun OSS](https://www.alibabacloud.com/product/oss) for object storage. ## Parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ParameterRequiredDescription
- accesskeyid - -yes - -Your access key ID. -
- accesskeysecret - -yes - -Your access key secret. -
- region - -yes - The name of the OSS region in which you would like to store objects (for example `oss-cn-beijing`). For a list of regions, you can look at -
- endpoint - -no - -An endpoint which defaults to `..aliyuncs.com` or `.-internal.aliyuncs.com` (when `internal=true`). You can change the default endpoint by changing this value. -
- internal - -no - An internal endpoint or the public endpoint for OSS access. The default is false. For a list of regions, you can look at -
- bucket - -yes - The name of your OSS bucket where you wish to store objects (needs to already be created prior to driver initialization). -
- encrypt - -no - Specifies whether you would like your data encrypted on the server side. Defaults to false if not specified. -
- secure - -no - Specifies whether to transfer data to the bucket over ssl or not. If you omit this value, `true` is used. -
- chunksize - -no - The default part size for multipart uploads (performed by WriteStream) to OSS. The default is 10 MB. Keep in mind that the minimum part size for OSS is 5MB. You might experience better performance for larger chunk sizes depending on the speed of your connection to OSS. -
- rootdirectory - -no - The root directory tree in which to store all registry files. Defaults to an empty string (bucket root). -
+| Parameter | Required | Description | +|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `accesskeyid` | yes | Your access key ID. | +| `accesskeysecret` | yes | Your access key secret. | +| `region` | yes | The name of the OSS region in which you would like to store objects (for example oss-cn-beijing). For a list of regions, you can look at the [official documentation](https://www.alibabacloud.com/help/doc-detail/31837.html). | +| `endpoint` | no | An endpoint which defaults to `[bucket].[region].aliyuncs.com` or `[bucket].[region]-internal.aliyuncs.com` (when `internal=true`). You can change the default endpoint by changing this value. | +| `internal` | no | An internal endpoint or the public endpoint for OSS access. The default is false. For a list of regions, you can look at the [official documentation](https://www.alibabacloud.com/help/doc-detail/31837.html). | +| `bucket` | yes | The name of your OSS bucket where you wish to store objects (needs to already be created prior to driver initialization). | +| `encrypt` | no | Specifies whether you would like your data encrypted on the server side. Defaults to false if not specified. | +| `secure` | no | Specifies whether to transfer data to the bucket over ssl or not. If you omit this value, `true` is used. | +| `chunksize` | no | The default part size for multipart uploads (performed by WriteStream) to OSS. The default is 10 MB. Keep in mind that the minimum part size for OSS is 5MB. You might experience better performance for larger chunk sizes depending on the speed of your connection to OSS. | +| `rootdirectory` | no | The root directory tree in which to store all registry files. Defaults to an empty string (bucket root). | diff --git a/docs/storage-drivers/s3.md b/docs/storage-drivers/s3.md index 30187db4..f33d5fbe 100644 --- a/docs/storage-drivers/s3.md +++ b/docs/storage-drivers/s3.md @@ -1,225 +1,39 @@ - +--- +description: Explains how to use the S3 storage drivers +keywords: registry, service, driver, images, storage, S3 +title: S3 storage driver +--- +{% include registry.md %} -# S3 storage driver - -An implementation of the `storagedriver.StorageDriver` interface which uses Amazon S3 or S3 compatible services for object storage. +An implementation of the `storagedriver.StorageDriver` interface which uses +Amazon S3 or S3 compatible services for object storage. ## Parameters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ParameterRequiredDescription
- accesskey - - yes - - Your AWS Access Key. -
- secretkey - - yes - - Your AWS Secret Key. -
- region - - yes - - The AWS region in which your bucket exists. For the moment, the Go AWS - library in use does not use the newer DNS based bucket routing. -
- regionendpoint - - no - - Endpoint for S3 compatible storage services (Minio, etc) -
- bucket - - yes - - The bucket name in which you want to store the registry's data. -
- encrypt - - no - - Specifies whether the registry stores the image in encrypted format or - not. A boolean value. The default is false. -
- keyid - - no - - Optional KMS key ID to use for encryption (encrypt must be true, or this - parameter will be ignored). The default is none. -
- secure - - no - - Indicates whether to use HTTPS instead of HTTP. A boolean value. The - default is true. -
- v4auth - - no - - Indicates whether the registry uses Version 4 of AWS's authentication. - Generally, you should set this to true unless you are using an - S3 compatible provider that does not support v4 signature signing. - If you set this to false then the storage driver will use v2 signature signing. - By default, this is true. - You can not use v2 signing if you are using AWS S3. -
- chunksize - - no - - The S3 API requires multipart upload chunks to be at least 5MB. This value - should be a number that is larger than 5*1024*1024. -
- multipartcopychunksize - - no - - Chunk size for all but the last Upload Part - Copy - operation of a copy that uses the multipart upload API. -
- multipartcopymaxconcurrency - - no - - Maximum number of concurrent Upload Part - Copy operations for a - copy that uses the multipart upload API. -
- multipartcopythresholdsize - - no - - Objects above this size will be copied using the multipart upload API. - PUT Object - Copy is used for objects at or below this size. -
- rootdirectory - - no - - This is a prefix that will be applied to all S3 keys to allow you to segment data in your bucket if necessary. -
- storageclass - - no - - The S3 storage class applied to each registry file. The default value is STANDARD. -
- objectacl - - no - - The S3 Canned ACL for objects. The default value is "private". -
+| Parameter | Required | Description | +|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `accesskey` | no | Your AWS Access Key. If you use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), omit to fetch temporary credentials from IAM. | +| `secretkey` | no | Your AWS Secret Key. If you use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), omit to fetch temporary credentials from IAM. | +| `region` | yes | The AWS region in which your bucket exists. For the moment, the Go AWS library in use does not use the newer DNS based bucket routing. | +| `regionendpoint` | no | Endpoint for S3 compatible storage services (Minio, etc). | +| `bucket` | yes | The bucket name in which you want to store the registry's data. | +| `encrypt` | no | Specifies whether the registry stores the image in encrypted format or not. A boolean value. The default is `false`. | +| `keyid` | no | Optional KMS key ID to use for encryption (encrypt must be true, or this parameter is ignored). The default is `none`. | +| `secure` | no | Indicates whether to use HTTPS instead of HTTP. A boolean value. The default is `true`. | +| `skipverify` | no | Skips TLS verification when the value is set to `true`. The default is `false`. | +| `v4auth` | no | Indicates whether the registry uses Version 4 of AWS's authentication. The default is `true`. | +| `chunksize` | no | The S3 API requires multipart upload chunks to be at least 5MB. This value should be a number that is larger than 5 * 1024 * 1024.| +| `rootdirectory` | no | This is a prefix that is applied to all S3 keys to allow you to segment data in your bucket if necessary. | +| `storageclass` | no | The S3 storage class applied to each registry file. The default is `STANDARD`. | +| `objectacl` | no | The S3 Canned ACL for objects. The default value is "private". | +> **Note** You can provide empty strings for your access and secret keys to run the driver +> on an ec2 instance and handles authentication with the instance's credentials. If you +> use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), +> omit these keys to fetch temporary credentials from IAM. -`accesskey`: Your aws access key. - -`secretkey`: Your aws secret key. - -**Note** You can provide empty strings for your access and secret keys if you plan on running the driver on an ec2 instance and will handle authentication with the instance's credentials. - -`region`: The name of the aws region in which you would like to store objects (for example `us-east-1`). For a list of regions, you can look at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html +`region`: The name of the aws region in which you would like to store objects (for example `us-east-1`). For a list of regions, see [Regions, Availability Zones, and Local Zones](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). `regionendpoint`: (optional) Endpoint URL for S3 compatible APIs. This should not be provided when using Amazon S3. @@ -227,64 +41,87 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Amaz `encrypt`: (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified). -`keyid`: (optional) Whether you would like your data encrypted with this KMS key ID (defaults to none if not specified, will be ignored if encrypt is not true). +`keyid`: (optional) Whether you would like your data encrypted with this KMS key ID (defaults to none if not specified, is ignored if encrypt is not true). -`secure`: (optional) Whether you would like to transfer data to the bucket over ssl or not. Defaults to true (meaning transferring over ssl) if not specified. Note that while setting this to false will improve performance, it is not recommended due to security concerns. +`secure`: (optional) Whether you would like to transfer data to the bucket over ssl or not. Defaults to true (meaning transferring over ssl) if not specified. While setting this to false improves performance, it is not recommended due to security concerns. -`v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to true if not specified (note that the eu-central-1 region does not work with version 2 signatures, so the driver will error out if initialized with this region and v4auth set to false) +`v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to `false` if not specified. The `eu-central-1` region does not work with version 2 signatures, so the driver errors out if initialized with this region and v4auth set to `false`. -`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance; faster connections will benefit from larger chunk sizes. +`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance; faster connections benefit from larger chunk sizes. -`rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root). +`rootdirectory`: (optional) The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root). -`storageclass`: (optional) The storage class applied to each registry file. Defaults to STANDARD. Valid options are NONE, STANDARD and REDUCED_REDUNDANCY. Use NONE if your S3 compatible provider does not support storage classes. +`storageclass`: (optional) The storage class applied to each registry file. Defaults to STANDARD. Valid options are STANDARD and REDUCED_REDUNDANCY. `objectacl`: (optional) The canned object ACL to be applied to each registry object. Defaults to `private`. If you are using a bucket owned by another AWS account, it is recommended that you set this to `bucket-owner-full-control` so that the bucket owner can access your objects. Other valid options are available in the [AWS S3 documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl). + ## S3 permission scopes -The following IAM permissions are required by the registry for push and pull. See [the S3 policy documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuAndPermissions.html) for more details. +The following AWS policy is required by the registry for push and pull. Make sure to replace `S3_BUCKET_NAME` with the name of your bucket. ``` - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "s3:ListBucket", - "s3:GetBucketLocation", - "s3:ListBucketMultipartUploads" - ], - "Resource": "arn:aws:s3:::mybucket" - }, - { - "Effect": "Allow", - "Action": [ - "s3:PutObject", - "s3:GetObject", - "s3:DeleteObject", - "s3:ListMultipartUploadParts", - "s3:AbortMultipartUpload" - ], - "Resource": "arn:aws:s3:::mybucket/*" - } -] +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:ListBucket", + "s3:GetBucketLocation", + "s3:ListBucketMultipartUploads" + ], + "Resource": "arn:aws:s3:::S3_BUCKET_NAME" + }, + { + "Effect": "Allow", + "Action": [ + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject", + "s3:ListMultipartUploadParts", + "s3:AbortMultipartUpload" + ], + "Resource": "arn:aws:s3:::S3_BUCKET_NAME/*" + } + ] +} ``` +See [the S3 policy documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuAndPermissions.html) for more details. + # CloudFront as Middleware with S3 backend ## Use Case -Adding CloudFront as a middleware for your S3 backed registry can dramatically improve pull times. Your registry will have the ability to retrieve your images from edge servers, rather than the geographically limited location of your S3 bucket. The farther your registry is from your bucket, the more improvements you will see. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/). +Adding CloudFront as a middleware for your S3 backed registry can dramatically +improve pull times. Your registry can retrieve your images +from edge servers, rather than the geographically limited location of your S3 +bucket. The farther your registry is from your bucket, the more improvements are +possible. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/). + +An alternative method for CloudFront that requires less configuration and will use +the same edge servers is [S3 Transfer Acceleration](https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html). +Please check acceleration [Requirements](https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html#transfer-acceleration-requirements) +to see whether you need CloudFront or S3 Transfer Acceleration. ## Configuring CloudFront for Distribution -If you are unfamiliar with creating a CloudFront distribution, see [Getting Started with Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html). +If you are unfamiliar with creating a CloudFront distribution, see [Getting +Started with +Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html). Defaults can be kept in most areas except: ### Origin: -The CloudFront distribution must be created such that the `Origin Path` is set to the directory level of the root "docker" key in S3. If your registry exists on the root of the bucket, this path should be left blank. + - The CloudFront distribution must be created such that the `Origin Path` is set + to the directory level of the root "docker" key in S3. If your registry exists + on the root of the bucket, this path should be left blank. + + - For private S3 buckets, you must set `Restrict Bucket Access` to `Yes`. See + the [CloudFront documentation](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html). + ### Behaviors: @@ -296,10 +133,13 @@ The CloudFront distribution must be created such that the `Origin Path` is set t ## Registry configuration -Here the `middleware` option is used. It is still important to keep the `storage` option as CloudFront will only handle `pull` actions; `push` actions are still directly written to S3. +Here the `middleware` option is used. It is still important to keep the +`storage` option, because CloudFront only handles `pull` actions; `push` actions +are still directly written to S3. -The following example shows what you will need at minimum: -``` +The following example shows a minimum configuration: + +```yaml ... storage: s3: @@ -317,4 +157,6 @@ middleware: ## CloudFront Key-Pair -A CloudFront key-pair is required for all AWS accounts needing access to your CloudFront distribution. For information, please see [Creating CloudFront Key Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). +A CloudFront key-pair is required for all AWS accounts needing access to your +CloudFront distribution. You must have access to your AWS account's root credentials to create the required Cloudfront keypair. For information, see [Creating CloudFront Key +Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). diff --git a/docs/storage-drivers/swift.md b/docs/storage-drivers/swift.md index 0bbf98c5..3b88e8bb 100644 --- a/docs/storage-drivers/swift.md +++ b/docs/storage-drivers/swift.md @@ -1,268 +1,44 @@ - +--- +description: Explains how to use the OpenStack swift storage driver +keywords: registry, service, driver, images, storage, swift +title: OpenStack Swift storage driver +--- +{% include registry.md %} -# OpenStack Swift storage driver - -An implementation of the `storagedriver.StorageDriver` interface that uses [OpenStack Swift](http://docs.openstack.org/developer/swift/) for object storage. +An implementation of the `storagedriver.StorageDriver` interface that uses +[OpenStack Swift](http://docs.openstack.org/developer/swift/) for object +storage. ## Parameters +| Parameter | Required | Description | +|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `authurl` | yes | URL for obtaining an auth token. https://storage.myprovider.com/v2.0 or https://storage.myprovider.com/v3/auth | +| `username` | yes | Your Openstack user name. | +| `password` | yes | Your Openstack password. | +| `region` | no | The Openstack region in which your container exists. | +| `container` | yes | The name of your Swift container where you wish to store the registry's data. The driver creates the named container during its initialization. | +| `tenant` | no | Your Openstack tenant name. You can either use `tenant` or `tenantid`. | +| `tenantid` | no | Your Openstack tenant name. You can either use `tenant` or `tenantid`. | +| `domain` | no | Your Openstack domain name for Identity v3 API. You can either use `domain` or `domainid`. | +| `domainid` | no | Your Openstack domain name for Identity v3 API. You can either use `domain` or `domainid`. | +| `tenantdomain` | no | Your tenant's Openstack domain name for Identity v3 API. Only necessary if different from the domain. You can either use `tenantdomain` or `tenantdomainid`. | +| `tenantdomainid` | no | Your tenant's Openstack domain id for Identity v3 API. Only necessary if different from the domain. You can either use `tenantdomain` or `tenantdomainid`. | +| `trustid` | no | Your Openstack trust ID for Identity v3 API. | +| `insecureskipverify` | no | Skips TLS verification if the value is wet to `true`. The default is `false`. | +| `chunksize` | no | Size of the data segments for the Swift Dynamic Large Objects. This value should be a number (defaults to 5M). | +| `prefix` | no | This is a prefix that is applied to all Swift keys to allow you to segment data in your container if necessary. Defaults to the empty string which is the container's root. | +| `secretkey` | no | The secret key used to generate temporary URLs. | +| `accesskey` | no | The access key to generate temporary URLs. It is used by HP Cloud Object Storage in addition to the `secretkey` parameter. | +| `authversion` | no | Specify the OpenStack Auth's version, for example `3`. By default the driver autodetects the auth's version from the AuthURL. | +| `endpointtype` | no | The endpoint type used when connecting to swift. Possible values are `public`, `internal`, and `admin`. The default is `public`. | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ParameterRequiredDescription
- authurl - - yes - - URL for obtaining an auth token. https://storage.myprovider.com/v2.0 or https://storage.myprovider.com/v3/auth -
- username - - yes - - Your Openstack user name. -
- password - - yes - - Your Openstack password. -
- region - - no - - The Openstack region in which your container exists. -
- container - - yes - - The name of your Swift container where you wish to store the registry's data. The driver creates the named container during its initialization. -
- tenant - - no - - Your Openstack tenant name. You can either use tenant or tenantid. -
- tenantid - - no - - Your Openstack tenant id. You can either use tenant or tenantid. -
- domain - - no - - Your user's Openstack domain name for Identity v3 API. You can either use domain or domainid. -
- domainid - - no - - Your user's Openstack domain id for Identity v3 API. You can either use domain or domainid. -
- tenantdomain - - no - - Your tenant's Openstack domain name for Identity v3 API. Only necessary if different from the domain. You can either use tenantdomain or tenantdomainid. -
- tenantdomainid - - no - - Your tenant's Openstack domain id for Identity v3 API. Only necessary if different from the domain. You can either use tenantdomain or tenantdomainid. -
- trustid - - no - - Your Openstack trust id for Identity v3 API. -
- insecureskipverify - - no - - true to skip TLS verification, false by default. -
- chunksize - - no - - Size of the data segments for the Swift Dynamic Large Objects. This value should be a number (defaults to 5M). -
- prefix - - no - - This is a prefix that will be applied to all Swift keys to allow you to segment data in your container if necessary. Defaults to the empty string which is the container's root. -
- secretkey - - no - - The secret key used to generate temporary URLs. -
- accesskey - - no - - The access key to generate temporary URLs. It is used by HP Cloud Object Storage in addition to the `secretkey` parameter. -
- authversion - - no - - Specify the OpenStack Auth's version,for example 3. By default the driver will autodetect the auth's version from the AuthURL. -
- endpointtype - - no - - The endpoint type used when connecting to swift. Possible values are `public`, `internal` and `admin`. Default is `public`. -
+The features supported by the Swift server are queried by requesting the `/info` +URL on the server. In case the administrator disabled that feature, the +configuration file can specify the following optional parameters : -The features supported by the Swift server are queried by requesting the `/info` URL on the server. In case the administrator -disabled that feature, the configuration file can specify the following optional parameters : - - - - - - - - - - -
- tempurlcontainerkey - -

- Specify whether to use container secret key to generate temporary URL when set to true, or the account secret key otherwise.

-

-
- tempurlmethods - -

- Array of HTTP methods that are supported by the TempURL middleware of the Swift server. Example:

- - - tempurlmethods: - - GET - - PUT - - HEAD - - POST - - DELETE - -

-
+| Optional parameter | Description | +|:--------------|:---------| +| `tempurlcontainerkey` | Specify whether to use container secret key to generate temporary URL when set to true, or the account secret key otherwise. | +| `tempurlmethods` | Array of HTTP methods that are supported by the TempURL middleware of the Swift server. For example: `["GET", "PUT", "HEAD", "POST", "DELETE"]` |