forked from TrueCloudLab/distribution
Various copyedits to reduce future tense, wordiness, and use of 'please' (#5788)
* Reword lots of instances of 'will' * Reword lots of instances of won't * Reword lots of instances of we'll * Eradicate you'll * Eradicate 'be able to' type of phrases * Eradicate 'unable to' type of phrases * Eradicate 'has / have to' type of phrases * Eradicate 'note that' type of phrases * Eradicate 'in order to' type of phrases * Redirect to official Chef and Puppet docs * Eradicate gratuitous 'please' * Reduce use of e.g. * Reduce use of i.e. * Reduce use of N.B. * Get rid of 'sexagesimal' and correct some errors
This commit is contained in:
parent
b5bbca9ed4
commit
f1fb06838a
14 changed files with 114 additions and 122 deletions
|
@ -5,20 +5,20 @@ title: Registry compatibility
|
||||||
---
|
---
|
||||||
|
|
||||||
## Synopsis
|
## Synopsis
|
||||||
*If a manifest is pulled by _digest_ from a registry 2.3 with Docker Engine 1.9
|
If a manifest is pulled by _digest_ from a registry 2.3 with Docker Engine 1.9
|
||||||
and older, and the manifest was pushed with Docker Engine 1.10, a security check
|
and older, and the manifest was pushed with Docker Engine 1.10, a security check
|
||||||
will cause the Engine to receive a manifest it cannot use and the pull will fail.*
|
causes the Engine to receive a manifest it cannot use and the pull fails.
|
||||||
|
|
||||||
## Registry manifest support
|
## Registry manifest support
|
||||||
|
|
||||||
Historically, the registry has supported a [single manifest type](./spec/manifest-v2-1.md)
|
Historically, the registry has supported a [single manifest type](./spec/manifest-v2-1.md)
|
||||||
known as _Schema 1_.
|
known as _Schema 1_.
|
||||||
|
|
||||||
With the move toward multiple architecture images the distribution project
|
With the move toward multiple architecture images, the distribution project
|
||||||
introduced two new manifest types: Schema 2 manifests and manifest lists. The
|
introduced two new manifest types: Schema 2 manifests and manifest lists. Registry
|
||||||
registry 2.3 supports all three manifest types and in order to be compatible
|
2.3 supports all three manifest types and sometimes performs an on-the-fly
|
||||||
with older Docker engines will, in certain cases, do an on-the-fly
|
transformation of a manifest before serving the JSON in the response, to
|
||||||
transformation of a manifest before serving the JSON in the response.
|
preserve compatibility with older versions of Docker Engine..
|
||||||
|
|
||||||
This conversion has some implications for pulling manifests by digest and this
|
This conversion has some implications for pulling manifests by digest and this
|
||||||
document enumerates these implications.
|
document enumerates these implications.
|
||||||
|
@ -28,7 +28,7 @@ document enumerates these implications.
|
||||||
|
|
||||||
Manifests are stored and retrieved in the registry by keying off a digest
|
Manifests are stored and retrieved in the registry by keying off a digest
|
||||||
representing a hash of the contents. One of the advantages provided by CAS is
|
representing a hash of the contents. One of the advantages provided by CAS is
|
||||||
security: if the contents are changed, then the digest will no longer match.
|
security: if the contents are changed, then the digest no longer matches.
|
||||||
This prevents any modification of the manifest by a MITM attack or an untrusted
|
This prevents any modification of the manifest by a MITM attack or an untrusted
|
||||||
third party.
|
third party.
|
||||||
|
|
||||||
|
@ -36,9 +36,9 @@ When a manifest is stored by the registry, this digest is returned in the HTTP
|
||||||
response headers and, if events are configured, delivered within the event. The
|
response headers and, if events are configured, delivered within the event. The
|
||||||
manifest can either be retrieved by the tag, or this digest.
|
manifest can either be retrieved by the tag, or this digest.
|
||||||
|
|
||||||
For registry versions 2.2.1 and below, the registry will always store and
|
For registry versions 2.2.1 and below, the registry always stores and
|
||||||
serve _Schema 1_ manifests. The Docker Engine 1.10 will first
|
serves _Schema 1_ manifests. Engine 1.10 first
|
||||||
attempt to send a _Schema 2_ manifest, falling back to sending a
|
attempts to send a _Schema 2_ manifest, falling back to sending a
|
||||||
Schema 1 type manifest when it detects that the registry does not
|
Schema 1 type manifest when it detects that the registry does not
|
||||||
support the new version.
|
support the new version.
|
||||||
|
|
||||||
|
@ -47,32 +47,32 @@ support the new version.
|
||||||
|
|
||||||
### Manifest push with Docker 1.10
|
### Manifest push with Docker 1.10
|
||||||
|
|
||||||
The docker engine will construct a _Schema 2_ manifest which the
|
The Engine constructs a _Schema 2_ manifest which the
|
||||||
registry will persist to disk.
|
registry persists to disk.
|
||||||
|
|
||||||
When the manifest is pulled by digest or tag with Docker Engine 1.10, a
|
When the manifest is pulled by digest or tag with Docker Engine 1.10, a
|
||||||
_Schema 2_ manifest will be returned. The Docker Engine 1.10
|
_Schema 2_ manifest is returned. Docker Engine 1.10
|
||||||
understands the new manifest format.
|
understands the new manifest format.
|
||||||
|
|
||||||
When the manifest is pulled by *tag* with Docker Engine 1.9 and older, the
|
When the manifest is pulled by *tag* with Docker Engine 1.9 and older, the
|
||||||
manifest is converted on-the-fly to _Schema 1_ and sent in the
|
manifest is converted on-the-fly to _Schema 1_ and sent in the
|
||||||
response. The Docker Engine 1.9 is compatible with this older format.
|
response. The Docker Engine 1.9 is compatible with this older format.
|
||||||
|
|
||||||
*When the manifest is pulled by _digest_ with Docker Engine 1.9 and older, the
|
When the manifest is pulled by _digest_ with Docker Engine 1.9 and older, the
|
||||||
same rewriting process will not happen in the registry. If this were to happen
|
same rewriting process does not happen in the registry. If it did,
|
||||||
the digest would no longer match the hash of the manifest and would violate the
|
the digest would no longer match the hash of the manifest and would violate the
|
||||||
constraints of CAS.*
|
constraints of CAS.
|
||||||
|
|
||||||
For this reason if a manifest is pulled by _digest_ from a registry 2.3 with Docker
|
For this reason if a manifest is pulled by _digest_ from a registry 2.3 with Docker
|
||||||
Engine 1.9 and older, and the manifest was pushed with Docker Engine 1.10, a
|
Engine 1.9 and older, and the manifest was pushed with Docker Engine 1.10, a
|
||||||
security check will cause the Engine to receive a manifest it cannot use and the
|
security check causes the Engine to receive a manifest it cannot use and the
|
||||||
pull will fail.
|
pull fails.
|
||||||
|
|
||||||
### Manifest push with Docker 1.9 and older
|
### Manifest push with Docker 1.9 and older
|
||||||
|
|
||||||
The Docker Engine will construct a _Schema 1_ manifest which the
|
The Docker Engine constructs a _Schema 1_ manifest which the
|
||||||
registry will persist to disk.
|
registry persists to disk.
|
||||||
|
|
||||||
When the manifest is pulled by digest or tag with any docker version, a
|
When the manifest is pulled by digest or tag with any docker version, a
|
||||||
_Schema 1_ manifest will be returned.
|
_Schema 1_ manifest is returned.
|
||||||
|
|
||||||
|
|
|
@ -183,7 +183,7 @@ service](#run-a-registry-as-a-service) below.
|
||||||
|
|
||||||
These examples assume the following:
|
These examples assume the following:
|
||||||
|
|
||||||
- Your registry will be accessible on `https://myregistry.domain.com/`.
|
- Your registry URL is `https://myregistry.domain.com/`.
|
||||||
- Your DNS, routing, and firewall settings allow access to the registry's host
|
- Your DNS, routing, and firewall settings allow access to the registry's host
|
||||||
on port 5000.
|
on port 5000.
|
||||||
- You have already obtained a certificate from a certificate authority (CA).
|
- You have already obtained a certificate from a certificate authority (CA).
|
||||||
|
@ -279,8 +279,8 @@ or a service with either only a single node or a node constraint.
|
||||||
fully replicated service. Each worker can write to the storage back-end
|
fully replicated service. Each worker can write to the storage back-end
|
||||||
without causing write conflicts.
|
without causing write conflicts.
|
||||||
|
|
||||||
- If you use a local bind mount or volume, each worker node will write to its
|
- If you use a local bind mount or volume, each worker node writes to its
|
||||||
own storage location, which means that each registry will contain a different
|
own storage location, which means that each registry contains a different
|
||||||
data set. You can solve this problem by using a single-replica service and a
|
data set. You can solve this problem by using a single-replica service and a
|
||||||
node constraint to ensure that only a single worker is writing to the bind
|
node constraint to ensure that only a single worker is writing to the bind
|
||||||
mount.
|
mount.
|
||||||
|
@ -348,15 +348,15 @@ the following must be the same:
|
||||||
- HTTP Secret
|
- HTTP Secret
|
||||||
- Redis Cache (if configured)
|
- Redis Cache (if configured)
|
||||||
|
|
||||||
If any of these are different, the registry will have trouble serving requests.
|
Differences in any of the above cause problems serving requests.
|
||||||
As an example, if you're using the filesystem driver, all registry instances
|
As an example, if you're using the filesystem driver, all registry instances
|
||||||
must have access to the same filesystem root, which means they should be in
|
must have access to the same filesystem root, on
|
||||||
the same machine. For other drivers, such as s3 or azure, they should be
|
the same machine. For other drivers, such as S3 or Azure, they should be
|
||||||
accessing the same resource, and will likely share an identical configuration.
|
accessing the same resource and share an identical configuration.
|
||||||
The _HTTP Secret_ coordinates uploads, so also must be the same across
|
The _HTTP Secret_ coordinates uploads, so also must be the same across
|
||||||
instances. Configuring different redis instances will work (at the time
|
instances. Configuring different redis instances works (at the time
|
||||||
of writing), but will not be optimal if the instances are not shared, causing
|
of writing), but is not optimal if the instances are not shared, because
|
||||||
more requests to be directed to the backend.
|
more requests are directed to the backend.
|
||||||
|
|
||||||
### Important/Required HTTP-Headers
|
### Important/Required HTTP-Headers
|
||||||
|
|
||||||
|
@ -377,11 +377,11 @@ without credentials. The response should include a `WWW-Authenticate`
|
||||||
challenge, providing guidance on how to authenticate, such as with basic auth
|
challenge, providing guidance on how to authenticate, such as with basic auth
|
||||||
or a token service. If the load balancer has health checks, it is recommended
|
or a token service. If the load balancer has health checks, it is recommended
|
||||||
to configure it to consider a 401 response as healthy and any other as down.
|
to configure it to consider a 401 response as healthy and any other as down.
|
||||||
This will secure your registry by ensuring that configuration problems with
|
This secures your registry by ensuring that configuration problems with
|
||||||
authentication don't accidentally expose an unprotected registry. If you're
|
authentication don't accidentally expose an unprotected registry. If you're
|
||||||
using a less sophisticated load balancer, such as Amazon's Elastic Load
|
using a less sophisticated load balancer, such as Amazon's Elastic Load
|
||||||
Balancer, that doesn't allow one to change the healthy response code, health
|
Balancer, that doesn't allow one to change the healthy response code, health
|
||||||
checks can be directed at "/", which will always return a `200 OK` response.
|
checks can be directed at "/", which always returns a `200 OK` response.
|
||||||
|
|
||||||
## Restricting access
|
## Restricting access
|
||||||
|
|
||||||
|
@ -436,7 +436,7 @@ secrets.
|
||||||
```
|
```
|
||||||
|
|
||||||
4. Try to pull an image from the registry, or push an image to the registry.
|
4. Try to pull an image from the registry, or push an image to the registry.
|
||||||
These commands will fail.
|
These commands fail.
|
||||||
|
|
||||||
5. Log in to the registry.
|
5. Log in to the registry.
|
||||||
|
|
||||||
|
@ -518,7 +518,7 @@ following:
|
||||||
distributable. This means that when you push an image based on one of these
|
distributable. This means that when you push an image based on one of these
|
||||||
images to your private registry, the non-distributable layers are **not**
|
images to your private registry, the non-distributable layers are **not**
|
||||||
pushed, but are always fetched from their authorized location. This is fine
|
pushed, but are always fetched from their authorized location. This is fine
|
||||||
for internet-connected hosts, but will not work in an air-gapped set-up.
|
for internet-connected hosts, but not in an air-gapped set-up.
|
||||||
|
|
||||||
In Docker 17.06 and higher, you can configure the Docker daemon to allow
|
In Docker 17.06 and higher, you can configure the Docker daemon to allow
|
||||||
pushing non-distributable layers to private registries, in this scenario.
|
pushing non-distributable layers to private registries, in this scenario.
|
||||||
|
@ -546,7 +546,7 @@ following:
|
||||||
3. Restart the registry if it does not start automatically.
|
3. Restart the registry if it does not start automatically.
|
||||||
|
|
||||||
4. When you push images to the registries in the list, their
|
4. When you push images to the registries in the list, their
|
||||||
non-distributable layers will be pushed to the registry.
|
non-distributable layers are pushed to the registry.
|
||||||
|
|
||||||
> **Warning**: Non-distributable artifacts typically have restrictions on
|
> **Warning**: Non-distributable artifacts typically have restrictions on
|
||||||
> how and where they can be distributed and shared. Only use this feature
|
> how and where they can be distributed and shared. Only use this feature
|
||||||
|
@ -557,7 +557,7 @@ following:
|
||||||
|
|
||||||
## Next steps
|
## Next steps
|
||||||
|
|
||||||
You will find more specific and advanced information in the following sections:
|
More specific and advanced information is available in the following sections:
|
||||||
|
|
||||||
- [Configuration reference](configuration.md)
|
- [Configuration reference](configuration.md)
|
||||||
- [Working with notifications](notifications.md)
|
- [Working with notifications](notifications.md)
|
||||||
|
|
|
@ -33,8 +33,8 @@ documentation [here](spec/api.md#deleting-a-layer) and
|
||||||
to the target and makes them eligible for garbage collection. It also makes them
|
to the target and makes them eligible for garbage collection. It also makes them
|
||||||
unable to be read via the API.
|
unable to be read via the API.
|
||||||
|
|
||||||
If a layer is deleted it will be removed from the filesystem when garbage collection
|
If a layer is deleted, it is removed from the filesystem when garbage collection
|
||||||
is run. If a manifest is deleted the layers to which it refers will be removed from
|
is run. If a manifest is deleted the layers to which it refers are removed from
|
||||||
the filesystem if no other manifests refers to them.
|
the filesystem if no other manifests refers to them.
|
||||||
|
|
||||||
|
|
||||||
|
@ -58,9 +58,9 @@ A -----> a B
|
||||||
```
|
```
|
||||||
|
|
||||||
In this state layer `c` no longer has a reference and is eligible for garbage
|
In this state layer `c` no longer has a reference and is eligible for garbage
|
||||||
collection. Layer `a` had one reference removed but will not be garbage
|
collection. Layer `a` had one reference removed but not garbage
|
||||||
collected as it is still referenced by manifest `A`. The blob representing
|
collected as it is still referenced by manifest `A`. The blob representing
|
||||||
manifest `B` will also be eligible for garbage collection.
|
manifest `B` is eligible for garbage collection.
|
||||||
|
|
||||||
After garbage collection has been run, manifest `A` and its blobs remain.
|
After garbage collection has been run, manifest `A` and its blobs remain.
|
||||||
|
|
||||||
|
@ -77,18 +77,14 @@ scans all the manifests in the registry. From these manifests, it constructs a
|
||||||
set of content address digests. This set is the 'mark set' and denotes the set
|
set of content address digests. This set is the 'mark set' and denotes the set
|
||||||
of blobs to *not* delete. Secondly, in the 'sweep' phase, the process scans all
|
of blobs to *not* delete. Secondly, in the 'sweep' phase, the process scans all
|
||||||
the blobs and if a blob's content address digest is not in the mark set, the
|
the blobs and if a blob's content address digest is not in the mark set, the
|
||||||
process will delete it.
|
process deletes it.
|
||||||
|
|
||||||
|
|
||||||
> **Note**: You should ensure that the registry is in read-only mode or not running at
|
> **Note**: You should ensure that the registry is in read-only mode or not running at
|
||||||
> all. If you were to upload an image while garbage collection is running, there is the
|
> all. If you were to upload an image while garbage collection is running, there is the
|
||||||
> risk that the image's layers will be mistakenly deleted, leading to a corrupted image.
|
> risk that the image's layers are mistakenly deleted leading to a corrupted image.
|
||||||
|
|
||||||
This type of garbage collection is known as stop-the-world garbage collection. In future
|
|
||||||
registry versions the intention is that garbage collection will be an automated background
|
|
||||||
action and this manual process will no longer apply.
|
|
||||||
|
|
||||||
|
|
||||||
|
This type of garbage collection is known as stop-the-world garbage collection.
|
||||||
|
|
||||||
## Run garbage collection
|
## Run garbage collection
|
||||||
|
|
||||||
|
@ -96,9 +92,9 @@ Garbage collection can be run as follows
|
||||||
|
|
||||||
`bin/registry garbage-collect [--dry-run] /path/to/config.yml`
|
`bin/registry garbage-collect [--dry-run] /path/to/config.yml`
|
||||||
|
|
||||||
The garbage-collect command accepts a `--dry-run` parameter, which will print the progress
|
The garbage-collect command accepts a `--dry-run` parameter, which prints the progress
|
||||||
of the mark and sweep phases without removing any data. Running with a log level of `info`
|
of the mark and sweep phases without removing any data. Running with a log level of `info`
|
||||||
will give a clear indication of what will and will not be deleted.
|
gives a clear indication of items eligible for deletion.
|
||||||
|
|
||||||
The config.yml file should be in the following format:
|
The config.yml file should be in the following format:
|
||||||
|
|
||||||
|
|
|
@ -110,12 +110,12 @@ This sections lists some common failures and how to recover from them.
|
||||||
### Failing...
|
### Failing...
|
||||||
|
|
||||||
Failing to configure the Engine daemon and trying to pull from a registry that is not using
|
Failing to configure the Engine daemon and trying to pull from a registry that is not using
|
||||||
TLS will results in the following message:
|
TLS results in the following message:
|
||||||
|
|
||||||
```none
|
```none
|
||||||
FATA[0000] Error response from daemon: v1 ping attempt failed with error:
|
FATA[0000] Error response from daemon: v1 ping attempt failed with error:
|
||||||
Get https://myregistrydomain.com:5000/v1/_ping: tls: oversized record received with length 20527.
|
Get https://myregistrydomain.com:5000/v1/_ping: tls: oversized record received with length 20527.
|
||||||
If this private registry supports only HTTP or HTTPS with an unknown CA certificate,please add
|
If this private registry supports only HTTP or HTTPS with an unknown CA certificate, add
|
||||||
`--insecure-registry myregistrydomain.com:5000` to the daemon's arguments.
|
`--insecure-registry myregistrydomain.com:5000` to the daemon's arguments.
|
||||||
In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag;
|
In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag;
|
||||||
simply place the CA certificate at /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt
|
simply place the CA certificate at /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt
|
||||||
|
|
|
@ -40,10 +40,10 @@ them to the configuration. A simple example follows:
|
||||||
The above would configure the registry with an endpoint to send events to
|
The above would configure the registry with an endpoint to send events to
|
||||||
`https://mylistener.example.com/event`, with the header "Authorization: Bearer
|
`https://mylistener.example.com/event`, with the header "Authorization: Bearer
|
||||||
<your token, if needed>". The request would timeout after 500 milliseconds. If
|
<your token, if needed>". The request would timeout after 500 milliseconds. If
|
||||||
5 failures happen consecutively, the registry will backoff for 1 second before
|
5 failures happen consecutively, the registry backs off for 1 second before
|
||||||
trying again.
|
trying again.
|
||||||
|
|
||||||
For details on the fields, please see the [configuration documentation](configuration.md#notifications).
|
For details on the fields, see the [configuration documentation](configuration.md#notifications).
|
||||||
|
|
||||||
A properly configured endpoint should lead to a log message from the registry
|
A properly configured endpoint should lead to a log message from the registry
|
||||||
upon startup:
|
upon startup:
|
||||||
|
@ -117,8 +117,8 @@ manifest:
|
||||||
|
|
||||||
|
|
||||||
The target struct of events which are sent when manifests and blobs are deleted
|
The target struct of events which are sent when manifests and blobs are deleted
|
||||||
will contain a subset of the data contained in Get and Put events. Specifically,
|
contains a subset of the data contained in Get and Put events. Specifically,
|
||||||
only the digest and repository will be sent.
|
only the digest and repository are sent.
|
||||||
|
|
||||||
```json
|
```json
|
||||||
"target": {
|
"target": {
|
||||||
|
@ -148,7 +148,7 @@ group unrelated events and send them in the same envelope to reduce the total
|
||||||
number of requests.
|
number of requests.
|
||||||
|
|
||||||
The full package has the mediatype
|
The full package has the mediatype
|
||||||
"application/vnd.docker.distribution.events.v1+json", which will be set on the
|
"application/vnd.docker.distribution.events.v1+json", which is set on the
|
||||||
request coming to an endpoint.
|
request coming to an endpoint.
|
||||||
|
|
||||||
An example of a full event may look as follows:
|
An example of a full event may look as follows:
|
||||||
|
@ -244,7 +244,7 @@ Content-Type: application/vnd.docker.distribution.events.v1+json
|
||||||
|
|
||||||
The registry is fairly accepting of the response codes from endpoints. If an
|
The registry is fairly accepting of the response codes from endpoints. If an
|
||||||
endpoint responds with any 2xx or 3xx response code (after following
|
endpoint responds with any 2xx or 3xx response code (after following
|
||||||
redirects), the message will be considered delivered and discarded.
|
redirects), the message is considered to have been delivered, and is discarded.
|
||||||
|
|
||||||
In turn, it is recommended that endpoints are accepting of incoming responses,
|
In turn, it is recommended that endpoints are accepting of incoming responses,
|
||||||
as well. While the format of event envelopes are standardized by media type,
|
as well. While the format of event envelopes are standardized by media type,
|
||||||
|
@ -312,15 +312,15 @@ monitor the size ("Pending" above) of the endpoint queues. If failures or
|
||||||
queue sizes are increasing, it can indicate a larger problem.
|
queue sizes are increasing, it can indicate a larger problem.
|
||||||
|
|
||||||
The logs are also a valuable resource for monitoring problems. A failing
|
The logs are also a valuable resource for monitoring problems. A failing
|
||||||
endpoint will lead to messages similar to the following:
|
endpoint leads to messages similar to the following:
|
||||||
|
|
||||||
```
|
```none
|
||||||
ERRO[0340] retryingsink: error writing events: httpSink{http://localhost:5003/callback}: error posting: Post http://localhost:5003/callback: dial tcp 127.0.0.1:5003: connection refused, retrying
|
ERRO[0340] retryingsink: error writing events: httpSink{http://localhost:5003/callback}: error posting: Post http://localhost:5003/callback: dial tcp 127.0.0.1:5003: connection refused, retrying
|
||||||
WARN[0340] httpSink{http://localhost:5003/callback} encountered too many errors, backing off
|
WARN[0340] httpSink{http://localhost:5003/callback} encountered too many errors, backing off
|
||||||
```
|
```
|
||||||
|
|
||||||
The above indicates that several errors have led to a backoff and the registry
|
The above indicates that several errors caused a backoff and the registry
|
||||||
will wait before retrying.
|
waits before retrying.
|
||||||
|
|
||||||
## Considerations
|
## Considerations
|
||||||
|
|
||||||
|
@ -328,7 +328,7 @@ Currently, the queues are inmemory, so endpoints should be _reasonably
|
||||||
reliable_. They are designed to make a best-effort to send the messages but if
|
reliable_. They are designed to make a best-effort to send the messages but if
|
||||||
an instance is lost, messages may be dropped. If an endpoint goes down, care
|
an instance is lost, messages may be dropped. If an endpoint goes down, care
|
||||||
should be taken to ensure that the registry instance is not terminated before
|
should be taken to ensure that the registry instance is not terminated before
|
||||||
the endpoint comes back up or messages will be lost.
|
the endpoint comes back up or messages are lost.
|
||||||
|
|
||||||
This can be mitigated by running endpoints in close proximity to the registry
|
This can be mitigated by running endpoints in close proximity to the registry
|
||||||
instances. One could run an endpoint that pages to disk and then forwards a
|
instances. One could run an endpoint that pages to disk and then forwards a
|
||||||
|
@ -338,6 +338,6 @@ The notification system is designed around a series of interchangeable _sinks_
|
||||||
which can be wired up to achieve interesting behavior. If this system doesn't
|
which can be wired up to achieve interesting behavior. If this system doesn't
|
||||||
provide acceptable guarantees, adding a transactional `Sink` to the registry
|
provide acceptable guarantees, adding a transactional `Sink` to the registry
|
||||||
is a possibility, although it may have an effect on request service time.
|
is a possibility, although it may have an effect on request service time.
|
||||||
Please see the
|
See the
|
||||||
[godoc](http://godoc.org/github.com/docker/distribution/notifications#Sink)
|
[godoc](http://godoc.org/github.com/docker/distribution/notifications#Sink)
|
||||||
for more information.
|
for more information.
|
||||||
|
|
|
@ -26,7 +26,7 @@ We also implement push restriction (to a limited user group) for the sake of the
|
||||||
|
|
||||||
While this model gives you the ability to use whatever authentication backend you want through the secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself.
|
While this model gives you the ability to use whatever authentication backend you want through the secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself.
|
||||||
|
|
||||||
Furthermore, introducing an extra http layer in your communication pipeline will make it more complex to deploy, maintain, and debug, and will possibly create issues.
|
Furthermore, introducing an extra http layer in your communication pipeline adds complexity when deploying, maintaining, and debugging.
|
||||||
|
|
||||||
## Setting things up
|
## Setting things up
|
||||||
|
|
||||||
|
@ -40,7 +40,7 @@ Run the following script:
|
||||||
mkdir -p auth
|
mkdir -p auth
|
||||||
mkdir -p data
|
mkdir -p data
|
||||||
|
|
||||||
# This is the main apache configuration you will use
|
# This is the main apache configuration
|
||||||
cat <<EOF > auth/httpd.conf
|
cat <<EOF > auth/httpd.conf
|
||||||
LoadModule headers_module modules/mod_headers.so
|
LoadModule headers_module modules/mod_headers.so
|
||||||
|
|
||||||
|
|
|
@ -4,15 +4,12 @@ keywords: registry, on-prem, images, tags, repository, distribution, recipes, ad
|
||||||
title: Recipes overview
|
title: Recipes overview
|
||||||
---
|
---
|
||||||
|
|
||||||
You will find here a list of "recipes", end-to-end scenarios for exotic or otherwise advanced use-cases.
|
This list of "recipes" provides end-to-end scenarios for exotic or otherwise advanced use-cases.
|
||||||
|
These recipes are not useful for most standard set-ups.
|
||||||
Most users are not expected to have a use for these.
|
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
You should have followed entirely the basic [deployment guide](../deploying.md).
|
Before following these steps, work through the [deployment guide](../deploying.md).
|
||||||
|
|
||||||
If you have not, please take the time to do so.
|
|
||||||
|
|
||||||
At this point, it's assumed that:
|
At this point, it's assumed that:
|
||||||
|
|
||||||
|
|
|
@ -8,12 +8,11 @@ redirect_from:
|
||||||
|
|
||||||
## Use-case
|
## Use-case
|
||||||
|
|
||||||
If you have multiple instances of Docker running in your environment (e.g.,
|
If you have multiple instances of Docker running in your environment, such as
|
||||||
multiple physical or virtual machines, all running the Docker daemon), each time
|
multiple physical or virtual machines all running Docker, each daemon goes out
|
||||||
one of them requires an image that it doesn’t have it will go out to the
|
to the internet and fetches an image it doesn't have locally, from the Docker
|
||||||
internet and fetch it from the public Docker registry. By running a local
|
repository. You can run a local registry mirror and point all your daemons
|
||||||
registry mirror, you can keep most of the redundant image fetch traffic on your
|
there, to avoid this extra internet traffic.
|
||||||
local network.
|
|
||||||
|
|
||||||
### Alternatives
|
### Alternatives
|
||||||
|
|
||||||
|
@ -42,15 +41,15 @@ serve the image from its own storage.
|
||||||
|
|
||||||
### What if the content changes on the Hub?
|
### What if the content changes on the Hub?
|
||||||
|
|
||||||
When a pull is attempted with a tag, the Registry will check the remote to
|
When a pull is attempted with a tag, the Registry checks the remote to
|
||||||
ensure if it has the latest version of the requested content. If it doesn't it
|
ensure if it has the latest version of the requested content. Otherwise, it
|
||||||
will fetch the latest content and cache it.
|
fetches and caches the latest content.
|
||||||
|
|
||||||
### What about my disk?
|
### What about my disk?
|
||||||
|
|
||||||
In environments with high churn rates, stale data can build up in the cache.
|
In environments with high churn rates, stale data can build up in the cache.
|
||||||
When running as a pull through cache the Registry will periodically remove old
|
When running as a pull through cache the Registry periodically removes old
|
||||||
content to save disk space. Subsequent requests for removed content will cause a
|
content to save disk space. Subsequent requests for removed content causes a
|
||||||
remote fetch and local re-caching.
|
remote fetch and local re-caching.
|
||||||
|
|
||||||
To ensure best performance and guarantee correctness the Registry cache should
|
To ensure best performance and guarantee correctness the Registry cache should
|
||||||
|
@ -62,15 +61,15 @@ The easiest way to run a registry as a pull through cache is to run the official
|
||||||
Registry image.
|
Registry image.
|
||||||
|
|
||||||
Multiple registry caches can be deployed over the same back-end. A single
|
Multiple registry caches can be deployed over the same back-end. A single
|
||||||
registry cache will ensure that concurrent requests do not pull duplicate data,
|
registry cache ensures that concurrent requests do not pull duplicate data,
|
||||||
but this property will not hold true for a registry cache cluster.
|
but this property does not hold true for a registry cache cluster.
|
||||||
|
|
||||||
### Configure the cache
|
### Configure the cache
|
||||||
|
|
||||||
To configure a Registry to run as a pull through cache, the addition of a
|
To configure a Registry to run as a pull through cache, the addition of a
|
||||||
`proxy` section is required to the config file.
|
`proxy` section is required to the config file.
|
||||||
|
|
||||||
In order to access private images on the Docker Hub, a username and password can
|
To access private images on the Docker Hub, a username and password can
|
||||||
be supplied.
|
be supplied.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
@ -81,11 +80,11 @@ proxy:
|
||||||
```
|
```
|
||||||
|
|
||||||
> **Warning**: If you specify a username and password, it's very important to
|
> **Warning**: If you specify a username and password, it's very important to
|
||||||
> understand that private resources that this user has access to Docker Hub will
|
> understand that private resources that this user has access to Docker Hub is
|
||||||
> be made available on your mirror. **You must secure your mirror** by
|
> made available on your mirror. **You must secure your mirror** by
|
||||||
> implementing authentication if you expect these resources to stay private!
|
> implementing authentication if you expect these resources to stay private!
|
||||||
|
|
||||||
> **Warning**: In order for the scheduler to clean up old entries, `delete` must
|
> **Warning**: For the scheduler to clean up old entries, `delete` must
|
||||||
> be enabled in the registry configuration. See
|
> be enabled in the registry configuration. See
|
||||||
> [Registry Configuration](/registry/configuration.md) for more details.
|
> [Registry Configuration](/registry/configuration.md) for more details.
|
||||||
|
|
||||||
|
|
|
@ -38,9 +38,9 @@ you want through the secondary authentication mechanism implemented inside your
|
||||||
proxy, it also requires that you move TLS termination from the Registry to the
|
proxy, it also requires that you move TLS termination from the Registry to the
|
||||||
proxy itself.
|
proxy itself.
|
||||||
|
|
||||||
Furthermore, introducing an extra http layer in your communication pipeline will
|
Furthermore, introducing an extra http layer in your communication pipeline
|
||||||
make it more complex to deploy, maintain, and debug, and will possibly create
|
makes it more complex to deploy, maintain, and debug. Make sure the extra
|
||||||
issues. Make sure the extra complexity is required.
|
complexity is required.
|
||||||
|
|
||||||
For instance, Amazon's Elastic Load Balancer (ELB) in HTTPS mode already sets
|
For instance, Amazon's Elastic Load Balancer (ELB) in HTTPS mode already sets
|
||||||
the following client header:
|
the following client header:
|
||||||
|
@ -61,7 +61,7 @@ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
```
|
```
|
||||||
|
|
||||||
Otherwise Nginx will reset the ELB's values, and the requests will not be routed
|
Otherwise Nginx resets the ELB's values, and the requests are not routed
|
||||||
properly. For more information, see
|
properly. For more information, see
|
||||||
[#970](https://github.com/docker/distribution/issues/970).
|
[#970](https://github.com/docker/distribution/issues/970).
|
||||||
|
|
||||||
|
@ -75,7 +75,7 @@ Review the [requirements](/registry/recipes/index.md#requirements), then follow
|
||||||
mkdir -p auth data
|
mkdir -p auth data
|
||||||
```
|
```
|
||||||
|
|
||||||
2. Create the main nginx configuration you will use. Paste this code block into a new file called `auth/nginx.conf`:
|
2. Create the main nginx configuration. Paste this code block into a new file called `auth/nginx.conf`:
|
||||||
|
|
||||||
```conf
|
```conf
|
||||||
events {
|
events {
|
||||||
|
@ -91,7 +91,7 @@ Review the [requirements](/registry/recipes/index.md#requirements), then follow
|
||||||
## Set a variable to help us decide if we need to add the
|
## Set a variable to help us decide if we need to add the
|
||||||
## 'Docker-Distribution-Api-Version' header.
|
## 'Docker-Distribution-Api-Version' header.
|
||||||
## The registry always sets this header.
|
## The registry always sets this header.
|
||||||
## In the case of nginx performing auth, the header will be unset
|
## In the case of nginx performing auth, the header is unset
|
||||||
## since nginx is auth-ing before proxying.
|
## since nginx is auth-ing before proxying.
|
||||||
map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
|
map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
|
||||||
'' 'registry/2.0';
|
'' 'registry/2.0';
|
||||||
|
@ -128,7 +128,7 @@ Review the [requirements](/registry/recipes/index.md#requirements), then follow
|
||||||
auth_basic "Registry realm";
|
auth_basic "Registry realm";
|
||||||
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
|
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
|
||||||
|
|
||||||
## If $docker_distribution_api_version is empty, the header will not be added.
|
## If $docker_distribution_api_version is empty, the header is not added.
|
||||||
## See the map directive above where this variable is defined.
|
## See the map directive above where this variable is defined.
|
||||||
add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
|
add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
|
||||||
|
|
||||||
|
|
|
@ -12,7 +12,7 @@ An implementation of the `storagedriver.StorageDriver` interface which uses [Mic
|
||||||
|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||||
| `accountname` | yes | Name of the Azure Storage Account. |
|
| `accountname` | yes | Name of the Azure Storage Account. |
|
||||||
| `accountkey` | yes | Primary or Secondary Key for the Storage Account. |
|
| `accountkey` | yes | Primary or Secondary Key for the Storage Account. |
|
||||||
| `container` | yes | Name of the Azure root storage container in which all registry data will be stored. Must comply the storage container name [requirements](https://docs.microsoft.com/rest/api/storageservices/fileservices/naming-and-referencing-containers--blobs--and-metadata). |
|
| `container` | yes | Name of the Azure root storage container in which all registry data is stored. Must comply the storage container name [requirements](https://docs.microsoft.com/rest/api/storageservices/fileservices/naming-and-referencing-containers--blobs--and-metadata). |
|
||||||
| `realm` | no | Domain name suffix for the Storage Service API endpoint. For example realm for "Azure in China" would be `core.chinacloudapi.cn` and realm for "Azure Government" would be `core.usgovcloudapi.net`. By default, this is `core.windows.net`. |
|
| `realm` | no | Domain name suffix for the Storage Service API endpoint. For example realm for "Azure in China" would be `core.chinacloudapi.cn` and realm for "Azure Government" would be `core.usgovcloudapi.net`. By default, this is `core.windows.net`. |
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -48,7 +48,7 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Goog
|
||||||
no
|
no
|
||||||
</td>
|
</td>
|
||||||
<td>
|
<td>
|
||||||
This is a prefix that will be applied to all Google Cloud Storage keys to allow you to segment data in your bucket if necessary.
|
This is a prefix that is applied to all Google Cloud Storage keys to allow you to segment data in your bucket if necessary.
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
@ -72,4 +72,4 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Goog
|
||||||
|
|
||||||
**Note** Instead of a key file you can use [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials).
|
**Note** Instead of a key file you can use [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials).
|
||||||
|
|
||||||
`rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root).
|
`rootdirectory`: (optional) The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root).
|
||||||
|
|
|
@ -36,7 +36,7 @@ The preferred method of selecting a storage driver is using the `StorageDriverFa
|
||||||
Storage driver factories may be registered by name using the
|
Storage driver factories may be registered by name using the
|
||||||
`factory.Register` method, and then later invoked by calling `factory.Create`
|
`factory.Register` method, and then later invoked by calling `factory.Create`
|
||||||
with a driver name and parameters map. If no such storage driver can be found,
|
with a driver name and parameters map. If no such storage driver can be found,
|
||||||
`factory.Create` will return an `InvalidStorageDriverError`.
|
`factory.Create` returns an `InvalidStorageDriverError`.
|
||||||
|
|
||||||
## Driver contribution
|
## Driver contribution
|
||||||
|
|
||||||
|
|
|
@ -92,7 +92,7 @@ Amazon S3 or S3 compatible services for object storage.
|
||||||
</td>
|
</td>
|
||||||
<td>
|
<td>
|
||||||
Optional KMS key ID to use for encryption (encrypt must be true, or this
|
Optional KMS key ID to use for encryption (encrypt must be true, or this
|
||||||
parameter will be ignored). The default is none.
|
parameter is ignored). The default is none.
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
@ -139,7 +139,7 @@ Amazon S3 or S3 compatible services for object storage.
|
||||||
no
|
no
|
||||||
</td>
|
</td>
|
||||||
<td>
|
<td>
|
||||||
This is a prefix that will be applied to all S3 keys to allow you to segment data in your bucket if necessary.
|
This is a prefix that is applied to all S3 keys to allow you to segment data in your bucket if necessary.
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
@ -161,7 +161,7 @@ Amazon S3 or S3 compatible services for object storage.
|
||||||
`secretkey`: Your aws secret key.
|
`secretkey`: Your aws secret key.
|
||||||
|
|
||||||
> **Note** You can provide empty strings for your access and secret keys to run the driver
|
> **Note** You can provide empty strings for your access and secret keys to run the driver
|
||||||
> on an ec2 instance and will handle authentication with the instance's credentials. If you
|
> on an ec2 instance and handles authentication with the instance's credentials. If you
|
||||||
> use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html),
|
> use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html),
|
||||||
> omit these keys to fetch temporary credentials from IAM.
|
> omit these keys to fetch temporary credentials from IAM.
|
||||||
|
|
||||||
|
@ -173,15 +173,15 @@ Amazon S3 or S3 compatible services for object storage.
|
||||||
|
|
||||||
`encrypt`: (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified).
|
`encrypt`: (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified).
|
||||||
|
|
||||||
`keyid`: (optional) Whether you would like your data encrypted with this KMS key ID (defaults to none if not specified, will be ignored if encrypt is not true).
|
`keyid`: (optional) Whether you would like your data encrypted with this KMS key ID (defaults to none if not specified, is ignored if encrypt is not true).
|
||||||
|
|
||||||
`secure`: (optional) Whether you would like to transfer data to the bucket over ssl or not. Defaults to true (meaning transferring over ssl) if not specified. Note that while setting this to false will improve performance, it is not recommended due to security concerns.
|
`secure`: (optional) Whether you would like to transfer data to the bucket over ssl or not. Defaults to true (meaning transferring over ssl) if not specified. While setting this to false improves performance, it is not recommended due to security concerns.
|
||||||
|
|
||||||
`v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to false if not specified (note that the eu-central-1 region does not work with version 2 signatures, so the driver will error out if initialized with this region and v4auth set to false)
|
`v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to `false` if not specified. The `eu-central-1` region does not work with version 2 signatures, so the driver errors out if initialized with this region and v4auth set to `false`.
|
||||||
|
|
||||||
`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance; faster connections will benefit from larger chunk sizes.
|
`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance; faster connections benefit from larger chunk sizes.
|
||||||
|
|
||||||
`rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root).
|
`rootdirectory`: (optional) The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root).
|
||||||
|
|
||||||
`storageclass`: (optional) The storage class applied to each registry file. Defaults to STANDARD. Valid options are STANDARD and REDUCED_REDUNDANCY.
|
`storageclass`: (optional) The storage class applied to each registry file. Defaults to STANDARD. Valid options are STANDARD and REDUCED_REDUNDANCY.
|
||||||
|
|
||||||
|
@ -223,10 +223,10 @@ See [the S3 policy documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev
|
||||||
## Use Case
|
## Use Case
|
||||||
|
|
||||||
Adding CloudFront as a middleware for your S3 backed registry can dramatically
|
Adding CloudFront as a middleware for your S3 backed registry can dramatically
|
||||||
improve pull times. Your registry will have the ability to retrieve your images
|
improve pull times. Your registry can retrieve your images
|
||||||
from edge servers, rather than the geographically limited location of your S3
|
from edge servers, rather than the geographically limited location of your S3
|
||||||
bucket. The farther your registry is from your bucket, the more improvements you
|
bucket. The farther your registry is from your bucket, the more improvements are
|
||||||
will see. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/).
|
possible. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/).
|
||||||
|
|
||||||
## Configuring CloudFront for Distribution
|
## Configuring CloudFront for Distribution
|
||||||
|
|
||||||
|
@ -257,10 +257,10 @@ Defaults can be kept in most areas except:
|
||||||
## Registry configuration
|
## Registry configuration
|
||||||
|
|
||||||
Here the `middleware` option is used. It is still important to keep the
|
Here the `middleware` option is used. It is still important to keep the
|
||||||
`storage` option as CloudFront will only handle `pull` actions; `push` actions
|
`storage` option, because CloudFront only handles `pull` actions; `push` actions
|
||||||
are still directly written to S3.
|
are still directly written to S3.
|
||||||
|
|
||||||
The following example shows what you will need at minimum:
|
The following example shows a minimum configuration:
|
||||||
|
|
||||||
```
|
```
|
||||||
...
|
...
|
||||||
|
@ -281,5 +281,5 @@ middleware:
|
||||||
## CloudFront Key-Pair
|
## CloudFront Key-Pair
|
||||||
|
|
||||||
A CloudFront key-pair is required for all AWS accounts needing access to your
|
A CloudFront key-pair is required for all AWS accounts needing access to your
|
||||||
CloudFront distribution. You must have access to your AWS account's root credentials to create the required Cloudfront keypair. For information, please see [Creating CloudFront Key
|
CloudFront distribution. You must have access to your AWS account's root credentials to create the required Cloudfront keypair. For information, see [Creating CloudFront Key
|
||||||
Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs).
|
Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs).
|
||||||
|
|
|
@ -157,7 +157,7 @@ storage.
|
||||||
no
|
no
|
||||||
</td>
|
</td>
|
||||||
<td>
|
<td>
|
||||||
This is a prefix that will be applied to all Swift keys to allow you to segment data in your container if necessary. Defaults to the empty string which is the container's root.
|
This is a prefix that is applied to all Swift keys to allow you to segment data in your container if necessary. Defaults to the empty string which is the container's root.
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
@ -190,7 +190,7 @@ storage.
|
||||||
no
|
no
|
||||||
</td>
|
</td>
|
||||||
<td>
|
<td>
|
||||||
Specify the OpenStack Auth's version,for example <code>3</code>. By default the driver will autodetect the auth's version from the AuthURL.
|
Specify the OpenStack Auth's version,for example <code>3</code>. By default the driver autodetects the auth's version from the AuthURL.
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
|
Loading…
Reference in a new issue