To simplify the vendoring story for the client, we have now removed the
requirement for `logrus` and the forked `context` package (usually
imported as `dcontext`). We inject the logger via the metrics tracker
for the blob cache and via options on the token handler. We preserve
logs on the proxy cache for that case. Clients expecting these log
messages may need to be updated accordingly.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.
To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The registry uses partial Named values which the named parsers
no longer support. To allow the registry service to continue
to operate without canonicalization, switch to use WithName.
In the future, the registry should start using fully canonical
values on the backend and WithName should no longer support
creating partial values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Enforces backwards compatibility with older authorization servers
without requiring the client to know about the compatibility
requirements.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Allow clients to handle errors being set in the WWW-Authenticate
rather than in the body. The WWW-Authenticate errors give a
more precise error describing what is needed to authorize
with the server.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Split challenges into its own package. Avoids possible
import cycle with challenges from client.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
If a user specifies `mydomain.com:443` in the `Host` configuration, the
PATCH request for the layer upload will fail because the challenge does not
appear to be in the map. To fix this, we normalize the map keys to always
use the Host:Port combination.
Closes https://github.com/docker/docker/issues/18469
Signed-off-by: Stan Hu <stanhu@gmail.com>
* tag service: properly handle error responses on HEAD requests by
re-issuing requests as GET for proper error details.
Fixes#1911.
Signed-off-by: dmitri <deemok@gmail.com>
* Simplify handling of failing HEAD requests in TagService and
make a GET request for cases:
- if the server does not handle HEAD
- if the response was an error to get error details
Signed-off-by: dmitri <deemok@gmail.com>
* Add a missing http.Response.Body.Close call for the GET request.
Signed-off-by: dmitri <deemok@gmail.com>
Allows using v2 for v1 endpoints.
The primary use case being for search which does not have a v2 specification.
Added a user scope for allowing v2 search
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The client may need the content digest to delete a manifest using the digest used by the registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
go1.5 doesn't export http.StatusTooManyRequests while
go1.6 does. Fix this by hardcoding the status code for now.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
If this slice ends up empty after parsing the HTTP response body, it
means the body is not well-formed. We've probably encountered an error
message produced by something that uses a different JSON schema, or
an error that just happens to validate as JSON.
An empty errcode.Errors slice is not a very useful thing to return,
since its Error() output is just `<nil>`. Detect this case, and instend
return an UnexpectedHTTPResponseError.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Updates registry storage code to use this for better resumable writes.
Implements this interface for the following drivers:
+ Inmemory
+ Filesystem
+ S3
+ Azure
Signed-off-by: Brian Bland <brian.bland@docker.com>
It is possible for a middlebox to lowercase the URL at somepoint causing a
lookup in the auth challenges table to fail. Lowercase hostname before
using as keys to challenge map.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Cross repository push tokens were not being cached and could not be used,
now any returned token will be used and the caching is hidden in the getToken function.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Login needs to add an offline token flag to ensure a refresh token is returned by the token endpoint.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Ensure Accept headers are sent with TagService.Get (which hits manifest
endpoints). Add support for remote Get and Put for the proxied blobstore.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Making this an exported error value will allow users of the
registry/client/auth module to have consistent behavior between
authentication failures and cases where no credentials are provided.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
A changeset under consideration for Go 1.7 would automatically copy
headers on redirect. This change future-proofs our code so we won't make
duplicate copies of the headers if net/http does it automatically in the
future.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This is needed for compatibility with some third-party registries that
send an inappropriate Content-Type header such as text/html.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
https://github.com/docker/distribution/pull/1249 changed token fetching
to parse HTTP error response bodies as serialized errcodes. However,
Docker Hub's authentication endpoint does not return error bodies in
this format. To work around this, convert its format into
ErrCodeUnauthorized or ErrCodeUnknown.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.
Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository
Signed-off-by: Brian Bland <brian.bland@docker.com>
When an auth request provides the "from" query parameter, the token
handler will add a "pull" scope for the provided repository, refreshing
the token if the overall scope has increased
Signed-off-by: Brian Bland <brian.bland@docker.com>
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.
Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.
Remove Signatures() from Repository interface.
Signatures are relevant only to schema1 manifests. Move access to the signature
store inside the schema1 manifestStore. Add some API tests to verify
signature roundtripping.
schema1
-------
Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.
tags
----
Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces. Implement a driver based tagService to be backward
compatible with the current tag service.
Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.
manifest store
--------------
Remove revision store and move all signing functionality into the signed manifeststore.
manifest registration
---------------------
Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.
client
------
Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs. Change functional arguments to be an interface type
to allow passing data without mutating shared state.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
The current implementation of digest.FromBytes returns an error. This
error can never be non-nil, but its presence in the function signature
means each call site needs error handling code for an error that is
always nil.
I verified that none of the hash.Hash implementations in the standard
library can return an error on Write. Nor can any of the hash.Hash
implementations vendored in distribution.
This commit changes digest.FromBytes not to return an error. If Write
returns an error, it will panic, but as discussed above, this should
never happen.
This commit also avoids using a bytes.Reader to feed data into the hash
function in FromBytes. This makes the hypothetical case that would panic
a bit more explicit, and should also be more performant.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This calls Stat before Open, which should be unnecessary because Open
can handle the case of a nonexistent blob. Removing the Stat saves a
round trip.
This is similar to the removal of stat in Open in #1226.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Without this commit, three round-trips are required to fetch a blob with
a progress bar. The first is a call to Stat (HEAD request), to determine
the size. Then Open is called, which also calls Stat, and finally
performs a GET request.
Only the GET request is actually needed. The size of the blob can be
sniffed from Content-Length in the GET response.
This commit changes HTTPReadSeeker to automatically detect the size from
Content-Length instead of requiring it to be passed in. The Stat call is
removed from Open because it is no longer necessary.
HTTPReadSeeker now takes an additional errorHandler callback argument which
translates an unsuccessful HTTP response into an appropriate API-level
error. Using a callback for this makes it possible to avoid leaking the
repsonse body to Read's caller, which would make lifecycle management
problematic.
Fixes#1223
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This extends the specification for the Bearer token response to include
information pertaining to when an issued Bearer token will expire.
This also allows the client to accept `access_token` as an alias for `token`.
Signed-off-by: Matt Moore <mattmoor@google.com>
To ensure that we only unmarshal the verified payload into the contained
manifest, we first copy the entire incoming buffer into Raw and then unmarshal
only the Payload portion of the incoming bytes. If the contents is later
verified, the caller can then be sure that the contents of the Manifest fields
can be trusted.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
There seems to be a need for a type that represents a way of pointing
to an image, irrespective of the implementation.
This patch defines a Reference interface and provides 3 implementations:
- TagReference: when only a tag is provided
- DigestReference: when a digest (according to the digest package) is
provided, can include optional tag as well
Validation of references are purely syntactic.
There is also a strong type for tags, analogous to digests, as well
as a strong type for Repository from which clients can access the
hostname alone, or the repository name without the hostname, or both
together via the String() method.
For Repository, the files names.go and names_test.go were moved from
the v2 package.
Signed-off-by: Tibor Vass <tibor@docker.com>
As we begin our march towards multi-arch, we must prepare for the reality of
multiple manifest schemas. This is the beginning of a set of changes to
facilitate this. We are both moving this package into its target position where
it may live peacefully next to other manfiest versions.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Several error codes are generally useful but tied to the v2 specification
definitions. This change moves these error code definitions into the common
package for use by the health package, which is not tied to the v2 API.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
with a new `proxy` section in the configuration file.
Create a new registry type which delegates storage to a proxyBlobStore
and proxyManifestStore. These stores will pull through data if not present
locally. proxyBlobStore takes care not to write duplicate data to disk.
Add a scheduler to cleanup expired content. The scheduler runs as a background
goroutine. When a blob or manifest is pulled through from the remote registry,
an entry is added to the scheduler with a TTL. When the TTL expires the
scheduler calls a pre-specified function to remove the fetched resource.
Add token authentication to the registry middleware. Get a token at startup
and preload the credential store with the username and password supplied in the
config file.
Allow resumable digest functionality to be disabled at runtime and disable
it when the registry is a pull through cache.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache. Deletion is configurable - if it is disabled API calls
will return an unsupported error.
We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.
Delete() is added throughout the relevant API to support this functionality.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
The way Repositories() was initially called was somewhat different than
other parts of the client bindings because there was no way to instantiate a
Namespace. This change implements a NewRegistry() function which changes
it so that Repositories() can be called the way one would expect.
It doesn't implement any of the other functions of Namespaces.
Signed-off-by: Patrick Devine <patrick.devine@docker.com>
