Commit graph

148 commits

Author SHA1 Message Date
Brian Bland
ff03381d49 Adds new storagedriver.FileWriter interface
Updates registry storage code to use this for better resumable writes.
Implements this interface for the following drivers:
 + Inmemory
 + Filesystem
 + S3
 + Azure

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-03-08 16:37:44 -08:00
Brian Bland
3ae504384c Merge pull request #1473 from miminar/pass-blob-size
Commit uploaded blob with size
2016-02-29 10:26:45 -08:00
Michal Minar
21a0878665 Commit blob with known size
Signed-off-by: Michal Minar <miminar@redhat.com>
2016-02-23 08:44:06 +01:00
Michal Minar
4d15bf071c Defined ErrAccessDenied error
Middleware code may perform additional checks on blobs written. Allow it
to return access denied errors that will result in 403 Forbidden.

Signed-off-by: Michal Minar <miminar@redhat.com>
2016-02-22 21:12:59 +01:00
Olivier Gambier
3c80f6fb5e Merge pull request #1451 from aaronlehmann/typo-fixes
Typo fixes in comments
2016-02-10 17:12:00 -08:00
Aaron Lehmann
d6a9b20971 Merge pull request #1445 from dmcgowan/fix-manifest-digest-header
Fix schema1 manifest etag and docker content digest header
2016-02-10 16:27:18 -08:00
Aaron Lehmann
aa80478b64 Typo fixes in comments
Correct spelling of words in source code comments.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-10 16:26:29 -08:00
Aaron Lehmann
a3213ff331 Merge pull request #1420 from dmcgowan/configurable-trust-key
Add option to disable signatures
2016-02-10 16:15:59 -08:00
Derek McGowan
b34e571bff Add option to disable signatures
Add option for specifying trust key for signing schema1 manifests.
Since schema1 signature key identifiers are not verified anywhere and deprecated, storing signatures is no longer a requirement.
Furthermore in schema2 there is no signature, requiring the registry to already add signatures to generated schema1 manifests.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-10 15:20:39 -08:00
Derek McGowan
350234898a Fix schema1 manifest etag and docker content digest header
When schema2 manifests are rewritten as schema1 currently the etag and docker content digest header keep the value for the schema2 manifest.

Fixes #1444

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-09 18:28:43 -08:00
Richard Scothern
b737de4856 Merge pull request #1408 from aaronlehmann/repository-interface
Rename Name method of Repository to Named
2016-02-04 16:43:35 -08:00
Aaron Lehmann
b0989446eb Rename Name method of Repository to Named
This makes code that gets the name as a string read like
repo.Named().Name() instead of repo.Name().Name().

Requested in
https://github.com/docker/docker/pull/19887#discussion_r51479753

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-04 09:55:36 -08:00
Aaron Lehmann
e10efb2ecc Merge pull request #1403 from dmcgowan/auth-const-keys
Update auth context keys to use constant
2016-02-01 16:29:07 -08:00
Aaron Lehmann
2fc586d2a4 Merge pull request #1381 from BrianBland/s3CustomUAString
Adds custom registry User-Agent header to s3 HTTP requests
2016-02-01 15:40:36 -08:00
Brian Bland
2dc1af12a1 Adds custom registry User-Agent header to s3 HTTP requests
Uses docker/goamz instead of AdRoll/goamz

Adds a registry UA string param to the storage parameters when
constructing the storage driver for the registry App.
This could be used by other storage drivers as well

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 13:43:46 -08:00
Derek McGowan
648a1343db Update auth context keys to use constant
Prevent using strings throughout the code to reference a string key defined in the auth package.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-28 17:02:09 -08:00
Aaron Lehmann
2b20b0167a Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann
4441333912 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Aaron Lehmann
8c1a000799 Fix content type for schema1 signed manifests
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.

Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 16:08:02 -08:00
Aaron Lehmann
5a2664e0b1 Do not require "charset=utf-8" for a schema1 with content type application/json
For compatibility with other registries that don't use this exact
variant of the Content-Type header, we need to be more flexible about
what we accept. Any form of "application/json" should be allowed. The
charset should not be included in the comparison.

See docker/docker#19400.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 09:59:50 -08:00
Brian Bland
3ef917c269 Fixes cross-repo blob mounting in the BlobUploadHandler
Accidentally checked for err != nil instead of err == nil :/
Also now ensures that only a non-nil option is appended to the create
options slice

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 19:21:45 -08:00
Brian Bland
ce88d8a6f4 Adds functional options arguments to the Blobs Create method
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 16:42:59 -08:00
Richard Scothern
3ea0d70896 Fix manifest API unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-11 12:52:21 -08:00
Richard Scothern
3a1220de01 Merge pull request #1319 from RichardScothern/update-tags
Remove tags referencing deleted manifests.
2016-01-11 11:33:22 -08:00
Brian Bland
5df21570a7 Adds cross-repository blob mounting behavior
Extends blob upload POST endpoint to support mount and from query
parameters as described in #634

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:18 -08:00
Aaron Lehmann
6d17423a6d Move MediaType into manifest.Versioned
This makes content type sniffing cleaner. The document just needs to be
decoded into a manifest.Versioned structure. It's no longer a two-step
process.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
697af09566 Recognize clients that don't support manifest lists
Convert a default platform's manifest to schema1 on the fly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
9c416f0e94 Add support for manifest list ("fat manifest")
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
9284810356 Add API unit testing for schema2 manifest
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Aaron Lehmann
3f746a8207 Recognize clients that don't support schema2, and convert manifests to schema1 on the fly
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Richard Scothern
94347c8611 Remove tags referencing deleted manifests.
When a manifest is deleted by digest, look up the referenced tags in the tag
store and remove all associations.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-05 13:47:52 -08:00
Richard Scothern
cb6f002350 Implementation of the Manifest Service API refactor.
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.

Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.

Remove Signatures() from Repository interface.

Signatures are relevant only to schema1 manifests.  Move access to the signature
store inside the schema1 manifestStore.  Add some API tests to verify
signature roundtripping.

schema1
-------

Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.

tags
----

Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces.  Implement a driver based tagService to be backward
compatible with the current tag service.

Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.

manifest store
--------------

Remove revision store and move all signing functionality into the signed manifeststore.

manifest registration
---------------------

Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.

client
------

Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs.  Change functional arguments to be an interface type
to allow passing data without mutating shared state.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2015-12-17 17:09:14 -08:00
Aaron Lehmann
4c850e7165 Remove tarsum support for digest package
tarsum is not actually used by the registry. Remove support for it.

Convert numerous uses in unit tests to SHA256.

Update docs to remove mentions of tarsums (which were often inaccurate).

Remove tarsum dependency.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-15 17:22:18 -08:00
Aaron Lehmann
31047c8113 Simplify digest.FromBytes calling convention
The current implementation of digest.FromBytes returns an error. This
error can never be non-nil, but its presence in the function signature
means each call site needs error handling code for an error that is
always nil.

I verified that none of the hash.Hash implementations in the standard
library can return an error on Write. Nor can any of the hash.Hash
implementations vendored in distribution.

This commit changes digest.FromBytes not to return an error. If Write
returns an error, it will panic, but as discussed above, this should
never happen.

This commit also avoids using a bytes.Reader to feed data into the hash
function in FromBytes. This makes the hypothetical case that would panic
a bit more explicit, and should also be more performant.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-14 14:30:51 -08:00
Troels Thomsen
300ce35c12 Map error type to error code
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:26:37 +01:00
Richard Scothern
dd32fbe615 Before allowing a schema1 manifest to be stored in the registry, ensure that it
contains equal length History and FSLayer arrays.

This is required to prevent malformed manifests being put to the registry and
failing external verification checks.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 15:19:46 -08:00
Stephen Day
dfe60f4cb1 Merge pull request #827 from aaronlehmann/read-only-mode-2
Add a read-only mode as a configuration option
2015-10-15 11:50:31 -07:00
Stephen J Day
76624704c3 Correct unmarshal order for SignedManifest
To ensure that we only unmarshal the verified payload into the contained
manifest, we first copy the entire incoming buffer into Raw and then unmarshal
only the Payload portion of the incoming bytes. If the contents is later
verified, the caller can then be sure that the contents of the Manifest fields
can be trusted.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-14 14:37:34 -07:00
Aaron Lehmann
0a6988195e Add http.host parameter
This allows the administrator to specify an externally-reachable URL for
the registry. It takes precedence over the X-Forwarded-Proto and
X-Forwarded-Host headers, and the hostname in the request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-30 15:12:01 -07:00
Aaron Lehmann
a601f92336 Add an "enabled" parameter under "readonly", and make it as if the mutable handlers don't exist when read-only mode is enabled
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:49:26 -07:00
Aaron Lehmann
c9bb330b71 Add a read-only mode as a configuration option
Add "readonly" under the storage/maintenance section. When this is set
to true, uploads and deletions will return 503 Service Unavailable
errors.

Document the parameter and add some unit testing.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:47:48 -07:00
Stephen J Day
530afa5234 Add WithVersion to context and other cleanup
By adding WithVersion to the context package, we can simplify context setup in
the application. This avoids some odd bugs where instantiation order can lead
to missing instance.id or version from log messages.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-09-14 17:14:31 -07:00
Aaron Lehmann
9b69e40c93 Move initialization code from main.go to the registry package
This makes it easier to embed a registry instance inside another
application.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-09 14:39:31 -07:00
Stephen J Day
6712e602b0 Move manifest package to schema1
As we begin our march towards multi-arch, we must prepare for the reality of
multiple manifest schemas. This is the beginning of a set of changes to
facilitate this. We are both moving this package into its target position where
it may live peacefully next to other manfiest versions.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-21 16:29:47 -07:00
Aaron Lehmann
b67aab2f60 Add headers parameter for HTTP checker
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:16 -07:00
Aaron Lehmann
e8f088fea6 Add a TCP health checker
Also, add timeout and status code parameters to the HTTP checker, and
remove the threshold parameter for the file checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:15 -07:00
Aaron Lehmann
b9b9cafa8f Expose a Registry type in health package, so unit tests can stay isolated from each other
Update docs.

Change health_test.go tests to create their own registries and register
the checks there. The tests now call CheckStatus directly instead of
polling the HTTP handler, which returns results from the default
registry.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:07:54 -07:00
Aaron Lehmann
79959f578a Switch tests to import "github.com/docker/distribution/context"
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
216df32510 Add storagedriver section to health check configuration
Add default storagedriver health check to example configuration files
with parameters matching the previous hardcoded configuration.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
b09b0ffcf9 Add configurable file-existence and HTTP health checks
Add a section to the config file called "health". Within this section,
"filecheckers" and "httpcheckers" list checks to run. Each check
specifies a file or URI, a time interval for the check, and a threshold
specifying how many times the check must fail to reach an unhealthy
state.

Document the new options in docs/configuration.md.

Add unit testing for both types of checkers. Add an UnregisterAll
function in the health package to support the unit tests, and an
Unregister function for consistency with Register.

Fix a string conversion problem in the health package's HTTP checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00