Commit graph

722 commits

Author SHA1 Message Date
Derek McGowan
582a0661bf Update to provide small and clear interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-10-09 16:05:49 -07:00
Tibor Vass
b72f1fd2e3 Add a new reference package abstracting repositories, tags and digests
There seems to be a need for a type that represents a way of pointing
to an image, irrespective of the implementation.

This patch defines a Reference interface and provides 3 implementations:
- TagReference: when only a tag is provided
- DigestReference: when a digest (according to the digest package) is
  provided, can include optional tag as well

Validation of references are purely syntactic.

There is also a strong type for tags, analogous to digests, as well
as a strong type for Repository from which clients can access the
hostname alone, or the repository name without the hostname, or both
together via the String() method.

For Repository, the files names.go and names_test.go were moved from
the v2 package.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-09 16:05:34 -07:00
Richard Scothern
36fa22c821 Fix a race condition in pull through cache population by removing the functionality
of readers joining current downloads.  Concurrent requests for the same blob
will not block, but only the first instance will be comitted locally.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-08 16:12:29 -07:00
Matt Moore
d5ca577ad1 Allow hostname components in component names.
Fixes https://github.com/docker/distribution/issues/1062

This relaxes the naming restrictions places on Docker images to permit
valid hostnames according to [RFC-2396](https://www.ietf.org/rfc/rfc2396.txt).

It deviates from the RFC in the following ways:
 1) Allow underscores where we allow hyphens (hostnames don't allow
 underscores, which we must for backwards compatibility).
 2) Leave "top-level" name segments unrestricted (domains require an
 alpha character to begin a top-level domain, e.g. "com").
 3) DO NOT allow a trailing dot, as permitted by FQDNs.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-08 11:29:54 -07:00
Richard Scothern
1039e2dc26 Merge pull request #950 from jackpgriffin/master
Skip creating swift container if already exists
2015-10-01 10:15:54 -07:00
Aaron Lehmann
6573d5c119 Add http.host parameter
This allows the administrator to specify an externally-reachable URL for
the registry. It takes precedence over the X-Forwarded-Proto and
X-Forwarded-Host headers, and the hostname in the request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-30 15:12:01 -07:00
Josh Hawn
fa4c33f5f3 [api spec] Update authN and authZ errors
Associate HTTP 401s with Authentication errors rather than Authorization
errors. Changes the meaning of the UNAUTHORIZED error to be authentication
specific.

Defines DENIED error code to be associated with authorization
errors which result in HTTP 403 responses.

Add 'No Such Repository' errors to more endpoints.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-09-30 09:12:31 -07:00
Stephen Day
3532bc3f2c Merge pull request #1024 from aaronlehmann/avoid-testing-import
Avoid importing "testing" in externally-facing code
2015-09-30 13:14:42 +02:00
Aaron Lehmann
b045aa2a3d Avoid importing "testing" in externally-facing code
The "testing" package adds some flags in its init function, so utilities
that import distribution code may print a page of extra testing flags in
their help output.

This commit solves the issue by moving an import of "testing" in the
registry/storage/cache package to a new
registry/storage/cache/cachecheck package, which is only imported by
tests.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-29 10:06:39 -07:00
Aaron Lehmann
8a05f9bb4f Merge pull request #1001 from RichardScothern/fix-atrocity
Don't return nil, nil from functions
2015-09-28 13:34:11 -07:00
Aaron Lehmann
cbf83ecd31 Add an "enabled" parameter under "readonly", and make it as if the mutable handlers don't exist when read-only mode is enabled
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:49:26 -07:00
Aaron Lehmann
df9758ba39 Add a read-only mode as a configuration option
Add "readonly" under the storage/maintenance section. When this is set
to true, uploads and deletions will return 503 Service Unavailable
errors.

Document the parameter and add some unit testing.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:47:48 -07:00
Nuutti Kotivuori
84e7c07c42 Remove initial access check from S3 driver
In the S3 storage driver there is currently an initial access permission check by listing the bucket. If this check fails, registry will panic and exit.

However, this check is broken in two ways. First of all it strips the final slash from the root directory path, meaning that any access permissions which limit access to a single directory will fail, because S3 treats the path as strict prefix match. Secondly it fails to strip any leading slash that might be present, unlike the other access places, which means that the path used is different as a leading slash is allowed and significant in a filename in S3.

Since there is also a periodic health check which correctly checks access permissions and shows the error more cleanly, the best solution seems to be to just remove this initial access check.

Signed-off-by: Nuutti Kotivuori <nuutti.kotivuori@poplatek.fi>
2015-09-19 12:47:12 +03:00
Richard Scothern
9fb5fe4fbb Don't return a nil array and a nil error if the Tags endpoint cannot be found
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-18 16:19:41 -07:00
Richard Scothern
b8a1ec4155 Avoid returning nil, nil when fetching a manifest by tag by introducing a new
error ErrManifestNotModified which can be checked by clients.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-18 16:19:37 -07:00
Stephen Day
5a8fabfee3 Merge pull request #974 from stevvooe/context-cleanup
context: WithVersion and context package cleanup
2015-09-14 19:24:33 -07:00
Stephen J Day
49f080acc8 Add WithVersion to context and other cleanup
By adding WithVersion to the context package, we can simplify context setup in
the application. This avoids some odd bugs where instantiation order can lead
to missing instance.id or version from log messages.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-09-14 17:14:31 -07:00
Stephen J Day
cabf1fd236 Allow interface{} keys when using logger
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-09-11 11:33:55 -07:00
Stephen Day
bffea2d94f Merge pull request #977 from aaronlehmann/use-distribution-context
Only use the distribution/context package in registry.go
2015-09-11 11:33:19 -07:00
Aaron Lehmann
6403bf64d5 Only use the distribution/context package in registry.go
This solves a issue from #909 where instance.id was not printed in logs,
because this file was using the background context from
golang.org/x/net/context instead of
github.com/docker/distribution/context.

It's cleaner to standardize on one package, so this commit removes the
import of golang.org/x/net/context entirely. The Context interfaces
defined in both packages are the same, so other code using
golang.org/x/net/context can still pass its context to NewRegistry.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-11 09:54:15 -07:00
Aaron Lehmann
045db61784 Add a cobra command that implements the entire main function for registry
Use this command in cmd/registry/main.go.

Move debug server to the main command, and change Serve to be a
ListenAndServe function.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-09 14:39:31 -07:00
Aaron Lehmann
8dd51d6460 Move initialization code from main.go to the registry package
This makes it easier to embed a registry instance inside another
application.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-09 14:39:31 -07:00
Jack Griffin
8ceca304b0 Skip creating swift container if already exists
Signed-off-by: Jack Griffin <jackpg14@gmail.com>
2015-09-04 18:53:17 +00:00
Hua Wang
8f5f6a4e59 Add TrustId parameter to swift driver
github/ncw/swift has added support for trust, so let's add it.

Signed-off-by: Hua Wang <wanghua.humble@gmail.com>
2015-09-02 15:05:29 +08:00
Stephen Day
0306c5c453 Merge pull request #934 from stevvooe/sanitize-url-correctly
Correctly sanitize location url preserving parameters
2015-08-27 22:30:21 -07:00
Stephen J Day
6e7718dfce Correctly sanitize location url preserving parameters
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-27 17:06:03 -07:00
Aaron Lehmann
8ae9f9da01 Merge pull request #912 from stevvooe/manifest-refactoring-schema1
Move manifest package to schema1
2015-08-21 16:47:09 -07:00
Stephen J Day
bb098c72a2 Move manifest package to schema1
As we begin our march towards multi-arch, we must prepare for the reality of
multiple manifest schemas. This is the beginning of a set of changes to
facilitate this. We are both moving this package into its target position where
it may live peacefully next to other manfiest versions.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-21 16:29:47 -07:00
Richard Scothern
f639a1af8c Merge pull request #901 from aaronlehmann/configurable-health-checks
Add configurable file-existence and HTTP health checks
2015-08-21 10:53:32 -07:00
Stephen J Day
7dd03e12bb More consistent return from ErrorCode.Error()
To bring ErrorCode into liine with Go conventions, ErrorCode.Error() now
returns the "nice" value of the error code. This ensures error message assembly
works similar to commonly used Go conventions when directly using ErrorCode as
an error.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-20 17:36:24 -07:00
Aaron Lehmann
5b804f7600 Add headers parameter for HTTP checker
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:16 -07:00
Aaron Lehmann
ca3d460278 Add a TCP health checker
Also, add timeout and status code parameters to the HTTP checker, and
remove the threshold parameter for the file checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:15 -07:00
Aaron Lehmann
cdc3143b7e Expose a Registry type in health package, so unit tests can stay isolated from each other
Update docs.

Change health_test.go tests to create their own registries and register
the checks there. The tests now call CheckStatus directly instead of
polling the HTTP handler, which returns results from the default
registry.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:07:54 -07:00
Aaron Lehmann
bbd4699166 Switch tests to import "github.com/docker/distribution/context"
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
68e8532cef Add storagedriver section to health check configuration
Add default storagedriver health check to example configuration files
with parameters matching the previous hardcoded configuration.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
c48e460933 Add configurable file-existence and HTTP health checks
Add a section to the config file called "health". Within this section,
"filecheckers" and "httpcheckers" list checks to run. Each check
specifies a file or URI, a time interval for the check, and a threshold
specifying how many times the check must fail to reach an unhealthy
state.

Document the new options in docs/configuration.md.

Add unit testing for both types of checkers. Add an UnregisterAll
function in the health package to support the unit tests, and an
Unregister function for consistency with Register.

Fix a string conversion problem in the health package's HTTP checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Stephen Day
0189e8c218 Merge pull request #893 from aaronlehmann/functional-opts
Functional options for NewRegistryWithDriver
2015-08-19 14:41:08 -07:00
Richard Scothern
fbd42e9e6f Merge pull request #896 from aaronlehmann/closenotifier-warnings
Fix CloseNotifier handling and avoid "the ResponseWriter does not implement CloseNotifier" warnings in logs
2015-08-19 12:41:03 -07:00
Aaron Lehmann
142b68aaa2 Add a unit test which verifies the ResponseWriter endpoints see implements CloseNotifier
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-19 11:37:53 -07:00
Aaron Lehmann
7fb68446cc Functional options for NewRegistryWithDriver
Clean up calling convention for NewRegistryWithDriver to use functional
arguments.

This is a first step towards the refactor described in #215. I plan to
add additional options in the process of moving configurable items from
the App structure to the registry structure.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-19 11:06:30 -07:00
Aaron Lehmann
11133181fc Fix CloseNotifier handling and avoid "the ResponseWriter does not implement CloseNotifier" warnings in logs
A change in #763 to address review comments caused problems. Originally,
instrumentedResponseWriter implemented the CloseNotifier interface, and
would panic if it was wrapping something that did not implement that
interface. This was split into a separate instrumentedResponseWriterCN
type that implements CloseNotifier, so there's a fallback if
instrumentedResponseWriter ever needs to wrap something that does not
implement this interface.

instrumentedResponseWriter's Value method would end up upcasting either
type back to instrumentedResponseWriter, which does not implement the
interface. In effect, instrumentedResponseWriterCN was never visible to
the handler.

This fixes the problem by implementing a wrapper Value method for
instrumentedResponseWriterCN.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-18 17:28:51 -07:00
Olivier Gambier
ae79542b91 Merge pull request #894 from aaronlehmann/fix-tests
Fix tests after #846
2015-08-18 13:57:13 -07:00
Aaron Lehmann
2e4c643419 Fix tests after #846
Change checkResponse to only expect the configured
X-Content-Type-Options header if it doesn't receive a 405 error, which
means the handler isn't registered for that method.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-18 13:33:26 -07:00
Richard Scothern
0c16d42a4a Merge pull request #846 from aaronlehmann/http-header-configuration
Add a section to the config file for HTTP headers to add to responses
2015-08-18 12:53:05 -07:00
Aaron Lehmann
027e7ff770 Merge pull request #889 from stevvooe/remove-pathmapper
Remove pathMapper object
2015-08-18 09:40:34 -07:00
Stephen J Day
614e8c8277 Remove pathMapper object
The use of the pathMapper is no longer needed the way we have organized the
code base. The extra level of indirection has proved unnecessary and confusing
so we've opted to clean it up. In the future, we may require more flexibility,
but now it is simply not required.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-17 18:51:05 -07:00
Richard Scothern
9174fc62d2 Merge pull request #864 from stevvooe/use-correct-manifest-link
registry/storage: use correct manifest link
2015-08-12 15:13:45 -07:00
Stephen J Day
5878a8f401 Maintain manifest link compatibility
Unfortunately, the 2.1 releease has written manfiest links into the wrong
directory. This doesn't affect new 2.1 deployments but fixing this to be 2.0
backwards compatible has broken 2.1.0 compatibility. To ensure we have
compatibility between 2.0, 2.1.0 and future releases, we now check one of
several locations to identify a manifest link.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-12 13:11:13 -07:00
Stephen J Day
5dd78c821a Use correct path for manifest revision path
Unfortunately, the refactor used the incorrect path for manifest links within a
repository. While this didn't stop the registry from working, it did break
compatibility with 2.0 deployments for manifest fetches.

Tests were added to ensure these are locked down to the appropriate paths.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-12 13:10:15 -07:00
Aaron Lehmann
d9a20377f3 Add a section to the config file for HTTP headers to add to responses
The example configuration files add X-Content-Type-Options: nosniff.

Add coverage in existing registry/handlers unit tests.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-11 16:41:38 -07:00