Commit graph

1487 commits

Author SHA1 Message Date
Richard Scothern
057284b593 Merge pull request #1156 from RichardScothern/manifest-verification
Manifest Verification
2015-11-04 14:09:12 -08:00
Jess Frazelle
0d91f2899e Merge pull request #17614 from LK4D4/fix_golint
Update linting tools to latest versions
2015-11-03 15:32:12 -08:00
Richard Scothern
78b6d648fa Before allowing a schema1 manifest to be stored in the registry, ensure that it
contains equal length History and FSLayer arrays.

This is required to prevent malformed manifests being put to the registry and
failing external verification checks.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 15:19:46 -08:00
Richard Scothern
573be447c6 Merge pull request #1150 from RichardScothern/carry-1070
Add Storage Driver Context.
2015-11-03 15:15:48 -08:00
Richard Scothern
e79324edd8 Add a generic error type to capture non-typed errors
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 11:19:44 -08:00
amitshukla
7840a5bc8f Fix for issue 664: https://github.com/docker/distribution/issues/664
Errors thrown by storage drivers don't have the name of the driver, causing user
confusion about whether the error is coming from Docker or from a storage driver.
This change adds the storage driver name to each error message.

This required changing ErrUnsupportedDriver to a type, leading to code changes
whenever ErrUnsupportedDriver is used.  The tests check whether the driver name
appears in the error message.

Signed-off-by: Amit Shukla <amit.shukla@docker.com>
2015-11-03 11:19:17 -08:00
Richard Scothern
bd958d8b88 Merge pull request #1114 from lebauce/swift-temp-url
Redirect support in Swift driver
2015-11-03 09:34:01 -08:00
Sylvain Baubeau
34c1d0ed50 Ensure read after write for segments
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-11-03 17:09:52 +01:00
Sylvain Baubeau
11546b5309 Add support for temporary URL for Swift driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-11-03 09:59:50 +01:00
Aaron Lehmann
afd61ce8f2 Vendor updated version of docker/distribution
This updates the vendored docker/distribution to the current master
branch.

Note the following changes:

- The manifest package was split into manifest/schema1. Most references
  to the manifest package in the engine needed to be updated to use
  schema1 instead.

- Validation functions in api/v2 were replaced by the
  distribution/reference package. The engine code has been updated to
  use the reference package for validation where necessary. A future PR
  will change the engine to use the types defined in
  distribution/reference more comprehensively.

- The reference package explicitly allows double _ characters in
  repository names. registry_test.go was updated for this.

- TestPullFailsWithAlteredManifest was corrupting the manifest JSON, now
  that the schema1 package unmarshals the correct payload. The test is
  being changed to modify the JSON without affecting its length, which
  allows the pull to succeed to the point where digest validation
  happens.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-02 12:40:18 -08:00
Stephen J Day
854fa0a4dd registry/storage: close filereader after allocation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-11-02 11:55:11 -08:00
Alexander Morozov
94913b8f7f Fix go vet warnings
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-11-02 08:28:34 -08:00
Richard Scothern
00f02b5fbc Buffer writing the scheduler entry state to disk by periodically checking for
changes to the entries index and saving it to the filesystem.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-29 11:35:52 -07:00
Richard Scothern
a31e93e1c4 Merge pull request #756 from aibaars/storage-gcs
Storage driver for Google Cloud Storage
2015-10-29 10:41:17 -07:00
Richard Scothern
cbf1f0797c Merge pull request #1096 from aaronlehmann/simplify-proxy-scheduler
Simplify proxy scheduler
2015-10-28 13:23:00 -07:00
Arthur Baars
98ad17f757 Storage driver for: Google Cloud Storage (gcs)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2015-10-28 20:21:56 +00:00
Brian Goff
24de26a805 Revert "dockerversion placeholder for library-import"
This reverts commit c2d5c29437.

Commit caused issues on systems with case-insensitive filesystems.
Revert for now

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-10-27 21:23:53 -04:00
Vincent Demeester
c2d5c29437 dockerversion placeholder for library-import
- Move autogen/dockerversion to version
- Update autogen and "builds" to use this package and a build flag

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-10-27 20:36:07 +01:00
Matt Moore
b38b98c8a8 Add expires_in and issued_at to the auth spec.
This extends the specification for the Bearer token response to include
information pertaining to when an issued Bearer token will expire.

This also allows the client to accept `access_token` as an alias for `token`.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-27 11:05:33 -07:00
Rusty Conover
9293e3db11 Fix failing test case
When building a URL don't include path components in the host parameter.

Closes #1124

Signed-off-by: Rusty Conover <rusty@luckydinosaur.com>
2015-10-25 13:03:22 -04:00
Stephen Day
8cc790cccf Merge pull request #1098 from denverdino/master
Redundant digest verification in validateBlob when pushing a new layer
2015-10-21 14:50:18 -07:00
Stephen Day
cc69dcca8d Merge pull request #1115 from tianon/fix-auth-servehttp-comment
Update "type auth.Challenge" comment example code
2015-10-21 12:56:14 -07:00
Tianon Gravi
8263cdeb57 Update "type auth.Challenge" comment example code
This interface was changed in 4a2300aaa9, but the comment wasn't ever updated to match.

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-10-21 11:54:13 -07:00
Richard Scothern
8806306f3d Merge pull request #1111 from aaronlehmann/golint-comments
Correct two golint comment issues
2015-10-20 09:53:14 -07:00
Aaron Lehmann
fc5ee720d1 Correct two golint comment issues
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-19 16:42:12 -07:00
Richard Scothern
0ce983b4e8 Merge pull request #1006 from RichardScothern/proxy-serialize
Fix a race condition in pull through cache population
2015-10-16 17:13:26 -07:00
Stephen Day
eb484b7ddf Merge pull request #827 from aaronlehmann/read-only-mode-2
Add a read-only mode as a configuration option
2015-10-15 11:50:31 -07:00
Aaron Lehmann
84595fc628 Simplify proxy scheduler
The proxy scheduler implemented its own timer state machine. It's
simpler and more efficient to leverage the Go runtime's timer heap by
using time.AfterFunc.

This commit adds a time.Timer to each scheduler entry, and starts and
stops those timers as necessary. Then the mainloop goroutine and its
associated logic are not needed.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-15 09:18:21 -07:00
Li Yi
c40b2e2341 Redundant digest verification in validateBlob
Change-Id: I03764edadae529db2cc3acf7ecca329570f18659
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-10-15 22:21:14 +08:00
Stephen J Day
26762a54fe Correct unmarshal order for SignedManifest
To ensure that we only unmarshal the verified payload into the contained
manifest, we first copy the entire incoming buffer into Raw and then unmarshal
only the Payload portion of the incoming bytes. If the contents is later
verified, the caller can then be sure that the contents of the Manifest fields
can be trusted.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-14 14:37:34 -07:00
Richard Scothern
da206d64cb Merge pull request #1035 from aaronlehmann/host-parameter-alternate
Add http.host parameter
2015-10-13 14:03:48 -07:00
Stephen Day
80783c058f Merge pull request #1033 from jlhawn/api_spec_update_auth_errors
[api spec] Update authN and authZ errors
2015-10-13 13:52:18 -07:00
Derek McGowan
582a0661bf Update to provide small and clear interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-10-09 16:05:49 -07:00
Tibor Vass
b72f1fd2e3 Add a new reference package abstracting repositories, tags and digests
There seems to be a need for a type that represents a way of pointing
to an image, irrespective of the implementation.

This patch defines a Reference interface and provides 3 implementations:
- TagReference: when only a tag is provided
- DigestReference: when a digest (according to the digest package) is
  provided, can include optional tag as well

Validation of references are purely syntactic.

There is also a strong type for tags, analogous to digests, as well
as a strong type for Repository from which clients can access the
hostname alone, or the repository name without the hostname, or both
together via the String() method.

For Repository, the files names.go and names_test.go were moved from
the v2 package.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-09 16:05:34 -07:00
Derek McGowan
ed69ef01ee Update distribution package
Pick up name regexp change in distribution to allow matching of hostnames as a valid component of a repository.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-10-08 17:16:43 -07:00
Richard Scothern
36fa22c821 Fix a race condition in pull through cache population by removing the functionality
of readers joining current downloads.  Concurrent requests for the same blob
will not block, but only the first instance will be comitted locally.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-08 16:12:29 -07:00
Matt Moore
d5ca577ad1 Allow hostname components in component names.
Fixes https://github.com/docker/distribution/issues/1062

This relaxes the naming restrictions places on Docker images to permit
valid hostnames according to [RFC-2396](https://www.ietf.org/rfc/rfc2396.txt).

It deviates from the RFC in the following ways:
 1) Allow underscores where we allow hyphens (hostnames don't allow
 underscores, which we must for backwards compatibility).
 2) Leave "top-level" name segments unrestricted (domains require an
 alpha character to begin a top-level domain, e.g. "com").
 3) DO NOT allow a trailing dot, as permitted by FQDNs.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-08 11:29:54 -07:00
Jessica Frazelle
0b543b4767 change flag name to better follow the other flags that start with disable;
Signed-off-by: Jessica Frazelle <acidburn@docker.com>
2015-10-07 10:16:37 -07:00
Vincent Demeester
60edb40c29 Merge pull request #16509 from HuKeping/search-problem
Fix docker search problem
2015-10-04 18:21:32 +02:00
Hu Keping
82965f6c84 Fix docker search problem
Search terms shouldn't be restricted to only full valid repository
names. It should be perfectly valid to search using a part of a name,
even if it ends with a period, dash or underscore.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-10-04 23:23:43 +08:00
Richard Scothern
1039e2dc26 Merge pull request #950 from jackpgriffin/master
Skip creating swift container if already exists
2015-10-01 10:15:54 -07:00
Aaron Lehmann
6573d5c119 Add http.host parameter
This allows the administrator to specify an externally-reachable URL for
the registry. It takes precedence over the X-Forwarded-Proto and
X-Forwarded-Host headers, and the hostname in the request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-30 15:12:01 -07:00
Josh Hawn
fa4c33f5f3 [api spec] Update authN and authZ errors
Associate HTTP 401s with Authentication errors rather than Authorization
errors. Changes the meaning of the UNAUTHORIZED error to be authentication
specific.

Defines DENIED error code to be associated with authorization
errors which result in HTTP 403 responses.

Add 'No Such Repository' errors to more endpoints.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-09-30 09:12:31 -07:00
Stephen Day
3532bc3f2c Merge pull request #1024 from aaronlehmann/avoid-testing-import
Avoid importing "testing" in externally-facing code
2015-09-30 13:14:42 +02:00
Aaron Lehmann
b045aa2a3d Avoid importing "testing" in externally-facing code
The "testing" package adds some flags in its init function, so utilities
that import distribution code may print a page of extra testing flags in
their help output.

This commit solves the issue by moving an import of "testing" in the
registry/storage/cache package to a new
registry/storage/cache/cachecheck package, which is only imported by
tests.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-29 10:06:39 -07:00
Aaron Lehmann
8a05f9bb4f Merge pull request #1001 from RichardScothern/fix-atrocity
Don't return nil, nil from functions
2015-09-28 13:34:11 -07:00
Richard Scothern
ebaa771c3b Prevent push and pull to v1 registries by filtering the available endpoints.
Add a daemon flag to control this behaviour.  Add a warning message when pulling
an image from a v1 registry.  The default order of pull is slightly altered
with this changset.

Previously it was:
https v2, https v1, http v2, http v1

now it is:
https v2, http v2, https v1, http v1

Prevent login to v1 registries by explicitly setting the version before ping to
prevent fallback to v1.

Add unit tests for v2 only mode.  Create a mock server that can register
handlers for various endpoints.  Assert no v1 endpoints are hit with legacy
registries disabled for the following commands:  pull, push, build, run and
login.  Assert the opposite when legacy registries are not disabled.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-25 15:59:42 -07:00
Aaron Lehmann
cbf83ecd31 Add an "enabled" parameter under "readonly", and make it as if the mutable handlers don't exist when read-only mode is enabled
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:49:26 -07:00
Aaron Lehmann
df9758ba39 Add a read-only mode as a configuration option
Add "readonly" under the storage/maintenance section. When this is set
to true, uploads and deletions will return 503 Service Unavailable
errors.

Document the parameter and add some unit testing.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:47:48 -07:00
Nuutti Kotivuori
84e7c07c42 Remove initial access check from S3 driver
In the S3 storage driver there is currently an initial access permission check by listing the bucket. If this check fails, registry will panic and exit.

However, this check is broken in two ways. First of all it strips the final slash from the root directory path, meaning that any access permissions which limit access to a single directory will fail, because S3 treats the path as strict prefix match. Secondly it fails to strip any leading slash that might be present, unlike the other access places, which means that the path used is different as a leading slash is allowed and significant in a filename in S3.

Since there is also a periodic health check which correctly checks access permissions and shows the error more cleanly, the best solution seems to be to just remove this initial access check.

Signed-off-by: Nuutti Kotivuori <nuutti.kotivuori@poplatek.fi>
2015-09-19 12:47:12 +03:00