Commit graph

716 commits

Author SHA1 Message Date
Brian Bland
f41a408e34 Adds custom registry User-Agent header to s3 HTTP requests
Uses docker/goamz instead of AdRoll/goamz

Adds a registry UA string param to the storage parameters when
constructing the storage driver for the registry App.
This could be used by other storage drivers as well

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 13:43:46 -08:00
Derek McGowan
badd8c49b6 Update auth context keys to use constant
Prevent using strings throughout the code to reference a string key defined in the auth package.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-28 17:02:09 -08:00
Aaron Lehmann
6c135e03ce Merge pull request #1393 from RichardScothern/putbydgst
Add manifest put by digest to the registry client
2016-01-28 14:41:07 -08:00
Derek McGowan
0e96574ea2 Merge pull request #1392 from aaronlehmann/client-range-requests
Support range requests in the client's httpReadSeeker
2016-01-28 13:44:43 -08:00
Richard Scothern
ccd11e4434 Merge pull request #1394 from RichardScothern/invalidate-bdc
Invalidate the blob store descriptor cache
2016-01-28 10:04:09 -08:00
Aaron Lehmann
8e571dff41 Add a CheckRedirect function to the HTTP client
Use it to preserve Accept and Range headers that were added to the
original request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Aaron Lehmann
a58b7625ba Support range requests in the client's httpReadSeeker
Remove buffering on the reader, because it's not useful. Also remove
artificial io.EOF return.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Richard Scothern
3e570e59f1 Invalidate the blob store descriptor caches when content is removed from from
the proxy.  Also, switch to reference in the scheduler API.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:59 -08:00
Richard Scothern
a7740f5d0f Correct test digest lengths and enable all unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:46 -08:00
Richard Scothern
f757372dd8 Add manifest put by digest to the registry client
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 14:20:23 -08:00
Derek McGowan
1eed0ddd07 Update token header struct to use json.RawMessage pointer
Since RawMessage json receivers take a pointer type, the Header structure should use points in order to call the json.RawMessage marshal and unmarshal functions

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-25 20:11:41 -08:00
Arthur Baars
7dee3d19d9 Storage: remove bufferedFileWriter (dead code)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-23 10:30:08 +00:00
Arthur Baars
586b3d47a7 Storage: blobwriter.Write/Seek test case
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-23 10:30:07 +00:00
Richard Scothern
609bc63dd4 Merge pull request #1333 from aaronlehmann/use-reference-package
Use reference package
2016-01-22 15:54:19 -08:00
Aaron Lehmann
6149a8c634 Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann
e9692b8037 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Aaron Lehmann
e9bcc96ad2 If the media type for a manifest is unrecognized, default to schema1
This is needed for compatibility with some third-party registries that
send an inappropriate Content-Type header such as text/html.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-21 09:34:06 -08:00
Stephen Day
0d46f0e786 Merge pull request #1379 from aaronlehmann/token-error
Handle nonstandard token endpoint errors
2016-01-20 15:07:47 -08:00
Aaron Lehmann
59254013be Handle nonstandard token endpoint errors
https://github.com/docker/distribution/pull/1249 changed token fetching
to parse HTTP error response bodies as serialized errcodes. However,
Docker Hub's authentication endpoint does not return error bodies in
this format. To work around this, convert its format into
ErrCodeUnauthorized or ErrCodeUnknown.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-20 14:45:08 -08:00
Richard Scothern
05b8317291 Merge pull request #1372 from aibaars/gcs-delete
StorageDriver GCS: improve test suite clean-up and add retrying to all GCS api calls
2016-01-20 12:21:22 -08:00
Arthur Baars
59a9607783 StorageDriver: GCS: retry all api calls
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Arthur Baars
ffc9527782 StorageDriver: Test suite: improve cleanup
Verify that the file(s) have been deleted after calling Delete,
and retry if this is not the case. Furthermore, report the error
if a Delete operation fails.

Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Richard Scothern
e4b654f2ce Merge pull request #1367 from aaronlehmann/signed-manifest-content-type
Fix content type for schema1 signed manifests
2016-01-19 11:00:47 -08:00
Aaron Lehmann
f9a3f028b5 Fix content type for schema1 signed manifests
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.

Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 16:08:02 -08:00
Richard Scothern
0640813179 Merge pull request #1363 from aaronlehmann/media-type-charset
Do not require "charset=utf-8" for a schema1 with content type application/json
2016-01-18 15:20:01 -08:00
Richard Scothern
b2876674f3 Merge pull request #1355 from hopkings2008/master
In testsuites.go, enlarge the size of randomBytes to 128M to fix the …
2016-01-18 12:38:26 -08:00
Richard Scothern
f4bd6cacd1 Merge pull request #1357 from aibaars/gcs-test
StorageDriver GCS test suite: try google.DefaultTokenSource first
2016-01-18 11:29:01 -08:00
Aaron Lehmann
3da0ee00d8 Do not require "charset=utf-8" for a schema1 with content type application/json
For compatibility with other registries that don't use this exact
variant of the Content-Type header, we need to be more flexible about
what we accept. Any form of "application/json" should be allowed. The
charset should not be included in the comparison.

See docker/docker#19400.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 09:59:50 -08:00
Arthur Baars
985c0d602f StorageDriver GCS: try google.DefaultTokenSource first
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-15 11:47:47 +00:00
yuzou
d3d9282a30 In testsuites.go, enlarge the size of randomBytes to 128M to fix the crash of running TestConcurrentStreamReads
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-01-15 17:22:43 +08:00
Richard Scothern
05dc6404fd Merge pull request #1332 from RichardScothern/gcs-params
Change the parameters to the GCS drivers to allow CircleCI testing.
2016-01-14 15:50:09 -08:00
Brian Bland
67aef89bc0 Splits up blob create options definitions to be package-specific
Redefines privately in both storage and client packages

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-14 10:47:33 -08:00
Brian Bland
e90578b27d Merge pull request #1348 from BrianBland/fixCrossRepoBlobMount
Fixes cross-repo blob mounting in the BlobUploadHandler
2016-01-14 09:44:30 -08:00
Brian Bland
e0d4a45c93 Fixes cross-repo blob mounting in the BlobUploadHandler
Accidentally checked for err != nil instead of err == nil :/
Also now ensures that only a non-nil option is appended to the create
options slice

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 19:21:45 -08:00
Richard Scothern
5d35fa34c1 Change the parameters to the GCS drivers to allow CircleCI testing.
Remove the requirement of file system access to run GCS unit tests.  Deconstruct
the input parameters to take the private key and email which can be specified on
the build system via environment variables.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-13 18:00:59 -08:00
Richard Scothern
b38b9ba385 Merge pull request #1344 from BrianBland/blobCreateWithOptions
Adds functional options arguments to the Blobs Create method, remove Mount operation
2016-01-13 17:35:49 -08:00
Brian Bland
36023174db Adds functional options arguments to the Blobs Create method
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 16:42:59 -08:00
Richard Scothern
66b492e69f Merge pull request #1335 from RichardScothern/fix-build
Fix manifest API unit tests
2016-01-11 14:17:36 -08:00
Richard Scothern
93b65847ca Fix manifest API unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-11 12:52:21 -08:00
Richard Scothern
b7aee20b81 Merge pull request #1319 from RichardScothern/update-tags
Remove tags referencing deleted manifests.
2016-01-11 11:33:22 -08:00
Richard Scothern
5fe3d68e9c Merge pull request #1269 from BrianBland/crossRepositoryPush
Adds cross-repository blob mounting behavior
2016-01-08 14:37:00 -08:00
Brian Bland
44d95e5841 Allows token authentication handler to request additional scopes
When an auth request provides the "from" query parameter, the token
handler will add a "pull" scope for the provided repository, refreshing
the token if the overall scope has increased

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:23 -08:00
Brian Bland
41e30f626b Adds cross-repository blob mounting behavior
Extends blob upload POST endpoint to support mount and from query
parameters as described in #634

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:18 -08:00
Richard Scothern
e5e0e2cc2a Merge pull request #1276 from denverdino/oss-test
Support large layer for OSS driver
2016-01-08 13:26:09 -08:00
Stephen Day
c426367881 Merge pull request #1281 from aaronlehmann/new-manifest
Implement schema2 manifest formats
2016-01-07 17:19:56 -08:00
Stephen Day
98fecde1c9 Merge pull request #911 from stevvooe/consistent-error-string
More consistent return from ErrorCode.Error()
2016-01-07 17:06:12 -08:00
Aaron Lehmann
bbabb55ccb Move MediaType into manifest.Versioned
This makes content type sniffing cleaner. The document just needs to be
decoded into a manifest.Versioned structure. It's no longer a two-step
process.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
fce65b72b3 Recognize clients that don't support manifest lists
Convert a default platform's manifest to schema1 on the fly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
7ef71988a8 Add support for manifest list ("fat manifest")
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
66a33baa36 Add API unit testing for schema2 manifest
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00