Commit graph

499 commits

Author SHA1 Message Date
Brian Bland
4688d1adc2 Adds "storageclass" configuration parameter for S3 driver.
Defaults to STANDARD, also supports REDUCED_REDUNDANCY.

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 16:01:19 -08:00
Aaron Lehmann
2fc586d2a4 Merge pull request #1381 from BrianBland/s3CustomUAString
Adds custom registry User-Agent header to s3 HTTP requests
2016-02-01 15:40:36 -08:00
Brian Bland
2dc1af12a1 Adds custom registry User-Agent header to s3 HTTP requests
Uses docker/goamz instead of AdRoll/goamz

Adds a registry UA string param to the storage parameters when
constructing the storage driver for the registry App.
This could be used by other storage drivers as well

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 13:43:46 -08:00
Aaron Lehmann
74c4a0e3e3 Merge pull request #1393 from RichardScothern/putbydgst
Add manifest put by digest to the registry client
2016-01-28 14:41:07 -08:00
Derek McGowan
70e85dde26 Merge pull request #1392 from aaronlehmann/client-range-requests
Support range requests in the client's httpReadSeeker
2016-01-28 13:44:43 -08:00
Aaron Lehmann
8fd84c4a57 Add a CheckRedirect function to the HTTP client
Use it to preserve Accept and Range headers that were added to the
original request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Aaron Lehmann
ccf8154a44 Support range requests in the client's httpReadSeeker
Remove buffering on the reader, because it's not useful. Also remove
artificial io.EOF return.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Richard Scothern
a8861549cf Invalidate the blob store descriptor caches when content is removed from from
the proxy.  Also, switch to reference in the scheduler API.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:59 -08:00
Richard Scothern
584c9b517c Correct test digest lengths and enable all unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:46 -08:00
Richard Scothern
9f72e8442d Add manifest put by digest to the registry client
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 14:20:23 -08:00
Aaron Lehmann
2b20b0167a Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann
4441333912 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Aaron Lehmann
c01fe47231 If the media type for a manifest is unrecognized, default to schema1
This is needed for compatibility with some third-party registries that
send an inappropriate Content-Type header such as text/html.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-21 09:34:06 -08:00
Aaron Lehmann
ec636bbfd2 Handle nonstandard token endpoint errors
https://github.com/docker/distribution/pull/1249 changed token fetching
to parse HTTP error response bodies as serialized errcodes. However,
Docker Hub's authentication endpoint does not return error bodies in
this format. To work around this, convert its format into
ErrCodeUnauthorized or ErrCodeUnknown.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-20 14:45:08 -08:00
Arthur Baars
2a4345ca4b StorageDriver: GCS: retry all api calls
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Arthur Baars
6ee339464c StorageDriver: Test suite: improve cleanup
Verify that the file(s) have been deleted after calling Delete,
and retry if this is not the case. Furthermore, report the error
if a Delete operation fails.

Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Aaron Lehmann
8c1a000799 Fix content type for schema1 signed manifests
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.

Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 16:08:02 -08:00
Richard Scothern
3cb403ae5b Merge pull request #1363 from aaronlehmann/media-type-charset
Do not require "charset=utf-8" for a schema1 with content type application/json
2016-01-18 15:20:01 -08:00
Richard Scothern
fc7ee6d789 Merge pull request #1355 from hopkings2008/master
In testsuites.go, enlarge the size of randomBytes to 128M to fix the …
2016-01-18 12:38:26 -08:00
Aaron Lehmann
5a2664e0b1 Do not require "charset=utf-8" for a schema1 with content type application/json
For compatibility with other registries that don't use this exact
variant of the Content-Type header, we need to be more flexible about
what we accept. Any form of "application/json" should be allowed. The
charset should not be included in the comparison.

See docker/docker#19400.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 09:59:50 -08:00
Arthur Baars
19d7c9e051 StorageDriver GCS: try google.DefaultTokenSource first
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-15 11:47:47 +00:00
yuzou
cc82b0d48e In testsuites.go, enlarge the size of randomBytes to 128M to fix the crash of running TestConcurrentStreamReads
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-01-15 17:22:43 +08:00
Richard Scothern
7378e21678 Merge pull request #1332 from RichardScothern/gcs-params
Change the parameters to the GCS drivers to allow CircleCI testing.
2016-01-14 15:50:09 -08:00
Brian Bland
8c5a6c13c0 Splits up blob create options definitions to be package-specific
Redefines privately in both storage and client packages

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-14 10:47:33 -08:00
Brian Bland
3ef917c269 Fixes cross-repo blob mounting in the BlobUploadHandler
Accidentally checked for err != nil instead of err == nil :/
Also now ensures that only a non-nil option is appended to the create
options slice

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 19:21:45 -08:00
Richard Scothern
81dbebc7d3 Change the parameters to the GCS drivers to allow CircleCI testing.
Remove the requirement of file system access to run GCS unit tests.  Deconstruct
the input parameters to take the private key and email which can be specified on
the build system via environment variables.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-13 18:00:59 -08:00
Brian Bland
ce88d8a6f4 Adds functional options arguments to the Blobs Create method
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 16:42:59 -08:00
Richard Scothern
3ea0d70896 Fix manifest API unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-11 12:52:21 -08:00
Richard Scothern
3a1220de01 Merge pull request #1319 from RichardScothern/update-tags
Remove tags referencing deleted manifests.
2016-01-11 11:33:22 -08:00
Richard Scothern
93d9070c8b Merge pull request #1269 from BrianBland/crossRepositoryPush
Adds cross-repository blob mounting behavior
2016-01-08 14:37:00 -08:00
Brian Bland
3a35a2d953 Allows token authentication handler to request additional scopes
When an auth request provides the "from" query parameter, the token
handler will add a "pull" scope for the provided repository, refreshing
the token if the overall scope has increased

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:23 -08:00
Brian Bland
5df21570a7 Adds cross-repository blob mounting behavior
Extends blob upload POST endpoint to support mount and from query
parameters as described in #634

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:18 -08:00
Richard Scothern
cc0b6b86ea Merge pull request #1276 from denverdino/oss-test
Support large layer for OSS driver
2016-01-08 13:26:09 -08:00
Stephen Day
a7ae88da45 Merge pull request #1281 from aaronlehmann/new-manifest
Implement schema2 manifest formats
2016-01-07 17:19:56 -08:00
Stephen Day
5ebf6391a7 Merge pull request #911 from stevvooe/consistent-error-string
More consistent return from ErrorCode.Error()
2016-01-07 17:06:12 -08:00
Aaron Lehmann
6d17423a6d Move MediaType into manifest.Versioned
This makes content type sniffing cleaner. The document just needs to be
decoded into a manifest.Versioned structure. It's no longer a two-step
process.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
697af09566 Recognize clients that don't support manifest lists
Convert a default platform's manifest to schema1 on the fly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
9c416f0e94 Add support for manifest list ("fat manifest")
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
9284810356 Add API unit testing for schema2 manifest
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Aaron Lehmann
3f746a8207 Recognize clients that don't support schema2, and convert manifests to schema1 on the fly
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Aaron Lehmann
befd4d6e3c Factor out schema-specific portions of manifestStore
Create signedManifestHandler and schema2ManifestHandler. Use these to
unmarshal and put the respective types of manifests from manifestStore.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Richard Scothern
9f83bfc071 Merge pull request #1321 from aibaars/gcs-fix-retry
GCS storage driver: fix retry function
2016-01-06 12:00:12 -08:00
Arthur Baars
bb1e9e0a82 GCS Storagedriver: fix test failure caused by #1187
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-06 18:36:28 +00:00
Arthur Baars
d3d40a8807 GCS driver: fix retry function
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-06 17:57:22 +00:00
Richard Scothern
94347c8611 Remove tags referencing deleted manifests.
When a manifest is deleted by digest, look up the referenced tags in the tag
store and remove all associations.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-05 13:47:52 -08:00
Stephen Day
74d719d83f Merge pull request #1249 from k4leung4/print-error-msg
Print error for failed HTTP auth request.
2016-01-05 12:23:05 -08:00
Stephen Day
0e73b8a37f Merge pull request #1097 from hopkings2008/master
use the scheme and host from x-forward-proto and x-forward-host if the…
2015-12-30 16:35:20 -08:00
Stephen Day
00b983b837 Merge pull request #1304 from BrianBland/fsDriverRelaxedPermissions
Relaxes filesystem driver permissions to 0777 (dirs) and 0666 (files)
2015-12-29 19:06:45 -08:00
Kenny Leung
7ff5042c1e Merge branch 'print-error-msg' of https://github.com/k4leung4/distribution into print-error-msg
Changed to use typed error instead of formatted string.
Added tests for new public method.

Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-29 16:26:13 -08:00
Kenny Leung
c28278f7a2 Print error for failed HTTP auth request.
Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-29 11:39:20 -08:00