Commit graph

532 commits

Author SHA1 Message Date
Richard Scothern
7d16fee7a4 To avoid any network use unless necessary, delay establishing authorization
challenges with the upstream until any proxied data is found not to be local.

Implement auth challenges behind an interface and add to unit tests.  Also,
remove a non-sensical unit test.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-02-16 11:53:42 -08:00
Brian Bland
881ef1096f Merge pull request #1385 from BrianBland/s3UseAWSLibrary
[driver/s3] Use aws/aws-sdk-go instead of goamz for s3 driver and cloudfront
2016-02-16 11:19:15 -08:00
liuchang0812
7ca24a7f5a fix gofmt
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-02-16 11:42:09 +08:00
liuchang0812
ad6a0735d2 closes #1461, enhance log message of oss driver
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-02-16 11:17:09 +08:00
Brian Bland
d5a38e4c5f Adds new s3 driver using aws-sdk-go instead of goamz
Keeps old s3 driver, renames to s3goamz, registers new s3 driver as both
"s3" and "s3aws"

Changes cloudfront middleware to use aws-sdk-go

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-12 15:59:26 -08:00
Olivier Gambier
7e0e141948 Merge pull request #1452 from aaronlehmann/export-no-credentials-error
Export "no basic auth credentials" as an error value
2016-02-11 10:09:22 -08:00
Olivier Gambier
ff9448cc4b Merge pull request #1451 from aaronlehmann/typo-fixes
Typo fixes in comments
2016-02-10 17:12:00 -08:00
Aaron Lehmann
cffb4bbbfd Export "no basic auth credentials" as an error value
Making this an exported error value will allow users of the
registry/client/auth module to have consistent behavior between
authentication failures and cases where no credentials are provided.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-10 16:46:57 -08:00
Aaron Lehmann
55906ee341 Merge pull request #1445 from dmcgowan/fix-manifest-digest-header
Fix schema1 manifest etag and docker content digest header
2016-02-10 16:27:18 -08:00
Aaron Lehmann
f77c82ebb3 Typo fixes in comments
Correct spelling of words in source code comments.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-10 16:26:29 -08:00
Aaron Lehmann
2177a6a1bf Merge pull request #1420 from dmcgowan/configurable-trust-key
Add option to disable signatures
2016-02-10 16:15:59 -08:00
Derek McGowan
956ece5c70 Add option to disable signatures
Add option for specifying trust key for signing schema1 manifests.
Since schema1 signature key identifiers are not verified anywhere and deprecated, storing signatures is no longer a requirement.
Furthermore in schema2 there is no signature, requiring the registry to already add signatures to generated schema1 manifests.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-10 15:20:39 -08:00
Derek McGowan
ae59517936 Fix schema1 manifest etag and docker content digest header
When schema2 manifests are rewritten as schema1 currently the etag and docker content digest header keep the value for the schema2 manifest.

Fixes #1444

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-09 18:28:43 -08:00
Brian Bland
4bb5f80885 Improves flexibility of configuration handling for S3 driver
Treats nil parameters the same as unprovided parameters (fixes issues
where certain parameters are printed to "<nil>").
Accepts "true" and "false" string values for boolean parameters.

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-09 17:27:16 -08:00
Richard Scothern
8745d31f60 Merge pull request #1419 from aaronlehmann/safer-header-copying
On redirect, only copy headers when they don't already exist in the redirected request
2016-02-05 12:14:01 -08:00
Richard Scothern
9894643c88 Correct type for repo reference
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-04 17:32:55 -08:00
Derek McGowan
0751bb3179 Merge pull request #1417 from RichardScothern/token-print
Print the correct token expiration time
2016-02-04 16:57:08 -08:00
Richard Scothern
fe0e76f322 Merge pull request #1350 from aibaars/storage-filewriter-pointer
Storage: remove bufferedFileWriter (dead code)
2016-02-04 16:47:52 -08:00
Richard Scothern
def3e430bb Merge pull request #1401 from BrianBland/s3StorageClass
Adds "storageclass" configuration parameter for S3 driver.
2016-02-04 16:44:13 -08:00
Richard Scothern
ab6c08691e Merge pull request #1408 from aaronlehmann/repository-interface
Rename Name method of Repository to Named
2016-02-04 16:43:35 -08:00
Aaron Lehmann
6158eb544d Rename Name method of Repository to Named
This makes code that gets the name as a string read like
repo.Named().Name() instead of repo.Name().Name().

Requested in
https://github.com/docker/docker/pull/19887#discussion_r51479753

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-04 09:55:36 -08:00
yuzou
95b9c7281b read the actual number of bytes according to the initial size.
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-02-04 16:14:35 +08:00
Aaron Lehmann
bbf983c061 On redirect, only copy headers when they don't already exist in the redirected request
A changeset under consideration for Go 1.7 would automatically copy
headers on redirect. This change future-proofs our code so we won't make
duplicate copies of the headers if net/http does it automatically in the
future.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-03 13:19:44 -08:00
Richard Scothern
091c12f86b Print the correct token expiration time
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-03 10:42:32 -08:00
Olivier Gambier
c4b79bda8a Merge pull request #1410 from aaronlehmann/failured
Correct ErrAuthenticationFailure message
2016-02-01 19:20:35 -08:00
Aaron Lehmann
95a50c7236 Correct ErrAuthenticationFailure message
This was "authentication failured". Change it to "authentication
failure".

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-01 17:03:41 -08:00
Aaron Lehmann
9b395d0789 Merge pull request #1403 from dmcgowan/auth-const-keys
Update auth context keys to use constant
2016-02-01 16:29:07 -08:00
Aaron Lehmann
db48ad4d0f Merge pull request #1390 from dmcgowan/token-server
Simple integration test token server
2016-02-01 16:27:49 -08:00
Brian Bland
a2ade36ecf Adds test for S3 storage class configuration option
Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 16:01:55 -08:00
Brian Bland
8e7910826e Adds "storageclass" configuration parameter for S3 driver.
Defaults to STANDARD, also supports REDUCED_REDUNDANCY.

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 16:01:19 -08:00
Aaron Lehmann
6417c05a27 Merge pull request #1381 from BrianBland/s3CustomUAString
Adds custom registry User-Agent header to s3 HTTP requests
2016-02-01 15:40:36 -08:00
Brian Bland
f41a408e34 Adds custom registry User-Agent header to s3 HTTP requests
Uses docker/goamz instead of AdRoll/goamz

Adds a registry UA string param to the storage parameters when
constructing the storage driver for the registry App.
This could be used by other storage drivers as well

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 13:43:46 -08:00
Derek McGowan
badd8c49b6 Update auth context keys to use constant
Prevent using strings throughout the code to reference a string key defined in the auth package.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-28 17:02:09 -08:00
Aaron Lehmann
6c135e03ce Merge pull request #1393 from RichardScothern/putbydgst
Add manifest put by digest to the registry client
2016-01-28 14:41:07 -08:00
Derek McGowan
0e96574ea2 Merge pull request #1392 from aaronlehmann/client-range-requests
Support range requests in the client's httpReadSeeker
2016-01-28 13:44:43 -08:00
Aaron Lehmann
8e571dff41 Add a CheckRedirect function to the HTTP client
Use it to preserve Accept and Range headers that were added to the
original request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Aaron Lehmann
a58b7625ba Support range requests in the client's httpReadSeeker
Remove buffering on the reader, because it's not useful. Also remove
artificial io.EOF return.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Richard Scothern
3e570e59f1 Invalidate the blob store descriptor caches when content is removed from from
the proxy.  Also, switch to reference in the scheduler API.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:59 -08:00
Richard Scothern
a7740f5d0f Correct test digest lengths and enable all unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:46 -08:00
Richard Scothern
f757372dd8 Add manifest put by digest to the registry client
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 14:20:23 -08:00
Derek McGowan
1eed0ddd07 Update token header struct to use json.RawMessage pointer
Since RawMessage json receivers take a pointer type, the Header structure should use points in order to call the json.RawMessage marshal and unmarshal functions

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-25 20:11:41 -08:00
Arthur Baars
7dee3d19d9 Storage: remove bufferedFileWriter (dead code)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-23 10:30:08 +00:00
Arthur Baars
586b3d47a7 Storage: blobwriter.Write/Seek test case
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-23 10:30:07 +00:00
Aaron Lehmann
6149a8c634 Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann
e9692b8037 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Aaron Lehmann
e9bcc96ad2 If the media type for a manifest is unrecognized, default to schema1
This is needed for compatibility with some third-party registries that
send an inappropriate Content-Type header such as text/html.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-21 09:34:06 -08:00
Aaron Lehmann
59254013be Handle nonstandard token endpoint errors
https://github.com/docker/distribution/pull/1249 changed token fetching
to parse HTTP error response bodies as serialized errcodes. However,
Docker Hub's authentication endpoint does not return error bodies in
this format. To work around this, convert its format into
ErrCodeUnauthorized or ErrCodeUnknown.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-20 14:45:08 -08:00
Arthur Baars
59a9607783 StorageDriver: GCS: retry all api calls
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Arthur Baars
ffc9527782 StorageDriver: Test suite: improve cleanup
Verify that the file(s) have been deleted after calling Delete,
and retry if this is not the case. Furthermore, report the error
if a Delete operation fails.

Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Aaron Lehmann
f9a3f028b5 Fix content type for schema1 signed manifests
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.

Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 16:08:02 -08:00