Commit graph

655 commits

Author SHA1 Message Date
Derek McGowan
c9880e6b05 Fetch token by credentials and refresh token
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-03-04 00:34:17 -08:00
Richard Scothern
62d46ef07d Merge pull request #1388 from aibaars/gcs-simplify-move
StorageDriver: GCS: remove support for directory Moves
2016-03-03 10:20:51 -08:00
Arthur Baars
1593c2413f StorageDriver: GCS: remove support for directory Moves
The Move operation is only used to move uploaded blobs
to their final destination. There is no point in implementing
Move on "folders". Apart from simplifying the code, this also
saves an HTTP request.

Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-03-03 13:28:13 +00:00
Richard Scothern
c8dff1bb57 Merge pull request #1364 from aaronlehmann/content-type-in-api-docs
Add information about manifest content types to API spec
2016-03-01 13:53:49 -08:00
Richard Scothern
5cbac00dd4 Merge pull request #1386 from andrewnguyen/garbage_collect_pick_from_1050
garbage collection
2016-02-29 15:01:48 -08:00
Andrew T Nguyen
feab4aafbc Implements garbage collection subcommand
- Includes a change in the command to run the registry. The registry
  server itself is now started up as a subcommand.
- Includes changes to the high level interfaces to support enumeration
  of various registry objects.

Signed-off-by: Andrew T Nguyen <andrew.nguyen@docker.com>
2016-02-29 14:15:21 -08:00
Brian Bland
3ae504384c Merge pull request #1473 from miminar/pass-blob-size
Commit uploaded blob with size
2016-02-29 10:26:45 -08:00
Aaron Lehmann
a5bd3e6144 Merge pull request #1474 from miminar/respect-errors
Respect errors returned from middleware code
2016-02-26 23:28:37 -08:00
Brian Bland
e430d77342 Merge pull request #1482 from stweil/master
Fix some typos in comments and strings
2016-02-23 14:08:21 -08:00
Stefan Weil
615c6dfced Fix some typos in comments and strings
All of them were found and fixed by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2016-02-23 22:33:38 +01:00
Aaron Lehmann
7571572305 Merge pull request #1471 from RichardScothern/tag-headers
Enable proxying registries to downgrade fetched manifests to Schema 1.
2016-02-23 11:08:46 -08:00
Michal Minar
21a0878665 Commit blob with known size
Signed-off-by: Michal Minar <miminar@redhat.com>
2016-02-23 08:44:06 +01:00
Richard Scothern
7144936612 Enable proxying registries to downgrade fetched manifests to Schema 1.
Ensure Accept headers are sent with TagService.Get (which hits manifest
endpoints).  Add support for remote Get and Put for the proxied blobstore.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-22 17:56:48 -08:00
Michal Minar
4d15bf071c Defined ErrAccessDenied error
Middleware code may perform additional checks on blobs written. Allow it
to return access denied errors that will result in 403 Forbidden.

Signed-off-by: Michal Minar <miminar@redhat.com>
2016-02-22 21:12:59 +01:00
xiekeyang
d58188c4d7 compare error output in tagstore unit test
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
2016-02-22 17:34:22 +08:00
Richard Scothern
ad1b181291 Merge pull request #1466 from RichardScothern/proxy-lazy-auth
Lazily evaluate auth challenges
2016-02-19 15:06:35 -08:00
Richard Scothern
4ce15476bd Extend authChallenger interface to remove type cast.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-17 13:07:14 -08:00
Brian Bland
5806f275bf Merge pull request #1469 from noxiouz/fix_storage_interface_desc
Fix description of StorageDriver.WriteStream
2016-02-17 10:46:11 -08:00
Anton Tiurin
a048a4c8d5 Fix description of StorageDriver.WriteStream
Offset can be more than CurrentSize as long as this case is checked
by DriverSuite.testContinueStreamAppend.

Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
2016-02-17 13:57:20 +03:00
HuKeping
5408d4e0a5 Cleanup: remove unused log
Signed-off-by: Hu Keping <hukeping@huawei.com>
2016-02-17 14:11:20 +08:00
Brian Bland
c770f28f04 [driver/s3aws] Fix TestStorageClass
Fixes bug in TestStorageClass for s3aws driver where the "standard" file
was checked for reduced-redundnancy storage.

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-16 17:50:55 -08:00
Richard Scothern
56af60ad24 To avoid any network use unless necessary, delay establishing authorization
challenges with the upstream until any proxied data is found not to be local.

Implement auth challenges behind an interface and add to unit tests.  Also,
remove a non-sensical unit test.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-02-16 11:53:42 -08:00
Brian Bland
b9907c4853 Merge pull request #1385 from BrianBland/s3UseAWSLibrary
[driver/s3] Use aws/aws-sdk-go instead of goamz for s3 driver and cloudfront
2016-02-16 11:19:15 -08:00
Derek McGowan
98620458e3 Add credential authenticator interface
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-16 10:23:35 -08:00
liuchang0812
0758653b42 fix gofmt
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-02-16 11:42:09 +08:00
liuchang0812
dbba9cf299 closes #1461, enhance log message of oss driver
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-02-16 11:17:09 +08:00
Brian Bland
1319be2cae Adds new s3 driver using aws-sdk-go instead of goamz
Keeps old s3 driver, renames to s3goamz, registers new s3 driver as both
"s3" and "s3aws"

Changes cloudfront middleware to use aws-sdk-go

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-12 15:59:26 -08:00
Olivier Gambier
7b66c50bb7 Merge pull request #1452 from aaronlehmann/export-no-credentials-error
Export "no basic auth credentials" as an error value
2016-02-11 10:09:22 -08:00
Olivier Gambier
3c80f6fb5e Merge pull request #1451 from aaronlehmann/typo-fixes
Typo fixes in comments
2016-02-10 17:12:00 -08:00
Aaron Lehmann
b046861e40 Export "no basic auth credentials" as an error value
Making this an exported error value will allow users of the
registry/client/auth module to have consistent behavior between
authentication failures and cases where no credentials are provided.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-10 16:46:57 -08:00
Aaron Lehmann
d6a9b20971 Merge pull request #1445 from dmcgowan/fix-manifest-digest-header
Fix schema1 manifest etag and docker content digest header
2016-02-10 16:27:18 -08:00
Aaron Lehmann
aa80478b64 Typo fixes in comments
Correct spelling of words in source code comments.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-10 16:26:29 -08:00
Aaron Lehmann
a3213ff331 Merge pull request #1420 from dmcgowan/configurable-trust-key
Add option to disable signatures
2016-02-10 16:15:59 -08:00
Derek McGowan
b34e571bff Add option to disable signatures
Add option for specifying trust key for signing schema1 manifests.
Since schema1 signature key identifiers are not verified anywhere and deprecated, storing signatures is no longer a requirement.
Furthermore in schema2 there is no signature, requiring the registry to already add signatures to generated schema1 manifests.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-10 15:20:39 -08:00
Derek McGowan
350234898a Fix schema1 manifest etag and docker content digest header
When schema2 manifests are rewritten as schema1 currently the etag and docker content digest header keep the value for the schema2 manifest.

Fixes #1444

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-02-09 18:28:43 -08:00
Brian Bland
52b4a328dd Improves flexibility of configuration handling for S3 driver
Treats nil parameters the same as unprovided parameters (fixes issues
where certain parameters are printed to "<nil>").
Accepts "true" and "false" string values for boolean parameters.

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-09 17:27:16 -08:00
Richard Scothern
dd58349b35 Merge pull request #1419 from aaronlehmann/safer-header-copying
On redirect, only copy headers when they don't already exist in the redirected request
2016-02-05 12:14:01 -08:00
Aaron Lehmann
7761a70389 Add information about manifest content types to API spec
Bring the spec up to date for schema2 changes.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-05 11:27:20 -08:00
Richard Scothern
4a1d36c22d Correct type for repo reference
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-04 17:32:55 -08:00
Derek McGowan
ab9b433fca Merge pull request #1417 from RichardScothern/token-print
Print the correct token expiration time
2016-02-04 16:57:08 -08:00
Richard Scothern
579981b979 Merge pull request #1350 from aibaars/storage-filewriter-pointer
Storage: remove bufferedFileWriter (dead code)
2016-02-04 16:47:52 -08:00
Richard Scothern
2cc6ccbded Merge pull request #1401 from BrianBland/s3StorageClass
Adds "storageclass" configuration parameter for S3 driver.
2016-02-04 16:44:13 -08:00
Richard Scothern
b737de4856 Merge pull request #1408 from aaronlehmann/repository-interface
Rename Name method of Repository to Named
2016-02-04 16:43:35 -08:00
Aaron Lehmann
b0989446eb Rename Name method of Repository to Named
This makes code that gets the name as a string read like
repo.Named().Name() instead of repo.Name().Name().

Requested in
https://github.com/docker/docker/pull/19887#discussion_r51479753

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-04 09:55:36 -08:00
yuzou
44c0b9ae87 read the actual number of bytes according to the initial size.
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-02-04 16:14:35 +08:00
Aaron Lehmann
4354f0a107 On redirect, only copy headers when they don't already exist in the redirected request
A changeset under consideration for Go 1.7 would automatically copy
headers on redirect. This change future-proofs our code so we won't make
duplicate copies of the headers if net/http does it automatically in the
future.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-03 13:19:44 -08:00
Richard Scothern
f5823ff0be Print the correct token expiration time
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-03 10:42:32 -08:00
Olivier Gambier
830bef975d Merge pull request #1410 from aaronlehmann/failured
Correct ErrAuthenticationFailure message
2016-02-01 19:20:35 -08:00
Aaron Lehmann
e387b63001 Correct ErrAuthenticationFailure message
This was "authentication failured". Change it to "authentication
failure".

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-01 17:03:41 -08:00
Aaron Lehmann
e10efb2ecc Merge pull request #1403 from dmcgowan/auth-const-keys
Update auth context keys to use constant
2016-02-01 16:29:07 -08:00
Aaron Lehmann
d3fb5c7064 Merge pull request #1390 from dmcgowan/token-server
Simple integration test token server
2016-02-01 16:27:49 -08:00
Brian Bland
695bac7b0f Adds test for S3 storage class configuration option
Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 16:01:55 -08:00
Brian Bland
4688d1adc2 Adds "storageclass" configuration parameter for S3 driver.
Defaults to STANDARD, also supports REDUCED_REDUNDANCY.

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 16:01:19 -08:00
Aaron Lehmann
2fc586d2a4 Merge pull request #1381 from BrianBland/s3CustomUAString
Adds custom registry User-Agent header to s3 HTTP requests
2016-02-01 15:40:36 -08:00
Brian Bland
2dc1af12a1 Adds custom registry User-Agent header to s3 HTTP requests
Uses docker/goamz instead of AdRoll/goamz

Adds a registry UA string param to the storage parameters when
constructing the storage driver for the registry App.
This could be used by other storage drivers as well

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-02-01 13:43:46 -08:00
Derek McGowan
648a1343db Update auth context keys to use constant
Prevent using strings throughout the code to reference a string key defined in the auth package.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-28 17:02:09 -08:00
Aaron Lehmann
74c4a0e3e3 Merge pull request #1393 from RichardScothern/putbydgst
Add manifest put by digest to the registry client
2016-01-28 14:41:07 -08:00
Derek McGowan
70e85dde26 Merge pull request #1392 from aaronlehmann/client-range-requests
Support range requests in the client's httpReadSeeker
2016-01-28 13:44:43 -08:00
Aaron Lehmann
8fd84c4a57 Add a CheckRedirect function to the HTTP client
Use it to preserve Accept and Range headers that were added to the
original request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Aaron Lehmann
ccf8154a44 Support range requests in the client's httpReadSeeker
Remove buffering on the reader, because it's not useful. Also remove
artificial io.EOF return.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Richard Scothern
a8861549cf Invalidate the blob store descriptor caches when content is removed from from
the proxy.  Also, switch to reference in the scheduler API.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:59 -08:00
Richard Scothern
584c9b517c Correct test digest lengths and enable all unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 17:02:46 -08:00
Richard Scothern
9f72e8442d Add manifest put by digest to the registry client
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 14:20:23 -08:00
Derek McGowan
fd17443988 Update token header struct to use json.RawMessage pointer
Since RawMessage json receivers take a pointer type, the Header structure should use points in order to call the json.RawMessage marshal and unmarshal functions

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-01-25 20:11:41 -08:00
Arthur Baars
3d67dae525 Storage: remove bufferedFileWriter (dead code)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-23 10:30:08 +00:00
Arthur Baars
26e7268a86 Storage: blobwriter.Write/Seek test case
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-23 10:30:07 +00:00
Aaron Lehmann
2b20b0167a Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann
4441333912 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Aaron Lehmann
c01fe47231 If the media type for a manifest is unrecognized, default to schema1
This is needed for compatibility with some third-party registries that
send an inappropriate Content-Type header such as text/html.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-21 09:34:06 -08:00
Aaron Lehmann
ec636bbfd2 Handle nonstandard token endpoint errors
https://github.com/docker/distribution/pull/1249 changed token fetching
to parse HTTP error response bodies as serialized errcodes. However,
Docker Hub's authentication endpoint does not return error bodies in
this format. To work around this, convert its format into
ErrCodeUnauthorized or ErrCodeUnknown.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-20 14:45:08 -08:00
Arthur Baars
2a4345ca4b StorageDriver: GCS: retry all api calls
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Arthur Baars
6ee339464c StorageDriver: Test suite: improve cleanup
Verify that the file(s) have been deleted after calling Delete,
and retry if this is not the case. Furthermore, report the error
if a Delete operation fails.

Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-20 13:24:09 +00:00
Aaron Lehmann
8c1a000799 Fix content type for schema1 signed manifests
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.

Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 16:08:02 -08:00
Richard Scothern
3cb403ae5b Merge pull request #1363 from aaronlehmann/media-type-charset
Do not require "charset=utf-8" for a schema1 with content type application/json
2016-01-18 15:20:01 -08:00
Richard Scothern
fc7ee6d789 Merge pull request #1355 from hopkings2008/master
In testsuites.go, enlarge the size of randomBytes to 128M to fix the …
2016-01-18 12:38:26 -08:00
Aaron Lehmann
5a2664e0b1 Do not require "charset=utf-8" for a schema1 with content type application/json
For compatibility with other registries that don't use this exact
variant of the Content-Type header, we need to be more flexible about
what we accept. Any form of "application/json" should be allowed. The
charset should not be included in the comparison.

See docker/docker#19400.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-18 09:59:50 -08:00
Arthur Baars
19d7c9e051 StorageDriver GCS: try google.DefaultTokenSource first
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-15 11:47:47 +00:00
yuzou
cc82b0d48e In testsuites.go, enlarge the size of randomBytes to 128M to fix the crash of running TestConcurrentStreamReads
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-01-15 17:22:43 +08:00
Richard Scothern
7378e21678 Merge pull request #1332 from RichardScothern/gcs-params
Change the parameters to the GCS drivers to allow CircleCI testing.
2016-01-14 15:50:09 -08:00
Brian Bland
8c5a6c13c0 Splits up blob create options definitions to be package-specific
Redefines privately in both storage and client packages

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-14 10:47:33 -08:00
Brian Bland
3ef917c269 Fixes cross-repo blob mounting in the BlobUploadHandler
Accidentally checked for err != nil instead of err == nil :/
Also now ensures that only a non-nil option is appended to the create
options slice

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 19:21:45 -08:00
Richard Scothern
81dbebc7d3 Change the parameters to the GCS drivers to allow CircleCI testing.
Remove the requirement of file system access to run GCS unit tests.  Deconstruct
the input parameters to take the private key and email which can be specified on
the build system via environment variables.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-13 18:00:59 -08:00
Brian Bland
ce88d8a6f4 Adds functional options arguments to the Blobs Create method
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 16:42:59 -08:00
Richard Scothern
3ea0d70896 Fix manifest API unit tests
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-11 12:52:21 -08:00
Richard Scothern
3a1220de01 Merge pull request #1319 from RichardScothern/update-tags
Remove tags referencing deleted manifests.
2016-01-11 11:33:22 -08:00
Richard Scothern
93d9070c8b Merge pull request #1269 from BrianBland/crossRepositoryPush
Adds cross-repository blob mounting behavior
2016-01-08 14:37:00 -08:00
Brian Bland
3a35a2d953 Allows token authentication handler to request additional scopes
When an auth request provides the "from" query parameter, the token
handler will add a "pull" scope for the provided repository, refreshing
the token if the overall scope has increased

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:23 -08:00
Brian Bland
5df21570a7 Adds cross-repository blob mounting behavior
Extends blob upload POST endpoint to support mount and from query
parameters as described in #634

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:18 -08:00
Richard Scothern
cc0b6b86ea Merge pull request #1276 from denverdino/oss-test
Support large layer for OSS driver
2016-01-08 13:26:09 -08:00
Stephen Day
a7ae88da45 Merge pull request #1281 from aaronlehmann/new-manifest
Implement schema2 manifest formats
2016-01-07 17:19:56 -08:00
Stephen Day
5ebf6391a7 Merge pull request #911 from stevvooe/consistent-error-string
More consistent return from ErrorCode.Error()
2016-01-07 17:06:12 -08:00
Aaron Lehmann
6d17423a6d Move MediaType into manifest.Versioned
This makes content type sniffing cleaner. The document just needs to be
decoded into a manifest.Versioned structure. It's no longer a two-step
process.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
697af09566 Recognize clients that don't support manifest lists
Convert a default platform's manifest to schema1 on the fly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
9c416f0e94 Add support for manifest list ("fat manifest")
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:27 -08:00
Aaron Lehmann
9284810356 Add API unit testing for schema2 manifest
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Aaron Lehmann
3f746a8207 Recognize clients that don't support schema2, and convert manifests to schema1 on the fly
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Aaron Lehmann
befd4d6e3c Factor out schema-specific portions of manifestStore
Create signedManifestHandler and schema2ManifestHandler. Use these to
unmarshal and put the respective types of manifests from manifestStore.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-07 15:26:26 -08:00
Richard Scothern
9f83bfc071 Merge pull request #1321 from aibaars/gcs-fix-retry
GCS storage driver: fix retry function
2016-01-06 12:00:12 -08:00
Arthur Baars
bb1e9e0a82 GCS Storagedriver: fix test failure caused by #1187
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-06 18:36:28 +00:00
Arthur Baars
d3d40a8807 GCS driver: fix retry function
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-01-06 17:57:22 +00:00
Richard Scothern
94347c8611 Remove tags referencing deleted manifests.
When a manifest is deleted by digest, look up the referenced tags in the tag
store and remove all associations.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-05 13:47:52 -08:00
Stephen Day
74d719d83f Merge pull request #1249 from k4leung4/print-error-msg
Print error for failed HTTP auth request.
2016-01-05 12:23:05 -08:00
Stephen Day
0e73b8a37f Merge pull request #1097 from hopkings2008/master
use the scheme and host from x-forward-proto and x-forward-host if the…
2015-12-30 16:35:20 -08:00
Stephen Day
00b983b837 Merge pull request #1304 from BrianBland/fsDriverRelaxedPermissions
Relaxes filesystem driver permissions to 0777 (dirs) and 0666 (files)
2015-12-29 19:06:45 -08:00
Kenny Leung
7ff5042c1e Merge branch 'print-error-msg' of https://github.com/k4leung4/distribution into print-error-msg
Changed to use typed error instead of formatted string.
Added tests for new public method.

Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-29 16:26:13 -08:00
Kenny Leung
c28278f7a2 Print error for failed HTTP auth request.
Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-29 11:39:20 -08:00
weiyuan.yl
75abeeca11 Replace 404 to http.StatusNotFound
Change-Id: Ia100975cb93c0a6d94ea5542b1c9ce386bc87649
Signed-off-by: weiyuan.yl <weiyuan.yl@alibaba-inc.com>
2015-12-29 12:09:04 +08:00
Brian Bland
f19deba809 Relaxes filesystem driver permissions to 0777 (dirs) and 0666 (files)
Leaves any further permissions restrictions to the process umask

Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-12-28 15:22:28 -08:00
Brian Bland
cf487a7911 Serve blobs when a storage driver supports redirects but are disabled
Fixes issue where an error was returned instead of serving the blob

Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-12-28 11:08:39 -08:00
yuzou
d724bb1d24 use the scheme and host from x-forward-proto and x-forward-host if they exits and correct the scheme for Location header during image upload
Signed-off-by: yuzou <zouyu7@huawei.com>
2015-12-28 17:28:32 +08:00
weiyuan.yl
3e4b554f8d In HEAD request for missing resource, only 404 NOT FOUND is returned
Change-Id: I73caf67b59519e6f4f82f7d78f5d4fd4ad9affcd
Signed-off-by: weiyuan.yl <weiyuan.yl@alibaba-inc.com>
2015-12-28 11:28:42 +08:00
Richard Scothern
67d3675d55 Merge pull request #1268 from RichardScothern/manifest-refactor-impl
Implementation of the Manifest Service API refactor.
2015-12-17 17:32:55 -08:00
Richard Scothern
cb6f002350 Implementation of the Manifest Service API refactor.
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.

Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.

Remove Signatures() from Repository interface.

Signatures are relevant only to schema1 manifests.  Move access to the signature
store inside the schema1 manifestStore.  Add some API tests to verify
signature roundtripping.

schema1
-------

Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.

tags
----

Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces.  Implement a driver based tagService to be backward
compatible with the current tag service.

Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.

manifest store
--------------

Remove revision store and move all signing functionality into the signed manifeststore.

manifest registration
---------------------

Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.

client
------

Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs.  Change functional arguments to be an interface type
to allow passing data without mutating shared state.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2015-12-17 17:09:14 -08:00
Stephen Day
7f41cd7587 Merge pull request #1257 from aaronlehmann/remove-extra-stat
Remove unnecessary stat from blob Get method
2015-12-17 14:10:12 -08:00
Aaron Lehmann
4c850e7165 Remove tarsum support for digest package
tarsum is not actually used by the registry. Remove support for it.

Convert numerous uses in unit tests to SHA256.

Update docs to remove mentions of tarsums (which were often inaccurate).

Remove tarsum dependency.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-15 17:22:18 -08:00
Aaron Lehmann
31047c8113 Simplify digest.FromBytes calling convention
The current implementation of digest.FromBytes returns an error. This
error can never be non-nil, but its presence in the function signature
means each call site needs error handling code for an error that is
always nil.

I verified that none of the hash.Hash implementations in the standard
library can return an error on Write. Nor can any of the hash.Hash
implementations vendored in distribution.

This commit changes digest.FromBytes not to return an error. If Write
returns an error, it will panic, but as discussed above, this should
never happen.

This commit also avoids using a bytes.Reader to feed data into the hash
function in FromBytes. This makes the hypothetical case that would panic
a bit more explicit, and should also be more performant.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-14 14:30:51 -08:00
Aaron Lehmann
be1f4114c7 Remove unnecessary stat from blob Get method
This calls Stat before Open, which should be unnecessary because Open
can handle the case of a nonexistent blob. Removing the Stat saves a
round trip.

This is similar to the removal of stat in Open in #1226.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-11 15:13:03 -08:00
Li Yi
54da47d636 Support large layer for OSS driver
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-12-11 10:56:50 +08:00
Richard Scothern
796d6e7915 Merge pull request #1187 from stevvooe/check-storage-drivers-list-path-not-found
[WIP] registry/storage/driver: checking that non-existent path returns PathNotFoundError
2015-12-08 16:32:02 -08:00
Stephen J Day
66cd2bf950 storage/driver/s3: adjust s3 driver to return unmunged path
This fixes both the s3 driver and the oss driver to return the unmunged path
when returning errors.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 15:48:55 -08:00
Stephen Day
e4208ad5a7 Merge pull request #1246 from stevvooe/decrease-driver-testsuite-allocations
storage/driver: decrease memory allocation done during testsuite
2015-12-08 14:52:59 -08:00
Kenny Leung
e128a821db Print error for failed HTTP auth request.
Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-08 14:24:03 -08:00
Li Yi
b6756a3d89 Fix the issue for listing root directory
Change-Id: I1c6181fa4e5666bd2e6ec69cb608c4778ae0fe48
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-12-08 12:04:04 -08:00
Li Yi
7161fa0559 Fix for stevvooe:check-storage-drivers-list-path-not-found in OSS driver
Change-Id: I5e96fe761d3833c962084fd2d597f47e8a72e7c2
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-12-08 12:04:04 -08:00
davidli
79d4d7f546 driver/swift: treat empty object list as a PathNotFoundError
Swift returns an empty object list when trying to read a non-existing object path, treat it as a
PathNotFoundError when trying to list a non existing virtual directory.

Signed-off-by: David li <wenquan.li@hpe.com>
2015-12-08 12:04:04 -08:00
Vincent Giersch
f25ccea279 driver/rados: treat OMAP EIO as a PathNotFoundError
RADOS returns a -EIO when trying to read a non-existing OMAP, treat it as a
PathNotFoundError when trying to list a non existing virtual directory.

Signed-off-by: Vincent Giersch <vincent@giersch.fr>
2015-12-08 12:04:04 -08:00
Stephen J Day
a889f46223 storage/driver/s3: correct response on list of missing directory
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 12:04:03 -08:00
Stephen J Day
b45078eb44 storage/driver/base: use correct error format style
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 12:04:03 -08:00
Stephen J Day
a4eae0917e driver/filesystem: address filesystem driver on behavior of List
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 12:04:03 -08:00
Stephen J Day
0322c3bf1f registry/storage/driver: checking that non-existent path returns PathNotFoundError
Issue #1186 describes a condition where a null tags response is returned when
using the s3 driver. The issue seems to be related to a missing
PathNotFoundError in s3. This change adds a test for that to get an idea of the
lack of compliance across storage drivers. If the failures are manageable,
we'll add this test condition and fix the s3 driver.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 12:04:03 -08:00
Stephen Day
807a80ff06 Merge pull request #1238 from noxiouz/fix_comment
Fix comment for PathRegexp
2015-12-08 10:41:16 -08:00
Stephen Day
ce17efc71e Merge pull request #1241 from pdevine/catalog-speedup
Make the catalog more efficient
2015-12-08 10:20:51 -08:00
Richard Scothern
95e80a8283 Merge pull request #1243 from RichardScothern/client-header-message
Add clearer messaging around missing content-length headers.
2015-12-08 09:54:18 -08:00
Stephen J Day
9dc3c6df0e storage/driver: decrease memory allocation done during testsuite
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-07 19:05:13 -08:00
Patrick Devine
1027f5e066 Make the catalog more efficient
This change removes the sort() from the Repositories() function since
we're now guaranteed to have a lexigraphically sorted walk.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>
2015-12-07 12:33:01 -08:00
Richard Scothern
74bb6f89c8 Merge pull request #1157 from lebauce/swift-bulk-delete-fixes
Use bulk delete to remove segments in Swift driver
2015-12-07 10:56:32 -08:00
Richard Scothern
f18bf5e343 Add clearer messaging around missing content-length headers.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-12-07 10:17:49 -08:00
Anton Tiurin
d4435b79d9 Fix comment for PathRegexp
Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
2015-12-04 22:14:21 +03:00
Sylvain Baubeau
ef4ab7a885 Use bulk delete to remove segments in Swift driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-12-04 12:16:49 +01:00
Stephen Day
4cf93714f3 Merge pull request #1231 from tonistiigi/digest-length-check
Validate digest length on parsing
2015-12-03 12:17:16 -08:00
Stephen Day
d46b6a8796 Merge pull request #1211 from tt/remove-name-verification
Remove name verification
2015-12-03 11:59:21 -08:00
Stephen Day
b66bb1287c Merge pull request #1227 from stevvooe/walk-sorted
storage: enforce sorted traversal during Walk
2015-12-02 18:37:44 -08:00
Tonis Tiigi
f015982f0f Validate digest length on parsing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-12-02 16:04:06 -08:00
Aaron Lehmann
6beeb935cd Avoid stat round-trips when fetching a blob
Without this commit, three round-trips are required to fetch a blob with
a progress bar. The first is a call to Stat (HEAD request), to determine
the size. Then Open is called, which also calls Stat, and finally
performs a GET request.

Only the GET request is actually needed. The size of the blob can be
sniffed from Content-Length in the GET response.

This commit changes HTTPReadSeeker to automatically detect the size from
Content-Length instead of requiring it to be passed in. The Stat call is
removed from Open because it is no longer necessary.

HTTPReadSeeker now takes an additional errorHandler callback argument which
translates an unsuccessful HTTP response into an appropriate API-level
error. Using a callback for this makes it possible to avoid leaking the
repsonse body to Read's caller, which would make lifecycle management
problematic.

Fixes #1223

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-02 14:21:13 -08:00
Stephen J Day
8aa3ee6923 storage: add further tests for Walk implementation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:55:10 -08:00
Stephen J Day
5d576bc0cd storage: enforce sorted traversal during Walk
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:25:30 -08:00
Stephen J Day
6ad10796ef storage: correctly handle error during Walk
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:25:29 -08:00
Troels Thomsen
300ce35c12 Map error type to error code
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:26:37 +01:00
Troels Thomsen
1ece510198 Use well-known error type
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:23:47 +01:00
Troels Thomsen
d309bce2d1 Verify manifest name format
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-26 10:28:35 +01:00