Commit graph

477 commits

Author SHA1 Message Date
Stephen J Day
4829e9685e registry/storage/driver: checking that non-existent path returns PathNotFoundError
Issue #1186 describes a condition where a null tags response is returned when
using the s3 driver. The issue seems to be related to a missing
PathNotFoundError in s3. This change adds a test for that to get an idea of the
lack of compliance across storage drivers. If the failures are manageable,
we'll add this test condition and fix the s3 driver.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 12:04:03 -08:00
Stephen Day
51c5b868b1 Merge pull request #1238 from noxiouz/fix_comment
Fix comment for PathRegexp
2015-12-08 10:41:16 -08:00
Stephen Day
9bf537dede Merge pull request #1241 from pdevine/catalog-speedup
Make the catalog more efficient
2015-12-08 10:20:51 -08:00
Richard Scothern
e7e0b72ae0 Merge pull request #1243 from RichardScothern/client-header-message
Add clearer messaging around missing content-length headers.
2015-12-08 09:54:18 -08:00
Patrick Devine
ecb84029ec Make the catalog more efficient
This change removes the sort() from the Repositories() function since
we're now guaranteed to have a lexigraphically sorted walk.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>
2015-12-07 12:33:01 -08:00
Richard Scothern
3b7e1a986a Merge pull request #1157 from lebauce/swift-bulk-delete-fixes
Use bulk delete to remove segments in Swift driver
2015-12-07 10:56:32 -08:00
Richard Scothern
fb2142147f Add clearer messaging around missing content-length headers.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-12-07 10:17:49 -08:00
Anton Tiurin
d6cc32965e Fix comment for PathRegexp
Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
2015-12-04 22:14:21 +03:00
Sylvain Baubeau
b596464d38 Use bulk delete to remove segments in Swift driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-12-04 12:16:49 +01:00
Stephen Day
fb89fd30f8 Merge pull request #1231 from tonistiigi/digest-length-check
Validate digest length on parsing
2015-12-03 12:17:16 -08:00
Stephen Day
0a71df72a1 Merge pull request #1211 from tt/remove-name-verification
Remove name verification
2015-12-03 11:59:21 -08:00
Stephen Day
0aee52fa3d Merge pull request #1227 from stevvooe/walk-sorted
storage: enforce sorted traversal during Walk
2015-12-02 18:37:44 -08:00
Tonis Tiigi
1f5f9bad39 Validate digest length on parsing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-12-02 16:04:06 -08:00
Aaron Lehmann
bf2cc0a9d6 Avoid stat round-trips when fetching a blob
Without this commit, three round-trips are required to fetch a blob with
a progress bar. The first is a call to Stat (HEAD request), to determine
the size. Then Open is called, which also calls Stat, and finally
performs a GET request.

Only the GET request is actually needed. The size of the blob can be
sniffed from Content-Length in the GET response.

This commit changes HTTPReadSeeker to automatically detect the size from
Content-Length instead of requiring it to be passed in. The Stat call is
removed from Open because it is no longer necessary.

HTTPReadSeeker now takes an additional errorHandler callback argument which
translates an unsuccessful HTTP response into an appropriate API-level
error. Using a callback for this makes it possible to avoid leaking the
repsonse body to Read's caller, which would make lifecycle management
problematic.

Fixes #1223

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-02 14:21:13 -08:00
Stephen J Day
6693e9667c storage: add further tests for Walk implementation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:55:10 -08:00
Stephen J Day
93f92498ce storage: enforce sorted traversal during Walk
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:25:30 -08:00
Stephen J Day
7bf8f846c2 storage: correctly handle error during Walk
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:25:29 -08:00
Troels Thomsen
e1cf7c418b Map error type to error code
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:26:37 +01:00
Troels Thomsen
beeff299f8 Use well-known error type
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:23:47 +01:00
Troels Thomsen
8299937613 Verify manifest name format
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-26 10:28:35 +01:00
Troels Thomsen
6fb6183083 Verify manifest name length
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-26 10:28:28 +01:00
Troels Thomsen
a9a1b57900 Remove name verification
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-25 21:16:28 +01:00
Troels Thomsen
8257e8c42a Use case of type name
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-25 21:12:49 +01:00
Stephen Day
c0d094a72a Merge pull request #1166 from treed/master
De-obfuscate error message
2015-11-11 17:18:25 -08:00
Richard Scothern
a53b1a9989 Merge pull request #1125 from rustyconover/fix-url-path-escaping
Fix failing test case for URL escaping problem.
2015-11-10 16:50:55 -08:00
Richard Scothern
aaf448b2de Merge pull request #1141 from lebauce/swift-no-missing-segment
Ensure read after write for segments
2015-11-10 10:39:43 -08:00
Richard Scothern
d7b999e903 Merge pull request #1172 from aaronlehmann/swift-empty-delete-request
Fix empty delete requests with Swift fs driver
2015-11-10 10:36:19 -08:00
Aaron Lehmann
accfa46f9b Fix empty delete requests with Swift fs driver
The Delete method lists objects under the given path and tries to delete
all of them with a bulk delete request. If the path has no objects
underneath it, the body of this request will be empty, which causes
HTTP-level issues. Specifically, Go's HTTP client senses the empty
request buffer and doesn't include a Content-Length, which causes the
Swift server to fail the request.

This commit fixes the problem by avoiding sending empty bulk delete
requests. This is the correct thing to do anyway, since there's no
reason to request deletion of zero objects.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-09 15:33:05 -08:00
Ted Reed
f01a70c8a6 De-obfuscate error message
Previously, this error message would stringify as a pointer address,
which isn't particularly helpful.

This change breaks out the elements of the challenge object such that
the error is appropriately represented.

Signed-off-by: Ted Reed <ted.reed@gmail.com>
2015-11-06 17:10:28 -08:00
Richard Scothern
057284b593 Merge pull request #1156 from RichardScothern/manifest-verification
Manifest Verification
2015-11-04 14:09:12 -08:00
Richard Scothern
78b6d648fa Before allowing a schema1 manifest to be stored in the registry, ensure that it
contains equal length History and FSLayer arrays.

This is required to prevent malformed manifests being put to the registry and
failing external verification checks.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 15:19:46 -08:00
Richard Scothern
573be447c6 Merge pull request #1150 from RichardScothern/carry-1070
Add Storage Driver Context.
2015-11-03 15:15:48 -08:00
Richard Scothern
e79324edd8 Add a generic error type to capture non-typed errors
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 11:19:44 -08:00
amitshukla
7840a5bc8f Fix for issue 664: https://github.com/docker/distribution/issues/664
Errors thrown by storage drivers don't have the name of the driver, causing user
confusion about whether the error is coming from Docker or from a storage driver.
This change adds the storage driver name to each error message.

This required changing ErrUnsupportedDriver to a type, leading to code changes
whenever ErrUnsupportedDriver is used.  The tests check whether the driver name
appears in the error message.

Signed-off-by: Amit Shukla <amit.shukla@docker.com>
2015-11-03 11:19:17 -08:00
Richard Scothern
bd958d8b88 Merge pull request #1114 from lebauce/swift-temp-url
Redirect support in Swift driver
2015-11-03 09:34:01 -08:00
Sylvain Baubeau
34c1d0ed50 Ensure read after write for segments
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-11-03 17:09:52 +01:00
Sylvain Baubeau
11546b5309 Add support for temporary URL for Swift driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-11-03 09:59:50 +01:00
Stephen J Day
854fa0a4dd registry/storage: close filereader after allocation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-11-02 11:55:11 -08:00
Richard Scothern
00f02b5fbc Buffer writing the scheduler entry state to disk by periodically checking for
changes to the entries index and saving it to the filesystem.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-29 11:35:52 -07:00
Richard Scothern
a31e93e1c4 Merge pull request #756 from aibaars/storage-gcs
Storage driver for Google Cloud Storage
2015-10-29 10:41:17 -07:00
Richard Scothern
cbf1f0797c Merge pull request #1096 from aaronlehmann/simplify-proxy-scheduler
Simplify proxy scheduler
2015-10-28 13:23:00 -07:00
Arthur Baars
98ad17f757 Storage driver for: Google Cloud Storage (gcs)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2015-10-28 20:21:56 +00:00
Matt Moore
b38b98c8a8 Add expires_in and issued_at to the auth spec.
This extends the specification for the Bearer token response to include
information pertaining to when an issued Bearer token will expire.

This also allows the client to accept `access_token` as an alias for `token`.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-27 11:05:33 -07:00
Rusty Conover
9293e3db11 Fix failing test case
When building a URL don't include path components in the host parameter.

Closes #1124

Signed-off-by: Rusty Conover <rusty@luckydinosaur.com>
2015-10-25 13:03:22 -04:00
Stephen Day
8cc790cccf Merge pull request #1098 from denverdino/master
Redundant digest verification in validateBlob when pushing a new layer
2015-10-21 14:50:18 -07:00
Stephen Day
cc69dcca8d Merge pull request #1115 from tianon/fix-auth-servehttp-comment
Update "type auth.Challenge" comment example code
2015-10-21 12:56:14 -07:00
Tianon Gravi
8263cdeb57 Update "type auth.Challenge" comment example code
This interface was changed in 4a2300aaa9, but the comment wasn't ever updated to match.

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-10-21 11:54:13 -07:00
Richard Scothern
8806306f3d Merge pull request #1111 from aaronlehmann/golint-comments
Correct two golint comment issues
2015-10-20 09:53:14 -07:00
Aaron Lehmann
fc5ee720d1 Correct two golint comment issues
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-19 16:42:12 -07:00
Richard Scothern
0ce983b4e8 Merge pull request #1006 from RichardScothern/proxy-serialize
Fix a race condition in pull through cache population
2015-10-16 17:13:26 -07:00