Commit graph

417 commits

Author SHA1 Message Date
Richard Scothern
74bb6f89c8 Merge pull request #1157 from lebauce/swift-bulk-delete-fixes
Use bulk delete to remove segments in Swift driver
2015-12-07 10:56:32 -08:00
Sylvain Baubeau
ef4ab7a885 Use bulk delete to remove segments in Swift driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-12-04 12:16:49 +01:00
Stephen Day
4cf93714f3 Merge pull request #1231 from tonistiigi/digest-length-check
Validate digest length on parsing
2015-12-03 12:17:16 -08:00
Stephen Day
d46b6a8796 Merge pull request #1211 from tt/remove-name-verification
Remove name verification
2015-12-03 11:59:21 -08:00
Stephen Day
b66bb1287c Merge pull request #1227 from stevvooe/walk-sorted
storage: enforce sorted traversal during Walk
2015-12-02 18:37:44 -08:00
Tonis Tiigi
f015982f0f Validate digest length on parsing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-12-02 16:04:06 -08:00
Aaron Lehmann
6beeb935cd Avoid stat round-trips when fetching a blob
Without this commit, three round-trips are required to fetch a blob with
a progress bar. The first is a call to Stat (HEAD request), to determine
the size. Then Open is called, which also calls Stat, and finally
performs a GET request.

Only the GET request is actually needed. The size of the blob can be
sniffed from Content-Length in the GET response.

This commit changes HTTPReadSeeker to automatically detect the size from
Content-Length instead of requiring it to be passed in. The Stat call is
removed from Open because it is no longer necessary.

HTTPReadSeeker now takes an additional errorHandler callback argument which
translates an unsuccessful HTTP response into an appropriate API-level
error. Using a callback for this makes it possible to avoid leaking the
repsonse body to Read's caller, which would make lifecycle management
problematic.

Fixes #1223

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-02 14:21:13 -08:00
Stephen J Day
8aa3ee6923 storage: add further tests for Walk implementation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:55:10 -08:00
Stephen J Day
5d576bc0cd storage: enforce sorted traversal during Walk
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:25:30 -08:00
Stephen J Day
6ad10796ef storage: correctly handle error during Walk
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-01 16:25:29 -08:00
Troels Thomsen
300ce35c12 Map error type to error code
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:26:37 +01:00
Troels Thomsen
1ece510198 Use well-known error type
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-12-01 22:23:47 +01:00
Troels Thomsen
d309bce2d1 Verify manifest name format
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-26 10:28:35 +01:00
Troels Thomsen
34c8194c95 Verify manifest name length
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-26 10:28:28 +01:00
Troels Thomsen
e8f8f4034e Remove name verification
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-25 21:16:28 +01:00
Troels Thomsen
ca9f0451a6 Use case of type name
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2015-11-25 21:12:49 +01:00
Stephen Day
0a3acb2625 Merge pull request #1166 from treed/master
De-obfuscate error message
2015-11-11 17:18:25 -08:00
Richard Scothern
bb745757de Merge pull request #1125 from rustyconover/fix-url-path-escaping
Fix failing test case for URL escaping problem.
2015-11-10 16:50:55 -08:00
Richard Scothern
a33095af80 Merge pull request #1141 from lebauce/swift-no-missing-segment
Ensure read after write for segments
2015-11-10 10:39:43 -08:00
Aaron Lehmann
6664a1384e Fix empty delete requests with Swift fs driver
The Delete method lists objects under the given path and tries to delete
all of them with a bulk delete request. If the path has no objects
underneath it, the body of this request will be empty, which causes
HTTP-level issues. Specifically, Go's HTTP client senses the empty
request buffer and doesn't include a Content-Length, which causes the
Swift server to fail the request.

This commit fixes the problem by avoiding sending empty bulk delete
requests. This is the correct thing to do anyway, since there's no
reason to request deletion of zero objects.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-09 15:33:05 -08:00
Ted Reed
fddbf73a57 De-obfuscate error message
Previously, this error message would stringify as a pointer address,
which isn't particularly helpful.

This change breaks out the elements of the challenge object such that
the error is appropriately represented.

Signed-off-by: Ted Reed <ted.reed@gmail.com>
2015-11-06 17:10:28 -08:00
Richard Scothern
362ae9cc41 Merge pull request #1156 from RichardScothern/manifest-verification
Manifest Verification
2015-11-04 14:09:12 -08:00
Richard Scothern
dd32fbe615 Before allowing a schema1 manifest to be stored in the registry, ensure that it
contains equal length History and FSLayer arrays.

This is required to prevent malformed manifests being put to the registry and
failing external verification checks.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 15:19:46 -08:00
Richard Scothern
bc6e4cdceb Add a generic error type to capture non-typed errors
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 11:19:44 -08:00
amitshukla
9637cb40cd Fix for issue 664: https://github.com/docker/distribution/issues/664
Errors thrown by storage drivers don't have the name of the driver, causing user
confusion about whether the error is coming from Docker or from a storage driver.
This change adds the storage driver name to each error message.

This required changing ErrUnsupportedDriver to a type, leading to code changes
whenever ErrUnsupportedDriver is used.  The tests check whether the driver name
appears in the error message.

Signed-off-by: Amit Shukla <amit.shukla@docker.com>
2015-11-03 11:19:17 -08:00
Richard Scothern
a9da0e5100 Merge pull request #1114 from lebauce/swift-temp-url
Redirect support in Swift driver
2015-11-03 09:34:01 -08:00
Sylvain Baubeau
3ff8af326b Ensure read after write for segments
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-11-03 17:09:52 +01:00
Sylvain Baubeau
7c3281861f Add support for temporary URL for Swift driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-11-03 09:59:50 +01:00
Stephen J Day
46dd057f5c registry/storage: close filereader after allocation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-11-02 11:55:11 -08:00
Richard Scothern
33428c37e1 Buffer writing the scheduler entry state to disk by periodically checking for
changes to the entries index and saving it to the filesystem.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-29 11:35:52 -07:00
Richard Scothern
9e8aaf7b40 Merge pull request #756 from aibaars/storage-gcs
Storage driver for Google Cloud Storage
2015-10-29 10:41:17 -07:00
Richard Scothern
987a69dd05 Merge pull request #1096 from aaronlehmann/simplify-proxy-scheduler
Simplify proxy scheduler
2015-10-28 13:23:00 -07:00
Arthur Baars
59784ecdd0 Storage driver for: Google Cloud Storage (gcs)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2015-10-28 20:21:56 +00:00
Matt Moore
0c7cd3fcf8 Add expires_in and issued_at to the auth spec.
This extends the specification for the Bearer token response to include
information pertaining to when an issued Bearer token will expire.

This also allows the client to accept `access_token` as an alias for `token`.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-27 11:05:33 -07:00
Rusty Conover
5f2205981b Fix failing test case
When building a URL don't include path components in the host parameter.

Closes #1124

Signed-off-by: Rusty Conover <rusty@luckydinosaur.com>
2015-10-25 13:03:22 -04:00
Stephen Day
e5d9837970 Merge pull request #1098 from denverdino/master
Redundant digest verification in validateBlob when pushing a new layer
2015-10-21 14:50:18 -07:00
Tianon Gravi
39d47e8eea Update "type auth.Challenge" comment example code
This interface was changed in a0fdfb9d4d, but the comment wasn't ever updated to match.

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-10-21 11:54:13 -07:00
Aaron Lehmann
5608db616c Correct two golint comment issues
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-19 16:42:12 -07:00
Richard Scothern
d4910831a3 Merge pull request #1006 from RichardScothern/proxy-serialize
Fix a race condition in pull through cache population
2015-10-16 17:13:26 -07:00
Stephen Day
dfe60f4cb1 Merge pull request #827 from aaronlehmann/read-only-mode-2
Add a read-only mode as a configuration option
2015-10-15 11:50:31 -07:00
Aaron Lehmann
c56e8c2533 Simplify proxy scheduler
The proxy scheduler implemented its own timer state machine. It's
simpler and more efficient to leverage the Go runtime's timer heap by
using time.AfterFunc.

This commit adds a time.Timer to each scheduler entry, and starts and
stops those timers as necessary. Then the mainloop goroutine and its
associated logic are not needed.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-15 09:18:21 -07:00
Li Yi
ad14545745 Redundant digest verification in validateBlob
Change-Id: I03764edadae529db2cc3acf7ecca329570f18659
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-10-15 22:21:14 +08:00
Stephen J Day
76624704c3 Correct unmarshal order for SignedManifest
To ensure that we only unmarshal the verified payload into the contained
manifest, we first copy the entire incoming buffer into Raw and then unmarshal
only the Payload portion of the incoming bytes. If the contents is later
verified, the caller can then be sure that the contents of the Manifest fields
can be trusted.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-14 14:37:34 -07:00
Richard Scothern
4c4112bdcf Merge pull request #1035 from aaronlehmann/host-parameter-alternate
Add http.host parameter
2015-10-13 14:03:48 -07:00
Stephen Day
bfd1f423c5 Merge pull request #1033 from jlhawn/api_spec_update_auth_errors
[api spec] Update authN and authZ errors
2015-10-13 13:52:18 -07:00
Derek McGowan
31a448a628 Update to provide small and clear interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-10-09 16:05:49 -07:00
Tibor Vass
f8c09b6a7d Add a new reference package abstracting repositories, tags and digests
There seems to be a need for a type that represents a way of pointing
to an image, irrespective of the implementation.

This patch defines a Reference interface and provides 3 implementations:
- TagReference: when only a tag is provided
- DigestReference: when a digest (according to the digest package) is
  provided, can include optional tag as well

Validation of references are purely syntactic.

There is also a strong type for tags, analogous to digests, as well
as a strong type for Repository from which clients can access the
hostname alone, or the repository name without the hostname, or both
together via the String() method.

For Repository, the files names.go and names_test.go were moved from
the v2 package.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-09 16:05:34 -07:00
Richard Scothern
515b9596e7 Fix a race condition in pull through cache population by removing the functionality
of readers joining current downloads.  Concurrent requests for the same blob
will not block, but only the first instance will be comitted locally.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-08 16:12:29 -07:00
Matt Moore
74c32a0a51 Allow hostname components in component names.
Fixes https://github.com/docker/distribution/issues/1062

This relaxes the naming restrictions places on Docker images to permit
valid hostnames according to [RFC-2396](https://www.ietf.org/rfc/rfc2396.txt).

It deviates from the RFC in the following ways:
 1) Allow underscores where we allow hyphens (hostnames don't allow
 underscores, which we must for backwards compatibility).
 2) Leave "top-level" name segments unrestricted (domains require an
 alpha character to begin a top-level domain, e.g. "com").
 3) DO NOT allow a trailing dot, as permitted by FQDNs.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-08 11:29:54 -07:00
Richard Scothern
61a4a1362c Merge pull request #950 from jackpgriffin/master
Skip creating swift container if already exists
2015-10-01 10:15:54 -07:00