Commit graph

2323 commits

Author SHA1 Message Date
Ahmet Alp Balkan
78d0660319
azure: fix race condition in PutContent()
See #2077 for background.

The PR #1438 which was not reviewed by azure folks basically introduced
a race condition around uploads to the same blob by multiple clients
concurrently as it used the "writer" type for PutContent(), introduced in #1438.
This does chunked upload of blobs using "AppendBlob" type, which was not atomic.

Usage of "writer" type and thus AppendBlobs on metadata files is currently not
concurrency-safe and generally, they are not the right type of blob for the job.

This patch fixes PutContent() to use the atomic upload operation that works
for uploads smaller than 64 MB and creates blobs with "BlockBlob" type. To be
backwards compatible, we query the type of the blob first and if it is not
a "BlockBlob" we delete the blob first before doing an atomic PUT. This
creates a small inconsistency/race window "only once". Once the blob is made
"BlockBlob", it is overwritten with a single PUT atomicallly next time.

Therefore, going forward, PutContent() will be producing BlockBlobs and it
will silently migrate the AppendBlobs introduced in #1438 to BlockBlobs with
this patch.

Tested with existing code side by side, both registries with and without this
patch work fine without breaking each other. So this should be good from a
backwards/forward compatiblity perspective, with a cost of doing an extra
HEAD checking the blob type.

Fixes #2077.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-11-30 12:40:43 -08:00
Derek McGowan
c599955707 Merge pull request #2087 from aaronlehmann/missing-digest-panic
schema1: Validate descriptors in AppendReference
2016-11-30 10:26:55 -08:00
Aaron Lehmann
ec5fa1f9d6 schema1: Validate descriptors in AppendReference
If the digest is malformed, it will cause a panic when building the
manifest.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-11-29 17:22:07 -08:00
Derek McGowan
bf27f260cd Merge pull request #2083 from mstanleyjones/fix_doubled_words
Fix doubled words
2016-11-28 13:03:35 -08:00
Misty Stanley-Jones
8e703afdc6 Fix doubled words
Signed-off-by: Misty Stanley-Jones <misty@docker.com>
2016-11-28 11:29:11 -08:00
Derek McGowan
acad7c43a8
Enable dependency validation
Re-enable dependency validation using vndr instead of godep

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-23 15:45:04 -08:00
Derek McGowan
a685e3fc98
Replace godep with vndr
Vndr has a simpler configuration and allows pointing to forked
packages. Additionally other docker projects are now using
vndr making vendoring in distribution more consistent.

Updates letsencrypt to use fork.
No longer uses sub-vendored packages.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-23 15:07:06 -08:00
Derek McGowan
38fbd03266 Merge pull request #1840 from dmcgowan/support-1.12-integration-tests
Integration tests support for docker 1.12
2016-11-23 14:19:09 -08:00
Derek McGowan
8d096a4f42 Merge pull request #1969 from TrumanLing/master
api.md:fix disordered placement of section "Deleting a Layer"
2016-11-23 13:59:17 -08:00
Derek McGowan
f2620ca7c1
Bump call to use 1.12.3
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-23 13:33:19 -08:00
Derek McGowan
00a505bc42
Support docker 1.12
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-23 13:33:19 -08:00
Derek McGowan
8f9abbd27f Merge pull request #2079 from dmcgowan/update-swift-vendor
Update swift vendor for upstreamed change
2016-11-23 13:19:53 -08:00
Derek McGowan
e9684b126e
Update swift vendor for upstreamed change
Gets rid of deviation between hash and upstream.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-23 11:45:56 -08:00
Derek McGowan
a6bf3dd064 Merge pull request #2067 from dmcgowan/add-repo-class
Add support for repository class
2016-11-21 20:23:04 -08:00
Derek McGowan
e02278f22a
Update registry server to support repository class
Use whitelist of allowed repository classes to enforce.
By default all repository classes are allowed.

Add authorized resources to context after authorization.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-21 16:36:36 -08:00
Derek McGowan
61e65ecd9d
Update contrib token server to support repository class
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-21 13:32:12 -08:00
Derek McGowan
01509db714
Add class to repository scope
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-21 13:32:12 -08:00
Derek McGowan
e10634f8c3
Update scope specification for resource class
Update grammar to support a resource class. Add
example for plugin repository class.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-21 13:32:12 -08:00
Kira
4accc8f2be filter listResponse.Contents in driver.List()
Signed-off-by: Kira <me@imkira.com>
2016-11-17 10:38:56 +08:00
Derek McGowan
7694c31658 Merge pull request #2063 from dmcgowan/fix-golint-error-newline
Remove newlines from end of error strings
2016-11-14 15:22:52 -08:00
Derek McGowan
8b71d39c34
Remove newlines from end of error strings
Golint now checks for new lines at the end of go error strings,
remove these unneeded new lines.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-14 14:03:01 -08:00
Derek McGowan
d22e09a668 Merge pull request #2062 from nishanttotla/with-default-tag
Adding utility functions - IsNameOnly and EnsureTagged
2016-11-11 15:34:52 -08:00
Nishant Totla
2caeb6192a
Adding utility functions - IsNameOnly and WithDefaultTag
Signed-off-by: Nishant Totla <nishanttotla@gmail.com>
2016-11-11 15:17:16 -08:00
Richard Scothern
e249b61e90 Merge pull request #2059 from RichardScothern/2.6-release-notes
Release notes for v2.6.0-rc1
2016-11-11 12:03:20 -08:00
Richard Scothern
ff9533dd6f Release notes for v2.6.0-rc1
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-11-11 11:54:35 -08:00
Richard Scothern
cb98aa744b Merge pull request #2060 from RichardScothern/release-prep
Release preperation: update AUTHORS and version file
2016-11-11 10:28:15 -08:00
Richard Scothern
cb2c78ce9b Release preperation: update AUTHORS and version file
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-11-11 09:23:05 -08:00
Derek McGowan
9cc7c94e4e Merge pull request #2052 from RichardScothern/fix-links
Fix storage driver links
2016-11-11 09:05:44 -08:00
Richard Scothern
6e62b39842 Merge pull request #2036 from pyr/fix/sort-v2-headers
v2 signer: correctly sort headers
2016-11-10 15:31:24 -08:00
Derek McGowan
8016d2d890 Merge pull request #2053 from dmcgowan/add-trim-name
Add trim name method helper to create name only references
2016-11-10 15:03:34 -08:00
Derek McGowan
31a91e419c
Add trim name method helper to create name only references
Add a simple helper to create name only references without having
to first get the name as a string and reparse. This method does
not fail so ignoring or checking error to simply trim reference
is not required.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-10 14:46:04 -08:00
Richard Scothern
4d65dd513e Merge pull request #2038 from spacexnice/master
fix: oss driver would get connection reset by peer when upload large image layer.
2016-11-10 14:44:32 -08:00
Richard Scothern
09b2d06d5d Fix storage driver links
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-11-10 14:28:28 -08:00
Stephen Day
4ad885a672 Merge pull request #2051 from vieux/update_plugin_media_typev1
switch to v1 and remove .image
2016-11-10 11:49:48 -08:00
Victor Vieux
b82a2efc33 switch to v1 and remove .image
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-11-10 11:39:15 -08:00
Derek McGowan
ac0493db12 Merge pull request #2040 from RichardScothern/release-checklist
Document registry release process
2016-11-10 10:33:30 -08:00
Derek McGowan
717134d7c2 Merge pull request #2048 from dmcgowan/use-api-errors
Update oauth errors to use api errors
2016-11-09 20:27:40 -08:00
Derek McGowan
051801f1d0
Update oauth errors to use api errors
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-09 19:29:18 -08:00
Stephen Day
26c9a77535 Merge pull request #2039 from vdemeester/add-match-support-to-reference
Add a new Match method to the reference package
2016-11-09 18:49:21 -08:00
Stephen Day
e04e6ddd2c Merge pull request #2047 from dmcgowan/fix-authorization-error
Add OAuth error for client
2016-11-09 18:03:38 -08:00
Derek McGowan
16396a7a80
Add OAuth error for client
Allow clients to handle errors being set in the WWW-Authenticate
rather than in the body. The WWW-Authenticate errors give a
more precise error describing what is needed to authorize
with the server.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-09 15:27:40 -08:00
Derek McGowan
a1a73884f9
Refactor authorization challenges to its own package
Split challenges into its own package. Avoids possible
import cycle with challenges from client.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-09 15:27:40 -08:00
Vincent Demeester
353e3a4c9d
Add a new Match method to the reference package
The Match method allows to see if a reference matches a specified
patterns.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-11-09 22:04:52 +01:00
Derek McGowan
02f4195788 Merge pull request #2044 from aaronlehmann/preserve-tag-and-digest
digest: Preserve tag and digest in With* functions
2016-11-08 15:10:22 -08:00
Aaron Lehmann
bc1e7aa603 digest: Preserve tag and digest in With* functions
When WithDigest is called on a reference that has a tag, it should
preserve the tag.

When WithTag is called on a reference that has digest, it should
preserve the digest.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-11-08 14:47:33 -08:00
yaoyao.xyy
a4a227e351 oss native large file copy consume too much time which will eventually lead to client timeout because of no data transmit throughout native copy. change maxCopySize to 128MB, ensure only sm all medium size file use oss native copy to avoid connection reset by peer. And fix Move function with CopyLargeFileInParallel to optimize oss upload copy
Signed-off-by: yaoyao.xyy <yaoyao.xyy@alibaba-inc.com>
2016-11-08 12:14:13 +08:00
Richard Scothern
1e501d0bd7 Document registry release process
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-11-07 12:48:55 -08:00
Derek McGowan
a2611c7520 Merge pull request #2027 from ahmetalpbalkan/pr-azure-memleak2
Update vendored azure-sdk-for-go
2016-11-04 10:08:40 -07:00
Ahmet Alp Balkan
2ab25288a2
Update vendored azure-sdk-for-go
Updating to a recent version of Azure Storage SDK to be
able to patch some memory leaks through configurable HTTP client
changes which were made possible by recent patches to it.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-11-03 13:24:57 -07:00
Derek McGowan
fbe6e8d212 Merge pull request #2008 from miminar/honor-x-forwarded-port
Honor X-Forwarded-Port header
2016-11-02 12:56:17 -07:00