See #2077 for background.
The PR #1438 which was not reviewed by azure folks basically introduced
a race condition around uploads to the same blob by multiple clients
concurrently as it used the "writer" type for PutContent(), introduced in #1438.
This does chunked upload of blobs using "AppendBlob" type, which was not atomic.
Usage of "writer" type and thus AppendBlobs on metadata files is currently not
concurrency-safe and generally, they are not the right type of blob for the job.
This patch fixes PutContent() to use the atomic upload operation that works
for uploads smaller than 64 MB and creates blobs with "BlockBlob" type. To be
backwards compatible, we query the type of the blob first and if it is not
a "BlockBlob" we delete the blob first before doing an atomic PUT. This
creates a small inconsistency/race window "only once". Once the blob is made
"BlockBlob", it is overwritten with a single PUT atomicallly next time.
Therefore, going forward, PutContent() will be producing BlockBlobs and it
will silently migrate the AppendBlobs introduced in #1438 to BlockBlobs with
this patch.
Tested with existing code side by side, both registries with and without this
patch work fine without breaking each other. So this should be good from a
backwards/forward compatiblity perspective, with a cost of doing an extra
HEAD checking the blob type.
Fixes#2077.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Use whitelist of allowed repository classes to enforce.
By default all repository classes are allowed.
Add authorized resources to context after authorization.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Update grammar to support a resource class. Add
example for plugin repository class.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Golint now checks for new lines at the end of go error strings,
remove these unneeded new lines.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add a simple helper to create name only references without having
to first get the name as a string and reparse. This method does
not fail so ignoring or checking error to simply trim reference
is not required.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Allow clients to handle errors being set in the WWW-Authenticate
rather than in the body. The WWW-Authenticate errors give a
more precise error describing what is needed to authorize
with the server.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Split challenges into its own package. Avoids possible
import cycle with challenges from client.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When WithDigest is called on a reference that has a tag, it should
preserve the tag.
When WithTag is called on a reference that has digest, it should
preserve the digest.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Updating to a recent version of Azure Storage SDK to be
able to patch some memory leaks through configurable HTTP client
changes which were made possible by recent patches to it.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
The current code determines the header order for the
"string-to-sign" payload by sorting on the concatenation
of headers and values, whereas it should only happen on the
key.
During multipart uploads, since `x-amz-copy-source-range` and
`x-amz-copy-source` headers are present, V2 signatures fail to
validate since header order is swapped.
This patch reverts to the expected behavior.
Signed-off-by: Pierre-Yves Ritschard <pyr@spootnik.org>
