78d0660319
See #2077 for background. The PR #1438 which was not reviewed by azure folks basically introduced a race condition around uploads to the same blob by multiple clients concurrently as it used the "writer" type for PutContent(), introduced in #1438. This does chunked upload of blobs using "AppendBlob" type, which was not atomic. Usage of "writer" type and thus AppendBlobs on metadata files is currently not concurrency-safe and generally, they are not the right type of blob for the job. This patch fixes PutContent() to use the atomic upload operation that works for uploads smaller than 64 MB and creates blobs with "BlockBlob" type. To be backwards compatible, we query the type of the blob first and if it is not a "BlockBlob" we delete the blob first before doing an atomic PUT. This creates a small inconsistency/race window "only once". Once the blob is made "BlockBlob", it is overwritten with a single PUT atomicallly next time. Therefore, going forward, PutContent() will be producing BlockBlobs and it will silently migrate the AppendBlobs introduced in #1438 to BlockBlobs with this patch. Tested with existing code side by side, both registries with and without this patch work fine without breaking each other. So this should be good from a backwards/forward compatiblity perspective, with a cost of doing an extra HEAD checking the blob type. Fixes #2077. Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com> |
||
|---|---|---|
| .. | ||
| azure | ||
| base | ||
| factory | ||
| filesystem | ||
| gcs | ||
| inmemory | ||
| middleware | ||
| oss | ||
| s3-aws | ||
| s3-goamz | ||
| swift | ||
| testdriver | ||
| testsuites | ||
| fileinfo.go | ||
| storagedriver.go | ||