This is an edge case when we are trying to upload an empty chunk of data using
a MultiPart upload. As a result we are trying to complete the MultipartUpload
with an empty slice of `completedUploadedParts` which will always lead to 400
being returned from S3 See: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#CompletedMultipartUpload
Solution: we upload an empty i.e. 0 byte part as a single part and then append it
to the completedUploadedParts slice used to complete the Multipart upload.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
gofumpt (https://github.com/mvdan/gofumpt) provides a supserset of `gofmt` / `go fmt`,
and addresses various formatting issues that linters may be checking for.
We can consider enabling the `gofumpt` linter to verify the formatting in CI, although
not every developer may have it installed, so for now this runs it once to get formatting
in shape.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Instead of first collecting all keys and then batch deleting them,
we will do the incremental delete _online_ per max allowed batch.
Doing this prevents frequent allocations for large S3 keyspaces
and OOM-kills that might happen as a result of those.
This commit introduces storagedriver.Errors type that allows to return
multierrors as a single error from any storage driver implementation.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
1, return the right upload offset for client when asks.
2, do not call ResumeBlobUpload on getting status.
3, return 416 rather than 404 on failed to patch chunk blob.
4, add the missing upload close
Signed-off-by: Wang Yan <wangyan@vmware.com>
Instead of letting the cache grow without bound, use a LRU to impose a
size limit.
The limit is configurable through a new `blobdescriptorsize` config key.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
These tests don't validate if options are valid for the storage-driver,
nor do they test if the storage-driver itself is valid. However, the tests
were using actual values (such as s3) and options (such as "region") which
may lead to the conclusion that it's also testing validity of those values.
This patch replaces the test-values with non-existing driver-names and
options to make it more clear these are fake values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
This simple change mainly affects the distribution client. By respecting
the context the caller passes in, timeouts and cancellations will work
as expected. Also, transports which rely on the context (such as tracing
transports that retrieve a span from the context) will work properly.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
The example shows a `GET` request to `/callback`, but it looks like it's
intending to show the `POST` request made by the registry to a
notificaitons endpoint. Unless I'm missing something, no `GET` request
should be involved.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
(*App).context, called in the HTTP handler on each request, creates a
URLBuilder, which involves calling Router(). This shows up in profiles a
hot spot because it involves compiling the regexps which define all the
routes. For efficiency, cache the router and return the same object each
time.
It appears to be safe to reuse the router because GetRoute is the only
method ever called on the returned router object.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
Without this, the log message for the user indicates a problem with the
yaml file, so identifying the actual error is hard. This change fixes
the output so that the incorrect environment variable is easy to spot.
Fixes#3653
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
As the project has transitioned from Docker to CNCF, clean up the
contributing.md to reflect that:
- CNCF slack
- CNCF maintainers
- Use with other container tools
Fixes#3550
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
