distribution/registry/storage/driver
Milos Gajdos ebc4234fd5
Delete S3 keys incrementally in batches
Instead of first collecting all keys and then batch deleting them,
we will do the incremental delete _online_ per max allowed batch.
Doing this prevents frequent allocations for large S3 keyspaces
and OOM-kills that might happen as a result of those.

This commit introduces storagedriver.Errors type that allows to return
multierrors as a single error from any storage driver implementation.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2022-10-30 19:10:24 +00:00
..
azure storagedriver/s3: Optimized Walk implementation + bugfix 2021-08-16 16:07:25 -04:00
base go.mod: change imports to github.com/distribution/distribution/v3 2021-02-08 18:30:46 +01:00
factory go.mod: change imports to github.com/distribution/distribution/v3 2021-02-08 18:30:46 +01:00
filesystem storagedriver/s3: Optimized Walk implementation + bugfix 2021-08-16 16:07:25 -04:00
gcs bump up golang v1.17 2021-10-27 15:51:30 +08:00
inmemory Fix panic in inmemory driver 2022-03-27 19:38:07 +08:00
middleware updatefrequency should not be saved into duration 2021-09-21 11:12:34 +02:00
oss bump up golang v1.17 2021-10-27 15:51:30 +08:00
s3-aws Delete S3 keys incrementally in batches 2022-10-30 19:10:24 +00:00
swift storagedriver/s3: Optimized Walk implementation + bugfix 2021-08-16 16:07:25 -04:00
testdriver go.mod: change imports to github.com/distribution/distribution/v3 2021-02-08 18:30:46 +01:00
testsuites go.mod: change imports to github.com/distribution/distribution/v3 2021-02-08 18:30:46 +01:00
fileinfo.go Move storagedriver package to registry/storage/driver 2015-02-11 12:43:04 -08:00
storagedriver.go Delete S3 keys incrementally in batches 2022-10-30 19:10:24 +00:00
walk.go storagedriver/s3: Optimized Walk implementation + bugfix 2021-08-16 16:07:25 -04:00
walk_test.go storagedriver/s3: Optimized Walk implementation + bugfix 2021-08-16 16:07:25 -04:00