distribution/registry/storage
Milos Gajdos b888b14b39
Optimise push in S3 driver
This commit cleans up and attempts to optimise the performance of image push in S3 driver.
There are 2 main changes:
* we refactor the S3 driver Writer where instead of using separate bytes
  slices for ready and pending parts which get constantly appended data
  into them causing unnecessary allocations we use optimised bytes
  buffers; we make sure these are used efficiently when written to.
* we introduce a memory pool that is used for allocating the byte
  buffers introduced above

These changes should alleviate high memory pressure on the push path to S3.

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-27 21:33:22 +01:00
..
cache deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
driver Optimise push in S3 driver 2023-09-27 21:33:22 +01:00
blob_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
blobserver.go go.mod: change imports to github.com/distribution/distribution/v3 2021-02-08 18:30:46 +01:00
blobstore.go deprecate ReadSeekCloser in favor of io.ReadSeekCloser 2022-11-10 23:10:32 +01:00
blobwriter.go Update Azure SDK and support additional authentication schemes 2023-04-25 17:23:20 +00:00
blobwriter_nonresumable.go bump up golang v1.17 2021-10-27 15:51:30 +08:00
blobwriter_resumable.go format code with gofumpt 2022-11-03 22:48:20 +01:00
catalog.go Merge pull request #4031 from thaJeztah/migrate_reference 2023-08-31 15:20:28 +01:00
catalog_test.go Merge pull request #4031 from thaJeztah/migrate_reference 2023-08-31 15:20:28 +01:00
doc.go Move storage package under registry package 2015-02-11 12:43:04 -08:00
error.go Remove old walk function 2018-01-18 13:06:07 -08:00
filereader.go replace deprecated io/ioutil 2022-11-04 23:47:52 +01:00
filereader_test.go Handle rand deprecations in go 1.20 2023-08-28 09:33:12 +01:00
garbagecollect.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
garbagecollect_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
io.go replace deprecated io/ioutil 2022-11-04 23:47:52 +01:00
linkedblobstore.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
linkedblobstore_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
manifestlisthandler.go Fix Go Idioms 2023-04-21 15:22:37 +01:00
manifeststore.go feat(storage)!: remove schema1 except manifeststore_test 2023-08-28 12:44:46 +02:00
manifeststore_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
ociindexhandler.go Split OCI Image Index from Docker Manifest List 2023-03-31 11:43:21 +01:00
ocimanifesthandler.go format code with gofumpt 2022-11-03 22:48:20 +01:00
ocimanifesthandler_test.go registry: verify digest and check blob presence when put manifest 2021-04-16 16:11:52 +08:00
paths.go remove not exist function name in comment 2023-09-21 16:53:59 +08:00
paths_test.go format code with gofumpt 2022-11-03 22:48:20 +01:00
purgeuploads.go format code with gofumpt 2022-11-03 22:48:20 +01:00
purgeuploads_test.go format code with gofumpt 2022-11-03 22:48:20 +01:00
registry.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
schema2manifesthandler.go feat(storage)!: remove schema1 except manifeststore_test 2023-08-28 12:44:46 +02:00
schema2manifesthandler_test.go registry: verify digest and check blob presence when put manifest 2021-04-16 16:11:52 +08:00
tagstore.go Enable prealloc linter 2023-09-03 22:41:51 +01:00
tagstore_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
vacuum.go go.mod: change imports to github.com/distribution/distribution/v3 2021-02-08 18:30:46 +01:00