distribution/registry/storage
Milos Gajdos a18cc8a656
S3 driver: Attempt HeadObject on Stat first, fail over to List
Stat always calls ListObjects when stat-ing S3 key.
Unfortauntely ListObjects is not a free call - both in terms of egress
and actual AWS costs (likely because of the egress).

This changes the behaviour of Stat such that we always attempt the
HeadObject call first and only ever fall through to ListObjects if the
HeadObject returns an AWS API error.

Note, that the official docs mention that the only error returned by
HEAD is NoSuchKey; experiments show that this is demonstrably wrong and
the AWS docs are simply outdated at the time of this commit.

HeadObject actually returns the following errors:
* NotFound: if the queried key does not exist
* NotFound: if the queried key contains subkeys i.e. it's a prefix
* BucketRegionError: if the bucket does not exist
* Forbidden: if Head operation is not allows via IAM/ACLs

Co-authored-by: Cory Snider <corhere@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-07-17 10:16:54 +01:00
..
cache Replace custom Redis config struct with go-redis UniversalOptions (adds sentinel & cluster support) (#4306) 2024-07-04 16:00:37 +01:00
driver S3 driver: Attempt HeadObject on Stat first, fail over to List 2024-07-17 10:16:54 +01:00
blob_test.go chore: fix some typos in comments 2024-04-23 12:04:03 +08:00
blobserver.go storage/driver: replace URLFor method 2023-10-27 10:58:37 -04:00
blobstore.go Move context package internal 2023-10-27 10:58:37 -04:00
blobwriter.go chore: remove repetitive words in comments 2024-03-27 17:34:22 +08:00
blobwriter_nonresumable.go Move context package internal 2023-10-27 10:58:37 -04:00
blobwriter_resumable.go format code with gofumpt 2022-11-03 22:48:20 +01:00
catalog.go Merge pull request #4031 from thaJeztah/migrate_reference 2023-08-31 15:20:28 +01:00
catalog_test.go Merge pull request #4031 from thaJeztah/migrate_reference 2023-08-31 15:20:28 +01:00
doc.go Move storage package under registry package 2015-02-11 12:43:04 -08:00
error.go Remove old walk function 2018-01-18 13:06:07 -08:00
filereader.go cleanup: make blob sizes easier to understand 2023-10-31 14:53:02 -07:00
filereader_test.go feat(linter): enable errcheck linter in golangci-lint 2023-11-18 07:19:24 +00:00
garbagecollect.go remove layer's link file by gc 2024-07-03 00:16:11 +08:00
garbagecollect_test.go remove layer's link file by gc 2024-07-03 00:16:11 +08:00
io.go cleanup: make blob sizes easier to understand 2023-10-31 14:53:02 -07:00
linkedblobstore.go Move context package internal 2023-10-27 10:58:37 -04:00
linkedblobstore_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
manifestlisthandler.go Enable configuration of index dependency validation 2024-05-28 09:56:14 +01:00
manifeststore.go Move context package internal 2023-10-27 10:58:37 -04:00
manifeststore_test.go Remove ManifestBuilder interface 2024-07-16 11:16:06 +02:00
ociindexhandler.go Move context package internal 2023-10-27 10:58:37 -04:00
ocimanifesthandler.go vendor: github.com/opencontainers/image-spec v1.1.0 2024-07-10 14:58:09 -05:00
ocimanifesthandler_test.go vendor: github.com/opencontainers/image-spec v1.1.0 2024-07-10 14:58:09 -05:00
paths.go add repositoriesRootPathSpec in pathFor documentation 2023-09-26 15:07:49 +08:00
paths_test.go format code with gofumpt 2022-11-03 22:48:20 +01:00
purgeuploads.go Set readStartAtFile context aware for purge uploads 2024-05-02 11:06:39 +02:00
purgeuploads_test.go feat(linter): enable errcheck linter in golangci-lint 2023-11-18 07:19:24 +00:00
registry.go Enable configuration of index dependency validation 2024-05-28 09:56:14 +01:00
schema2manifesthandler.go Move context package internal 2023-10-27 10:58:37 -04:00
schema2manifesthandler_test.go Enable configuration of index dependency validation 2024-05-28 09:56:14 +01:00
tagstore.go add concurrency limits for tag lookup and untag 2024-04-26 22:32:21 +08:00
tagstore_test.go deprecate reference package, migrate to github.com/distribution/reference 2023-08-31 15:47:06 +02:00
vacuum.go remove layer's link file by gc 2024-07-03 00:16:11 +08:00