distribution/registry/storage/driver/s3-aws
Milos Gajdos a18cc8a656
S3 driver: Attempt HeadObject on Stat first, fail over to List
Stat always calls ListObjects when stat-ing S3 key.
Unfortauntely ListObjects is not a free call - both in terms of egress
and actual AWS costs (likely because of the egress).

This changes the behaviour of Stat such that we always attempt the
HeadObject call first and only ever fall through to ListObjects if the
HeadObject returns an AWS API error.

Note, that the official docs mention that the only error returned by
HEAD is NoSuchKey; experiments show that this is demonstrably wrong and
the AWS docs are simply outdated at the time of this commit.

HeadObject actually returns the following errors:
* NotFound: if the queried key does not exist
* NotFound: if the queried key contains subkeys i.e. it's a prefix
* BucketRegionError: if the bucket does not exist
* Forbidden: if Head operation is not allows via IAM/ACLs

Co-authored-by: Cory Snider <corhere@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-07-17 10:16:54 +01:00
..
s3.go S3 driver: Attempt HeadObject on Stat first, fail over to List 2024-07-17 10:16:54 +01:00
s3_test.go Allow setting s3 forcepathstyle without regionendpoint 2024-04-08 12:45:26 +02:00
s3_v2_signer.go feat(linter): enable errcheck linter in golangci-lint 2023-11-18 07:19:24 +00:00