These tests show issues when with foo/repo and foo-bar/repo are
compared, because - is below / in the ascii table and affects path based
sorting.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
With the current walk algorithm, it was very tricky to make sure that we
didn't regress #1854. Refactoring to start with the hint and work our
way up the tree makes the code clearer and more efficient.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
The start after hint should really match a path that would have been
returned by the walk, instead of being an addition under the walked
directory, otherwise the user has to do processing on the returned value
to use it.
Also, add tests to ensure that the S3 driver correctly uses the hint to
limit the returned results.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
Other storage drivers will only return children and below, s3 should do
the same. The only reason it was returning was because of the addition
of a / to ensure we treat the from as a directory.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
With the current implementation of using ErrSkipDir to end a walk if it
is returned on a file, it is not triggered for the catalog walk. Every
file within the repositories directory is within a directory that starts
with a _, so no files actually get processed, meaning there is no
trigger.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
Storage drivers may be able to take advantage of the hint to start
their walk more efficiently. Start with s3, whose API takes a
start-after parameter. Registries with many repositories can drastically
reduce calls to s3 by telling s3 to only list results lexographically
after the last parameter.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
This puts back the original flow where old clients are fetching manifest
lists schema1 images where we want to try returning some image for the
default architecture. This was incorrectly removed by one of the
previous commits.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
schema1 package was deprecated a while ago so we are removing
any references to it from handlers. in preparation to
removing it from the codebase altogether.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
This commit removes `oss` storage driver from distribution as well as
`alicdn` storage middleware which only works with the `oss` driver.
There are several reasons for it:
* no real-life expertise among the maintainers
* oss is compatible with S3 API operations required by S3 storage driver
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
The Azure tests fail if there is no Azure configuration available,
instead they should be skipped.
Also, one of the Azure tests is wrong and doesn't match the code.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
This commit removes swift storage driver from distribution.
There are several reasons for it:
* no real life expertise among the maintainers
* swift is compatible with S3 API operations required by S3 storage driver
This will also remove depedencies that are also hard to keep up with.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
Client ReadFrom doesn't set Content-Type header leading to server
side implementor to assume it's application/octet-stream. This commit
makes this explicit on the client side.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
schema1 package was deprecated a while ago so we are removing
any references to it from the proxy package in preparation to
removing it from the codebase altogether.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
Redis introduced an Access Control List (ACL) mechanism since version 6.0. This commit implements the necessary changes to support configuring the username for Redis. Users can now define a specific username to authenticate with Redis and enhance security through the ACL feature.
Signed-off-by: chlins <chenyuzh@vmware.com>
Split request and hit metrics into separate metrics, rather than using
labels. This avoids duplication of data and makes metric math easier.
* Count cache errors separately to avoid weird math.
* Hit ratio: `registry_storage_cache_hits_total / registry_storage_cache_requests_total`
* Miss ratio: `1 - (registry_storage_cache_hits_total / registry_storage_cache_requests_total`
* Misses: `registry_storage_cache_requests_total -
registry_storage_cache_hits_total`
Signed-off-by: Ben Kochie <superq@gmail.com>
This enables go build tags so the GCS and OSS driver support is
available in the binary distributed via the image build by Dockerfile.
This led to quite a few fixes in the GCS and OSS packages raised as
warning by golang-ci linter.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
Something seems broken on azure/azure sdk side - it is currently not
possible to copy a blob of type AppendBlob using `CopyFromURL`.
Using the AppendBlob client via NewAppendBlobClient does not work
either.
According to Azure the correct way to do this is by using
StartCopyFromURL. Because this is an async operation, we need to do
polling ourselves. A simple backoff mechanism is used, where during each
iteration, the configured delay is multiplied by the retry number.
Also introduces two new config options for the Azure driver:
copy_status_poll_max_retry, and copy_status_poll_delay.
Signed-off-by: Flavian Missi <fmissi@redhat.com>
