Commit graph

4993 commits

Author SHA1 Message Date
James Hewitt
e680634060
Clean up failure cases
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-21 13:00:07 +01:00
James Hewitt
6deeec4699
Clearer comment
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:25 +01:00
James Hewitt
3c8b280408
Use start after hint in walk fallback
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:25 +01:00
James Hewitt
2245836358
The start after hint should match returned path
The start after hint should really match a path that would have been
returned by the walk, instead of being an addition under the walked
directory, otherwise the user has to do processing on the returned value
to use it.

Also, add tests to ensure that the S3 driver correctly uses the hint to
limit the returned results.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:25 +01:00
James Hewitt
fbb9c484d9
Don't return the from of a walk
Other storage drivers will only return children and below, s3 should do
the same. The only reason it was returning was because of the addition
of a / to ensure we treat the from as a directory.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:25 +01:00
James Hewitt
fcc3632801
Switch to using functional options
Keeps the interface cleaner and avoids changes from callers.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:25 +01:00
James Hewitt
36b9d78fed
Use an explicit error to end walk early
With the current implementation of using ErrSkipDir to end a walk if it
is returned on a file, it is not triggered for the catalog walk. Every
file within the repositories directory is within a directory that starts
with a _, so no files actually get processed, meaning there is no
trigger.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:25 +01:00
James Hewitt
48959ebac0
Pass the last paging flag to storage drivers
Storage drivers may be able to take advantage of the hint to start
their walk more efficiently. Start with s3, whose API takes a
start-after parameter. Registries with many repositories can drastically
reduce calls to s3 by telling s3 to only list results lexographically
after the last parameter.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 16:32:23 +01:00
Milos Gajdos
ed256e07f1
Merge pull request #4004 from Jamstah/revert-3902
Revert 3902
2023-08-18 16:26:39 +01:00
Milos Gajdos
a45c5785ab
Merge pull request #3998 from milosgajdos/update-docs
Small update of API docs
2023-08-18 16:24:55 +01:00
Milos Gajdos
9139f52ddb
Merge pull request #4003 from DavidSpek/remove-contrib-folder
remove contrib folder
2023-08-18 14:57:33 +01:00
Milos Gajdos
bfe4a37b7d
Merge pull request #3997 from milosgajdos/module-grouping
Regroup direct and indirect dependencies
2023-08-18 14:56:00 +01:00
James Hewitt
37a213dc4b
Revert "optimize catalog last param"
This reverts commit 65f4ce4d93.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:37 +01:00
James Hewitt
8fd504debe
Revert "Rename catalog funcs and update their godocs."
This reverts commit 230cc72a8b.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:28 +01:00
James Hewitt
ad11105052
Revert "removed redundant check"
This reverts commit 0f846853fe.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:19 +01:00
James Hewitt
8e4a8517c5
Revert "fix: resolve most comments"
This reverts commit 6a5846b32e.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:06 +01:00
David van der Spek
6fea54890d
remove contrib folder
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-18 12:58:50 +02:00
Milos Gajdos
279fa01b2a
Small update of API docs
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-18 08:47:04 +01:00
Milos Gajdos
c0a15e6448
Regroup direct and indirect dependencies
For some reason we let these be ungrouped and mixed all over the place.
This commit groups direct and indirect Go module dependencies.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-18 08:28:26 +01:00
Milos Gajdos
97957b12b1
Merge pull request #3623 from TaylorKanper/security-compilation
Add some secure compilation options, especially PIE and RELRO.
2023-08-18 08:26:15 +01:00
Milos Gajdos
d302c58264
Merge pull request #3995 from waynr/waynr-patch-1
Remove waynr from MAINTAINERS
2023-08-18 08:18:35 +01:00
Milos Gajdos
bf3c2df6b2
Merge pull request #3902 from pluralsh/catalog-opti-fix-rebase
Optimise catalog function rebase of #3145
2023-08-18 08:11:29 +01:00
Milos Gajdos
ac32466188
Merge pull request #3993 from DavidSpek/update-golang-lru
Update github.com/hashicorp/golang-lru to v2
2023-08-18 08:09:11 +01:00
Hayley Swimelar
ac7a334069
Merge pull request #3987 from milosgajdos/remove-schema1-handlers
Remove references to schema1 pacakge from handlers
2023-08-17 14:18:48 -07:00
Milos Gajdos
40c56bf1b6
Keep returning image for default arch to old clients fetching lists
This puts back the original flow where old clients are fetching manifest
lists schema1 images where we want to try returning some image for the
default architecture. This was incorrectly removed by one of the
previous commits.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-17 20:57:02 +01:00
Hayley Swimelar
61280d107f
Merge pull request #3994 from DavidSpek/james-reviewer
Propose James Hewitt as a new reviewer
2023-08-17 11:36:54 -07:00
Hayley Swimelar
d7241d788b
Merge pull request #3990 from DavidSpek/update-aws-sdk
fix(deps): update module github.com/aws/aws-sdk-go to v1.44.325
2023-08-17 11:36:27 -07:00
wayne
f48e0ecd2a
Remove waynr from MAINTAINERS
Signed-off-by: wayne <wayne.warren.s@gmail.com>
2023-08-17 09:32:28 -06:00
David van der Spek
e51cfa6605
Add James Hewitt as a reviewer
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-17 16:47:54 +02:00
David van der Spek
0f006548a1
update golang-lru to v2
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-17 13:41:54 +02:00
David van der Spek
9d862f0982
fix(deps): update module github.com/aws/aws-sdk-go to v1.44.325
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-17 12:02:28 +02:00
Milos Gajdos
f517191da1
Add small update to api tests
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-17 09:37:26 +01:00
Milos Gajdos
7e39a7c6dc
Remove references to schema1 pacakge from handlers
schema1 package was deprecated a while ago so we are removing
any references to it from handlers. in preparation to
removing it from the codebase altogether.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-17 08:35:54 +01:00
Hayley Swimelar
6ccc551f96
Merge pull request #3985 from milosgajdos/remove-oss-alibaba
Remove oss storage driver and alicdn storage driver middleware
2023-08-16 08:20:46 -07:00
Milos Gajdos
3f1859af26
Remove oss storage driver and alicdn storage driver middleware
This commit removes `oss` storage driver from distribution as well as
`alicdn` storage middleware which only works with the `oss` driver.

There are several reasons for it:
* no real-life expertise among the maintainers
* oss is compatible with S3 API operations required by S3 storage driver

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-16 08:39:20 +01:00
Milos Gajdos
65b57464f9
Merge pull request #3982 from milosgajdos/remove-swift-storage-driver
Remove SWIFT storage driver
2023-08-16 07:47:42 +01:00
Milos Gajdos
fff194fc46
Merge pull request #3983 from Jamstah/azure-tests
Fix Azure tests
2023-08-15 18:31:05 +01:00
James Hewitt
46ff5f8528
Fix Azure tests
The Azure tests fail if there is no Azure configuration available,
instead they should be skipped.

Also, one of the Azure tests is wrong and doesn't match the code.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-15 16:46:36 +01:00
Milos Gajdos
2918c3244d
Merge pull request #3981 from milosgajdos/set-content-type-client-readfrom
Set Content-Type header in registry client ReadFrom
2023-08-15 15:21:20 +01:00
Milos Gajdos
72dc264241
Merge pull request #3511 from andriisoldatenko/add-support-env-vars-slices
Added support for configuring array values with environment variables
2023-08-15 14:14:30 +01:00
Milos Gajdos
1813dd13b4
Merge pull request #3976 from milosgajdos/remove-schema1-notifcations
Remove references to schema1 pacakge from notifications package
2023-08-15 10:18:32 +01:00
Milos Gajdos
c6b9944ab1
Remove SWIFT storage driver
This commit removes swift storage driver from distribution.
There are several reasons for it:
* no real life expertise among the maintainers
* swift is compatible with S3 API operations required by S3 storage driver

This will also remove depedencies that are also hard to keep up with.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-15 09:14:11 +01:00
Milos Gajdos
24de708d22
Set Content-Type header in registry client ReadFrom
Client ReadFrom doesn't set Content-Type header leading to server
side implementor to assume it's application/octet-stream. This commit
makes this explicit on the client side.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-15 08:46:48 +01:00
Milos Gajdos
ff46bf1c41
Merge pull request #2897 from SuperQ/cache_metrics
Cleanup storage cache metrics
2023-08-15 07:44:14 +01:00
Hayley Swimelar
f1529a7782
Merge pull request #3977 from milosgajdos/remove-schema1-proxystore
Remove references to schema1 pacakge from proxy package
2023-08-14 16:24:45 -07:00
Milos Gajdos
0e18af15f8
Merge pull request #3741 from sashashura/patch-1
GitHub Workflows security hardening
2023-08-14 19:21:28 +01:00
Wang Yan
5410c55820
Merge pull request #3979 from milosgajdos/user-agent
Drop docker prefix from storage driver API user agent
2023-08-15 02:02:47 +08:00
Milos Gajdos
745a29a0e2
Merge pull request #3980 from milosgajdos/new-reviewer
Propose David van der Spek as a new reviewer
2023-08-14 14:00:34 +01:00
Milos Gajdos
c1147cc11f
Add David van der Spek as a new reviewer
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-14 09:16:24 +01:00
Milos Gajdos
02a92efba8
Drop docker prefix from storage driver API user agent
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-13 22:27:55 +01:00