Commit graph

1268 commits

Author SHA1 Message Date
Milos Gajdos
6baa31a273
Remove duplicate code that instruments Redis OTLP
This was somehow overlooked in https://github.com/distribution/distribution/pull/4019

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-04 17:52:38 +01:00
Milos Gajdos
9790bc806c
Merge pull request #4037 from milosgajdos/enable-prealloc
Enable prealloc linter
2023-09-04 16:57:29 +01:00
Milos Gajdos
823cd4a370
Add a comment why prealloc linter is disabled when configuring endpoints
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 23:33:20 +01:00
Milos Gajdos
1089800643
Preallocate created slice in S3 tests
In case drvr.PutContent fails and returns error we'd have
some extra memory allocated, though in this case
(test with known size of the slice being iterated), that's fine.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 23:26:32 +01:00
Milos Gajdos
a9d31ec7b9
Avoid unnecessary type assertion in mfs driver
We already make sure the node in *dir

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 23:23:25 +01:00
Milos Gajdos
59fd8656ac
Enable prealloc linter
This will give us nice little performance gains in some code paths.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 22:41:51 +01:00
Milos Gajdos
dcdd8bb740
Propagate storage driver context to S3 API calls
Only some of the S3 storage driver calls were propagating context to the
S3 API calls. This commit updates the S3 storage drivers so the context
is propagated to all the S3 API calls.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 21:54:54 +01:00
Milos Gajdos
db4bd9933e
Merge pull request #4031 from thaJeztah/migrate_reference
deprecate reference package, migrate to github.com/distribution/reference
2023-08-31 15:20:28 +01:00
Sebastiaan van Stijn
152af63ec5
deprecate reference package, migrate to github.com/distribution/reference
This integrates the new module, which was extracted from this repository
at commit b9b19409cf458dcb9e1253ff44ba75bd0620faa6;

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    # create a temporary clone of docker
    cd ~/Projects
    git clone https://github.com/distribution/distribution.git reference
    cd reference

    # commit taken from
    git rev-parse --verify HEAD
    b9b19409cf

    # remove all code, except for general files, 'reference/', and rename to /
    git filter-repo \
      --path .github/workflows/codeql-analysis.yml \
      --path .github/workflows/fossa.yml \
      --path .golangci.yml \
      --path distribution-logo.svg \
      --path CODE-OF-CONDUCT.md \
      --path CONTRIBUTING.md \
      --path GOVERNANCE.md \
      --path README.md \
      --path LICENSE \
      --path MAINTAINERS \
      --path-glob 'reference/*.*' \
      --path-rename reference/:

    # initialize go.mod
    go mod init github.com/distribution/reference
    go mod tidy -go=1.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:47:06 +02:00
Milos Gajdos
17552d864d
Merge pull request #3685 from Jamstah/aws-paging
Work with the storage driver to minimise work when paging
2023-08-31 08:27:26 +01:00
bin liu
2513dd1f96 fix typos in registry/storage/paths.go
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-30 22:36:14 +08:00
Milos Gajdos
b9b19409cf
Merge pull request #4028 from liubin/delete-duplicated-code 2023-08-30 07:45:51 +01:00
bin liu
eda5fe2d67 remove duplicated code
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-30 07:56:56 +08:00
James Hewitt
e22f7cbc73
Pass the last paging flag to storage drivers
Storage drivers may be able to take advantage of the hint to start
their walk more efficiently.

For S3: The API takes a start-after parameter. Registries with many
repositories can drastically reduce calls to s3 by telling s3 to only
list results lexographically after the last parameter.

For the fallback: We can start deeper in the tree and avoid statting
the files and directories before the hint in a walk. For a filesystem
this improves performance a little, but many of the API based drivers
are currently treated like a filesystem, so this drastically improves
the performance of GCP and Azure blob.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-29 11:27:42 +01:00
Milos Gajdos
3a44c2e10e
Merge pull request #4022 from Jamstah/catalog-optimisation
Don't make a new buffer for catalog listing
2023-08-29 11:24:39 +01:00
James Hewitt
a41613ba3a
Don't make a new buffer for catalog listing
We are given a slice to fill, write catalog entries directly to the
slice until it is full.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-29 11:11:13 +01:00
bin liu
b889cc2eb4 Add username to create redis client
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-29 16:27:43 +08:00
Wang Yan
5f8b59177b
Merge pull request #4019 from milosgajdos/replace-redigo-redis
Replace redigo with redis-go
2023-08-29 09:53:32 +08:00
Milos Gajdos
884cf14d30
Merge pull request #4013 from milosgajdos/nonjson-error-client
Dont parse errors as JSON unless Content-Type is set to JSON
2023-08-28 14:28:48 +01:00
Milos Gajdos
45b7b9cec3
Dont parse errors as JSON unless Content-Type is set to JSON
Client attempts to parse the body of every error it receives as JSON
regardless of the content-type. This commit rectifies by only parsing
he error body as JSON if the Content-Type header is set to
either "application/json" or "application/vnd.api+json".

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-28 12:35:33 +01:00
David van der Spek
c7bdabadcf
add back getKeys + cleanup manifeststore test
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-28 12:44:49 +02:00
David van der Spek
f9bc9220eb
feat(storage)!: remove schema1 except manifeststore_test
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-28 12:44:46 +02:00
James Hewitt
1a3e73cb84
Handle rand deprecations in go 1.20
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-28 09:33:12 +01:00
Milos Gajdos
fcbc25e789
Replace redigo with redis-go
We are replacing the very outdated redigo Go module with the official
redis Go module, go-redis.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-26 07:44:02 +01:00
Milos Gajdos
4f7424c8eb
Merge pull request #4000 from DavidSpek/remove-bugsnag
Remove bugsnag
2023-08-23 15:21:18 +01:00
Milos Gajdos
4a85aab30f
Merge pull request #4012 from liubin/fix-typo
fix typo in comment and log
2023-08-23 15:20:05 +01:00
bin liu
1284c48781 fix typo in comment and log
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-23 19:39:19 +08:00
Milos Gajdos
59dd684cc8
Merge pull request #3713 from Jamstah/s3-tests 2023-08-21 13:48:43 +01:00
Milos Gajdos
1a4638a06c
Merge pull request #4007 from milosgajdos/remove-schema1-client 2023-08-21 13:46:29 +01:00
Sebastiaan van Stijn
5b3be39870
s3: add interface assertion
This was added for the other drivers in 6b388b1ba6,
but it missed the s3 storage driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 13:54:13 +02:00
Milos Gajdos
df2787c6cf
Update test asserts
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-21 10:27:46 +01:00
David van der Spek
3e4c4ead4c
Remove bugsnag
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-21 10:24:36 +02:00
Milos Gajdos
11c341a369
Remove schema1 references from registry client
We've replaced all the schema1 references with OCI schema manifest.
Note, there are some TODO items that must be addressed at some point in
the future once the schema1 package is removed completely from the
codebase.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-20 09:07:59 +01:00
Milos Gajdos
3dbfbc7255
Enable bodyclose linter
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-19 09:45:44 +01:00
Milos Gajdos
597e8a6b34
Merge pull request #4001 from DavidSpek/remove-newrelic
Remove NewRelic
2023-08-18 22:35:16 +01:00
James Hewitt
37a213dc4b
Revert "optimize catalog last param"
This reverts commit 65f4ce4d93.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:37 +01:00
James Hewitt
8fd504debe
Revert "Rename catalog funcs and update their godocs."
This reverts commit 230cc72a8b.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:28 +01:00
James Hewitt
ad11105052
Revert "removed redundant check"
This reverts commit 0f846853fe.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:19 +01:00
James Hewitt
8e4a8517c5
Revert "fix: resolve most comments"
This reverts commit 6a5846b32e.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-18 13:52:06 +01:00
David van der Spek
77c33cd243
remove NewRelic
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-18 12:16:18 +02:00
Milos Gajdos
bf3c2df6b2
Merge pull request #3902 from pluralsh/catalog-opti-fix-rebase
Optimise catalog function rebase of #3145
2023-08-18 08:11:29 +01:00
Milos Gajdos
ac32466188
Merge pull request #3993 from DavidSpek/update-golang-lru
Update github.com/hashicorp/golang-lru to v2
2023-08-18 08:09:11 +01:00
Milos Gajdos
40c56bf1b6
Keep returning image for default arch to old clients fetching lists
This puts back the original flow where old clients are fetching manifest
lists schema1 images where we want to try returning some image for the
default architecture. This was incorrectly removed by one of the
previous commits.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-17 20:57:02 +01:00
David van der Spek
0f006548a1
update golang-lru to v2
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-17 13:41:54 +02:00
Milos Gajdos
f517191da1
Add small update to api tests
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-17 09:37:26 +01:00
Milos Gajdos
7e39a7c6dc
Remove references to schema1 pacakge from handlers
schema1 package was deprecated a while ago so we are removing
any references to it from handlers. in preparation to
removing it from the codebase altogether.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-17 08:35:54 +01:00
Milos Gajdos
3f1859af26
Remove oss storage driver and alicdn storage driver middleware
This commit removes `oss` storage driver from distribution as well as
`alicdn` storage middleware which only works with the `oss` driver.

There are several reasons for it:
* no real-life expertise among the maintainers
* oss is compatible with S3 API operations required by S3 storage driver

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-16 08:39:20 +01:00
Milos Gajdos
65b57464f9
Merge pull request #3982 from milosgajdos/remove-swift-storage-driver
Remove SWIFT storage driver
2023-08-16 07:47:42 +01:00
Milos Gajdos
fff194fc46
Merge pull request #3983 from Jamstah/azure-tests
Fix Azure tests
2023-08-15 18:31:05 +01:00
James Hewitt
46ff5f8528
Fix Azure tests
The Azure tests fail if there is no Azure configuration available,
instead they should be skipped.

Also, one of the Azure tests is wrong and doesn't match the code.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-15 16:46:36 +01:00