Commit graph

1183 commits

Author SHA1 Message Date
Hayley Swimelar
87b280718d
Merge pull request #2752 from davidswu/registry-driver-middleware
Add content cache required changes to distribution
2023-06-21 10:06:13 -07:00
David Wu
db1d0cbf35 Add registry middleware access to storage drivers
Signed-off-by: David Wu <dwu7401@gmail.com>
2023-06-20 17:18:05 -07:00
Wang Yan
6a57630cf4
Merge pull request #3934 from stonezdj/add_caller_info
Add option to enable caller information in logger
2023-06-11 21:53:14 +08:00
Irene Diez
421a97ffab registry/api/v2: fix ContentType in RouteNameBlobUpload
Sets 'application/octet-stream' as the ContentType in the
Body of RouteNameBlobUpload.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-06-08 11:05:34 +02:00
stonezdj
2338ee4f25 Add option to enable caller information in logger
Signed-off-by: stonezdj <stonezdj@gmail.com>
2023-06-06 13:14:02 +08:00
Laura Brehm
afe5a2a9b7 Support ztsd compression as Content-Encoding
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-06-01 14:06:25 +01:00
Milos Gajdos
5902a24fd0
Merge pull request #3929 from flavianmissi/fix-gcs-oss-file-writer
Fix gcs storage driver

Thanks to @jmontleon who laid the first bricks in #3702
2023-06-01 09:56:57 +01:00
Flavian Missi
0207adaa5c registry/storage/driver/gcs: fix code to use updated gcs driver
Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-31 09:28:03 +02:00
Flavian Missi
d0bc83d8e4 registry/storage/driver: receive context on Cancel methods
both oss and gcs driver were missing the context parameter that is
required to satisfy the storagedriver.FileWriter interface.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-31 09:14:51 +02:00
Milos Gajdos
ba46c769b3
Merge pull request #3932 from flavianmissi/fix-azure-test-parameters
registry/storage/driver: test call to Stat(ctx, "/")
2023-05-30 21:49:32 +01:00
Flavian Missi
0d20e7ae9e registry/storage/driver/testsuites: use 4MB for Azure append test
Fixes #3931.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:08 +02:00
Flavian Missi
0c33bb1092 registry/storage/driver/azure: consider CannotVerifyCopySource as 404
Azure will return CannotVerifyCopySource with a 404 status code from a
call to Move when the source blob does not exist.
Details: https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-error-codes.

This fixes the TestMoveNonexistent test case for the Azure driver.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:08 +02:00
Flavian Missi
d2e16fc74a registry/storage/driver/azure: fix driver parameters on tests
fixes "azure: no Host in request URL" tests failure, allowing azure
driver tests to run.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:08 +02:00
Flavian Missi
90ece48d77 registry/storage/driver: add test call to Stat on "/"
Stat(ctx, "/") is called by the registry healthcheck.
Also fixes blob name building in the Azure driver so it no longer
returns empty blob names. This was causing errors in the healthcheck
call to Stat for Azure.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:00 +02:00
Lucas França de Oliveira
035a8ec52a
Fix panic in the s3 backend walk logic
Signed-off-by: Lucas França de Oliveira <lucasfdo@palantir.com>
2023-05-25 14:56:05 -07:00
Milos Gajdos
f55a6552b0
Merge pull request from GHSA-hqxw-f8mx-cpmw
Fix runaway allocation on /v2/_catalog
2023-05-09 21:21:54 +01:00
Sebastiaan van Stijn
ebe9d67446
ignore SA1019: ac.(*accessController).rootCerts.Subjects has been deprecated
We need to look into this; can we remove it, or is there a replacement?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Sebastiaan van Stijn
999527f978
Ignore SA1019: "schema1 is deprecated" linting errors
We need to use this for backward compatibility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Sebastiaan van Stijn
3c144f2264
registry/auth/token: fix the surrounding loop is unconditionally terminate
registry/auth/token/types_test.go:83:3: SA4004: the surrounding loop is unconditionally terminated (staticcheck)
                   return
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Sebastiaan van Stijn
4052d269f5
reference, registry: fix loop variable captured by func literal (govet)
reference/normalize_test.go:274:40: loopclosure: loop variable r captured by func literal (govet)
                                   named, err := ParseNormalizedNamed(r)
                                                                      ^
    reference/normalize_test.go:276:29: loopclosure: loop variable r captured by func literal (govet)
                        t.Fatalf("ref=%s: %v", r, err)
                                               ^
    registry/api/errcode/errors_test.go:45:7: loopclosure: loop variable ec captured by func literal (govet)
                if ec != desc.Code {
                   ^
    registry/api/errcode/errors_test.go:46:66: loopclosure: loop variable ec captured by func literal (govet)
                    t.Fatalf("error code in descriptor isn't correct, %q != %q", ec, desc.Code)
                                                                                 ^
    registry/api/errcode/errors_test.go:49:23: loopclosure: loop variable desc captured by func literal (govet)
                if idToDescriptors[desc.Value].Code != ec {
                                   ^
    registry/api/errcode/errors_test.go:50:80: loopclosure: loop variable desc captured by func literal (govet)
                    t.Fatalf("error code in idToDesc isn't correct, %q != %q", idToDescriptors[desc.Value].Code, ec)
                                                                                               ^
    registry/api/errcode/errors_test.go:53:7: loopclosure: loop variable ec captured by func literal (govet)
                if ec.Message() != desc.Message {
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:08 +02:00
Sebastiaan van Stijn
f03d966ef7
cloudfront: use consistent names for test-tables, t.Parallel()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:39:58 +02:00
Sebastiaan van Stijn
5301ae14bf
cloudfront: rename vars that collided with type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:00:17 +02:00
Sebastiaan van Stijn
85028a801b
registry/handlers: use consistent names for test-tables
Also marked assertBlobUploadStateEquals as t.Helper()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:00:17 +02:00
Sebastiaan van Stijn
f4acd98865
registry/api/v2: checkTestRouter(): use sub-tests, t.Parallel()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:00:05 +02:00
Sebastiaan van Stijn
2444c3282d
registry/api/v2: use consistent names for test-tables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:16 +02:00
Sebastiaan van Stijn
9266220c2a
registry/auth/token: TestAudienceList_Unmarshal: t.Parallel()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:16 +02:00
Sebastiaan van Stijn
5e67a40e08
registry/auth/token: use consistent names for test-tables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:16 +02:00
Sebastiaan van Stijn
f884a079df
registry/api/errorcode: TestErrorCodes: use sub-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:11 +02:00
Wang Yan
8900e90699
Merge pull request #3895 from pluralsh/bug-fix-5001-server-rebase
Fix the issue that the debug server with port 5001 run twice
2023-05-05 13:21:55 +08:00
Milos Gajdos
08f7a47bc9
Merge pull request #2766 from glefloch/remove-testdriver
Remove registry storage testdriver
2023-05-03 21:40:53 +01:00
David van der Spek
99a8ad00ea fix: rename log to logrus
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-05-03 19:02:03 +02:00
Feng Honglin
f0fdaff0a5 Fix the issue that the debug server with port 5001 run twice
When configuring bugsnag, bugsnag will fork the process, resulting the port 5001 listened twice. The PR fix this error by moving the initialization of prometheus server after the configuration of bugsnag

Signed-off-by: Honglin Feng <tifayuki@gmail.com>
(cherry picked from commit 5a6a2d6ae06453136f5e1cfb5e9efa20c27085d9)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-05-03 18:37:32 +02:00
AdamKorcz
e2a43ec8d3
Fuzzing: Move over two fuzzers from cncf-fuzzing
Signed-off-by: AdamKorcz <adam@adalogics.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-02 23:03:57 +02:00
glefloch
9fb201e23d Remove registry storage testdriver
Signed-off-by: glefloch <glfloch@gmail.com>
2023-05-02 16:37:33 +02:00
Milos Gajdos
788598e23f
Merge pull request #3881 from vvoland/handle-forbidden
registry/errors: Parse http forbidden as denied
2023-05-02 09:38:36 +01:00
Sebastiaan van Stijn
3fa6d5a33b
remove dot-imports for gopkg.in/check.v1
Dot-imports were only used in a couple of places, and replacing them
makes it more explicit what's imported.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 17:08:47 +02:00
Paweł Gronowski
5f1df02149
registry/errors: Parse http forbidden as denied
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-27 19:48:32 +02:00
Milos Gajdos
29b5e79f82
Merge pull request #3742 from sagikazarmark/fix-aud-claim-list
Accept list of strings in audience claim in token auth
2023-04-26 18:39:26 +01:00
Milos Gajdos
2fb8dbdeca
Merge pull request #3839 from kirat-singh/feature.azure-sdk-update
Update Azure SDK and support additional authentication schemes
2023-04-25 19:35:34 +01:00
Kirat Singh
ba4a6bbe02 Update Azure SDK and support additional authentication schemes
Microsoft has updated the golang Azure SDK significantly.  Update the
azure storage driver to use the new SDK.  Add support for client
secret and MSI authentication schemes in addition to shared key
authentication.

Implement rootDirectory support for the azure storage driver to mirror
the S3 driver.

Signed-off-by: Kirat Singh <kirat.singh@beacon.io>

Co-authored-by: Cory Snider <corhere@gmail.com>
2023-04-25 17:23:20 +00:00
Jose D. Gomez R
4c1561e9fb
Fix runaway allocation on /v2/_catalog
Introduced a Catalog entry in the configuration struct. With it,
it's possible to control the maximum amount of entries returned
by /v2/catalog (`GetCatalog` in registry/handlers/catalog.go).

It's set to a default value of 1000.

`GetCatalog` returns 100 entries by default if no `n` is
provided. When provided it will be validated to be between `0`
and `MaxEntries` defined in Configuration. When `n` is outside
the aforementioned boundary, an error response is returned.

`GetCatalog` now handles `n=0` gracefully with an empty response
as well.

Signed-off-by: José D. Gómez R. <1josegomezr@gmail.com>
2023-03-31 13:17:43 +02:00
Milos Gajdos
0c958010ac
Merge pull request #3763 from distribution/multipart-upload-empty-files
Enable pushing empty blobs
2023-03-27 10:18:44 +01:00
Milos Gajdos
5fa926a609
Enable pushing empty blobs
This is an edge case when we are trying to upload an empty chunk of data using
a MultiPart upload. As a result we are trying to complete the MultipartUpload
with an empty slice of `completedUploadedParts` which will always lead to 400
being returned from S3 See: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#CompletedMultipartUpload
Solution: we upload an empty i.e. 0 byte part as a single part and then append it
to the completedUploadedParts slice used to complete the Multipart upload.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-03-27 10:11:07 +01:00
Aaron Lehmann
2074688be9 Fix S3 multipart upload pagination loop condition
The loop that iterates over paginated lists of S3 multipart upload parts
appears to be using the wrong variable in its loop condition. Nothing
inside the loop affects the value of `resp.IsTruncated`, so this loop
will either be wrongly skipped or loop forever.

It looks like this is a regression caused by commit
7736319f2e. The return value of
`ListMultipartUploads` used to be assigned to a variable named `resp`,
but it was renamed to `partsList` without updating the for loop
condition.

I believe this is causing an error we're seeing with large layer uploads
at commit time:

    upload resumed at wrong offset: 5242880000 != 5815706782

Missing parts of the multipart S3 upload would cause an incorrect size
calculation in `newWriter`.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2023-02-21 20:57:50 -08:00
Milos Gajdos
772cef6b4e
Merge pull request #3736 from aaronlehmann/log-username-on-successful-requests
Log username on successful requests
2023-02-19 19:05:35 +00:00
Kirat Singh
3117e2eb2f
Use default http.Transport for AWS S3 session
Previously we used a custom Transport in order to modify the user agent header.
This prevented the AWS SDK from being able to customize SSL and other client TLS
parameters since it could not understand the Transport type.

Instead we can simply use the SDK function MakeAddToUserAgentFreeFormHandler to
customize the UserAgent if necessary and leave all the TLS configuration to the
AWS SDK.

The only exception being SkipVerify which we have to handle, but we can set it
onto the standard http.Transport which does not interfere with the SDKs ability
to set other options.

Signed-off-by: Kirat Singh <kirat.singh@gmail.com>
2023-02-15 13:37:01 -05:00
Aaron Lehmann
a811c1bb57 Log username on successful requests
Currently, "response completed with error" log lines include an
`auth.user.name` key, but successful "response completed" lines do not
include this, because they are logged a few stack frames up where
`auth.user.name` is not present on the `Context`. Move the successful
request logging inside the `dispatcher` closure, where the logger on the
context automatically includes this key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2023-01-30 09:32:39 -08:00
Milos Gajdos
ac302d9ce5
Merge pull request #3807 from thaJeztah/replace_types_for_oci_step1
minor fixes and enhancements
2022-11-29 10:49:12 +00:00
Milos Gajdos
8cc58797e8
Merge pull request #3794 from AdamKorcz/fuzz1
Fuzzing: Rewrite existing fuzzers to native go fuzzers
2022-11-29 09:57:09 +00:00
Sebastiaan van Stijn
f2db7faa2f
registry/storage: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00