Commit graph

1190 commits

Author SHA1 Message Date
Milos Gajdos
69023c7f85
Merge pull request #3880 from wzshiming/proxy-cache-configurable 2023-07-14 08:43:19 +01:00
Milos Gajdos
6b388b1ba6
Enable Go build tags
This enables go build tags so the GCS and OSS driver support is
available in the binary distributed via the image build by Dockerfile.

This led to quite a few fixes in the GCS and OSS packages raised as
warning by golang-ci linter.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-06-28 11:41:22 +01:00
Hayley Swimelar
71a6c56fbf
Merge pull request #3948 from flavianmissi/log-healthcheck-error
registry/handlers/app: log healthcheck error before return
2023-06-27 14:06:31 -07:00
Flavian Missi
87081252ba registry/handlers/app: log healthcheck error before return
Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-06-27 09:50:15 +02:00
Milos Gajdos
22725209e3
Merge pull request #3936 from flavianmissi/azure-path-not-found
Fix path not found error in Azure
2023-06-26 13:21:00 +01:00
Flavian Missi
2b72c4d1ca registry/storage/driver/azure: fix Move method
Something seems broken on azure/azure sdk side - it is currently not
possible to copy a blob of type AppendBlob using `CopyFromURL`.
Using the AppendBlob client via NewAppendBlobClient does not work
either.

According to Azure the correct way to do this is by using
StartCopyFromURL. Because this is an async operation, we need to do
polling ourselves. A simple backoff mechanism is used, where during each
iteration, the configured delay is multiplied by the retry number.

Also introduces two new config options for the Azure driver:
copy_status_poll_max_retry, and copy_status_poll_delay.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-06-26 13:47:30 +02:00
Hayley Swimelar
87b280718d
Merge pull request #2752 from davidswu/registry-driver-middleware
Add content cache required changes to distribution
2023-06-21 10:06:13 -07:00
David Wu
db1d0cbf35 Add registry middleware access to storage drivers
Signed-off-by: David Wu <dwu7401@gmail.com>
2023-06-20 17:18:05 -07:00
Wang Yan
6a57630cf4
Merge pull request #3934 from stonezdj/add_caller_info
Add option to enable caller information in logger
2023-06-11 21:53:14 +08:00
Irene Diez
421a97ffab registry/api/v2: fix ContentType in RouteNameBlobUpload
Sets 'application/octet-stream' as the ContentType in the
Body of RouteNameBlobUpload.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-06-08 11:05:34 +02:00
stonezdj
2338ee4f25 Add option to enable caller information in logger
Signed-off-by: stonezdj <stonezdj@gmail.com>
2023-06-06 13:14:02 +08:00
Laura Brehm
afe5a2a9b7 Support ztsd compression as Content-Encoding
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-06-01 14:06:25 +01:00
Milos Gajdos
5902a24fd0
Merge pull request #3929 from flavianmissi/fix-gcs-oss-file-writer
Fix gcs storage driver

Thanks to @jmontleon who laid the first bricks in #3702
2023-06-01 09:56:57 +01:00
Flavian Missi
0207adaa5c registry/storage/driver/gcs: fix code to use updated gcs driver
Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-31 09:28:03 +02:00
Flavian Missi
d0bc83d8e4 registry/storage/driver: receive context on Cancel methods
both oss and gcs driver were missing the context parameter that is
required to satisfy the storagedriver.FileWriter interface.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-31 09:14:51 +02:00
Milos Gajdos
ba46c769b3
Merge pull request #3932 from flavianmissi/fix-azure-test-parameters
registry/storage/driver: test call to Stat(ctx, "/")
2023-05-30 21:49:32 +01:00
Flavian Missi
0d20e7ae9e registry/storage/driver/testsuites: use 4MB for Azure append test
Fixes #3931.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:08 +02:00
Flavian Missi
0c33bb1092 registry/storage/driver/azure: consider CannotVerifyCopySource as 404
Azure will return CannotVerifyCopySource with a 404 status code from a
call to Move when the source blob does not exist.
Details: https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-error-codes.

This fixes the TestMoveNonexistent test case for the Azure driver.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:08 +02:00
Flavian Missi
d2e16fc74a registry/storage/driver/azure: fix driver parameters on tests
fixes "azure: no Host in request URL" tests failure, allowing azure
driver tests to run.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:08 +02:00
Flavian Missi
90ece48d77 registry/storage/driver: add test call to Stat on "/"
Stat(ctx, "/") is called by the registry healthcheck.
Also fixes blob name building in the Azure driver so it no longer
returns empty blob names. This was causing errors in the healthcheck
call to Stat for Azure.

Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-05-30 15:12:00 +02:00
Lucas França de Oliveira
035a8ec52a
Fix panic in the s3 backend walk logic
Signed-off-by: Lucas França de Oliveira <lucasfdo@palantir.com>
2023-05-25 14:56:05 -07:00
Milos Gajdos
f55a6552b0
Merge pull request from GHSA-hqxw-f8mx-cpmw
Fix runaway allocation on /v2/_catalog
2023-05-09 21:21:54 +01:00
Sebastiaan van Stijn
ebe9d67446
ignore SA1019: ac.(*accessController).rootCerts.Subjects has been deprecated
We need to look into this; can we remove it, or is there a replacement?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Sebastiaan van Stijn
999527f978
Ignore SA1019: "schema1 is deprecated" linting errors
We need to use this for backward compatibility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Sebastiaan van Stijn
3c144f2264
registry/auth/token: fix the surrounding loop is unconditionally terminate
registry/auth/token/types_test.go:83:3: SA4004: the surrounding loop is unconditionally terminated (staticcheck)
                   return
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Sebastiaan van Stijn
4052d269f5
reference, registry: fix loop variable captured by func literal (govet)
reference/normalize_test.go:274:40: loopclosure: loop variable r captured by func literal (govet)
                                   named, err := ParseNormalizedNamed(r)
                                                                      ^
    reference/normalize_test.go:276:29: loopclosure: loop variable r captured by func literal (govet)
                        t.Fatalf("ref=%s: %v", r, err)
                                               ^
    registry/api/errcode/errors_test.go:45:7: loopclosure: loop variable ec captured by func literal (govet)
                if ec != desc.Code {
                   ^
    registry/api/errcode/errors_test.go:46:66: loopclosure: loop variable ec captured by func literal (govet)
                    t.Fatalf("error code in descriptor isn't correct, %q != %q", ec, desc.Code)
                                                                                 ^
    registry/api/errcode/errors_test.go:49:23: loopclosure: loop variable desc captured by func literal (govet)
                if idToDescriptors[desc.Value].Code != ec {
                                   ^
    registry/api/errcode/errors_test.go:50:80: loopclosure: loop variable desc captured by func literal (govet)
                    t.Fatalf("error code in idToDesc isn't correct, %q != %q", idToDescriptors[desc.Value].Code, ec)
                                                                                               ^
    registry/api/errcode/errors_test.go:53:7: loopclosure: loop variable ec captured by func literal (govet)
                if ec.Message() != desc.Message {
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:08 +02:00
Sebastiaan van Stijn
f03d966ef7
cloudfront: use consistent names for test-tables, t.Parallel()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:39:58 +02:00
Sebastiaan van Stijn
5301ae14bf
cloudfront: rename vars that collided with type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:00:17 +02:00
Sebastiaan van Stijn
85028a801b
registry/handlers: use consistent names for test-tables
Also marked assertBlobUploadStateEquals as t.Helper()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:00:17 +02:00
Sebastiaan van Stijn
f4acd98865
registry/api/v2: checkTestRouter(): use sub-tests, t.Parallel()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 14:00:05 +02:00
Sebastiaan van Stijn
2444c3282d
registry/api/v2: use consistent names for test-tables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:16 +02:00
Sebastiaan van Stijn
9266220c2a
registry/auth/token: TestAudienceList_Unmarshal: t.Parallel()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:16 +02:00
Sebastiaan van Stijn
5e67a40e08
registry/auth/token: use consistent names for test-tables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:16 +02:00
Sebastiaan van Stijn
f884a079df
registry/api/errorcode: TestErrorCodes: use sub-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:55:11 +02:00
Wang Yan
8900e90699
Merge pull request #3895 from pluralsh/bug-fix-5001-server-rebase
Fix the issue that the debug server with port 5001 run twice
2023-05-05 13:21:55 +08:00
Milos Gajdos
08f7a47bc9
Merge pull request #2766 from glefloch/remove-testdriver
Remove registry storage testdriver
2023-05-03 21:40:53 +01:00
David van der Spek
99a8ad00ea fix: rename log to logrus
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-05-03 19:02:03 +02:00
Feng Honglin
f0fdaff0a5 Fix the issue that the debug server with port 5001 run twice
When configuring bugsnag, bugsnag will fork the process, resulting the port 5001 listened twice. The PR fix this error by moving the initialization of prometheus server after the configuration of bugsnag

Signed-off-by: Honglin Feng <tifayuki@gmail.com>
(cherry picked from commit 5a6a2d6ae06453136f5e1cfb5e9efa20c27085d9)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-05-03 18:37:32 +02:00
AdamKorcz
e2a43ec8d3
Fuzzing: Move over two fuzzers from cncf-fuzzing
Signed-off-by: AdamKorcz <adam@adalogics.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-02 23:03:57 +02:00
glefloch
9fb201e23d Remove registry storage testdriver
Signed-off-by: glefloch <glfloch@gmail.com>
2023-05-02 16:37:33 +02:00
Milos Gajdos
788598e23f
Merge pull request #3881 from vvoland/handle-forbidden
registry/errors: Parse http forbidden as denied
2023-05-02 09:38:36 +01:00
Sebastiaan van Stijn
3fa6d5a33b
remove dot-imports for gopkg.in/check.v1
Dot-imports were only used in a couple of places, and replacing them
makes it more explicit what's imported.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 17:08:47 +02:00
Paweł Gronowski
5f1df02149
registry/errors: Parse http forbidden as denied
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-27 19:48:32 +02:00
Milos Gajdos
29b5e79f82
Merge pull request #3742 from sagikazarmark/fix-aud-claim-list
Accept list of strings in audience claim in token auth
2023-04-26 18:39:26 +01:00
Milos Gajdos
2fb8dbdeca
Merge pull request #3839 from kirat-singh/feature.azure-sdk-update
Update Azure SDK and support additional authentication schemes
2023-04-25 19:35:34 +01:00
Kirat Singh
ba4a6bbe02 Update Azure SDK and support additional authentication schemes
Microsoft has updated the golang Azure SDK significantly.  Update the
azure storage driver to use the new SDK.  Add support for client
secret and MSI authentication schemes in addition to shared key
authentication.

Implement rootDirectory support for the azure storage driver to mirror
the S3 driver.

Signed-off-by: Kirat Singh <kirat.singh@beacon.io>

Co-authored-by: Cory Snider <corhere@gmail.com>
2023-04-25 17:23:20 +00:00
Manish Tomar
8fe4ca4038 Option to configure proxy cache TTL
Currently when registry is run as proxy it tries to cleanup unused blobs
from its cache after 7 days which is hard-coded. This PR makes that
value configurable.

Co-authored-by: Shiming Zhang <wzshiming@foxmail.com>
Co-authored-by: Manish Tomar <manish.tomar@docker.com>
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2023-04-20 13:03:39 +08:00
Jose D. Gomez R
4c1561e9fb
Fix runaway allocation on /v2/_catalog
Introduced a Catalog entry in the configuration struct. With it,
it's possible to control the maximum amount of entries returned
by /v2/catalog (`GetCatalog` in registry/handlers/catalog.go).

It's set to a default value of 1000.

`GetCatalog` returns 100 entries by default if no `n` is
provided. When provided it will be validated to be between `0`
and `MaxEntries` defined in Configuration. When `n` is outside
the aforementioned boundary, an error response is returned.

`GetCatalog` now handles `n=0` gracefully with an empty response
as well.

Signed-off-by: José D. Gómez R. <1josegomezr@gmail.com>
2023-03-31 13:17:43 +02:00
Milos Gajdos
0c958010ac
Merge pull request #3763 from distribution/multipart-upload-empty-files
Enable pushing empty blobs
2023-03-27 10:18:44 +01:00
Milos Gajdos
5fa926a609
Enable pushing empty blobs
This is an edge case when we are trying to upload an empty chunk of data using
a MultiPart upload. As a result we are trying to complete the MultipartUpload
with an empty slice of `completedUploadedParts` which will always lead to 400
being returned from S3 See: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#CompletedMultipartUpload
Solution: we upload an empty i.e. 0 byte part as a single part and then append it
to the completedUploadedParts slice used to complete the Multipart upload.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-03-27 10:11:07 +01:00