Commit graph

4825 commits

Author SHA1 Message Date
Sebastiaan van Stijn
fa1d14c513
reference: TestParseAnyReference(): use sub-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:54:37 +02:00
Sebastiaan van Stijn
fcbddfc6ae
reference: use consistent names for test-tables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 13:54:37 +02:00
Wang Yan
8900e90699
Merge pull request #3895 from pluralsh/bug-fix-5001-server-rebase
Fix the issue that the debug server with port 5001 run twice
2023-05-05 13:21:55 +08:00
Milos Gajdos
08f7a47bc9
Merge pull request #2766 from glefloch/remove-testdriver
Remove registry storage testdriver
2023-05-03 21:40:53 +01:00
David van der Spek
99a8ad00ea fix: rename log to logrus
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-05-03 19:02:03 +02:00
Feng Honglin
f0fdaff0a5 Fix the issue that the debug server with port 5001 run twice
When configuring bugsnag, bugsnag will fork the process, resulting the port 5001 listened twice. The PR fix this error by moving the initialization of prometheus server after the configuration of bugsnag

Signed-off-by: Honglin Feng <tifayuki@gmail.com>
(cherry picked from commit 5a6a2d6ae06453136f5e1cfb5e9efa20c27085d9)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-05-03 18:37:32 +02:00
Milos Gajdos
4adbb690c1
Merge pull request #3795 from AdamKorcz/fuzz222
Fuzzing: Move over two fuzzers from cncf-fuzzing
2023-05-03 15:59:20 +01:00
AdamKorcz
e2a43ec8d3
Fuzzing: Move over two fuzzers from cncf-fuzzing
Signed-off-by: AdamKorcz <adam@adalogics.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-02 23:03:57 +02:00
Milos Gajdos
0ea92a1a8d
Merge pull request #3883 from thaJeztah/reference_update_tests_and_add_benchmark
reference: rewrite test to use sub-tests, add benchmark
2023-05-02 20:55:05 +01:00
glefloch
9fb201e23d Remove registry storage testdriver
Signed-off-by: glefloch <glfloch@gmail.com>
2023-05-02 16:37:33 +02:00
Milos Gajdos
788598e23f
Merge pull request #3881 from vvoland/handle-forbidden
registry/errors: Parse http forbidden as denied
2023-05-02 09:38:36 +01:00
Milos Gajdos
1062ccc7ef
Merge pull request #3882 from thaJeztah/no_dot_imports
remove dot-imports for gopkg.in/check.v1
2023-05-01 16:21:11 +01:00
Sebastiaan van Stijn
49e2de2830
reference: add BenchmarkParse
BenchmarkParse
    BenchmarkParse-10    	   12696	     93805 ns/op	    9311 B/op	     185 allocs/op
    PASS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 19:31:21 +02:00
Sebastiaan van Stijn
b50c049fc6
reference: TestParseRepositoryInfo: use subtests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 18:58:09 +02:00
Sebastiaan van Stijn
af36dd698f
reference: TestParseDockerRef: capture test in loop
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 18:58:01 +02:00
Sebastiaan van Stijn
3fa6d5a33b
remove dot-imports for gopkg.in/check.v1
Dot-imports were only used in a couple of places, and replacing them
makes it more explicit what's imported.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 17:08:47 +02:00
Paweł Gronowski
5f1df02149
registry/errors: Parse http forbidden as denied
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-27 19:48:32 +02:00
Milos Gajdos
29b5e79f82
Merge pull request #3742 from sagikazarmark/fix-aud-claim-list
Accept list of strings in audience claim in token auth
2023-04-26 18:39:26 +01:00
Milos Gajdos
2fb8dbdeca
Merge pull request #3839 from kirat-singh/feature.azure-sdk-update
Update Azure SDK and support additional authentication schemes
2023-04-25 19:35:34 +01:00
Kirat Singh
ba4a6bbe02 Update Azure SDK and support additional authentication schemes
Microsoft has updated the golang Azure SDK significantly.  Update the
azure storage driver to use the new SDK.  Add support for client
secret and MSI authentication schemes in addition to shared key
authentication.

Implement rootDirectory support for the azure storage driver to mirror
the S3 driver.

Signed-off-by: Kirat Singh <kirat.singh@beacon.io>

Co-authored-by: Cory Snider <corhere@gmail.com>
2023-04-25 17:23:20 +00:00
Milos Gajdos
0c958010ac
Merge pull request #3763 from distribution/multipart-upload-empty-files
Enable pushing empty blobs
2023-03-27 10:18:44 +01:00
Milos Gajdos
5fa926a609
Enable pushing empty blobs
This is an edge case when we are trying to upload an empty chunk of data using
a MultiPart upload. As a result we are trying to complete the MultipartUpload
with an empty slice of `completedUploadedParts` which will always lead to 400
being returned from S3 See: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#CompletedMultipartUpload
Solution: we upload an empty i.e. 0 byte part as a single part and then append it
to the completedUploadedParts slice used to complete the Multipart upload.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-03-27 10:11:07 +01:00
Milos Gajdos
e5d5810851
Merge pull request #3847 from aaronlehmann/s3-pagination-loop
Fix S3 multipart upload pagination loop condition
2023-02-23 07:28:52 +00:00
Aaron Lehmann
2074688be9 Fix S3 multipart upload pagination loop condition
The loop that iterates over paginated lists of S3 multipart upload parts
appears to be using the wrong variable in its loop condition. Nothing
inside the loop affects the value of `resp.IsTruncated`, so this loop
will either be wrongly skipped or loop forever.

It looks like this is a regression caused by commit
7736319f2e. The return value of
`ListMultipartUploads` used to be assigned to a variable named `resp`,
but it was renamed to `partsList` without updating the for loop
condition.

I believe this is causing an error we're seeing with large layer uploads
at commit time:

    upload resumed at wrong offset: 5242880000 != 5815706782

Missing parts of the multipart S3 upload would cause an incorrect size
calculation in `newWriter`.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2023-02-21 20:57:50 -08:00
Milos Gajdos
a38eb86fa6
Merge pull request #3845 from distribution/dependabot/go_modules/golang.org/x/net-0.7.0 2023-02-20 14:00:06 +00:00
Milos Gajdos
772cef6b4e
Merge pull request #3736 from aaronlehmann/log-username-on-successful-requests
Log username on successful requests
2023-02-19 19:05:35 +00:00
dependabot[bot]
9594fbcfeb
build(deps): bump golang.org/x/net from 0.4.0 to 0.7.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.4.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-17 22:42:11 +00:00
Hayley Swimelar
a4d64adabc
Merge pull request #3838 from smorimoto/update-fossa-action
Update fossa-contrib/fossa-action action to v2
2023-02-17 14:28:03 -08:00
Milos Gajdos
e64b08ada6
Merge pull request #3841 from kirat-singh/feature/aws-ca-bundle
Support AWS_CA_BUNDLE when talking to the S3 API
2023-02-17 09:36:20 +00:00
Kirat Singh
3117e2eb2f
Use default http.Transport for AWS S3 session
Previously we used a custom Transport in order to modify the user agent header.
This prevented the AWS SDK from being able to customize SSL and other client TLS
parameters since it could not understand the Transport type.

Instead we can simply use the SDK function MakeAddToUserAgentFreeFormHandler to
customize the UserAgent if necessary and leave all the TLS configuration to the
AWS SDK.

The only exception being SkipVerify which we have to handle, but we can set it
onto the standard http.Transport which does not interfere with the SDKs ability
to set other options.

Signed-off-by: Kirat Singh <kirat.singh@gmail.com>
2023-02-15 13:37:01 -05:00
Milos Gajdos
36d8c594d7
Merge pull request #3823 from drornir/patch-1
add double newlines in ADOPTERS.md
2023-02-14 15:00:26 +00:00
drornir
2179a5f66c add double newlines in ADOPTERS.md
Using double newlines in markdown makes it easier to read in markdown readers, e.g github

Signed-off-by: dror.nir <dror@minutemedia.com>
2023-02-14 14:51:41 +02:00
Sora Morimoto
165fd5f9ac Update fossa-contrib/fossa-action action to v2
Signed-off-by: Sora Morimoto <sora@morimoto.io>
2023-02-14 01:43:23 +09:00
Milos Gajdos
cf87e8d07e
Merge pull request #3818 from ckw017/ckw/fixseparatorregex
Fix separator regex to disallow empty strings
2023-01-31 08:15:13 +00:00
Aaron Lehmann
a811c1bb57 Log username on successful requests
Currently, "response completed with error" log lines include an
`auth.user.name` key, but successful "response completed" lines do not
include this, because they are logged a few stack frames up where
`auth.user.name` is not present on the `Context`. Move the successful
request logging inside the `dispatcher` closure, where the logger on the
context automatically includes this key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2023-01-30 09:32:39 -08:00
Milos Gajdos
9b629737cb
Merge pull request #3804 from thaJeztah/deprecate_schema1
manifest/schema1: mark docker manifest v2, schema 1 deprecated
2023-01-30 16:16:38 +00:00
Chris K. Wong
d1c186812e Fix separator regex
Signed-off-by: Chris K. Wong <chriskw.xyz@gmail.com>
2022-12-22 14:50:50 -08:00
Milos Gajdos
362910506b
Merge pull request #3816 from thaJeztah/bump_golang_net
vendor: golang.org/x/net v0.4.0
2022-12-08 16:53:59 +00:00
Sebastiaan van Stijn
345be95498
vendor: golang.org/x/net v0.4.0
golang.org/x/net contains a fix for CVE-2022-41717, which was addressed
in stdlib in go1.19.4 and go1.18.9;

> net/http: limit canonical header cache by bytes, not entries
>
> An attacker can cause excessive memory growth in a Go server accepting
> HTTP/2 requests.
>
> HTTP/2 server connections contain a cache of HTTP header keys sent by
> the client. While the total number of entries in this cache is capped,
> an attacker sending very large keys can cause the server to allocate
> approximately 64 MiB per open connection.
>
> This issue is also fixed in golang.org/x/net/http2 v0.4.0,
> for users manually configuring HTTP/2.

full diff: https://github.com/golang/net/compare/v0.2.0...v0.4.0

other dependency updates (due to (circular) dependencies):

- golang.org/x/sys v0.3.0: https://github.com/golang/sys/compare/3c1f35247d10...v0.3.0
- golang.org/x/text v0.5.0: https://github.com/golang/text/compare/v0.3.7...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 10:39:04 +01:00
Hayley Swimelar
7847817946
Merge pull request #3814 from thaJeztah/remove_fuzz_leftovers
remove script directory and fuzz left-overs
2022-12-05 15:46:40 -08:00
Sebastiaan van Stijn
59f1357775
remove script directory and fuzz left-overs
commit 9337b8df66 rewrote the fuzzers to
native go fuzzers, so the script was no longer needed. With this, the
script directory is no longer used, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 17:15:16 +01:00
Wang Yan
92d136e113
Merge pull request #3810 from thaJeztah/align_descriptor_with_oci
Descriptor: align field order with OCI image specification
2022-12-01 16:32:18 +08:00
Sebastiaan van Stijn
19233195b7
Align code to match order of fields
This is just cosmetic; alighn the fields with the order in which they appear
in the struct (and JSON output).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 23:24:10 +01:00
Sebastiaan van Stijn
86cd830fb3
Descriptor: align field order with OCI image specification
I am looking at aligning the types defined in this repository with the
OCI image specification, and potentially exchanging local types with
those from the specification.

This patch is a stepping-stone towards that effort, but as this changes
the format of the serialized JSON, I wanted to put this up first before
proceeding with the other work in case there are concerns.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 23:24:10 +01:00
Milos Gajdos
ac302d9ce5
Merge pull request #3807 from thaJeztah/replace_types_for_oci_step1
minor fixes and enhancements
2022-11-29 10:49:12 +00:00
Milos Gajdos
8cc58797e8
Merge pull request #3794 from AdamKorcz/fuzz1
Fuzzing: Rewrite existing fuzzers to native go fuzzers
2022-11-29 09:57:09 +00:00
Sebastiaan van Stijn
0b4311d5ce
manifest: improve test output and use const
Use consts to make clear these values are fixed, and improve the output
to make it clearer which part is the expected output, and which part
the actual.

Before this:

    === RUN   TestManifest
        manifest_test.go:87: manifest bytes not equal: "{\n   \"schemaVersion\": 2,\n   \"mediaType\": \"application/vnd.oci.image.manifest.v1+json\",\n   \"config\": {\n      \"mediaType\": \"application/vnd.oci.image.config.v1+json\",\n      \"size\": 985,\n      \"digest\": \"sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b\",\n      \"annotations\": {\n         \"apple\": \"orange\"\n      }\n   },\n   \"layers\": [\n      {\n         \"mediaType\": \"application/vnd.oci.image.layer.v1.tar+gzip\",\n         \"size\": 153263,\n         \"digest\": \"sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b\",\n         \"annotations\": {\n            \"lettuce\": \"wrap\"\n         }\n      }\n   ],\n   \"annotations\": {\n      \"hot\": \"potato\"\n   }\n}" != "{\n   \"schemaVersion\": 2,\n   \"mediaType\": \"application/vnd.oci.image.manifest.v1+json\",\n   \"config\": {\n      \"mediaType\": \"application/vnd.oci.image.config.v1+json\",\n      \"size\": 985,\n      \"digest\": \"sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b\",\n      \"annotations\": {\n         \"apple\": \"orange\"\n      }\n   },\n   \"layers\": [\n      {\n         \"mediaType\": \"application/vnd.oci.image.layer.v1.tar+gzip\",\n         \"size\": 153263,\n         \"digest\": \"sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b\",\n         \"annotations\": {\n            \"lettuce\": \"wrap\"\n         }\n      }\n   ],\n   \"annotations\": {\n      \"hot\": \"potato\"\n   }\n}"
    --- FAIL: TestManifest (0.00s)

After this:

    === RUN   TestManifest
        manifest_test.go:72: manifest bytes not equal:
            expected:
            {
               "schemaVersion": 2,
               "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
               "config": {
                  "mediaType": "application/vnd.docker.container.image.v1+json",
                  "size": 985,
                  "digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b"
               },
               "layers": [
                  {
                     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                     "size": 153263,
                     "digest": "sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b"
                  }
               ]
            }
            actual:
            {
               "schemaVersion": 2,
               "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
               "config": {
                  "mediaType": "application/vnd.docker.container.image.v1+json",
                  "size": 985,
                  "digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b"
               },
               "layers": [
                  {
                     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                     "size": 153263,
                     "digest": "sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b"
                  }
               ]
            }
    --- FAIL: TestManifest (0.00s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00
Sebastiaan van Stijn
f2db7faa2f
registry/storage: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00
Sebastiaan van Stijn
030489ca66
testutil: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00
Sebastiaan van Stijn
0e3efe749b
manifest: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:02 +01:00