Commit graph

5383 commits

Author SHA1 Message Date
Milos Gajdos
788598e23f
Merge pull request #3881 from vvoland/handle-forbidden
registry/errors: Parse http forbidden as denied
2023-05-02 09:38:36 +01:00
Milos Gajdos
1062ccc7ef
Merge pull request #3882 from thaJeztah/no_dot_imports
remove dot-imports for gopkg.in/check.v1
2023-05-01 16:21:11 +01:00
James Hewitt
434cc087cc
Bump to 2.8.x as the current version
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-05-01 09:39:58 +01:00
James Hewitt
d141b65734
We already had instructions for this, use them.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-05-01 09:39:58 +01:00
James Hewitt
7e51e717fb
Add information about security
Versions that will receive security fixes, and
how to report vulnerabilities to the maintainers.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-05-01 09:39:58 +01:00
Sebastiaan van Stijn
49e2de2830
reference: add BenchmarkParse
BenchmarkParse
    BenchmarkParse-10    	   12696	     93805 ns/op	    9311 B/op	     185 allocs/op
    PASS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 19:31:21 +02:00
Sebastiaan van Stijn
b50c049fc6
reference: TestParseRepositoryInfo: use subtests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 18:58:09 +02:00
Sebastiaan van Stijn
af36dd698f
reference: TestParseDockerRef: capture test in loop
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 18:58:01 +02:00
Sebastiaan van Stijn
3fa6d5a33b
remove dot-imports for gopkg.in/check.v1
Dot-imports were only used in a couple of places, and replacing them
makes it more explicit what's imported.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 17:08:47 +02:00
Paweł Gronowski
5f1df02149
registry/errors: Parse http forbidden as denied
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-27 19:48:32 +02:00
Milos Gajdos
29b5e79f82
Merge pull request #3742 from sagikazarmark/fix-aud-claim-list
Accept list of strings in audience claim in token auth
2023-04-26 18:39:26 +01:00
Milos Gajdos
2fb8dbdeca
Merge pull request #3839 from kirat-singh/feature.azure-sdk-update
Update Azure SDK and support additional authentication schemes
2023-04-25 19:35:34 +01:00
Kirat Singh
ba4a6bbe02 Update Azure SDK and support additional authentication schemes
Microsoft has updated the golang Azure SDK significantly.  Update the
azure storage driver to use the new SDK.  Add support for client
secret and MSI authentication schemes in addition to shared key
authentication.

Implement rootDirectory support for the azure storage driver to mirror
the S3 driver.

Signed-off-by: Kirat Singh <kirat.singh@beacon.io>

Co-authored-by: Cory Snider <corhere@gmail.com>
2023-04-25 17:23:20 +00:00
Bracken Dawson
973bfbb676
Fix Go Idioms
- DRY out SchemaVersion literals
- Better name the predefined Versioned struct for the Image Index
- Var names, declarations, else cases.

Co-authored-by: Milos Gajdos <milosthegajdos@gmail.com>
Signed-off-by: Bracken Dawson <abdawson@gmail.com>
2023-04-21 15:22:37 +01:00
Manish Tomar
8fe4ca4038 Option to configure proxy cache TTL
Currently when registry is run as proxy it tries to cleanup unused blobs
from its cache after 7 days which is hard-coded. This PR makes that
value configurable.

Co-authored-by: Shiming Zhang <wzshiming@foxmail.com>
Co-authored-by: Manish Tomar <manish.tomar@docker.com>
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2023-04-20 13:03:39 +08:00
Jakob Ackermann
c624b9eda7
Use docker-compose spec v3 in nginx receipt
Previously v1 was used.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2023-04-07 19:13:56 +02:00
Bracken Dawson
88646f54da
Support annotations in the OCI Image Index
Empty platform structs were already supported after splitting OCI Image
Index out from Docker Manifest List.

Signed-off-by: Bracken Dawson <abdawson@gmail.com>
2023-03-31 14:01:30 +01:00
Jose D. Gomez R
4c1561e9fb
Fix runaway allocation on /v2/_catalog
Introduced a Catalog entry in the configuration struct. With it,
it's possible to control the maximum amount of entries returned
by /v2/catalog (`GetCatalog` in registry/handlers/catalog.go).

It's set to a default value of 1000.

`GetCatalog` returns 100 entries by default if no `n` is
provided. When provided it will be validated to be between `0`
and `MaxEntries` defined in Configuration. When `n` is outside
the aforementioned boundary, an error response is returned.

`GetCatalog` now handles `n=0` gracefully with an empty response
as well.

Signed-off-by: José D. Gómez R. <1josegomezr@gmail.com>
2023-03-31 13:17:43 +02:00
Bracken Dawson
e72294d075
Split OCI Image Index from Docker Manifest List
Move implementation of the index from the manifestlist package to the ocischema package so that other modules making empty imports support the manifest types their authors would expect. This is a breaking change to distribution as a library but not the registry.

As OCI 1.0 released the manifest and index together, that is a good package from which to initialise both manifests. The docker manifest and manifest list remain in separate packages because one was released later.

The image index and manifest list still share common code in many functions not intended for import by other modules.

Signed-off-by: Bracken Dawson <abdawson@gmail.com>
2023-03-31 11:43:21 +01:00
Milos Gajdos
0c958010ac
Merge pull request #3763 from distribution/multipart-upload-empty-files
Enable pushing empty blobs
2023-03-27 10:18:44 +01:00
Milos Gajdos
5fa926a609
Enable pushing empty blobs
This is an edge case when we are trying to upload an empty chunk of data using
a MultiPart upload. As a result we are trying to complete the MultipartUpload
with an empty slice of `completedUploadedParts` which will always lead to 400
being returned from S3 See: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#CompletedMultipartUpload
Solution: we upload an empty i.e. 0 byte part as a single part and then append it
to the completedUploadedParts slice used to complete the Multipart upload.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-03-27 10:11:07 +01:00
Milos Gajdos
e5d5810851
Merge pull request #3847 from aaronlehmann/s3-pagination-loop
Fix S3 multipart upload pagination loop condition
2023-02-23 07:28:52 +00:00
Aaron Lehmann
2074688be9 Fix S3 multipart upload pagination loop condition
The loop that iterates over paginated lists of S3 multipart upload parts
appears to be using the wrong variable in its loop condition. Nothing
inside the loop affects the value of `resp.IsTruncated`, so this loop
will either be wrongly skipped or loop forever.

It looks like this is a regression caused by commit
7736319f2e. The return value of
`ListMultipartUploads` used to be assigned to a variable named `resp`,
but it was renamed to `partsList` without updating the for loop
condition.

I believe this is causing an error we're seeing with large layer uploads
at commit time:

    upload resumed at wrong offset: 5242880000 != 5815706782

Missing parts of the multipart S3 upload would cause an incorrect size
calculation in `newWriter`.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2023-02-21 20:57:50 -08:00
Milos Gajdos
a38eb86fa6
Merge pull request #3845 from distribution/dependabot/go_modules/golang.org/x/net-0.7.0 2023-02-20 14:00:06 +00:00
Milos Gajdos
772cef6b4e
Merge pull request #3736 from aaronlehmann/log-username-on-successful-requests
Log username on successful requests
2023-02-19 19:05:35 +00:00
dependabot[bot]
9594fbcfeb
build(deps): bump golang.org/x/net from 0.4.0 to 0.7.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.4.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-17 22:42:11 +00:00
Hayley Swimelar
a4d64adabc
Merge pull request #3838 from smorimoto/update-fossa-action
Update fossa-contrib/fossa-action action to v2
2023-02-17 14:28:03 -08:00
Milos Gajdos
e64b08ada6
Merge pull request #3841 from kirat-singh/feature/aws-ca-bundle
Support AWS_CA_BUNDLE when talking to the S3 API
2023-02-17 09:36:20 +00:00
Kirat Singh
3117e2eb2f
Use default http.Transport for AWS S3 session
Previously we used a custom Transport in order to modify the user agent header.
This prevented the AWS SDK from being able to customize SSL and other client TLS
parameters since it could not understand the Transport type.

Instead we can simply use the SDK function MakeAddToUserAgentFreeFormHandler to
customize the UserAgent if necessary and leave all the TLS configuration to the
AWS SDK.

The only exception being SkipVerify which we have to handle, but we can set it
onto the standard http.Transport which does not interfere with the SDKs ability
to set other options.

Signed-off-by: Kirat Singh <kirat.singh@gmail.com>
2023-02-15 13:37:01 -05:00
Milos Gajdos
36d8c594d7
Merge pull request #3823 from drornir/patch-1
add double newlines in ADOPTERS.md
2023-02-14 15:00:26 +00:00
drornir
2179a5f66c add double newlines in ADOPTERS.md
Using double newlines in markdown makes it easier to read in markdown readers, e.g github

Signed-off-by: dror.nir <dror@minutemedia.com>
2023-02-14 14:51:41 +02:00
Sora Morimoto
165fd5f9ac Update fossa-contrib/fossa-action action to v2
Signed-off-by: Sora Morimoto <sora@morimoto.io>
2023-02-14 01:43:23 +09:00
Milos Gajdos
cf87e8d07e
Merge pull request #3818 from ckw017/ckw/fixseparatorregex
Fix separator regex to disallow empty strings
2023-01-31 08:15:13 +00:00
Aaron Lehmann
a811c1bb57 Log username on successful requests
Currently, "response completed with error" log lines include an
`auth.user.name` key, but successful "response completed" lines do not
include this, because they are logged a few stack frames up where
`auth.user.name` is not present on the `Context`. Move the successful
request logging inside the `dispatcher` closure, where the logger on the
context automatically includes this key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2023-01-30 09:32:39 -08:00
Milos Gajdos
9b629737cb
Merge pull request #3804 from thaJeztah/deprecate_schema1
manifest/schema1: mark docker manifest v2, schema 1 deprecated
2023-01-30 16:16:38 +00:00
Chris K. Wong
d1c186812e Fix separator regex
Signed-off-by: Chris K. Wong <chriskw.xyz@gmail.com>
2022-12-22 14:50:50 -08:00
Milos Gajdos
362910506b
Merge pull request #3816 from thaJeztah/bump_golang_net
vendor: golang.org/x/net v0.4.0
2022-12-08 16:53:59 +00:00
Sebastiaan van Stijn
345be95498
vendor: golang.org/x/net v0.4.0
golang.org/x/net contains a fix for CVE-2022-41717, which was addressed
in stdlib in go1.19.4 and go1.18.9;

> net/http: limit canonical header cache by bytes, not entries
>
> An attacker can cause excessive memory growth in a Go server accepting
> HTTP/2 requests.
>
> HTTP/2 server connections contain a cache of HTTP header keys sent by
> the client. While the total number of entries in this cache is capped,
> an attacker sending very large keys can cause the server to allocate
> approximately 64 MiB per open connection.
>
> This issue is also fixed in golang.org/x/net/http2 v0.4.0,
> for users manually configuring HTTP/2.

full diff: https://github.com/golang/net/compare/v0.2.0...v0.4.0

other dependency updates (due to (circular) dependencies):

- golang.org/x/sys v0.3.0: https://github.com/golang/sys/compare/3c1f35247d10...v0.3.0
- golang.org/x/text v0.5.0: https://github.com/golang/text/compare/v0.3.7...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 10:39:04 +01:00
Hayley Swimelar
7847817946
Merge pull request #3814 from thaJeztah/remove_fuzz_leftovers
remove script directory and fuzz left-overs
2022-12-05 15:46:40 -08:00
Sebastiaan van Stijn
59f1357775
remove script directory and fuzz left-overs
commit 9337b8df66 rewrote the fuzzers to
native go fuzzers, so the script was no longer needed. With this, the
script directory is no longer used, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 17:15:16 +01:00
Wang Yan
92d136e113
Merge pull request #3810 from thaJeztah/align_descriptor_with_oci
Descriptor: align field order with OCI image specification
2022-12-01 16:32:18 +08:00
Sebastiaan van Stijn
19233195b7
Align code to match order of fields
This is just cosmetic; alighn the fields with the order in which they appear
in the struct (and JSON output).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 23:24:10 +01:00
Sebastiaan van Stijn
86cd830fb3
Descriptor: align field order with OCI image specification
I am looking at aligning the types defined in this repository with the
OCI image specification, and potentially exchanging local types with
those from the specification.

This patch is a stepping-stone towards that effort, but as this changes
the format of the serialized JSON, I wanted to put this up first before
proceeding with the other work in case there are concerns.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 23:24:10 +01:00
Milos Gajdos
ac302d9ce5
Merge pull request #3807 from thaJeztah/replace_types_for_oci_step1
minor fixes and enhancements
2022-11-29 10:49:12 +00:00
Milos Gajdos
8cc58797e8
Merge pull request #3794 from AdamKorcz/fuzz1
Fuzzing: Rewrite existing fuzzers to native go fuzzers
2022-11-29 09:57:09 +00:00
Sebastiaan van Stijn
0b4311d5ce
manifest: improve test output and use const
Use consts to make clear these values are fixed, and improve the output
to make it clearer which part is the expected output, and which part
the actual.

Before this:

    === RUN   TestManifest
        manifest_test.go:87: manifest bytes not equal: "{\n   \"schemaVersion\": 2,\n   \"mediaType\": \"application/vnd.oci.image.manifest.v1+json\",\n   \"config\": {\n      \"mediaType\": \"application/vnd.oci.image.config.v1+json\",\n      \"size\": 985,\n      \"digest\": \"sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b\",\n      \"annotations\": {\n         \"apple\": \"orange\"\n      }\n   },\n   \"layers\": [\n      {\n         \"mediaType\": \"application/vnd.oci.image.layer.v1.tar+gzip\",\n         \"size\": 153263,\n         \"digest\": \"sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b\",\n         \"annotations\": {\n            \"lettuce\": \"wrap\"\n         }\n      }\n   ],\n   \"annotations\": {\n      \"hot\": \"potato\"\n   }\n}" != "{\n   \"schemaVersion\": 2,\n   \"mediaType\": \"application/vnd.oci.image.manifest.v1+json\",\n   \"config\": {\n      \"mediaType\": \"application/vnd.oci.image.config.v1+json\",\n      \"size\": 985,\n      \"digest\": \"sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b\",\n      \"annotations\": {\n         \"apple\": \"orange\"\n      }\n   },\n   \"layers\": [\n      {\n         \"mediaType\": \"application/vnd.oci.image.layer.v1.tar+gzip\",\n         \"size\": 153263,\n         \"digest\": \"sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b\",\n         \"annotations\": {\n            \"lettuce\": \"wrap\"\n         }\n      }\n   ],\n   \"annotations\": {\n      \"hot\": \"potato\"\n   }\n}"
    --- FAIL: TestManifest (0.00s)

After this:

    === RUN   TestManifest
        manifest_test.go:72: manifest bytes not equal:
            expected:
            {
               "schemaVersion": 2,
               "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
               "config": {
                  "mediaType": "application/vnd.docker.container.image.v1+json",
                  "size": 985,
                  "digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b"
               },
               "layers": [
                  {
                     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                     "size": 153263,
                     "digest": "sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b"
                  }
               ]
            }
            actual:
            {
               "schemaVersion": 2,
               "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
               "config": {
                  "mediaType": "application/vnd.docker.container.image.v1+json",
                  "size": 985,
                  "digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b"
               },
               "layers": [
                  {
                     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                     "size": 153263,
                     "digest": "sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b"
                  }
               ]
            }
    --- FAIL: TestManifest (0.00s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00
Sebastiaan van Stijn
f2db7faa2f
registry/storage: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00
Sebastiaan van Stijn
030489ca66
testutil: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:05 +01:00
Sebastiaan van Stijn
0e3efe749b
manifest: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 10:08:02 +01:00
Milos Gajdos
b9f9073d1a
Merge pull request #3805 from crazy-max/update-gha
ci: update github actions
2022-11-27 10:56:15 +00:00