Commit graph

5165 commits

Author SHA1 Message Date
baojiangnan
17952924f3 avoid redundant blob fetching
Signed-off-by: baojiangnan <baojn1998@163.com>
2023-09-18 10:40:25 +08:00
Milos Gajdos
612ad42609
Merge pull request #4040 from thaJeztah/move_api_errors 2023-09-15 09:36:36 +01:00
Milos Gajdos
73af930009
Merge pull request #4052 from thaJeztah/client_refactor_errhandling 2023-09-15 09:35:57 +01:00
Hayley Swimelar
b56fb385f6
Merge pull request #4055 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-12 08:52:39 -07:00
Sebastiaan van Stijn
23115ff634
update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-12 00:07:34 +02:00
Sebastiaan van Stijn
c8ba5d7081
registry/client: combine SuccessStatus and HandleErrorResponse
The SuccessStatus acted on the response's status code, and was used to return
early, before checking the same status code with HandleErrorResponse.

This patch combines both functions into a HandleHTTPResponseError, which
returns an error for "non-success" status-codes, which simplifies handling
of responses, and makes some logic slightly more idiomatic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:40:41 +02:00
Milos Gajdos
285b601af9
Merge pull request #4049 from distribution/dependabot/go_modules/github.com/cyphar/filepath-securejoin-0.2.4
Bump github.com/cyphar/filepath-securejoin from 0.2.3 to 0.2.4
2023-09-08 10:32:50 +01:00
dependabot[bot]
e4dd28b886
Bump github.com/cyphar/filepath-securejoin from 0.2.3 to 0.2.4
Bumps [github.com/cyphar/filepath-securejoin](https://github.com/cyphar/filepath-securejoin) from 0.2.3 to 0.2.4.
- [Release notes](https://github.com/cyphar/filepath-securejoin/releases)
- [Commits](https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4)

---
updated-dependencies:
- dependency-name: github.com/cyphar/filepath-securejoin
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-07 13:06:27 +00:00
Wang Yan
90939f1173
Merge pull request #4042 from milosgajdos/libtrust-handlers
Remove libtrust from handler tests
2023-09-05 15:59:38 +08:00
Milos Gajdos
612a30a7e7
Remove libtrust from handler tests
It was used for signing schema v1 manifests in tests which have now been
removed so there is no point in keeping these there anymore.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-05 08:41:29 +01:00
Milos Gajdos
6787846b9d
Merge pull request #4041 from milosgajdos/duplicate-code
Remove duplicate code that instruments Redis otel
2023-09-04 19:30:06 +01:00
Milos Gajdos
6baa31a273
Remove duplicate code that instruments Redis OTLP
This was somehow overlooked in https://github.com/distribution/distribution/pull/4019

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-04 17:52:38 +01:00
Sebastiaan van Stijn
0104adf4a8
registry/api/errcode: split Register to internal / exported
Use the non-exported function to all errors; there's currently no external
consumers of this function (perhaps it should be deprecated).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 18:03:00 +02:00
Sebastiaan van Stijn
292e30bc61
registry/api: move all errors to "errcode" package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 18:02:54 +02:00
Milos Gajdos
9790bc806c
Merge pull request #4037 from milosgajdos/enable-prealloc
Enable prealloc linter
2023-09-04 16:57:29 +01:00
Milos Gajdos
b6d0d3802e
Merge pull request #4036 from milosgajdos/s3-context
Propagate storage driver context to S3 API calls
2023-09-04 16:57:11 +01:00
Milos Gajdos
823cd4a370
Add a comment why prealloc linter is disabled when configuring endpoints
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 23:33:20 +01:00
Milos Gajdos
1089800643
Preallocate created slice in S3 tests
In case drvr.PutContent fails and returns error we'd have
some extra memory allocated, though in this case
(test with known size of the slice being iterated), that's fine.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 23:26:32 +01:00
Milos Gajdos
a9d31ec7b9
Avoid unnecessary type assertion in mfs driver
We already make sure the node in *dir

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 23:23:25 +01:00
Milos Gajdos
59fd8656ac
Enable prealloc linter
This will give us nice little performance gains in some code paths.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 22:41:51 +01:00
Milos Gajdos
dcdd8bb740
Propagate storage driver context to S3 API calls
Only some of the S3 storage driver calls were propagating context to the
S3 API calls. This commit updates the S3 storage drivers so the context
is propagated to all the S3 API calls.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 21:54:54 +01:00
Milos Gajdos
a2e65220ae
Merge pull request #4035 from milosgajdos/small-docs-cleanup 2023-09-02 08:30:24 +01:00
Milos Gajdos
0634160074
Remove outdated docs
This commit removes Registry v1 -> Registry v2 migration guide
as Registry v1 was deprecated long time ago and is no long longer
supported.

We also remove some references to "Future" roadmap which are wildly
outdated, too.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-01 11:04:33 +01:00
Wang Yan
88087f80ed
Merge pull request #4034 from Jamstah/custom-drivers
Add note on custom storage drivers
2023-09-01 16:16:09 +08:00
James Hewitt
8c7eea7621
Add note on custom storage drivers
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-31 15:56:53 +01:00
Milos Gajdos
2e50e6d2e6
Merge pull request #4033 from thaJeztah/remove_comment
go.mod: remove outdated comment
2023-08-31 15:39:46 +01:00
Milos Gajdos
db4bd9933e
Merge pull request #4031 from thaJeztah/migrate_reference
deprecate reference package, migrate to github.com/distribution/reference
2023-08-31 15:20:28 +01:00
Sebastiaan van Stijn
acf804a2dd
go.mod: remove outdated comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 16:10:15 +02:00
Sebastiaan van Stijn
152af63ec5
deprecate reference package, migrate to github.com/distribution/reference
This integrates the new module, which was extracted from this repository
at commit b9b19409cf458dcb9e1253ff44ba75bd0620faa6;

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    # create a temporary clone of docker
    cd ~/Projects
    git clone https://github.com/distribution/distribution.git reference
    cd reference

    # commit taken from
    git rev-parse --verify HEAD
    b9b19409cf

    # remove all code, except for general files, 'reference/', and rename to /
    git filter-repo \
      --path .github/workflows/codeql-analysis.yml \
      --path .github/workflows/fossa.yml \
      --path .golangci.yml \
      --path distribution-logo.svg \
      --path CODE-OF-CONDUCT.md \
      --path CONTRIBUTING.md \
      --path GOVERNANCE.md \
      --path README.md \
      --path LICENSE \
      --path MAINTAINERS \
      --path-glob 'reference/*.*' \
      --path-rename reference/:

    # initialize go.mod
    go mod init github.com/distribution/reference
    go mod tidy -go=1.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:47:06 +02:00
Milos Gajdos
17552d864d
Merge pull request #3685 from Jamstah/aws-paging
Work with the storage driver to minimise work when paging
2023-08-31 08:27:26 +01:00
Milos Gajdos
17ccf825b2
Merge pull request #4030 from liubin/fix-typos-in-paths
fix typos in registry/storage/paths.go
2023-08-30 17:06:54 +01:00
bin liu
2513dd1f96 fix typos in registry/storage/paths.go
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-30 22:36:14 +08:00
Milos Gajdos
b9b19409cf
Merge pull request #4028 from liubin/delete-duplicated-code 2023-08-30 07:45:51 +01:00
bin liu
eda5fe2d67 remove duplicated code
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-30 07:56:56 +08:00
Milos Gajdos
df16076366
Merge pull request #4023 from DavidSpek/remove-schema1-manifest
feat!: remove schema1 manifest and config options
2023-08-29 11:38:45 +01:00
James Hewitt
e22f7cbc73
Pass the last paging flag to storage drivers
Storage drivers may be able to take advantage of the hint to start
their walk more efficiently.

For S3: The API takes a start-after parameter. Registries with many
repositories can drastically reduce calls to s3 by telling s3 to only
list results lexographically after the last parameter.

For the fallback: We can start deeper in the tree and avoid statting
the files and directories before the hint in a walk. For a filesystem
this improves performance a little, but many of the API based drivers
are currently treated like a filesystem, so this drastically improves
the performance of GCP and Azure blob.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-29 11:27:42 +01:00
Milos Gajdos
3a44c2e10e
Merge pull request #4022 from Jamstah/catalog-optimisation
Don't make a new buffer for catalog listing
2023-08-29 11:24:39 +01:00
David van der Spek
0742b56677
feat!: remove schema1 manifest
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-29 12:17:44 +02:00
James Hewitt
a41613ba3a
Don't make a new buffer for catalog listing
We are given a slice to fill, write catalog entries directly to the
slice until it is full.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-29 11:11:13 +01:00
Milos Gajdos
b0276a2793
Merge pull request #4027 from liubin/add-redis-username
Add username to create redis client
2023-08-29 11:07:55 +01:00
bin liu
b889cc2eb4 Add username to create redis client
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-08-29 16:27:43 +08:00
Wang Yan
5f8b59177b
Merge pull request #4019 from milosgajdos/replace-redigo-redis
Replace redigo with redis-go
2023-08-29 09:53:32 +08:00
Milos Gajdos
884cf14d30
Merge pull request #4013 from milosgajdos/nonjson-error-client
Dont parse errors as JSON unless Content-Type is set to JSON
2023-08-28 14:28:48 +01:00
Milos Gajdos
45b7b9cec3
Dont parse errors as JSON unless Content-Type is set to JSON
Client attempts to parse the body of every error it receives as JSON
regardless of the content-type. This commit rectifies by only parsing
he error body as JSON if the Content-Type header is set to
either "application/json" or "application/vnd.api+json".

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-28 12:35:33 +01:00
Milos Gajdos
932be63dc2
Merge pull request #4002 from DavidSpek/remove-schemav1-storage
Remove references to schema1 pacakge from storage
2023-08-28 11:58:56 +01:00
David van der Spek
c7bdabadcf
add back getKeys + cleanup manifeststore test
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-28 12:44:49 +02:00
David van der Spek
f9bc9220eb
feat(storage)!: remove schema1 except manifeststore_test
Signed-off-by: David van der Spek <vanderspek.david@gmail.com>
2023-08-28 12:44:46 +02:00
Milos Gajdos
db460d2636
Merge pull request #4021 from Jamstah/go-1.20
Update to go 1.20
2023-08-28 11:42:35 +01:00
Milos Gajdos
e948e1b05d
Merge pull request #4018 from milosgajdos/makefile-cleanup
Small Makefie update
2023-08-28 11:30:39 +01:00
Milos Gajdos
d0c0b7bdd5
Update Makefile
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-08-28 11:19:01 +01:00