Commit graph

5248 commits

Author SHA1 Message Date
Milos Gajdos
5aee8e1917
feat: Add context to storagedriver.(Filewriter).Commit() (#4109) 2023-10-19 11:41:55 +01:00
Milos Gajdos
cb0d083d8d
feat: Add context to storagedriver.(Filewriter).Commit()
This commit changes storagedriver.Filewriter interface
by adding context.Context as an argument to its Commit
func.

We pass the context appropriately where need be throughout
the distribution codebase to all the writers and tests.

S3 driver writer unfortunately must maintain the context
passed down to it from upstream so it contnues to
implement io.Writer and io.Closer interfaces which do not
allow accepting the context in any of their funcs.

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-19 11:27:27 +01:00
Milos Gajdos
5ad2c45b8c
update to go1.20.10, test go1.21.3 (#4116) 2023-10-19 10:54:01 +01:00
Sebastiaan van Stijn
46d13ff75b
update to go1.20.10, test go1.21.3
go1.20.10 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.20.10 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.9...go1.20.10

From the security mailing:

[security] Go 1.21.3 and Go 1.20.10 are released

Hello gophers,

We have just released Go versions 1.21.3 and 1.20.10, minor point releases.

These minor releases include 1 security fixes following the security policy:

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 10:45:12 +02:00
Sebastiaan van Stijn
9cc6e5b27f
update to go1.20.9, test go1.21.2
go1.20.9 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the go command and the linker. See the Go 1.20.9
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.9+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.8...go1.20.9

From the security mailing:

[security] Go 1.21.2 and Go 1.20.9 are released

Hello gophers,

We have just released Go versions 1.21.2 and 1.20.9, minor point releases.

These minor releases include 1 security fixes following the security policy:

- cmd/go: line directives allows arbitrary execution during build

  "//line" directives can be used to bypass the restrictions on "//go:cgo_"
  directives, allowing blocked linker and compiler flags to be passed during
  compliation. This can result in unexpected execution of arbitrary code when
  running "go build". The line directive requires the absolute path of the file in
  which the directive lives, which makes exploting this issue significantly more
  complex.

  This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 10:44:38 +02:00
Milos Gajdos
5592968cbc
reference: fix broken alias for DomainRegexp (#4113) 2023-10-19 09:25:17 +01:00
Sebastiaan van Stijn
c8c2bc279c
reference: fix broken alias for DomainRegexp
An incorrect alias snuck into 152af63ec5,
and DomainRegexp was aliased to the regex for digests (DigestRegexp).

This didn't affect this repository, as it didn't use the aliases and migrated
to the new module, but does affect user of the old module that depend on the
aliases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 10:06:38 +02:00
Wang Yan
d24243730f
refactor: Storage driver errors (#4108) 2023-10-19 01:12:14 +08:00
Milos Gajdos
ea41722902
refactor: Storage driver errors
Small refactoring of storagedriver errors.
We change the Enclosed field to Detail and make sure
Errors get properly serialized to JSON.
We also add tests.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-18 10:02:21 +01:00
Milos Gajdos
915ad2d5a6
json encode storage driver enclosed error (#4099) 2023-10-17 21:44:42 +01:00
James Hewitt
eac199875e
Remove test for nested file delete on S3
Nested files aren't supported on MinIO, and as our storage layout is
filesystem based, we don't actually use nest files in the code.

Remove the test so that we can support MinIO.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-17 09:13:15 +01:00
James Hewitt
647ec33c33
Bump minio version and test less storage classes
This fixes some of the tests for minio.

The walk tests needs a version of minio that contains https://github.com/minio/minio/pull/18099

The storage classes minio supports are a subset of the s3 classes.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-17 02:10:43 +01:00
Milos Gajdos
1d410148ef
Update dockerhub-readme GH Action (#4105) 2023-10-16 19:10:39 +01:00
Milos Gajdos
71217a0c7b
Update docs GHA and renamed a doc file (#4102) 2023-10-16 13:59:03 +01:00
Milos Gajdos
078c0546a4
Add annotation for descriptor (#4106) 2023-10-16 13:36:07 +01:00
Tosone
4dce8b866e Add annotation for descriptor
Signed-off-by: Tosone <i@tosone.cn>
2023-10-16 19:03:12 +08:00
Milos Gajdos
777ad03208
Update docs GHA
Add missing steps to the job, pick up the path automatically, trigger
the job on config file changes.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-15 11:03:23 +01:00
Milos Gajdos
4f506663d3
Update dockerhub-readme GH Action
We were missing GitHub checkout step before running the update.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-14 18:16:01 +01:00
Milos Gajdos
c78d6f99ae
Move dockerhub-readme workflow to the correct path (#4103) 2023-10-14 18:09:52 +01:00
Milos Gajdos
dc07c42810
Move dockerhub-readme workflow to the correct path
We've incorrectly added the dockerhub-readme workflow into .github
path from where it can not be triggered:

https://docs.github.com/en/actions/using-workflows/triggering-a-workflow

This commit addresses it and update the workflow paths.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-13 11:05:12 +01:00
Milos Gajdos
ebba01efea
docs: add hugo website (#4101) 2023-10-12 17:08:54 +01:00
David Karlsson
a66f6c37cb ci: add github pages workflow for docs
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 13:44:58 +02:00
James Hewitt
c3ae793f85
And other content...
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-12 12:00:15 +01:00
James Hewitt
83dd4ff0a6
Cleanup of naming in docs
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-12 11:39:36 +01:00
David Karlsson
31707d54f3 docs: add github link in header
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 10:00:43 +02:00
David Karlsson
1596da6813 docs: add tests
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 09:32:55 +02:00
David Karlsson
b911020c1f docs: fix markup and broken links
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 09:32:37 +02:00
Milos Gajdos
fe98c99860
Bump golang.org/x/net from 0.8.0 to 0.17.0 (#4100) 2023-10-12 08:13:49 +01:00
Glyn Owen Hanmer
fee6faef70 json encode storage driver enclosed error
Signed-off-by: Glyn Owen Hanmer <1295698+glynternet@users.noreply.github.com>
2023-10-11 17:53:27 -06:00
dependabot[bot]
758c0f9d77
Bump golang.org/x/net from 0.8.0 to 0.17.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.8.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.8.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 23:39:45 +00:00
David Karlsson
e2ae76f1f2 docs: add hugo site
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-11 16:45:16 +02:00
Milos Gajdos
f7b3869062
Merge pull request #4091 from dvdksn/docs-jwt-rendering-bug
docs: remove blank line
2023-10-04 14:35:49 +01:00
David Karlsson
6183f23092 docs: remove blank line
This blank line confuses the markdown parser to think
that this is an indented code block.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-04 15:20:18 +02:00
Milos Gajdos
a70964c2fc
Merge pull request #4076 from flavianmissi/s3-loglevel
registry: add loglevel support for aws s3 storage driver
2023-10-04 14:13:15 +01:00
Milos Gajdos
ed8423176f
Merge pull request #4081 from liubin/fix/refactor-redis
refactor redis cache
2023-10-03 16:01:07 +01:00
Milos Gajdos
a2101447d4
Merge pull request #4087 from milosgajdos/update-dockerhub-docs
Update Docker Hub README and keep it in sync with this repository.
2023-10-02 22:22:02 +01:00
Milos Gajdos
f2a72d7f77
Update Docker Hub README and keep it in sync with this repository.
This commit
* adds a new docs page (`dockerhub.md`) that contains Docker Hub README
* updates the default config that gets backed into the docker image
* updates CI with a new workflow job that keeps Docker Hub README in
  sync with the contents of the `docs/dockerhub.md` file

Co-authored-by: CrazyMax <github@crazymax.dev>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-02 22:03:35 +01:00
Milos Gajdos
93a64460fe
Merge pull request #4086 from milosgajdos/indent-prometheus-docs
Properly indent prometheus docs
2023-10-02 20:40:14 +01:00
Milos Gajdos
504a3bafc5
Properly indent prometheus docs
Incorrect section indentation of the prometheus docs confuses some
folks. This commit fixes that by indenting the prometheus section
under the debug configuration section.

Co-authored-by: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-02 18:03:14 +01:00
Milos Gajdos
e4d98bf5d0
Merge pull request #4089 from ialidzhikov/enh/debug-endpoint
Add few more sentences for the debug endpoint
2023-10-02 16:58:57 +01:00
Flavian Missi
3df7e28f44 registry: add loglevel support for aws s3 storage driver
based on the work from
https://github.com/distribution/distribution/pull/3057.

Co-authored-by: Simon Compston <compston@gmail.com>
Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-10-02 15:47:02 +02:00
ialidzhikov
993af6fefd Add few more sentences for the debug endpoint
Initially I misunderstood that the debug endpoint has to be disabled in production environments. That's why I created https://github.com/distribution/distribution/issues/4084 and https://github.com/distribution/distribution/issues/4085.
But it turns out that the docs want to state the the debug endpoint should not be exposed publicly to the internet.

Signed-off-by: ialidzhikov <i.alidjikov@gmail.com>
2023-10-02 11:10:15 +03:00
Milos Gajdos
735c161b53
Merge pull request #4066 from milosgajdos/optimise-s3-push
Optimise push in S3 driver
2023-09-29 13:47:20 +01:00
Milos Gajdos
23083ac9d2
Merge pull request #4077 from liubin/fix/use-manifestTagsPathSpec-all-tag-all
use manifestTagsPathSpec for listing all tags
2023-09-28 17:37:25 +01:00
Milos Gajdos
4fce3c0028
Move completedParts type back to the original position
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-28 15:58:02 +01:00
Milos Gajdos
73e7952c36
Merge pull request #4080 from liubin/fix/comment-typos
fix comment typos
2023-09-28 15:50:06 +01:00
bin liu
46a9da160e refactor redis cache
This commit removes some `conn` parameters of private functions, which can
be obtain from the struct itself. The `conn` is for the old `redisgo` library,
which is replaced by `go-redis` in #4019.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 18:40:50 +08:00
bin liu
dca71db976 fix comment typos
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 17:48:21 +08:00
bin liu
6c724a1a95 use manifestTagsPathSpec for listing all tags
In terms of results, a`manifestTagsPathSpec{ name: "repo" }` equals
`manifestTagPathSpec{ name: "repo", tag: "" }`, but from the intention,
the `manifestTagsPathSpec` should be used.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 10:44:46 +08:00
Milos Gajdos
3fc1216dc3
Merge pull request #4072 from NeilW/zero-byte-append-check
driver testsuite: Add zero byte file checks
2023-09-27 22:04:05 +01:00