Several storage drivers and storage middlewares need to introspect the
client HTTP request in order to construct content-redirect URLs. The
request is indirectly passed into the driver interface method URLFor()
through the context argument, which is bad practice. The request should
be passed in as an explicit argument as the method is only called from
request handlers.
Replace the URLFor() method with a RedirectURL() method which takes an
HTTP request as a parameter instead of a context. Drop the options
argument from URLFor() as in practice it only ever encoded the request
method, which can now be fetched directly from the request. No URLFor()
callers ever passed in an "expiry" option, either.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The specifics of how the authorization for a request is propagated
through the registry app are private implementation details. Hide those
details from outsiders so they can be changed as needed without fear of
breaking third-party code. Move the utilities for attaching a request's
authorization status to its context and retrieving it from the context
into the registry/handlers package as unexported symbols.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The details of how request-scoped information is propagated through the
registry server app should be left as private implementation details so
they can be changed without fear of breaking compatibility with
third-party code which imports the distribution module. The
AccessController interface unnecessarily bakes into the public API
details of how authorization grants are propagated through request
contexts. In practice the only values the in-tree authorizers attach to
the request contexts are the UserInfo and Resources for the request.
Change the AccessController interface to return the UserInfo and
Resources directly to allow us to change how request contexts are used
within the app without altering the AccessController interface contract.
Signed-off-by: Cory Snider <csnider@mirantis.com>
The RemoteAddr and RemoteIP functions operate on *http.Request values,
not contexts. They have very low cohesion with the rest of the package.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Our context package predates the establishment of current best practices
regarding context usage and it shows. It encourages bad practices such
as using contexts to propagate non-request-scoped values like the
application version and using string-typed keys for context values. Move
the package internal to remove it from the API surface of
distribution/v3@v3.0.0 so we are free to iterate on it without being
constrained by compatibility.
Signed-off-by: Cory Snider <csnider@mirantis.com>
This commit make the S3 driver chunk size constants more straightforward
to understand -- instead of remembering the bit shifts we make this more
explicit.
We are also updating append parameter to the `(writer).Write` to follow
the new convention we are trying to establish.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
* include storage integration tests in the build matrix
* add a new CI job that runs E2E tests backed by S3 storage driver
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
This addition enables pushing distribution images into GHCR.
This is in addition to the Docker Hub push which remains in place.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
Nowadays there are much, much better UUID implementations to choose
from, such as github.com/google/uuid. Prevent external users from
importing our bespoke implementation so that we can change or migrate
away from it internally without introducing breaking changes.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Our registry client is not currently in a good place to be used as the
reference OCI Distribution client implementation. But the registry proxy
currently depends on it. Make the registry client internal to the
distribution application to remove it from the API surface area (and any
implied compatibility promises) of distribution/v3@v3.0.0 without
breaking the proxy.
Signed-off-by: Cory Snider <csnider@mirantis.com>
docker/libtrust repository has been archived for several years now.
This commit replaces all the libtrust JWT machinery with go-jose/go-jose module.
Some of the code has been adopted from libtrust and adjusted for some of
the use cases covered by the token authorization flow especially in the
tests.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
This commit changes storagedriver.Filewriter interface
by adding context.Context as an argument to its Commit
func.
We pass the context appropriately where need be throughout
the distribution codebase to all the writers and tests.
S3 driver writer unfortunately must maintain the context
passed down to it from upstream so it contnues to
implement io.Writer and io.Closer interfaces which do not
allow accepting the context in any of their funcs.
Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
go1.20.10 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.20.10 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.20.10+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.20.9...go1.20.10
From the security mailing:
[security] Go 1.21.3 and Go 1.20.10 are released
Hello gophers,
We have just released Go versions 1.21.3 and 1.20.10, minor point releases.
These minor releases include 1 security fixes following the security policy:
- net/http: rapid stream resets can cause excessive work
A malicious HTTP/2 client which rapidly creates requests and
immediately resets them can cause excessive server resource consumption.
While the total number of requests is bounded to the
http2.Server.MaxConcurrentStreams setting, resetting an in-progress
request allows the attacker to create a new request while the existing
one is still executing.
HTTP/2 servers now bound the number of simultaneously executing
handler goroutines to the stream concurrency limit. New requests
arriving when at the limit (which can only happen after the client
has reset an existing, in-flight request) will be queued until a
handler exits. If the request queue grows too large, the server
will terminate the connection.
This issue is also fixed in golang.org/x/net/http2 v0.17.0,
for users manually configuring HTTP/2.
The default stream concurrency limit is 250 streams (requests)
per HTTP/2 connection. This value may be adjusted using the
golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
setting and the ConfigureServer function.
This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
This is also tracked by CVE-2023-44487.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.20.9 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the go command and the linker. See the Go 1.20.9
milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.20.9+label%3ACherryPickApproved
full diff: https://github.com/golang/go/compare/go1.20.8...go1.20.9
From the security mailing:
[security] Go 1.21.2 and Go 1.20.9 are released
Hello gophers,
We have just released Go versions 1.21.2 and 1.20.9, minor point releases.
These minor releases include 1 security fixes following the security policy:
- cmd/go: line directives allows arbitrary execution during build
"//line" directives can be used to bypass the restrictions on "//go:cgo_"
directives, allowing blocked linker and compiler flags to be passed during
compliation. This can result in unexpected execution of arbitrary code when
running "go build". The line directive requires the absolute path of the file in
which the directive lives, which makes exploting this issue significantly more
complex.
This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
An incorrect alias snuck into 152af63ec5,
and DomainRegexp was aliased to the regex for digests (DigestRegexp).
This didn't affect this repository, as it didn't use the aliases and migrated
to the new module, but does affect user of the old module that depend on the
aliases.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Small refactoring of storagedriver errors.
We change the Enclosed field to Detail and make sure
Errors get properly serialized to JSON.
We also add tests.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
Nested files aren't supported on MinIO, and as our storage layout is
filesystem based, we don't actually use nest files in the code.
Remove the test so that we can support MinIO.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
This fixes some of the tests for minio.
The walk tests needs a version of minio that contains https://github.com/minio/minio/pull/18099
The storage classes minio supports are a subset of the s3 classes.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
