Commit graph

87 commits

Author SHA1 Message Date
Milos Gajdos
2c6b6482fc
Include headers when serving blob through proxy (#4273) 2024-05-14 14:27:09 +01:00
Milos Gajdos
6a9b0cfb71
Add support for Basic Authentication to proxyingRegistry (#4263)
Merging despite CodeQL warnings. see this for more details, why we decided to merge: https://github.com/github/codeql/issues/16486
2024-05-14 10:43:56 +01:00
Dimitar Kostadinov
062309c08b Stop proxy scheduler on system exit
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2024-05-13 17:01:35 +03:00
Dimitar Kostadinov
6ca646caad Initialize proxy prometheus counters values to 0 to prevent gaps after registry restart
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2024-02-21 14:35:49 +02:00
oliver-goetz
1e8ea03173
Add support for Basic Authentication to proxyingRegistry
Signed-off-by: oliver-goetz <o.goetz@sap.com>
2024-02-07 03:08:12 +01:00
Mikel Rychliski
041824555c Include headers when serving blob through proxy
In commit 17952924f3 we updated ServeBlob() to use an io.MultiWriter to
write simultaneously to the local store and the HTTP response.

However, copyContent was using a type assertion to only add headers if
the io.Writer was a http.ResponseWriter. Therefore, this change caused
us to stop sending the expected headers (i.e. Content-Length, Etag,
etc.) on the first request for a blob.

Resolve the issue by explicitly passing in http.Header and setting it
unconditionally.

Signed-off-by: Mikel Rychliski <mikel@mikelr.com>
2024-02-01 19:31:53 -05:00
Milos Gajdos
d8ff41a344
cleanup: move init funcs to the top of the source
We make sure they're not hiding at the bottom or in the middle
which makes debugging an utter nightmare!

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-11-28 06:50:48 +00:00
João Pereira
17872ebbc0
feat(linter): enable errcheck linter in golangci-lint (#4158) 2023-11-21 11:57:34 +00:00
Milos Gajdos
13fe08d87b
Fix proxy statistics (#4045) 2023-11-20 10:22:14 +00:00
Milos Gajdos
7ce129d63b
feat(linter): enable errcheck linter in golangci-lint
Also, bump the linter version to the latest available version.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-11-18 07:19:24 +00:00
Milos Gajdos
bd0e476910
Hide our misuses of contexts from the public interface (#4128) 2023-11-03 05:05:19 +00:00
Milos Gajdos
7430651536
cleanup: make blob sizes easier to understand
This is a followup to https://github.com/distribution/distribution/pull/4139

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-31 14:53:02 -07:00
Dimitar Kostadinov
bf933f546f Fix proxy statistics
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2023-10-31 12:16:41 +02:00
Cory Snider
d0f5aa670b Move context package internal
Our context package predates the establishment of current best practices
regarding context usage and it shows. It encourages bad practices such
as using contexts to propagate non-request-scoped values like the
application version and using string-typed keys for context values. Move
the package internal to remove it from the API surface of
distribution/v3@v3.0.0 so we are free to iterate on it without being
constrained by compatibility.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-27 10:58:37 -04:00
Milos Gajdos
daf3d00a32
Add prometheus proxy related metrics (#4047) 2023-10-26 16:11:10 +01:00
Cory Snider
cc23fdacff Move registry client internal
Our registry client is not currently in a good place to be used as the
reference OCI Distribution client implementation. But the registry proxy
currently depends on it. Make the registry client internal to the
distribution application to remove it from the API surface area (and any
implied compatibility promises) of distribution/v3@v3.0.0 without
breaking the proxy.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-24 12:26:38 -04:00
Dimitar Kostadinov
2ce5c81f47 Address PR review feedback
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2023-10-23 19:59:22 +03:00
Dimitar Kostadinov
9861a46d99 Add prometheus proxy related metrics
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2023-10-23 19:59:22 +03:00
baojiangnan
17952924f3 avoid redundant blob fetching
Signed-off-by: baojiangnan <baojn1998@163.com>
2023-09-18 10:40:25 +08:00
Milos Gajdos
59fd8656ac
Enable prealloc linter
This will give us nice little performance gains in some code paths.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-03 22:41:51 +01:00
Sebastiaan van Stijn
152af63ec5
deprecate reference package, migrate to github.com/distribution/reference
This integrates the new module, which was extracted from this repository
at commit b9b19409cf458dcb9e1253ff44ba75bd0620faa6;

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    # create a temporary clone of docker
    cd ~/Projects
    git clone https://github.com/distribution/distribution.git reference
    cd reference

    # commit taken from
    git rev-parse --verify HEAD
    b9b19409cf

    # remove all code, except for general files, 'reference/', and rename to /
    git filter-repo \
      --path .github/workflows/codeql-analysis.yml \
      --path .github/workflows/fossa.yml \
      --path .golangci.yml \
      --path distribution-logo.svg \
      --path CODE-OF-CONDUCT.md \
      --path CONTRIBUTING.md \
      --path GOVERNANCE.md \
      --path README.md \
      --path LICENSE \
      --path MAINTAINERS \
      --path-glob 'reference/*.*' \
      --path-rename reference/:

    # initialize go.mod
    go mod init github.com/distribution/reference
    go mod tidy -go=1.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:47:06 +02:00
James Hewitt
1a3e73cb84
Handle rand deprecations in go 1.20
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-08-28 09:33:12 +01:00
Milos Gajdos
ae0001f54d
Remove references to schema1 pacakge from proxy package
schema1 package was deprecated a while ago so we are removing
any references to it from the proxy package in preparation to
removing it from the codebase altogether.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-13 14:57:55 +01:00
Milos Gajdos
69023c7f85
Merge pull request #3880 from wzshiming/proxy-cache-configurable 2023-07-14 08:43:19 +01:00
Sebastiaan van Stijn
999527f978
Ignore SA1019: "schema1 is deprecated" linting errors
We need to use this for backward compatibility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Manish Tomar
8fe4ca4038 Option to configure proxy cache TTL
Currently when registry is run as proxy it tries to cleanup unused blobs
from its cache after 7 days which is hard-coded. This PR makes that
value configurable.

Co-authored-by: Shiming Zhang <wzshiming@foxmail.com>
Co-authored-by: Manish Tomar <manish.tomar@docker.com>
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2023-04-20 13:03:39 +08:00
Milos Gajdos
21622ca699
Merge pull request #3787 from thaJeztah/simplify_mocks 2022-11-18 08:53:26 +00:00
Sebastiaan van Stijn
019ead86f5
deprecate ReadSeekCloser in favor of io.ReadSeekCloser
Go's io package in stdlib now defines this interface, so we can switch
to using that instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-10 23:10:32 +01:00
Sebastiaan van Stijn
2cd52d5c0c
simplify mocks
Embed the interface that we're mocking; calling any of it's methods
that are not implemented will panic, so should give the same result
as before.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-10 17:09:24 +01:00
Sebastiaan van Stijn
f8b3af78fc
replace deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 23:47:52 +01:00
Hayley Swimelar
52d948a9f5
Merge pull request #3766 from thaJeztah/gofumpt
format code with gofumpt
2022-11-04 12:19:53 +01:00
Sebastiaan van Stijn
e0281dc609
format code with gofumpt
gofumpt (https://github.com/mvdan/gofumpt) provides a supserset of `gofmt` / `go fmt`,
and addresses various formatting issues that linters may be checking for.

We can consider enabling the `gofumpt` linter to verify the formatting in CI, although
not every developer may have it installed, so for now this runs it once to get formatting
in shape.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-03 22:48:20 +01:00
Sebastiaan van Stijn
f9ccd2c6ea
use http consts for request methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 23:31:47 +01:00
Aaron Lehmann
e36cb0a5d8 registry/storage/cache/memory: Use LRU cache to bound cache size
Instead of letting the cache grow without bound, use a LRU to impose a
size limit.

The limit is configurable through a new `blobdescriptorsize` config key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-07 07:20:06 -07:00
baojiangnan
4363fb1ef4 disable insecure cipher suites
This commit removes the following cipher suites that are known to be insecure:

TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
TLS_ECDHE_RSA_WITH_RC4_128_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256

And this commit deletes the tlsVersions of tls1.0 and tls1.1. The tls1.2 is the minimal supported tls version for creating a safer tls configuration.

Signed-off-by: david.bao <baojn1998@163.com>
2022-01-25 17:18:44 +08:00
baojiangnan
706f2170bd fix image cache incompletely
Signed-off-by: baojiangnan <baojn1998@163.com>
2022-01-14 23:58:44 +08:00
Sebastiaan van Stijn
1d33874951
go.mod: change imports to github.com/distribution/distribution/v3
Go 1.13 and up enforce import paths to be versioned if a project
contains a go.mod and has released v2 or up.

The current v2.x branches (and releases) do not yet have a go.mod,
and therefore are still allowed to be imported with a non-versioned
import path (go modules add a `+incompatible` annotation in that case).

However, now that this project has a `go.mod` file, incompatible
import paths will not be accepted by go modules, and attempting
to use code from this repository will fail.

This patch uses `v3` for the import-paths (not `v2`), because changing
import paths itself is a breaking change, which means that  the
next release should increment the "major" version to comply with
SemVer (as go modules dictate).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-08 18:30:46 +01:00
zhipengzuo
f361d443b7 clean up code because err is always nil
Signed-off-by: zhipengzuo <zuozhipeng@baidu.com>
2020-07-22 10:54:46 +08:00
Manish Tomar
da8db4666b Fix gometalint errors
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2019-02-04 16:01:04 -08:00
liyongxin
6133840f49 typo fix from existant to existent
Signed-off-by: liyongxin <yxli@alauda.io>
Signed-off-by: Yongxin Li <yxli@alauda.io>
2018-09-13 19:37:13 +08:00
Derek McGowan
ef859e1b21
Merge pull request #2474 from vikstrous/disable-v1-master
disable schema1 by default, add a config flag to enable it
2018-08-24 10:58:39 -07:00
Derek McGowan
db0a4ec1c8
Enable static checks
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 14:49:11 -07:00
Viktor Stanchev
e9864ce8b9 disable schema1 by default, add a config flag to enable it
port of #2473

Signed-off-by: Viktor Stanchev <me@viktorstanchev.com>
2017-12-19 10:23:25 -08:00
Stephen J Day
4abf680c76
registry: feed the linter by removing redundant err check
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-29 13:27:49 -07:00
Tibor Vass
2c58ce1a7f Remove context in NewRegistry and NewRepository
The context parameter was either not used or misused.

Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 10:34:56 -07:00
Stephen J Day
860b28c5b9
registry: remove dependency on logrus for client
To simplify the vendoring story for the client, we have now removed the
requirement for `logrus` and the forked `context` package (usually
imported as `dcontext`). We inject the logger via the metrics tracker
for the blob cache and via options on the token handler. We preserve
logs on the proxy cache for that case. Clients expecting these log
messages may need to be updated accordingly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-15 10:28:38 -07:00
Stephen J Day
9c88801a12
context: remove definition of Context
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.

To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-11 15:53:31 -07:00
Derek McGowan
d8fcbeeb71
Update registry to use WithName for creating Named values
The registry uses partial Named values which the named parsers
no longer support. To allow the registry service to continue
to operate without canonicalization, switch to use WithName.
In the future, the registry should start using fully canonical
values on the backend and WithName should no longer support
creating partial values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-01-18 13:56:56 -08:00
Stephen J Day
532ec9f036
digest: migrate to opencontainers/go-digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 15:42:03 -08:00
Derek McGowan
a1a73884f9
Refactor authorization challenges to its own package
Split challenges into its own package. Avoids possible
import cycle with challenges from client.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-09 15:27:40 -08:00