Commit graph

5440 commits

Author SHA1 Message Date
Milos Gajdos
d3cc664fa2
Update docs: JWKS credentials and AZ identity
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-07-06 10:13:29 +01:00
Milos Gajdos
4dd0ac977e
feat: implement 'rewrite' storage middleware (#4146) 2024-07-04 16:16:29 +01:00
Milos Gajdos
306f4ff71e
Replace custom Redis config struct with go-redis UniversalOptions (adds sentinel & cluster support) (#4306) 2024-07-04 16:00:37 +01:00
Andrey Smirnov
558ace1391
feat: implement 'rewrite' storage middleware
This allows to rewrite 'URLFor' of the storage driver to use a specific
host/trim the base path.

It is different from the 'redirect' middleware, as it still calls the
storage driver URLFor.

For example, with Azure storage provider, this allows to transform the
SAS Azure Blob Storage URL into the URL compatible with Azure Front
Door.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-04 18:49:25 +04:00
Milos Gajdos
6d5911900a
Update Redis configuration docs with TLS options
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-07-04 15:44:41 +01:00
Milos Gajdos
3a8499541a
docs: disable base element override (#4391) 2024-07-04 09:00:57 +01:00
Milos Gajdos
10d90f7290
remove layer's link file by gc (#4344) 2024-07-02 18:08:56 +01:00
Liang Zheng
d9050bb917 remove layer's link file by gc
The garbage-collect should remove unsed layer link file

P.S. This was originally contributed by @m-masataka, now I would like to take over it.
Thanks @m-masataka efforts with PR https://github.com/distribution/distribution/pull/2288

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2024-07-03 00:16:11 +08:00
Milos Gajdos
2b036a9fc1
Update dockerhub.md (#4394) 2024-07-01 19:04:39 +01:00
Mahmoud Kandil
43a64480ef
Update dockerhub.md
Signed-off-by: Mahmoud Kandil <47168819+MahmoudKKandil@users.noreply.github.com>
2024-07-01 13:53:43 +03:00
David Karlsson
f36b44ff73 docs: disable base element override
Setting the HTML <base> element causes page-internal links to point to
the root of the website, rather than local anchors on the same page.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-07-01 10:07:44 +02:00
Milos Gajdos
a008d360b4
Create type alias for redis.UniversalOptions
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-06-30 11:20:51 +01:00
Milos Gajdos
f27799d1aa
Add custom TLS config to Redis
We also update the Redis TLS config initialization in the app.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-06-28 22:03:22 +01:00
Milos Gajdos
5f804a9df7
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity from 1.3.0 to 1.6.0 (#4380) 2024-06-26 09:39:21 +01:00
Anders Ingemann
b63cbb3318
Replace custom Redis config struct with go-redis UniversalOptions
Huge help from @milosgajdos who figured out how to do the entire
marshalling/unmarshalling for the configs

Signed-off-by: Anders Ingemann <aim@orbit.online>
2024-06-14 10:31:09 +02:00
dependabot[bot]
050e1a3ee7
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.3.0 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.0...sdk/azcore/v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-11 20:09:16 +00:00
Milos Gajdos
e1ec19ae60
New path for distribution config (#4365) 2024-06-11 12:19:40 +01:00
Milos Gajdos
675d7e27f5
feature: Bump go-jose and require signing algorithms in auth (#4349) 2024-05-30 20:54:20 +01:00
Milos Gajdos
52d68216c0
feature: Bump go-jose and require signing algorithms in auth
This bumps go-jose to the latest available version: v4.0.3.
This slightly breaks the backwards compatibility with the existing
registry deployments but brings more security with it.

We now require the users to specify the list of token signing algorithms in
the configuration. We do strive to maintain the b/w compat by providing
a list of supported algorithms, though, this isn't something we
recommend due to security issues, see:
* https://github.com/go-jose/go-jose/issues/64
* https://github.com/go-jose/go-jose/pull/69

As part of this change we now return to the original flow of the token
signature validation:
1. X2C (tls) headers
2. JWKS
3. KeyID

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-05-30 20:44:35 +01:00
Milos Gajdos
975613d4a0
New path for distribution config
The original path was referencing a docker directory which no longer
makes much sense.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-05-29 22:05:22 +01:00
Milos Gajdos
37b83869a9
Add option to enable sparse indexes (#3536) 2024-05-28 10:15:02 +01:00
James Hewitt
c40c4b289a
Enable configuration of index dependency validation
Enable configuration options that can selectively disable validation
that dependencies exist within the registry before the image index
is uploaded.

This enables sparse indexes, where a registry holds a manifest index that
could be signed (so the digest must not change) but does not hold every
referenced image in the index. The use case for this is when a registry
mirror does not need to mirror all platforms, but does need to maintain
the digests of all manifests either because they are signed or because
they are pulled by digest.

The registry administrator can also select specific image architectures
that must exist in the registry, enabling a registry operator to select
only the platforms they care about and ensure all image indexes uploaded
to the registry are valid for those platforms.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2024-05-28 09:56:14 +01:00
Milos Gajdos
e0a54de7fc
Add a go.mod toolchain version (#4347) 2024-05-16 19:51:27 +01:00
Milos Gajdos
ad69db3fd5
docs: update location of filesystem.md (#4355) 2024-05-16 14:14:00 +01:00
Emmanuel Ferdman
119c608fad
docs: update location of filesystem.md
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2024-05-16 15:43:41 +03:00
Milos Gajdos
2c6b6482fc
Include headers when serving blob through proxy (#4273) 2024-05-14 14:27:09 +01:00
Milos Gajdos
6a9b0cfb71
Add support for Basic Authentication to proxyingRegistry (#4263)
Merging despite CodeQL warnings. see this for more details, why we decided to merge: https://github.com/github/codeql/issues/16486
2024-05-14 10:43:56 +01:00
Milos Gajdos
56a020f7f1
Stop proxy scheduler on system exit (#4293) 2024-05-13 17:31:23 +01:00
Dimitar Kostadinov
062309c08b Stop proxy scheduler on system exit
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2024-05-13 17:01:35 +03:00
James Hewitt
421a359b26
Add a go.mod toolchain version
go 1.21 added toolchain support. We should now specify a toolchain
version in go.mod.

https://go.dev/doc/toolchain

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2024-05-13 14:47:07 +01:00
Milos Gajdos
c49220d492
Fix #2902: ‘autoRedirect’ hardcode ‘https’ scheme (#2903) 2024-05-04 15:32:25 +01:00
Milos Gajdos
cb3a2010c4
Set readStartAtFile context aware for purge uploads (#4339) 2024-05-02 19:00:43 +01:00
Sylvain DESGRAIS
f1875862cf Set readStartAtFile context aware for purge uploads
Signed-off-by: Sylvain DESGRAIS <sylvain.desgrais@gmail.com>
2024-05-02 11:06:39 +02:00
Milos Gajdos
c8e22f6723
Add Shutdown method to registry.Registry (#4338) 2024-05-01 15:05:44 +01:00
Robin Ketelbuters
16a305ebaf Add registry.Shutdown method for graceful shutdown of embedded registry
Signed-off-by: Robin Ketelbuters <robin.ketelbuters@gmail.com>
2024-04-29 20:18:58 +02:00
Milos Gajdos
e0795fcfe3
add bounded concurrency for tag lookup and untag (#4329) 2024-04-26 19:59:59 +01:00
Liang Zheng
a2afe23f38 add concurrency limits for tag lookup and untag
Harbor is using the distribution for it's (harbor-registry) registry component.
The harbor GC will call into the registry to delete the manifest, which in turn
then does a lookup for all tags that reference the deleted manifest.
To find the tag references, the registry will iterate every tag in the repository
and read it's link file to check if it matches the deleted manifest (i.e. to see
if uses the same sha256 digest). So, the more tags in repository, the worse the
performance will be (as there will be more s3 API calls occurring for the tag
directory lookups and tag file reads).

Therefore, we can use concurrent lookup and untag to optimize performance as described in https://github.com/goharbor/harbor/issues/12948.

P.S. This optimization was originally contributed by @Antiarchitect, now I would like to take it over.
Thanks @Antiarchitect's efforts with PR https://github.com/distribution/distribution/pull/3890.

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2024-04-26 22:32:21 +08:00
Liang Zheng
a5882d6646 vendor: update manifest dependencies
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2024-04-26 22:22:49 +08:00
Kyle Squizzato
47a9dac250
fix: ignore error of manifest tag path not found in gc (#4331) 2024-04-25 10:25:54 -07:00
Liang Zheng
112156321f fix: ignore error of manifest tag path not found in gc
it is reasonable to ignore the error that the manifest tag path does not exist when querying
all tags of the specified repository when executing gc.

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2024-04-25 17:13:06 +08:00
Milos Gajdos
e6d1d182bf
Allow setting s3 forcepathstyle without regionendpoint (#4291) 2024-04-24 08:34:01 +01:00
Milos Gajdos
03e58dfcf8
chore: fix some typos in comments (#4335) 2024-04-24 08:33:32 +01:00
Milos Gajdos
d61d8ebc16
build(deps): bump golang.org/x/net from 0.20.0 to 0.23.0 (#4333) 2024-04-23 16:18:48 +01:00
guoguangwu
2fe3442035 chore: fix some typos in comments
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
2024-04-23 17:48:53 +08:00
Milos Gajdos
e8ea4e5951
chore: fix some typos in comments (#4332) 2024-04-23 09:03:51 +01:00
Milos Gajdos
bdd3d31fae
proxy: Do not configure HTTP secret for proxy registry (#4305) 2024-04-23 08:17:50 +01:00
goodactive
e0a1ce14a8 chore: fix some typos in comments
Signed-off-by: goodactive <goodactive@qq.com>
2024-04-23 12:04:03 +08:00
Milos Gajdos
df98374764
Fix garbage-collect --delete-untagged to handle schema 2 manifest list and OCI image index (#4285) 2024-04-21 09:18:41 +01:00
Anthony Ramahay
601b37d98b Handle OCI image index and V2 manifest list during garbage collection
Signed-off-by: Anthony Ramahay <thewolt@gmail.com>
2024-04-20 16:41:50 +02:00
dependabot[bot]
2db0a598cc
build(deps): bump golang.org/x/net from 0.20.0 to 0.23.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-19 12:59:08 +00:00