Commit graph

1885 commits

Author SHA1 Message Date
Andrey Smirnov
558ace1391
feat: implement 'rewrite' storage middleware
This allows to rewrite 'URLFor' of the storage driver to use a specific
host/trim the base path.

It is different from the 'redirect' middleware, as it still calls the
storage driver URLFor.

For example, with Azure storage provider, this allows to transform the
SAS Azure Blob Storage URL into the URL compatible with Azure Front
Door.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-04 18:49:25 +04:00
Milos Gajdos
3a8499541a
docs: disable base element override (#4391) 2024-07-04 09:00:57 +01:00
Mahmoud Kandil
43a64480ef
Update dockerhub.md
Signed-off-by: Mahmoud Kandil <47168819+MahmoudKKandil@users.noreply.github.com>
2024-07-01 13:53:43 +03:00
David Karlsson
f36b44ff73 docs: disable base element override
Setting the HTML <base> element causes page-internal links to point to
the root of the website, rather than local anchors on the same page.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-07-01 10:07:44 +02:00
Milos Gajdos
e1ec19ae60
New path for distribution config (#4365) 2024-06-11 12:19:40 +01:00
Milos Gajdos
675d7e27f5
feature: Bump go-jose and require signing algorithms in auth (#4349) 2024-05-30 20:54:20 +01:00
Milos Gajdos
52d68216c0
feature: Bump go-jose and require signing algorithms in auth
This bumps go-jose to the latest available version: v4.0.3.
This slightly breaks the backwards compatibility with the existing
registry deployments but brings more security with it.

We now require the users to specify the list of token signing algorithms in
the configuration. We do strive to maintain the b/w compat by providing
a list of supported algorithms, though, this isn't something we
recommend due to security issues, see:
* https://github.com/go-jose/go-jose/issues/64
* https://github.com/go-jose/go-jose/pull/69

As part of this change we now return to the original flow of the token
signature validation:
1. X2C (tls) headers
2. JWKS
3. KeyID

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-05-30 20:44:35 +01:00
Milos Gajdos
975613d4a0
New path for distribution config
The original path was referencing a docker directory which no longer
makes much sense.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2024-05-29 22:05:22 +01:00
James Hewitt
c40c4b289a
Enable configuration of index dependency validation
Enable configuration options that can selectively disable validation
that dependencies exist within the registry before the image index
is uploaded.

This enables sparse indexes, where a registry holds a manifest index that
could be signed (so the digest must not change) but does not hold every
referenced image in the index. The use case for this is when a registry
mirror does not need to mirror all platforms, but does need to maintain
the digests of all manifests either because they are signed or because
they are pulled by digest.

The registry administrator can also select specific image architectures
that must exist in the registry, enabling a registry operator to select
only the platforms they care about and ensure all image indexes uploaded
to the registry are valid for those platforms.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2024-05-28 09:56:14 +01:00
Emmanuel Ferdman
119c608fad
docs: update location of filesystem.md
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2024-05-16 15:43:41 +03:00
Milos Gajdos
c49220d492
Fix #2902: ‘autoRedirect’ hardcode ‘https’ scheme (#2903) 2024-05-04 15:32:25 +01:00
Liang Zheng
a2afe23f38 add concurrency limits for tag lookup and untag
Harbor is using the distribution for it's (harbor-registry) registry component.
The harbor GC will call into the registry to delete the manifest, which in turn
then does a lookup for all tags that reference the deleted manifest.
To find the tag references, the registry will iterate every tag in the repository
and read it's link file to check if it matches the deleted manifest (i.e. to see
if uses the same sha256 digest). So, the more tags in repository, the worse the
performance will be (as there will be more s3 API calls occurring for the tag
directory lookups and tag file reads).

Therefore, we can use concurrent lookup and untag to optimize performance as described in https://github.com/goharbor/harbor/issues/12948.

P.S. This optimization was originally contributed by @Antiarchitect, now I would like to take it over.
Thanks @Antiarchitect's efforts with PR https://github.com/distribution/distribution/pull/3890.

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2024-04-26 22:32:21 +08:00
Milos Gajdos
e6d1d182bf
Allow setting s3 forcepathstyle without regionendpoint (#4291) 2024-04-24 08:34:01 +01:00
Benjamin Schanzel
8654a0ee45
Allow setting s3 forcepathstyle without regionendpoint
Currently, the `forcepathstyle` parameter for the s3 storage driver is
considered only if the `regionendpoint` parameter is set. Since setting
a region endpoint explicitly is discouraged with AWS s3, it is not clear
how to enforce path style URLs with AWS s3.
This also means, that the default value (true) only applies if a region
endpoint is configured.

This change makes sure we always forward the `forcepathstyle` parameter
to the aws-sdk if present in the config. This is a breaking change where
a `regionendpoint` is configured but no explicit `forcepathstyle` value
is set.

Signed-off-by: Benjamin Schanzel <benjamin.schanzel@bmw.de>
2024-04-08 12:45:26 +02:00
Tadeusz Dudkiewicz
de450c903a update: support redirects in gcs storage with default credentials
Signed-off-by: Tadeusz Dudkiewicz <tadeusz.dudkiewicz@rtbhouse.com>
2024-03-11 21:05:03 +01:00
icefed
63eb22d74b
Fix: ‘autoRedirect’ hardcode ‘https’ scheme
Signed-off-by: icefed <zlwangel@gmail.com>
2024-03-05 20:50:09 +08:00
Chad Faragher
1c3d44eccd
Update notifications.md
_setup_ is a noun , _set up_ is the verb.

Signed-off-by: Chad Faragher <wyckster@hotmail.com>
2024-02-28 13:32:59 -05:00
James Hewitt
5bebd152be
Add a trademarks and docs license link
Fixes #4264

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2024-02-06 16:36:46 +00:00
erezrokah
11f50c034e
feat: Add HTTP2 for unencrypted HTTP
Signed-off-by: erezrokah <erezrokah@users.noreply.github.com>
2024-01-17 20:59:02 +00:00
David Karlsson
5e75227fb2 docs: fix broken links and improve link resolution
Update the formatting of links and add a Markdown render hook for
handling relative internal links. Cross-references between markdown
files are now resolved the same way in both GitHub and Hugo.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-01-14 21:33:55 +01:00
Milos Gajdos
fb6ccc33d1
update: readme cleanup and fxes (#4208) 2023-12-21 22:18:07 +00:00
Milos Gajdos
e29a5c8e68
update: readme cleanup and fxes
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-12-21 22:05:56 +00:00
Steven Kalt
0e0d74b037
docs: remove legacy kramdown options from link
I was reading https://distribution.github.io/distribution/recipes/mirror/#gotcha when I noticed some unexpected annotations after the "fair use policy" link. According to [Stack Overflow](https://stackoverflow.com/a/4705645/6571327), these are kramdown options that the current hugo documentation site isn't respecting. I searched the hugo docs and couldn't find an easy way to preserve `rel="noopener" target="_blank"` behavior, so I removed the annotation.

Signed-off-by: Steven Kalt <SKalt@users.noreply.github.com>
2023-12-21 08:00:21 -05:00
Wang Yan
4a360f9da2
fix: remove disabling of multipart combine small parts (#4193) 2023-12-19 16:10:19 +08:00
Milos Gajdos
7fb303e922
Update s3.md
Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-12-18 16:43:54 +00:00
Milos Gajdos
290dba5d47
update: S3 driver docs
We were missing documentation for some of the configuration options.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-12-18 10:21:41 +00:00
Milos Gajdos
7ba91015f5
fix: remove disabling of multipart combine small parts
This reverts https://github.com/distribution/distribution/pull/3556

This feature is currently broken and requires more fundamental changes
in the S3 driver. Until then it's better to remove it.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-12-18 09:52:19 +00:00
Milos Gajdos
79ef555f8a
Update the gotcha in the proxy guide (#4164) 2023-12-15 18:20:52 +00:00
Ismail Alidzhikov
4f84c086fd Update the gotcha in the proxy guide
Signed-off-by: Ismail Alidzhikov <i.alidjikov@gmail.com>
2023-12-15 18:10:58 +02:00
Michael Bonfils
a1b262f083
Update nginx.md
htpasswd is no more available in registry:2 container, switch to httpd official image to create auth file

Signed-off-by: Michael Bonfils <bonfils.michael@protonmail.com>
2023-12-10 16:28:19 +01:00
Jakub Mikulas
c90b0b2649
docs: fix typo in mirror.md
Signed-off-by: Jakub Mikulas <jakub@mikul.as>
2023-11-21 15:16:51 +01:00
Sebastiaan van Stijn
186d522da9
docs: remove unused go.mod
This go.mod was used to allow vendoring the docs with Hugo, but this
was never used, so we can remove the go.mod altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-01 15:56:05 +01:00
Milos Gajdos
daf3d00a32
Add prometheus proxy related metrics (#4047) 2023-10-26 16:11:10 +01:00
David Karlsson
e5f16beae7 docs: remove unused images
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-26 13:04:56 +02:00
David Karlsson
57a6fa46b2 docs: fix broken image references
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-26 13:03:53 +02:00
David Karlsson
9d12b47bbb docs: allow unsafe HTML
Enable rendering of unsafe HTML in Markdown files

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-26 12:55:39 +02:00
Dimitar Kostadinov
9861a46d99 Add prometheus proxy related metrics
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
2023-10-23 19:59:22 +03:00
Milos Gajdos
fe21f43911
feat: replace docker/libtrust with go-jose/go-jose
docker/libtrust repository has been archived for several years now.
This commit replaces all the libtrust JWT machinery with go-jose/go-jose module.
Some of the code has been adopted from libtrust and adjusted for some of
the use cases covered by the token authorization flow especially in the
tests.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-19 15:32:59 +01:00
James Hewitt
c3ae793f85
And other content...
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-12 12:00:15 +01:00
James Hewitt
83dd4ff0a6
Cleanup of naming in docs
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-12 11:39:36 +01:00
David Karlsson
31707d54f3 docs: add github link in header
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 10:00:43 +02:00
David Karlsson
1596da6813 docs: add tests
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 09:32:55 +02:00
David Karlsson
b911020c1f docs: fix markup and broken links
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 09:32:37 +02:00
David Karlsson
e2ae76f1f2 docs: add hugo site
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-11 16:45:16 +02:00
Milos Gajdos
f7b3869062
Merge pull request #4091 from dvdksn/docs-jwt-rendering-bug
docs: remove blank line
2023-10-04 14:35:49 +01:00
David Karlsson
6183f23092 docs: remove blank line
This blank line confuses the markdown parser to think
that this is an indented code block.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-04 15:20:18 +02:00
Milos Gajdos
a70964c2fc
Merge pull request #4076 from flavianmissi/s3-loglevel
registry: add loglevel support for aws s3 storage driver
2023-10-04 14:13:15 +01:00
Milos Gajdos
a2101447d4
Merge pull request #4087 from milosgajdos/update-dockerhub-docs
Update Docker Hub README and keep it in sync with this repository.
2023-10-02 22:22:02 +01:00
Milos Gajdos
f2a72d7f77
Update Docker Hub README and keep it in sync with this repository.
This commit
* adds a new docs page (`dockerhub.md`) that contains Docker Hub README
* updates the default config that gets backed into the docker image
* updates CI with a new workflow job that keeps Docker Hub README in
  sync with the contents of the `docs/dockerhub.md` file

Co-authored-by: CrazyMax <github@crazymax.dev>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-02 22:03:35 +01:00
Milos Gajdos
93a64460fe
Merge pull request #4086 from milosgajdos/indent-prometheus-docs
Properly indent prometheus docs
2023-10-02 20:40:14 +01:00