Commit graph

5290 commits

Author SHA1 Message Date
Milos Gajdos
ea41722902
refactor: Storage driver errors
Small refactoring of storagedriver errors.
We change the Enclosed field to Detail and make sure
Errors get properly serialized to JSON.
We also add tests.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-18 10:02:21 +01:00
Milos Gajdos
915ad2d5a6
json encode storage driver enclosed error (#4099) 2023-10-17 21:44:42 +01:00
James Hewitt
eac199875e
Remove test for nested file delete on S3
Nested files aren't supported on MinIO, and as our storage layout is
filesystem based, we don't actually use nest files in the code.

Remove the test so that we can support MinIO.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-17 09:13:15 +01:00
James Hewitt
647ec33c33
Bump minio version and test less storage classes
This fixes some of the tests for minio.

The walk tests needs a version of minio that contains https://github.com/minio/minio/pull/18099

The storage classes minio supports are a subset of the s3 classes.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-17 02:10:43 +01:00
Milos Gajdos
1d410148ef
Update dockerhub-readme GH Action (#4105) 2023-10-16 19:10:39 +01:00
Milos Gajdos
71217a0c7b
Update docs GHA and renamed a doc file (#4102) 2023-10-16 13:59:03 +01:00
Milos Gajdos
078c0546a4
Add annotation for descriptor (#4106) 2023-10-16 13:36:07 +01:00
Tosone
4dce8b866e Add annotation for descriptor
Signed-off-by: Tosone <i@tosone.cn>
2023-10-16 19:03:12 +08:00
Milos Gajdos
777ad03208
Update docs GHA
Add missing steps to the job, pick up the path automatically, trigger
the job on config file changes.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-15 11:03:23 +01:00
Milos Gajdos
4f506663d3
Update dockerhub-readme GH Action
We were missing GitHub checkout step before running the update.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-14 18:16:01 +01:00
Milos Gajdos
c78d6f99ae
Move dockerhub-readme workflow to the correct path (#4103) 2023-10-14 18:09:52 +01:00
Milos Gajdos
dc07c42810
Move dockerhub-readme workflow to the correct path
We've incorrectly added the dockerhub-readme workflow into .github
path from where it can not be triggered:

https://docs.github.com/en/actions/using-workflows/triggering-a-workflow

This commit addresses it and update the workflow paths.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-13 11:05:12 +01:00
Milos Gajdos
ebba01efea
docs: add hugo website (#4101) 2023-10-12 17:08:54 +01:00
David Karlsson
a66f6c37cb ci: add github pages workflow for docs
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 13:44:58 +02:00
James Hewitt
c3ae793f85
And other content...
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-12 12:00:15 +01:00
James Hewitt
83dd4ff0a6
Cleanup of naming in docs
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2023-10-12 11:39:36 +01:00
David Karlsson
31707d54f3 docs: add github link in header
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 10:00:43 +02:00
David Karlsson
1596da6813 docs: add tests
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 09:32:55 +02:00
David Karlsson
b911020c1f docs: fix markup and broken links
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-12 09:32:37 +02:00
Milos Gajdos
fe98c99860
Bump golang.org/x/net from 0.8.0 to 0.17.0 (#4100) 2023-10-12 08:13:49 +01:00
Glyn Owen Hanmer
fee6faef70 json encode storage driver enclosed error
Signed-off-by: Glyn Owen Hanmer <1295698+glynternet@users.noreply.github.com>
2023-10-11 17:53:27 -06:00
dependabot[bot]
758c0f9d77
Bump golang.org/x/net from 0.8.0 to 0.17.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.8.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.8.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 23:39:45 +00:00
David Karlsson
e2ae76f1f2 docs: add hugo site
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-11 16:45:16 +02:00
Milos Gajdos
f7b3869062
Merge pull request #4091 from dvdksn/docs-jwt-rendering-bug
docs: remove blank line
2023-10-04 14:35:49 +01:00
David Karlsson
6183f23092 docs: remove blank line
This blank line confuses the markdown parser to think
that this is an indented code block.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-04 15:20:18 +02:00
Milos Gajdos
a70964c2fc
Merge pull request #4076 from flavianmissi/s3-loglevel
registry: add loglevel support for aws s3 storage driver
2023-10-04 14:13:15 +01:00
Milos Gajdos
ed8423176f
Merge pull request #4081 from liubin/fix/refactor-redis
refactor redis cache
2023-10-03 16:01:07 +01:00
Milos Gajdos
a2101447d4
Merge pull request #4087 from milosgajdos/update-dockerhub-docs
Update Docker Hub README and keep it in sync with this repository.
2023-10-02 22:22:02 +01:00
Milos Gajdos
f2a72d7f77
Update Docker Hub README and keep it in sync with this repository.
This commit
* adds a new docs page (`dockerhub.md`) that contains Docker Hub README
* updates the default config that gets backed into the docker image
* updates CI with a new workflow job that keeps Docker Hub README in
  sync with the contents of the `docs/dockerhub.md` file

Co-authored-by: CrazyMax <github@crazymax.dev>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-02 22:03:35 +01:00
Milos Gajdos
93a64460fe
Merge pull request #4086 from milosgajdos/indent-prometheus-docs
Properly indent prometheus docs
2023-10-02 20:40:14 +01:00
Milos Gajdos
504a3bafc5
Properly indent prometheus docs
Incorrect section indentation of the prometheus docs confuses some
folks. This commit fixes that by indenting the prometheus section
under the debug configuration section.

Co-authored-by: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-10-02 18:03:14 +01:00
Milos Gajdos
e4d98bf5d0
Merge pull request #4089 from ialidzhikov/enh/debug-endpoint
Add few more sentences for the debug endpoint
2023-10-02 16:58:57 +01:00
Flavian Missi
3df7e28f44 registry: add loglevel support for aws s3 storage driver
based on the work from
https://github.com/distribution/distribution/pull/3057.

Co-authored-by: Simon Compston <compston@gmail.com>
Signed-off-by: Flavian Missi <fmissi@redhat.com>
2023-10-02 15:47:02 +02:00
ialidzhikov
993af6fefd Add few more sentences for the debug endpoint
Initially I misunderstood that the debug endpoint has to be disabled in production environments. That's why I created https://github.com/distribution/distribution/issues/4084 and https://github.com/distribution/distribution/issues/4085.
But it turns out that the docs want to state the the debug endpoint should not be exposed publicly to the internet.

Signed-off-by: ialidzhikov <i.alidjikov@gmail.com>
2023-10-02 11:10:15 +03:00
Milos Gajdos
735c161b53
Merge pull request #4066 from milosgajdos/optimise-s3-push
Optimise push in S3 driver
2023-09-29 13:47:20 +01:00
Milos Gajdos
23083ac9d2
Merge pull request #4077 from liubin/fix/use-manifestTagsPathSpec-all-tag-all
use manifestTagsPathSpec for listing all tags
2023-09-28 17:37:25 +01:00
Milos Gajdos
4fce3c0028
Move completedParts type back to the original position
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-28 15:58:02 +01:00
Milos Gajdos
73e7952c36
Merge pull request #4080 from liubin/fix/comment-typos
fix comment typos
2023-09-28 15:50:06 +01:00
bin liu
46a9da160e refactor redis cache
This commit removes some `conn` parameters of private functions, which can
be obtain from the struct itself. The `conn` is for the old `redisgo` library,
which is replaced by `go-redis` in #4019.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 18:40:50 +08:00
bin liu
dca71db976 fix comment typos
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 17:48:21 +08:00
bin liu
6c724a1a95 use manifestTagsPathSpec for listing all tags
In terms of results, a`manifestTagsPathSpec{ name: "repo" }` equals
`manifestTagPathSpec{ name: "repo", tag: "" }`, but from the intention,
the `manifestTagsPathSpec` should be used.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 10:44:46 +08:00
Milos Gajdos
3fc1216dc3
Merge pull request #4072 from NeilW/zero-byte-append-check
driver testsuite: Add zero byte file checks
2023-09-27 22:04:05 +01:00
Milos Gajdos
b888b14b39
Optimise push in S3 driver
This commit cleans up and attempts to optimise the performance of image push in S3 driver.
There are 2 main changes:
* we refactor the S3 driver Writer where instead of using separate bytes
  slices for ready and pending parts which get constantly appended data
  into them causing unnecessary allocations we use optimised bytes
  buffers; we make sure these are used efficiently when written to.
* we introduce a memory pool that is used for allocating the byte
  buffers introduced above

These changes should alleviate high memory pressure on the push path to S3.

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-27 21:33:22 +01:00
Milos Gajdos
4144538c72
Merge pull request #4069 from milosgajdos/makefile-local-environment
Add make targets to allow starting local cloud storage environment.
2023-09-27 15:19:20 +01:00
Milos Gajdos
58a76344de
Merge pull request #4073 from liubin/fix-docs-url
docs: remove README.md that point to Docker's repo
2023-09-27 11:38:37 +01:00
bin liu
0b72b0b8c7 docs: remove README.md that point to Docker's repo
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-27 16:40:13 +08:00
Milos Gajdos
a5c04b3688
Update Makefile
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 17:24:04 +01:00
Milos Gajdos
cf95610635
Update BUILDING.md
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 15:33:43 +01:00
Milos Gajdos
98ffc56af7
Only set COMPOSE if it doesnt have a value
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-26 14:44:10 +01:00
Milos Gajdos
14361b3ab5
Update Makefile and docker-compose
* make COMPOSE overrideable
* remove minio trace command from minio init

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-26 14:41:42 +01:00