Commit graph

5105 commits

Author SHA1 Message Date
Milos Gajdos
735c161b53
Merge pull request #4066 from milosgajdos/optimise-s3-push
Optimise push in S3 driver
2023-09-29 13:47:20 +01:00
Milos Gajdos
23083ac9d2
Merge pull request #4077 from liubin/fix/use-manifestTagsPathSpec-all-tag-all
use manifestTagsPathSpec for listing all tags
2023-09-28 17:37:25 +01:00
Milos Gajdos
4fce3c0028
Move completedParts type back to the original position
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-28 15:58:02 +01:00
Milos Gajdos
73e7952c36
Merge pull request #4080 from liubin/fix/comment-typos
fix comment typos
2023-09-28 15:50:06 +01:00
bin liu
dca71db976 fix comment typos
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 17:48:21 +08:00
bin liu
6c724a1a95 use manifestTagsPathSpec for listing all tags
In terms of results, a`manifestTagsPathSpec{ name: "repo" }` equals
`manifestTagPathSpec{ name: "repo", tag: "" }`, but from the intention,
the `manifestTagsPathSpec` should be used.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-28 10:44:46 +08:00
Milos Gajdos
3fc1216dc3
Merge pull request #4072 from NeilW/zero-byte-append-check
driver testsuite: Add zero byte file checks
2023-09-27 22:04:05 +01:00
Milos Gajdos
b888b14b39
Optimise push in S3 driver
This commit cleans up and attempts to optimise the performance of image push in S3 driver.
There are 2 main changes:
* we refactor the S3 driver Writer where instead of using separate bytes
  slices for ready and pending parts which get constantly appended data
  into them causing unnecessary allocations we use optimised bytes
  buffers; we make sure these are used efficiently when written to.
* we introduce a memory pool that is used for allocating the byte
  buffers introduced above

These changes should alleviate high memory pressure on the push path to S3.

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-27 21:33:22 +01:00
Milos Gajdos
4144538c72
Merge pull request #4069 from milosgajdos/makefile-local-environment
Add make targets to allow starting local cloud storage environment.
2023-09-27 15:19:20 +01:00
Milos Gajdos
58a76344de
Merge pull request #4073 from liubin/fix-docs-url
docs: remove README.md that point to Docker's repo
2023-09-27 11:38:37 +01:00
bin liu
0b72b0b8c7 docs: remove README.md that point to Docker's repo
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-27 16:40:13 +08:00
Milos Gajdos
a5c04b3688
Update Makefile
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 17:24:04 +01:00
Milos Gajdos
cf95610635
Update BUILDING.md
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 15:33:43 +01:00
Milos Gajdos
98ffc56af7
Only set COMPOSE if it doesnt have a value
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-26 14:44:10 +01:00
Milos Gajdos
14361b3ab5
Update Makefile and docker-compose
* make COMPOSE overrideable
* remove minio trace command from minio init

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-26 14:41:42 +01:00
Milos Gajdos
8e630ae2a5
Update BUILDING.md readme file.
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-26 14:37:10 +01:00
Milos Gajdos
ecf492ab5c
Update tests/docker-compose-storage.yml
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 14:29:06 +01:00
Milos Gajdos
dfb8514a9f
Update Makefile
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 14:21:56 +01:00
Milos Gajdos
6f05474fe0
Update tests/docker-compose-storage.yml
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 14:18:21 +01:00
Milos Gajdos
8af25245f3
Update tests/docker-compose-storage.yml
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 14:18:10 +01:00
Milos Gajdos
075d81d7bf
Update Makefile
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 14:17:48 +01:00
Milos Gajdos
6b0c391865
Update Makefile
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Milos Gajdos <milosgajdos83@gmail.com>
2023-09-26 14:17:28 +01:00
Neil Wilson
71c532e60c
driver testsuite: Add zero byte file checks
Add two new checks to the testsuite that check
the driver can handle zero byte files and appends to zero
byte files correctly

Signed-off-by: Neil Wilson <neil@aldur.co.uk>
2023-09-26 10:48:46 +01:00
Milos Gajdos
f7e792417a
Merge pull request #4070 from liubin/add-repositoriesRootPathSpec
add repositoriesRootPathSpec in pathFor documentation
2023-09-26 08:47:19 +01:00
Milos Gajdos
11e93bf454
Merge pull request #4071 from liubin/delete-old-version-gobuild-directive
remove go build directive for older go version
2023-09-26 08:47:06 +01:00
bin liu
06acf2def5 remove go build directive for older go version
Go 1.4 is too old and should be dropped safely.

Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-26 15:14:57 +08:00
bin liu
a0d9279e8f add repositoriesRootPathSpec in pathFor documentation
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-26 15:07:49 +08:00
Milos Gajdos
797b1e3927
Add make targets to allow starting local cloud storage environment.
Requirements:
* docker deamon
* docker compose installed

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-25 23:47:49 +01:00
Milos Gajdos
5e6af2f13f
Merge pull request #4067 from milosgajdos/dont-close-request-body
Do not close HTTP request body in HTTP handler
2023-09-23 23:08:04 +01:00
Milos Gajdos
f4d5210b25
Do not close HTTP request body in HTTP handler
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-22 16:21:54 +01:00
Milos Gajdos
8d12329a8f
Merge pull request #4061 from sagikazarmark/deprecate-resource-class
document resource class deprecation
2023-09-21 15:46:44 +01:00
Milos Gajdos
7038ccbd31
Merge pull request #4062 from liubin/fix/remove-not-exist-fn-in-comment
remove not exist function name in comment
2023-09-21 10:15:50 +01:00
bin liu
34654f6c4a remove not exist function name in comment
Signed-off-by: bin liu <liubin0329@gmail.com>
2023-09-21 16:53:59 +08:00
Milos Gajdos
f0e27fde4d
Merge pull request #4020 from PhracturedBlue/socket-activation
Support systemd socket-activation
2023-09-21 09:08:55 +01:00
Mark Sagi-Kazar
ca1b875374
document resource class deprecation
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2023-09-21 01:36:29 +02:00
Geoffrey Hausheer
a9399e9ea2 Improve socket-activation message
Co-authored-by: James Hewitt <james.hewitt@gmail.com>
Signed-off-by: Geoffrey Hausheer <rc2012@pblue.org>
2023-09-20 10:31:02 -07:00
Geoffrey Hausheer
9721db9504 Add info message regarding socket-activation
Signed-off-by: Geoffrey Hausheer <rc2012@pblue.org>
2023-09-20 09:56:25 -07:00
Geoffrey Hausheer
741f9bb564 Add documentation for socket activation
Signed-off-by: Geoffrey Hausheer <rc2012@pblue.org>
2023-09-20 09:37:26 -07:00
Geoffrey Hausheer
2435def474 Support systemd socket-activation
Signed-off-by: Geoffrey Hausheer <rc2012@pblue.org>
2023-09-20 09:37:22 -07:00
Milos Gajdos
42ce5d4d51
Merge pull request #3569 from justadogistaken/optimize/avoid-redundant-blob-fetching
optimize: avoid redundant blob fetching
2023-09-18 08:01:14 +01:00
baojiangnan
17952924f3 avoid redundant blob fetching
Signed-off-by: baojiangnan <baojn1998@163.com>
2023-09-18 10:40:25 +08:00
Milos Gajdos
612ad42609
Merge pull request #4040 from thaJeztah/move_api_errors 2023-09-15 09:36:36 +01:00
Milos Gajdos
73af930009
Merge pull request #4052 from thaJeztah/client_refactor_errhandling 2023-09-15 09:35:57 +01:00
Hayley Swimelar
b56fb385f6
Merge pull request #4055 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-12 08:52:39 -07:00
Sebastiaan van Stijn
23115ff634
update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-12 00:07:34 +02:00
Sebastiaan van Stijn
c8ba5d7081
registry/client: combine SuccessStatus and HandleErrorResponse
The SuccessStatus acted on the response's status code, and was used to return
early, before checking the same status code with HandleErrorResponse.

This patch combines both functions into a HandleHTTPResponseError, which
returns an error for "non-success" status-codes, which simplifies handling
of responses, and makes some logic slightly more idiomatic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:40:41 +02:00
Milos Gajdos
285b601af9
Merge pull request #4049 from distribution/dependabot/go_modules/github.com/cyphar/filepath-securejoin-0.2.4
Bump github.com/cyphar/filepath-securejoin from 0.2.3 to 0.2.4
2023-09-08 10:32:50 +01:00
dependabot[bot]
e4dd28b886
Bump github.com/cyphar/filepath-securejoin from 0.2.3 to 0.2.4
Bumps [github.com/cyphar/filepath-securejoin](https://github.com/cyphar/filepath-securejoin) from 0.2.3 to 0.2.4.
- [Release notes](https://github.com/cyphar/filepath-securejoin/releases)
- [Commits](https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4)

---
updated-dependencies:
- dependency-name: github.com/cyphar/filepath-securejoin
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-07 13:06:27 +00:00
Wang Yan
90939f1173
Merge pull request #4042 from milosgajdos/libtrust-handlers
Remove libtrust from handler tests
2023-09-05 15:59:38 +08:00
Milos Gajdos
612a30a7e7
Remove libtrust from handler tests
It was used for signing schema v1 manifests in tests which have now been
removed so there is no point in keeping these there anymore.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-09-05 08:41:29 +01:00