Commit graph

215 commits

Author SHA1 Message Date
Milos Gajdos
612ad42609
Merge pull request #4040 from thaJeztah/move_api_errors 2023-09-15 09:36:36 +01:00
Sebastiaan van Stijn
c8ba5d7081
registry/client: combine SuccessStatus and HandleErrorResponse
The SuccessStatus acted on the response's status code, and was used to return
early, before checking the same status code with HandleErrorResponse.

This patch combines both functions into a HandleHTTPResponseError, which
returns an error for "non-success" status-codes, which simplifies handling
of responses, and makes some logic slightly more idiomatic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:40:41 +02:00
Sebastiaan van Stijn
292e30bc61
registry/api: move all errors to "errcode" package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 18:02:54 +02:00
Sebastiaan van Stijn
152af63ec5
deprecate reference package, migrate to github.com/distribution/reference
This integrates the new module, which was extracted from this repository
at commit b9b19409cf458dcb9e1253ff44ba75bd0620faa6;

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    # create a temporary clone of docker
    cd ~/Projects
    git clone https://github.com/distribution/distribution.git reference
    cd reference

    # commit taken from
    git rev-parse --verify HEAD
    b9b19409cf

    # remove all code, except for general files, 'reference/', and rename to /
    git filter-repo \
      --path .github/workflows/codeql-analysis.yml \
      --path .github/workflows/fossa.yml \
      --path .golangci.yml \
      --path distribution-logo.svg \
      --path CODE-OF-CONDUCT.md \
      --path CONTRIBUTING.md \
      --path GOVERNANCE.md \
      --path README.md \
      --path LICENSE \
      --path MAINTAINERS \
      --path-glob 'reference/*.*' \
      --path-rename reference/:

    # initialize go.mod
    go mod init github.com/distribution/reference
    go mod tidy -go=1.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:47:06 +02:00
Milos Gajdos
45b7b9cec3
Dont parse errors as JSON unless Content-Type is set to JSON
Client attempts to parse the body of every error it receives as JSON
regardless of the content-type. This commit rectifies by only parsing
he error body as JSON if the Content-Type header is set to
either "application/json" or "application/vnd.api+json".

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-28 12:35:33 +01:00
Milos Gajdos
1a4638a06c
Merge pull request #4007 from milosgajdos/remove-schema1-client 2023-08-21 13:46:29 +01:00
Milos Gajdos
df2787c6cf
Update test asserts
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-21 10:27:46 +01:00
Milos Gajdos
11c341a369
Remove schema1 references from registry client
We've replaced all the schema1 references with OCI schema manifest.
Note, there are some TODO items that must be addressed at some point in
the future once the schema1 package is removed completely from the
codebase.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-20 09:07:59 +01:00
Milos Gajdos
3dbfbc7255
Enable bodyclose linter
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-19 09:45:44 +01:00
Milos Gajdos
24de708d22
Set Content-Type header in registry client ReadFrom
Client ReadFrom doesn't set Content-Type header leading to server
side implementor to assume it's application/octet-stream. This commit
makes this explicit on the client side.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2023-08-15 08:46:48 +01:00
Laura Brehm
afe5a2a9b7 Support ztsd compression as Content-Encoding
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-06-01 14:06:25 +01:00
Sebastiaan van Stijn
999527f978
Ignore SA1019: "schema1 is deprecated" linting errors
We need to use this for backward compatibility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:04:17 +02:00
Paweł Gronowski
5f1df02149
registry/errors: Parse http forbidden as denied
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-27 19:48:32 +02:00
Milos Gajdos
5357c45703
Merge pull request #3788 from thaJeztah/deprecate_ReadSeekCloser 2022-11-18 08:53:15 +00:00
Sebastiaan van Stijn
1d8cd5e443
registry/client: use struct literals
Remove some intermediate variables, and use struct literals instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-10 23:11:12 +01:00
Sebastiaan van Stijn
d71ad5b3a6
transport.NewHTTPReadSeeker: return concrete type, deprecate ReadSeekCloser
General convention is to define interfaces on the receiver side, and
to return concrete types.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-10 23:10:39 +01:00
Sebastiaan van Stijn
019ead86f5
deprecate ReadSeekCloser in favor of io.ReadSeekCloser
Go's io package in stdlib now defines this interface, so we can switch
to using that instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-10 23:10:32 +01:00
Sebastiaan van Stijn
3b391d3290
replace strings.Split(N) for strings.Cut() or alternatives
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:

```go
func BenchmarkSplit(b *testing.B) {
	b.ReportAllocs()
	data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
	for i := 0; i < b.N; i++ {
		for _, s := range data {
			_ = strings.SplitN(s, "=", 2)[0]
		}
	}
}

func BenchmarkCut(b *testing.B) {
	b.ReportAllocs()
	data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
	for i := 0; i < b.N; i++ {
		for _, s := range data {
			_, _, _ = strings.Cut(s, "=")
		}
	}
}
```

    BenchmarkSplit
    BenchmarkSplit-10    	 8244206	       128.0 ns/op	     128 B/op	       4 allocs/op
    BenchmarkCut
    BenchmarkCut-10      	54411998	        21.80 ns/op	       0 B/op	       0 allocs/op

While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints;

- for cases where an specific number of items is expected, I used `strings.SplitN()`
  with a suitable limit. This prevents (theoretical) unlimited splits.
- in some cases it we were using `strings.Split()`, but _actually_ were trying to match
  a prefix; for those I replaced the code to just match (and/or strip) the prefix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-10 22:38:12 +01:00
Sebastiaan van Stijn
f8b3af78fc
replace deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 23:47:52 +01:00
Hayley Swimelar
52d948a9f5
Merge pull request #3766 from thaJeztah/gofumpt
format code with gofumpt
2022-11-04 12:19:53 +01:00
Sebastiaan van Stijn
e0281dc609
format code with gofumpt
gofumpt (https://github.com/mvdan/gofumpt) provides a supserset of `gofmt` / `go fmt`,
and addresses various formatting issues that linters may be checking for.

We can consider enabling the `gofumpt` linter to verify the formatting in CI, although
not every developer may have it installed, so for now this runs it once to get formatting
in shape.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-03 22:48:20 +01:00
Hayley Swimelar
ebfa2a0ac0
Merge pull request #3768 from thaJeztah/http_consts
use http consts for request methods
2022-11-03 13:52:52 +01:00
Sebastiaan van Stijn
f9ccd2c6ea
use http consts for request methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 23:31:47 +01:00
Nicolas De Loof
9c04d0b30a
Revert "registry/client: set Accept: identity header when getting layers"
This reverts commit 16f086a0ec.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-10-20 14:44:10 +02:00
Aaron Lehmann
e36cb0a5d8 registry/storage/cache/memory: Use LRU cache to bound cache size
Instead of letting the cache grow without bound, use a LRU to impose a
size limit.

The limit is configurable through a new `blobdescriptorsize` config key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-07 07:20:06 -07:00
Aaron Lehmann
fbdfd1ac35 Use http.NewRequestWithContext for outgoing HTTP requests
This simple change mainly affects the distribution client. By respecting
the context the caller passes in, timeouts and cancellations will work
as expected. Also, transports which rely on the context (such as tracing
transports that retrieve a span from the context) will work properly.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-08-10 10:38:30 -07:00
João Pereira
6ae6df7d75
Add tag delete API
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-05-27 23:27:02 +01:00
Sebastiaan van Stijn
1d33874951
go.mod: change imports to github.com/distribution/distribution/v3
Go 1.13 and up enforce import paths to be versioned if a project
contains a go.mod and has released v2 or up.

The current v2.x branches (and releases) do not yet have a go.mod,
and therefore are still allowed to be imported with a non-versioned
import path (go modules add a `+incompatible` annotation in that case).

However, now that this project has a `go.mod` file, incompatible
import paths will not be accepted by go modules, and attempting
to use code from this repository will fail.

This patch uses `v3` for the import-paths (not `v2`), because changing
import paths itself is a breaking change, which means that  the
next release should increment the "major" version to comply with
SemVer (as go modules dictate).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-08 18:30:46 +01:00
João Pereira
f82e1c2332
Merge pull request #3300 from jubalh/sp
Fix minor spelling mistakes
2021-02-06 16:03:09 +00:00
João Pereira
038a5060de
Merge pull request #2905 from 2opremio/master
Honor contexts passed to registry client methods
2021-02-06 15:25:03 +00:00
Michael Vetter
f0c93f65a2 Fix typo in NewSimpleManager() documentation
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
2021-02-03 13:11:53 +01:00
Smasherr
c8d90f904f Remove empty Content-Type header
Fixes #3288

Signed-off-by: Smasherr <soundcracker@gmail.com>
2020-11-03 12:24:02 +01:00
Derek McGowan
78c2ab6646
Fix gosimple checks
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-18 15:52:34 -07:00
Alfonso Acosta
15f7bd29a5 Remove {get,head}WithContext()
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 13:36:43 +01:00
Alfonso Acosta
282351e954 Use http.NewRequestWithContext()
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 13:36:31 +01:00
Alfonso Acosta
58331abf58 Honor contexts passed to registry client methods
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 12:41:15 +01:00
sayboras
66809646d9 Migrate to golangci-lint
Signed-off-by: Tam Mach <sayboras@yahoo.com>
2020-02-14 08:11:16 +11:00
Derek McGowan
bdf3438b52
Merge pull request #2985 from novas0x2a/default-transport
make it possible to wrap the client transport in another one
2019-12-13 11:36:12 -08:00
Ryan Abrams
740d4d1211
Merge pull request #2918 from dmathieu/test-blob-writer-write
Test httpBlobUpload.Write method
2019-09-05 08:16:36 -07:00
Mike Lundy
c486db2d71
make it possible to wrap the client transport in another one
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
2019-08-22 17:37:47 -04:00
Damien Mathieu
dd3bdee21c implement Repository Blobs upload resuming
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-26 09:46:49 +02:00
Ryan Abrams
90dfea7952
Merge pull request #2921 from dmathieu/repository-serve-blob
Implement Repository ServeBlob
2019-06-25 19:07:38 -07:00
Damien Mathieu
898b1f2a53 test httpBlobUpload.Write method
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:32:58 +02:00
Damien Mathieu
c5d5f938e3 fast-stop ServeBlob if we're doing a HEAD request
A registry pointing to ECR is having issues if we try loading the blob

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu
3800c47fd2 Implement Repository ServeBlob
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu
a45e5cb13f handle create blob if the uuid couldn't be retrieved from headers or URL
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Damien Mathieu
8b31a894bd deduce blob UUID from location if it wasn't provided in the headers
Some registries (ECR) don't provide a `Docker-Upload-UUID` when creating
a blob. So we can't rely on that header. Fallback to reading it from the
URL.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Ryan Abrams
84f47e7bb3
Merge pull request #2900 from sevki/no-token-err-fix
fix no error returned in fetchTokenWithOAuth
2019-06-19 19:16:18 -07:00
Damien Mathieu
0e2d080a8a append the written bytes to the blob writer's size
Any byte written should append to the size. Otherwise, the full Size is
always zero.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-05-09 14:17:58 +02:00
Sevki Hasirci
5afbf32400 fix no error returned in fetchTokenWithOAuth
fetchTokenWithBasicAuth checks if a token is in the token response
but fetchTokenWithOAuth does not

these changes implements the same behaviour for the latter
returning a `ErrNoToken` if a token is not found in the resposne

Signed-off-by: Sevki Hasirci <sevki@cloudflare.com>
2019-04-14 11:05:59 +01:00