Commit graph

4782 commits

Author SHA1 Message Date
Milos Gajdos
9bb63e6e46
Merge pull request #3775 from thaJeztah/remove_digestset 2022-11-08 22:48:25 +00:00
Milos Gajdos
47c3610bff
Merge pull request #3679 from Jamstah/building-doc-updates 2022-11-08 22:46:09 +00:00
Milos Gajdos
75093e0aab
Merge pull request #3777 from thaJeztah/reference_add_sorting 2022-11-08 22:45:19 +00:00
Sebastiaan van Stijn
7b651a9692
digestset: deprecate package in favor of go-digest/digestset
This package was only used for the deprecated "shortid" syntax. Now that
support for this syntax was removed, we can also remove this package.

This patch deprecates and removes the package, adding temporary aliases pointing
to the new location to ease migration from docker/distribution to the new
distribution/distribution/v3. We should remove those aliases in a future update.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 23:17:10 +01:00
Derek McGowan
1052518d9f
reference: implement Sort()
This upstreams `Sort()` as originally implemented in containerd in
https://github.com/containerd/containerd/0886ceaea2470edc7339dfc5ebe0e3257ae84d06

From that commit:

> Fix reference ordering in CRI image store
>
> Currently image references end up being stored in a
> random order due to the way maps are iterated through
> in Go. This leads to inconsistent identifiers being
> resolved when a single reference is needed to identify
> an image and the ordering of the references is used for
> the selection.
>
> Sort references in a consistent and ranked manner,
> from higher information formats to lower.
>
> Note: A `name + tag` reference is considered higher
> information than a `name + digest` reference since a
> registry may be used to resolve the digest from a
> `name + tag` reference.

Co-Authored-by: Derek McGowan <derek@mcg.dev>
Co-Authored-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 22:38:47 +01:00
Milos Gajdos
fee8c686bb
Merge pull request #3776 from thaJeztah/ref_replace_deprecated
reference: replace deprecated function SplitHostname
2022-11-08 17:30:59 +00:00
Sebastiaan van Stijn
bf72536440
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 13:51:11 +01:00
Hayley Swimelar
e9a25da7a4
Merge pull request #3772 from thaJeztah/registry_logging_simplify
registry: configureLogging() simplify logic a bit
2022-11-08 09:17:20 +01:00
Hayley Swimelar
c7758d1637
Merge pull request #3774 from thaJeztah/remove_deprecated_shortid
reference: remove support for deprecated "shortid" refs
2022-11-08 09:14:18 +01:00
Hayley Swimelar
4906c6ee88
Merge pull request #3767 from thaJeztah/replace_ioutils
replace deprecated io/ioutil
2022-11-08 09:13:08 +01:00
Milos Gajdos
ad6249fa92
Merge pull request #3681 from Jamstah/api-doc-fix
Fix API doc parameter placeholder: last
2022-11-07 15:34:07 +00:00
James Hewitt
30f5f41d8c
Use GitHub https url
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2022-11-07 15:06:46 +00:00
Milos Gajdos
08764d51bf
Merge pull request #3680 from Jamstah/env-var-clash
Raise a specific error for env var parsing issues
2022-11-07 11:26:09 +00:00
Sebastiaan van Stijn
3c71f4933d
referene: fix formatting of "deprecated" comment.
Go requires "deprecated" comments to have an empty line before them,
and to not be all-caps.

This updates to the comment so that it's correctly picked up as deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 22:52:01 +01:00
zounengren
79d1901549
replace deprecated function
Signed-off-by: Zou Nengren <zouyee1989@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 22:49:41 +01:00
Sebastiaan van Stijn
6d4f62d7fd
reference: remove support for deprecated "shortid" refs
The "shortid" syntax was added in d26a3b37a6,
and allowed for matching an image on its ID prefix (this is before images were
content-addressable). With the introduction of content-addressable references,
this syntax became problematic, and Docker deprecated this syntax in 2016
(Docker v1.13.0) through commit; 5fc71599a0

> The `repository:shortid` syntax for referencing images is very little used,
> collides with tag references, and can be confused with digest references.

Support for this syntax was removed in 2017 (Docker 17.12) through commit:
a942c92dd7

containerd uses a fork of the reference package with this syntax removed, and
does not support this syntax:
901bcb2231

This patch removes the deprecated syntax, the ParseAnyReferenceWithSet function,
and the ShortIdentifierRegexp regex.

As there are no external consumers for this function, nor the regexp, I'm
skipping a deprecation cycle for this;

- https://grep.app/search?q=.ShortIdentifierRegexp
- https://grep.app/search?q=.ParseAnyReferenceWithSet%28

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 20:23:30 +01:00
Sebastiaan van Stijn
b73c038000
registry: configureLogging() simplify logic a bit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 13:26:13 +01:00
Sebastiaan van Stijn
f1dff3e434
registry: use consts for some defaults
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 13:26:05 +01:00
Sebastiaan van Stijn
f8b3af78fc
replace deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 23:47:52 +01:00
Hayley Swimelar
e3509fc1de
Merge pull request #3635 from milosgajdos/make-s3-driver-delete-faster
Delete S3 keys incrementally in batches
2022-11-04 16:56:41 +01:00
Hayley Swimelar
52d948a9f5
Merge pull request #3766 from thaJeztah/gofumpt
format code with gofumpt
2022-11-04 12:19:53 +01:00
Milos Gajdos
526d594ef6
Merge pull request #3770 from Jamstah/log-stderr-doc
Update doc to reflect that logs go to stderr
2022-11-03 22:19:51 +00:00
Sebastiaan van Stijn
e0281dc609
format code with gofumpt
gofumpt (https://github.com/mvdan/gofumpt) provides a supserset of `gofmt` / `go fmt`,
and addresses various formatting issues that linters may be checking for.

We can consider enabling the `gofumpt` linter to verify the formatting in CI, although
not every developer may have it installed, so for now this runs it once to get formatting
in shape.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-03 22:48:20 +01:00
James Hewitt
6dbb55ada5
Update doc to reflect that logs go to stderr
Closes #2855

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2022-11-03 16:37:00 +00:00
Hayley Swimelar
ebfa2a0ac0
Merge pull request #3768 from thaJeztah/http_consts
use http consts for request methods
2022-11-03 13:52:52 +01:00
Milos Gajdos
6a2594c5b0
Merge pull request #3754 from ndeloof/accept-encoding
Revert "registry/client: set Accept: identity header when getting layers
2022-11-03 11:06:17 +00:00
Sebastiaan van Stijn
f9ccd2c6ea
use http consts for request methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 23:31:47 +01:00
Sebastiaan van Stijn
b1285c33a8
go.mod: github.com/spf13/cobra v1.6.1
We were using v1.0.0 of Cobra as newer versions added spf13/viper as dependency,
which came with many indirect dependencies. Cobra v1.6.0 and up no longer depend
on viper, so we can now safely upgrade to the latest version.

full diff: https://github.com/spf13/cobra/compare/v1.0.0...v1.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 19:48:45 +01:00
Milos Gajdos
ebc4234fd5
Delete S3 keys incrementally in batches
Instead of first collecting all keys and then batch deleting them,
we will do the incremental delete _online_ per max allowed batch.
Doing this prevents frequent allocations for large S3 keyspaces
and OOM-kills that might happen as a result of those.

This commit introduces storagedriver.Errors type that allows to return
multierrors as a single error from any storage driver implementation.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2022-10-30 19:10:24 +00:00
Milos Gajdos
7f9f86c411
Merge pull request #3755 from wy65701436/fix-getstatus 2022-10-28 08:20:15 +01:00
Wang Yan
bad5dcb602 fit get status issue
1, return the right upload offset for client when asks.
2, do not call ResumeBlobUpload on getting status.
3, return 416 rather than 404 on failed to patch chunk blob.
4, add the missing upload close

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-10-26 23:33:39 +08:00
Milos Gajdos
c47a966fde
Merge pull request #3727 from thaJeztah/bump_x_net
vendor: golang.org/x/net v0.0.0-20220906165146-f3363e06e74c
2022-10-21 10:26:57 +01:00
Nicolas De Loof
9c04d0b30a
Revert "registry/client: set Accept: identity header when getting layers"
This reverts commit 16f086a0ec.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-10-20 14:44:10 +02:00
Milos Gajdos
fb2188868d
Merge pull request #3365 from brackendawson/3122-remove-workaround
Remove workaround from 2.1.1 for faulty 2.1.0 manifest links
2022-10-19 09:04:24 +01:00
Wang Yan
ff42b8bf8e
Merge pull request #3748 from gliptak/patch-1
Add Go 1.19 to GHA
2022-10-17 14:33:46 +08:00
Gábor Lipták
8cc5b4f5aa
Add Go 1.19 to GHA
Signed-off-by: Gábor Lipták gliptak@gmail.com
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2022-10-09 16:52:41 -04:00
Milos Gajdos
78b9c98c5c
Merge pull request #3689 from aaronlehmann/inmemory-descriptor-cache-lru
registry/storage/cache/memory: Use LRU cache to bound cache size
2022-09-07 16:52:24 +01:00
Aaron Lehmann
e36cb0a5d8 registry/storage/cache/memory: Use LRU cache to bound cache size
Instead of letting the cache grow without bound, use a LRU to impose a
size limit.

The limit is configurable through a new `blobdescriptorsize` config key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-07 07:20:06 -07:00
Sebastiaan van Stijn
f9dee9dc90
vendor: golang.org/x/net v0.0.0-20220906165146-f3363e06e74c
Update to the latest version that contains a fix for CVE-2022-27664;
f3363e06e7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 23:09:44 +02:00
Milos Gajdos
0122d7ddae
Merge pull request #3724 from Littlemoon917/main
fix all json syntax error in error api spec
2022-09-02 13:51:04 +01:00
Littlemoon917
e24be4e834 fix all json syntax error
Signed-off-by: Littlemoon917 <Littlemoon917@users.noreply.github.com>
2022-09-02 16:40:01 +08:00
Wang Yan
3413bf8e14
Merge pull request #3682 from Jamstah/contributing-review
Review contributing for Docker references
2022-08-22 11:44:24 +08:00
Milos Gajdos
bc6b7455cb
Merge pull request #3687 from Jamstah/add-build-tags
Add build tags to BUILDING.md
2022-08-20 11:24:08 +01:00
Milos Gajdos
cec2cad801
Merge pull request #3706 from kaovilai/s3-no-hostport
configuration: use "fake" values for tests
2022-08-18 18:11:35 +01:00
Tiger Kaovilai
567158c365 configuration: use "fake" values for tests
These tests don't validate if options are valid for the storage-driver,
nor do they test if the storage-driver itself is valid. However, the tests
were using actual values (such as s3) and options (such as "region") which
may lead to the conclusion that it's also testing validity of those values.

This patch replaces the test-values with non-existing driver-names and
options to make it more clear these are fake values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-08-18 10:23:17 -04:00
Milos Gajdos
6c237953cb
Merge pull request #3711 from aaronlehmann/request-with-context
Use http.NewRequestWithContext for outgoing HTTP requests
2022-08-16 16:03:28 +01:00
Milos Gajdos
1db54ecaf6
Merge pull request #3688 from Ailuridae/patch-1
Fix Grammar Mistake
2022-08-16 15:25:38 +01:00
Aaron Lehmann
fbdfd1ac35 Use http.NewRequestWithContext for outgoing HTTP requests
This simple change mainly affects the distribution client. By respecting
the context the caller passes in, timeouts and cancellations will work
as expected. Also, transports which rely on the context (such as tracing
transports that retrieve a span from the context) will work properly.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-08-10 10:38:30 -07:00
Milos Gajdos
26163d8256
Merge pull request #3683 from aaronlehmann/dont-recreate-router
Do not recreate mux router for each incoming request
2022-07-29 17:30:34 +01:00
Milos Gajdos
4bf3547399
Merge pull request #3489 from aojea/ipv6_domains
Ipv6 registries
2022-07-25 14:31:11 +01:00