Commit graph

92 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e0281dc609
format code with gofumpt
gofumpt (https://github.com/mvdan/gofumpt) provides a supserset of `gofmt` / `go fmt`,
and addresses various formatting issues that linters may be checking for.

We can consider enabling the `gofumpt` linter to verify the formatting in CI, although
not every developer may have it installed, so for now this runs it once to get formatting
in shape.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-03 22:48:20 +01:00
Aaron Lehmann
e36cb0a5d8 registry/storage/cache/memory: Use LRU cache to bound cache size
Instead of letting the cache grow without bound, use a LRU to impose a
size limit.

The limit is configurable through a new `blobdescriptorsize` config key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-07 07:20:06 -07:00
Aaron Lehmann
fbdfd1ac35 Use http.NewRequestWithContext for outgoing HTTP requests
This simple change mainly affects the distribution client. By respecting
the context the caller passes in, timeouts and cancellations will work
as expected. Also, transports which rely on the context (such as tracing
transports that retrieve a span from the context) will work properly.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-08-10 10:38:30 -07:00
João Pereira
6ae6df7d75
Add tag delete API
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-05-27 23:27:02 +01:00
Sebastiaan van Stijn
1d33874951
go.mod: change imports to github.com/distribution/distribution/v3
Go 1.13 and up enforce import paths to be versioned if a project
contains a go.mod and has released v2 or up.

The current v2.x branches (and releases) do not yet have a go.mod,
and therefore are still allowed to be imported with a non-versioned
import path (go modules add a `+incompatible` annotation in that case).

However, now that this project has a `go.mod` file, incompatible
import paths will not be accepted by go modules, and attempting
to use code from this repository will fail.

This patch uses `v3` for the import-paths (not `v2`), because changing
import paths itself is a breaking change, which means that  the
next release should increment the "major" version to comply with
SemVer (as go modules dictate).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-08 18:30:46 +01:00
João Pereira
038a5060de
Merge pull request #2905 from 2opremio/master
Honor contexts passed to registry client methods
2021-02-06 15:25:03 +00:00
Smasherr
c8d90f904f Remove empty Content-Type header
Fixes #3288

Signed-off-by: Smasherr <soundcracker@gmail.com>
2020-11-03 12:24:02 +01:00
Alfonso Acosta
15f7bd29a5 Remove {get,head}WithContext()
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 13:36:43 +01:00
Alfonso Acosta
282351e954 Use http.NewRequestWithContext()
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 13:36:31 +01:00
Alfonso Acosta
58331abf58 Honor contexts passed to registry client methods
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 12:41:15 +01:00
sayboras
66809646d9 Migrate to golangci-lint
Signed-off-by: Tam Mach <sayboras@yahoo.com>
2020-02-14 08:11:16 +11:00
Damien Mathieu
dd3bdee21c implement Repository Blobs upload resuming
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-26 09:46:49 +02:00
Ryan Abrams
90dfea7952
Merge pull request #2921 from dmathieu/repository-serve-blob
Implement Repository ServeBlob
2019-06-25 19:07:38 -07:00
Damien Mathieu
c5d5f938e3 fast-stop ServeBlob if we're doing a HEAD request
A registry pointing to ECR is having issues if we try loading the blob

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu
3800c47fd2 Implement Repository ServeBlob
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu
a45e5cb13f handle create blob if the uuid couldn't be retrieved from headers or URL
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Damien Mathieu
8b31a894bd deduce blob UUID from location if it wasn't provided in the headers
Some registries (ECR) don't provide a `Docker-Upload-UUID` when creating
a blob. So we can't rely on that header. Fallback to reading it from the
URL.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Derek McGowan
32e2260be2
Enable struct check
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 13:55:43 -07:00
Kevin Lin
1bfbeca726 Properly follow relative links when listing tags
The previous code assumed that the link returned when listing tags was
always absolute. However, some registries, such as quay.io, return the
link as a relative link (e.g. the second page for the quay.io/coreos/etcd
image is /v2/coreos/etcd/tags/list?next_page=<truncated>&n=50). Because
the relative link was retrieved directly, the fetch failed (with the
error `unsupported protocol scheme ""`).

Signed-off-by: Kevin Lin <kevin@kelda.io>
2017-11-18 22:04:19 -08:00
Clayton Coleman
3c5f85abd1
Allow clients to request specific manifest media types
The current registry/client sends the registered manifest types in
random order. Allow clients to request a single specific manifest type
or a preferred order as per the HTTP spec.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-10-03 17:06:11 -04:00
Tibor Vass
13e0608bc8 gofmt -w -s registry/client/repository.go
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 13:14:43 -07:00
Tibor Vass
2c58ce1a7f Remove context in NewRegistry and NewRepository
The context parameter was either not used or misused.

Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 10:34:56 -07:00
Clayton Coleman
a2015272c1
Support HEAD requests without Docker-Content-Digest header
A statically hosted registry that responds correctly to GET with a
manifest will load the right digest (by looking at the manifest body and
calculating the digest). If the registry returns a HEAD without
`Docker-Content-Digest`, then the client Tags().Get() call will return
an empty digest.

This commit changes the client to fallback to loading the tag via GET if
the `Docker-Content-Digest` header is not set.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-08-25 17:18:01 -04:00
Stephen J Day
9c88801a12
context: remove definition of Context
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.

To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-11 15:53:31 -07:00
Stephen J Day
532ec9f036
digest: migrate to opencontainers/go-digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 15:42:03 -08:00
Stephen J Day
01dfa0fcb9
digest: use digest.Parse over ParseDigest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-15 15:07:42 -08:00
a-palchikov
82609180a1 tag service: properly handle error responses on HEAD requests by (#1918)
* tag service: properly handle error responses on HEAD requests by
re-issuing requests as GET for proper error details.

Fixes #1911.

Signed-off-by: dmitri <deemok@gmail.com>

* Simplify handling of failing HEAD requests in TagService and
make a GET request for cases:
  - if the server does not handle HEAD
  - if the response was an error to get error details

Signed-off-by: dmitri <deemok@gmail.com>

* Add a missing http.Response.Body.Close call for the GET request.

Signed-off-by: dmitri <deemok@gmail.com>
2016-08-26 14:35:04 -07:00
Richard Scothern
857d0f15c0 Merge pull request #1757 from miminar/cross-mount-middleware-repository
Allow to use repo middleware wrapper during cross-repo mounts
2016-07-19 17:43:10 -07:00
Michal Minář
3f1434525b Export storage.CreateOptions in top-level package
Let the options for `BlobStore.Create()` be modified in middleware
wrappers.

Signed-off-by: Michal Minar <miminar@redhat.com>
2016-07-06 16:40:46 +02:00
Brian Bland
a1f9f71e67 Changes the client Tags All() method to follow links
This returns all tags even when the registry forces pagination.

Signed-off-by: Brian Bland <brian.t.bland@gmail.com>
2016-06-28 15:49:14 -07:00
Derek McGowan
125f4ff7d7 Add option to get content digest from manifest get
The client may need the content digest to delete a manifest using the digest used by the registry.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 17:02:29 -07:00
Richard Scothern
87a997249d Merge pull request #1522 from RichardScothern/tag-events
Send tag events to notification listeners
2016-03-23 16:41:29 -07:00
Richard Scothern
bc9c820e4b Enable URLs returned from the registry to be configured as relative.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 15:14:32 -07:00
Richard Scothern
afe2bdd1c5 Propogate tag as a functional argument into the notification system to attach
tags to manifest push and pull event notifications.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 14:57:52 -07:00
Antonio Murdaca
75b5a1b1f6 registry: client: repository: close response body
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 19:53:17 +01:00
Richard Scothern
7144936612 Enable proxying registries to downgrade fetched manifests to Schema 1.
Ensure Accept headers are sent with TagService.Get (which hits manifest
endpoints).  Add support for remote Get and Put for the proxied blobstore.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-22 17:56:48 -08:00
Richard Scothern
dd58349b35 Merge pull request #1419 from aaronlehmann/safer-header-copying
On redirect, only copy headers when they don't already exist in the redirected request
2016-02-05 12:14:01 -08:00
Aaron Lehmann
b0989446eb Rename Name method of Repository to Named
This makes code that gets the name as a string read like
repo.Named().Name() instead of repo.Name().Name().

Requested in
https://github.com/docker/docker/pull/19887#discussion_r51479753

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-04 09:55:36 -08:00
Aaron Lehmann
4354f0a107 On redirect, only copy headers when they don't already exist in the redirected request
A changeset under consideration for Go 1.7 would automatically copy
headers on redirect. This change future-proofs our code so we won't make
duplicate copies of the headers if net/http does it automatically in the
future.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-03 13:19:44 -08:00
Aaron Lehmann
74c4a0e3e3 Merge pull request #1393 from RichardScothern/putbydgst
Add manifest put by digest to the registry client
2016-01-28 14:41:07 -08:00
Aaron Lehmann
8fd84c4a57 Add a CheckRedirect function to the HTTP client
Use it to preserve Accept and Range headers that were added to the
original request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Richard Scothern
9f72e8442d Add manifest put by digest to the registry client
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 14:20:23 -08:00
Aaron Lehmann
2b20b0167a Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann
4441333912 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Brian Bland
8c5a6c13c0 Splits up blob create options definitions to be package-specific
Redefines privately in both storage and client packages

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-14 10:47:33 -08:00
Brian Bland
ce88d8a6f4 Adds functional options arguments to the Blobs Create method
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 16:42:59 -08:00
Brian Bland
5df21570a7 Adds cross-repository blob mounting behavior
Extends blob upload POST endpoint to support mount and from query
parameters as described in #634

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:18 -08:00
Kenny Leung
c28278f7a2 Print error for failed HTTP auth request.
Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-29 11:39:20 -08:00
Richard Scothern
67d3675d55 Merge pull request #1268 from RichardScothern/manifest-refactor-impl
Implementation of the Manifest Service API refactor.
2015-12-17 17:32:55 -08:00
Richard Scothern
cb6f002350 Implementation of the Manifest Service API refactor.
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.

Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.

Remove Signatures() from Repository interface.

Signatures are relevant only to schema1 manifests.  Move access to the signature
store inside the schema1 manifestStore.  Add some API tests to verify
signature roundtripping.

schema1
-------

Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.

tags
----

Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces.  Implement a driver based tagService to be backward
compatible with the current tag service.

Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.

manifest store
--------------

Remove revision store and move all signing functionality into the signed manifeststore.

manifest registration
---------------------

Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.

client
------

Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs.  Change functional arguments to be an interface type
to allow passing data without mutating shared state.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2015-12-17 17:09:14 -08:00