This integrates the new module, which was extracted from this repository
at commit b9b19409cf458dcb9e1253ff44ba75bd0620faa6;
# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo
# create a temporary clone of docker
cd ~/Projects
git clone https://github.com/distribution/distribution.git reference
cd reference
# commit taken from
git rev-parse --verify HEAD
b9b19409cf
# remove all code, except for general files, 'reference/', and rename to /
git filter-repo \
--path .github/workflows/codeql-analysis.yml \
--path .github/workflows/fossa.yml \
--path .golangci.yml \
--path distribution-logo.svg \
--path CODE-OF-CONDUCT.md \
--path CONTRIBUTING.md \
--path GOVERNANCE.md \
--path README.md \
--path LICENSE \
--path MAINTAINERS \
--path-glob 'reference/*.*' \
--path-rename reference/:
# initialize go.mod
go mod init github.com/distribution/reference
go mod tidy -go=1.20
This commit is based on 152af63ec5 in the main branch,
but adjusted for the 2.8 branch, with some differences:
- the Sort functions have not been kept, as they were not part of the v2 package,
and introduced in 1052518d9f
- the ParseAnyReferenceWithSet and ShortIdentifierRegexp were kept (but deprecated)
as removing happened in 6d4f62d7fd, which is not
in the 2.8 branch.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
OCI Image manifests and indexes are supported both with and without
an embeded MediaType (the field is reserved according to the spec).
Test storing and retrieving both types from the manifest store.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
In the OCI image specification, the MediaType field is reserved
and otherwise undefined; assume that manifests without a media
in storage are OCI images or image indexes, and determine which
by looking at what fields are in the JSON. We do keep a check
that when unmarshalling an OCI image or image index, if it has
a MediaType field, it must match that media type of the upload.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
When unmarshalling manifests from JSON, check that the MediaType field
corresponds to the type that we are unmarshalling as. This makes sure
that when we retrieve a manifest from the manifest store, it will have
the same type as it was handled as before storing it in the manifest
store.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.
To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The registry uses partial Named values which the named parsers
no longer support. To allow the registry service to continue
to operate without canonicalization, switch to use WithName.
In the future, the registry should start using fully canonical
values on the backend and WithName should no longer support
creating partial values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Modify manifest builder so it can be used to build
manifests with different configuration media types.
Rename config media type const to image config.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
To allow generic manifest walking, we define an interface method of
`References` that returns the referenced items in the manifest. The
current implementation does not return the config target from schema2,
making this useless for most applications.
The garbage collector has been modified to show the utility of this
correctly formed `References` method. We may be able to make more
generic traversal methods with this, as well.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
When we push a schema1 manifest, we encode history information from the
image JSON into v1Compatibility strings for the respective layers. The
"author" field was not being set in these v1Compatibility strings, so if
a parent layer had an author set, it would not be preserved after
pushing through a schema1 manifest and repulling, so the image ID would
change after the pull. This change preserves the authorship information
for parent layers so that the image ID does not change.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The schema2 manifest builder fills in this part of the manifest based on
the descriptor it gets back from BlobIngester's Put method. It passes
the correct media type to Put, but Put ends up replacing this value with
application/octet-stream in its return value.
This commit works around the issue in the manifest builder. Arguably Put
should not be changing the media type in its return value, but this
commit is a targeted fix to keep it very low-risk for possible inclusion
in Docker 1.11.
Fixes#1621 (but maybe we should open a separate issue for the media
type behavior in the distribution client, and the unnecessary stat).
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Add option for specifying trust key for signing schema1 manifests.
Since schema1 signature key identifiers are not verified anywhere and deprecated, storing signatures is no longer a requirement.
Furthermore in schema2 there is no signature, requiring the registry to already add signatures to generated schema1 manifests.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The Payload function for schema1 currently returns a signed manifest,
but indicates the content type is that of a manifest that isn't signed.
Note that this breaks compatibility with Registry 2.3 alpha 1 and
Docker 1.10-rc1, because they use the incorrect content type.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
For compatibility with other registries that don't use this exact
variant of the Content-Type header, we need to be more flexible about
what we accept. Any form of "application/json" should be allowed. The
charset should not be included in the comparison.
See docker/docker#19400.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository
Signed-off-by: Brian Bland <brian.bland@docker.com>
This makes content type sniffing cleaner. The document just needs to be
decoded into a manifest.Versioned structure. It's no longer a two-step
process.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Add schema2 manifest implementation.
Add a schema2 builder that creates a schema2 manifest from descriptors
and a configuration. It will add the configuration to the blob store if
necessary.
Rename the original schema1 manifest builder to ReferenceBuilder, and
create a ConfigBuilder variant that can build a schema1 manifest from an
image configuration and set of descriptors. This will be used to
translate schema2 manifests to the schema1 format for backward
compatibliity, by adding the descriptors from the existing schema2
manifest to the schema1 builder. It will also be used by engine-side
push code to create schema1 manifests from the new-style image
configration, when necessary to push a schema1 manifest.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.
Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.
Remove Signatures() from Repository interface.
Signatures are relevant only to schema1 manifests. Move access to the signature
store inside the schema1 manifestStore. Add some API tests to verify
signature roundtripping.
schema1
-------
Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.
tags
----
Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces. Implement a driver based tagService to be backward
compatible with the current tag service.
Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.
manifest store
--------------
Remove revision store and move all signing functionality into the signed manifeststore.
manifest registration
---------------------
Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.
client
------
Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs. Change functional arguments to be an interface type
to allow passing data without mutating shared state.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
