Commit graph

2614 commits

Author SHA1 Message Date
Stephen J Day
1ba5b3b553
registry/storage: ignore missing tag on delete
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-02-06 15:25:51 -08:00
Stephen Day
fea8bd5114
Merge pull request #2471 from GarageDeveloper/master
fixes #2249: sanitize tenant and tenantid
2018-02-02 16:29:34 -08:00
Derek McGowan
85b4d46b4f
Merge pull request #2507 from mistyhacks/fix-storage-driver-link
Fixed broken storage driver link
2018-02-01 13:49:34 -08:00
Felix Bünemann
4ecb17cc4c registry: support whitelisting letsencrypt hosts
This adds a configuration setting `HTTP.TLS.LetsEncrypt.Hosts` which can
be set to a list of hosts that the registry will whitelist for retrieving
certificates from Let's Encrypt. HTTPS connections with SNI hostnames
that are not whitelisted will be closed with an "unknown host" error.
It is required to avoid lots of unsuccessful registrations attempts that
are triggered by malicious clients connecting with bogus SNI hostnames.

NOTE: Due to a bug in the deprecated vendored rsc.io/letsencrypt library
clearing the host list requires deleting or editing of the cachefile to
reset the hosts list to null.

Signed-off-by: Felix Buenemann <felix.buenemann@gmail.com>
2018-02-01 21:16:58 +01:00
Misty Stanley-Jones
a4c32bce50 Fixed broken storage driver link
Signed-off-by: Misty Stanley-Jones <misty@docker.com>
2018-01-25 14:54:50 -08:00
Derek McGowan
5cb406d511
Merge pull request #2455 from sargun/make-walk-faster
s3: improve walk performance
2018-01-19 13:00:03 -08:00
Sargun Dhillon
cbcbcb02c5 Remove old walk function
This removes the old global walk function, and changes all
the code to use the per-driver walk functions.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-18 13:06:07 -08:00
Sargun Dhillon
c7b0da2622 Use the new walk method for catalog enumeration
This change is primarily to make GC faster.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-18 11:58:46 -08:00
Derek McGowan
b5db8eeeb8
Merge pull request #2500 from stevvooe/remove-net-context-vestiges
context, registry/handlers: remove net/context references
2018-01-18 10:26:24 -08:00
Stephen J Day
585cdeb571
context, registry/handlers: remove net/context references
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-17 16:01:41 -08:00
Sargun Dhillon
35b29a609e Use the New Driver Walk method for catalog enumeration
This changes the Walk Method used for catalog enumeration. Just to show
how much an effect this has on our s3 storage:
Original:
List calls: 6839

real    3m16.636s
user    0m0.000s
sys    0m0.016s

New:
ListObjectsV2 Calls: 1805

real    0m49.970s
user    0m0.008s
sys    0m0.000s

This is because it no longer performs a list and stat per item, and instead
is able to use the metadata gained from the list as a replacement to stat.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-07 22:45:17 -08:00
Sargun Dhillon
32ac467992 Introduce Walk Method Per Storage Driver
Move the Walk types into registry/storage/driver, and add a Walk method to each
storage driver. Although this is yet another API to implement, there is a fall
back implementation that relies on List and Stat. For some filesystems this is
very slow.

Also, this WalkDir Method conforms better do a traditional WalkDir (a la filepath).

This change is in preparation for refactoring.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-07 22:45:17 -08:00
Sargun Dhillon
9f664468ea Upgrade AWS library to v1.12.36
This upgrade, and vendors aws-sdk-go to version v1.12.36.
This is because it has new API calls accessible to the S3 client,
specifically S3.ListObjectsV2PagesWithContext

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-07 22:45:17 -08:00
Derek McGowan
277ed486c9
Merge pull request #2302 from zetaab/oma
add possibility to clean untagged manifests
2018-01-05 15:27:52 -08:00
Jesse Haka
ff87ad884c add possibility to clean untagged manifests
add tests

add possibility to clean untagged manifests

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

add dry tests

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

remove underscores

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

fixes

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

opts struct+use camelcase

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

doublecheck manifest in paths.go

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

add gofmt

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

fix lint

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

add log print

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

move log to dryrun as well

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

remove counter

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

remove manifest tag references

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

add tag to tests

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

manifestsWithoutTags -> removeUntagged

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

remove RemoveManifestTagReferences and use removemanifests

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

remove comment

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

remove pathfor

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

move removemanifest out of manifestenumerator, it does not work correctly if we delete stuff in it

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

add comment

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

fix context -> dcontext

Signed-off-by: Jesse Haka <haka.jesse@gmail.com>

fix gofmt
2018-01-05 12:20:01 +02:00
Stephen Day
db0bc08b85
Merge pull request #2484 from perlun/patch-1
Update CONTRIBUTING.md
2018-01-02 16:07:18 -08:00
Stephen Day
70c5e31509
Merge pull request #2485 from perlun/patch-2
api.md: Fixed incorrect grammar
2018-01-02 16:06:25 -08:00
Per Lundberg
aac2f6c8b7 api.md: Fixed incorrect grammar
I noted this while reading at https://docs.docker.com/registry/spec/api/

Signed-off-by: Per Lundberg <perlun@gmail.com>
2018-01-02 12:49:31 +02:00
Per Lundberg
eaa90a3c6d Update CONTRIBUTING.md
Fixed the headers. They were rendering incorrectly on GitHub for some weird reason; I copied the strings and rewrote the ## characters at the beginning and now they render as they should.

Signed-off-by: Per Lundberg <per.lundberg@ecraft.com>
2017-12-29 14:41:43 +02:00
Viktor Stanchev
e9864ce8b9 disable schema1 by default, add a config flag to enable it
port of #2473

Signed-off-by: Viktor Stanchev <me@viktorstanchev.com>
2017-12-19 10:23:25 -08:00
Raphaël Enrici
8777e97b72 fixes #2249: sanitize tenant and tenantid
If tenant or tenantid are passed as env variables, we systematically use Sprint to make sure they are string and not integer as it would make mapstructure fail.

Signed-off-by: Raphaël Enrici <raphael@root-42.com>
2017-12-17 18:33:34 +01:00
Derek McGowan
f411848591
Merge pull request #2447 from tifayuki/cloudfront-s3-filter
add s3 region filters for cloudfront
2017-12-07 10:04:35 -08:00
tifayuki
e8ecc6dc55 add s3 region filters for cloudfront
Signed-off-by: tifayuki <tifayuki@gmail.com>
2017-12-01 15:58:58 -08:00
Stephen Day
bc3c7b0525
Merge pull request #2435 from zhouhaibing089/digestset-ut
digestset: refine some words on unit test
2017-11-22 16:42:22 -08:00
Stephen Day
8cedd7b3a4
Merge pull request #2448 from kklin/follow-relative-links
Properly follow relative links when listing tags
2017-11-22 16:38:47 -08:00
Stephen Day
1503fa2109
Merge pull request #2431 from mistyhacks/fix-keywords
Fix keyword format for downstream docs
2017-11-22 16:34:27 -08:00
Monika Katiyar
118c8ee1f1 Added nginx configuration for uploading large sized blobs (layer size) to registry when using nginx as reverse proxy from monikakatiyar16
Signed-off-by: Monika Katiyar <monika@jeavio.com>
2017-11-20 15:17:47 +05:30
Kevin Lin
1bfbeca726 Properly follow relative links when listing tags
The previous code assumed that the link returned when listing tags was
always absolute. However, some registries, such as quay.io, return the
link as a relative link (e.g. the second page for the quay.io/coreos/etcd
image is /v2/coreos/etcd/tags/list?next_page=<truncated>&n=50). Because
the relative link was retrieved directly, the fetch failed (with the
error `unsupported protocol scheme ""`).

Signed-off-by: Kevin Lin <kevin@kelda.io>
2017-11-18 22:04:19 -08:00
gbarr01
e5b5e44386 Revert "Add text about win os version ordering"
This reverts commit 1a860d8c19.
2017-11-09 14:49:04 -08:00
gbarr01
1a860d8c19
Add text about win os version ordering 2017-11-09 14:15:00 -08:00
zhouhaibing089
f74613907d digestset: refine some words on unit test
1. when lookup an entry which is missing, it should say NotFound.
2. when add duplicated entry, the entries size should be increased.
3. when add entry which has different algorithm, it should be allowed.

Signed-off-by: zhouhaibing089 <zhouhaibing089@gmail.com>
2017-10-31 16:33:36 +08:00
Misty Stanley-Jones
fda42e5ef9 Fix keyword format for downstream docs
Signed-off-by: Misty Stanley-Jones <misty@docker.com>
2017-10-24 15:26:28 -07:00
Stephen Day
3800056b88 Merge pull request #2428 from zhouhaibing089/algo-import
cmd/digest: import crypto algorithms
2017-10-23 10:07:44 +02:00
zhouhaibing089
d66208108d cmd/digest: import crypto algorithms
the digest cli does not work if we do not import this two packages,
tested in go1.9. basically, we have to make several algorithms to
be available by calling crypto.RegisterHash in init functions.

Signed-off-by: zhouhaibing089 <zhouhaibing089@gmail.com>
2017-10-14 12:36:55 +08:00
Derek McGowan
7484e51bf6 Merge pull request #2402 from smarterclayton/order_manifest
Allow clients to request specific manifest media types
2017-10-11 10:17:12 -07:00
Clayton Coleman
3c5f85abd1
Allow clients to request specific manifest media types
The current registry/client sends the registered manifest types in
random order. Allow clients to request a single specific manifest type
or a preferred order as per the HTTP spec.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-10-03 17:06:11 -04:00
Derek McGowan
bb49a1685d Merge pull request #2416 from stevvooe/feed-the-linter
registry: feed the linter by removing redundant err check
2017-09-29 13:51:02 -07:00
Stephen J Day
4abf680c76
registry: feed the linter by removing redundant err check
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-29 13:27:49 -07:00
Derek McGowan
8710fa58ae Merge pull request #2377 from stevvooe/healthcheck-storage-ignore-notfound
registry/handlers: ignore notfound on storage driver healthcheck
2017-09-29 13:14:15 -07:00
Derek McGowan
5db89f0ca6 Merge pull request #2388 from tiborvass/remove-misused-context
Remove context in NewRegistry and NewRepository
2017-09-05 13:44:47 -07:00
Tibor Vass
13e0608bc8 gofmt -w -s registry/client/repository.go
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 13:14:43 -07:00
Tibor Vass
2c58ce1a7f Remove context in NewRegistry and NewRepository
The context parameter was either not used or misused.

Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 10:34:56 -07:00
Derek McGowan
30578ca329 Merge pull request #2384 from smarterclayton/client_retry
Support HEAD requests without Docker-Content-Digest header
2017-08-25 15:06:52 -07:00
Clayton Coleman
a2015272c1
Support HEAD requests without Docker-Content-Digest header
A statically hosted registry that responds correctly to GET with a
manifest will load the right digest (by looking at the manifest body and
calculating the digest). If the registry returns a HEAD without
`Docker-Content-Digest`, then the client Tags().Get() call will return
an empty digest.

This commit changes the client to fallback to loading the tag via GET if
the `Docker-Content-Digest` header is not set.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-08-25 17:18:01 -04:00
Derek McGowan
d9e0121fef Merge pull request #2382 from smarterclayton/scope_check
If the request already has the scope, don't fetchToken again
2017-08-24 15:50:00 -07:00
Clayton Coleman
23f8ca88e1
If the request already has the scope, don't force token fetch
AuthorizeRequest() injects the 'pull' scope if `from` is set
unconditionally. If the current token already has that scope, it will
be inserted into the scope list twice and `addedScopes` will be set to
true, resulting in a new token being fetched that has no net new scopes.

Instead, check whether `additionalScopes` are actually new.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-08-23 19:27:37 -04:00
Stephen J Day
1618b49d5b
registry/handlers: ignore notfound on storage driver healthcheck
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-21 15:04:31 -07:00
Stephen Day
5f6282db7d Merge pull request #2371 from lazize/patch-1
Fix registry version argument
2017-08-17 10:56:59 -07:00
Leonardo Azize Martins
3f771adca6 Fix registry version argument
Signed-off-by: Leonardo Azize Martins <leonardo.azize@scania.com>
2017-08-17 08:35:39 +02:00
Derek McGowan
e0b4f55f2b Merge pull request #2366 from stevvooe/remove-logging-dependencies
registry: remove dependency on logrus for client
2017-08-15 11:03:52 -07:00