Commit graph

1207 commits

Author SHA1 Message Date
João Pereira
8ef268df25
Add tests for tags list pagination
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-05-22 15:31:18 +01:00
João Pereira
d80a63f1ea
Merge pull request #3143 from eyJhb/pagination
OCI: Add pagination on `/v2/<name>/tags/list`
2021-05-22 15:05:18 +01:00
Wang Yan
f65a33d3c8
Merge pull request #3227 from wy65701436/fix-conformance-416
OCI: add content range handling in patch blob
2021-05-11 23:51:17 +08:00
wang yan
d7a2b14489 add content range handling in patch blob
Fixes #3141

1, return 416 for Out-of-order blob upload
2, return 400 for content length and content size mismatch

Signed-off-by: wang yan <wangyan@vmware.com>
2021-05-11 23:35:15 +08:00
Hayley Swimelar
9329f6a62b
Merge pull request #3395 from fuweid/verify-layer-config-descriptor
OCI: verify digest and check blob presence when put manifest
2021-05-07 10:38:45 -07:00
eyjhb
4da2712b52
added pagination to v2/<name>/tags/list endpoint
Signed-off-by: eyjhb <eyjhbb@gmail.com>
2021-04-23 18:06:04 +02:00
eyjhbb@gmail.com
febc8733d2
added error codes for pagination
Signed-off-by: eyjhb <eyjhbb@gmail.com>
2021-04-23 18:06:03 +02:00
João Pereira
a27b0c4952
Merge pull request #3161 from bloodorangeio/redis-tls
Add configuration option for Redis TLS
2021-04-23 16:52:54 +01:00
Wei Fu
9e618c90c3 registry: verify digest and check blob presence when put manifest
According to OCI image spec, the descriptor's digest field is required.
For the normal config/layer blobs, the valivation should check the
presence of the blob when put manifest.

REF: https://github.com/opencontainers/image-spec/blob/v1.0.1/descriptor.md

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-04-16 16:11:52 +08:00
sangluo
334a7e7ff6 close the io.ReadCloser from storage drive
Signed-off-by: sangluo <sangluo@pinduoduo.com>
2021-03-24 17:58:35 +08:00
Wang Yan
077b38b95e
Merge pull request #3364 from Agilicus/escape-json
fix: disable DisableHTMLEscape on logrus json logging
2021-03-24 11:16:55 +08:00
Derek McGowan
a01c71e247
Merge pull request #2815 from bainsy88/issue_2814
Add code to handle pagination of parts. Fixes max layer size of 10GB bug
2021-03-16 09:12:03 -07:00
Josh Dolitsky
32ccbf193d
Add configuration option for Redis TLS
Signed-off-by: Josh Dolitsky <josh@dolit.ski>
2021-03-01 18:55:56 -05:00
Adam Wolfe Gordon
43e502590f log: Include configured fields in all logs
It's possible to configure log fields in the configuration file, and we would
like these fields to be included in all logs. Previously these fields were
included only in logs produced using the main routine's context, meaning that
any logs from a request handler were missing the fields since those use a
context based on the HTTP request's context.

Add a configurable default logger to the `context` package, and set it when
configuring logging at startup time.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
2021-03-01 14:44:56 -07:00
Milos Gajdos
17ab8afeec
Merge pull request #3174 from adamwg/manifest-delete-by-tag
manifests: Return UNSUPPORTED when deleting manifests by tag
2021-03-01 17:21:03 +00:00
Milos Gajdos
5a76dc8df1
Merge pull request #3169 from d-luu/configurable_ciphersuites
Added flag for user-configurable cipher suites

Thanks, @d-luu for putting the effort into this. Much appreciated!
2021-02-26 08:17:32 +00:00
David Luu
1e625d0076 Added flag for user configurable cipher suites
Configuration of list of cipher suites allows a user to disable use
of weak ciphers or continue to support them for legacy usage if they
so choose.

List of available cipher suites at:
https://golang.org/pkg/crypto/tls/#pkg-constants

Default cipher suites have been updated to:
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_AES_128_GCM_SHA256
- TLS_CHACHA20_POLY1305_SHA256
- TLS_AES_256_GCM_SHA384

MinimumTLS has also been updated to include TLS 1.3 as an option
and now defaults to TLS 1.2 since 1.0 and 1.1 have been deprecated.

Signed-off-by: David Luu <david@davidluu.info>
2021-02-25 14:19:56 -06:00
Don Bowman
9c43ba9dcc
fix: disable DisableHTMLEscape on logrus json logging
Fixes #3363

Without this, we emit illegal json logs, the user-agent
ends up as:

```
"http.request.useragent": "docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.10.10-051010-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 \(linux\))"
```

which is not valid according to [spec](https://www.json.org/json-en.html)

specifically, string: "<any codepoint except " or \ or control>*"

Signed-off-by: Don Bowman <don@agilicus.com>
2021-02-15 13:16:27 -05:00
olegburov
907e7be545 Bump Logstash hook for logrus to v1.0.0.
Signed-off-by: olegburov <oleg.burov@outlook.com>
2021-02-11 21:51:13 -08:00
Sebastiaan van Stijn
1d33874951
go.mod: change imports to github.com/distribution/distribution/v3
Go 1.13 and up enforce import paths to be versioned if a project
contains a go.mod and has released v2 or up.

The current v2.x branches (and releases) do not yet have a go.mod,
and therefore are still allowed to be imported with a non-versioned
import path (go modules add a `+incompatible` annotation in that case).

However, now that this project has a `go.mod` file, incompatible
import paths will not be accepted by go modules, and attempting
to use code from this repository will fail.

This patch uses `v3` for the import-paths (not `v2`), because changing
import paths itself is a breaking change, which means that  the
next release should increment the "major" version to comply with
SemVer (as go modules dictate).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-08 18:30:46 +01:00
João Pereira
6200038bc7
Merge pull request #3204 from stefannica/fsDriverRelaxedPermissions
Relax filesystem driver folder permissions to 0777
2021-02-06 16:12:02 +00:00
João Pereira
f82e1c2332
Merge pull request #3300 from jubalh/sp
Fix minor spelling mistakes
2021-02-06 16:03:09 +00:00
João Pereira
038a5060de
Merge pull request #2905 from 2opremio/master
Honor contexts passed to registry client methods
2021-02-06 15:25:03 +00:00
Michael Vetter
4d34a31762 Correct spelling: decription -> description
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
2021-02-03 13:12:23 +01:00
Michael Vetter
084c0bd100 Fix typo in docu of NewURLBuilderFromString()
And one more minor typo.

Signed-off-by: Michael Vetter <jubalh@iodoru.org>
2021-02-03 13:12:16 +01:00
Michael Vetter
f0c93f65a2 Fix typo in NewSimpleManager() documentation
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
2021-02-03 13:11:53 +01:00
Ricardo Maraschini
87cbd09fa7 Ignore self reference object on empty prefix
When a given prefix is empty and we attempt to list its content AWS
returns that the prefix contains one object with key defined as the
prefix with an extra "/" at the end.

e.g.

If we call ListObjects() passing to it an existing but empty prefix,
say "my/empty/prefix", AWS will return that "my/empty/prefix/" is an
object inside "my/empty/prefix" (ListObjectsOutput.Contents).

This extra "/" causes the upload purging process to panic. On normal
circunstances we never find empty prefixes on S3 but users may touch
it.

Signed-off-by: Ricardo Maraschini <rmarasch@redhat.com>
2020-11-30 13:04:14 +01:00
Arko Dasgupta
551158e600
Merge pull request #3289 from Smasherr/master
#3288 Remove empty Content-Type header
2020-11-13 18:03:30 -08:00
Arko Dasgupta
065aec5688
Merge pull request #3239 from olegburov/bump-redigo
Upgrade Redigo to `1.8.2`.
2020-11-04 18:19:51 -08:00
Smasherr
c8d90f904f Remove empty Content-Type header
Fixes #3288

Signed-off-by: Smasherr <soundcracker@gmail.com>
2020-11-03 12:24:02 +01:00
olegburov
34f1322664 Fix hardcoded credential provides.
Signed-off-by: olegburov <oleg.burov@outlook.com>
2020-09-02 17:32:34 -07:00
olegburov
264e26fd8c Bump Redigo to v1.8.2.
Signed-off-by: olegburov <oleg.burov@outlook.com>
2020-08-31 18:07:54 -07:00
Manish Tomar
2800ab0224
Merge pull request #2973 from redmatter/support-ecs-instance-profile-in-s3-driver
Support ECS TaskRole in S3 storage driver
2020-08-26 15:24:20 -07:00
Adam Wolfe Gordon
95f1eea5f5 manifests: Return UNSUPPORTED when deleting manifests by tag
The OCI distribution spec allows implementations to support deleting manifests
by tag, but also permits returning the `UNSUPPORTED` error code for such
requests. docker/distribution has never supported deleting manifests by tag, but
previously returned `DIGEST_INVALID`.

The `Tag` and `Digest` fields of the `manifestHandler` are already correctly
populated based on which kind of reference was given in the request URL. Return
`UNSUPPORTED` if the `Tag` field is populated.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
2020-08-04 11:40:27 -06:00
Adam Wolfe Gordon
a784441b62 catalog: List repositories with no unique layers
A repository need not contain any unique layers, if its images use only layers
mounted from other repositories. But, the catalog endpoint was looking for the
_layers directory to indicate that a directory was a repository.

Use the _manifests directory as the marker instead, since any repository with
revisions will contain a _manifests directory.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
2020-08-04 11:38:49 -06:00
zhipengzuo
f361d443b7 clean up code because err is always nil
Signed-off-by: zhipengzuo <zuozhipeng@baidu.com>
2020-07-22 10:54:46 +08:00
Stefan Nica
2672c0ebe2 Relax filesystem driver folder permissions to 0777 (cont)
There was a previous PR relaxing the filsystem driver permissions
for files and folders to 0666 and 0777 respectively [1][2], but it was
incomplete. This is required to get the registry to honor the umask
value.

[1] https://github.com/docker/distribution/pull/1304/
[2] https://github.com/docker/distribution/issues/1295

Signed-off-by: Stefan Nica <snica@suse.com>
2020-07-16 11:59:08 +02:00
Andrew Bulford
9690d843fa Support ECS TaskRole in S3 storage driver
Instead of constructing the list of credential providers manually, if we
use the default list we can take advantage of the AWS SDK checking the
environment and returning either the EC2RoleProvider or the generic HTTP
credentials provider, configured to use the ECS credentials endpoint.

Also, use the `defaults.Config()` function instead of `aws.NewConfig()`,
as this results in an initialised HTTP client which prevents a fatal
error when retrieving credentials from the ECS credentials endpoint.

Fixes #2960

Signed-off-by: Andrew Bulford <andrew.bulford@redmatter.com>
2020-07-01 08:42:56 +01:00
Derek McGowan
742aab907b
Merge pull request #3127 from dmage/err-shadow
Fix err shadowing in gcs driver
2020-03-19 10:36:57 -07:00
Derek McGowan
78c2ab6646
Fix gosimple checks
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-18 15:52:34 -07:00
Oleg Bulatov
cdb4ba947a
Fix err shadowing in gcs driver
Signed-off-by: Oleg Bulatov <oleg@bulatov.me>
2020-03-18 11:41:34 +01:00
Derek McGowan
581be91482
Merge pull request #3113 from dmcgowan/upstream-redis-fixes
Redis cache fixes and metrics
2020-03-09 13:36:53 -07:00
Derek McGowan
be29c05a1e
Remove deprecated cache metrics code
The metrics tracker in cached blob statter was replaced with prometheus
metrics and no longer needed.
Remove unused log wrapping.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-09 13:11:54 -07:00
Derek McGowan
495a4af7cf
Fix goimports
Separate fix for cherry-picked code

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Grant Watters
74d442a058
Consider redis.ErrNil as distribution.ErrBlobUnknown for Stat HGET
* Update redis.go

If the dgst key does not exist in the cache when calling HGET, `redis.String` will return an `ErrNil` which we need to translate into `distribution.ErrBlobUnknown` so that the error being returned can be properly handled. This will ensure that `SetDescriptor` is properly called from `cachedBlobStatter::Stat` for `repositoryScopedRedisBlobDescriptorService` which will update the redis cache and be considered as a Miss rather than an Error.

cc @manishtomar

* Update suite.go

Add unit test to ensure missing blobs for scoped repo properly return ErrBlobUnknown when HGET returns redis.ErrNil.

(cherry picked from commit dca6b9526a1d30dd218a9f321c4f84ecc4b5e62e)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Manish Tomar
795892662b
redis metrics
* redis metrics

it is working but metrics are not very useful since default buckets
start from 5ms and almost all of them are in that range.

* remove extra comment

(cherry picked from commit ba1a1d74e7eb047dd1056548ccf0695e8846782c)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Manish Tomar
ce101280fe
fix redis caching issue
* fix redis caching issue

earlier redis cache was updated when there was any error including any
temporary connectivity issue. This would trigger set calls which would
further increase load and possibly connectivity errors from redis
leaving the system with continuous errors and high latency. Now the
cache is updated only when it is genuine cache miss. Other errors do not
trigger a cache update.

* add back tracker Hit() and Miss() calls

*squashed commits*
(cherry picked from commit 6f3e1c10260ef59ba4e9c42e939329fad9fdd8c3)
(cherry picked from commit 6738ff3320cf82cc2df919a95a1bde2f7789a501)

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Derek McGowan
4c7c63b557
Add cache unit test
Test base functionality of the cache statter

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Elliot Pahl
800cb95821
Use go-events package
TBD: Queue not converted yet

Signed-off-by: Elliot Pahl <elliot.pahl@gmail.com>
2020-03-04 12:49:32 -08:00
Alfonso Acosta
15f7bd29a5 Remove {get,head}WithContext()
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 13:36:43 +01:00
Alfonso Acosta
282351e954 Use http.NewRequestWithContext()
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 13:36:31 +01:00
Alfonso Acosta
58331abf58 Honor contexts passed to registry client methods
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
2020-02-25 12:41:15 +01:00
Derek McGowan
6b972e50fe
Merge pull request #2272 from naveedjamil/fips
Increase Unit Test Code Coverage
2020-02-22 17:40:41 -08:00
Derek McGowan
e65b3f1316
Fix CI for test updates
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-02-22 17:27:55 -08:00
Naveed Jamil
efdba4f210
Increase Unit Test Code Coverage
Unit test coverge was increased to cover the usages of crypto. This helps to ensure that everything is working fine with fips mode enabled.
Also updated sha1 to sha256 in registry/storage/driver/testsuites/testsuites.go because sha1 is not supported in fips mode.

Signed-off-by: Naveed Jamil <naveed.jamil@tenpearl.com>
2020-02-22 17:09:00 -08:00
Derek McGowan
bfa6b923e7
Merge pull request #3020 from kd7lxl/updatefrequency
Fix typo cloudfront updatefrenquency
2020-02-22 16:54:03 -08:00
sayboras
66809646d9 Migrate to golangci-lint
Signed-off-by: Tam Mach <sayboras@yahoo.com>
2020-02-14 08:11:16 +11:00
Derek McGowan
a837179414
Merge pull request #3072 from fermayo/fix-TestRegistryAsCacheMutationAPIs
Fix TestRegistryAsCacheMutationAPIs
2019-12-15 20:48:56 -08:00
Fernando Mayo Fernandez
6ca7b9e9fa
Fix TestRegistryAsCacheMutationAPIs
Use a synthetic upstream registry when creating the testing mirror configuration
to avoid the test fail when trying to reach http://example.com

Signed-off-by: Fernando Mayo Fernandez <fernando@undefinedlabs.com>
2019-12-15 13:51:25 +01:00
Tom Hayward
0f5e2753a6 Fix typo cloudfront updatefrenquency
Signed-off-by: Tom Hayward <thayward@infoblox.com>
2019-12-13 12:11:42 -08:00
Derek McGowan
bdf3438b52
Merge pull request #2985 from novas0x2a/default-transport
make it possible to wrap the client transport in another one
2019-12-13 11:36:12 -08:00
Guillaume Rose
c9c3324300 Add unit tests for BlobEnumerator
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-06 09:38:13 +01:00
Manish Tomar
5538da4923 fixes to make layersPathSpec work
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-05 20:48:33 +01:00
Manish Tomar
fa7d949408 allow Repository.BlobStore to enumerate over blobs
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-05 20:48:33 +01:00
Manish Tomar
cf77113795 add pathspec for repo _layers directory
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-05 20:48:33 +01:00
Ryan Abrams
ae2e973db9
Merge pull request #2748 from manishtomar/tag-digests
API to retrive tag's digests
2019-10-08 12:14:56 -07:00
Ryan Abrams
f656e60de5
Merge pull request #2984 from bouk/rempo
registry: Fix typo in RepositoryRemover warning
2019-09-05 08:23:18 -07:00
Ryan Abrams
740d4d1211
Merge pull request #2918 from dmathieu/test-blob-writer-write
Test httpBlobUpload.Write method
2019-09-05 08:16:36 -07:00
Bouke van der Bijl
1c481d34d9 registry: Fix typo in RepositoryRemover warning
Signed-off-by: Bouke van der Bijl <me@bou.ke>
2019-09-02 16:07:34 +00:00
Mike Lundy
c486db2d71
make it possible to wrap the client transport in another one
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
2019-08-22 17:37:47 -04:00
Ryan Abrams
1fb7fffdb2
Merge pull request #2950 from terinjokes/patches/swift-segment-hash
swift: correct segment path generation
2019-07-11 15:35:31 -07:00
Damien Mathieu
dd3bdee21c implement Repository Blobs upload resuming
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-26 09:46:49 +02:00
Ryan Abrams
90dfea7952
Merge pull request #2921 from dmathieu/repository-serve-blob
Implement Repository ServeBlob
2019-06-25 19:07:38 -07:00
Ryan Abrams
6c72ec2e85
Merge pull request #2927 from dmathieu/blob-create-uuid
Handle Blob Create when the underlying registry doesn't provide 'Docker-Upload-UUID'
2019-06-25 19:06:46 -07:00
Damien Mathieu
898b1f2a53 test httpBlobUpload.Write method
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:32:58 +02:00
Damien Mathieu
c5d5f938e3 fast-stop ServeBlob if we're doing a HEAD request
A registry pointing to ECR is having issues if we try loading the blob

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu
3800c47fd2 Implement Repository ServeBlob
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu
a45e5cb13f handle create blob if the uuid couldn't be retrieved from headers or URL
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Damien Mathieu
8b31a894bd deduce blob UUID from location if it wasn't provided in the headers
Some registries (ECR) don't provide a `Docker-Upload-UUID` when creating
a blob. So we can't rely on that header. Fallback to reading it from the
URL.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Damien Mathieu
94097512db extract blob upload resume into its own method
I've found this logic being in a single method to be quite hard to get.
I believe extracting it makes it easier to read, as we can then more
easily see what the main method does and possibly ignore the intricacies
of `ResumeBlobUpload`.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:25:56 +02:00
Terin Stock
b23dd1ef37 swift: correct segment path generation
When uploading segments to Swift, the registry generates a random file,
by taking the hash of the container path and 32-bytes of random data.
The registry attempts to shard across multiple directory paths, by
taking the first three hex characters as leader.

The implementation in registry, unfortunately, takes the hash of
nothing, and appends it to the path and random data. This results in all
segments being created in one directory.

Fixes: #2407
Fixes: #2311
Signed-off-by: Terin Stock <terinjokes@gmail.com>
2019-06-22 23:44:43 -07:00
Ryan Abrams
84f47e7bb3
Merge pull request #2900 from sevki/no-token-err-fix
fix no error returned in fetchTokenWithOAuth
2019-06-19 19:16:18 -07:00
Ryan Abrams
6c329e56a2
Merge pull request #2920 from dmathieu/blob_writer_offset
Append the written bytes to the blob writer's size
2019-06-19 19:14:49 -07:00
Ryan Abrams
0c394fdd84
Merge pull request #2861 from yuwaMSFT2/master
Fixes #2835 Process Accept header MIME types in case-insensitive way
2019-06-19 19:09:29 -07:00
Ryan Abrams
d80a17d8e0
Merge pull request #2879 from tbe/fix-s3-ceph
Fix s3 driver for supporting ceph radosgw
2019-06-19 19:09:07 -07:00
Ryan Abrams
55287010ce
Merge pull request #2894 from jabrown85/fix-offset-typos
Fix typo: offest -> offset
2019-06-19 19:08:22 -07:00
Tariq Ibrahim
8f9c8094fb
replace rsc.io/letsencrypt in favour of golang.org/x/crypto
Signed-off-by: Tariq Ibrahim <tariq181290@gmail.com>
2019-06-04 12:04:18 -07:00
Damien Mathieu
0e2d080a8a append the written bytes to the blob writer's size
Any byte written should append to the size. Otherwise, the full Size is
always zero.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-05-09 14:17:58 +02:00
Ryan Abrams
3226863cbc
Merge pull request #2849 from Shawnpku/master
support Alibaba Cloud CDN storage middleware
2019-04-16 18:43:06 -07:00
Sevki Hasirci
5afbf32400 fix no error returned in fetchTokenWithOAuth
fetchTokenWithBasicAuth checks if a token is in the token response
but fetchTokenWithOAuth does not

these changes implements the same behaviour for the latter
returning a `ErrNoToken` if a token is not found in the resposne

Signed-off-by: Sevki Hasirci <sevki@cloudflare.com>
2019-04-14 11:05:59 +01:00
Jesse Brown
74f429a5ad Fix typo: offest -> offset
Signed-off-by: Jesse Brown <jabrown85@gmail.com>
2019-04-05 14:20:20 -05:00
Shawn Chen
fd77cf43a6 change package name & format document
Signed-off-by: Shawn Chen <chen8132@gmail.com>
2019-03-18 11:35:46 +08:00
Thomas Berger
c18c6c33b2 S3 Driver: added comment for missing KeyCount workaround
Signed-off-by: Thomas Berger <loki@lokis-chaos.de>
2019-03-15 21:05:21 +01:00
Eohyung Lee
f877726503 Fix s3 driver for supporting ceph radosgw
Radosgw does not support S3 `GET Bucket` API v2 API but v1.
This API has backward compatibility, so most of this API is working
correctly but we can not get `KeyCount` in v1 API and which is only
for v2 API.

Signed-off-by: Eohyung Lee <liquidnuker@gmail.com>
2019-03-08 12:45:20 +01:00
Shawn Chen
3390f32aec fix Context issue
Signed-off-by: Shawn Chen <chen8132@gmail.com>
2019-03-04 17:48:32 +08:00
Shawnpku
6e10631d9c fix default cdn auth duration
Signed-off-by: Shawnpku <chen8132@gmail.com>
2019-03-04 14:53:48 +08:00
Vishesh Jindal
f9a0506191
Bugfix: Make ipfilteredby not required
Signed-off-by: Vishesh Jindal <vishesh92@gmail.com>
2019-03-02 08:58:52 +05:30
Ryan Abrams
d3ddc3572c
Merge pull request #2854 from manishtomar/log-authed-name
Log authorized username
2019-03-01 14:45:43 -08:00
Ryan Abrams
c192a281f8
Merge pull request #2813 from lucab/ups/spec-json-binary
registry: fix binary JSON content-type
2019-03-01 14:26:54 -08:00
Yu Wang
a683c7c235 Fixes #2835 Process Accept header MIME types in case-insensitive way
Use mime.ParseMediaType to parse the media types in Accept header in manifest request. Ignore the failed ones.

Signed-off-by: Yu Wang <yuwa@microsoft.com>
2019-02-21 15:11:41 -08:00
Shawnpku
bbc9885aa2 fix func name
Signed-off-by: Shawnpku <chen8132@gmail.com>
2019-02-20 15:54:21 +08:00
Manish Tomar
ec6566c02b Log authorized username
This is useful to know which user pulled/pushed which repo.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2019-02-13 08:49:37 -08:00
Shawnpku
3aa2a282f7 support alicdn middleware
Signed-off-by: Shawnpku <chen8132@gmail.com>
2019-02-11 15:11:26 +08:00
Manish Tomar
48818fdea7 Remove err nil check
since type checking nil will not panic and return appropriately

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2019-02-04 16:42:44 -08:00
Manish Tomar
da8db4666b Fix gometalint errors
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2019-02-04 16:01:04 -08:00
Derek McGowan
b75069ef13
Merge pull request #2791 from AliyunContainerService/support-oss-byok
Support BYOK for OSS storage driver
2019-01-16 16:37:37 -08:00
Jack Baines
bda79219b2 Add code to handle pagination of parts. Fixes max layer size of 10GB bug
Signed-off-by: Jack Baines <jack.baines@uk.ibm.com>
2019-01-14 14:54:55 +00:00
Luca Bruno
15b0204758
registry: fix binary JSON content-type
This fixes registry endpoints to return the proper `application/json`
content-type for JSON content, also updating spec examples for that.

As per IETF specification and IANA registry [0], the `application/json`
type is a binary media, so the content-type label does not need any
text-charset selector. Additionally, the media type definition
explicitly states that it has no required nor optional parameters,
which makes the current registry headers non-compliant.

[0]: https://www.iana.org/assignments/media-types/application/json

Signed-off-by: Luca Bruno <lucab@debian.org>
2019-01-14 09:04:42 +00:00
Greg Rebholz
cdb62b2b77 Registry - make minimum TLS version user configurable
Signed-off-by: J. Gregory Rebholz <gregrebholz@gmail.com>
2019-01-11 18:11:03 -05:00
David Wu
eb1a2cd911 default autoredirect to false
Signed-off-by: David Wu <david.wu@docker.com>
2019-01-04 11:05:12 -08:00
Li Yi
90bed67126 Support BYOK for OSS storage driver
Change-Id: I423ad03e63bd38aded3abfcba49079ff2fbb3b74
Signed-off-by: Li Yi <denverdino@gmail.com>
2018-12-25 08:30:40 +08:00
Derek McGowan
aa985ba889
Merge pull request #2711 from davidswu/autoredirect
add autoredirect auth config
2018-11-27 15:48:25 -08:00
Manish Tomar
1251e51ad0 better name and updated tests
- use ManifestDigests name instead of Indexes
- update tests to validate against multiple tags

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-11-01 10:31:08 -07:00
Manish Tomar
9ebf151ac2 API to retrive tag's digests
Add an interface alongside TagStore that provides API to retreive
digests of all manifests that a tag historically pointed to. It also
includes currently linked tag.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-10-29 18:19:05 -07:00
Matt Tescher
7c4d584e58 add bugsnag logrus hook
Signed-off-by: Matt Tescher <matthew.tescher@docker.com>
2018-10-25 14:52:10 -07:00
Yongxin Li
de8636b78c typo fix about overridden
Signed-off-by: Yongxin Li <yxli@alauda.io>
2018-09-27 20:27:09 +08:00
Rui Cao
569d18aef9 Fix some typos
Signed-off-by: Rui Cao <ruicao@alauda.io>
2018-09-24 09:05:44 +08:00
David Wu
2e1e6307dd add autoredirect to option
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-20 19:33:06 -07:00
David Wu
b2bd465760 fix checks
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-20 15:24:27 -07:00
Viktor Stanchev
f730f3ab77 add autoredirect auth config
It redirects the user to to the Host header's domain whenever they try to use
token auth.

Signed-off-by: David Wu <david.wu@docker.com>
2018-09-20 14:47:43 -07:00
Olivier Gambier
16128bbac4
Merge pull request #2707 from davidswu/go-1.11
remove dependencies on resumable
2018-09-20 12:47:44 -07:00
liyongxin
6133840f49 typo fix from existant to existent
Signed-off-by: liyongxin <yxli@alauda.io>
Signed-off-by: Yongxin Li <yxli@alauda.io>
2018-09-13 19:37:13 +08:00
David Wu
a927fbdb9b track digest offset in blobwriter
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-12 14:53:27 -07:00
David Wu
bd41413d57 remove closenotifier
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-11 16:14:10 -07:00
David Wu
166874ade9 fix gofmt and goimports
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-11 16:14:10 -07:00
David Wu
877d706b38 remove dependencies on resumable
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-11 08:21:27 -07:00
Derek McGowan
642075f42c
Merge pull request #2631 from whoshuu/feature/improve-gcs-driver
Improve gcs driver
2018-09-05 17:48:03 -07:00
Derek McGowan
15de837aa8
Merge pull request #2704 from dmcgowan/fix-2703
Fix registry stripping newlines from manifests
2018-09-05 17:46:56 -07:00
Huu Nguyen
7a195dd5ca Add back include_gcs build constraint
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
2018-09-05 15:48:30 -07:00
Huu Nguyen
69299d93d9 Use existing jwtConf instead of creating a scoped one
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
2018-09-05 15:48:30 -07:00
Tony Holdstock-Brown
f9187b2572 Add regulator to GCS
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
2018-09-05 15:48:30 -07:00
Andrey Kostov
b424c3d870 Better error handling for GCS credential argument addition
Signed-off-by: Andrey Kostov <kostov.andrey@gmail.com>
2018-09-05 15:48:29 -07:00
Andrey Kostov
78238ef1a0 Add credentials argument for GCS driver
Signed-off-by: Andrey Kostov <kostov.andrey@gmail.com>
2018-09-05 15:48:29 -07:00
Derek McGowan
c88728f217
Fix registry stripping newlines from manifests
Content must be preserved exactly

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-05 13:40:42 -07:00
Ryan Abrams
6b73a9ab89 Ignore missing paths during enumeration
It's possible to run into a race condition in which the enumerator lists
lots of repositories and then starts the long process of enumerating through
them. In that time if someone deletes a repo, the enumerator may error out.

Signed-off-by: Ryan Abrams <rdabrams@gmail.com>
2018-09-05 10:17:08 -07:00
David Wu
8d7e4cd388 fix goimports and gofmt
Signed-off-by: David Wu <david.wu@docker.com>
2018-09-04 17:45:39 -07:00
Olivier Gambier
90705d2fb8
Merge pull request #2362 from twistlock/populate_htpasswd
Create and populate htpasswd file if missing
2018-08-31 00:25:37 -07:00
Derek McGowan
b12bd4004a
Merge pull request #2639 from andrew-leung/manifesteventlayers
Add configurable layers in manifest events
2018-08-28 16:03:05 -07:00
Derek McGowan
059f301d54
Merge pull request #2685 from manishtomar/mani-graceful-shutdown
Graceful shutdown
2018-08-27 14:24:53 -07:00
Olivier Gambier
3354cf98e3
Merge pull request #2680 from manishtomar/mani-fix-mem-leak
fix memory leak introduced in PR #2648
2018-08-24 14:35:12 -07:00
Derek McGowan
ef859e1b21
Merge pull request #2474 from vikstrous/disable-v1-master
disable schema1 by default, add a config flag to enable it
2018-08-24 10:58:39 -07:00
Olivier
53bd46af5c
Merge pull request #2651 from mikebrow/manifest-version-test-on-put
adds validation testing for schema version values
2018-08-20 12:19:40 -07:00
Olivier
6411087274
Merge pull request #2681 from dmcgowan/update-yaml
Update yaml parser
2018-08-20 12:18:21 -07:00
David Wu
0b0d470281 use aws sdk to validate regions
Signed-off-by: David Wu <david.wu@docker.com>
2018-08-20 11:02:14 -07:00
Andrew Leung
5e4b81a578 Use references terminology instead of layers.
Signed-off-by: Andrew Leung <anwleung@gmail.com>
2018-08-20 10:01:40 -07:00
Manish Tomar
40efb602d6
Add support to gracefully shutdown the server
This is done by draining the connections for configured time after registry receives a SIGTERM signal.
This adds a `draintimeout` setting under `HTTP`. Registry doesn't drain
if draintimeout is not provided.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-08-20 10:01:26 -07:00
Derek McGowan
f0ee5720a5
Update yaml parser
Mark the top level Loglevel field as deprecated

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-14 16:42:36 -07:00
Manish Tomar
13f8189f2a fix memory leak introduced in PR #2648
context.App.repoRemover is single registry instance stored throughout
app run. It was wrapped in another remover when processing each request.
This remover happened to be remover got from previous request. This way
every remover created was stored in infinite linked list causing memory
leak. Fixing it by storing the wrapped remover inside the request context
which will get gced when request context is gced. This was introduced in
PR #2648.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-08-14 16:41:45 -07:00
Mike Brown
2fdb2ac270 adds validation testing for schema version values
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-08-14 08:53:54 -05:00
Derek McGowan
5f37adaa41
Merge pull request #2673 from dmp42/TLS
Remove ciphers that do not support perfect forward secrecy
2018-08-10 16:03:59 -07:00
Derek McGowan
88530ef7a0
Merge pull request #2671 from dmp42/goamz
Remove goamz
2018-08-10 15:54:04 -07:00
Olivier
7d9f067716
Merge pull request #2632 from whoshuu/feature/improve-s3-driver
Improve s3 driver
2018-08-10 15:01:11 -07:00
Olivier
d260b18f2f Remove ciphers that do not support perfect forward secrecy
Signed-off-by: Olivier <o+github@gambier.email>
2018-08-10 14:58:51 -07:00
dmp
9caa7a81bc Remove goamz
Signed-off-by: Olivier <o+github@gambier.email>
2018-08-10 11:11:53 -07:00
Olivier
b7446e89bf
Merge pull request #2599 from legionus/storage-inmemory-deadlock
Fix deadlock in the inmemory storage driver
2018-08-09 15:26:12 -07:00
Olivier
0cbe144826
Merge pull request #2602 from dmage/errmessage
Use e.Message field instead of e.Code.Message()
2018-08-09 15:21:32 -07:00
Olivier
4a75b72fd3
Merge pull request #2662 from dmcgowan/enable-structcheck
Enable static checks
2018-08-07 19:01:38 -07:00
Olivier
eefe9670bd
Merge pull request #2661 from dmcgowan/enable-goimports
Enable goimports check
2018-08-07 14:59:49 -07:00
Derek McGowan
003aa051b4
Merge pull request #2648 from manishtomar/tag-deleted-event
add repo and tag deletion event
2018-08-07 11:50:56 -07:00
Derek McGowan
db0a4ec1c8
Enable static checks
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 14:49:11 -07:00
Derek McGowan
795e11d5fb
Enable goimports check
Validates that goimports has been run on all files

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 14:37:44 -07:00
Derek McGowan
32e2260be2
Enable struct check
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 13:55:43 -07:00
Manish Tomar
8c05756141 lint fix
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-08-06 09:46:42 -07:00
Manish Tomar
8f6758278d take handler update
forgot to commit this earlier

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-08-02 23:13:37 -07:00
Manish Tomar
328069bb4d add support for repo deleted event also
by having another interface RepositoryRemover that is implemented by
registry instance and is injected in app context for event tracking

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
2018-08-02 23:05:05 -07:00
Mike Brown
20aecf1d7b added test for initial oci schema version
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-07-19 19:41:31 -05:00
Mike Brown
5f588fbf9b address review comment regarding panic use
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-07-19 16:07:26 -05:00
Andrew Leung
276fdce3d9 Add configurable layers in manifest events
Signed-off-by: Andrew Leung <anwleung@gmail.com>
2018-06-27 09:27:24 -07:00
Mike Brown
e8d7941ca6 address lint and gofmt issues
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-25 15:45:15 -05:00
Huu Nguyen
f5c6357c6d Update s3_test to account for SkipVerify parameter
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
2018-06-22 12:00:28 -07:00
Huu Nguyen
7655a3d91f Add option to skip certificate verification for the s3 driver
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
2018-06-22 12:00:21 -07:00
Owen W. Taylor
132abc6de5 Test storing OCI image manifests and indexes with/without a media type
OCI Image manifests and indexes are supported both with and without
an embeded MediaType (the field is reserved according to the spec).
Test storing and retrieving both types from the manifest store.

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2018-06-19 11:24:25 -04:00
Owen W. Taylor
60d9c5dfad Handle OCI manifests and image indexes without a media type
In the OCI image specification, the MediaType field is reserved
and otherwise undefined; assume that manifests without a media
in storage are OCI images or image indexes, and determine which
by looking at what fields are in the JSON. We do keep a check
that when unmarshalling an OCI image or image index, if it has
a MediaType field, it must match that media type of the upload.

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2018-06-19 11:24:25 -04:00
Mike Brown
ad7ab0853c folow commit 9c88801a12
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
7b47fb13cf update url policy support; testing for annoations in index
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
ec2aa05cdf addressing comments from stevvooe
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
6bae7ca597 refactor adding enum for storage types
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
9e3f78b8c8 addresses minor debug comments
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
b0cef05626 removes oci. namespace feature
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
426afb3a4c address get manifest issue with oci. namespace; and comment descriptions
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:55:24 -05:00
Mike Brown
c94f28805e OCI media types; annotation support; oci index
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:48:58 -05:00
Mike Brown
6fcea22b0a add an ocischema manifest handler for the registry
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:48:58 -05:00
Mike Brown
9986e8ca7c adds support for oci manifests and manifestlists
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2018-06-18 13:48:58 -05:00
Yu Wang
62797237b9 closes #2496 and #2552
Update Azure SDK with release v16.2.1
Update Azure autorest SDK with release v10.8.1

Signed-off-by: Yu Wang <yuwa@microsoft.com>
2018-06-08 16:53:02 -07:00
Oleg Bulatov
7bc438a534
Use e.Message field instead of e.Code.Message()
Signed-off-by: Oleg Bulatov <oleg@bulatov.me>
2018-06-04 17:29:46 +02:00
Gladkov Alexey
fc7e8f42d7 Fix deadlock in the inmemory storage driver
According golang documentation [1]: no goroutine should expect to be
able to acquire a read lock until the initial read lock is released.

[1] https://golang.org/pkg/sync/#RWMutex

Signed-off-by: Gladkov Alexey <agladkov@redhat.com>
2018-05-30 18:01:08 +02:00
Olivier Gambier
23bef416bd
Merge pull request #2538 from tianon/challanges
Fix minor "Challanges" typo
2018-03-21 09:58:14 -07:00
Derek McGowan
607ae5d128
Merge pull request #2501 from xiaonancc77/master
Added ignore event types into notifications
2018-03-14 10:37:02 -07:00
Michal Minář
492844e09c redis: delete right size attribute
Signed-off-by: Michal Minář <miminar@redhat.com>
2018-03-13 15:25:32 +01:00
elsanli(李楠)
fc1d3647c6 Added ignore event typs into notifications
Signed-off-by: elsanli(李楠) <elsanli@tencent.com>
2018-03-13 16:00:44 +08:00
Tianon Gravi
f2805894c8 Fix minor "Challanges" typo
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2018-03-01 11:25:05 -08:00
Stephen Day
6664ec7039
Merge pull request #2466 from tifayuki/prometheus_go_metrics
add prometheus metrics
2018-02-09 15:49:04 -08:00
tifayuki
e3c37a46e2 Add Prometheus Metrics
at the first iteration, only the following metrics are collected:

  - HTTP metrics of each API endpoint
  - cache counter for request/hit/miss
  - histogram of storage actions, including:
    GetContent, PutContent, Stat, List, Move, and Delete

Signed-off-by: tifayuki <tifayuki@gmail.com>
2018-02-09 14:27:51 -08:00
Derek McGowan
13076371a6
Merge pull request #2520 from ywk253100/ignore_path_not_found_error
Ignore path not found error when look up tags
2018-02-09 11:19:52 -08:00
Wenkai Yin
005c6e0236 ignore path not found error when look up tags
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2018-02-08 17:32:25 +08:00
Stephen Day
d707ea2428
Merge pull request #2469 from felixbuenemann/letsencrypt-host-whitelist-support
Let's Encrypt Host Whitelist Support
2018-02-06 16:11:20 -08:00
Stephen J Day
1ba5b3b553
registry/storage: ignore missing tag on delete
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-02-06 15:25:51 -08:00
Stephen Day
fea8bd5114
Merge pull request #2471 from GarageDeveloper/master
fixes #2249: sanitize tenant and tenantid
2018-02-02 16:29:34 -08:00
Felix Bünemann
4ecb17cc4c registry: support whitelisting letsencrypt hosts
This adds a configuration setting `HTTP.TLS.LetsEncrypt.Hosts` which can
be set to a list of hosts that the registry will whitelist for retrieving
certificates from Let's Encrypt. HTTPS connections with SNI hostnames
that are not whitelisted will be closed with an "unknown host" error.
It is required to avoid lots of unsuccessful registrations attempts that
are triggered by malicious clients connecting with bogus SNI hostnames.

NOTE: Due to a bug in the deprecated vendored rsc.io/letsencrypt library
clearing the host list requires deleting or editing of the cachefile to
reset the hosts list to null.

Signed-off-by: Felix Buenemann <felix.buenemann@gmail.com>
2018-02-01 21:16:58 +01:00
Derek McGowan
5cb406d511
Merge pull request #2455 from sargun/make-walk-faster
s3: improve walk performance
2018-01-19 13:00:03 -08:00