alexvanin/distribution
1
0
Fork 0
forked from TrueCloudLab/distribution
Code Pull requests Activity
2956 commits 2 branches 59 tags 37 MiB
Commit graph

1061 commits

Author SHA1 Message Date
Manish Tomar
ce101280fe
fix redis caching issue 
* fix redis caching issue

earlier redis cache was updated when there was any error including any
temporary connectivity issue. This would trigger set calls which would
further increase load and possibly connectivity errors from redis
leaving the system with continuous errors and high latency. Now the
cache is updated only when it is genuine cache miss. Other errors do not
trigger a cache update.

* add back tracker Hit() and Miss() calls

*squashed commits*
(cherry picked from commit 6f3e1c10260ef59ba4e9c42e939329fad9fdd8c3)
(cherry picked from commit 6738ff3320cf82cc2df919a95a1bde2f7789a501)

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2020-03-04 17:51:37 -08:00
Derek McGowan
4c7c63b557
Add cache unit test 
Test base functionality of the cache statter

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2020-03-04 17:51:37 -08:00
Elliot Pahl
800cb95821
Use go-events package 
TBD: Queue not converted yet

Signed-off-by: Elliot Pahl <elliot.pahl@gmail.com>
 2020-03-04 12:49:32 -08:00
Alfonso Acosta
15f7bd29a5 Remove {get,head}WithContext() 
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
 2020-02-25 13:36:43 +01:00
Alfonso Acosta
282351e954 Use http.NewRequestWithContext() 
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
 2020-02-25 13:36:31 +01:00
Alfonso Acosta
58331abf58 Honor contexts passed to registry client methods 
Signed-off-by: Alfonso Acosta <fons@syntacticsugar.consulting>
 2020-02-25 12:41:15 +01:00
Derek McGowan
6b972e50fe
Merge pull request #2272 from naveedjamil/fips 
Increase Unit Test Code Coverage
 2020-02-22 17:40:41 -08:00
Derek McGowan
e65b3f1316
Fix CI for test updates 
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2020-02-22 17:27:55 -08:00
Naveed Jamil
efdba4f210
Increase Unit Test Code Coverage 
Unit test coverge was increased to cover the usages of crypto. This helps to ensure that everything is working fine with fips mode enabled.
Also updated sha1 to sha256 in registry/storage/driver/testsuites/testsuites.go because sha1 is not supported in fips mode.

Signed-off-by: Naveed Jamil <naveed.jamil@tenpearl.com>
 2020-02-22 17:09:00 -08:00
Derek McGowan
bfa6b923e7
Merge pull request #3020 from kd7lxl/updatefrequency 
Fix typo cloudfront updatefrenquency
 2020-02-22 16:54:03 -08:00
sayboras
66809646d9 Migrate to golangci-lint 
Signed-off-by: Tam Mach <sayboras@yahoo.com>
 2020-02-14 08:11:16 +11:00
Derek McGowan
a837179414
Merge pull request #3072 from fermayo/fix-TestRegistryAsCacheMutationAPIs 
Fix TestRegistryAsCacheMutationAPIs
 2019-12-15 20:48:56 -08:00
Fernando Mayo Fernandez
6ca7b9e9fa
Fix TestRegistryAsCacheMutationAPIs 
Use a synthetic upstream registry when creating the testing mirror configuration
to avoid the test fail when trying to reach http://example.com

Signed-off-by: Fernando Mayo Fernandez <fernando@undefinedlabs.com>
 2019-12-15 13:51:25 +01:00
Tom Hayward
0f5e2753a6 Fix typo cloudfront updatefrenquency 
Signed-off-by: Tom Hayward <thayward@infoblox.com>
 2019-12-13 12:11:42 -08:00
Derek McGowan
bdf3438b52
Merge pull request #2985 from novas0x2a/default-transport 
make it possible to wrap the client transport in another one
 2019-12-13 11:36:12 -08:00
Guillaume Rose
c9c3324300 Add unit tests for BlobEnumerator 
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
 2019-12-06 09:38:13 +01:00
Manish Tomar
5538da4923 fixes to make layersPathSpec work 
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
 2019-12-05 20:48:33 +01:00
Manish Tomar
fa7d949408 allow Repository.BlobStore to enumerate over blobs 
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
 2019-12-05 20:48:33 +01:00
Manish Tomar
cf77113795 add pathspec for repo _layers directory 
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
 2019-12-05 20:48:33 +01:00
Ryan Abrams
ae2e973db9
Merge pull request #2748 from manishtomar/tag-digests 
API to retrive tag's digests
 2019-10-08 12:14:56 -07:00
Ryan Abrams
f656e60de5
Merge pull request #2984 from bouk/rempo 
registry: Fix typo in RepositoryRemover warning
 2019-09-05 08:23:18 -07:00
Ryan Abrams
740d4d1211
Merge pull request #2918 from dmathieu/test-blob-writer-write 
Test httpBlobUpload.Write method
 2019-09-05 08:16:36 -07:00
Bouke van der Bijl
1c481d34d9 registry: Fix typo in RepositoryRemover warning 
Signed-off-by: Bouke van der Bijl <me@bou.ke>
 2019-09-02 16:07:34 +00:00
Mike Lundy
c486db2d71
make it possible to wrap the client transport in another one 
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
 2019-08-22 17:37:47 -04:00
Ryan Abrams
1fb7fffdb2
Merge pull request #2950 from terinjokes/patches/swift-segment-hash 
swift: correct segment path generation
 2019-07-11 15:35:31 -07:00
Damien Mathieu
dd3bdee21c implement Repository Blobs upload resuming 
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-26 09:46:49 +02:00
Ryan Abrams
90dfea7952
Merge pull request #2921 from dmathieu/repository-serve-blob 
Implement Repository ServeBlob
 2019-06-25 19:07:38 -07:00
Ryan Abrams
6c72ec2e85
Merge pull request #2927 from dmathieu/blob-create-uuid 
Handle Blob Create when the underlying registry doesn't provide 'Docker-Upload-UUID'
 2019-06-25 19:06:46 -07:00
Damien Mathieu
898b1f2a53 test httpBlobUpload.Write method 
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-25 09:32:58 +02:00
Damien Mathieu
c5d5f938e3 fast-stop ServeBlob if we're doing a HEAD request 
A registry pointing to ECR is having issues if we try loading the blob

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-25 09:30:22 +02:00
Damien Mathieu
3800c47fd2 Implement Repository ServeBlob 
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-25 09:30:22 +02:00
Damien Mathieu
a45e5cb13f handle create blob if the uuid couldn't be retrieved from headers or URL 
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-25 09:29:38 +02:00
Damien Mathieu
8b31a894bd deduce blob UUID from location if it wasn't provided in the headers 
Some registries (ECR) don't provide a `Docker-Upload-UUID` when creating
a blob. So we can't rely on that header. Fallback to reading it from the
URL.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-25 09:29:38 +02:00
Damien Mathieu
94097512db extract blob upload resume into its own method 
I've found this logic being in a single method to be quite hard to get.
I believe extracting it makes it easier to read, as we can then more
easily see what the main method does and possibly ignore the intricacies
of `ResumeBlobUpload`.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-06-25 09:25:56 +02:00
Terin Stock
b23dd1ef37 swift: correct segment path generation 
When uploading segments to Swift, the registry generates a random file,
by taking the hash of the container path and 32-bytes of random data.
The registry attempts to shard across multiple directory paths, by
taking the first three hex characters as leader.

The implementation in registry, unfortunately, takes the hash of
nothing, and appends it to the path and random data. This results in all
segments being created in one directory.

Fixes: #2407
Fixes: #2311
Signed-off-by: Terin Stock <terinjokes@gmail.com>
 2019-06-22 23:44:43 -07:00
Ryan Abrams
84f47e7bb3
Merge pull request #2900 from sevki/no-token-err-fix 
fix no error returned in fetchTokenWithOAuth
 2019-06-19 19:16:18 -07:00
Ryan Abrams
6c329e56a2
Merge pull request #2920 from dmathieu/blob_writer_offset 
Append the written bytes to the blob writer's size
 2019-06-19 19:14:49 -07:00
Ryan Abrams
0c394fdd84
Merge pull request #2861 from yuwaMSFT2/master 
Fixes #2835 Process Accept header MIME types in case-insensitive way
 2019-06-19 19:09:29 -07:00
Ryan Abrams
d80a17d8e0
Merge pull request #2879 from tbe/fix-s3-ceph 
Fix s3 driver for supporting ceph radosgw
 2019-06-19 19:09:07 -07:00
Ryan Abrams
55287010ce
Merge pull request #2894 from jabrown85/fix-offset-typos 
Fix typo: offest -> offset
 2019-06-19 19:08:22 -07:00
Tariq Ibrahim
8f9c8094fb
replace rsc.io/letsencrypt in favour of golang.org/x/crypto 
Signed-off-by: Tariq Ibrahim <tariq181290@gmail.com>
 2019-06-04 12:04:18 -07:00
Damien Mathieu
0e2d080a8a append the written bytes to the blob writer's size 
Any byte written should append to the size. Otherwise, the full Size is
always zero.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
 2019-05-09 14:17:58 +02:00
Ryan Abrams
3226863cbc
Merge pull request #2849 from Shawnpku/master 
support Alibaba Cloud CDN storage middleware
 2019-04-16 18:43:06 -07:00
Sevki Hasirci
5afbf32400 fix no error returned in fetchTokenWithOAuth 
fetchTokenWithBasicAuth checks if a token is in the token response
but fetchTokenWithOAuth does not

these changes implements the same behaviour for the latter
returning a `ErrNoToken` if a token is not found in the resposne

Signed-off-by: Sevki Hasirci <sevki@cloudflare.com>
 2019-04-14 11:05:59 +01:00
Jesse Brown
74f429a5ad Fix typo: offest -> offset 
Signed-off-by: Jesse Brown <jabrown85@gmail.com>
 2019-04-05 14:20:20 -05:00
Shawn Chen
fd77cf43a6 change package name & format document 
Signed-off-by: Shawn Chen <chen8132@gmail.com>
 2019-03-18 11:35:46 +08:00
Thomas Berger
c18c6c33b2 S3 Driver: added comment for missing KeyCount workaround 
Signed-off-by: Thomas Berger <loki@lokis-chaos.de>
 2019-03-15 21:05:21 +01:00
Eohyung Lee
f877726503 Fix s3 driver for supporting ceph radosgw 
Radosgw does not support S3 `GET Bucket` API v2 API but v1.
This API has backward compatibility, so most of this API is working
correctly but we can not get `KeyCount` in v1 API and which is only
for v2 API.

Signed-off-by: Eohyung Lee <liquidnuker@gmail.com>
 2019-03-08 12:45:20 +01:00
Shawn Chen
3390f32aec fix Context issue 
Signed-off-by: Shawn Chen <chen8132@gmail.com>
 2019-03-04 17:48:32 +08:00
Shawnpku
6e10631d9c fix default cdn auth duration 
Signed-off-by: Shawnpku <chen8132@gmail.com>
 2019-03-04 14:53:48 +08:00
Vishesh Jindal
f9a0506191
Bugfix: Make ipfilteredby not required 
Signed-off-by: Vishesh Jindal <vishesh92@gmail.com>
 2019-03-02 08:58:52 +05:30
Ryan Abrams
d3ddc3572c
Merge pull request #2854 from manishtomar/log-authed-name 
Log authorized username
 2019-03-01 14:45:43 -08:00
Ryan Abrams
c192a281f8
Merge pull request #2813 from lucab/ups/spec-json-binary 
registry: fix binary JSON content-type
 2019-03-01 14:26:54 -08:00
Yu Wang
a683c7c235 Fixes #2835 Process Accept header MIME types in case-insensitive way 
Use mime.ParseMediaType to parse the media types in Accept header in manifest request. Ignore the failed ones.

Signed-off-by: Yu Wang <yuwa@microsoft.com>
 2019-02-21 15:11:41 -08:00
Shawnpku
bbc9885aa2 fix func name 
Signed-off-by: Shawnpku <chen8132@gmail.com>
 2019-02-20 15:54:21 +08:00
Manish Tomar
ec6566c02b Log authorized username 
This is useful to know which user pulled/pushed which repo.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
 2019-02-13 08:49:37 -08:00
Shawnpku
3aa2a282f7 support alicdn middleware 
Signed-off-by: Shawnpku <chen8132@gmail.com>
 2019-02-11 15:11:26 +08:00
Manish Tomar
48818fdea7 Remove err nil check 
since type checking nil will not panic and return appropriately

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
 2019-02-04 16:42:44 -08:00
Manish Tomar
da8db4666b Fix gometalint errors 
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
 2019-02-04 16:01:04 -08:00
Derek McGowan
b75069ef13
Merge pull request #2791 from AliyunContainerService/support-oss-byok 
Support BYOK for OSS storage driver
 2019-01-16 16:37:37 -08:00
Jack Baines
bda79219b2 Add code to handle pagination of parts. Fixes max layer size of 10GB bug 
Signed-off-by: Jack Baines <jack.baines@uk.ibm.com>
 2019-01-14 14:54:55 +00:00
Luca Bruno
15b0204758
registry: fix binary JSON content-type 
This fixes registry endpoints to return the proper `application/json`
content-type for JSON content, also updating spec examples for that.

As per IETF specification and IANA registry [0], the `application/json`
type is a binary media, so the content-type label does not need any
text-charset selector. Additionally, the media type definition
explicitly states that it has no required nor optional parameters,
which makes the current registry headers non-compliant.

[0]: https://www.iana.org/assignments/media-types/application/json

Signed-off-by: Luca Bruno <lucab@debian.org>
 2019-01-14 09:04:42 +00:00
Greg Rebholz
cdb62b2b77 Registry - make minimum TLS version user configurable 
Signed-off-by: J. Gregory Rebholz <gregrebholz@gmail.com>
 2019-01-11 18:11:03 -05:00
David Wu
eb1a2cd911 default autoredirect to false 
Signed-off-by: David Wu <david.wu@docker.com>
 2019-01-04 11:05:12 -08:00
Li Yi
90bed67126 Support BYOK for OSS storage driver 
Change-Id: I423ad03e63bd38aded3abfcba49079ff2fbb3b74
Signed-off-by: Li Yi <denverdino@gmail.com>
 2018-12-25 08:30:40 +08:00
Derek McGowan
aa985ba889
Merge pull request #2711 from davidswu/autoredirect 
add autoredirect auth config
 2018-11-27 15:48:25 -08:00
Manish Tomar
1251e51ad0 better name and updated tests 
- use ManifestDigests name instead of Indexes
- update tests to validate against multiple tags

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
 2018-11-01 10:31:08 -07:00
Manish Tomar
9ebf151ac2 API to retrive tag's digests 
Add an interface alongside TagStore that provides API to retreive
digests of all manifests that a tag historically pointed to. It also
includes currently linked tag.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
 2018-10-29 18:19:05 -07:00
Matt Tescher
7c4d584e58 add bugsnag logrus hook 
Signed-off-by: Matt Tescher <matthew.tescher@docker.com>
 2018-10-25 14:52:10 -07:00
Yongxin Li
de8636b78c typo fix about overridden 
Signed-off-by: Yongxin Li <yxli@alauda.io>
 2018-09-27 20:27:09 +08:00
Rui Cao
569d18aef9 Fix some typos 
Signed-off-by: Rui Cao <ruicao@alauda.io>
 2018-09-24 09:05:44 +08:00
David Wu
2e1e6307dd add autoredirect to option 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-20 19:33:06 -07:00
David Wu
b2bd465760 fix checks 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-20 15:24:27 -07:00
Viktor Stanchev
f730f3ab77 add autoredirect auth config 
It redirects the user to to the Host header's domain whenever they try to use
token auth.

Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-20 14:47:43 -07:00
Olivier Gambier
16128bbac4
Merge pull request #2707 from davidswu/go-1.11 
remove dependencies on resumable
 2018-09-20 12:47:44 -07:00
liyongxin
6133840f49 typo fix from existant to existent 
Signed-off-by: liyongxin <yxli@alauda.io>
Signed-off-by: Yongxin Li <yxli@alauda.io>
 2018-09-13 19:37:13 +08:00
David Wu
a927fbdb9b track digest offset in blobwriter 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-12 14:53:27 -07:00
David Wu
bd41413d57 remove closenotifier 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-11 16:14:10 -07:00
David Wu
166874ade9 fix gofmt and goimports 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-11 16:14:10 -07:00
David Wu
877d706b38 remove dependencies on resumable 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-11 08:21:27 -07:00
Derek McGowan
642075f42c
Merge pull request #2631 from whoshuu/feature/improve-gcs-driver 
Improve gcs driver
 2018-09-05 17:48:03 -07:00
Derek McGowan
15de837aa8
Merge pull request #2704 from dmcgowan/fix-2703 
Fix registry stripping newlines from manifests
 2018-09-05 17:46:56 -07:00
Huu Nguyen
7a195dd5ca Add back include_gcs build constraint 
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
 2018-09-05 15:48:30 -07:00
Huu Nguyen
69299d93d9 Use existing jwtConf instead of creating a scoped one 
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
 2018-09-05 15:48:30 -07:00
Tony Holdstock-Brown
f9187b2572 Add regulator to GCS 
Signed-off-by: Huu Nguyen <whoshuu@gmail.com>
 2018-09-05 15:48:30 -07:00
Andrey Kostov
b424c3d870 Better error handling for GCS credential argument addition 
Signed-off-by: Andrey Kostov <kostov.andrey@gmail.com>
 2018-09-05 15:48:29 -07:00
Andrey Kostov
78238ef1a0 Add credentials argument for GCS driver 
Signed-off-by: Andrey Kostov <kostov.andrey@gmail.com>
 2018-09-05 15:48:29 -07:00
Derek McGowan
c88728f217
Fix registry stripping newlines from manifests 
Content must be preserved exactly

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2018-09-05 13:40:42 -07:00
Ryan Abrams
6b73a9ab89 Ignore missing paths during enumeration 
It's possible to run into a race condition in which the enumerator lists
lots of repositories and then starts the long process of enumerating through
them. In that time if someone deletes a repo, the enumerator may error out.

Signed-off-by: Ryan Abrams <rdabrams@gmail.com>
 2018-09-05 10:17:08 -07:00
David Wu
8d7e4cd388 fix goimports and gofmt 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-09-04 17:45:39 -07:00
Olivier Gambier
90705d2fb8
Merge pull request #2362 from twistlock/populate_htpasswd 
Create and populate htpasswd file if missing
 2018-08-31 00:25:37 -07:00
Derek McGowan
b12bd4004a
Merge pull request #2639 from andrew-leung/manifesteventlayers 
Add configurable layers in manifest events
 2018-08-28 16:03:05 -07:00
Derek McGowan
059f301d54
Merge pull request #2685 from manishtomar/mani-graceful-shutdown 
Graceful shutdown
 2018-08-27 14:24:53 -07:00
Olivier Gambier
3354cf98e3
Merge pull request #2680 from manishtomar/mani-fix-mem-leak 
fix memory leak introduced in PR #2648
 2018-08-24 14:35:12 -07:00
Derek McGowan
ef859e1b21
Merge pull request #2474 from vikstrous/disable-v1-master 
disable schema1 by default, add a config flag to enable it
 2018-08-24 10:58:39 -07:00
Olivier
53bd46af5c
Merge pull request #2651 from mikebrow/manifest-version-test-on-put 
adds validation testing for schema version values
 2018-08-20 12:19:40 -07:00
Olivier
6411087274
Merge pull request #2681 from dmcgowan/update-yaml 
Update yaml parser
 2018-08-20 12:18:21 -07:00
David Wu
0b0d470281 use aws sdk to validate regions 
Signed-off-by: David Wu <david.wu@docker.com>
 2018-08-20 11:02:14 -07:00
Andrew Leung
5e4b81a578 Use references terminology instead of layers. 
Signed-off-by: Andrew Leung <anwleung@gmail.com>
 2018-08-20 10:01:40 -07:00
Manish Tomar
40efb602d6
Add support to gracefully shutdown the server 
This is done by draining the connections for configured time after registry receives a SIGTERM signal.
This adds a `draintimeout` setting under `HTTP`. Registry doesn't drain
if draintimeout is not provided.

Signed-off-by: Manish Tomar <manish.tomar@docker.com>
 2018-08-20 10:01:26 -07:00