Commit graph

1094 commits

Author SHA1 Message Date
Milos Gajdos
8eab5d1bd6
Update s3 ListObjects to V2 API
Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2022-04-09 12:16:46 +01:00
Simone Locci
80952c9e2b
Rename s3accelerate parameter to accelerate
Signed-off-by: Simone Locci <simonelocci88@gmail.com>
2022-04-04 19:35:21 +02:00
Simone Locci
ea27621d4a
Fix review
Signed-off-by: Simone Locci <simonelocci88@gmail.com>
2022-04-04 19:35:09 +02:00
Kirat Singh
51c0c8148a
Add new parameter s3accelerate to S3 storage driver.
If s3accelerate is set to true then we turn on S3 Transfer
Acceleration via the AWS SDK.  It defaults to false since this is an
opt-in feature on the S3 bucket.

Signed-off-by: Kirat Singh <kirat.singh@wsq.io>
Signed-off-by: Simone Locci <simonelocci88@gmail.com>
2022-04-04 19:34:57 +02:00
Milos Gajdos
d2c9f72c6b
Merge pull request #3615 from zhsj/inmem-panic
Fix panic in inmemory driver
2022-03-27 16:20:31 +01:00
Shengjing Zhu
1a75c71907 Fix panic in inmemory driver
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-03-27 19:38:07 +08:00
James Hewitt
25bd1f704d
Incorrect variable in test output
Looks like a copy-paste bug from the same test for the image manifest.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2022-03-27 11:05:47 +01:00
João Pereira
514cbd71be
Merge pull request #3519 from jtherin/mpu-paginate
fix: paginate through s3 multipart uploads
2022-03-11 16:06:46 +00:00
Milos Gajdos
676691ce6d
Fix: Avoid a false type assertion in the inmemory driver
This issue was discovered by the following fuzzer:
https://github.com/cncf/cncf-fuzzing/blob/main/projects/distribution/inmemory_fuzzer.go#L24

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2022-01-28 11:00:37 +00:00
baojiangnan
4363fb1ef4 disable insecure cipher suites
This commit removes the following cipher suites that are known to be insecure:

TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
TLS_ECDHE_RSA_WITH_RC4_128_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256

And this commit deletes the tlsVersions of tls1.0 and tls1.1. The tls1.2 is the minimal supported tls version for creating a safer tls configuration.

Signed-off-by: david.bao <baojn1998@163.com>
2022-01-25 17:18:44 +08:00
João Pereira
5f1974ab8b
Merge pull request #3567 from justadogistaken/fix/image-cache-incomplete
fix image cache incompletely
2022-01-19 13:57:27 +00:00
baojiangnan
706f2170bd fix image cache incompletely
Signed-off-by: baojiangnan <baojn1998@163.com>
2022-01-14 23:58:44 +08:00
libo.huang
117757a5cb feat: add option to disable combining the pending part
Signed-off-by: Libo Huang <huanglibo2010@gmail.com>
2022-01-07 18:20:31 +08:00
Adam Kaplan
e2caaf9cba Add dualstack option to S3 storage driver
Allow the storage driver to optionally use AWS SDK's dualstack mode.
This allows the registry to communicate with S3 in IPv6 environments.

Signed-off-by: Adam Kaplan <adam.kaplan@redhat.com>
2022-01-04 17:19:05 -05:00
Milos Gajdos
020bcce59d
Merge pull request #3458 from AdamKorcz/fuzz1
Fuzzing: Add 3 fuzzers
2021-12-20 09:34:15 +00:00
AdamKorcz
d0ca0c3303 Fuzzing: Add 3 fuzzers
Signed-off-by: AdamKorcz <adam@adalogics.com>
2021-11-29 20:59:28 +00:00
Wang Yan
f637481c67 fix go check issues
1, Fix GoSec G404: Use of weak random number generator (math/rand instead of crypto/rand)
2, Fix Static check: ST1019: package "github.com/sirupsen/logrus" is being imported more than once

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-11-15 17:31:33 +08:00
Milos Gajdos
6248a88d03
Merge pull request #3515 from wyckster/patch-1
Fixed typo in error message
2021-11-01 14:28:38 +00:00
Chad Faragher
f619db7336 Fixed typo in error message
The wording of the error message had a typo (missing the word "not") that gave it the opposite meaning from the intended meaning.

Signed-off-by: Chad Faragher <wyckster@hotmail.com>
2021-10-28 14:50:04 -04:00
Wang Yan
3f4c558dac bump up golang v1.17
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-10-27 15:51:30 +08:00
Jeremy THERIN
7736319f2e fix: paginate through s3 multipart uploads
Signed-off-by: Jeremy THERIN <jtherin@scaleway.com>
2021-10-26 17:55:03 +02:00
Milos Gajdos
1563384b69
Merge pull request #3480 from CollinShoop/optimize-s3-walk
Optimize storagedriver/s3 Walk (up to ~500x) + small bugfix
2021-09-26 10:24:39 +01:00
Oleg Bulatov
f5709b285a updatefrequency should not be saved into duration
When updatefrequency is set and is a string, its value should be saved
into updateFrequency, and it shouldn't override duration.

Signed-off-by: Oleg Bulatov <oleg@bulatov.me>
2021-09-21 11:12:34 +02:00
Collin Shoop
cf81f67a16 storagedriver/s3: Optimized Walk implementation + bugfix
Optimized S3 Walk impl by no longer listing files recursively. Overall gives a huge performance increase both in terms of runtime and S3 calls (up to ~500x).

Fixed a bug in WalkFallback where ErrSkipDir for was not handled as documented for non-directory.

Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-16 16:07:25 -04:00
Collin Shoop
9e873f31ec storagedriver/s3: Adding back missing import.
Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-12 11:56:13 -04:00
Collin Shoop
e625bc7160 storagedriver/s3: Removed temporary S3 test
(cherry picked from commit ce80e98cea1d15aa2a2ab931c8b9a1161fc6e218)
Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-12 11:56:13 -04:00
Collin Shoop
dc5b77101d storagedriver/s3: Cleaning up tests
(cherry picked from commit 483ba26165ca66bcf18a1eaadf41ebe4d3bd5f85)
Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-12 11:56:13 -04:00
Collin Shoop
6da7217b99 storagedriver/s3: Optimize s3 Delete test cleanup.
(cherry picked from commit e4af4dc3a6da6da724e7cff18cf5b6da6ef2a3fd)
Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-12 11:56:13 -04:00
Collin Shoop
03f9eb3a18 storagedriver/s3: Fixed a Delete noop edgecase
Delete was not working when the subpath immediately followed the given path started with an ascii lower than "/" such as dash "-" and underscore "_" and requests no files to be deleted.

(cherry picked from commit 5d8fa0ce94b68cce70237805db92cdd8d40de282)
Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-12 11:56:13 -04:00
Collin Shoop
05a258e711 storagedriver/s3: Added Delete tests to s3_test
(cherry picked from commit 1e3b6b67a8e6d7f01307518370f0731212935d05)
Signed-off-by: Collin Shoop <cshoop@digitalocean.com>
2021-08-12 10:57:24 -04:00
Brandon Mitchell
9c7967a32d Update PUT and PATCH APIs
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2021-06-29 14:16:33 -04:00
João Pereira
6d75bd043a
Merge pull request #3425 from lostsquirrel/patch-1
Fix the /v2/_catalog n parameter description
2021-06-17 11:10:14 +01:00
lostsquirrel
6d9a3aba04 fix the /v2/_catalog n parameter description
If `n` is not present only 100 entries returned

Signed-off-by: lisong <lisong@cdsunrise.net>
2021-06-17 17:33:55 +08:00
Milos Gajdos
af8ac80933
Merge pull request #3427 from joaodrp/tag-delete
Add tag delete API
2021-06-14 17:52:26 +01:00
João Pereira
033683d629
apply feedback
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-06-06 19:54:55 +01:00
João Pereira
81f081f91b
Group case values
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-05-28 10:33:40 +01:00
João Pereira
6ae6df7d75
Add tag delete API
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-05-27 23:27:02 +01:00
João Pereira
8ef268df25
Add tests for tags list pagination
Signed-off-by: João Pereira <484633+joaodrp@users.noreply.github.com>
2021-05-22 15:31:18 +01:00
João Pereira
d80a63f1ea
Merge pull request #3143 from eyJhb/pagination
OCI: Add pagination on `/v2/<name>/tags/list`
2021-05-22 15:05:18 +01:00
Wang Yan
f65a33d3c8
Merge pull request #3227 from wy65701436/fix-conformance-416
OCI: add content range handling in patch blob
2021-05-11 23:51:17 +08:00
wang yan
d7a2b14489 add content range handling in patch blob
Fixes #3141

1, return 416 for Out-of-order blob upload
2, return 400 for content length and content size mismatch

Signed-off-by: wang yan <wangyan@vmware.com>
2021-05-11 23:35:15 +08:00
Hayley Swimelar
9329f6a62b
Merge pull request #3395 from fuweid/verify-layer-config-descriptor
OCI: verify digest and check blob presence when put manifest
2021-05-07 10:38:45 -07:00
eyjhb
4da2712b52
added pagination to v2/<name>/tags/list endpoint
Signed-off-by: eyjhb <eyjhbb@gmail.com>
2021-04-23 18:06:04 +02:00
eyjhbb@gmail.com
febc8733d2
added error codes for pagination
Signed-off-by: eyjhb <eyjhbb@gmail.com>
2021-04-23 18:06:03 +02:00
João Pereira
a27b0c4952
Merge pull request #3161 from bloodorangeio/redis-tls
Add configuration option for Redis TLS
2021-04-23 16:52:54 +01:00
Wei Fu
9e618c90c3 registry: verify digest and check blob presence when put manifest
According to OCI image spec, the descriptor's digest field is required.
For the normal config/layer blobs, the valivation should check the
presence of the blob when put manifest.

REF: https://github.com/opencontainers/image-spec/blob/v1.0.1/descriptor.md

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-04-16 16:11:52 +08:00
sangluo
334a7e7ff6 close the io.ReadCloser from storage drive
Signed-off-by: sangluo <sangluo@pinduoduo.com>
2021-03-24 17:58:35 +08:00
Wang Yan
077b38b95e
Merge pull request #3364 from Agilicus/escape-json
fix: disable DisableHTMLEscape on logrus json logging
2021-03-24 11:16:55 +08:00
Derek McGowan
a01c71e247
Merge pull request #2815 from bainsy88/issue_2814
Add code to handle pagination of parts. Fixes max layer size of 10GB bug
2021-03-16 09:12:03 -07:00
Josh Dolitsky
32ccbf193d
Add configuration option for Redis TLS
Signed-off-by: Josh Dolitsky <josh@dolit.ski>
2021-03-01 18:55:56 -05:00