Commit graph

2855 commits

Author SHA1 Message Date
wang yan
bf56f348be Update oci library
1, update go-digest to v1.0.0-rc1
2, update image-spec to v1.0.1

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-10 23:19:29 +08:00
Derek McGowan
581be91482
Merge pull request #3113 from dmcgowan/upstream-redis-fixes
Redis cache fixes and metrics
2020-03-09 13:36:53 -07:00
Derek McGowan
be29c05a1e
Remove deprecated cache metrics code
The metrics tracker in cached blob statter was replaced with prometheus
metrics and no longer needed.
Remove unused log wrapping.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-09 13:11:54 -07:00
Derek McGowan
495a4af7cf
Fix goimports
Separate fix for cherry-picked code

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Grant Watters
74d442a058
Consider redis.ErrNil as distribution.ErrBlobUnknown for Stat HGET
* Update redis.go

If the dgst key does not exist in the cache when calling HGET, `redis.String` will return an `ErrNil` which we need to translate into `distribution.ErrBlobUnknown` so that the error being returned can be properly handled. This will ensure that `SetDescriptor` is properly called from `cachedBlobStatter::Stat` for `repositoryScopedRedisBlobDescriptorService` which will update the redis cache and be considered as a Miss rather than an Error.

cc @manishtomar

* Update suite.go

Add unit test to ensure missing blobs for scoped repo properly return ErrBlobUnknown when HGET returns redis.ErrNil.

(cherry picked from commit dca6b9526a1d30dd218a9f321c4f84ecc4b5e62e)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Manish Tomar
795892662b
redis metrics
* redis metrics

it is working but metrics are not very useful since default buckets
start from 5ms and almost all of them are in that range.

* remove extra comment

(cherry picked from commit ba1a1d74e7eb047dd1056548ccf0695e8846782c)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Manish Tomar
ce101280fe
fix redis caching issue
* fix redis caching issue

earlier redis cache was updated when there was any error including any
temporary connectivity issue. This would trigger set calls which would
further increase load and possibly connectivity errors from redis
leaving the system with continuous errors and high latency. Now the
cache is updated only when it is genuine cache miss. Other errors do not
trigger a cache update.

* add back tracker Hit() and Miss() calls

*squashed commits*
(cherry picked from commit 6f3e1c10260ef59ba4e9c42e939329fad9fdd8c3)
(cherry picked from commit 6738ff3320cf82cc2df919a95a1bde2f7789a501)

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Derek McGowan
4c7c63b557
Add cache unit test
Test base functionality of the cache statter

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 17:51:37 -08:00
Derek McGowan
492a10376c
Merge pull request #3111 from dmcgowan/update-report-issues
Update reporting issues guidelines
2020-03-04 14:39:19 -08:00
Derek McGowan
c79aa81772
Merge pull request #2550 from halcyonCorsair/halcyonCorsair/use-go-events-1
Use go-events package
2020-03-04 14:38:15 -08:00
Elliot Pahl
800cb95821
Use go-events package
TBD: Queue not converted yet

Signed-off-by: Elliot Pahl <elliot.pahl@gmail.com>
2020-03-04 12:49:32 -08:00
Derek McGowan
2d1126ecc1
Merge pull request #3080 from thaJeztah/bump_go_metrics
vendor: update docker/go-metrics v0.0.1
2020-03-04 10:21:48 -08:00
Derek McGowan
3b4d1e3ddf
Merge pull request #3110 from dmcgowan/update-maintainers
Update governance and maintainers
2020-03-04 10:19:46 -08:00
Derek McGowan
1e25ecefe4
Update governance and maintainers
Update format of governance and bring in language from containerd.
Update maintainers to reflect active maintainers.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-04 10:04:38 -08:00
Derek McGowan
db3c418ada
Update reporting issues guidelines
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-03 16:56:46 -08:00
Sebastiaan van Stijn
98dcc5195e
vendor: update docker/go-metrics v0.0.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-02 20:14:15 +01:00
Derek McGowan
6b972e50fe
Merge pull request #2272 from naveedjamil/fips
Increase Unit Test Code Coverage
2020-02-22 17:40:41 -08:00
Derek McGowan
e65b3f1316
Fix CI for test updates
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-02-22 17:27:55 -08:00
Naveed Jamil
efdba4f210
Increase Unit Test Code Coverage
Unit test coverge was increased to cover the usages of crypto. This helps to ensure that everything is working fine with fips mode enabled.
Also updated sha1 to sha256 in registry/storage/driver/testsuites/testsuites.go because sha1 is not supported in fips mode.

Signed-off-by: Naveed Jamil <naveed.jamil@tenpearl.com>
2020-02-22 17:09:00 -08:00
Derek McGowan
e9a023180f
Merge pull request #3071 from dmcgowan/update-readme-docs
Update readme and contributing docs
2020-02-22 16:58:53 -08:00
Derek McGowan
f1cb8a56c8
Merge pull request #3105 from thaJeztah/bump_golang_1.13.8
Update Golang 1.13.8
2020-02-22 16:54:49 -08:00
Derek McGowan
bfa6b923e7
Merge pull request #3020 from kd7lxl/updatefrequency
Fix typo cloudfront updatefrenquency
2020-02-22 16:54:03 -08:00
Derek McGowan
f593b8d975
Merge pull request #2446 from legionus/docker-configuration-ptr
Fix the pointer initialization
2020-02-22 16:42:17 -08:00
Sebastiaan van Stijn
1f77c9a57f
Update Golang 1.13.8
full diff: https://github.com/golang/go/compare/go1.13.7...go1.13.8

go1.13.8 (released 2020/02/12) includes fixes to the runtime, the crypto/x509,
and net/http packages. See the Go 1.13.8 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.8+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-22 17:59:03 +01:00
Derek McGowan
861aa2a621
Merge pull request #3089 from thaJeztah/bump_golang_1.13.7
Update Golang 1.13.7, golang.org/x/crypto (CVE-2020-0601, CVE-2020-7919)
2020-02-21 18:08:27 -08:00
Sebastiaan van Stijn
9b6a019081
prevent dev-tools from updating go.mod
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-21 23:47:14 +01:00
Sebastiaan van Stijn
7e290869e7
vendor: update golang.org/x/crypto v0.0.0-20200128174031-69ecbb4d6d5d (CVE-2020-7919)
Includes 69ecbb4d6d
(forward-port of 8b5121be2f),
which fixes CVE-2020-7919:

- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
  On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
  functions of golang.org/x/crypto/cryptobyte can lead to a panic.
  The malformed certificate can be delivered via a crypto/tls connection to a
  client, or to a server that accepts client certificates. net/http clients can
  be made to crash by an HTTPS server, while net/http servers that accept client
  certificates will recover the panic and are unaffected.
  Thanks to Project Wycheproof for providing the test cases that led to the
  discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-21 23:45:56 +01:00
Sebastiaan van Stijn
016549532f
Dockerfile: use alpine 3.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-21 23:45:54 +01:00
Sebastiaan van Stijn
974375f66c
re-vendor with go 1.13
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-21 23:45:52 +01:00
Sebastiaan van Stijn
4ae059c714
Update Golang 1.13.7 (CVE-2020-0601, CVE-2020-7919)
full diff: https://github.com/golang/go/compare/go1.13.4...go1.13.7

go1.13.7 (released 2020/01/28) includes two security fixes. One mitigates
the CVE-2020-0601 certificate verification bypass on Windows. The other affects
only 32-bit architectures.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.7+label%3ACherryPickApproved

- X.509 certificate validation bypass on Windows 10
  A Windows vulnerability allows attackers to spoof valid certificate chains when
  the system root store is in use. These releases include a mitigation for Go
  applications, but it’s strongly recommended that affected users install the
  Windows security update to protect their system.
  This issue is CVE-2020-0601 and Go issue golang.org/issue/36834.
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
  On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
  functions of golang.org/x/crypto/cryptobyte can lead to a panic.
  The malformed certificate can be delivered via a crypto/tls connection to a
  client, or to a server that accepts client certificates. net/http clients can
  be made to crash by an HTTPS server, while net/http servers that accept client
  certificates will recover the panic and are unaffected.
  Thanks to Project Wycheproof for providing the test cases that led to the
  discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
  This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of golang.org/x/crypto/cryptobyte.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-21 23:45:49 +01:00
Derek McGowan
d054b13dc3
Merge pull request #3086 from thaJeztah/forward_port_ISS-2819
[master] Use same env var in Dockerfile and Makefile
2020-02-21 10:24:25 -08:00
Derek McGowan
244d5246c2
Merge pull request #3023 from sayboras/feature/golangci-lint
Migrate to golangci-lint
2020-02-21 10:21:15 -08:00
sayboras
66809646d9 Migrate to golangci-lint
Signed-off-by: Tam Mach <sayboras@yahoo.com>
2020-02-14 08:11:16 +11:00
Ryan Abrams
23f6bdd743
Use same env var in Dockerfile and Makefile
Ensures that build tags get set in the Dockerfile so that OSS and GCS drivers
are built into the official registry binary.

Closes #2819

Signed-off-by: Ryan Abrams <rdabrams@gmail.com>
(cherry picked from commit bf74e4f91d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-22 10:54:02 +01:00
Derek McGowan
a837179414
Merge pull request #3072 from fermayo/fix-TestRegistryAsCacheMutationAPIs
Fix TestRegistryAsCacheMutationAPIs
2019-12-15 20:48:56 -08:00
Fernando Mayo Fernandez
6ca7b9e9fa
Fix TestRegistryAsCacheMutationAPIs
Use a synthetic upstream registry when creating the testing mirror configuration
to avoid the test fail when trying to reach http://example.com

Signed-off-by: Fernando Mayo Fernandez <fernando@undefinedlabs.com>
2019-12-15 13:51:25 +01:00
Derek McGowan
f5e84a4939
Update readme and contributing docs
Update project details to align more closely with community
updates in OCI and containerd.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-12-13 17:20:45 -08:00
Tom Hayward
0f5e2753a6 Fix typo cloudfront updatefrenquency
Signed-off-by: Tom Hayward <thayward@infoblox.com>
2019-12-13 12:11:42 -08:00
Derek McGowan
bdf3438b52
Merge pull request #2985 from novas0x2a/default-transport
make it possible to wrap the client transport in another one
2019-12-13 11:36:12 -08:00
Derek McGowan
29c300c106
Merge pull request #3050 from ducksecops/patch-1
bump golang to 1.13.4
2019-12-13 11:09:05 -08:00
Derek McGowan
4798651387
Merge pull request #3053 from adrian-plata/table-fix
Fixing broken table
2019-12-13 11:08:00 -08:00
Derek McGowan
f18781257e
Merge pull request #3061 from guillaumerose/reconciliate
Add pathspec for repo _layers directory and allow Repository.BlobStore to enumerate over blobs
2019-12-06 11:27:13 -08:00
Guillaume Rose
c9c3324300 Add unit tests for BlobEnumerator
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-06 09:38:13 +01:00
Manish Tomar
5538da4923 fixes to make layersPathSpec work
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-05 20:48:33 +01:00
Manish Tomar
fa7d949408 allow Repository.BlobStore to enumerate over blobs
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-05 20:48:33 +01:00
Manish Tomar
cf77113795 add pathspec for repo _layers directory
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2019-12-05 20:48:33 +01:00
Adrian Plata
b4694b0d2d Fixing broken table
Signed-off-by: Adrian Plata <adrian.plata@docker.com>
2019-11-26 17:38:40 -08:00
ducksecops
070cc010f7 bump golang to 1.13.4
Signed-off-by: Daniel Sutton <daniel@ducksecops.uk>
2019-11-13 23:15:11 +00:00
Derek McGowan
dee21c0394
Merge pull request #2991 from ad-m/patch-1
use latest version of alpine when building the Docker container
2019-10-24 15:54:08 -07:00
Ryan Abrams
ae2e973db9
Merge pull request #2748 from manishtomar/tag-digests
API to retrive tag's digests
2019-10-08 12:14:56 -07:00