Commit graph

211 commits

Author SHA1 Message Date
Derek McGowan
607ae5d128
Merge pull request #2501 from xiaonancc77/master
Added ignore event types into notifications
2018-03-14 10:37:02 -07:00
elsanli(李楠)
fc1d3647c6 Added ignore event typs into notifications
Signed-off-by: elsanli(李楠) <elsanli@tencent.com>
2018-03-13 16:00:44 +08:00
Stephen Day
6664ec7039
Merge pull request #2466 from tifayuki/prometheus_go_metrics
add prometheus metrics
2018-02-09 15:49:04 -08:00
tifayuki
e3c37a46e2 Add Prometheus Metrics
at the first iteration, only the following metrics are collected:

  - HTTP metrics of each API endpoint
  - cache counter for request/hit/miss
  - histogram of storage actions, including:
    GetContent, PutContent, Stat, List, Move, and Delete

Signed-off-by: tifayuki <tifayuki@gmail.com>
2018-02-09 14:27:51 -08:00
Stephen J Day
585cdeb571
context, registry/handlers: remove net/context references
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-17 16:01:41 -08:00
Stephen J Day
1618b49d5b
registry/handlers: ignore notfound on storage driver healthcheck
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-21 15:04:31 -07:00
Stephen J Day
9c88801a12
context: remove definition of Context
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.

To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-11 15:53:31 -07:00
Derek McGowan
edc3ab29cd Merge pull request #2316 from igmor/logrus_case_dep_update
Update logrus and logrus-logstash-hook libraries
2017-07-26 10:46:10 -07:00
Stephen J Day
55ea440428
registry/{storage,handlers}: limit content sizes
Under certain circumstances, the use of `StorageDriver.GetContent` can
result in unbounded memory allocations. In particualr, this happens when
accessing a layer through the manifests endpoint.

This problem is mitigated by setting a 4MB limit when using to access
content that may have been accepted from a user. In practice, this means
setting the limit with the use of `BlobProvider.Get` by wrapping
`StorageDriver.GetContent` in a helper that uses `StorageDriver.Reader`
with a `limitReader` that returns an error.

When mitigating this security issue, we also noticed that the size of
manifests uploaded to the registry is also unlimited. We apply similar
logic to the request body of payloads that are full buffered.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-06 17:13:39 -07:00
Igor Morozov
a97d7c0c15 moved Sirupsen to sirupsen on a case sensitive system
Signed-off-by: Igor Morozov <igor@adhoc05-sjc1.prod.uber.internal>
2017-06-23 20:28:48 +00:00
Oleg Bulatov
5b1b6afae2 Update Docker-Content-Digest if manifest list is rewritten
If the client doesn't support manifest lists, the registry will
rewrite a manifest list into the old format. The Docker-Content-Digest
header should be updated in this case.

Signed-off-by: Oleg Bulatov <oleg@bulatov.me>
2017-06-16 16:20:14 +02:00
Troels Thomsen
1935c8d50b Return early to prevent nil pointer dereference
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2017-04-27 14:59:37 +02:00
Stephen J Day
1f0a9dbca0
register/handlers: remove context manager
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-04-10 15:27:30 -07:00
Derek McGowan
1bcc73251d
Update handler test to use WithName
Fix test panic in handler test.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-01-19 15:21:40 -08:00
Derek McGowan
11cb04e994 Merge pull request #2142 from dmcgowan/reference-enforce-canonical-parsing
reference: ParseNamed updated to enforce canonical format
2017-01-18 17:06:08 -08:00
Derek McGowan
d8fcbeeb71
Update registry to use WithName for creating Named values
The registry uses partial Named values which the named parsers
no longer support. To allow the registry service to continue
to operate without canonicalization, switch to use WithName.
In the future, the registry should start using fully canonical
values on the backend and WithName should no longer support
creating partial values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-01-18 13:56:56 -08:00
yuwaMSFT2
0bb696c5bf add test for manifest handler where storage driver fails
Signed-off-by: Yu Wang (UC) <yuwa@microsoft.com>
2017-01-18 11:39:45 -08:00
Yu Wang (UC)
2e94c22fe4 issue#2135 image pull returns 404 on manifest request if there is storage error
When get manifest, the handler will try to retrieve it from storage driver. When storage driver is cloud storage, it can fail due to various reasons even if the manifest exists
(like 500, 503, etc. from storage server). Currently manifest handler blindly return 404 which can be confusing to user.

This change will return 404 if the manifest blob doesn't exist, and return 500 UnknownError for all other errors (consistent with the behavior of other handlers).

Signed-off-by: Yu Wang (UC) <yuwa@microsoft.com>
2017-01-11 12:34:11 -08:00
Stephen J Day
c91a68ca05
registy/handlers: move images to manifests
Once upon a time, we referred to manifests and images interchangably.
That simple past is no more. As we grow, we update our nomenclature and
so follows our code.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-10 16:09:15 -08:00
Derek McGowan
2bc4a9459c Merge pull request #2121 from nwt/delete-action
Change DELETE action from "*" to "delete"
2017-01-09 12:34:23 -08:00
Stephen Day
7dba427612 Merge pull request #2111 from stevvooe/use-digest-package
digest: use new go-digest package
2017-01-06 16:25:57 -08:00
Stephen J Day
532ec9f036
digest: migrate to opencontainers/go-digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 15:42:03 -08:00
Noah Treuhaft
ccb839e0e3 Change DELETE action from "*" to "delete"
With token authentication, requiring the "*" action for DELETE requests
makes it impossible to administratively lock a repository against pushes
and pulls but still allow deletion.  This change adds a new "delete"
action for DELETE requests to make that possible.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2017-01-03 12:27:12 -08:00
yixi zhang
8e915d69f4 Use app.driver.Stat for registry health check
`app.driver.List` on `"/"` is very expensive if registry contains significant amount of images. And the result isn't used anyways.
In most (if not all) storage drivers, `Stat` has a cheaper implementation, so use it instead to achieve the same goal.

Signed-off-by: yixi zhang <yixi@memsql.com>
2016-12-21 17:12:43 -08:00
Stephen J Day
1040c824ba
registry/handlers: fix incorrect use of Digest.Verifiers
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-15 18:33:26 -08:00
Derek McGowan
729b8c5b91 Merge pull request #2105 from stevvooe/algorithm-own-file
digest: cleanup digester and verifier creation
2016-12-15 16:55:37 -08:00
Stephen J Day
e37baed88e
digest: cleanup digester and verifier creation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-15 15:17:21 -08:00
Stephen J Day
01dfa0fcb9
digest: use digest.Parse over ParseDigest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-15 15:07:42 -08:00
Derek McGowan
8867e8fac3
Update schema2 builder to take media type
Modify manifest builder so it can be used to build
manifests with different configuration media types.
Rename config media type const to image config.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2016-12-14 16:17:20 -08:00
Antonio Murdaca
0fb25dd094
registry/handles/app: always append default urls regexps
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-12-05 20:28:51 +01:00
Derek McGowan
e02278f22a
Update registry server to support repository class
Use whitelist of allowed repository classes to enforce.
By default all repository classes are allowed.

Add authorized resources to context after authorization.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-21 16:36:36 -08:00
Stephen J Day
c9aaff00f8
manifest: references should cover all children
To allow generic manifest walking, we define an interface method of
`References` that returns the referenced items in the manifest. The
current implementation does not return the config target from schema2,
making this useless for most applications.

The garbage collector has been modified to show the utility of this
correctly formed `References` method. We may be able to make more
generic traversal methods with this, as well.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-10-18 11:43:33 -07:00
Derek McGowan
d35d94dcec
Update to fix lint errors
Context should use type values instead of strings.
Updated direct calls to WithValue, but still other uses of string keys.
Update Acl to ACL in s3 driver.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-10-05 17:47:12 -07:00
Noah Treuhaft
ad6bb66faf Add notification filtering by target media type
The Hub registry generates a large volume of notifications, many of
which are uninteresting based on target media type.  Discarding them
within the notification endpoint consumes considerable resources that
could be saved by discarding them within the registry.  To that end,
this change adds registry configuration options to restrict the
notifications sent to an endpoint based on target media type.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-09-16 12:01:03 -07:00
Stephen Day
5e8d18f615 Merge pull request #1945 from stevvooe/better-report-on-invalid-secret
handlers: provide better log message on mismatched secret
2016-09-07 12:55:48 -07:00
Stephen J Day
668b0a5f40
handlers: provide better log message on mismatched secret
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-09-02 15:24:35 -07:00
Derek McGowan
798b7331ca Add shutdown to test environment
Ensures test http server is fully closed before moving on with test.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-26 16:23:07 -07:00
Derek McGowan
38ee36eee4 Disable upload purging during test
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-26 16:23:07 -07:00
Edgar Lee
379312c148 Stop ErrFinishedWalk from escaping from Repositories walk
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-08-05 17:21:48 -07:00
Richard Scothern
2b72dd3927 Merge pull request #1829 from nwt/foreign-layer-host-whitelist
Add a foreign layer URL host whitelist
2016-07-21 16:02:20 -07:00
Noah Treuhaft
61e5803b56 Add control over validation of URLs in pushed manifests
Until we have some experience hosting foreign layer manifests, the Hub
operators wish to limit foreign layers on Hub. To that end, this change
adds registry configuration options to restrict the URLs that may appear
in pushed manifests.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-07-19 14:38:42 -07:00
Edgar Lee
a82f661ef0 Use typecast over reflect for error type checking
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-07-14 15:03:18 -07:00
Edgar Lee
aeb9a29499 Handle new errors returned from catalog repository listing
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-07-14 13:24:16 -07:00
Cezar Sa Espinola
e015cbadd6
registry: avoid formatting errors with %#v
Signed-off-by: Cezar Sa Espinola <cezarsa@gmail.com>
2016-07-01 11:30:58 -03:00
Richard Scothern
ccfa25cf00 If resumable digest support is disabled, detct this when closing the blobwriter
and allow the close to continue.  Also update the name of the function.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-06-13 17:35:06 -07:00
Richard Scothern
1fc752c718 Merge pull request #1706 from aibaars/registry-size-close
Blobwriter: call BlobWriter.Size after BlobWriter.Close
2016-06-13 16:29:35 -07:00
Tianon Gravi
8907f7d189 Update "Accept" header parsing for list values
In Go's header parsing, the same header multiple times results in multiple entries in the `r.Header[...]` slice, but Go does no further parsing beyond that (and in https://golang.org/cl/4528086 it was determined that until/unless the stdlib itself needs it, Go will not do so).

The consequence here for parsing of `Accept:` headers is that we support the way Go outputs headers, but not all language HTTP libraries have a facility to output multiple headers instead of a single list header.

This change ensures that the following (valid) header blocks all parse to the same result for the purposes of what is being tested here:

```
Accept: a/b
Accept: b/c
Accept: d/e
```

```
Accept: a/b; q=0.5, b/c
Accept: d/e
```

```
Accept: a/b; q=0.1, b/c; q=0.2, d/e; q=0.8
```

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2016-06-10 16:52:27 -07:00
allencloud
db90724ab0 fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-06-02 23:03:27 +08:00
Richard Scothern
641f102967 Merge pull request #1687 from RichardScothern/signature-store
Remove signature store from registry.
2016-05-31 09:09:39 -07:00
Richard Scothern
0c15ab6952 Remove signature store from registry. Return a generated signature for manifest
pull.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-05-27 13:19:26 -07:00