Commit graph

4869 commits

Author SHA1 Message Date
Stephen Day
f5a34a009a Merge pull request #317 from stevvooe/notification-docs
doc: document event notification system
2015-04-03 13:49:24 -07:00
Stephen Day
d43adcd910 Merge pull request #320 from RichardScothern/docs
doc: document storage middleware.
2015-04-03 13:48:43 -07:00
Stephen Day
aea2965636 Merge pull request #316 from stevvooe/eventual-consistency-approach
doc: document approach to eventual consistency
2015-04-03 13:46:19 -07:00
Richard
3573c6c466 Document storage middleware. 2015-04-03 13:45:40 -07:00
Doug Davis
e5408bd911 Remove engine.Table from docker search and fix missing field
registry/SearchResults was missing the "is_automated" field.
I added it back in.

Pull this 'table' removal one from the others because it fixed
a bug too

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-04-03 11:27:30 -07:00
Olivier Gambier
38b8ddbe59 Merge pull request #318 from CpuID/cpuid_fix_api_issue_link
Fix GitHub issue link, simple typo
2015-04-03 10:02:36 -07:00
Nathan Sullivan
06ff65c579 fix github issue link, simple typo 2015-04-04 01:03:43 +10:00
Stephen J Day
91413606a3 Document event notification system
An overview the notification system is provided, covering topics on
architecture, configuration, implementation and formats. This may need to
filled out with further details covering format specifications.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 22:09:25 -07:00
Stephen J Day
6b748a74ef Move expvar under the registry section
For consistency with other systems, the redis and caching monitoring data has
been moved under the "registry" section in expvar. This ensures the entire
registry state is kept to a single section.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 21:30:27 -07:00
Stephen J Day
09d48c7192 Move expvar under the registry section
For consistency with other systems, the redis and caching monitoring data has
been moved under the "registry" section in expvar. This ensures the entire
registry state is kept to a single section.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 21:30:27 -07:00
Stephen J Day
2df6b670c6 Document approach to eventual consistency
There is probably a better place for this documentation but we'd like to move
this elsewhere than a github issue. We can move this to a more appropriate
location with the documentation effort.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:24:18 -07:00
Stephen J Day
4e1ecad6cc Allow control over which storage cache to use
This allows one to better control the usage of the cache and turn it off
completely. The storage configuration module was modified to allow parameters
to be passed to just the storage implementation, rather than to the driver.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:16 -07:00
Stephen J Day
919d972e65 Allow control over which storage cache to use
This allows one to better control the usage of the cache and turn it off
completely. The storage configuration module was modified to allow parameters
to be passed to just the storage implementation, rather than to the driver.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:16 -07:00
Stephen J Day
6ab228f798 Integrate layer info cache with registry and storage
This changeset integrates the layer info cache with the registry webapp and
storage backend. The main benefit is to cache immutable layer meta data,
reducing backend roundtrips. The cache can be configured to use either redis or
an inmemory cache.

This provides massive performance benefits for HEAD http checks on layer blobs
and manifest verification.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:16 -07:00
Stephen J Day
44b14ceadc Integrate layer info cache with registry and storage
This changeset integrates the layer info cache with the registry webapp and
storage backend. The main benefit is to cache immutable layer meta data,
reducing backend roundtrips. The cache can be configured to use either redis or
an inmemory cache.

This provides massive performance benefits for HEAD http checks on layer blobs
and manifest verification.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:16 -07:00
Stephen J Day
a7c2dceea5 Define and implement layer info cache
This changeset defines the interface for layer info caches. Layer info caches
speed up access to layer meta data accessed in storage driver backends. The
two main operations are tests for repository membership and resolving path and
size information for backend blobs.

Two implementations are available. The main implementation leverages redis to
store layer info. An alternative implementation simply caches layer info in
maps, which should speed up resolution for less sophisticated implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:09 -07:00
Stephen J Day
b1f616cbff Define and implement layer info cache
This changeset defines the interface for layer info caches. Layer info caches
speed up access to layer meta data accessed in storage driver backends. The
two main operations are tests for repository membership and resolving path and
size information for backend blobs.

Two implementations are available. The main implementation leverages redis to
store layer info. An alternative implementation simply caches layer info in
maps, which should speed up resolution for less sophisticated implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:09 -07:00
Stephen Day
dff57726f9 Merge pull request #295 from jlhawn/use_resumable_digest
digest, registry/storage: use resumable digest
2015-04-02 20:12:57 -07:00
Stephen Day
cf5839b019 Merge pull request #295 from jlhawn/use_resumable_digest
digest, registry/storage: use resumable digest
2015-04-02 20:12:57 -07:00
Stephen Day
721b39d0cb Merge pull request #314 from stevvooe/lockdown-uuid-route
registry/api/v2: stronger validation for uuid field in urls
2015-04-02 13:57:17 -07:00
Stephen Day
7c3b73482f Merge pull request #314 from stevvooe/lockdown-uuid-route
registry/api/v2: stronger validation for uuid field in urls
2015-04-02 13:57:17 -07:00
Stephen Day
277c68d51e Merge pull request #313 from stevvooe/move-panic-fix
registry/storage/driver/inmemory: avoid crash on invalid Move arguments
2015-04-02 13:56:40 -07:00
Josh Hawn
b96de45be8 Use resumable digest for efficient upload finish
By using a resumable digester and storing the state of upload digests between
subsequent upload chunks, finalizing an upload no longer requires reading back
all of the uploaded data to verify the client's expected digest.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-02 11:20:31 -07:00
Josh Hawn
18c9a1cdd8 Use resumable digest for efficient upload finish
By using a resumable digester and storing the state of upload digests between
subsequent upload chunks, finalizing an upload no longer requires reading back
all of the uploaded data to verify the client's expected digest.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-02 11:20:31 -07:00
Stephen J Day
06acde06cb Avoid crash on invalid Move arguments
This chnage prevents a crash when moving from a non-existent directory that has
a file as a parent. To prevent this, we simply check that the node is a
directory and throws an error if it is not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 19:03:02 -07:00
Stephen J Day
f26a283a48 Avoid crash on invalid Move arguments
This chnage prevents a crash when moving from a non-existent directory that has
a file as a parent. To prevent this, we simply check that the node is a
directory and throws an error if it is not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 19:03:02 -07:00
Stephen J Day
6eb804a1ec Stronger validation for uuid field in urls
This change adds strong validation for the uuid variable for v2 routes. This is
a minor specification change but is okay since the uuid field is controlled by
the server. The character set is restricted to avoid path traversal, allowing
for alphanumeric values and urlsafe base64 encoding.

This change has no effect on client implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 18:57:59 -07:00
Stephen J Day
1944be9db3 Stronger validation for uuid field in urls
This change adds strong validation for the uuid variable for v2 routes. This is
a minor specification change but is okay since the uuid field is controlled by
the server. The character set is restricted to avoid path traversal, allowing
for alphanumeric values and urlsafe base64 encoding.

This change has no effect on client implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 18:57:59 -07:00
Josh Hawn
60b6748c95 Use resumable hash in digest package
Vendored resumable sha256/sha512 library. Digest package new exports a
resumable variant of the Digester.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-01 17:11:47 -07:00
Stephen Day
da9d49d186 Merge pull request #310 from jlhawn/improve_context_pkg
context: improve context package
2015-04-01 16:39:59 -07:00
Stephen J Day
38ae1cb461 Add redis pool to registry webapp
Redis has been integrated with the web application for use with various
services. The configuraiton exposes connection details, timeouts and pool
parameters. Documentation has been updated accordingly.

A few convenience methods have been added to the context package to get loggers
with certain fields, exposing some missing functionality from logrus.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 16:27:24 -07:00
Stephen J Day
3cad3c7b6a Add redis pool to registry webapp
Redis has been integrated with the web application for use with various
services. The configuraiton exposes connection details, timeouts and pool
parameters. Documentation has been updated accordingly.

A few convenience methods have been added to the context package to get loggers
with certain fields, exposing some missing functionality from logrus.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 16:27:24 -07:00
Stephen J Day
fcdfdd2ae0 Add redigo dependency to Godeps
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 15:55:30 -07:00
Josh Hawn
731e0b0066 Improve context package
You shouldn't have to import both:

  github.com/docker/distribution/context
  golang.org/x/net/context

just to use the distribution tools and implement the distribution interfaces.

By pulling the Context interface from golang.org/x/net/context into the
context package within the distribution project, you no longer have to import
both packages.

Note: You do not have to change anything anywhere else yet! All current uses
of both packages together will still work correctly because the Context
interface from either package is identical.

I've also made some other minor changes:

- Added a RemoteIP function. It's like RemoteAddr but discards the port suffix
- Added `.String()` to the response duration context value so that JSON log
  formatting shows human-parseable duration and not just number of nano-seconds
- Added WithMapContext(...) to the context package. This is a useful function
  so I pulled it out of the main.go in cmd/registry so that it can be used
  elsewhere.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-01 15:40:21 -07:00
Stephen Day
5052dc692f Merge pull request #283 from sthulb/client-certs
registry, configuration, doc: client certificates for authentication purposes.
2015-03-31 16:45:03 -07:00
Michael Crosby
638ccff564 Remove jobs from registry.Service
This makes `registry.Service` a first class type and does not use jobs
to interact with this type.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-03-31 16:38:04 -07:00
Derek McGowan
67e5c940c4 Use vendored v2 registry api
Update registry package to use the v2 registry api from distribution. Update interfaces to directly take in digests.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-31 15:02:27 -07:00
Antonio Murdaca
5fa2d814f8 Refactor utils/http.go, fixes #11899
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-30 17:59:57 +02:00
Brian Goff
9e9a8add19 Merge pull request #11788 from reikani/pchoi
Changed snake case naming to camelCase.
2015-03-26 23:55:50 -07:00
Antonio Murdaca
d5045d054b Replace aliased imports of logrus, fixes #11762
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-26 23:22:04 +01:00
Peter Choi
b085d5556e Changed snake case naming to camelCase
Signed-off-by: Peter Choi <phkchoi89@gmail.com>
2015-03-26 15:05:45 -06:00
Don Kjer
eff5278d12 Fix for issue 9922: private registry search with auth returns 401
Signed-off-by: Don Kjer <don.kjer@gmail.com>
2015-03-26 18:59:16 +00:00
Stephen Day
fd3373b91c Merge pull request #302 from RichardScothern/richardscothern-298
Attempt to identify remote IP addresses for requests which come through proxies.
2015-03-25 15:43:34 -07:00
Alexander Morozov
e4afe03dcc Merge pull request #11761 from runcom/11731-remove-engine-status
Remove engine.Status, replace with standard go error
2015-03-25 15:41:02 -07:00
Antonio Murdaca
9c08a43624 Remove engine.Status and replace it with standard go error
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-25 22:32:08 +01:00
Richard
78562258b2 Log invalid remote IPs 2015-03-25 13:44:16 -07:00
Stephen Day
e9c69ffe14 Merge pull request #293 from stevvooe/log-formatter-configuration
main, configuration, doc: allow logging formatter to be configured
2015-03-25 13:34:42 -07:00
Richard
e21a425f88 Verify IP addresses
Fix lint errors
Add more test
2015-03-25 11:11:46 -07:00
Simon Thulbourn
c8f3800f1c Add client certificate CA option to authenticate with client certs
Add the ability to authenticate against multiple client CA certificates.

Signed-off-by: Simon Thulbourn <simon+github@thulbourn.com>
2015-03-25 10:53:03 +00:00
Stephen J Day
ac73963d7e Add support for configuration static logging fields
To allow flexibility in log message context information, this changeset
provides the ability to configure static fields that are included in the
context. Such fields can be set via configuration or environment variables.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-03-24 18:47:22 -07:00