Commit graph

716 commits

Author SHA1 Message Date
Stephen J Day
65b0d73cb7 Move storagedriver package to registry/storage/driver
This change is slightly more complex than previous package maves in that the
package name changed. To address this, we simply always reference the package
driver as storagedriver to avoid compatbility issues with existing code. While
unfortunate, this can be cleaned up over time.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-11 12:43:04 -08:00
Stephen J Day
3468fbd4a8 Move storage package under registry package
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-11 12:43:04 -08:00
Stephen J Day
2832ec7f5c Move client package under registry package
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-11 12:42:56 -08:00
Josh Hawn
89a0b6fa8b Merge pull request #110 from jlhawn/token_auth_spec
doc/spec: documentation for token based auth
2015-02-10 18:29:13 -08:00
Josh Hawn
a46af29783 Documentation for Token based Auth
Outlines the format of the tokens and how they are verified.

Outlines how clients should respond to bearer token authorization
challenges.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-02-10 18:15:14 -08:00
Stephen Day
fe20d2c38f Merge pull request #166 from BrianBland/config-omitempty
configuration: Adds yaml omitempty tags to config fields
2015-02-10 17:42:24 -08:00
Stephen J Day
0371f648bf Move auth package under registry package
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-10 17:34:04 -08:00
Stephen J Day
e4b811f489 Move registry api definitions under registry package
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-10 17:32:22 -08:00
Stephen J Day
0f08b6961a Move registry package into handler package
The goal is to free up the distribution/registry package to include common
registry types. This moves the webapp definitions out of the way to allow for
this change in the future.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-10 17:25:40 -08:00
Olivier Gambier
90c2e4b460 Merge pull request #165 from stevvooe/project-management-labels
define meaning of project management labels for issues and PRs
2015-02-10 16:57:15 -08:00
Brian Bland
c69764a745 Adds yaml omitempty tags to config fields 2015-02-10 16:56:19 -08:00
Olivier Gambier
ecef2eb99e Merge pull request #163 from stevvooe/bug-return-non-authorized
Correctly return when repo name is not available
2015-02-10 16:11:17 -08:00
Stephen J Day
12b55bcba0 Define meaning of project management labels for issues and PRs
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-10 15:55:36 -08:00
Stephen J Day
287de9f60a Correctly return when repo name is not available
The branch that executes after a failed request authorization due to a missing
repo name now correctly returns an error. This is somewhat superficial since
the response would have already been executed. Although, unintended repository
operations may have occurred.

Documentations and comments have also been updated to be in line with
surrounding changes.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-10 15:19:02 -08:00
Stephen Day
34c09cb56c Merge pull request #76 from ahmetalpbalkan/azure-fixes
storagedriver/azure: fix azure storage driver methods, implement Stat
2015-02-10 15:02:48 -08:00
Stephen Day
4be40d16eb Merge pull request #151 from stevvooe/context-aware-logging
context, registry, auth, auth/token, cmd/registry: context aware logging
2015-02-10 15:02:21 -08:00
Ahmet Alp Balkan
d606948fd7 Incorporate storagedriver base in azure, remove path checks
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-02-10 14:13:02 -08:00
Ahmet Alp Balkan
e7485c831f Support writes to random offsets in Azure storage driver
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-02-10 14:13:02 -08:00
Ahmet Alp Balkan
bc42f53ec8 Fix azure storagedriver methods, implement Stat, URLFor
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-02-10 11:17:41 -08:00
Ahmet Alp Balkan
bdc268bca3 Vendor azure-sdk-for-go with godep
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-02-10 11:17:41 -08:00
Stephen J Day
d9d84ae269 Integrate context with storage package
This changeset integrates context with the storage package. Debug messages have
been added to exported methods. Existing log messages will now include
contextual details through logger fields to aid in debugging. This integration
focuses on logging and may be followed up with a metric-oriented change in the
future.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-09 14:44:58 -08:00
Stephen Day
d224eda04c Merge pull request #154 from noxiouz/fix_deinitialization_deadlock
[IPC/Client] Get rid of an unnecessary stopChan sending
2015-02-09 14:41:20 -08:00
Olivier Gambier
cc91aeab00 Merge pull request #153 from stevvooe/move-spec-api
doc/spec: move API specification into correct position
2015-02-09 10:45:38 -08:00
Anton Tiurin
66eeadd754 [IPC/Client] There's no need to send struct{}{} to notify a
waiting goroutine with an infinite loop about the stop. It's enough
to close stopChan. Moreover it could lead to a deadlock if
driver.Stop is called before driver.handleSubprocessExit.

Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
2015-02-09 09:19:14 +03:00
Stephen J Day
b429176dd7 Move API specification into correct position
The goal is to maintain a specification heirarchy under doc/spec. This change
sets the example. The Makefile has also been changed update the AUTHORS file
and can now generate the specification.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 17:49:50 -08:00
Stephen J Day
d2bfb5825c Integrate contextual logging with regsitry app
This changeset integrates contextual logging into the registry web application.
Idiomatic context use is attempted within the current webapp layout. The
functionality is centered around making lifecycle objects (application and
request context) into contexts themselves. Relevant data has been moved into
the context where appropriate.  We still have some work to do to factor out the
registry.Context object and the dispatching functionality to remove some
awkward portions.

The api tests were slightly refactored to use a test environment to eliminate
common code.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 16:51:53 -08:00
Stephen J Day
9e33fb0f95 Update auth/token to use context package utils
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 16:20:38 -08:00
Stephen J Day
4b14eef595 Context package to support context-aware logging
The new context package supports context-aware logging, integrating with
logrus. Several utilities are provided to associate http requests with a
context, ensuring that one can trace log messages all the way through a
context-aware call stack.

A full description of this functionality is available in doc.go.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 16:20:38 -08:00
Stephen J Day
13382e45ba Update logrus dependency
This dependency added a method to access the current standard logger. This is
required to properly configure the logger for context awareness. The plan is to
have all loggers descend from the standard logger.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 16:20:38 -08:00
Stephen J Day
29135602ec Update go-uuid dependency
There don't seem to be any major changes but we'll update this while we're at
it. This dependency is pretty important for coming changes.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 16:20:38 -08:00
Stephen Day
10a194c775 Merge pull request #152 from stevvooe/makefile-improvements
Add common targets to Makefile
2015-02-06 16:15:19 -08:00
Stephen Day
05600271d7 Merge pull request #149 from AndreyKostov/s3-change-default-signature
storagedriver/s3: Change the default signature to v2 auth
2015-02-06 15:59:35 -08:00
Stephen J Day
d513d07e5f Add common targets to Makefile
A nice set of common default targets have been added to the Makefile to make
running tests and building binaries more straightforward. Running the Makefile
before committing should make build failures less common.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-06 15:39:11 -08:00
Stephen Day
7028457ef4 Merge pull request #150 from BrianBland/layerUploadReadFrom
Makes LayerUpload an io.ReaderFrom
2015-02-05 18:00:12 -08:00
Andrey Kostov
97be787339 Change the default signature to v2 auth
The reason we prefer v2 is that it performs better since it does not
hash the payloads.
2015-02-05 17:41:04 -08:00
Brian Bland
0bc84606cc Makes LayerUpload an io.ReaderFrom
This allows wrappers of LayerUpload to implement io.ReadFrom, which
prevents io.Copy on LayerUpload implementations from using repeated 32kB
Writes.

This has a huge performance implication, especially for s3/azure storage
drivers.
2015-02-05 17:37:52 -08:00
Stephen Day
9b9ea55797 Merge pull request #140 from jlhawn/137_auth_context
Use context for auth access controllers
2015-02-04 19:25:10 -08:00
Josh Hawn
2c3d738a05 Use context for auth access controllers
The auth package has been updated to use "golang.org/x/net/context" for
passing information between the application and the auth backend.

AccessControllers should now set a "auth.user" context value to a AuthUser
struct containing a single "Name" field for now with possible, optional, values
in the future.

The "silly" auth backend always sets the name to "silly", while the "token" auth
backend will set the name to match the "subject" claim of the JWT.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-02-04 19:17:33 -08:00
Stephen Day
15df84768d Merge pull request #145 from AndreyKostov/s3-test-fix
Skip s3 tests if no region is specified
2015-02-04 12:00:22 -08:00
Andrey Kostov
85768b7638 Skip s3 tests if no region is specified 2015-02-04 11:52:59 -08:00
Stephen Day
1a1d87801a Merge pull request #139 from stevvooe/storagedriver-base
Move shared checks to base storage driver
2015-02-04 10:58:01 -08:00
Stephen Day
d7c62e06f4 Merge pull request #141 from stevvooe/put-manifest-http-status
Manifest PUT should return 202 Accepted status
2015-02-04 10:57:17 -08:00
Stephen J Day
f5c01af13f Manifest PUT should return 202 Accepted status
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-03 18:27:40 -08:00
Stephen J Day
f265a29f24 Move shared checks to base storage driver
This changeset moves common checks from driver implementations into base
package. The Base type can be embedded in a driver implementation, ensuring
that incoming arguments are checked for validity.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-03 17:05:50 -08:00
Stephen Day
c1c7d3dabf Merge pull request #113 from stevvooe/event-sink-endpoint
Webhook notification support in registry webapp
2015-02-03 16:25:03 -08:00
Stephen Day
624fb9b565 Merge pull request #78 from ahmetalpbalkan/TestPutContentMultipleTimes
Add TestPutContentMultipleTimes to storage driver suite
2015-02-03 13:35:55 -08:00
Stephen J Day
080e329cb1 Separate request data from actor in Event
To clarify the role of actor, the request data that initiates an event has been
separated. The ActorRecord is pared down to just the username. This eliminates
confusion about where event related data should be added.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-03 13:32:37 -08:00
Stephen J Day
0a29b59e14 Webhook notification support in registry webapp
Endpoints are now created at applications startup time, using notification
configuration. The instances are then added to a Broadcaster instance, which
becomes the main event sink for the application. At request time, an event
bridge is configured to listen to repository method calls. The actor and source
of the eventBridge are created from the requeest context and application,
respectively. The result is notifications are dispatched with calls to the
context's Repository instance and are queued to each endpoint via the
broadcaster.

This commit also adds the concept of a RequestID and App.InstanceID. The
request id uniquely identifies each request and the InstanceID uniquely
identifies a run of the registry. These identifiers can be used in the future
to correlate log messages with generated events to support rich debugging.

The fields of the app were slightly reorganized for clarity and a few horrid
util functions have been removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-03 13:32:37 -08:00
Stephen J Day
499382dd0b Add debug server to support pprof and expvar
If configured, a debug http server will be started to serve default registered
endpoints, such as pprof and expvar. The endpoint should be secured carefully
and not available to external traffic. It is disabled by default but the
development config has been modified to make it available on localhost.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-03 13:32:37 -08:00
Stephen J Day
e5de2594ad Remove decorator package
After implementing notifications end to end, it was found that decorating
repositories was more straightforward that previously thought. It's unfortunate
to can this package, but it led to the techniques employed in
storage/notifications/listeners.go. The ultimate result turned out much better.
2015-02-03 13:30:20 -08:00