Commit graph

164 commits

Author SHA1 Message Date
Aaron Vinson
db36173705 add info about redis eviction policy recommended setting 2015-08-26 18:19:13 -07:00
Aaron Lehmann
b67aab2f60 Add headers parameter for HTTP checker
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:16 -07:00
Aaron Lehmann
e8f088fea6 Add a TCP health checker
Also, add timeout and status code parameters to the HTTP checker, and
remove the threshold parameter for the file checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:15 -07:00
Aaron Lehmann
216df32510 Add storagedriver section to health check configuration
Add default storagedriver health check to example configuration files
with parameters matching the previous hardcoded configuration.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
b09b0ffcf9 Add configurable file-existence and HTTP health checks
Add a section to the config file called "health". Within this section,
"filecheckers" and "httpcheckers" list checks to run. Each check
specifies a file or URI, a time interval for the check, and a threshold
specifying how many times the check must fail to reach an unhealthy
state.

Document the new options in docs/configuration.md.

Add unit testing for both types of checkers. Add an UnregisterAll
function in the health package to support the unit tests, and an
Unregister function for consistency with Register.

Fix a string conversion problem in the health package's HTTP checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Richard Scothern
f169359798 Merge pull request #846 from aaronlehmann/http-header-configuration
Add a section to the config file for HTTP headers to add to responses
2015-08-18 12:53:05 -07:00
Miquel Sabaté
d07ae75f90 docs: added the "delete" subsection in the configuration.md file
Signed-off-by: Miquel Sabaté <msabate@suse.com>
2015-08-13 20:18:35 +02:00
Aaron Lehmann
9c3bed6b88 Add a section to the config file for HTTP headers to add to responses
The example configuration files add X-Content-Type-Options: nosniff.

Add coverage in existing registry/handlers unit tests.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-11 16:41:38 -07:00
Richard Scothern
7b0d831e6d Merge pull request #853 from dmp42/4.docs-2.1
Additional fixes
2015-08-11 14:32:57 -07:00
Richard Scothern
c78bfc5c59 Merge pull request #852 from ahmetalpbalkan/docs/typo
docs: Small typo fix in config.md
2015-08-11 14:31:35 -07:00
Ahmet Alp Balkan
c471d986d7 docs: Small typo fix
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-08-11 21:29:33 +00:00
Olivier Gambier
c198f8f279 Additional fixes
Couples of nits that where not addressed.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-08-11 14:19:49 -07:00
Ahmet Alp Balkan
047877c079 docs: Fix markdown in configuration.md
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-08-11 19:03:53 +00:00
Aaron Lehmann
143aae8bc5 Merge pull request #837 from dmp42/4.docs-2.1
[WIP] Updating documentation for 2.1
2015-08-10 14:01:52 -07:00
Olivier Gambier
34067d7d43 Documentation work
- move away insecure & self-signed
- introduce native basic auth
- move "down" nginx based authentication
- overall shortening / smoothing of "deploying" documentation

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-08-10 13:57:35 -07:00
Dejan Golja
8221fd39bd Added missing auth type in the documentation
Signed-off-by: Dejan Golja <dejan@golja.org>
2015-08-11 00:11:17 +10:00
Alex Chan
51bd34eaed Fix a few typos in the docs
Signed-off-by: Alex Chan <alex.chan@metaswitch.com>
2015-07-31 13:36:43 +01:00
Aaron Lehmann
6c2ef78aa7 Rename dev-config.yml and example-config.yml to config-dev.yml and config-example.yml
Better for sort order.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-30 15:51:50 -07:00
Stephen Day
ad1b7d8f35 Merge pull request #740 from stevvooe/disable-storage-redirects
Allow disabling of storage driver redirects
2015-07-30 14:28:31 -07:00
Richard Scothern
7eabb7800e Merge pull request #790 from aaronlehmann/config-files
Sample config file changes
2015-07-30 13:20:43 -07:00
Aaron Lehmann
f0c5284d2d Sample config file changes
Rename config.yml to dev-config.yml

Add example-config.yml, a simple configuration file for the official
This was originally made for the the distribution-library-image repo,
but is being moved here to make sure it stays in sync.

Update Dockerfile and docs for the rename.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-30 12:14:39 -07:00
Dave Tucker
9348141616 Fix dead links in configuration.md
Fixes #672

Signed-off-by: Dave Tucker <dt@docker.com>
2015-07-30 17:56:04 +01:00
Aaron Lehmann
70d9a9d543 Automatically generate a HTTP secret if none is provided
Log a warning if the registry generates its own secret.

Update configuration doc, and remove the default secret from the
development config file.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-29 14:28:56 -07:00
Stephen J Day
9f9a7f230b Allow disabling of starage driver redirects
Storage drivers can implement a method called URLFor which can return a direct
url for a given path. The functionality allows the registry to direct clients
to download content directly from the backend storage. This is commonly used
with s3 and cloudfront. Under certain conditions, such as when the registry is
not local to the backend, these redirects can hurt performance and waste
incoming bandwidth on pulls. This feature addition allows one to disable this
feature, if required.

Signed-off-by: Stephen J Day <stephen.day@docker.com>

Conflicts:
	configuration/configuration.go
	registry/handlers/app.go
	registry/storage/catalog_test.go
	registry/storage/manifeststore_test.go
	registry/storage/registry.go
2015-07-24 16:59:35 -07:00
davidli
b5c7001bc9 Update configuration doc for authurl with adding the Openstack Identity v1.0 example
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:10 +02:00
Olivier Jacques
470f84078e Swift documentation updates
Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
2015-07-21 23:55:10 +02:00
davidli
af99dbd6bf Add support for Openstack Identity v3 API
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ea7c53df08 Add Openstack Swift storage driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:08 +02:00
Ayose Cazorla
dbf3ce3cde Fix link to token specification
Link was broken in cf9b4ab5e9

Signed-off-by: Ayose <ayosec@gmail.com>
2015-07-16 04:51:34 +01:00
Olivier Jacques
f03e25221f Fix indentation for several subs
Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
2015-07-11 10:38:39 +02:00
Olivier Jacques
b3683863dd docs: fix yaml sections format at docs.docker.com
yaml sections in the documentation does not display well on
docs.docker.com. This is due to the syntax highlighting
which uses highlight.js and does not support yaml
currently.
The fix is to remove triple back ticks and indent instead.
We loose yaml syntax highlighting on github, but it displays
an acceptable version on both github and docs.docker.com.

Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
2015-07-10 23:22:06 +02:00
Olivier Gambier
4e95367e55 Fixed doc generation
+ rollback the (now) unecessary changes

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-17 18:58:53 -07:00
Mary Anthony
1aa8b00bdf Adding in the better sed
Renaming to index.md;rereading of Hugo showed me my mistake; removing commented out/Markdown has no comment feature
Updating with Olivier. Yay! It looks great

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-17 17:31:13 -07:00
moxiegirl
020bd0b45f Merge pull request #631 from moxiegirl/test-tooling
Updating for new docs build and tooling infrastructure 1.7 release; Thank you Olivier. I promise to look at all of this after Dockercon.  We'll make it perfect.
2015-06-15 17:57:25 -07:00
Olivier Gambier
d678e3dc57 Merge pull request #608 from dmcgowan/http-basic-auth
Implementation of a basic authentication scheme using standard .htpasswd
2015-06-15 17:40:58 -07:00
Mary Anthony
832cb9d52c Updating for Hugo
Updating for tooling tests
Updating with the new sed scripts to protect links
updating with new image
Updating with comments

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-15 14:39:33 -07:00
Stephen J Day
d9d55bcbab Minor formatting fixes related to htpasswd auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-11 17:06:35 -07:00
Stephen J Day
01f730ad71 Document usage of htpasswd access controller
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-11 15:23:40 -07:00
Olivier Gambier
6e0cfc17dc Saner default data location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-10 20:54:24 -07:00
Olivier Gambier
c1c638d01f Minor keyword add
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
c3b3802503 Small fixes
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
b7b8e64f1d Moved instructions up
Make it more obvious that environment variables is the way to go.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
26d87ed1a5 No reason to detail here
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Ahmet Alp Balkan
89f183639a docs: Update azure docs in configuration.md
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-05 17:25:17 -07:00
Stephen Day
0d40913b9a Merge pull request #394 from xiekeyang/feature-panic-hook
Feature: Add Hook for Web Application Panic
2015-06-01 13:23:32 -07:00
xiekeyang
47aa47e3f6 Feature: Web Panic Reporting via hooks
This PR is for issue of "email after registry webapp panic" #41, improving my
previous design (closed).
It use self setting up hooks, to catch panic in web application.
And, send email in hooks handle directly, to no use new http server and
handler.

Signed-off-by: xiekeyang <keyangxie@126.com>
2015-05-31 14:21:22 +00:00
Stephen Day
318af0b1ce Merge pull request #443 from gierschv/driver-rados
Storage Driver: Ceph Object Storage (RADOS)
2015-05-29 18:10:49 -07:00
Stephen J Day
dfa8504408 Revise description of debug endpoint
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-20 15:24:25 -07:00
Vincent Giersch
394eea0231 Storage Driver: Ceph Object Storage (RADOS)
This driver implements the storagedriver.StorageDriver interface and
uses Ceph Object Storage as storage backend.

Since RADOS is an object storage and no hierarchy notion, the
following convention is used to keep the filesystem notions stored in
this backend:

* All the objects data are stored with opaque UUID names prefixed
  (e.g. "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164).
* All the hierarchy information are stored in rados omaps, where the
  omap object identifier is the virtual directory name, the keys in
  a specific are the relative filenames and the values the blob
  object identifier (or empty value for a sub directory).

  e.g. For the following hierarchy:

     /directory1
     /directory1/object1
     /directory1/object2
     /directory1/directory2/object3

  The omap "/directory1" will contains the following key / values:
    - "object1" "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164"
    - "object2" "blob:db2e359d-4af0-4bfb-ba1d-d2fd029866a0"
    - "directory2" ""

  The omap "/directory1/directory2" will contains:
    - "object3" "blob:9ae2371c-81fc-4945-80ac-8bf7f566a5d9"

* The MOVE is implemented by changing the reference to a specific
  blob in its parent virtual directory omap.

This driver stripes rados objects to a fixed size (e.g. 4M). The idea
is to keep small objects (as done by RBD on the top of RADOS) that
will be easily synchronized accross OSDs. The information of the
original object (i.e total size of the chunks) is stored as a Xattr
in the first chunk object.

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-05-20 01:44:34 +00:00
Mary Anthony
8d407c81ae Closes #485
Signed-off-by: Mary Anthony <mary@docker.com>

Tweak per Stephen

Signed-off-by: Mary Anthony <mary@docker.com>
2015-05-18 12:56:17 -07:00
Stephen J Day
593bbccdb5 Refactor Blob Service API
This PR refactors the blob service API to be oriented around blob descriptors.
Identified by digests, blobs become an abstract entity that can be read and
written using a descriptor as a handle. This allows blobs to take many forms,
such as a ReadSeekCloser or a simple byte buffer, allowing blob oriented
operations to better integrate with blob agnostic APIs (such as the `io`
package). The error definitions are now better organized to reflect conditions
that can only be seen when interacting with the blob API.

The main benefit of this is to separate the much smaller metadata from large
file storage. Many benefits also follow from this. Reading and writing has
been separated into discrete services. Backend implementation is also
simplified, by reducing the amount of metadata that needs to be picked up to
simply serve a read. This also improves cacheability.

"Opening" a blob simply consists of an access check (Stat) and a path
calculation. Caching is greatly simplified and we've made the mapping of
provisional to canonical hashes a first-class concept. BlobDescriptorService
and BlobProvider can be combined in different ways to achieve varying effects.

Recommend Review Approach
-------------------------

This is a very large patch. While apologies are in order, we are getting a
considerable amount of refactoring. Most changes follow from the changes to
the root package (distribution), so start there. From there, the main changes
are in storage. Looking at (*repository).Blobs will help to understand the how
the linkedBlobStore is wired. One can explore the internals within and also
branch out into understanding the changes to the caching layer. Following the
descriptions below will also help to guide you.

To reduce the chances for regressions, it was critical that major changes to
unit tests were avoided. Where possible, they are left untouched and where
not, the spirit is hopefully captured. Pay particular attention to where
behavior may have changed.

Storage
-------

The primary changes to the `storage` package, other than the interface
updates, were to merge the layerstore and blobstore. Blob access is now
layered even further. The first layer, blobStore, exposes a global
`BlobStatter` and `BlobProvider`. Operations here provide a fast path for most
read operations that don't take access control into account. The
`linkedBlobStore` layers on top of the `blobStore`, providing repository-
scoped blob link management in the backend. The `linkedBlobStore` implements
the full `BlobStore` suite, providing access-controlled, repository-local blob
writers. The abstraction between the two is slightly broken in that
`linkedBlobStore` is the only channel under which one can write into the global
blob store. The `linkedBlobStore` also provides flexibility in that it can act
over different link sets depending on configuration. This allows us to use the
same code for signature links, manifest links and blob links.  Eventually, we
will fully consolidate this storage.

The improved cache flow comes from the `linkedBlobStatter` component
of `linkedBlobStore`. Using a `cachedBlobStatter`, these combine together to
provide a simple cache hierarchy that should streamline access checks on read
and write operations, or at least provide a single path to optimize. The
metrics have been changed in a slightly incompatible way since the former
operations, Fetch and Exists, are no longer relevant.

The fileWriter and fileReader have been slightly modified to support the rest
of the changes. The most interesting is the removal of the `Stat` call from
`newFileReader`. This was the source of unnecessary round trips that were only
present to look up the size of the resulting reader. Now, one must simply pass
in the size, requiring the caller to decide whether or not the `Stat` call is
appropriate. In several cases, it turned out the caller already had the size
already. The `WriterAt` implementation has been removed from `fileWriter`,
since it is no longer required for `BlobWriter`, reducing the number of paths
which writes may take.

Cache
-----

Unfortunately, the `cache` package required a near full rewrite. It was pretty
mechanical in that the cache is oriented around the `BlobDescriptorService`
slightly modified to include the ability to set the values for individual
digests. While the implementation is oriented towards caching, it can act as a
primary store. Provisions are in place to have repository local metadata, in
addition to global metadata. Fallback is implemented as a part of the storage
package to maintain this flexibility.

One unfortunate side-effect is that caching is now repository-scoped, rather
than global. This should have little effect on performance but may increase
memory usage.

Handlers
--------

The `handlers` package has been updated to leverage the new API. For the most
part, the changes are superficial or mechanical based on the API changes. This
did expose a bug in the handling of provisional vs canonical digests that was
fixed in the unit tests.

Configuration
-------------

One user-facing change has been made to the configuration and is updated in
the associated documentation. The `layerinfo` cache parameter has been
deprecated by the `blobdescriptor` cache parameter. Both are equivalent and
configuration files should be backward compatible.

Notifications
-------------

Changes the `notification` package are simply to support the interface
changes.

Context
-------

A small change has been made to the tracing log-level. Traces have been moved
from "info" to "debug" level to reduce output when not needed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-15 17:05:18 -07:00
Anton Tiurin
ad80cbe1ea [Server] Listen and serve on a unix socket
Allow to use a unix socket as a listener.
To specify an endpoint type we use an optional configuration
field 'net', as there's no way to distinguish a relative
socket path from a hostname.

Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
2015-05-11 16:00:14 +03:00
Stephen J Day
aac8e0d0bf Remove configuration/README.md
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-06 15:31:12 -07:00
Mary Anthony
68c0682e00 Fixes Issue #471 with Publish
- Add sed to Dockerfile; this sed exists on publish script; breaks headings/nav in files without metadata
- Ensure sed runs over storage-driver/ subdir
- Add metadata to all the files (including specs) that don't have it; this ensures they display correctly on publish
- Implement the fix for the showing up in Github
- Update template with GITHUB IGNORES

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-30 15:39:40 -07:00
Mary Anthony
5043bdbc93 Updating env var documentation
The position was a bit too early. Fleshed out the example.
Also, using the _ underscore for emphasis was confusing because it is also used to indicate
a level change.

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-27 15:49:19 -07:00
Stephen Day
70427e3d9f Merge pull request #381 from RichardScothern/purge-config
Add configuration for upload purging
2015-04-27 14:33:34 -07:00
Richard
937c356585 Add environment variable override instructions
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 11:23:37 -07:00
Richard
6460ddb2cb Add configuration for upload purging
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 11:06:15 -07:00
Mary Anthony
d588ed13b2 Fixes #391 for registry top page
Fixes #390 to add building link to README
Docker Registry Service to Docker Registry

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-18 17:32:57 -07:00
Michael Prokop
647e0a48b3 docs: drop newlines in URLs to avoid 404
The docs as available at
https://github.com/docker/distribution/blob/master/docs/configuration.md
result in 404 errors:

  https://github.com/docker/distribution/blob/master/cmd/registry/%0Aconfig.yml
  http://docs.aws.amazon.com/AWSSecurityCredentials/1.0/%0AAboutAWSCredentials.html#KeyPairs

instead of pointing to the correct ones, being:

  https://github.com/docker/distribution/blob/master/cmd/registry/config.yml
  http://docs.aws.amazon.com/AWSSecurityCredentials/1.0/AboutAWSCredentials.html#KeyPairs

So avoid the newlines in the corresponding source files.

Signed-off-by: Michael Prokop <mika@grml.org>
2015-04-17 23:09:43 +02:00
Mary Anthony
b939d6d118 Updating with new RC work
Fixing typos and adding tables
Updating with testing material
Tweak after read through
Adding in Stephen's comments
Adding in Richard's comments. Fixing the broken images
closes issue #363
Another try

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-16 10:44:30 -07:00
Derek McGowan
142d62798e Rename top level registry interface to namespace
Registry is intended to be used as a repository service than an abstract collection of repositories. Namespace better describes a collection of repositories retrievable by name.
The registry service serves any repository in the global scope.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-04-09 19:21:33 -07:00
Mary Anthony
cf9b4ab5e9 Breaking out README
Adding new material
Adding in template chomped in error
Cover install/deploy in README
Adding in Stephen's comments
Fixing you tabs!
Updating with commentary from pr
Updating with last minute comments

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-09 17:50:46 -07:00
Mary Anthony
636a19b212 Retooling to allow for docs build
Adding docs build to the Makefile
Adding in Sven's changes to the Makefile
Removing DS_store file
Updating per Stephen's comments
Update with Stephen's final comment

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-03 14:55:24 -07:00
Renamed from doc/configuration.md (Browse further)