Commit graph

124 commits

Author SHA1 Message Date
Olivier Gambier
81e8657d7b Fix here doc conf generation
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:05 -07:00
Olivier Gambier
8c1784c838 Keeping in synch
- commenting out both the "JSON" and "token" specs, unless someone thinks they should be here
- added help, glossary, introduction and authentication documents

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:05 -07:00
Olivier Gambier
96d79eb30e Adding authentication
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:05 -07:00
Olivier Gambier
c1c638d01f Minor keyword add
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
c405f3717a Minor fixes
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
56ff32c683 Revising glossary
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
10ba376a99 Minor cleanup in the index
- adding glossary
- removing empty "advanced"
- commenting out building and architecture for now
- minor text enhancements

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
c3b3802503 Small fixes
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
b7b8e64f1d Moved instructions up
Make it more obvious that environment variables is the way to go.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
26d87ed1a5 No reason to detail here
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
431e829fa5 Simplified index
Simplified index again, to make access to information more obvious and more direct.
Added a TLDR for people in a hurry.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
d9dfe54308 Separated "help"
Help page is separate now.
Removed the bulk of it from various pages.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
4eb8d907b9 Separate introduction
Use cases, generalities, image naming overview.
Removed most of it from index, and some of it from other random pages.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
2fb5c97a97 Deployment rewrote
Strong focus on simplification.
Entirely removed custom build instructions.
Providing sane, one-liner defaults.
Verified, easy-to-use TLS instructions.
Removed hybrid instructions.
Removed authentication with nginx instructions (either wait for native support, or move it to advanced topics).

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Vincent Batts
0165b85861 spec/manifest: clarify the v2-s1 is provisional
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-06-08 16:48:48 -04:00
Ahmet Alp Balkan
89f183639a docs: Update azure docs in configuration.md
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-05 17:25:17 -07:00
Stephen J Day
c54adb667f Clarify that manifests can only be deleted by digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-04 18:59:08 -07:00
Stephen J Day
93aff60741 Disable building of azure storage driver by default
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-03 18:14:53 -07:00
Olivier Gambier
00388a14aa Merge pull request from stevvooe/building-environment
Build environment requires proper checkout of project
2015-06-03 12:24:44 -07:00
Stephen J Day
4700c27eb5 Build environment requires proper checkout of project
This clarifies the importance of properly setting a Go build environment when
building targets. Typically, users seem to editorialize the checkout location,
either ignoring the first section or have limited experience with the Go
development environment. We clarify the checkout requirements and point to the
documentation on how to setup Go.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-03 11:59:56 -07:00
Vincent Giersch
7b3f571555 doc: coherence between requests and parameters + typo
In the request parameters lists `tag` was used instead of
`reference` present in the HTTP requests paths

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-06-02 14:36:55 +00:00
Stephen Day
0d40913b9a Merge pull request from xiekeyang/feature-panic-hook
Feature: Add Hook for Web Application Panic
2015-06-01 13:23:32 -07:00
W. Trevor King
e1c553fee9 docs/storagedrivers.md: Update filesystem.go -> driver.go reference
Catch up after 88795e0a (Lots of various golint fixes, 2014-11-17, )
and 65b0d73c (Move storagedriver package to registry/storage/driver,
2015-02-10, ).

Signed-off-by: W. Trevor King <wking@tremily.us>
2015-05-31 15:23:31 -07:00
xiekeyang
47aa47e3f6 Feature: Web Panic Reporting via hooks
This PR is for issue of "email after registry webapp panic" , improving my
previous design (closed).
It use self setting up hooks, to catch panic in web application.
And, send email in hooks handle directly, to no use new http server and
handler.

Signed-off-by: xiekeyang <keyangxie@126.com>
2015-05-31 14:21:22 +00:00
Stephen Day
318af0b1ce Merge pull request from gierschv/driver-rados
Storage Driver: Ceph Object Storage (RADOS)
2015-05-29 18:10:49 -07:00
Stephen Day
63d2e51529 Merge pull request from konstruktoid/patch-1
sha256 when generating certificates
2015-05-29 17:09:26 -07:00
Oilbeater
b78b395ae2 Fix typo
Signed-off-by: Mengxin Liu <liumengxinfly@gmail.com>
2015-05-26 21:19:49 +08:00
Thomas Sjögren
58981a4535 increase key size to 4096. @diogomonica
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
2015-05-24 22:02:25 +02:00
Olivier Gambier
8b537e0888 Revamp index
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-05-20 16:15:20 -07:00
Stephen J Day
dfa8504408 Revise description of debug endpoint
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-20 15:24:25 -07:00
Thomas Sjögren
e39583cc9d sha256 when generating certificates
Small detail, but when generating certificates using sha256 is recommended. See for example http://googleonlinesecurity.blogspot.se/2014/09/gradually-sunsetting-sha-1.html.

Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
2015-05-20 23:44:17 +02:00
Vincent Giersch
394eea0231 Storage Driver: Ceph Object Storage (RADOS)
This driver implements the storagedriver.StorageDriver interface and
uses Ceph Object Storage as storage backend.

Since RADOS is an object storage and no hierarchy notion, the
following convention is used to keep the filesystem notions stored in
this backend:

* All the objects data are stored with opaque UUID names prefixed
  (e.g. "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164).
* All the hierarchy information are stored in rados omaps, where the
  omap object identifier is the virtual directory name, the keys in
  a specific are the relative filenames and the values the blob
  object identifier (or empty value for a sub directory).

  e.g. For the following hierarchy:

     /directory1
     /directory1/object1
     /directory1/object2
     /directory1/directory2/object3

  The omap "/directory1" will contains the following key / values:
    - "object1" "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164"
    - "object2" "blob:db2e359d-4af0-4bfb-ba1d-d2fd029866a0"
    - "directory2" ""

  The omap "/directory1/directory2" will contains:
    - "object3" "blob:9ae2371c-81fc-4945-80ac-8bf7f566a5d9"

* The MOVE is implemented by changing the reference to a specific
  blob in its parent virtual directory omap.

This driver stripes rados objects to a fixed size (e.g. 4M). The idea
is to keep small objects (as done by RBD on the top of RADOS) that
will be easily synchronized accross OSDs. The information of the
original object (i.e total size of the chunks) is stored as a Xattr
in the first chunk object.

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-05-20 01:44:34 +00:00
Mary Anthony
e1a1e3a8da Fixes
Signed-off-by: Mary Anthony <mary@docker.com>
2015-05-18 13:33:29 -07:00
Mary Anthony
8d407c81ae Closes
Signed-off-by: Mary Anthony <mary@docker.com>

Tweak per Stephen

Signed-off-by: Mary Anthony <mary@docker.com>
2015-05-18 12:56:17 -07:00
Stephen J Day
593bbccdb5 Refactor Blob Service API
This PR refactors the blob service API to be oriented around blob descriptors.
Identified by digests, blobs become an abstract entity that can be read and
written using a descriptor as a handle. This allows blobs to take many forms,
such as a ReadSeekCloser or a simple byte buffer, allowing blob oriented
operations to better integrate with blob agnostic APIs (such as the `io`
package). The error definitions are now better organized to reflect conditions
that can only be seen when interacting with the blob API.

The main benefit of this is to separate the much smaller metadata from large
file storage. Many benefits also follow from this. Reading and writing has
been separated into discrete services. Backend implementation is also
simplified, by reducing the amount of metadata that needs to be picked up to
simply serve a read. This also improves cacheability.

"Opening" a blob simply consists of an access check (Stat) and a path
calculation. Caching is greatly simplified and we've made the mapping of
provisional to canonical hashes a first-class concept. BlobDescriptorService
and BlobProvider can be combined in different ways to achieve varying effects.

Recommend Review Approach
-------------------------

This is a very large patch. While apologies are in order, we are getting a
considerable amount of refactoring. Most changes follow from the changes to
the root package (distribution), so start there. From there, the main changes
are in storage. Looking at (*repository).Blobs will help to understand the how
the linkedBlobStore is wired. One can explore the internals within and also
branch out into understanding the changes to the caching layer. Following the
descriptions below will also help to guide you.

To reduce the chances for regressions, it was critical that major changes to
unit tests were avoided. Where possible, they are left untouched and where
not, the spirit is hopefully captured. Pay particular attention to where
behavior may have changed.

Storage
-------

The primary changes to the `storage` package, other than the interface
updates, were to merge the layerstore and blobstore. Blob access is now
layered even further. The first layer, blobStore, exposes a global
`BlobStatter` and `BlobProvider`. Operations here provide a fast path for most
read operations that don't take access control into account. The
`linkedBlobStore` layers on top of the `blobStore`, providing repository-
scoped blob link management in the backend. The `linkedBlobStore` implements
the full `BlobStore` suite, providing access-controlled, repository-local blob
writers. The abstraction between the two is slightly broken in that
`linkedBlobStore` is the only channel under which one can write into the global
blob store. The `linkedBlobStore` also provides flexibility in that it can act
over different link sets depending on configuration. This allows us to use the
same code for signature links, manifest links and blob links.  Eventually, we
will fully consolidate this storage.

The improved cache flow comes from the `linkedBlobStatter` component
of `linkedBlobStore`. Using a `cachedBlobStatter`, these combine together to
provide a simple cache hierarchy that should streamline access checks on read
and write operations, or at least provide a single path to optimize. The
metrics have been changed in a slightly incompatible way since the former
operations, Fetch and Exists, are no longer relevant.

The fileWriter and fileReader have been slightly modified to support the rest
of the changes. The most interesting is the removal of the `Stat` call from
`newFileReader`. This was the source of unnecessary round trips that were only
present to look up the size of the resulting reader. Now, one must simply pass
in the size, requiring the caller to decide whether or not the `Stat` call is
appropriate. In several cases, it turned out the caller already had the size
already. The `WriterAt` implementation has been removed from `fileWriter`,
since it is no longer required for `BlobWriter`, reducing the number of paths
which writes may take.

Cache
-----

Unfortunately, the `cache` package required a near full rewrite. It was pretty
mechanical in that the cache is oriented around the `BlobDescriptorService`
slightly modified to include the ability to set the values for individual
digests. While the implementation is oriented towards caching, it can act as a
primary store. Provisions are in place to have repository local metadata, in
addition to global metadata. Fallback is implemented as a part of the storage
package to maintain this flexibility.

One unfortunate side-effect is that caching is now repository-scoped, rather
than global. This should have little effect on performance but may increase
memory usage.

Handlers
--------

The `handlers` package has been updated to leverage the new API. For the most
part, the changes are superficial or mechanical based on the API changes. This
did expose a bug in the handling of provisional vs canonical digests that was
fixed in the unit tests.

Configuration
-------------

One user-facing change has been made to the configuration and is updated in
the associated documentation. The `layerinfo` cache parameter has been
deprecated by the `blobdescriptor` cache parameter. Both are equivalent and
configuration files should be backward compatible.

Notifications
-------------

Changes the `notification` package are simply to support the interface
changes.

Context
-------

A small change has been made to the tracing log-level. Traces have been moved
from "info" to "debug" level to reduce output when not needed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-15 17:05:18 -07:00
Anton Tiurin
ad80cbe1ea [Server] Listen and serve on a unix socket
Allow to use a unix socket as a listener.
To specify an endpoint type we use an optional configuration
field 'net', as there's no way to distinguish a relative
socket path from a hostname.

Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
2015-05-11 16:00:14 +03:00
Adam Enger
0180615a82 Fixing grammatical mistake in docs/deploying.md
Signed-off-by: Adam Enger <adamenger@gmail.com>
2015-05-08 11:27:57 -05:00
Stephen Day
fbd022e452 Merge pull request from dmcgowan/patch-support
Modify blob upload API
2015-05-06 16:12:54 -07:00
Derek McGowan
d9003dfc25 Add documentation for client version header
Add documentation for Docker-Distribution-API-Version header required by clients

closes 

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-06 15:58:48 -07:00
Derek McGowan
e842662ede Move pre-release 2.0 changes into the 2.0 API specification
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-06 15:58:48 -07:00
Derek McGowan
2cf40ab790 Modify blob upload API
- Ensures new uploads and resumed upload statuses always return an offset of 0. This allows future clients which support resumable upload to not attempt resumable upload on this version which does not support it.
- Add PATCH support for streaming data on upload.
- Add messaging to specification that PATCH with content range is currently not supported.
- Update PUT blob to only support full data or no data, no more last chunk messaging as it was not supported.

closes 

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-06 15:43:23 -07:00
Stephen J Day
aac8e0d0bf Remove configuration/README.md
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-06 15:31:12 -07:00
Derek McGowan
7d6e6aa980 Update API spec to reference digest instead of tarsum
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-01 17:14:38 -07:00
Mary Anthony
68c0682e00 Fixes Issue with Publish
- Add sed to Dockerfile; this sed exists on publish script; breaks headings/nav in files without metadata
- Ensure sed runs over storage-driver/ subdir
- Add metadata to all the files (including specs) that don't have it; this ensures they display correctly on publish
- Implement the fix for the showing up in Github
- Update template with GITHUB IGNORES

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-30 15:39:40 -07:00
Mary Anthony
7c93c8c265 Fixing headings
Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-28 15:10:52 -07:00
Stephen Day
1f071786d4 Merge pull request from kelseyhightower/document-osx-setup
docs: document running Registry natively on OS X
2015-04-28 10:36:33 -07:00
Mary Anthony
5043bdbc93 Updating env var documentation
The position was a bit too early. Fleshed out the example.
Also, using the _ underscore for emphasis was confusing because it is also used to indicate
a level change.

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-27 15:49:19 -07:00
Stephen Day
70427e3d9f Merge pull request from RichardScothern/purge-config
Add configuration for upload purging
2015-04-27 14:33:34 -07:00
Stephen Day
7ba2bdb7dc Merge pull request from RichardScothern/docs
Add registry configuration environment variable documentation
2015-04-27 13:41:11 -07:00
Mary Anthony
d5c0300986 Pushing fix for mkdocs.yml introduced
Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-27 11:27:49 -07:00
Richard
937c356585 Add environment variable override instructions
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 11:23:37 -07:00
Richard
6460ddb2cb Add configuration for upload purging
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 11:06:15 -07:00
Kelsey Hightower
cd83b75d4c docs: document running Registry natively on OS X
Signed-off-by: Kelsey Hightower <kelsey.hightower@gmail.com>
2015-04-26 18:41:36 -07:00
Ian Babrou
824e7e8ae2 docs: fixed links for storage drivers
Signed-off-by: Ian Babrou <ibobrik@gmail.com>
2015-04-24 21:57:09 +03:00
Mary Anthony
d588ed13b2 Fixes for registry top page
Fixes  to add building link to README
Docker Registry Service to Docker Registry

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-18 17:32:57 -07:00
Stephen Day
a9a9658e41 Merge pull request from mika/master
docs: drop newlines in URLs to avoid 404
2015-04-17 16:54:09 -07:00
Shawn Falkner-Horine
85cc6bc4bb Fix deployment guide typos re: image name consistency.
Signed-off-by: Shawn Falkner-Horine <dreadpirateshawn@gmail.com>
2015-04-17 21:13:24 +00:00
Michael Prokop
647e0a48b3 docs: drop newlines in URLs to avoid 404
The docs as available at
https://github.com/docker/distribution/blob/master/docs/configuration.md
result in 404 errors:

  https://github.com/docker/distribution/blob/master/cmd/registry/%0Aconfig.yml
  http://docs.aws.amazon.com/AWSSecurityCredentials/1.0/%0AAboutAWSCredentials.html#KeyPairs

instead of pointing to the correct ones, being:

  https://github.com/docker/distribution/blob/master/cmd/registry/config.yml
  http://docs.aws.amazon.com/AWSSecurityCredentials/1.0/AboutAWSCredentials.html#KeyPairs

So avoid the newlines in the corresponding source files.

Signed-off-by: Michael Prokop <mika@grml.org>
2015-04-17 23:09:43 +02:00
Spencer Rinehart
570fedb2ab Fix registry link to point to localhost in deploy doc.
Signed-off-by: Spencer Rinehart <anubis@overthemonkey.com>
2015-04-16 17:16:42 -05:00
Ben Firshman
5e4deba5a4 Use registry:2.0 tag in docker run
Signed-off-by: Ben Firshman <ben@firshman.co.uk>
2015-04-16 11:11:03 -07:00
Mary Anthony
b939d6d118 Updating with new RC work
Fixing typos and adding tables
Updating with testing material
Tweak after read through
Adding in Stephen's comments
Adding in Richard's comments. Fixing the broken images
closes issue 
Another try

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-16 10:44:30 -07:00
Jessie Frazelle
399b11cf29 Merge pull request from bfirsh/update-landing-page
docs: add landing page for Registry to docs
2015-04-16 10:15:54 -07:00
Ben Firshman
ec9b9d560f Add landing page for Registry to docs
- Explain why you would want to use a registry
- Explain difference between Docker Hub and Registry
- List some features of Registry
- Add table of contents for documentation - particularly "deploying a registry" call to action
- Use standard "Getting help" section from orchestration projects

Signed-off-by: Ben Firshman <ben@firshman.co.uk>
2015-04-16 10:01:36 -07:00
Stephen Day
e16c23bb43 Merge pull request from dmcgowan/upload-api-docs-update
Update final upload chunk api doc
2015-04-15 19:09:00 -07:00
Derek McGowan
21504560e1 Update final upload chunk api doc
Updates description about content length and location

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-04-15 17:55:15 -07:00
Daisuke Fujita
4cacbb4faf Modify markdown format in distribution.md
Signed-off-by: Daisuke Fujita <dtanshi45@gmail.com>
2015-04-16 09:22:24 +09:00
Kenneth Lim
f5a104dbcd server --> serve
Possible typo?
2015-04-13 22:11:48 +08:00
Stephen Day
aa752f68a1 Merge pull request from dmcgowan/nginx_v1_v2_deployment
docs: add nginx configuration for v1 and v2 registry
2015-04-10 16:35:50 -07:00
Stephen Day
ad76574d74 Merge pull request from RichardScothern/docs
doc/spec: documentation for the Image Manifest V2 format
2015-04-10 16:26:27 -07:00
Richard
e564ac59c3 Documentation for the Image Manifest V2 specification 2015-04-10 16:13:32 -07:00
Derek McGowan
6f087829c9 Add nginx configuration for v1 and v2 registry
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-04-10 15:16:13 -07:00
Derek McGowan
142d62798e Rename top level registry interface to namespace
Registry is intended to be used as a repository service than an abstract collection of repositories. Namespace better describes a collection of repositories retrievable by name.
The registry service serves any repository in the global scope.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-04-09 19:21:33 -07:00
Mary Anthony
cf9b4ab5e9 Breaking out README
Adding new material
Adding in template chomped in error
Cover install/deploy in README
Adding in Stephen's comments
Fixing you tabs!
Updating with commentary from pr
Updating with last minute comments

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-09 17:50:46 -07:00
Mary Anthony
636a19b212 Retooling to allow for docs build
Adding docs build to the Makefile
Adding in Sven's changes to the Makefile
Removing DS_store file
Updating per Stephen's comments
Update with Stephen's final comment

Signed-off-by: Mary Anthony <mary@docker.com>
2015-04-03 14:55:24 -07:00