Commit graph

1165 commits

Author SHA1 Message Date
Misty Stanley-Jones
70c7657c69 Update branding for macOS
Apple has changed their branding guidelines from 'OS X' to 'macOS'
so we should update ours to be within trademark / branding
guidelines. See http://www.apple.com/macos/sierra/

Signed-off-by: Misty Stanley-Jones <misty@docker.com>
2016-10-18 09:21:09 -07:00
Richard Scothern
b206e8b2a4 Add note about configuring a registry cache with delete enabled
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-10-14 14:24:14 -07:00
John Mulhausen
50bd0cce07 Delete api.md.tmpl
Deleting per comments in https://github.com/docker/distribution/pull/1985
2016-10-11 12:00:08 -07:00
Misty Stanley-Jones
677eaaa3cc Change 'draft: true' to 'published: false' for Jekyll 2016-10-10 16:19:47 -07:00
John Mulhausen
9204a64925 Rendering fixes, part 2 2016-10-03 16:05:09 -07:00
John Mulhausen
e8bd2f2c1c Merge pull request #21 from pborreli/typos
Fixed typos
2016-09-30 11:51:42 -07:00
John Mulhausen
856dacadfc Content rendering fixes 2016-09-30 01:51:56 -07:00
John Mulhausen
b3b099f079 Removing merge detritus 2016-09-29 23:38:12 -07:00
John Mulhausen
9e3c43c60c Removing empty front-matter 2016-09-29 22:39:30 -07:00
Pascal Borreli
2d32aa43eb Fixed typos 2016-09-30 01:52:12 +01:00
Misty Stanley-Jones
0fb207c822 Convert TOML to YAML, tweaks to work with Jekyll 2016-09-29 17:16:03 -07:00
Misty Stanley-Jones
1f6cc853d3 Initial import of https://github.com/docker/docker 2016-09-29 11:17:56 -07:00
Misty Stanley-Jones
dd41410647 Moved registry docs to registry subdirectory 2016-09-28 14:46:28 -07:00
Aaron Lehmann
1f3cc59124 Document TOOMANYREQUESTS error code
Add entries with this error code in registry/api/v2/descriptors.go.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-07-28 10:28:13 -07:00
Sebastiaan van Stijn
8d287d4332 Improve flag help consistency, and update docs
This adds the `--live-restore` option to the documentation.

Also synched usage description in the documentation
with the actual description, and re-phrased some
flag descriptions to be a bit more consistent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 64a8317a5a306dffd0ec080d9ec5b4ceb2479a01)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-25 23:15:24 -07:00
Derek McGowan
93f029e87c Allow v1 search to use v2 auth with identity token
Updates the v1 search endpoint to also support v2 auth when an identity token is given.
Only search v1 endpoint is supported since there is not v2 search currently defined to replace it.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit 19d48f0b8ba59eea9f2cac4ad1c7977712a6b7ac)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-25 23:15:10 -07:00
Cezar Sa Espinola
c8aba9b484 registry: avoid formatting errors with %#v
Signed-off-by: Cezar Sa Espinola <cezarsa@gmail.com>
2016-07-01 11:30:58 -03:00
Derek McGowan
b63146e35c Merge pull request #1808 from BrianBland/clientAllTagsFollowsLinks
Changes the client Tags All() method to follow links
2016-06-29 12:02:48 -07:00
Brian Bland
9e211edc9d Changes the client Tags All() method to follow links
This returns all tags even when the registry forces pagination.

Signed-off-by: Brian Bland <brian.t.bland@gmail.com>
2016-06-28 15:49:14 -07:00
Josh Chorlton
1c5cb12745 fixed s3 Delete bug due to read-after-delete inconsistency
Signed-off-by: Josh Chorlton <josh.chorlton@docker.com>
2016-06-28 14:22:15 -07:00
bin liu
6eadd3f4dc fix typos
Signed-off-by: bin liu <liubin0329@gmail.com>
2016-06-22 12:40:21 +08:00
Richard Scothern
d303d5c1ac Merge pull request #1739 from cezarsa/master
[Swift] Expose EndpointType parameter in driver
2016-06-15 10:33:48 -07:00
Richard Scothern
d2e5d5c22c If resumable digest support is disabled, detct this when closing the blobwriter
and allow the close to continue.  Also update the name of the function.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-06-13 17:35:06 -07:00
Richard Scothern
22e8510f3f Merge pull request #1706 from aibaars/registry-size-close
Blobwriter: call BlobWriter.Size after BlobWriter.Close
2016-06-13 16:29:35 -07:00
Cezar Sa Espinola
7b97265d95 Expose EndpointType parameter in swift storage driver
Signed-off-by: Cezar Sa Espinola <cezarsa@gmail.com>
2016-06-13 19:28:45 -03:00
Richard Scothern
70928bcf90 Merge pull request #1465 from dmcgowan/token-server-oauth
Integration token server supporting oauth
2016-06-13 15:01:06 -07:00
Richard Scothern
3b1b58e9c5 Merge pull request #1775 from dmcgowan/get-content-digest
Add option to get content digest from manifest get
2016-06-13 14:59:05 -07:00
Richard Scothern
05083bbef8 Merge pull request #1782 from tianon/accept-lists
Update "Accept" header parsing for list values
2016-06-13 14:33:46 -07:00
Aaron Lehmann
e484576b95 Merge pull request #1779 from dmcgowan/letsencrypt-support
Let's Encrypt support
2016-06-13 10:48:55 -10:00
Derek McGowan
9a27ea7323 Add support for Let's Encrypt
Add configuration and certificate manager to use letsencrypt

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-13 11:30:42 -07:00
Yong Tang
a58c74303c Fix logrus formatting
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.

This fix fixes #23459.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-06-11 13:16:55 -07:00
Tianon Gravi
5de53e3448 Update "Accept" header parsing for list values
In Go's header parsing, the same header multiple times results in multiple entries in the `r.Header[...]` slice, but Go does no further parsing beyond that (and in https://golang.org/cl/4528086 it was determined that until/unless the stdlib itself needs it, Go will not do so).

The consequence here for parsing of `Accept:` headers is that we support the way Go outputs headers, but not all language HTTP libraries have a facility to output multiple headers instead of a single list header.

This change ensures that the following (valid) header blocks all parse to the same result for the purposes of what is being tested here:

```
Accept: a/b
Accept: b/c
Accept: d/e
```

```
Accept: a/b; q=0.5, b/c
Accept: d/e
```

```
Accept: a/b; q=0.1, b/c; q=0.2, d/e; q=0.8
```

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2016-06-10 16:52:27 -07:00
Derek McGowan
f3ae941cca Add option to get content digest from manifest get
The client may need the content digest to delete a manifest using the digest used by the registry.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 17:02:29 -07:00
Richard Scothern
4746b30e31 Merge pull request #1774 from RichardScothern/catalog-clarifcation
Clarify API documentation around catalog fetch behavior
2016-06-08 12:59:09 -07:00
Richard Scothern
ec7c591381 Clarify API documentation around catalog fetch behavior
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-06-08 10:39:17 -07:00
Richard Scothern
4739eabc69 Merge pull request #1772 from runcom/go1.6const
registry: use const for status code 429
2016-06-08 10:22:57 -07:00
Antonio Murdaca
4e09e1b658 registry: use const for status code 429
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-08 10:19:15 +02:00
Yong Tang
08426ad10d Add --limit option to docker search
This fix tries to address the issue raised in #23055.
Currently `docker search` result caps at 25 and there is
no way to allow getting more results (if exist).

This fix adds the flag `--limit` so that it is possible
to return more results from the `docker search`.

Related documentation has been updated.

Additional tests have been added to cover the changes.

This fix fixes #23055.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-06-02 19:12:20 -07:00
allencloud
6d0db0e2dd fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-06-02 23:03:27 +08:00
allencloud
d265da7356 fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-06-02 17:17:22 +08:00
Richard Scothern
51d692a29a Merge pull request #1627 from luckyraul/swift_auth_url
Swift auth version param
2016-06-01 11:23:23 -07:00
Richard Scothern
b370cfc940 Merge pull request #1687 from RichardScothern/signature-store
Remove signature store from registry.
2016-05-31 09:09:39 -07:00
Richard Scothern
d3b61b612f Remove signature store from registry. Return a generated signature for manifest
pull.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-05-27 13:19:26 -07:00
Richard Scothern
97ca375a41 Merge pull request #1688 from tt/pass-through-known-errors
Pass through known errors
2016-05-27 11:35:34 -07:00
Richard Scothern
84cb77c871 Merge pull request #1744 from tonyhb/add-config-to-middleware
Pass in `app` as context to apply{N}Middleware
2016-05-25 14:08:09 -07:00
Tony Holdstock-Brown
f1b815ed9f Pass in app as context to apply{N}Middleware
This lets us access registry config within middleware for additional
configuration of whatever it is that you're overriding.

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-24 11:08:57 -07:00
Richard Scothern
242093047a Merge pull request #1734 from legionus/blob-access-controller
Add support for  blobAccessController middleware
2016-05-24 10:34:36 -07:00
Vincent Demeester
04476ff5a9 Add Unit test to daemon.SearchRegistryForImages…
… and refactor a little bit some daemon on the way.

- Move `SearchRegistryForImages` to a new file (`daemon/search.go`) as
  `daemon.go` is getting pretty big.
- `registry.Service` is now an interface (allowing us to decouple it a
  little bit and thus unit test easily).
- Add some unit test for `SearchRegistryForImages`.
- Use UniqueExactMatch for search filters
- And use empty restore id for now in client.ContainerStart.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-05-24 11:07:11 +02:00
John Starks
dd66aabeba Add support for layers from foreign sources
This will be used to support downloading Windows base layers from
Microsoft URLs.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-05-20 12:31:10 -07:00
Alexey Gladkov
50e6eef076 Add support for blobAccessController middleware
Signed-off-by: Michal Minar <miminar@redhat.com>
Signed-off-by: Alexey Gladkov <gladkov.alexey@gmail.com>
2016-05-19 14:02:15 +02:00
Richard Scothern
bb841197c2 Add 'us-gov-west-1' to the valid region list.
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-05-09 16:38:16 +01:00
Arthur Baars
e57fd4faa6 StorageDriver: GCS: allow Cancel on a closed FileWriter
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-05-06 13:04:30 +01:00
Arthur Baars
af00617b99 Blobwriter: call BlobWriter.Size after BlobWriter.Close
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-05-06 13:04:30 +01:00
Arthur Baars
ddec546466 StorageDriver: Test case for #1698
Signed-off-by: Arthur Baars <arthur@semmle.com>
2016-05-06 13:04:30 +01:00
Richard Scothern
c8592da977 Merge pull request #1695 from tonyhb/add-regulator-to-filesystem
Add regulator to filesystem
2016-05-04 10:05:51 -07:00
Tony Holdstock-Brown
1e05d81a71 Don't wrap thead limits when using a negative int
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 16:03:44 -07:00
Antonio Murdaca
db274d3c00 registry: do not use http.StatusTooManyRequests
go1.5 doesn't export http.StatusTooManyRequests while
go1.6 does. Fix this by hardcoding the status code for now.

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-03 21:24:45 +02:00
Antonio Murdaca
8762c800f1 registry: type too many requests error
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-03 18:41:54 +02:00
Tony Holdstock-Brown
cbae4dd7bf Implement regulator in filesystem driver
This commit refactors base.regulator into the 2.4 interfaces and adds a
filesystem configuration option `maxthreads` to configure the regulator.

By default `maxthreads` is set to 100. This means the FS driver is
limited to 100 concurrent blocking file operations. Any subsequent
operations will block in Go until previous filesystem operations
complete.

This ensures that the registry can never open thousands of simultaneous
threads from os filesystem operations.

Note that `maxthreads` can never be less than 25.

Add test case covering parsable string maxthreads

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 09:33:22 -07:00
Richard Scothern
03abc6ebae Merge pull request #1650 from majewsky/swift/wait-for-dlo-segments
[Swift] wait for DLO segments to show up when Close()ing the writer
2016-05-02 13:41:26 -07:00
Derek McGowan
32585845b7 Merge pull request #1669 from RichardScothern/close-after-commit
Clean uploads
2016-04-29 16:18:29 -07:00
Troels Thomsen
28be207bc0 Pass through known errors
Signed-off-by: Troels Thomsen <troels@thomsen.io>
2016-04-29 23:34:24 +02:00
Richard Scothern
c0dcad3a48 Merge pull request #1677 from RichardScothern/tonyhb-fix-s3-gc-error
Move GC into storage package and add tests
2016-04-28 14:09:58 -07:00
Richard Scothern
ea4f065672 Merge pull request #1665 from andrewhsu/middleware-redirect
add middleware storage driver for redirect
2016-04-27 15:05:52 -07:00
Richard Scothern
2a2577d7b1 When a blob upload is committed prevent writing out hashstate in the
subsequent close.

When a blob upload is cancelled close the blobwriter before removing
upload state to ensure old hashstates don't persist.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 14:42:00 -07:00
Richard Scothern
63d28d3b81 Add a test with a missing _manifests directory
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 13:34:25 -07:00
Richard Scothern
3a034b477e Move garbage collect code into storage package
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 13:34:25 -07:00
Tony Holdstock-Brown
898fdb48a1 Ensure GC continues marking if _manifests is nonexistent
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-27 13:33:36 -07:00
Serge Dubrouski
8775da93d6 Fix wording for dry-run flag in useage message for garbage collector.
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
2016-04-26 19:44:23 -06:00
Josh Hawn
a88088a59d Regulate filesystem driver to max of 100 calls
It's easily possible for a flood of requests to trigger thousands of
concurrent file accesses on the storage driver. Each file I/O call creates
a new OS thread that is not reaped by the Golang runtime. By limiting it
to only 100 at a time we can effectively bound the number of OS threads
in use by the storage driver.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-26 14:44:13 -07:00
Andrew Hsu
54edbdfee6 separate the go/non-go imports and reorder
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-26 14:33:54 -07:00
Richard Scothern
51972682c7 Merge pull request #1660 from jhaohai/cn-north-1-fix
Add cn-north-1 to valid check
2016-04-25 16:07:54 -07:00
Richard Scothern
f91dc3ca51 Merge pull request #1666 from sergeyfd/master
Add blobWrtiter.Close() call into blobWriter.Commit()
2016-04-25 16:02:48 -07:00
Brett Higgins
c4778ea1be Respect ALL_PROXY during registry operations
Use sockets.DialerFromEnvironment, as is done in other places,
to transparently support SOCKS proxy config from ALL_PROXY
environment variable.

Requires the *engine* have the ALL_PROXY env var set, which
doesn't seem ideal. Maybe it should be a CLI option somehow?

Only tested with push and a v2 registry so far. I'm happy to look
further into testing more broadly, but I wanted to get feedback on
the general idea first.

Signed-off-by: Brett Higgins <brhiggins@arbor.net>
2016-04-25 15:08:30 -04:00
Andrew Hsu
3336cc13e4 modify redirect test to include port
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:46 -07:00
Andrew Hsu
fba2e3a206 scheme and host mandatory in baseurl
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:25 -07:00
Andrew Hsu
cec7248bd1 separate the go/non-go imports and reorder
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:03 -07:00
Anis Elleuch
d11a979591 Sorting completed parts by part number for a better accordance with the S3 spec
Signed-off-by: Anis Elleuch <vadmeste@gmail.com>
2016-04-23 22:36:04 +01:00
Yong Tang
56480ce80a Add default serveraddress value in remote API /auth
This fix tries to address the issue in #22244 where the remote
API `/auth` will not set the default value of `serveraddress`
if not provided. This behavior happens after only in 1.11.0
and is a regression as in 1.10.3 `serveraddress` will be assigned
with `IndexServer` if no value is provided.

The default value `IndexServer` is assigned to `serveraddress` if
no value provided in this fix.

An integration test `TestAuthApi` has been added to cover this change

This fix fixes #22244.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-04-22 20:00:47 -07:00
Serge Dubrouski
6615b77a09 Add blobWrtiter.Close() call into blobWriter.Commit()
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
2016-04-22 19:23:17 -06:00
Andrew Hsu
a691d82aee add middleware storage driver for redirect
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-21 16:02:52 -07:00
jhaohai
fdb0fb77df add cn-north-1 to valid check
Signed-off-by: jhaohai <jhaohai@foxmail.com>
2016-04-21 11:51:34 +08:00
Stefan Majewsky
ea5abc9935 wait for DLO segments to show up when Close()ing the writer
Not just when Commit()ing the result. This fixes some errors I observed
when the layer (i.e. the DLO) is Stat()ed immediately after closing,
and reports the wrong file size because the container listing is not
yet up-to-date.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-04-19 14:31:49 +02:00
Nikita Tarasov
346bfed907 docs + fix test
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-17 20:05:51 +03:00
Nikita Tarasov
007af250b4 fix test
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>

Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-17 19:52:40 +03:00
Nikita
b55719daaa test
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-13 18:49:38 +03:00
Nikita
63fe2d1429 Update swift.go
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-13 18:49:25 +03:00
Richard Scothern
406d03bdb0 Merge pull request #1604 from ArdaXi/custom-s3-skip-region-check
Only check validity of S3 region if not using custom endpoint
2016-04-08 15:38:24 -07:00
Richard Scothern
ede3c5033e Merge pull request #1605 from majewsky/swift/deal-with-outdated-container-listings
registry/storage/swift: detect and fix outdated container listings
2016-04-08 15:38:06 -07:00
Stefan Majewsky
84aa48b56c detect outdated container listings during Stat() and getAllSegments()
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-04-06 15:21:27 +02:00
Arien Holthuizen
c655241209 Only check validity of S3 region if not using custom endpoint
Signed-off-by: Arien Holthuizen <aholthuizen@schubergphilis.com>
2016-04-06 13:38:09 +02:00
Tony Holdstock-Brown
86ca50dfe5 Ensure we log io.Copy errors and bytes copied/total in uploads
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-04 17:21:36 -07:00
Vincent Demeester
4430d42db9 Merge pull request #21607 from allencloud/change-validateNoSchema-to-validateNoScheme
change validateNoSchema to validateNoScheme
2016-03-31 10:14:23 +02:00
allencloud
53a8806b40 1.change validateNoSchema into validateNoScheme
2.change schema into scheme in docs and some annotations.

Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-03-31 12:29:15 +08:00
Richard Scothern
2a3d48fb82 Merge pull request #1578 from majewsky/do-not-swallow-error
don't swallow errors in Swift driver's GetContent()
2016-03-30 17:46:27 -07:00
Richard Scothern
3d4b652b58 Update the gc documentation.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-30 16:57:21 -07:00
Richard Scothern
31ece3d3b6 Fix signature handling with GC.
If a schema 1 manifest is uploaded with the `disablesignaturestore` option set
to true, then no signatures will exist.  Handle this case.

If a schema 1 manifest is pushed, deleted, garbage collected and pushed again, the
repository will contain signature links from the first version, but the blobs will
not exist.  Disable the signature store in the garbage-collect command so
signatures are not fetched.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-03-30 16:10:27 -07:00
Richard Scothern
15e3ffb3f2 Add a --dry-run flag. If enabled this will print the mark and sweep process
with removing any files.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-30 16:10:27 -07:00
Aaron Lehmann
59ef6d2d40 garbagecollect: Clean up errors
- Clean up error messages

- Add a missing error check on the result of blobService.Enumerate.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-30 11:35:24 -07:00
Stefan Majewsky
b015bf0676 don't swallow errors in Swift driver's GetContent()
In 000dec3c6f, which was only intended to
be a refactoring commit, the behavior of this block subtly changed so
that unknown types of errors would be swallowed instead of propagated.

I noticed this while investigating an error similar to #1539 aka
docker/docker#21290. It appears that during GetContent() for a
hashstate, the Swift proxy produces an error. Since this error was
silently swallowed, an empty []byte is used to restart the hash, then
producing the digest of the empty string instead of the layer's digest.

This PR will not fix the issue, but it should make the actual error more
visible by propagating it into `blobWriter#resumeDigest' and
'blobWriter#validateBlob', respectively.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-03-30 16:11:29 +02:00