Commit graph

1943 commits

Author SHA1 Message Date
Richard Scothern
c047d34b22 Merge pull request #1695 from tonyhb/add-regulator-to-filesystem
Add regulator to filesystem
2016-05-04 10:05:51 -07:00
Richard Scothern
9ec0d742d6 Merge pull request #1696 from runcom/go1.6-lol
registry: do not use http.StatusTooManyRequests
2016-05-04 09:45:18 -07:00
Tony Holdstock-Brown
c9c62380ff Don't wrap thead limits when using a negative int
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 16:03:44 -07:00
Antonio Murdaca
f60f275c29 registry: do not use http.StatusTooManyRequests
go1.5 doesn't export http.StatusTooManyRequests while
go1.6 does. Fix this by hardcoding the status code for now.

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-03 21:24:45 +02:00
Aaron Lehmann
d4be7016ef Merge pull request #1693 from runcom/too-many-requests
registry: type too many requests error
2016-05-03 10:16:21 -07:00
Antonio Murdaca
a264f9ae29 registry: type too many requests error
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-03 18:41:54 +02:00
Tony Holdstock-Brown
d0352a7448 Add documentation
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 09:33:25 -07:00
Tony Holdstock-Brown
33c448f147 Implement regulator in filesystem driver
This commit refactors base.regulator into the 2.4 interfaces and adds a
filesystem configuration option `maxthreads` to configure the regulator.

By default `maxthreads` is set to 100. This means the FS driver is
limited to 100 concurrent blocking file operations. Any subsequent
operations will block in Go until previous filesystem operations
complete.

This ensures that the registry can never open thousands of simultaneous
threads from os filesystem operations.

Note that `maxthreads` can never be less than 25.

Add test case covering parsable string maxthreads

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 09:33:22 -07:00
Sven Dowideit
b7088d29c6 Merge pull request #1680 from SvenDowideit/use-docs-base-oss
convert docs Dockerfiles to use docs/base:oss
2016-05-03 10:48:11 +10:00
Richard Scothern
5d08dfa70c Merge pull request #1650 from majewsky/swift/wait-for-dlo-segments
[Swift] wait for DLO segments to show up when Close()ing the writer
2016-05-02 13:41:26 -07:00
Derek McGowan
a1d7463d67 Merge pull request #1669 from RichardScothern/close-after-commit
Clean uploads
2016-04-29 16:18:29 -07:00
Aaron Lehmann
024a9ed6c7 Merge pull request #1681 from aaronlehmann/preserve-authors-schema1
Preserve author information in schema1 manifests
2016-04-29 14:40:24 -07:00
Derek McGowan
72c9bfe1b5 Merge pull request #1683 from dmcgowan/update-golem-test-versions
Use official docker 1.11.1 image to test docker 1.11
2016-04-29 09:54:28 -07:00
Derek McGowan
8cded9db0d Use official docker 1.11.1 image to test docker 1.11
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-04-28 14:51:13 -07:00
Richard Scothern
ba927007b0 Merge pull request #1677 from RichardScothern/tonyhb-fix-s3-gc-error
Move GC into storage package and add tests
2016-04-28 14:09:58 -07:00
Aaron Lehmann
620a030217 Preserve author information in schema1 manifests
When we push a schema1 manifest, we encode history information from the
image JSON into v1Compatibility strings for the respective layers. The
"author" field was not being set in these v1Compatibility strings, so if
a parent layer had an author set, it would not be preserved after
pushing through a schema1 manifest and repulling, so the image ID would
change after the pull. This change preserves the authorship information
for parent layers so that the image ID does not change.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-04-28 11:57:48 -07:00
Sven Dowideit
e728c8bbed convert docs Dockerfiles to use docs/base:oss
Signed-off-by: Sven Dowideit <SvenDowideit@home.org.au>
2016-04-28 12:57:10 +00:00
Richard Scothern
a7dda2ce93 Merge pull request #1665 from andrewhsu/middleware-redirect
add middleware storage driver for redirect
2016-04-27 15:05:52 -07:00
Richard Scothern
d2e29acce0 When a blob upload is committed prevent writing out hashstate in the
subsequent close.

When a blob upload is cancelled close the blobwriter before removing
upload state to ensure old hashstates don't persist.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 14:42:00 -07:00
Richard Scothern
69ba30dc03 Add a test with a missing _manifests directory
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 13:34:25 -07:00
Richard Scothern
ea492aca1a Move garbage collect code into storage package
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-27 13:34:25 -07:00
Tony Holdstock-Brown
a5aaae1f06 Ensure GC continues marking if _manifests is nonexistent
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-27 13:33:36 -07:00
Richard Scothern
6a992e1348 Merge pull request #1675 from sergeyfd/master
Fix wording for dry-run flag in usage message for garbage collector.
2016-04-27 10:08:53 -07:00
Serge Dubrouski
32193bdcf0 Fix wording for dry-run flag in useage message for garbage collector.
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
2016-04-26 19:44:23 -06:00
Josh Hawn
e4dd3359cc Regulate filesystem driver to max of 100 calls
It's easily possible for a flood of requests to trigger thousands of
concurrent file accesses on the storage driver. Each file I/O call creates
a new OS thread that is not reaped by the Golang runtime. By limiting it
to only 100 at a time we can effectively bound the number of OS threads
in use by the storage driver.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-26 14:44:13 -07:00
Andrew Hsu
09a9b0cf90 separate the go/non-go imports and reorder
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-26 14:33:54 -07:00
Richard Scothern
47d14555c0 Merge pull request #1644 from fh1ch/clarify-kid-format
Clarify kid format for JWT token auth in docs
2016-04-25 17:01:15 -07:00
Richard Scothern
d654cfd985 Merge pull request #1623 from dmcgowan/docker-integration-readme
Integration test readme update
2016-04-25 16:10:19 -07:00
Richard Scothern
c83afea0c9 Merge pull request #1660 from jhaohai/cn-north-1-fix
Add cn-north-1 to valid check
2016-04-25 16:07:54 -07:00
Richard Scothern
ef32134592 Merge pull request #1666 from sergeyfd/master
Add blobWrtiter.Close() call into blobWriter.Commit()
2016-04-25 16:02:48 -07:00
Richard Scothern
96f796fb01 Merge pull request #1670 from vadmeste/fix_parts_sorting
s3 driver: Sorting completed parts by part number for a better accordance with S3 spec
2016-04-25 16:02:17 -07:00
Andrew Hsu
c4df027d41 modify redirect test to include port
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:46 -07:00
Andrew Hsu
fe9509f8f3 added config doc for redirect middleware
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:39 -07:00
Andrew Hsu
80248c3d3a scheme and host mandatory in baseurl
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:25 -07:00
Andrew Hsu
059bc5f5ef separate the go/non-go imports and reorder
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-25 11:52:03 -07:00
Anis Elleuch
987faca8a6 Sorting completed parts by part number for a better accordance with the S3 spec
Signed-off-by: Anis Elleuch <vadmeste@gmail.com>
2016-04-23 22:36:04 +01:00
Serge Dubrouski
21f38a74e6 Add blobWrtiter.Close() call into blobWriter.Commit()
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
2016-04-22 19:23:17 -06:00
Andrew Hsu
4b217ccbf5 add middleware storage driver for redirect
Signed-off-by: Andrew Hsu <andrewhsu@acm.org> (github: andrewhsu)
2016-04-21 16:02:52 -07:00
jhaohai
f76c622d8c add cn-north-1 to valid check
Signed-off-by: jhaohai <jhaohai@foxmail.com>
2016-04-21 11:51:34 +08:00
Richard Scothern
cd27f179f2 Merge pull request #1635 from hopkings2008/notifytype
use context.GetLogger to replace logrus in listener
2016-04-19 10:04:23 -07:00
Stefan Majewsky
9a67520af7 wait for DLO segments to show up when Close()ing the writer
Not just when Commit()ing the result. This fixes some errors I observed
when the layer (i.e. the DLO) is Stat()ed immediately after closing,
and reports the wrong file size because the container listing is not
yet up-to-date.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-04-19 14:31:49 +02:00
Fabio Huser
17756eb43e Clarify kid format for JWT token auth in docs
The kid value can have an arbitrary format according JOSE specification, but Docker distribution expects a specific format (libtrust fingerprint) to work. This is not written in the documentation so far and is only mentioned in the libtrust source code itself.

Signed-off-by: Fabio Huser <fabio@fh1.ch>
2016-04-17 12:04:15 +02:00
Richard Scothern
9d491698cc Merge pull request #1641 from RichardScothern/ifollowtherules
Correction for JSON example.
2016-04-15 09:52:20 -07:00
Richard Scothern
b72d74464a Correction for JSON example.
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-04-15 09:22:44 -07:00
yuzou
f2686b8db4 use context.GetLogger to replace logrus in listener
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-04-15 11:18:26 +08:00
Richard Scothern
05b0ab0302 Merge pull request #1630 from hopkings2008/notifytype
fix typepo for log message of layer push event in blobServiceListener…
2016-04-14 10:39:58 -07:00
yuzou
098005177f fix typepo for log message of layer push event in blobServiceListener Put function.
Signed-off-by: yuzou <zouyu7@huawei.com>
2016-04-14 16:41:35 +08:00
Richard Scothern
e90ff92895 Merge pull request #1625 from moxiegirl/fix-1598
Updated per conversation with Richard
2016-04-13 09:46:43 -07:00
Mary Anthony
6bce49d51d Updated per conversation with Richard
Removing draft
Richard's comments and some fixes

Signed-off-by: Mary Anthony <mary@docker.com>
2016-04-13 06:43:11 -07:00
Derek McGowan
ab2394446c Integration test readme update
Updates the readme to mention running the tests using golem.
Also provides instructions for making test development easier.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-04-12 14:03:56 -07:00