go1.5 doesn't export http.StatusTooManyRequests while
go1.6 does. Fix this by hardcoding the status code for now.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
This commit refactors base.regulator into the 2.4 interfaces and adds a
filesystem configuration option `maxthreads` to configure the regulator.
By default `maxthreads` is set to 100. This means the FS driver is
limited to 100 concurrent blocking file operations. Any subsequent
operations will block in Go until previous filesystem operations
complete.
This ensures that the registry can never open thousands of simultaneous
threads from os filesystem operations.
Note that `maxthreads` can never be less than 25.
Add test case covering parsable string maxthreads
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
subsequent close.
When a blob upload is cancelled close the blobwriter before removing
upload state to ensure old hashstates don't persist.
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
It's easily possible for a flood of requests to trigger thousands of
concurrent file accesses on the storage driver. Each file I/O call creates
a new OS thread that is not reaped by the Golang runtime. By limiting it
to only 100 at a time we can effectively bound the number of OS threads
in use by the storage driver.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
Use sockets.DialerFromEnvironment, as is done in other places,
to transparently support SOCKS proxy config from ALL_PROXY
environment variable.
Requires the *engine* have the ALL_PROXY env var set, which
doesn't seem ideal. Maybe it should be a CLI option somehow?
Only tested with push and a v2 registry so far. I'm happy to look
further into testing more broadly, but I wanted to get feedback on
the general idea first.
Signed-off-by: Brett Higgins <brhiggins@arbor.net>
This fix tries to address the issue in #22244 where the remote
API `/auth` will not set the default value of `serveraddress`
if not provided. This behavior happens after only in 1.11.0
and is a regression as in 1.10.3 `serveraddress` will be assigned
with `IndexServer` if no value is provided.
The default value `IndexServer` is assigned to `serveraddress` if
no value provided in this fix.
An integration test `TestAuthApi` has been added to cover this change
This fix fixes#22244.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Not just when Commit()ing the result. This fixes some errors I observed
when the layer (i.e. the DLO) is Stat()ed immediately after closing,
and reports the wrong file size because the container listing is not
yet up-to-date.
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
If a schema 1 manifest is uploaded with the `disablesignaturestore` option set
to true, then no signatures will exist. Handle this case.
If a schema 1 manifest is pushed, deleted, garbage collected and pushed again, the
repository will contain signature links from the first version, but the blobs will
not exist. Disable the signature store in the garbage-collect command so
signatures are not fetched.
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
In 000dec3c6f, which was only intended to
be a refactoring commit, the behavior of this block subtly changed so
that unknown types of errors would be swallowed instead of propagated.
I noticed this while investigating an error similar to #1539 aka
docker/docker#21290. It appears that during GetContent() for a
hashstate, the Swift proxy produces an error. Since this error was
silently swallowed, an empty []byte is used to restart the hash, then
producing the digest of the empty string instead of the layer's digest.
This PR will not fix the issue, but it should make the actual error more
visible by propagating it into `blobWriter#resumeDigest' and
'blobWriter#validateBlob', respectively.
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
This commit adds context-specific documentation on StorageDriver,
StorageDriverFactory, and the factory’s Register func, explaining how
the internal registration mechanism should be used.
This documentation follows from the thread starting at
https://github.com/deis/builder/pull/262/files#r56720200.
cc/ @stevvooe
Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
If this slice ends up empty after parsing the HTTP response body, it
means the body is not well-formed. We've probably encountered an error
message produced by something that uses a different JSON schema, or
an error that just happens to validate as JSON.
An empty errcode.Errors slice is not a very useful thing to return,
since its Error() output is just `<nil>`. Detect this case, and instend
return an UnexpectedHTTPResponseError.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Going forward, Docker won't use a different default registry on Windows.
This changes Windows to use the standard Docker Hub registry as the
default registry.
There is a plan in place to migrate existing images from the Windows
registry to Hub's normal registry, in advance of the 1.11 release. In
the mean time, images on the Windows registry can be accessed by
prefixing them with `registry-win-tp3.docker.io/`.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Use token handler options for initialization.
Update auth endpoint to set identity token in response.
Update credential store to match distribution interface changes.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Updates registry storage code to use this for better resumable writes.
Implements this interface for the following drivers:
+ Inmemory
+ Filesystem
+ S3
+ Azure
Signed-off-by: Brian Bland <brian.bland@docker.com>
It is possible for a middlebox to lowercase the URL at somepoint causing a
lookup in the auth challenges table to fail. Lowercase hostname before
using as keys to challenge map.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Cross repository push tokens were not being cached and could not be used,
now any returned token will be used and the caching is hidden in the getToken function.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Login needs to add an offline token flag to ensure a refresh token is returned by the token endpoint.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The Move operation is only used to move uploaded blobs
to their final destination. There is no point in implementing
Move on "folders". Apart from simplifying the code, this also
saves an HTTP request.
Signed-off-by: Arthur Baars <arthur@semmle.com>
Further differentiate the APIEndpoint used with V2 with the endpoint type which is only used for v1 registry interactions
Rename Endpoint to V1Endpoint and remove version ambiguity
Use distribution token handler for login
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
