Commit Graph

746 Commits (b89a5943550d4adc08ef3e518fb64cf74108083d)

Author SHA1 Message Date
Derek McGowan d35d94dcec
Update to fix lint errors
Context should use type values instead of strings.
Updated direct calls to WithValue, but still other uses of string keys.
Update Acl to ACL in s3 driver.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-10-05 17:47:12 -07:00
Richard Scothern 431cfa3179 Merge pull request #1966 from nwt/fix-redis-tests
Fix connection pool exhaustion in Redis tests
2016-09-19 16:27:56 -07:00
Richard Scothern 8cb75f692f Merge pull request #1868 from stanhu/fix-missing-credentials-with-default-port
Fix missing auth headers with PATCH HTTP request when pushing to default port
2016-09-19 15:55:12 -07:00
Noah Treuhaft cafeae4ecd Fix connection pool exhaustion in Redis tests
The Redis tests were failing with a "connection pool exhausted" error
from Redigo. Closing the connection used for FLUSHDB fixes the problem.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-09-19 15:30:32 -07:00
Stan Hu 6d952c92cf Rename CanonicalAddr -> canonicalAddr
Update comment for hasPort

Signed-off-by: Stan Hu <stanhu@gmail.com>
2016-09-19 14:16:02 -07:00
Richard Scothern cfad4321c1 Merge pull request #1957 from nwt/notification-filtering
Add notification filtering by target media type
2016-09-19 10:41:03 -07:00
Noah Treuhaft ad6bb66faf Add notification filtering by target media type
The Hub registry generates a large volume of notifications, many of
which are uninteresting based on target media type.  Discarding them
within the notification endpoint consumes considerable resources that
could be saved by discarding them within the registry.  To that end,
this change adds registry configuration options to restrict the
notifications sent to an endpoint based on target media type.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-09-16 12:01:03 -07:00
Richard Scothern b6e0cfbdaa Merge pull request #1959 from nwt/disable-access-logging
Add configuration option to disable access logging
2016-09-15 09:45:56 -07:00
Noah Treuhaft 4034ff65f0 Add configuration option to disable access logging
Access logging is great.  Access logging you can turn off is even
better.  This change adds a configuration option for that.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-09-14 14:33:30 -07:00
Richard Scothern 835b04c004 Merge pull request #1958 from spacexnice/master
remoteURL parameter error cause  challenge cache not working which eventually lead to client manifest request timeout
2016-09-14 12:33:18 -07:00
Derek McGowan 17fb0bb6b3 Merge pull request #1934 from jheiss/token_ssl_pem_type
Check PEM block type when reading token cert file
2016-09-13 09:45:06 -07:00
spacexnice 3c4b551a7c remoteURL parameter error cause challenge cache not working,critical area consume to much time, which eventually lead to client manifest request timeout(90s) in massive(which i test 80 concurrent) concurrent manifest request situation.
Signed-off-by: spacexnice <yaoyao.xyy@alibaba-inc.com>
2016-09-13 09:58:12 +08:00
Stephen Day 5e8d18f615 Merge pull request #1945 from stevvooe/better-report-on-invalid-secret
handlers: provide better log message on mismatched secret
2016-09-07 12:55:48 -07:00
Noah Treuhaft 91f268e5a5 Downgrade token auth JWT logging from error to info
The token auth package logs JWT validation and verification failures at
the `error` level.  But from the server's perspective, these aren't
errors.  They're the expected response to bad input.  Logging them at
the `info` level better reflects that distinction.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-09-07 10:45:06 -07:00
Stephen J Day 668b0a5f40
handlers: provide better log message on mismatched secret
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-09-02 15:24:35 -07:00
Stan Hu 462bb55c3f Fix missing auth headers with PATCH HTTP request when pushing to default port
If a user specifies `mydomain.com:443` in the `Host` configuration, the
PATCH request for the layer upload will fail because the challenge does not
appear to be in the map. To fix this, we normalize the map keys to always
use the Host:Port combination.

Closes https://github.com/docker/docker/issues/18469

Signed-off-by: Stan Hu <stanhu@gmail.com>
2016-09-01 14:46:31 -07:00
Fabio Berchtold 7dcac52f18 Add v2 signature signing to S3 storage driver (#1800)
* Add v2 signature signing to S3 storage driver

Closes #1796
Closes #1606

Signed-off-by: Fabio Berchtold <fabio.berchtold@swisscom.com>

* use Logrus for debug logging

Signed-off-by: Fabio Berchtold <fabio.berchtold@swisscom.com>

* use 'date' instead of 'x-amz-date' in request header

Signed-off-by: Fabio Berchtold <fabio.berchtold@swisscom.com>

* only allow v4 signature signing against AWS S3

Signed-off-by: Fabio Berchtold <fabio.berchtold@swisscom.com>
2016-09-01 13:52:40 -07:00
Jason Heiss d04481e388 Check PEM block type when reading token cert file
closes #1909

Signed-off-by: Jason Heiss <jheiss@twosigma.com>
2016-09-01 16:48:55 -04:00
Richard Scothern 49da29ee46 Merge pull request #1925 from dmcgowan/reenable-race-detector
Re-enable race detector in circleci
2016-09-01 13:36:12 -07:00
Derek McGowan 41f383fb9a Merge pull request #1928 from ghostplant/master
Dynamically Parsing the Latest HTPassword File
2016-08-30 10:29:35 -07:00
Derek McGowan 205e606a4c Update proxy scheduler test to account for race
Running with the race detector may cause some parts
of the code to run slower causing a race in the scheduler
ordering.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-29 14:27:32 -07:00
Derek McGowan b3e276ff93 Add hash map locking to proxy tests
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-29 13:39:24 -07:00
Derek McGowan a50ce1ab93 Add locking to repository access in memory cache
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-26 18:10:42 -07:00
Derek McGowan dabdc5e52b Fix access race in proxy scheduler
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-26 17:40:21 -07:00
Derek McGowan b1b100cf01 Merge pull request #1923 from spacexnice/master
fix simpleAuthChallge concurrent problem
2016-08-26 17:31:09 -07:00
Derek McGowan 798b7331ca Add shutdown to test environment
Ensures test http server is fully closed before moving on with test.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-26 16:23:07 -07:00
Derek McGowan 38ee36eee4 Disable upload purging during test
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-08-26 16:23:07 -07:00
a-palchikov 82609180a1 tag service: properly handle error responses on HEAD requests by (#1918)
* tag service: properly handle error responses on HEAD requests by
re-issuing requests as GET for proper error details.

Fixes #1911.

Signed-off-by: dmitri <deemok@gmail.com>

* Simplify handling of failing HEAD requests in TagService and
make a GET request for cases:
  - if the server does not handle HEAD
  - if the response was an error to get error details

Signed-off-by: dmitri <deemok@gmail.com>

* Add a missing http.Response.Body.Close call for the GET request.

Signed-off-by: dmitri <deemok@gmail.com>
2016-08-26 14:35:04 -07:00
cuiwei13 169ff1d098 Dynamically Parsing the Latest HTPassword File
To parse the latest account list dynamically instead of restarting the distribution service frequently.

Signed-off-by: CUI Wei <ghostplant@qq.com>
2016-08-27 04:28:33 +08:00
spacexnice 68243306fb correct onerror behavior
Signed-off-by: spacexnice <yaoyao.xyy@alibaba-inc.com>
2016-08-26 09:41:54 +08:00
spacexnice 22cf89c02b gofmt -s do code format
Signed-off-by: spacexnice <yaoyao.xyy@alibaba-inc.com>
2016-08-25 07:34:36 +08:00
Matthew Green dea554fc7c Swift driver now bulk deletes in chunks specified by the server (#1915)
Swift driver now bulk deletes in chunks specified by the server

Signed-off-by: Matthew Green <matthew.green@uk.ibm.com>
2016-08-24 10:09:25 -07:00
spacexnice ca2b78dc0a adjust for loop
Signed-off-by: spacexnice <yaoyao.xyy@alibaba-inc.com>
2016-08-24 09:31:27 +08:00
spacexnice 44b8157572 fix simpleAuthChallge concurrent problem
Signed-off-by: spacexnice <yaoyao.xyy@alibaba-inc.com>
2016-08-23 22:09:18 +08:00
Richard Scothern 010e063270 Merge pull request #1906 from nwt/s3-multipart-copy
Use multipart upload API in S3 Move method
2016-08-17 15:22:32 -07:00
Richard Scothern c4297ef9da Merge pull request #1839 from adamvduke/adamvduke/allow-http2-registry-clients
Allow registry clients to connect via http2
2016-08-16 11:48:06 -07:00
Noah Treuhaft 63468ef4a8 Use multipart upload API in S3 Move method
This change to the S3 Move method uses S3's multipart upload API to copy
objects whose size exceeds a threshold.  Parts are copied concurrently.
The level of concurrency, part size, and threshold are all configurable
with reasonable defaults.

Using the multipart upload API has two benefits.

* The S3 Move method can now handle objects over 5 GB, fixing #886.

* Moving most objects, and espectially large ones, is faster.  For
  example, moving a 1 GB object averaged 30 seconds but now averages 10.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-08-16 10:53:24 -07:00
Stefan Majewsky a7c6bfd59f [swift] support different user-domain and tenant-domain
This is already supported by ncw/swift, so we just need to pass the
parameters from the storage driver.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-08-15 11:21:42 +02:00
Adam Duke ac009c86f1 Allow registry clients to connect via http2
Http2 will be enabled by default and can be disabled with a configuration option.

Signed-off-by: Adam Duke <adam.v.duke@gmail.com>
2016-08-13 22:07:42 -04:00
Richard Scothern c9fd26e9ef Merge pull request #1895 from hinshun/improve-enumerate
Improve catalog enumerate runtime by an order of magnitude
2016-08-11 10:53:36 -07:00
Richard Scothern baca174469 Merge pull request #1892 from stevvooe/use-math-rand-reader
testutil, storage: use math/rand.Read where possible
2016-08-11 10:04:36 -07:00
Stephen J Day 040db51795
testutil, storage: use math/rand.Read where possible
Use the much faster math/rand.Read function where cryptographic
guarantees are not required. The unit test suite should speed up a
little bit but we've already optimized around this, so it may not
matter.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-10 14:26:12 -07:00
Edgar Lee 2f81b3b058 Improve catalog enumerate runtime by an order of magnitude
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-08-09 17:42:26 -07:00
Richard Scothern 9000745401 Merge pull request #1888 from stevvooe/more-efficient-path-comparison
registry/storage: more efficient path compare in catalog
2016-08-09 13:45:07 -07:00
Stephen J Day 308faf00f1
catalog: add benchmarks for overridden path comparison
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-08 17:09:49 -07:00
Stephen J Day bba5a0d05c
registry/storage: more efficient path compare in catalog
Previous component-wise path comparison is recursive and generates a
large amount of garbage. This more efficient version simply replaces the
path comparison with the zero-value to sort before everything. We do
this by replacing the byte-wise comparison that swaps a single character
inline for the separator comparison, such that separators sort first.

The resulting implementation provides component-wise path comparison
with no cost incurred for allocation or stack frame.

Direction of the comparison is also reversed to match Go style.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-08 15:55:26 -07:00
Edgar Lee 379312c148 Stop ErrFinishedWalk from escaping from Repositories walk
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-08-05 17:21:48 -07:00
Richard Scothern bfa0a9c097 Merge pull request #1877 from spacexnice/master
function copyContent() cause socket leak.
2016-08-02 09:15:23 -07:00
yaoyao.xyy 64c8dd3252 when deploy registry as a pull through cache ,function copeContent() may cause a socket leak when docker user canceled its pull operation.
Signed-off-by: yaoyao.xyy <yaoyao.xyy@alibaba-inc.com>
2016-08-02 11:10:10 +08:00
Michal Minář 7365003236 Provide stat descriptor for Create method during cross-repo mount (#1857)
* Allow precomputed stats on cross-mounted blobs

Signed-off-by: Michal Minář <miminar@redhat.com>

* Extended cross-repo mount tests

Signed-off-by: Michal Minář <miminar@redhat.com>
2016-08-01 19:07:11 -07:00