distribution/registry/storage
Noah Treuhaft 63468ef4a8 Use multipart upload API in S3 Move method
This change to the S3 Move method uses S3's multipart upload API to copy
objects whose size exceeds a threshold.  Parts are copied concurrently.
The level of concurrency, part size, and threshold are all configurable
with reasonable defaults.

Using the multipart upload API has two benefits.

* The S3 Move method can now handle objects over 5 GB, fixing #886.

* Moving most objects, and espectially large ones, is faster.  For
  example, moving a 1 GB object averaged 30 seconds but now averages 10.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-08-16 10:53:24 -07:00
..
cache Add support for layers from foreign sources 2016-05-20 12:31:10 -07:00
driver Use multipart upload API in S3 Move method 2016-08-16 10:53:24 -07:00
blob_test.go Merge pull request #1706 from aibaars/registry-size-close 2016-06-13 16:29:35 -07:00
blobcachemetrics.go Open cache interface 2015-05-15 17:56:28 -07:00
blobserver.go Serve blobs when a storage driver supports redirects but are disabled 2015-12-28 11:08:39 -08:00
blobstore.go registry: avoid formatting errors with %#v 2016-07-01 11:30:58 -03:00
blobwriter.go If resumable digest support is disabled, detct this when closing the blobwriter 2016-06-13 17:35:06 -07:00
blobwriter_nonresumable.go If resumable digest support is disabled, detct this when closing the blobwriter 2016-06-13 17:35:06 -07:00
blobwriter_resumable.go Adds new storagedriver.FileWriter interface 2016-03-08 16:37:44 -08:00
catalog.go Improve catalog enumerate runtime by an order of magnitude 2016-08-09 17:42:26 -07:00
catalog_test.go Improve catalog enumerate runtime by an order of magnitude 2016-08-09 17:42:26 -07:00
doc.go Move storage package under registry package 2015-02-11 12:43:04 -08:00
filereader.go Adds new storagedriver.FileWriter interface 2016-03-08 16:37:44 -08:00
filereader_test.go testutil, storage: use math/rand.Read where possible 2016-08-10 14:26:12 -07:00
garbagecollect.go Remove signature store from registry. Return a generated signature for manifest 2016-05-27 13:19:26 -07:00
garbagecollect_test.go Add control over validation of URLs in pushed manifests 2016-07-19 14:38:42 -07:00
linkedblobstore.go Provide stat descriptor for Create method during cross-repo mount (#1857) 2016-08-01 19:07:11 -07:00
linkedblobstore_test.go Provide stat descriptor for Create method during cross-repo mount (#1857) 2016-08-01 19:07:11 -07:00
manifestlisthandler.go Decouple storage components by redefining dependencies as interfaces instead of concrete types 2016-07-20 15:09:11 -07:00
manifeststore.go fix typos 2016-06-22 12:40:21 +08:00
manifeststore_test.go Remove signature store from registry. Return a generated signature for manifest 2016-05-27 13:19:26 -07:00
paths.go Remove signature store from registry. Return a generated signature for manifest 2016-05-27 13:19:26 -07:00
paths_test.go Remove signature store from registry. Return a generated signature for manifest 2016-05-27 13:19:26 -07:00
purgeuploads.go Remove pathMapper object 2015-08-17 18:51:05 -07:00
purgeuploads_test.go Remove pathMapper object 2015-08-17 18:51:05 -07:00
registry.go Fix the build. 2016-07-21 17:16:47 -07:00
schema2manifesthandler.go Fix the build. 2016-07-21 17:16:47 -07:00
schema2manifesthandler_test.go Add control over validation of URLs in pushed manifests 2016-07-19 14:38:42 -07:00
signedmanifesthandler.go Decouple storage components by redefining dependencies as interfaces instead of concrete types 2016-07-20 15:09:11 -07:00
tagstore.go Rename Name method of Repository to Named 2016-02-04 09:55:36 -08:00
tagstore_test.go compare error output in tagstore unit test 2016-02-22 17:34:22 +08:00
util.go Refactor Blob Service API 2015-05-15 17:05:18 -07:00
vacuum.go Implements garbage collection subcommand 2016-02-29 14:15:21 -08:00
walk.go storage: enforce sorted traversal during Walk 2015-12-01 16:25:30 -08:00
walk_test.go Fix some typos in comments and strings 2016-02-23 22:33:38 +01:00