Commit graph

31 commits

Author SHA1 Message Date
Stephen Day
dd58ffe6ed Merge pull request #410 from RichardScothern/driver-context
Add golang/x/net/context.Context to storage driver method calls
2015-05-04 19:18:30 -07:00
Stephen J Day
69e477f2d5 Use done channel to avoid goroutine leak
This deals with a memory leak, caused by goroutines, experienced when using the
s3 driver. Unfortunately, this section of the code leaks goroutines like a
sieve. There is probably some refactoring that could be done to avoid this but
instead, we have a done channel that will cause waiting goroutines to exit.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-28 14:40:20 -07:00
Richard
ae216e365a Make Storage Driver API calls context aware.
- Change driver interface to take a context as its first argument
     - Make newFileReader take a context as its first argument
     - Make newFileWriter take a context as its first argument
     - Make blobstore exists and delete take a context as a first argument
     - Pass the layerreader's context to the storage layer
     - Pass the app's context to purgeuploads
     - Store the app's context into the blobstore (was previously null)
     - Pass the trace'd context to the storage drivers

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 15:58:58 -07:00
Olivier Gambier
9b6d49fdb0 Merge pull request #430 from stevvooe/address-s3-timeout-error
Attempt to address intermittent s3 RequestTimeout error
2015-04-24 11:20:30 -07:00
Stephen Day
528869ac03 Merge pull request #420 from stevvooe/expose-drivernames
Expose storage driver names for tracing
2015-04-24 10:53:40 -07:00
Stephen J Day
e6715c5cec Attempt to address intermittent s3 RequestTimeout error
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-23 20:07:32 -07:00
Stephen J Day
6dd27cd2cb Correctly check s3 chunksize parameter
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-23 16:31:41 -07:00
Stephen J Day
d7eb17fd72 Include driver name in trace messsages
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 17:30:31 -07:00
Stephen J Day
9e146437e4 Require storage drivers to report their name
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 17:30:01 -07:00
Stephen Day
73960f4024 Merge pull request #419 from stevvooe/pool-buffers-s3
Pool buffers used in S3.WriteStream
2015-04-22 16:41:28 -07:00
Stephen J Day
1adab029f6 Pool buffers used in S3.WriteStream
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 16:25:53 -07:00
Stephen J Day
72a702d0d9 Allow configuration of chunksize parameter
The code using values from the yaml package wasn't careful enought with the
possible incoming types. Turns out, it is just an int but we've made this
section somewhat bulletproof in case that package changes the behavior.

This code likely never worked. The configuration system should be decoupled
from the object instantiation.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 14:31:34 -07:00
Stephen Day
7c25fcf698 Merge pull request #375 from RichardScothern/path-logging
registry/storage/driver: add path and other info to filesytem trace methods.
2015-04-17 14:09:23 -07:00
Richard
6b1c2e9b1f Add path and other info to filesytem trace methods.
Also fix Delete (was 'Move').
2015-04-17 13:55:18 -07:00
bin liu
e0521d2d01 fix some typos in source comments
Signed-off-by: bin liu <liubin0329@gmail.com>
2015-04-17 12:39:52 +00:00
Stephen J Day
730ce83cdf Trace function calls to Base storage driver
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-10 16:42:36 -07:00
Stephen J Day
e23ca5ac5f Defer case-sensitive support to storage backend
Rather than enforce lowercase paths for all drivers, support for
case-sensitivity has been deferred to the driver. There are a few caveats to
this approach:

1. There are possible security implications for tags that only differ in their
case. For instance, a tag "A" may be equivalent to tag "a" on certain file
system backends.
2. All system paths should not use case-sensitive identifiers where possible.
This might be problematic in a blob store that uses case-sensitive ids. For
now, since digest hex ids are all case-insensitive, this will not be an issue.

The recommend workaround is to not run the registry on a case-insensitive
filesystem driver in security sensitive applications.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-07 14:14:45 -07:00
Josh Hawn
0eb599329a Handle cloudFront bucket prefix issue
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-06 19:46:05 -07:00
Stephen J Day
f26a283a48 Avoid crash on invalid Move arguments
This chnage prevents a crash when moving from a non-existent directory that has
a file as a parent. To prevent this, we simply check that the node is a
directory and throws an error if it is not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 19:03:02 -07:00
Stephen Day
02b8c7b8cf Merge pull request #294 from fkautz/pr_out_updating_msopentech_azure_sdk_for_go_to_latest_master
Updating MSOpenTech/azure-sdk-for-go to latest master
2015-03-24 13:27:19 -07:00
Frederick F. Kautz IV
11db8185bc Updating MSOpenTech/azure-sdk-for-go to latest master
Signed-off-by: Frederick F. Kautz IV <fkautz@alumni.cmu.edu>
2015-03-23 21:59:21 -07:00
Ahmet Alp Balkan
ce9f500a31 storage/driver/azure: Allow non-default realms
This enables Azure storage driver to be used with non-default
cloud endpoints like Azure China or Azure Government that does
not use `.blob.core.windows.net` FQDN suffix.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-03-23 20:41:16 -07:00
Josh Hawn
db5689aa86 Refactor Layer interface to return a Handler
... Rather than ServeHTTP directly.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-03-12 21:59:07 -07:00
Andy Goldstein
b5a63d75ea Fix Godoc typos
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2015-03-11 08:56:29 -07:00
Andy Goldstein
a20abfbf3c Middleware!
Convert middleware in the config to be a map of type->[]Middleware

Add support for registry & repository middleware.

Some naming updates as well.

Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2015-03-11 08:56:28 -07:00
David Lawrence
4acda57e05 Refactoring cloudfactory layer handler into a more generic storage
middleware concept.

This also breaks the dependency the storage package had on goamz
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2015-03-11 08:56:28 -07:00
Stephen J Day
41b1e22f97 doc: move storage driver readmes into docs
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-03-05 17:23:33 -08:00
Andrey Kostov
df71f3451a Fix S3 driver's list when the root directory is either "" or "/" 2015-02-19 16:31:34 -08:00
Andrey Kostov
da5510b55e Add an empty root directory s3 driver specific test 2015-02-19 16:28:32 -08:00
Ahmet Alp Balkan
76f1abfa07 storagedriver/azure: Add README
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-02-11 14:40:51 -08:00
Stephen J Day
65b0d73cb7 Move storagedriver package to registry/storage/driver
This change is slightly more complex than previous package maves in that the
package name changed. To address this, we simply always reference the package
driver as storagedriver to avoid compatbility issues with existing code. While
unfortunate, this can be cleaned up over time.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-11 12:43:04 -08:00