Commit graph

12 commits

Author SHA1 Message Date
Tony Holdstock-Brown
c9c62380ff Don't wrap thead limits when using a negative int
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 16:03:44 -07:00
Tony Holdstock-Brown
33c448f147 Implement regulator in filesystem driver
This commit refactors base.regulator into the 2.4 interfaces and adds a
filesystem configuration option `maxthreads` to configure the regulator.

By default `maxthreads` is set to 100. This means the FS driver is
limited to 100 concurrent blocking file operations. Any subsequent
operations will block in Go until previous filesystem operations
complete.

This ensures that the registry can never open thousands of simultaneous
threads from os filesystem operations.

Note that `maxthreads` can never be less than 25.

Add test case covering parsable string maxthreads

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-05-03 09:33:22 -07:00
Josh Hawn
e4dd3359cc Regulate filesystem driver to max of 100 calls
It's easily possible for a flood of requests to trigger thousands of
concurrent file accesses on the storage driver. Each file I/O call creates
a new OS thread that is not reaped by the Golang runtime. By limiting it
to only 100 at a time we can effectively bound the number of OS threads
in use by the storage driver.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-26 14:44:13 -07:00
Brian Bland
ff03381d49 Adds new storagedriver.FileWriter interface
Updates registry storage code to use this for better resumable writes.
Implements this interface for the following drivers:
 + Inmemory
 + Filesystem
 + S3
 + Azure

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-03-08 16:37:44 -08:00
Brian Bland
f19deba809 Relaxes filesystem driver permissions to 0777 (dirs) and 0666 (files)
Leaves any further permissions restrictions to the process umask

Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-12-28 15:22:28 -08:00
Stephen J Day
a4eae0917e driver/filesystem: address filesystem driver on behavior of List
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-12-08 12:04:03 -08:00
Richard Scothern
bc6e4cdceb Add a generic error type to capture non-typed errors
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-11-03 11:19:44 -08:00
amitshukla
9637cb40cd Fix for issue 664: https://github.com/docker/distribution/issues/664
Errors thrown by storage drivers don't have the name of the driver, causing user
confusion about whether the error is coming from Docker or from a storage driver.
This change adds the storage driver name to each error message.

This required changing ErrUnsupportedDriver to a type, leading to code changes
whenever ErrUnsupportedDriver is used.  The tests check whether the driver name
appears in the error message.

Signed-off-by: Amit Shukla <amit.shukla@docker.com>
2015-11-03 11:19:17 -08:00
Olivier Gambier
6e0cfc17dc Saner default data location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-10 20:54:24 -07:00
Richard
ae216e365a Make Storage Driver API calls context aware.
- Change driver interface to take a context as its first argument
     - Make newFileReader take a context as its first argument
     - Make newFileWriter take a context as its first argument
     - Make blobstore exists and delete take a context as a first argument
     - Pass the layerreader's context to the storage layer
     - Pass the app's context to purgeuploads
     - Store the app's context into the blobstore (was previously null)
     - Pass the trace'd context to the storage drivers

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 15:58:58 -07:00
Stephen J Day
9e146437e4 Require storage drivers to report their name
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 17:30:01 -07:00
Stephen J Day
65b0d73cb7 Move storagedriver package to registry/storage/driver
This change is slightly more complex than previous package maves in that the
package name changed. To address this, we simply always reference the package
driver as storagedriver to avoid compatbility issues with existing code. While
unfortunate, this can be cleaned up over time.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-02-11 12:43:04 -08:00
Renamed from storagedriver/filesystem/driver.go (Browse further)