From 41b1e22f97ce50343c85acefaeb57a1b977bf6a0 Mon Sep 17 00:00:00 2001 From: Stephen J Day Date: Thu, 5 Mar 2015 17:23:33 -0800 Subject: [PATCH 1/2] doc: move storage driver readmes into docs Signed-off-by: Stephen J Day --- .../README.md => doc/storagedriver/azure.md | 0 .../storagedriver/filesystem.md | 0 .../storagedriver/inmemory.md | 0 .../s3/README.md => doc/storagedriver/s3.md | 0 doc/storagedrivers.md | 52 +++++++++++++++++-- registry/storage/driver/README.md | 49 ----------------- 6 files changed, 48 insertions(+), 53 deletions(-) rename registry/storage/driver/azure/README.md => doc/storagedriver/azure.md (100%) rename registry/storage/driver/filesystem/README.md => doc/storagedriver/filesystem.md (100%) rename registry/storage/driver/inmemory/README.md => doc/storagedriver/inmemory.md (100%) rename registry/storage/driver/s3/README.md => doc/storagedriver/s3.md (100%) delete mode 100644 registry/storage/driver/README.md diff --git a/registry/storage/driver/azure/README.md b/doc/storagedriver/azure.md similarity index 100% rename from registry/storage/driver/azure/README.md rename to doc/storagedriver/azure.md diff --git a/registry/storage/driver/filesystem/README.md b/doc/storagedriver/filesystem.md similarity index 100% rename from registry/storage/driver/filesystem/README.md rename to doc/storagedriver/filesystem.md diff --git a/registry/storage/driver/inmemory/README.md b/doc/storagedriver/inmemory.md similarity index 100% rename from registry/storage/driver/inmemory/README.md rename to doc/storagedriver/inmemory.md diff --git a/registry/storage/driver/s3/README.md b/doc/storagedriver/s3.md similarity index 100% rename from registry/storage/driver/s3/README.md rename to doc/storagedriver/s3.md diff --git a/doc/storagedrivers.md b/doc/storagedrivers.md index f8e4d675..b603503e 100644 --- a/doc/storagedrivers.md +++ b/doc/storagedrivers.md @@ -1,5 +1,49 @@ -# Storage Drivers +Docker-Registry Storage Driver +============================== -**TODO(stevvooe):** This should include detailed overviews of what a storage -driver is and information about each storage driver implementation. -Considerations when implementing a storage driver should also be present. \ No newline at end of file +This document describes the registry storage driver model, implementation, and explains how to contribute new storage drivers. + +Provided Drivers +================ + +This storage driver package comes bundled with three default drivers. + +1. filesystem: A local storage driver configured to use a directory tree in the local filesystem. +2. s3: A driver storing objects in an Amazon Simple Storage Solution (S3) bucket. +3. inmemory: A temporary storage driver using a local inmemory map. This exists solely for reference and testing. + +Storage Driver API +================== + +The storage driver API is designed to model a filesystem-like key/value storage in a manner abstract enough to support a range of drivers from the local filesystem to Amazon S3 or other distributed object storage systems. + +Storage drivers are required to implement the `storagedriver.StorageDriver` interface provided in `storagedriver.go`, which includes methods for reading, writing, and deleting content, as well as listing child objects of a specified prefix key. + +Storage drivers are intended (but not required) to be written in go, providing compile-time validation of the `storagedriver.StorageDriver` interface, although an IPC driver wrapper means that it is not required for drivers to be included in the compiled registry. The `storagedriver/ipc` package provides a client/server protocol for running storage drivers provided in external executables as a managed child server process. + +Driver Selection and Configuration +================================== + +The preferred method of selecting a storage driver is using the `StorageDriverFactory` interface in the `storagedriver/factory` package. These factories provide a common interface for constructing storage drivers with a parameters map. The factory model is based off of the [Register](http://golang.org/pkg/database/sql/#Register) and [Open](http://golang.org/pkg/database/sql/#Open) methods in the builtin [database/sql](http://golang.org/pkg/database/sql) package. + +Storage driver factories may be registered by name using the `factory.Register` method, and then later invoked by calling `factory.Create` with a driver name and parameters map. If no driver is registered with the given name, this factory will attempt to find an executable storage driver with the executable name "registry-storage-\" and return an IPC storage driver wrapper managing the driver subprocess. If no such storage driver can be found, `factory.Create` will return an `InvalidStorageDriverError`. + +Driver Contribution +=================== + +## Writing new storage drivers +To create a valid storage driver, one must implement the `storagedriver.StorageDriver` interface and make sure to expose this driver via the factory system and as a distributable IPC server executable. + +### In-process drivers +Storage drivers should call `factory.Register` with their driver name in an `init` method, allowing callers of `factory.New` to construct instances of this driver without requiring modification of imports throughout the codebase. + +### Out-of-process drivers +As many users will run the registry as a pre-constructed docker container, storage drivers should also be distributable as IPC server executables. Drivers written in go should model the main method provided in `storagedriver/filesystem/registry-storage-filesystem/filesystem.go`. Parameters to IPC drivers will be provided as a JSON-serialized map in the first argument to the process. These parameters should be validated and then a blocking call to `ipc.StorageDriverServer` should be made with a new storage driver. + +Out-of-process drivers must also implement the `ipc.IPCStorageDriver` interface, which exposes a `Version` check for the storage driver. This is used to validate storage driver api compatibility at driver load-time. + +## Testing +Storage driver test suites are provided in `storagedriver/testsuites/testsuites.go` and may be used for any storage driver written in go. Two methods are provided for registering test suites, `RegisterInProcessSuite` and `RegisterIPCSuite`, which run the same set of tests for the driver imported or managed over IPC respectively. + +## Drivers written in other languages +Although storage drivers are strongly recommended to be written in go for consistency, compile-time validation, and support, the IPC framework allows for a level of language-agnosticism. Non-go drivers must implement the storage driver protocol by mimicing StorageDriverServer in `storagedriver/ipc/server.go`. As the IPC framework is a layer on top of [docker/libchan](https://github.com/docker/libchan), this currently limits language support to Java via [ndeloof/chan](https://github.com/ndeloof/jchan) and Javascript via [GraftJS/jschan](https://github.com/GraftJS/jschan), although contributions to the libchan project are welcome. diff --git a/registry/storage/driver/README.md b/registry/storage/driver/README.md deleted file mode 100644 index b603503e..00000000 --- a/registry/storage/driver/README.md +++ /dev/null @@ -1,49 +0,0 @@ -Docker-Registry Storage Driver -============================== - -This document describes the registry storage driver model, implementation, and explains how to contribute new storage drivers. - -Provided Drivers -================ - -This storage driver package comes bundled with three default drivers. - -1. filesystem: A local storage driver configured to use a directory tree in the local filesystem. -2. s3: A driver storing objects in an Amazon Simple Storage Solution (S3) bucket. -3. inmemory: A temporary storage driver using a local inmemory map. This exists solely for reference and testing. - -Storage Driver API -================== - -The storage driver API is designed to model a filesystem-like key/value storage in a manner abstract enough to support a range of drivers from the local filesystem to Amazon S3 or other distributed object storage systems. - -Storage drivers are required to implement the `storagedriver.StorageDriver` interface provided in `storagedriver.go`, which includes methods for reading, writing, and deleting content, as well as listing child objects of a specified prefix key. - -Storage drivers are intended (but not required) to be written in go, providing compile-time validation of the `storagedriver.StorageDriver` interface, although an IPC driver wrapper means that it is not required for drivers to be included in the compiled registry. The `storagedriver/ipc` package provides a client/server protocol for running storage drivers provided in external executables as a managed child server process. - -Driver Selection and Configuration -================================== - -The preferred method of selecting a storage driver is using the `StorageDriverFactory` interface in the `storagedriver/factory` package. These factories provide a common interface for constructing storage drivers with a parameters map. The factory model is based off of the [Register](http://golang.org/pkg/database/sql/#Register) and [Open](http://golang.org/pkg/database/sql/#Open) methods in the builtin [database/sql](http://golang.org/pkg/database/sql) package. - -Storage driver factories may be registered by name using the `factory.Register` method, and then later invoked by calling `factory.Create` with a driver name and parameters map. If no driver is registered with the given name, this factory will attempt to find an executable storage driver with the executable name "registry-storage-\" and return an IPC storage driver wrapper managing the driver subprocess. If no such storage driver can be found, `factory.Create` will return an `InvalidStorageDriverError`. - -Driver Contribution -=================== - -## Writing new storage drivers -To create a valid storage driver, one must implement the `storagedriver.StorageDriver` interface and make sure to expose this driver via the factory system and as a distributable IPC server executable. - -### In-process drivers -Storage drivers should call `factory.Register` with their driver name in an `init` method, allowing callers of `factory.New` to construct instances of this driver without requiring modification of imports throughout the codebase. - -### Out-of-process drivers -As many users will run the registry as a pre-constructed docker container, storage drivers should also be distributable as IPC server executables. Drivers written in go should model the main method provided in `storagedriver/filesystem/registry-storage-filesystem/filesystem.go`. Parameters to IPC drivers will be provided as a JSON-serialized map in the first argument to the process. These parameters should be validated and then a blocking call to `ipc.StorageDriverServer` should be made with a new storage driver. - -Out-of-process drivers must also implement the `ipc.IPCStorageDriver` interface, which exposes a `Version` check for the storage driver. This is used to validate storage driver api compatibility at driver load-time. - -## Testing -Storage driver test suites are provided in `storagedriver/testsuites/testsuites.go` and may be used for any storage driver written in go. Two methods are provided for registering test suites, `RegisterInProcessSuite` and `RegisterIPCSuite`, which run the same set of tests for the driver imported or managed over IPC respectively. - -## Drivers written in other languages -Although storage drivers are strongly recommended to be written in go for consistency, compile-time validation, and support, the IPC framework allows for a level of language-agnosticism. Non-go drivers must implement the storage driver protocol by mimicing StorageDriverServer in `storagedriver/ipc/server.go`. As the IPC framework is a layer on top of [docker/libchan](https://github.com/docker/libchan), this currently limits language support to Java via [ndeloof/chan](https://github.com/ndeloof/jchan) and Javascript via [GraftJS/jschan](https://github.com/GraftJS/jschan), although contributions to the libchan project are welcome. From 52865bb128b722354edcb18b5451a3a6ad4ea850 Mon Sep 17 00:00:00 2001 From: Stephen J Day Date: Thu, 5 Mar 2015 17:26:50 -0800 Subject: [PATCH 2/2] doc/storagedrivers: update storage driver documentation links Signed-off-by: Stephen J Day --- doc/storagedrivers.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/doc/storagedrivers.md b/doc/storagedrivers.md index b603503e..c230c116 100644 --- a/doc/storagedrivers.md +++ b/doc/storagedrivers.md @@ -6,11 +6,12 @@ This document describes the registry storage driver model, implementation, and e Provided Drivers ================ -This storage driver package comes bundled with three default drivers. +This storage driver package comes bundled with several drivers: -1. filesystem: A local storage driver configured to use a directory tree in the local filesystem. -2. s3: A driver storing objects in an Amazon Simple Storage Solution (S3) bucket. -3. inmemory: A temporary storage driver using a local inmemory map. This exists solely for reference and testing. +- [inmemory](storagedriver/inmemory.md): A temporary storage driver using a local inmemory map. This exists solely for reference and testing. +- [filesystem](storagedriver/filesystem.md): A local storage driver configured to use a directory tree in the local filesystem. +- [s3](storagedriver/s3.md): A driver storing objects in an Amazon Simple Storage Solution (S3) bucket. +- [azure](storagedriver/azure.md): A driver storing objects in [Microsoft Azure Blob Storage](http://azure.microsoft.com/en-us/services/storage/). Storage Driver API ==================