This change is slightly more complex than previous package maves in that the
package name changed. To address this, we simply always reference the package
driver as storagedriver to avoid compatbility issues with existing code. While
unfortunate, this can be cleaned up over time.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
To address the possibility of confusing registry name components with
repository paths, path components that abut user provided repository names are
escaped with a prefixed underscore. This works because repository name
components are no allowed to start with underscores. The requirements on
backend driver path names have been relaxed greatly to support this use case.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Also removed ModTime checks on directories as it is not
required and some drivers might fail to provide it.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
We now also have a storagedriver error variable for identifying
api calls that are not implemented by drivers (the URLFor method
is not implemented by either the filesystem or inmemory drivers)
No longer requires that file paths match the repository naming scheme,
but instead allows path components as short as a single character, as to
accommodate for single-character tag names.
This requires some discussion of how we will handle errors due to network problems
and after further changes in that direction some more stress testing. There is also an
upcomming commit implementing zero fill on WriteStream when offset is greater than
the current size of the file.
Requires all paths in the inmemory and filesystem drivers to begin with
a slash, and then contain only valid path components (2+ alphanumeric
characters with optional period, hyphen, and underscore separators)
delimited by slashes.
Also updates the storage driver test suites to construct paths of this
format, and causes the suite to abort if files are not cleaned up after
the test run.
Also makes this test respect the Short flag, reducing the number of
threads by a factor of 4 and space usage by a factor of 16
Note: this test is probably unreasonable to run on the inmemory driver
without the Short flag
Creates trees instead of flat files for TestConcurrentFileStreams
Adds TestConcurrentStreamReads, which writes a large file (smaller in
Short mode), and then ensures that several concurrent readers properly
read their portions of the file with random offsets
Ensures that Move will properly overwrite the file at the destination
location.
Also checks that Move of a nonexistent source file will NOT delete the
file at the destination.
This replaces only using flat filenames, to better test nested file
behaviors.
Fixed inmemory/mfs.go and filesystem/driver.go after finding bugs with
the new tests and test behavior.
The packages causing build errors are being disabled for now to let us split up
the work in the different driver implementations without blocking integration
into the main branch. The s3 and azure implementations need some effort to add
Stat support. The ipc package needs that work plus some care around hanging
send calls.
Several checks for ReadStream with offset around boundary conditions were
missing. The new checks ensure negative offsets are detected and io.EOF is
returned properly when trying to read past the end of a file. The filesystem
and inmemory driver have been updated accordingly.
An outline of missing checks for List are also part of this commit. Action will
be taken here based on discussion in issue #819.
This change updates the testsuite to migrate to the new driver interface. This
includes the new Stat call, changes to int64 over uint64 and the changes to the
WriteStream signature. Several test cases have been added to vet
implementations against various assumptions.
This change brings the storagedriver API in line with the Go standard library's
use of int64 for offsets. The main benefit is simplicity in interfacing with
the io library reducing the number of type conversions in simple code.
Fixes/tests listing for keys beginning with "/"
No longer extraneously wraps Closers in ioutil.NopClosers
Uses omitempty for all ipc struct type fields
This only works for a specific whitelist of error types, which is
currently all errors in the storagedriver package.
Also improves storagedriver tests to enforce proper error types are
returned
Custom storage drivers can register a factory to create the driver by
name, similar to the database/sql package's Register and Open
factory.Create returns an in-process driver if registered or an IPC
driver if one can be found, erroring otherwise
This standardizes parameter passing for creation of storage drivers
Also adds documentation for storagedriver package and children
