Commit graph

281 commits

Author SHA1 Message Date
Richard
390bb97a88 Manifest and layer soft deletion.
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache.  Deletion is configurable - if it is disabled API calls
will return an unsupported error.

We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.

Delete() is added throughout the relevant API to support this functionality.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-24 09:57:20 -07:00
Doug Davis
0ec762c0f0 Remove dead code
thanks to @tiborvass for noticing

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-23 07:09:48 -07:00
Stephen Day
96efb19952 Merge pull request #493 from nevermosby/storage-support-openstack-swift
Storage support openstack swift
2015-07-22 23:53:36 -07:00
Stephen Day
a1ce8d81f7 Merge pull request #653 from pdevine/catalog-api
Catalog for V2 API Implementation
2015-07-22 18:54:48 -07:00
Aaron Lehmann
355c620139 Merge pull request #724 from aaronlehmann/repository-name-regexp-tests
Add additional test coverage for the regexp contained in RepositoryNameRegexp
2015-07-22 18:41:44 -07:00
Aaron Lehmann
683dc19778 Unify the testcases for the two tests in names_test.go
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-22 18:33:15 -07:00
Patrick Devine
a49594a0e1 Add Registry to client bindings for Repositories
The way Repositories() was initially called was somewhat different than
other parts of the client bindings because there was no way to instantiate a
Namespace.  This change implements a NewRegistry() function which changes
it so that Repositories() can be called the way one would expect.

It doesn't implement any of the other functions of Namespaces.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>
2015-07-22 17:48:15 -07:00
Doug Davis
aae59d54ef Make Error.Error() return the post-arg-substitution Message
Missed this during the removal of the args property

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-22 12:44:59 -07:00
Stephen Day
4e72e13d36 Merge pull request #709 from hopkings2008/master
Close reader after the test is finished.
2015-07-22 12:22:22 -07:00
Patrick Devine
bf62b7ebb7 Create Repositories method
This change removes the Catalog Service and replaces it with a more
simplistic Repositories() method for obtaining a catalog of all
repositories.  The Repositories method takes a pre-allocated slice
and fills it up to the size of the slice and returns the amount
filled.  The catalog is returned lexicographically and will start
being filled from the last entry passed to Repositories().  If there
are no more entries to fill, io.EOF will be returned.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>

Conflicts:
	registry/client/repository.go
	registry/handlers/api_test.go
2015-07-21 21:45:14 -07:00
Patrick Devine
f3207e76c8 Catalog for V2 API Implementation
This change adds a basic catalog endpoint to the API, which returns a list,
or partial list, of all of the repositories contained in the registry.  Calls
to this endpoint are somewhat expensive, as every call requires walking a
large part of the registry.

Instead, to maintain a list of repositories, you would first call the catalog
endpoint to get an initial list, and then use the events API to maintain
any future repositories.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>
2015-07-21 21:45:14 -07:00
Stephen J Day
0790a298ed Paginate catalog and tag results with Link header
Move the specification to use a Link header, rather than a "next" entry in the
json results. This prevents requiring clients from parsing the request body to
issue the next request. It also ensures that the returned response body does
not change in between requests.

The ordering of the specification has been slightly tweaked, as well. Listing
image tags has been moved after the catalog specification. Tag pagination now
heavily references catalog pagination.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-21 21:45:14 -07:00
Stephen J Day
1d68d81b42 Catalog V2 API specification proposal
This contains a proposal for a catalog API, provided access to the internal
contents of a registry instance. The API endpoint is prefixed with an
underscore, which is illegal in images names, to prevent collisions with
repositories names. To avoid issues with large result sets, a paginated version
of the API is proposed. We make an addition to the tags API to support
pagination to ensure the specification is conistent.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-21 21:45:14 -07:00
Aaron Lehmann
ceb2c7de44 Add additional test coverage for the regexp contained in RepositoryNameRegexp
This was inspired by problems found with new regexps proposed in PR #690

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-21 17:10:36 -07:00
davidli
b2935158b2 Remove IPC support from test file
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
81765f8cbb Catch either missing containers or objects
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
52d28ec81a Do not use Swift server side copy for manifests to handle >5G files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
000dec3c6f Inline Swift errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
661f197f68 Retrieve all the objects using pagination
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
704e082254 Do not create objects for directories
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f190aa4a7c Refactor segment path concatenation code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
0807282859 Use http.StatusRequestedRangeNotSatisfiable instead of error code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7a5aa32a64 Use file instead of filepath as it may cause troubles on Windows
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
91d74a3ee2 Protect against deletion of objects with the same prefix
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
01686e2c07 Show distribution version in User-Agent
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
913fe195fd Do not use suite style testing for Swift specific tests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1b28eea232 Rename environment variables to run Swift testsuite
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
2524f300dc Check file has been opened before closing it
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
fbc74a6457 Rename DriverParameters structure to Parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
80bfcb68a8 Change folder mime type to application/vnc.swift.directory
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7b0276dce5 Add code documentation
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
5cce023aa9 Do not read segment if no padding is necessary
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
d91c4cb694 Improve 404 errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
9ab55eae39 Use only one Swift container for both files and manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
a1ae7f7122 Increase default chunk size to 20M
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
davidli
062d6266cf Add support for Openstack Identity v3 API
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
3f9e7ed169 Use 'prefix' parameter instead of 'path' when listing files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1d46bb2bcc Create full folder hierarchy instead of just the top level folder
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
4e619bc9b1 Remove one level of indentation in swift path handling code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
75ce67c469 Use mitchellh/mapstructure library to parse Swift parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ea81e208a4 Move Dynamic Large Object handling to dedicated methods
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
8a22c0f4e1 Simplify code that handles non existing manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
16a49ade16 Handle error during copy of original content
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
nevermosby
9f7f23e373 Update the import path for swift driver test
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
davidli
1f4eb7b735 Use gofmt to format the code of swift driver.
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
cce4956131 Add Openstack Swift storage driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:08 +02:00
Stephen Day
50de6d914c Merge pull request #713 from stevvooe/descriptor-size-field
Use "Size" field to describe blobs over "Length"
2015-07-17 22:06:59 -07:00
Stephen J Day
249ad3b76d Use "Size" field to describe blobs over "Length"
After consideration, we've changed the main descriptor field name to for number
of bytes to "size" to match convention. While this may be a subjective
argument, commonly we refer to files by their "size" rather than their
"length". This will match other conventions, like `(FileInfo).Size()` and
methods on `io.SizeReaderAt`. Under more broad analysis, this argument doesn't
necessarily hold up. If anything, "size" is shorter than "length".

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-17 17:07:11 -07:00
yuzou
feebd69d26 Close reader after the test is finished.
Signed-off-by: yuzou <zouyu7@huawei.com>
2015-07-17 14:55:31 +08:00
Stephen Day
5c6c88196d Merge pull request #705 from stevvooe/export-servejson-errors
Export ServeJSON for serving error codes
2015-07-16 14:35:27 -07:00