Commit graph

81 commits

Author SHA1 Message Date
tgic
fc20dd72d6 check access key and secret before run 2015-07-31 13:22:55 +08:00
tgic
d28a3fa28a add endpoint support 2015-07-31 13:22:54 +08:00
Li Yi
46148721e1 Add the secure access with HTTPS
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-07-31 13:22:54 +08:00
Li Yi
bffce5722e Fix the warning of golint
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-07-31 13:22:54 +08:00
Li Yi
d79179884a Support OSS driver
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-07-31 13:22:54 +08:00
Stephen Day
96efb19952 Merge pull request #493 from nevermosby/storage-support-openstack-swift
Storage support openstack swift
2015-07-22 23:53:36 -07:00
davidli
b2935158b2 Remove IPC support from test file
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
81765f8cbb Catch either missing containers or objects
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
52d28ec81a Do not use Swift server side copy for manifests to handle >5G files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
000dec3c6f Inline Swift errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
661f197f68 Retrieve all the objects using pagination
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
704e082254 Do not create objects for directories
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f190aa4a7c Refactor segment path concatenation code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
0807282859 Use http.StatusRequestedRangeNotSatisfiable instead of error code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7a5aa32a64 Use file instead of filepath as it may cause troubles on Windows
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
91d74a3ee2 Protect against deletion of objects with the same prefix
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
01686e2c07 Show distribution version in User-Agent
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
913fe195fd Do not use suite style testing for Swift specific tests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1b28eea232 Rename environment variables to run Swift testsuite
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
2524f300dc Check file has been opened before closing it
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
fbc74a6457 Rename DriverParameters structure to Parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
80bfcb68a8 Change folder mime type to application/vnc.swift.directory
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7b0276dce5 Add code documentation
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
5cce023aa9 Do not read segment if no padding is necessary
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
d91c4cb694 Improve 404 errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
9ab55eae39 Use only one Swift container for both files and manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
a1ae7f7122 Increase default chunk size to 20M
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
davidli
062d6266cf Add support for Openstack Identity v3 API
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
3f9e7ed169 Use 'prefix' parameter instead of 'path' when listing files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1d46bb2bcc Create full folder hierarchy instead of just the top level folder
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
4e619bc9b1 Remove one level of indentation in swift path handling code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
75ce67c469 Use mitchellh/mapstructure library to parse Swift parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ea81e208a4 Move Dynamic Large Object handling to dedicated methods
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
8a22c0f4e1 Simplify code that handles non existing manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
16a49ade16 Handle error during copy of original content
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
nevermosby
9f7f23e373 Update the import path for swift driver test
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
davidli
1f4eb7b735 Use gofmt to format the code of swift driver.
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
cce4956131 Add Openstack Swift storage driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:08 +02:00
yuzou
feebd69d26 Close reader after the test is finished.
Signed-off-by: yuzou <zouyu7@huawei.com>
2015-07-17 14:55:31 +08:00
Stephen Day
940b865bc0 Merge pull request #670 from stevvooe/remove-ipc
Remove half-baked Storage Driver IPC support
2015-06-30 19:21:54 -07:00
Richard Scothern
fa17f9254f Pass correct context into tracer
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-30 10:28:14 -07:00
Stephen J Day
6167220cdd Remove half-baked Storage Driver IPC support
This removes documentation and code related to IPC based storage driver
plugins. The existence of this functionality was an original feature goal but
is now not maintained and actively confusing incoming contributions. We will
likely explore some driver plugin mechanism in the future but we don't need
this laying around in the meantime.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-29 16:52:12 -07:00
Stephen J Day
f9e152d912 Ensure that rados is disabled without build tag
This ensures that rados is not required when building the registry. This was
slightly tricky in that when the flags were applied, the rados package was
completely missing. This led to a problem where rados was basically unlistable
and untestable as a package. This was fixed by simply adding a doc.go file that
is included whether rados is built or not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-16 20:00:02 -07:00
Stephen Day
d796729b6b Merge pull request #623 from ahmetalpbalkan/azure-vendor
storage/driver/azure: Update vendored Azure SDK
2015-06-16 17:41:38 -07:00
Ahmet Alp Balkan
5c372ded1b storage/driver/azure: Update vendored Azure SDK
This change refreshes the updated version of Azure SDK
for Go that has the latest changes.

I manually vendored the new SDK (github.com/Azure/azure-sdk-for-go)
and I removed `management/` `core/` packages manually simply because
they're not used here and they have a fork of `net/http` and `crypto/tls`
for a particular reason. It was introducing a 44k SLOC change otherwise...

This also undoes the `include_azure` flag (actually Steven removed the
driver from imports but forgot to add the build flag apparently, so the
flag wasn't really including azure. 😄 ). This also must be obsolete
now.

Fixes #620, #175.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-16 17:13:44 -07:00
Olivier Gambier
280b9c50ac Saner default data location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-10 20:54:24 -07:00
Vincent Giersch
4bc53818cb Fix rados build, remove uuid dependency
Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-06-01 09:57:40 +00:00
Stephen Day
af987fd3dc Merge pull request #443 from gierschv/driver-rados
Storage Driver: Ceph Object Storage (RADOS)
2015-05-29 18:10:49 -07:00
Stephen J Day
d4c50637f9 Better error message when failing to get AWS auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 16:45:45 -07:00
Vincent Giersch
2c1a83f940 Storage Driver: Ceph Object Storage (RADOS)
This driver implements the storagedriver.StorageDriver interface and
uses Ceph Object Storage as storage backend.

Since RADOS is an object storage and no hierarchy notion, the
following convention is used to keep the filesystem notions stored in
this backend:

* All the objects data are stored with opaque UUID names prefixed
  (e.g. "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164).
* All the hierarchy information are stored in rados omaps, where the
  omap object identifier is the virtual directory name, the keys in
  a specific are the relative filenames and the values the blob
  object identifier (or empty value for a sub directory).

  e.g. For the following hierarchy:

     /directory1
     /directory1/object1
     /directory1/object2
     /directory1/directory2/object3

  The omap "/directory1" will contains the following key / values:
    - "object1" "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164"
    - "object2" "blob:db2e359d-4af0-4bfb-ba1d-d2fd029866a0"
    - "directory2" ""

  The omap "/directory1/directory2" will contains:
    - "object3" "blob:9ae2371c-81fc-4945-80ac-8bf7f566a5d9"

* The MOVE is implemented by changing the reference to a specific
  blob in its parent virtual directory omap.

This driver stripes rados objects to a fixed size (e.g. 4M). The idea
is to keep small objects (as done by RBD on the top of RADOS) that
will be easily synchronized accross OSDs. The information of the
original object (i.e total size of the chunks) is stored as a Xattr
in the first chunk object.

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-05-20 01:44:34 +00:00