TrueCloudLab/distribution
13
0
Fork 3
Code Issues 3 Pull requests Projects Releases 2 Packages Wiki Activity Actions
1915 commits 2 branches 61 tags 129 MiB
Commit graph

608 commits

Author SHA1 Message Date
Richard Scothern
ea492aca1a Move garbage collect code into storage package 
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
 2016-04-27 13:34:25 -07:00
Tony Holdstock-Brown
a5aaae1f06 Ensure GC continues marking if _manifests is nonexistent 
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
 2016-04-27 13:33:36 -07:00
Serge Dubrouski
32193bdcf0 Fix wording for dry-run flag in useage message for garbage collector. 
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
 2016-04-26 19:44:23 -06:00
Richard Scothern
c83afea0c9 Merge pull request #1660 from jhaohai/cn-north-1-fix 
Add cn-north-1 to valid check
 2016-04-25 16:07:54 -07:00
Richard Scothern
ef32134592 Merge pull request #1666 from sergeyfd/master 
Add blobWrtiter.Close() call into blobWriter.Commit()
 2016-04-25 16:02:48 -07:00
Anis Elleuch
987faca8a6 Sorting completed parts by part number for a better accordance with the S3 spec 
Signed-off-by: Anis Elleuch <vadmeste@gmail.com>
 2016-04-23 22:36:04 +01:00
Serge Dubrouski
21f38a74e6 Add blobWrtiter.Close() call into blobWriter.Commit() 
Signed-off-by: Serge Dubrouski <sergeyfd@gmail.com>
 2016-04-22 19:23:17 -06:00
jhaohai
f76c622d8c add cn-north-1 to valid check 
Signed-off-by: jhaohai <jhaohai@foxmail.com>
 2016-04-21 11:51:34 +08:00
Richard Scothern
4c119524f1 Merge pull request #1604 from ArdaXi/custom-s3-skip-region-check 
Only check validity of S3 region if not using custom endpoint
 2016-04-08 15:38:24 -07:00
Richard Scothern
437eeeda44 Merge pull request #1605 from majewsky/swift/deal-with-outdated-container-listings 
registry/storage/swift: detect and fix outdated container listings
 2016-04-08 15:38:06 -07:00
Stefan Majewsky
67321cb622 detect outdated container listings during Stat() and getAllSegments() 
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
 2016-04-06 15:21:27 +02:00
Arien Holthuizen
dbb6e28da2 Only check validity of S3 region if not using custom endpoint 
Signed-off-by: Arien Holthuizen <aholthuizen@schubergphilis.com>
 2016-04-06 13:38:09 +02:00
Tony Holdstock-Brown
25c5efdef9 Ensure we log io.Copy errors and bytes copied/total in uploads 
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
 2016-04-04 17:21:36 -07:00
Richard Scothern
04ff3c0359 Merge pull request #1578 from majewsky/do-not-swallow-error 
don't swallow errors in Swift driver's GetContent()
 2016-03-30 17:46:27 -07:00
Richard Scothern
eb0b7f0173 Update the gc documentation. 
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
 2016-03-30 16:57:21 -07:00
Richard Scothern
4324b70c50 Fix signature handling with GC. 
If a schema 1 manifest is uploaded with the `disablesignaturestore` option set
to true, then no signatures will exist.  Handle this case.

If a schema 1 manifest is pushed, deleted, garbage collected and pushed again, the
repository will contain signature links from the first version, but the blobs will
not exist.  Disable the signature store in the garbage-collect command so
signatures are not fetched.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
 2016-03-30 16:10:27 -07:00
Richard Scothern
80b310ca44 Add a --dry-run flag. If enabled this will print the mark and sweep process 
with removing any files.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
 2016-03-30 16:10:27 -07:00
Aaron Lehmann
267e268775 garbagecollect: Clean up errors 
- Clean up error messages

- Add a missing error check on the result of blobService.Enumerate.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
 2016-03-30 11:35:24 -07:00
Stefan Majewsky
4c9bed2507 don't swallow errors in Swift driver's GetContent() 
In 326c3a9c49, which was only intended to
be a refactoring commit, the behavior of this block subtly changed so
that unknown types of errors would be swallowed instead of propagated.

I noticed this while investigating an error similar to #1539 aka
docker/docker#21290. It appears that during GetContent() for a
hashstate, the Swift proxy produces an error. Since this error was
silently swallowed, an empty []byte is used to restart the hash, then
producing the digest of the empty string instead of the layer's digest.

This PR will not fix the issue, but it should make the actual error more
visible by propagating it into `blobWriter#resumeDigest' and
'blobWriter#validateBlob', respectively.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
 2016-03-30 16:11:29 +02:00
Stephen Day
0eefe17730 Merge pull request #1546 from arschles/doc-storage 
Add documentation for how to register new StorageDrivers
 2016-03-29 15:58:51 -07:00
Aaron Schlesinger
f4bdc6287a Remove the example 
Instead, direct users to the one in the factory package

Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
 2016-03-29 14:42:28 -07:00
Aaron Schlesinger
204ad474e4 Add documentation for how to register new StorageDrivers 
This commit adds context-specific documentation on StorageDriver,
StorageDriverFactory, and the factory’s Register func, explaining how
the internal registration mechanism should be used.

This documentation follows from the thread starting at
https://github.com/deis/builder/pull/262/files#r56720200.

cc/ @stevvooe

Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
 2016-03-29 14:42:19 -07:00
Tony Holdstock-Brown
31d5dd5175 utulize config log format within gc 
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
 2016-03-24 14:29:43 -07:00
Richard Scothern
87a997249d Merge pull request #1522 from RichardScothern/tag-events 
Send tag events to notification listeners
 2016-03-23 16:41:29 -07:00
Stephen Day
9e690c7fa2 Merge pull request #1491 from RichardScothern/relative-url 
Return relative URLs
 2016-03-23 16:19:39 -07:00
Richard Scothern
bc9c820e4b Enable URLs returned from the registry to be configured as relative. 
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
 2016-03-23 15:14:32 -07:00
Richard Scothern
afe2bdd1c5 Propogate tag as a functional argument into the notification system to attach 
tags to manifest push and pull event notifications.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
 2016-03-23 14:57:52 -07:00
姜继忠
92a9ee34d4 fix manifest revision search, closes #1535 
Signed-off-by: 姜继忠 <jizhong.jiangjz@alibaba-inc.com>
 2016-03-19 19:11:25 +08:00
Antonio Murdaca
ea148448a2 registry: client: auth: type errors 
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
 2016-03-18 09:34:50 +01:00
Richard Scothern
ec6ac0c05e Merge pull request #1531 from aaronlehmann/empty-errors-slice 
Don't return empty errcode.Errors slices
 2016-03-17 10:46:51 -07:00
Antonio Murdaca
75b5a1b1f6 registry: client: repository: close response body 
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
 2016-03-16 19:53:17 +01:00
Richard Scothern
2c635d1f7e Merge pull request #1532 from RichardScothern/azure-error-types 
Update missing blob error checking with latest Azure API
 2016-03-15 10:08:36 -07:00
Aaron Lehmann
3a2231fe39 Include status code in UnexpectedHTTPResponseError 
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
 2016-03-15 09:03:56 -07:00
Richard Scothern
a5d372535e Merge pull request #1502 from RichardScothern/auth-url-endpoints 
URL parse auth endpoints to normalize hostname to lowercase.
 2016-03-14 16:28:24 -07:00
Richard Scothern
772e6f4057 Update missing blob error checking with latest Azure API 
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
 2016-03-14 15:59:03 -07:00
Aaron Lehmann
cd6482ecb8 Don't return empty errcode.Errors slices 
If this slice ends up empty after parsing the HTTP response body, it
means the body is not well-formed. We've probably encountered an error
message produced by something that uses a different JSON schema, or
an error that just happens to validate as JSON.

An empty errcode.Errors slice is not a very useful thing to return,
since its Error() output is just `<nil>`. Detect this case, and instend
return an UnexpectedHTTPResponseError.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
 2016-03-14 10:27:01 -07:00
Matt Duch
fcb247dfce registry/storage/driver/s3-aws kms support 
Signed-off-by: Matt Duch <matt@learnmetrics.com>
 2016-03-11 17:19:01 -06:00
Brian Bland
c03b5fc5ee Merge pull request #1438 from BrianBland/newStorageDriverWriter 
Adds new StorageDriver.FileWriter interface
 2016-03-11 15:06:07 -08:00
Brian Bland
5967d33342 Removes ceph rados driver in favor of Swift API gateway support 
Signed-off-by: Brian Bland <brian.bland@docker.com>
 2016-03-10 16:49:08 -08:00
Brian Bland
ab49f85db8 Merge pull request #1512 from kmala/master 
Added support to specifiy custom endpoint
 2016-03-10 13:48:59 -08:00
Keerthan Mala
2be1b4ef4f Added support to specifiy custom endpoint 
Signed-off-by: Keerthan Reddy Mala <keerthan.mala@gmail.com>
 2016-03-09 16:12:20 -07:00
Derek McGowan
a59448784b Add client ID to token fetch to GET endpoint 
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
 2016-03-09 12:44:55 -08:00
Brian Bland
7fd1db9312 Updates Swift driver to support new storagedriver.FileWriter interface 
Signed-off-by: Brian Bland <brian.bland@docker.com>
 2016-03-08 16:38:39 -08:00
Li Yi
a9bf7a2aae Support FileWriter interface for OSS storage driver 
Change-Id: Ie5533ad85f944800499ca1040fd67bf1378815e0
Signed-off-by: Li Yi <denverdino@gmail.com>
 2016-03-08 16:38:39 -08:00
Arthur Baars
307504713f Storagedriver: GCS: add chunksize parameter 
Signed-off-by: Arthur Baars <arthur@semmle.com>
 2016-03-08 16:38:39 -08:00
Arthur Baars
7162cb19c6 Storagedriver: GCS: implement resumable uploads 
Signed-off-by: Arthur Baars <arthur@semmle.com>
 2016-03-08 16:38:39 -08:00
Arthur Baars
666273d9f6 StorageDriver: Testsuite: call Close before getting Size 
Signed-off-by: Arthur Baars <arthur@semmle.com>
 2016-03-08 16:38:38 -08:00
Brian Bland
ff03381d49 Adds new storagedriver.FileWriter interface 
Updates registry storage code to use this for better resumable writes.
Implements this interface for the following drivers:
 + Inmemory
 + Filesystem
 + S3
 + Azure

Signed-off-by: Brian Bland <brian.bland@docker.com>
 2016-03-08 16:37:44 -08:00
Aaron Lehmann
87e34bd307 Fix two misspellings in source code comments 
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
 2016-03-08 15:13:24 -08:00
Richard Scothern
8bd613aa11 URL parse auth endpoints to normalize hostname to lowercase. 
It is possible for a middlebox to lowercase the URL at somepoint causing a
lookup in the auth challenges table to fail.  Lowercase hostname before
using as keys to challenge map.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
 2016-03-07 14:08:21 -08:00