This removes the old global walk function, and changes all
the code to use the per-driver walk functions.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
* Improve Cloudfront notes regarding private buckets
* Point to CloudFront docs
This is better than outlining the steps specifically. The API steps will be different and the specific parts of the web UI may change over time. Amazon's docs are more likely to be up to date.
This changes the Walk Method used for catalog enumeration. Just to show
how much an effect this has on our s3 storage:
Original:
List calls: 6839
real 3m16.636s
user 0m0.000s
sys 0m0.016s
New:
ListObjectsV2 Calls: 1805
real 0m49.970s
user 0m0.008s
sys 0m0.000s
This is because it no longer performs a list and stat per item, and instead
is able to use the metadata gained from the list as a replacement to stat.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Move the Walk types into registry/storage/driver, and add a Walk method to each
storage driver. Although this is yet another API to implement, there is a fall
back implementation that relies on List and Stat. For some filesystems this is
very slow.
Also, this WalkDir Method conforms better do a traditional WalkDir (a la filepath).
This change is in preparation for refactoring.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
This upgrade, and vendors aws-sdk-go to version v1.12.36.
This is because it has new API calls accessible to the S3 client,
specifically S3.ListObjectsV2PagesWithContext
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Fixed the headers. They were rendering incorrectly on GitHub for some weird reason; I copied the strings and rewrote the ## characters at the beginning and now they render as they should.
Signed-off-by: Per Lundberg <per.lundberg@ecraft.com>
If tenant or tenantid are passed as env variables, we systematically use Sprint to make sure they are string and not integer as it would make mapstructure fail.
Signed-off-by: Raphaël Enrici <raphael@root-42.com>
The previous code assumed that the link returned when listing tags was
always absolute. However, some registries, such as quay.io, return the
link as a relative link (e.g. the second page for the quay.io/coreos/etcd
image is /v2/coreos/etcd/tags/list?next_page=<truncated>&n=50). Because
the relative link was retrieved directly, the fetch failed (with the
error `unsupported protocol scheme ""`).
Signed-off-by: Kevin Lin <kevin@kelda.io>
If the overwriteStruct() finds an uninitialized pointer, it tries to initialize it,
but does it incorrectly. It tries to assign a pointer to pointer, instead of pointer.
Signed-off-by: Gladkov Alexey <agladkov@redhat.com>
Using a daemon configuration file is preferred over
using command-line flags, as it allows reloading
this configuration without restarting the
daemon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If htpasswd authentication option is configured but the htpasswd file is
missing, populate it with a default user and automatically generated
password.
The password will be printed to stdout.
Signed-off-by: Liron Levin <liron@twistlock.com>
1. when lookup an entry which is missing, it should say NotFound.
2. when add duplicated entry, the entries size should be increased.
3. when add entry which has different algorithm, it should be allowed.
Signed-off-by: zhouhaibing089 <zhouhaibing089@gmail.com>
the digest cli does not work if we do not import this two packages,
tested in go1.9. basically, we have to make several algorithms to
be available by calling crypto.RegisterHash in init functions.
Signed-off-by: zhouhaibing089 <zhouhaibing089@gmail.com>
The current registry/client sends the registered manifest types in
random order. Allow clients to request a single specific manifest type
or a preferred order as per the HTTP spec.
Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
