Several API tests were added to ensure correct acceptance of zero-size and
empty tar files. This led to several changes in the storage backend around the
guarantees of remote file reading, which backs the layer and layer upload type.
In support of these changes, zero-length and empty checks have been added to
the digest package. These provide a sanity check against upstream tarsum
changes. The fileReader has been modified to be more robust when reading and
seeking on zero-length or non-existent files. The file no longer needs to exist
for the reader to be created. Seeks can now move beyond the end of the file,
causing reads to issue an io.EOF. This eliminates errors during certain race
conditions for reading files which should be detected by stat calls. As a part
of this, a few error types were factored out and the read buffer size was
increased to something more reasonable.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Several API tests were added to ensure correct acceptance of zero-size and
empty tar files. This led to several changes in the storage backend around the
guarantees of remote file reading, which backs the layer and layer upload type.
In support of these changes, zero-length and empty checks have been added to
the digest package. These provide a sanity check against upstream tarsum
changes. The fileReader has been modified to be more robust when reading and
seeking on zero-length or non-existent files. The file no longer needs to exist
for the reader to be created. Seeks can now move beyond the end of the file,
causing reads to issue an io.EOF. This eliminates errors during certain race
conditions for reading files which should be detected by stat calls. As a part
of this, a few error types were factored out and the read buffer size was
increased to something more reasonable.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Handles an issue where mux.Route does not set the desired scheme
when building a url and always uses `http`.
Now uses X-Forwarded-Proto when creating a URLBuilder from a request.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
When requesting a token, the basic auth header is always being set even
if there is no username value. This patch corrects this and does not set
the basic auth header if the username is empty.
Also fixes an issue where pulling all tags from a v2 registry succeeds
when the image does not actually exist on the registry.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
When getting the URL from a v2 registry url builder, it does not
honor the scheme from the endpoint object and will cause an https
endpoint to return urls starting with http.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This changeset addresses intermittent internal server errors encountered during
pushes. The root cause has been isolated to layers that result in identical,
empty filesystems but may have some path declarations (imaginge "./"),
resulting in different tarsums. The main error message reported during these
upload problems was a 500 error, which was not correct. Further investigation
showed the errors to be rooted in digest verification when finishing uploads.
Inspection of the surrounding code also identified a few issues. PutLayerChunk
was slightly refactered into PutLayerUploadComplete. Helper methods were
avoided to make handler less confusing. This simplification leveraged an
earlier change in the spec that moved non-complete chunk uploads to the PATCH
method. Simple logging was also added in the unknown error case that should
help to avoid mysterious 500 errors in the future.
At the same time, the glaring omission of a proper layer upload cancel method
was rectified. This has been added in this change so it is not missed in the
future.
In the future, we may want to refactor the handler code to be more
straightforward, hopefully letting us avoid these problems in the future.
Added test cases that reproduce these errors and drove these changes include
the following:
1. Push a layer with an empty body results in invalid blob upload.
2. Push a layer with a different tarsum (in this case, empty tar)
3. Deleting a layer upload works.
4. Getting status on a deleted layer upload returns 404.
Common functionality was grouped into shared functions to remove repitition.
The API tests will still require future love.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This changeset addresses intermittent internal server errors encountered during
pushes. The root cause has been isolated to layers that result in identical,
empty filesystems but may have some path declarations (imaginge "./"),
resulting in different tarsums. The main error message reported during these
upload problems was a 500 error, which was not correct. Further investigation
showed the errors to be rooted in digest verification when finishing uploads.
Inspection of the surrounding code also identified a few issues. PutLayerChunk
was slightly refactered into PutLayerUploadComplete. Helper methods were
avoided to make handler less confusing. This simplification leveraged an
earlier change in the spec that moved non-complete chunk uploads to the PATCH
method. Simple logging was also added in the unknown error case that should
help to avoid mysterious 500 errors in the future.
At the same time, the glaring omission of a proper layer upload cancel method
was rectified. This has been added in this change so it is not missed in the
future.
In the future, we may want to refactor the handler code to be more
straightforward, hopefully letting us avoid these problems in the future.
Added test cases that reproduce these errors and drove these changes include
the following:
1. Push a layer with an empty body results in invalid blob upload.
2. Push a layer with a different tarsum (in this case, empty tar)
3. Deleting a layer upload works.
4. Getting status on a deleted layer upload returns 404.
Common functionality was grouped into shared functions to remove repitition.
The API tests will still require future love.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
We've added support to the registry command to report the current version of
the distribution package. The version package is generated with a shell script
that gets the latest tag and add "+unknown". This allows builds from "go get"
and "go install" to have a rough version number. Generated periodically, it
will provide a decent indication of what code built the binary. For more
accurate versioning, one can build with the "binaries" make target. Linker
flags are used to replace the version string with the actual current tag at
build time.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Under certain cases, such as when putting a manifest or check for the existence
of a layer, the status code checks in session_v2.go were too narrow for their
purpose. In the case of putting a manifest, the handler only cares that an
error is not returned. Whether it is a 304 or 202 does not matter, as long as
the server reports success. Having the client only accept specific http codes
inhibits future protocol evolution.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Also removed ModTime checks on directories as it is not
required and some drivers might fail to provide it.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Since the Docker-Distribution-API-Version header value may contain multiple
space delimited versions as well as many instances of the header key, the
header value is now split on whitespace characters to iterate over all versions
that may be listed in one instance of the header.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
