forked from TrueCloudLab/distribution
Specify and implement Docker-Upload-UUID
This changeset adds support for a header to identify docker upload uuids. This id can be used as a key to manage local state for resumable uploads. The goal is remove the necessity for a client to parse the url to get an upload uuid. The restrictions for clients to use the location header are still strongly in place. Signed-off-by: Stephen J Day <stephen.day@docker.com>
This commit is contained in:
Stephen J Day
parent
47a8ad7a61
commit
996235dc59
5 changed files with 72 additions and 11 deletions
|
|
@ -347,6 +347,7 @@ with the upload URL in the `Location` header:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: bytes=0-<offset>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
The rest of the upload process can be carried out with the returned url,
|
||||
|
|
@ -358,6 +359,10 @@ try to assemble the it. While the `uuid` parameter may be an actual UUID, this
|
|||
proposal imposes no constraints on the format and clients should never impose
|
||||
any.
|
||||
|
||||
If clients need to correlate local upload state with remote upload state, the
|
||||
contents of the `Docker-Upload-UUID` header should be used. Such an id can be
|
||||
used to key the last used location header when implementing resumable uploads.
|
||||
|
||||
##### Upload Progress
|
||||
|
||||
The progress and chunk coordination of the upload process will be coordinated
|
||||
|
|
@ -384,6 +389,7 @@ The response will be similar to the above, except will return 204 status:
|
|||
204 No Content
|
||||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: bytes=0-<offset>
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
Note that the HTTP `Range` header byte ranges are inclusive and that will be
|
||||
|
|
@ -453,6 +459,7 @@ current status:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: 0-<last valid range>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
If this response is received, the client should resume from the "last valid
|
||||
|
|
@ -471,6 +478,7 @@ be returned, including a `Range` header with the current upload status:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: bytes=0-<offset>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
##### Completed Upload
|
||||
|
|
@ -1787,6 +1795,7 @@ The following parameters should be specified on the request:
|
|||
201 Created
|
||||
Location: <blob location>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
The blob has been created in the registry and is available at the provided location.
|
||||
|
|
@ -1797,6 +1806,7 @@ The following headers will be returned with the response:
|
|||
|----|-----------|
|
||||
|`Location`||
|
||||
|`Content-Length`|The `Content-Length` header must be zero and the body must be empty.|
|
||||
|`Docker-Upload-UUID`|Identifies the docker upload uuid for the current request.|
|
||||
|
||||
|
||||
|
||||
|
|
@ -1890,6 +1900,7 @@ The following parameters should be specified on the request:
|
|||
Content-Length: 0
|
||||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: 0-0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
The upload has been created. The `Location` header must be used to complete the upload. The response should be identical to a `GET` request on the contents of the returned `Location` header.
|
||||
|
|
@ -1901,6 +1912,7 @@ The following headers will be returned with the response:
|
|||
|`Content-Length`|The `Content-Length` header must be zero and the body must be empty.|
|
||||
|`Location`|The location of the created upload. Clients should use the contents verbatim to complete the upload, adding parameters where required.|
|
||||
|`Range`|Range header indicating the progress of the upload. When starting an upload, it will return an empty range, since no content has been received.|
|
||||
|`Docker-Upload-UUID`|Identifies the docker upload uuid for the current request.|
|
||||
|
||||
|
||||
|
||||
|
|
@ -2004,6 +2016,7 @@ The following parameters should be specified on the request:
|
|||
204 No Content
|
||||
Range: 0-<offset>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
The upload is known and in progress. The last received offset is available in the `Range` header.
|
||||
|
|
@ -2014,6 +2027,7 @@ The following headers will be returned with the response:
|
|||
|----|-----------|
|
||||
|`Range`|Range indicating the current progress of the upload.|
|
||||
|`Content-Length`|The `Content-Length` header must be zero and the body must be empty.|
|
||||
|`Docker-Upload-UUID`|Identifies the docker upload uuid for the current request.|
|
||||
|
||||
|
||||
|
||||
|
|
@ -2161,6 +2175,7 @@ The following parameters should be specified on the request:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: 0-<offset>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
The chunk of data has been accepted and the current progress is available in the range header. The updated upload location is available in the `Location` header.
|
||||
|
|
@ -2172,6 +2187,7 @@ The following headers will be returned with the response:
|
|||
|`Location`|The location of the upload. Clients should assume this changes after each request. Clients should use the contents verbatim to complete the upload, adding parameters where required.|
|
||||
|`Range`|Range indicating the current progress of the upload.|
|
||||
|`Content-Length`|The `Content-Length` header must be zero and the body must be empty.|
|
||||
|`Docker-Upload-UUID`|Identifies the docker upload uuid for the current request.|
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -347,6 +347,7 @@ with the upload URL in the `Location` header:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: bytes=0-<offset>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
The rest of the upload process can be carried out with the returned url,
|
||||
|
|
@ -358,6 +359,10 @@ try to assemble the it. While the `uuid` parameter may be an actual UUID, this
|
|||
proposal imposes no constraints on the format and clients should never impose
|
||||
any.
|
||||
|
||||
If clients need to correlate local upload state with remote upload state, the
|
||||
contents of the `Docker-Upload-UUID` header should be used. Such an id can be
|
||||
used to key the last used location header when implementing resumable uploads.
|
||||
|
||||
##### Upload Progress
|
||||
|
||||
The progress and chunk coordination of the upload process will be coordinated
|
||||
|
|
@ -384,6 +389,7 @@ The response will be similar to the above, except will return 204 status:
|
|||
204 No Content
|
||||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: bytes=0-<offset>
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
Note that the HTTP `Range` header byte ranges are inclusive and that will be
|
||||
|
|
@ -453,6 +459,7 @@ current status:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: 0-<last valid range>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
If this response is received, the client should resume from the "last valid
|
||||
|
|
@ -471,6 +478,7 @@ be returned, including a `Range` header with the current upload status:
|
|||
Location: /v2/<name>/blobs/uploads/<uuid>
|
||||
Range: bytes=0-<offset>
|
||||
Content-Length: 0
|
||||
Docker-Upload-UUID: <uuid>
|
||||
```
|
||||
|
||||
##### Completed Upload
|
||||
|
|
|
|||
|
|
@ -72,6 +72,13 @@ var (
|
|||
Format: "0",
|
||||
}
|
||||
|
||||
dockerUploadUUIDHeader = ParameterDescriptor{
|
||||
Name: "Docker-Upload-UUID",
|
||||
Description: "Identifies the docker upload uuid for the current request.",
|
||||
Type: "uuid",
|
||||
Format: "<uuid>",
|
||||
}
|
||||
|
||||
unauthorizedResponse = ResponseDescriptor{
|
||||
Description: "The client does not have access to the repository.",
|
||||
StatusCode: http.StatusUnauthorized,
|
||||
|
|
@ -898,6 +905,7 @@ var routeDescriptors = []RouteDescriptor{
|
|||
Format: "<blob location>",
|
||||
},
|
||||
contentLengthZeroHeader,
|
||||
dockerUploadUUIDHeader,
|
||||
},
|
||||
},
|
||||
},
|
||||
|
|
@ -941,6 +949,7 @@ var routeDescriptors = []RouteDescriptor{
|
|||
Format: "0-0",
|
||||
Description: "Range header indicating the progress of the upload. When starting an upload, it will return an empty range, since no content has been received.",
|
||||
},
|
||||
dockerUploadUUIDHeader,
|
||||
},
|
||||
},
|
||||
},
|
||||
|
|
@ -994,6 +1003,7 @@ var routeDescriptors = []RouteDescriptor{
|
|||
Description: "Range indicating the current progress of the upload.",
|
||||
},
|
||||
contentLengthZeroHeader,
|
||||
dockerUploadUUIDHeader,
|
||||
},
|
||||
},
|
||||
},
|
||||
|
|
@ -1077,6 +1087,7 @@ var routeDescriptors = []RouteDescriptor{
|
|||
Description: "Range indicating the current progress of the upload.",
|
||||
},
|
||||
contentLengthZeroHeader,
|
||||
dockerUploadUUIDHeader,
|
||||
},
|
||||
},
|
||||
},
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import (
|
|||
"net/http/httputil"
|
||||
"net/url"
|
||||
"os"
|
||||
"path"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
|
|
@ -97,8 +98,20 @@ func TestLayerAPI(t *testing.T) {
|
|||
checkResponse(t, "checking head on non-existent layer", resp, http.StatusNotFound)
|
||||
|
||||
// ------------------------------------------
|
||||
// Start an upload and cancel
|
||||
uploadURLBase := startPushLayer(t, env.builder, imageName)
|
||||
// Start an upload, check the status then cancel
|
||||
uploadURLBase, uploadUUID := startPushLayer(t, env.builder, imageName)
|
||||
|
||||
// A status check should work
|
||||
resp, err = http.Get(uploadURLBase)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error getting upload status: %v", err)
|
||||
}
|
||||
checkResponse(t, "status of deleted upload", resp, http.StatusNoContent)
|
||||
checkHeaders(t, resp, http.Header{
|
||||
"Location": []string{"*"},
|
||||
"Range": []string{"0-0"},
|
||||
"Docker-Upload-UUID": []string{uploadUUID},
|
||||
})
|
||||
|
||||
req, err := http.NewRequest("DELETE", uploadURLBase, nil)
|
||||
if err != nil {
|
||||
|
|
@ -121,7 +134,7 @@ func TestLayerAPI(t *testing.T) {
|
|||
|
||||
// -----------------------------------------
|
||||
// Do layer push with an empty body and different digest
|
||||
uploadURLBase = startPushLayer(t, env.builder, imageName)
|
||||
uploadURLBase, uploadUUID = startPushLayer(t, env.builder, imageName)
|
||||
resp, err = doPushLayer(t, env.builder, imageName, layerDigest, uploadURLBase, bytes.NewReader([]byte{}))
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error doing bad layer push: %v", err)
|
||||
|
|
@ -137,7 +150,7 @@ func TestLayerAPI(t *testing.T) {
|
|||
t.Fatalf("unexpected error digesting empty buffer: %v", err)
|
||||
}
|
||||
|
||||
uploadURLBase = startPushLayer(t, env.builder, imageName)
|
||||
uploadURLBase, uploadUUID = startPushLayer(t, env.builder, imageName)
|
||||
pushLayer(t, env.builder, imageName, zeroDigest, uploadURLBase, bytes.NewReader([]byte{}))
|
||||
|
||||
// -----------------------------------------
|
||||
|
|
@ -150,7 +163,7 @@ func TestLayerAPI(t *testing.T) {
|
|||
t.Fatalf("unexpected error digesting empty tar: %v", err)
|
||||
}
|
||||
|
||||
uploadURLBase = startPushLayer(t, env.builder, imageName)
|
||||
uploadURLBase, uploadUUID = startPushLayer(t, env.builder, imageName)
|
||||
pushLayer(t, env.builder, imageName, emptyDigest, uploadURLBase, bytes.NewReader(emptyTar))
|
||||
|
||||
// ------------------------------------------
|
||||
|
|
@ -158,7 +171,7 @@ func TestLayerAPI(t *testing.T) {
|
|||
layerLength, _ := layerFile.Seek(0, os.SEEK_END)
|
||||
layerFile.Seek(0, os.SEEK_SET)
|
||||
|
||||
uploadURLBase = startPushLayer(t, env.builder, imageName)
|
||||
uploadURLBase, uploadUUID = startPushLayer(t, env.builder, imageName)
|
||||
pushLayer(t, env.builder, imageName, layerDigest, uploadURLBase, layerFile)
|
||||
|
||||
// ------------------------
|
||||
|
|
@ -284,7 +297,7 @@ func TestManifestAPI(t *testing.T) {
|
|||
expectedLayers[dgst] = rs
|
||||
unsignedManifest.FSLayers[i].BlobSum = dgst
|
||||
|
||||
uploadURLBase := startPushLayer(t, env.builder, imageName)
|
||||
uploadURLBase, _ := startPushLayer(t, env.builder, imageName)
|
||||
pushLayer(t, env.builder, imageName, dgst, uploadURLBase, rs)
|
||||
}
|
||||
|
||||
|
|
@ -411,7 +424,7 @@ func putManifest(t *testing.T, msg, url string, v interface{}) *http.Response {
|
|||
return resp
|
||||
}
|
||||
|
||||
func startPushLayer(t *testing.T, ub *v2.URLBuilder, name string) string {
|
||||
func startPushLayer(t *testing.T, ub *v2.URLBuilder, name string) (location string, uuid string) {
|
||||
layerUploadURL, err := ub.BuildBlobUploadURL(name)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error building layer upload url: %v", err)
|
||||
|
|
@ -424,12 +437,20 @@ func startPushLayer(t *testing.T, ub *v2.URLBuilder, name string) string {
|
|||
defer resp.Body.Close()
|
||||
|
||||
checkResponse(t, fmt.Sprintf("pushing starting layer push %v", name), resp, http.StatusAccepted)
|
||||
|
||||
u, err := url.Parse(resp.Header.Get("Location"))
|
||||
if err != nil {
|
||||
t.Fatalf("error parsing location header: %v", err)
|
||||
}
|
||||
|
||||
uuid = path.Base(u.Path)
|
||||
checkHeaders(t, resp, http.Header{
|
||||
"Location": []string{"*"},
|
||||
"Content-Length": []string{"0"},
|
||||
"Location": []string{"*"},
|
||||
"Content-Length": []string{"0"},
|
||||
"Docker-Upload-UUID": []string{uuid},
|
||||
})
|
||||
|
||||
return resp.Header.Get("Location")
|
||||
return resp.Header.Get("Location"), uuid
|
||||
}
|
||||
|
||||
// doPushLayer pushes the layer content returning the url on success returning
|
||||
|
|
|
|||
|
|
@ -138,6 +138,8 @@ func (luh *layerUploadHandler) StartLayerUpload(w http.ResponseWriter, r *http.R
|
|||
luh.Errors.Push(v2.ErrorCodeUnknown, err)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Docker-Upload-UUID", luh.Upload.UUID())
|
||||
w.WriteHeader(http.StatusAccepted)
|
||||
}
|
||||
|
||||
|
|
@ -155,6 +157,7 @@ func (luh *layerUploadHandler) GetUploadStatus(w http.ResponseWriter, r *http.Re
|
|||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Docker-Upload-UUID", luh.UUID)
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
|
|
@ -235,6 +238,7 @@ func (luh *layerUploadHandler) CancelLayerUpload(w http.ResponseWriter, r *http.
|
|||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Docker-Upload-UUID", luh.UUID)
|
||||
if err := luh.Upload.Cancel(); err != nil {
|
||||
ctxu.GetLogger(luh).Errorf("error encountered canceling upload: %v", err)
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
|
|
@ -277,6 +281,7 @@ func (luh *layerUploadHandler) layerUploadResponse(w http.ResponseWriter, r *htt
|
|||
return err
|
||||
}
|
||||
|
||||
w.Header().Set("Docker-Upload-UUID", luh.UUID)
|
||||
w.Header().Set("Location", uploadURL)
|
||||
w.Header().Set("Content-Length", "0")
|
||||
w.Header().Set("Range", fmt.Sprintf("0-%d", luh.State.Offset))
|
||||
|
|
|
|||
Loading…
Reference in a new issue