Fast multipart uploads #63

New issue

Closed

opened 2023-03-21 12:11:35 +00:00 by alexvanin · 2 comments

alexvanin commented

2023-03-21 12:11:35 +00:00

Owner

S3 Gateway implements CompleteMultipartUpload this way (5c62010331):

start creating new object,
start reading payload of first parts and pipe it into new object,
finish creating new object and link new object ID with multipart object.

This is quite slow and inefficient, because in the end user awaits up to x2 upload time (for the parts and for the new object at complete multipart upload). Also it requires to delete parts, which may cause some issues in case of failures.

As the solution I propose to define special combine object. This object should be marked in the tree service and store list of object IDs. It is similar to link object. Unlike it, combine object stores IDs of any object and not just small objects. This should be safe enough, because combine objects are stored according to placement policy.

🟢 Fast complete multipart upload.
🟢 No changes in native protocol: S3 uses object names to identify objects instead of payload+header hash in native protocol, so it can't be a part of native protocol.
🟢 Removes some delete operations from multipart upload case.
🔴 Slower GET operations, especially for small objects which can be upload as multipart (see S3 Browser implementation, which uses 8MiB parts).
🔴 Requires support in HTTP Gateway to process combine objects.

Consider some extra optimizations like storing small combine objects fully in tree service.

We can create small PoC and check how badly it will affect GET operations (for small and large objects).

/cc @TrueCloudLab/storage-services-committers

S3 Gateway implements `CompleteMultipartUpload` this way (https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/commit/5c62010331ad5771fd6787ec0e1993b69c439b32): 1) start creating new object, 2) start reading payload of first parts and pipe it into new object, 3) finish creating new object and link new object ID with multipart object. This is quite slow and inefficient, because in the end user awaits up to x2 upload time (for the parts and for the new object at complete multipart upload). Also it requires to delete parts, which may cause some issues in case of failures. As the solution I propose to define special _combine_ object. This object should be marked in the tree service and store list of object IDs. It is similar to [link object](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/5fd4059061b8922d671b7e6e4615e1a865edf527/object/types.proto#L230). Unlike it, _combine_ object stores IDs of any object and not just small objects. This should be safe enough, because _combine_ objects are stored according to placement policy. 🟢 Fast complete multipart upload. 🟢 No changes in native protocol: S3 uses object names to identify objects instead of payload+header hash in native protocol, so it can't be a part of native protocol. 🟢 Removes some delete operations from multipart upload case. 🔴 Slower GET operations, especially for small objects which can be upload as multipart (see S3 Browser implementation, which uses 8MiB parts). 🔴 Requires support in HTTP Gateway to process _combine_ objects. Consider some extra optimizations like storing small _combine_ objects fully in tree service. We can create small PoC and check how badly it will affect GET operations (for small and large objects). /cc @TrueCloudLab/storage-services-committers