In order not to accidentally take outdated
information about downloaded parts from other
nodes, now when the multipart is abort or complete,
the root node of the multipart upload with the
finish flag remains in the tree.
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
With new retry policy of tree service pool, gateway should avoid
deletion of system nodes from tree. Absence of node in the tree
will trigger retry. Other storage in the network may return already
deleted node while tree is not completely synced, and client will
get unexpected result.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements
says that
"When the number of responses exceeds the value of MaxKeys,
NextVersionIdMarker specifies the first object version not returned
that satisfies the search criteria. Use this value for the
version-id-marker request parameter in a subsequent request."
the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
When object exists in tree but missing in storage, we can't remove
bucket. While storage node does not sync tree service and object
service, the only way to delete such broken bucket is to ignore
'object not found' error, clear cache and do not include missing
objects in the listing result.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
