Commit graph

345 commits

Author SHA1 Message Date
51e591877b [#207] Fix list parts with empty list
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-21 11:27:20 +03:00
a4c612614a [#210] Fix multipart object reader
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 16:30:08 +03:00
12cf29aed2 [#207] Fix part-number-marker handling
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
6617adc22b [#185] Use correct object size when object is combined or encrypted
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
8efcc957ea [#96] Move log messages to constants
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b728fef87 [#192] Add tests to make sure client_cut flag is passed to sdk
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
6b1f365e65 [#192] Support client cut
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
94a42fa273 [#51] Update frostfs-sdk-go
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
40d7f844e3 [#137] Refactor context data retrievers
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
18878b66d3 [#175] Use gate owner as object owner
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
fe897ec588 [#178] wrapReader: Fix goroutine leak
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-01 17:59:34 +03:00
1a09041cd1 [#63] Simplify multiObjectReader and add tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
bcf5a85aab [#63] multipart: Fix copying
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
ad81b599dd [#63] Add fast multipart upload
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
14ef9ff091 [#158] Separate init object reader from read itself
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:32:05 +03:00
fc90981c03 [#149] Update inner imports after moving middlewares
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
d531b13866 [#143] Add more context to some s3 errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:08:33 +03:00
be03c5178f [#143] Fix NoSuchKey error on get/head
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
23593eee3d [#111] Use request scope logger
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
7a380fa46c [#135] frostfs: Add SEARCH operation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:41 +03:00
462589fc0c [#103] Return 504 http code on timeout errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-14 09:16:21 +03:00
4a6e3a19ce [#125] Handle negative Content-Length on put
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
34bbbcf1ed [#78] Do not show objects missing from object service
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
fae03c2b50 [#78] Process 'not found' error when object exists in tree
When object exists in tree but missing in storage, we can't remove
bucket. While storage node does not sync tree service and object
service, the only way to delete such broken bucket is to ignore
'object not found' error, clear cache and do not include missing
objects in the listing result.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 15:31:22 +03:00
81e860481d [#122] Fix linter warnings
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-31 19:43:03 +03:00
a0f0d792b8 [#78] layer: Clean up already removed object from tree
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-24 17:42:46 +03:00
3fbad97ae4 [#1] Rename files with mentions of previous project
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-19 07:30:04 +00:00
Denis Kirillov
b366e75366 [#81] Use impersonate bearer token
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-03 17:22:52 +03:00
e487ee5b7d [#70] Add arrays of copies numbers for location constraints
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
70ec5a0a5b [#83] Don't create extra delete marker
We shouldn't create delete marker if:
1. object doesn't exist at all
2. last version is already a delete marker

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-19 17:56:11 +03:00
fb99b26209 [#57] api: Clean up List and Name caches when object is missing in Tree service 2023-04-04 17:15:04 +03:00
a0937126cb [#64] Support new system attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-22 11:46:58 +03:00
813aa2f173 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 17:38:08 +03:00
740acadd37 [TrueCloudLab#36] Fix cors object payload
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-20 19:22:48 +03:00
8ac630ee71 [TrueCloudLab#25] Add zone data to BucketInfo
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-02-17 13:47:48 +03:00
8e14ccad9f [#754] Remove duplicating CID/OID logging calls
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7265fecc897a2526419bb748847c668)
2022-12-28 17:06:32 +03:00
af7fbd6f32 [#754] Add CID and OID to logs
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2022-12-28 17:06:32 +03:00
000d9ed42a [#2] Update issue links
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-21 11:17:45 +03:00
388482e230 [#2] Rename internals
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-21 11:17:45 +03:00
96dff367db [#1] Build S3 Gateway with FrostFS dependencies
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-15 12:43:52 +03:00
Angira Kekteeva
f5fe9a9b4b [#741] Fix data race
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-11-16 18:19:31 +03:00
Denis Kirillov
094eb12578 [#726] Use client time on regular requests
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 14:53:55 +03:00
Denis Kirillov
16ad0cadcb [#733] Fix container name
User system name instead of regular.
Set name as CID if name attribute is missed

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-26 17:52:02 +03:00
Angira Kekteeva
da61fd8bcb [#728] Add error message to AbortMultipartUpload
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-10-25 19:04:11 +03:00
Denis Kirillov
821df3d648 [#669] Optimize getNodeVersion for tags operation
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-20 17:56:09 +03:00
Denis Kirillov
90eb4f0188 [#713] Add check access control for system caches
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 15:26:42 +03:00
Denis Kirillov
cb55d36063 [#713] Update tests
Add bearer token to test context

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 15:26:42 +03:00
Denis Kirillov
3d31c2ab4a [#713] Fix cache access
User can access only keys that he puts into caches

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 15:26:42 +03:00
Denis Kirillov
2a41929be3 [#702] Reload resolvers and TLS certs on SIGHUP
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-28 15:07:31 +03:00
Denis Kirillov
be4600988e [#709] Add expiration attribute for legal hold
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-26 16:51:53 +03:00