alexvanin/frostfs-s3-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-s3-gw
Code Issues Pull requests Projects Releases Packages Wiki Activity
1436 commits 4 branches 0 tags 19 MiB
Commit graph

73 commits

Author SHA1 Message Date
Nikita Zinkevich
9fadfbbc2f [#488] Renamed api/errors, layer/frostfs and layer/tree package names 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
827ea1a41e [#488] Move layer/frostfs.go to layer/frostfs/frostfs.go 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
968f10a72f [#488] Move layer/tree_service.go to layer/tree/tree_service.go 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Marina Biryukova
f187141ae5 [#486] Fix PUT object with negative Content-Length 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-16 08:45:46 +00:00
Marina Biryukova
d0e4d55772 [#460] Add network info cache 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-13 09:56:24 +00:00
Denis Kirillov
056f168d77 [#448] multipart: Support removing duplicated parts 
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-09-03 13:20:38 +00:00
Marina Biryukova
0644067496 [#412] Store creation epoch in tree service 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Denis Kirillov
69e77aecc9 [#451] Don't skip [Next]PartNumberMarker in response 
In ListParts response we should always keep PartNumberMarker (even if it's zero)
and NextPartNumberMarker (even if response isn't truncated)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-06 15:44:56 +03:00
Denis Kirillov
465eaa816a [#372] Drop [e]ACL related code 
Always consider buckets as APE compatible

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-01 16:58:44 +03:00
Denis Kirillov
414f3943e2 [#410] Drop layer.Client interface 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-25 15:57:55 +03:00
Roman Loginov
6f9ee3da76 [#275] Change logic delete multipart upload 
In order not to accidentally take outdated
information about downloaded parts from other
nodes, now when the multipart is abort or complete,
the root node of the multipart upload with the
finish flag remains in the tree.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-12-27 13:06:45 +03:00
Roman Loginov
861454e499 [#218] Add check content sha256 header 
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-11-22 11:33:52 +00:00
Marina Biryukova
b28ecef43b [#219] Return ETag in quotes 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-11-22 11:12:32 +00:00
Marina Biryukova
fe796ba538 [#217] Consider Copy-Source-SSE-* headers during copy 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-11-13 13:22:58 +00:00
Marina Biryukova
25bb581fee [#205] Add md5 checksum in header 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-25 11:04:19 +03:00
Marina Biryukova
51e591877b [#207] Fix list parts with empty list 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-21 11:27:20 +03:00
Marina Biryukova
12cf29aed2 [#207] Fix part-number-marker handling 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-19 12:43:07 +03:00
Denis Kirillov
6617adc22b [#185] Use correct object size when object is combined or encrypted 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-24 18:03:08 +03:00
Roman Loginov
8efcc957ea [#96] Move log messages to constants 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-08-23 18:32:31 +03:00
Denis Kirillov
18878b66d3 [#175] Use gate owner as object owner 
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-08 12:22:33 +03:00
Denis Kirillov
bcf5a85aab [#63] multipart: Fix copying 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-07-20 14:00:15 +03:00
Denis Kirillov
ad81b599dd [#63] Add fast multipart upload 
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-07-20 14:00:15 +03:00
Denis Kirillov
14ef9ff091 [#158] Separate init object reader from read itself 
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-07-11 17:32:05 +03:00
Denis Kirillov
d531b13866 [#143] Add more context to some s3 errors 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-30 12:08:33 +03:00
Denis Kirillov
23593eee3d [#111] Use request scope logger 
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-19 13:54:51 +03:00
Denis Kirillov
4a6e3a19ce [#125] Handle negative Content-Length on put 
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-02 14:16:19 +00:00
Artem Tataurov
e487ee5b7d [#70] Add arrays of copies numbers for location constraints 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-03 13:48:26 +03:00
Alex Vanin
813aa2f173 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:38:08 +03:00
Artem Tataurov
8e14ccad9f [#754] Remove duplicating CID/OID logging calls 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7265fecc897a2526419bb748847c668)
 2022-12-28 17:06:32 +03:00
Artem Tataurov
af7fbd6f32 [#754] Add CID and OID to logs 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2022-12-28 17:06:32 +03:00
Denis Kirillov
388482e230 [#2] Rename internals 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-21 11:17:45 +03:00
Alex Vanin
96dff367db [#1] Build S3 Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 12:43:52 +03:00
Denis Kirillov
094eb12578 [#726] Use client time on regular requests 
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-14 14:53:55 +03:00
Angira Kekteeva
da61fd8bcb [#728] Add error message to AbortMultipartUpload 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-10-25 19:04:11 +03:00
Denis Kirillov
821df3d648 [#669] Optimize getNodeVersion for tags operation 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-20 17:56:09 +03:00
Denis Kirillov
3d31c2ab4a [#713] Fix cache access 
User can access only keys that he puts into caches

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 15:26:42 +03:00
Denis Kirillov
80beedf13e [#504] Use bktInfo in tree service to check owner 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov
c3ad6d2faf [#637] Add header to override CopiesNumber 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-19 10:31:26 +03:00
Angira Kekteeva
5307211398 [#634] Add CopiesNumber in NeoFS requests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-16 17:49:15 +03:00
Denis Kirillov
94a6a55919 [#595] Move decrypter to separate package 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
87c05eb514 [#595] Add SSE with customer key 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Alex Vanin
5219965105 [#498] Remove log string sanitizer 
`zap` lib does all the job anyway.
Suppress CWE-117 from CodeQL.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-29 09:32:37 +03:00
Denis Kirillov
c88a9842db [#488] Sync using oid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
85c203e157 [#488] Sync using cid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
42a761c1f2 [#547] Cache ObjectInfo instead of Object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
3f5afabd4a [#515] Fix nil reference when a part is missing 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9dfc7e043f [#476] Save full parts info to multipart object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
c8e8ba9f6a [#476] Fix parts info for GetObjectAttributes 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9b1ccd39be [#475] Add extra attributes to partInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
1a456eaa8b [#452] Remove unused 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00