dkirillov/frostfs-s3-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-s3-gw
Code Issues Pull requests Projects Releases Packages Wiki Activity
877 commits 13 branches 31 tags 19 MiB
Commit graph

123 commits

Author SHA1 Message Date
Denis Kirillov
6aab3936d9 [#625] Limit listing generator with maxKeys 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-16 13:47:37 +03:00
Denis Kirillov
3824151699 [#616] Reduce number of requests during listing 
Check if object is directory before request to NeoFS

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-16 13:47:37 +03:00
Denis Kirillov
94a6a55919 [#595] Move decrypter to separate package 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
87c05eb514 [#595] Add SSE with customer key 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Angira Kekteeva
f66e4a77cd [#577] Add IsDeleteMarker() to NodeVersion 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
2a933f48be [#577] Remove redundant set of isLatest 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
245e64900d [#577] Replace ObjInfo with ExtObjInfo in cache 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
37c46e926e [#577] Add Version() to ExtendedObjectInfo 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Denis Kirillov
5dd658302d [#612] Fix comment 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-26 17:36:21 +03:00
Denis Kirillov
b2e8b1cfb3 [#612] Make listing more robust 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-26 17:36:21 +03:00
Angira Kekteeva
4483c6f57a [#619] Filter system metadata 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-26 15:37:13 +03:00
Alex Vanin
e0136feb73 [#611] Split name processing from object info creation procedure 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-25 14:23:30 +03:00
Alex Vanin
97f81d3270 [#611] Process delimiters on the object info cache hit 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-25 14:23:30 +03:00
Denis Kirillov
85eacdb970 [#602] Fix removal non-empty bucket 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
0057f6b7db [#546] Add size and etag in nodeVersionInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
72013e08ec [#569] Refactor versioning in tree service 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
88c392d024 [#490] Optimize GetObjectTaggingAndLock 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
c88a9842db [#488] Sync using oid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
85c203e157 [#488] Sync using cid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
42a761c1f2 [#547] Cache ObjectInfo instead of Object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
5984f2c2bc [#525] Add comment to clarify anonymous func 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
ad95d1745c [#525] Parallelize listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
12a2060dd0 [#524] Optimize listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
4f3d206422 [#478] Don't fetch epoch at object upload 
Creation epoch was used for versioning. With
tree service, versioning is done on the NeoFS
side in the tree.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
a02900a4f7 [#474] Use appropriate null version during listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9b1ccd39be [#475] Add extra attributes to partInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
1a456eaa8b [#452] Remove unused 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9fb3fb1274 [#452] Fix versions tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
dd534e8738 [#451] Handle lock objects using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
70957d75fd [#417] Complete multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9c74cca9af [#420] Using tree service to list object versions 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
25477cdaf8 [#416] Use tree service to list objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
37b1baed41 [#413] Refactor 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
977f176713 [#413] Use tree service to delete object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
49bd77d9cf [#413] Use tree service to head objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
36f3c43af5 [#413] Use tree service to put object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9f740b9683 [#289] Add detect mimetype by extension 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-29 19:13:16 +03:00
Denis Kirillov
7ca519cb32 [#539] Add context to errors 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-27 02:23:19 +04:00
Denis Kirillov
f00ca1b6c4 [#438] Drop layer/neofs package 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-07 13:59:45 +03:00
Angira Kekteeva
e904ed51c7 [#487] Optimize bucketInfo in initObjectPayloadReader 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-03 10:57:56 +03:00
Angira Kekteeva
4767eeed8c [#487] Remove attach of bearer token 
When bucket owner is not an issuer of the bearer token

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-03 10:57:56 +03:00
Alex Vanin
12d9eb62cb [#489] Sanitize log records that may contain user input 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-06-02 19:42:31 +04:00
Denis Kirillov
933ef2bc71 [#441] Remove unused 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-05-31 14:48:58 +03:00
Denis Kirillov
2bca4755f9 [#441] Optimize put objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-05-31 14:48:58 +03:00
Leonard Lyubich
087d500c5f [#458] *: Refactor working with NeoFS identities 
Pull latest changes from NeoFS SDK Go library. Decrease redundant and
unsafe usage of ID pointers. Use `EncodeToString` method in order to
calculate protocol strings.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 17:22:37 +04:00
Alex Vanin
385437aace [#441] Don't produce extra object.Head request at regular object upload 
Hash can be calculated locally in S3 gateway.
Creation epoch used for versioning and will be
fetched during get and list requests. To avoid
conflicts, put method do not update cache anymore.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-05-20 10:56:36 +03:00
Alex Vanin
b778c2e072 [#431] Fix ETag value 
ETag is a string with SHA256 of NeoFS object payload.
Hash is taken from object header by neofs-sdk-go getter.
Checksum type in neofs-sdk-go has changes `String()`
output from `<hash>` to `SHA256:<hash>` in latest update.

S3 gateway should not be relied on unstable string format
implementations and use raw value.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-05-12 21:19:59 +04:00
Denis Kirillov
e3c16a32dd [#409] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-26 12:51:52 +04:00
Denis Kirillov
6e91074b50 [#367] Check errors using status 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-22 15:30:42 +04:00
Denis Kirillov
3c5c2f20d8 [#399] Use bearer token for system objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-13 16:44:32 +04:00