TrueCloudLab/frostfs-s3-gw
17
3
Fork 19
Code Issues 28 Pull requests 7 Projects Releases 18 Packages Activity Actions
1408 commits 5 branches 43 tags 21 MiB
Commit graph

408 commits

Author SHA1 Message Date
Marina Biryukova
d919e6cce2 [#482] Fix containers resolving
All checks were successful
/ DCO (pull_request) Successful in 1m8s
Details
/ Builds (pull_request) Successful in 1m7s
Details
/ Vulncheck (pull_request) Successful in 1m12s
Details
/ Lint (pull_request) Successful in 2m24s
Details
/ Tests (pull_request) Successful in 1m24s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-05 12:33:14 +03:00
Denis Kirillov
056f168d77 [#448] multipart: Support removing duplicated parts 
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-09-03 13:20:38 +00:00
Marina Biryukova
d6b506f6d9 [#466] Implement PATCH for multipart objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
b08f476ea7 [#462] Implement PATCH for regular objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
fa68a4ce40 [#412] Store creation epoch of delete markers 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Marina Biryukova
0644067496 [#412] Store creation epoch in tree service 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Marina Biryukova
481520705a [#42] Support expiration lifecycle 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Denis Kirillov
69e77aecc9 [#451] Don't skip [Next]PartNumberMarker in response 
In ListParts response we should always keep PartNumberMarker (even if it's zero)
and NextPartNumberMarker (even if response isn't truncated)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-06 15:44:56 +03:00
Denis Kirillov
f5326b9f04 [#437] tree: Support removing old split system nodes 
It's need to fit user expectation on deleting CORS for example.
Previously after removing cors (that was uploaded in split manner)
we can still get some data (from other node)
because deletion worked only for latest node version.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-26 06:54:10 +00:00
Pavel Pogodaev
51c5c227c2 [#31] Add force bucket delete flag
All checks were successful
/ DCO (pull_request) Successful in 1m11s
Details
/ Vulncheck (pull_request) Successful in 1m22s
Details
/ Builds (1.21) (pull_request) Successful in 1m26s
Details
/ Builds (1.22) (pull_request) Successful in 2m23s
Details
/ Lint (pull_request) Successful in 2m22s
Details
/ Tests (1.21) (pull_request) Successful in 1m23s
Details
/ Tests (1.22) (pull_request) Successful in 1m25s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-07-25 14:04:54 +03:00
Denis Kirillov
6cb0026007 [#427] layer: Split FrostFS ReadObject to separate methods
All checks were successful
/ Builds (1.21) (pull_request) Successful in 1m20s
Details
/ Builds (1.22) (pull_request) Successful in 1m15s
Details
/ Vulncheck (pull_request) Successful in 1m6s
Details
/ DCO (pull_request) Successful in 57s
Details
/ Lint (pull_request) Successful in 2m59s
Details
/ Tests (1.21) (pull_request) Successful in 1m9s
Details
/ Tests (1.22) (pull_request) Successful in 1m23s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-23 16:53:59 +03:00
Marina Biryukova
971006a28c [#422] Support separate container for CORS 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-23 12:33:29 +00:00
Denis Kirillov
977a20760b [#430] Delete all split version at once
All checks were successful
/ DCO (pull_request) Successful in 8m40s
Details
/ Vulncheck (pull_request) Successful in 13m10s
Details
/ Builds (1.21) (pull_request) Successful in 14m11s
Details
/ Builds (1.22) (pull_request) Successful in 14m18s
Details
/ Lint (pull_request) Successful in 16m31s
Details
/ Tests (1.21) (pull_request) Successful in 10m48s
Details
/ Tests (1.22) (pull_request) Successful in 14m31s
Details 
Previously after split we can get two `null` versioned object with the same key
and deleting such key removes only one node/object.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-19 11:26:51 +03:00
Denis Kirillov
465eaa816a [#372] Drop [e]ACL related code
All checks were successful
/ DCO (pull_request) Successful in 2m15s
Details
/ Vulncheck (pull_request) Successful in 2m55s
Details
/ Builds (1.20) (pull_request) Successful in 3m46s
Details
/ Builds (1.21) (pull_request) Successful in 3m48s
Details
/ Lint (pull_request) Successful in 5m26s
Details
/ Tests (1.20) (pull_request) Successful in 3m34s
Details
/ Tests (1.21) (pull_request) Successful in 3m18s
Details 
Always consider buckets as APE compatible

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-01 16:58:44 +03:00
Denis Kirillov
414f3943e2 [#410] Drop layer.Client interface
All checks were successful
/ DCO (pull_request) Successful in 2m1s
Details
/ Vulncheck (pull_request) Successful in 2m31s
Details
/ Builds (1.20) (pull_request) Successful in 2m39s
Details
/ Builds (1.21) (pull_request) Successful in 2m31s
Details
/ Lint (pull_request) Successful in 3m14s
Details
/ Tests (1.20) (pull_request) Successful in 2m34s
Details
/ Tests (1.21) (pull_request) Successful in 2m10s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-25 15:57:55 +03:00
Denis Kirillov
9432782ce6 [#401] Drop notifications
All checks were successful
/ DCO (pull_request) Successful in 2m5s
Details
/ Builds (1.20) (pull_request) Successful in 2m40s
Details
/ Builds (1.21) (pull_request) Successful in 2m33s
Details
/ Vulncheck (pull_request) Successful in 2m22s
Details
/ Lint (pull_request) Successful in 4m24s
Details
/ Tests (1.20) (pull_request) Successful in 2m48s
Details
/ Tests (1.21) (pull_request) Successful in 2m45s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-25 15:49:37 +03:00
Marina Biryukova
2ab655b909 [#380] Add test for credentials versioning 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-05-03 07:24:13 +00:00
Marina Biryukova
e22ff52165 [#367] Add check of AccessBox attributes 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-19 06:25:26 +00:00
Marina Biryukova
3ff027587c [#357] Add check of request and resource tags 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-17 07:06:58 +00:00
Denis Kirillov
8307c73fef [#364] Fix removing combined object
Some checks failed
/ Vulncheck (pull_request) Failing after 3m8s
Details
/ DCO (pull_request) Successful in 3m49s
Details
/ Builds (1.20) (pull_request) Successful in 5m35s
Details
/ Builds (1.21) (pull_request) Successful in 4m16s
Details
/ Lint (pull_request) Successful in 6m55s
Details
/ Tests (1.20) (pull_request) Successful in 5m14s
Details
/ Tests (1.21) (pull_request) Successful in 4m29s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-12 14:56:38 +03:00
Denis Kirillov
8669bf6b50 [#346] acl: Update APE and fix using
All checks were successful
/ DCO (pull_request) Successful in 2m57s
Details
/ Vulncheck (pull_request) Successful in 3m33s
Details
/ Lint (pull_request) Successful in 4m44s
Details
/ Tests (1.20) (pull_request) Successful in 3m38s
Details
/ Tests (1.21) (pull_request) Successful in 3m29s
Details
/ Builds (1.20) (pull_request) Successful in 1m12s
Details
/ Builds (1.21) (pull_request) Successful in 3m23s
Details 
* Remove native policy when remove bucket policy
* Allow policies that contain only s3 compatible statements
(now deny rules cannot be converted to native rules)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-02 12:43:04 +00:00
Denis Kirillov
80c7b73eb9 [#306] In APE buckets forbid canned acl except private
Some checks failed
/ DCO (pull_request) Successful in 2m50s
Details
/ Vulncheck (pull_request) Failing after 3m15s
Details
/ Builds (1.20) (pull_request) Successful in 3m39s
Details
/ Builds (1.21) (pull_request) Successful in 3m41s
Details
/ Lint (pull_request) Successful in 5m48s
Details
/ Tests (1.20) (pull_request) Successful in 4m0s
Details
/ Tests (1.21) (pull_request) Successful in 3m53s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-19 16:57:26 +03:00
Denis Kirillov
f958eef2b3 [#325] Use default empty data.LockInfo in get/head in case of error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
8050ca2d51 [#306] Use session token for container read operations
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Vulncheck (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m59s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m19s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 18:14:33 +03:00
Denis Kirillov
c12e264697 [#306] Simplify cid resolver for metrics 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 17:46:16 +03:00
Denis Kirillov
e9f38a49e4 [#306] Fix forming key for bucket cache 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 16:09:40 +03:00
Marina Biryukova
31da31862a [#300] Update error logging in DeleteMultipleObjects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-02-29 14:24:32 +00:00
Denis Kirillov
bac1b3fb2d [#306] Use zero basic acl to mark APE containers 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
3d0d2032c6 [#306] acl: Handle put/get acl for APE buckets 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
1f2cf0ed67 [#306] Use APE instead of eACL on bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
a74d498df2 [#165] Return sort after HEAD in listing
Some checks failed
/ DCO (pull_request) Successful in 1m16s
Details
/ Builds (1.20) (pull_request) Successful in 2m9s
Details
/ Builds (1.21) (pull_request) Successful in 1m39s
Details
/ Vulncheck (pull_request) Successful in 1m52s
Details
/ Lint (pull_request) Failing after 1m47s
Details
/ Tests (1.20) (pull_request) Successful in 2m27s
Details
/ Tests (1.21) (pull_request) Successful in 2m13s
Details 
We have to sort object after HEAD because we make request in different goroutines,
so the order is not deterministic.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:08 +03:00
Denis Kirillov
69297a4a38 [#165] Delete object from tree in case of storage error 
Extend storage node errors in case of which we continue deleting from tree
with 'object not found' error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:08 +03:00
Denis Kirillov
71d82d1cc8 [#165] Fix lint issues 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:08 +03:00
Denis Kirillov
fafe4af529 [#165] Fix real object size in listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:03 +03:00
Denis Kirillov
88f1acbdfc [#165] Cancel context in outdated list session 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
4e15452853 [#165] Fix lint errors 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
da642a498a [#165] Listing fix data race 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
2d7973b3f1 [#165] Refactor list versions 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
6d52f46012 [#165] Fix v1 listing bookmark 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
0ae49eaab0 [#165] Generalize allObjectListingParams 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
093de13f54 [#165] Add stream listing tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
cf4fc3b602 [#165] Extend error on getting listing containers not in current namespace 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
c7ee628ab0 [#165] Fix versions listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
4ad84b9b94 [#165] listing: Use NodeVersion instead of ObjectInfo 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
3e20f736a6 [#165] Move listing function to one file 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
b52552e8c2 [#165] Add batching in streamin listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
6e8960b2ab [#165] Add list session cache 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
29ac91dfd5 [#165] Support streaming listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Roman Loginov
6f9ee3da76 [#275] Change logic delete multipart upload
All checks were successful
/ Vulncheck (pull_request) Successful in 1m13s
Details
/ DCO (pull_request) Successful in 1m43s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m36s
Details
/ Lint (pull_request) Successful in 3m33s
Details
/ Tests (1.20) (pull_request) Successful in 2m34s
Details
/ Tests (1.21) (pull_request) Successful in 2m29s
Details 
In order not to accidentally take outdated
information about downloaded parts from other
nodes, now when the multipart is abort or complete,
the root node of the multipart upload with the
finish flag remains in the tree.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-12-27 13:06:45 +03:00
Denis Kirillov
43cae9ee04 [#248] Correct object versions response markers
All checks were successful
/ DCO (pull_request) Successful in 1m40s
Details
/ Vulncheck (pull_request) Successful in 1m42s
Details
/ Builds (1.20) (pull_request) Successful in 2m29s
Details
/ Builds (1.21) (pull_request) Successful in 1m37s
Details
/ Lint (pull_request) Successful in 3m55s
Details
/ Tests (1.20) (pull_request) Successful in 2m23s
Details
/ Tests (1.21) (pull_request) Successful in 2m16s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-21 10:53:50 +03:00