TrueCloudLab/frostfs-s3-gw
18
3
Fork 20
Code Issues 30 Pull requests 9 Projects Releases 23 Packages Activity Actions
1483 commits 6 branches 48 tags 24 MiB
Commit graph

810 commits

Author SHA1 Message Date
Denis Kirillov
056f168d77 [#448] multipart: Support removing duplicated parts 
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-09-03 13:20:38 +00:00
Marina Biryukova
d6b506f6d9 [#466] Implement PATCH for multipart objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
b08f476ea7 [#462] Implement PATCH for regular objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Roman Ognev
664f83b2b7 [#480] Add fuzzing tests
All checks were successful
/ DCO (pull_request) Successful in 1m34s
Details
/ Vulncheck (pull_request) Successful in 1m45s
Details
/ Builds (1.22) (pull_request) Successful in 1m51s
Details
/ Builds (1.23) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m3s
Details
/ Tests (1.22) (pull_request) Successful in 1m52s
Details
/ Tests (1.23) (pull_request) Successful in 1m51s
Details 
Signed-off-by: Roman Ognev <r.ognev@yadro.com>
 2024-09-02 15:59:07 +03:00
Denis Kirillov
d76c4fe2a2 [#472] tree: Don't use sorted GetSubTree for nodes without FileName 
Sorted GetSubTree doesn't return nodes without FileName attribute
if there are more than 1000 nodes in subtree

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-27 09:43:29 +00:00
Roman Loginov
bf00fa6aa9 [#449] Add support headers for vhs and servername 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-23 08:35:05 +00:00
Roman Loginov
ff690ce996 [#446] Add tests for address style middleware 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-23 08:35:05 +00:00
Roman Loginov
534ae7f0f1 [#446] Add support virtual-hosted-style 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-23 08:35:05 +00:00
Denis Kirillov
e73f11c251 [#452] tree: Fix logging
All checks were successful
/ DCO (pull_request) Successful in 1m6s
Details
/ Vulncheck (pull_request) Successful in 1m10s
Details
/ Builds (1.21) (pull_request) Successful in 1m32s
Details
/ Builds (1.22) (pull_request) Successful in 1m31s
Details
/ Lint (pull_request) Successful in 1m46s
Details
/ Tests (1.21) (pull_request) Successful in 1m51s
Details
/ Tests (1.22) (pull_request) Successful in 1m45s
Details 
Don't log parsing tags error in listing

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-21 10:38:36 +03:00
Marina Biryukova
5cb77018f8 [#42] Change valid time format for lifecycle configuration 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:36 +03:00
Marina Biryukova
fa68a4ce40 [#412] Store creation epoch of delete markers 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Marina Biryukova
0644067496 [#412] Store creation epoch in tree service 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Marina Biryukova
481520705a [#42] Support expiration lifecycle 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Denis Kirillov
20719bd85c [#456] PostObject: check object key for emptiness
All checks were successful
/ DCO (pull_request) Successful in 1m1s
Details
/ Vulncheck (pull_request) Successful in 1m12s
Details
/ Builds (1.21) (pull_request) Successful in 1m29s
Details
/ Builds (1.22) (pull_request) Successful in 1m33s
Details
/ Lint (pull_request) Successful in 2m20s
Details
/ Tests (1.21) (pull_request) Successful in 1m41s
Details
/ Tests (1.22) (pull_request) Successful in 1m39s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-13 09:27:26 +03:00
Denis Kirillov
4f27e34974 [#456] auth: Fix authentication for POST Object 
During post object operation field AuthHeader in middleware.Box wasn't set
that led to panic

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-12 15:25:53 +03:00
Denis Kirillov
3dc989d7fe [#451] Support Location in CompleteMultipart response
All checks were successful
/ Vulncheck (pull_request) Successful in 1m15s
Details
/ DCO (pull_request) Successful in 1m13s
Details
/ Builds (1.21) (pull_request) Successful in 1m31s
Details
/ Builds (1.22) (pull_request) Successful in 1m25s
Details
/ Lint (pull_request) Successful in 2m26s
Details
/ Tests (1.21) (pull_request) Successful in 1m40s
Details
/ Tests (1.22) (pull_request) Successful in 1m43s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-06 15:45:09 +03:00
Denis Kirillov
69e77aecc9 [#451] Don't skip [Next]PartNumberMarker in response 
In ListParts response we should always keep PartNumberMarker (even if it's zero)
and NextPartNumberMarker (even if response isn't truncated)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-06 15:44:56 +03:00
Denis Kirillov
f5326b9f04 [#437] tree: Support removing old split system nodes 
It's need to fit user expectation on deleting CORS for example.
Previously after removing cors (that was uploaded in split manner)
we can still get some data (from other node)
because deletion worked only for latest node version.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-26 06:54:10 +00:00
Pavel Pogodaev
51c5c227c2 [#31] Add force bucket delete flag
All checks were successful
/ DCO (pull_request) Successful in 1m11s
Details
/ Vulncheck (pull_request) Successful in 1m22s
Details
/ Builds (1.21) (pull_request) Successful in 1m26s
Details
/ Builds (1.22) (pull_request) Successful in 2m23s
Details
/ Lint (pull_request) Successful in 2m22s
Details
/ Tests (1.21) (pull_request) Successful in 1m23s
Details
/ Tests (1.22) (pull_request) Successful in 1m25s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-07-25 14:04:54 +03:00
Denis Kirillov
6cb0026007 [#427] layer: Split FrostFS ReadObject to separate methods
All checks were successful
/ Builds (1.21) (pull_request) Successful in 1m20s
Details
/ Builds (1.22) (pull_request) Successful in 1m15s
Details
/ Vulncheck (pull_request) Successful in 1m6s
Details
/ DCO (pull_request) Successful in 57s
Details
/ Lint (pull_request) Successful in 2m59s
Details
/ Tests (1.21) (pull_request) Successful in 1m9s
Details
/ Tests (1.22) (pull_request) Successful in 1m23s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-23 16:53:59 +03:00
Marina Biryukova
971006a28c [#422] Support separate container for CORS 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-23 12:33:29 +00:00
Marina Biryukova
3213dd7236 [#404] Add processing of encoding-type in versions listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-23 12:10:35 +00:00
Denis Kirillov
977a20760b [#430] Delete all split version at once
All checks were successful
/ DCO (pull_request) Successful in 8m40s
Details
/ Vulncheck (pull_request) Successful in 13m10s
Details
/ Builds (1.21) (pull_request) Successful in 14m11s
Details
/ Builds (1.22) (pull_request) Successful in 14m18s
Details
/ Lint (pull_request) Successful in 16m31s
Details
/ Tests (1.21) (pull_request) Successful in 10m48s
Details
/ Tests (1.22) (pull_request) Successful in 14m31s
Details 
Previously after split we can get two `null` versioned object with the same key
and deleting such key removes only one node/object.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-19 11:26:51 +03:00
Denis Kirillov
c0011ebb8d [#430] tree: Fix multipart having system name 
Previously if multipart key has the same name as some system node
(e.g. bucket-settings, bucket-cors etc.) it shadows real system node
and bucket started to be unversioned again for example.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-19 11:24:50 +03:00
Denis Kirillov
456319d2f1 [#430] Fix split tree 
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated, so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-19 11:24:46 +03:00
Alex Vanin
f86b82351a [#398] Fix parameter parsing in bucket retryer
Some checks failed
/ Builds (1.20) (pull_request) Successful in 2m7s
Details
/ Builds (1.21) (pull_request) Successful in 1m38s
Details
/ DCO (pull_request) Successful in 2m3s
Details
/ Vulncheck (pull_request) Failing after 2m23s
Details
/ Lint (pull_request) Successful in 4m21s
Details
/ Tests (1.20) (pull_request) Successful in 2m46s
Details
/ Tests (1.21) (pull_request) Successful in 2m42s
Details 
RetryStrategyExponential should use jitter backoff
instead of constant delay function

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-03 13:42:24 +03:00
Denis Kirillov
465eaa816a [#372] Drop [e]ACL related code
All checks were successful
/ DCO (pull_request) Successful in 2m15s
Details
/ Vulncheck (pull_request) Successful in 2m55s
Details
/ Builds (1.20) (pull_request) Successful in 3m46s
Details
/ Builds (1.21) (pull_request) Successful in 3m48s
Details
/ Lint (pull_request) Successful in 5m26s
Details
/ Tests (1.20) (pull_request) Successful in 3m34s
Details
/ Tests (1.21) (pull_request) Successful in 3m18s
Details 
Always consider buckets as APE compatible

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-01 16:58:44 +03:00
Denis Kirillov
77f8bdac58 [#372] Drop kludge.acl_enabled flag 
Now only APE container can be created using s3-gw

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-01 16:26:19 +03:00
Denis Kirillov
91541a432d [#411] Check uniqueness in DeleteMultipleObjects
All checks were successful
/ Builds (1.20) (pull_request) Successful in 4m57s
Details
/ Builds (1.21) (pull_request) Successful in 4m42s
Details
/ DCO (pull_request) Successful in 4m52s
Details
/ Vulncheck (pull_request) Successful in 4m38s
Details
/ Lint (pull_request) Successful in 6m46s
Details
/ Tests (1.20) (pull_request) Successful in 4m28s
Details
/ Tests (1.21) (pull_request) Successful in 4m7s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-26 16:39:06 +03:00
Denis Kirillov
943b30d9f4 [#411] Don't check object tags on deletion 
By specification https://docs.aws.amazon.com/AmazonS3/latest/userguide/tagging-and-policies.html
we shouldn't check object tags on PUT and DELETE

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-26 16:38:56 +03:00
Denis Kirillov
414f3943e2 [#410] Drop layer.Client interface
All checks were successful
/ DCO (pull_request) Successful in 2m1s
Details
/ Vulncheck (pull_request) Successful in 2m31s
Details
/ Builds (1.20) (pull_request) Successful in 2m39s
Details
/ Builds (1.21) (pull_request) Successful in 2m31s
Details
/ Lint (pull_request) Successful in 3m14s
Details
/ Tests (1.20) (pull_request) Successful in 2m34s
Details
/ Tests (1.21) (pull_request) Successful in 2m10s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-25 15:57:55 +03:00
Denis Kirillov
9432782ce6 [#401] Drop notifications
All checks were successful
/ DCO (pull_request) Successful in 2m5s
Details
/ Builds (1.20) (pull_request) Successful in 2m40s
Details
/ Builds (1.21) (pull_request) Successful in 2m33s
Details
/ Vulncheck (pull_request) Successful in 2m22s
Details
/ Lint (pull_request) Successful in 4m24s
Details
/ Tests (1.20) (pull_request) Successful in 2m48s
Details
/ Tests (1.21) (pull_request) Successful in 2m45s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-25 15:49:37 +03:00
Denis Kirillov
280d11c794 [#407] Don't set full_control for bucket owner
All checks were successful
/ DCO (pull_request) Successful in 1m29s
Details
/ Builds (1.20) (pull_request) Successful in 2m14s
Details
/ Builds (1.21) (pull_request) Successful in 1m47s
Details
/ Vulncheck (pull_request) Successful in 1m57s
Details
/ Lint (pull_request) Successful in 4m16s
Details
/ Tests (1.20) (pull_request) Successful in 2m46s
Details
/ Tests (1.21) (pull_request) Successful in 2m29s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-19 10:55:24 +03:00
Roman Loginov
ed34b2cae4 [#402] auth: Extend test coverage 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-06-14 10:06:00 +00:00
Denis Kirillov
76f553d292 [#403] Set resource tags into resource properties
All checks were successful
/ DCO (pull_request) Successful in 6m17s
Details
/ Vulncheck (pull_request) Successful in 8m13s
Details
/ Builds (1.20) (pull_request) Successful in 9m45s
Details
/ Builds (1.21) (pull_request) Successful in 9m8s
Details
/ Lint (pull_request) Successful in 18m4s
Details
/ Tests (1.20) (pull_request) Successful in 9m52s
Details
/ Tests (1.21) (pull_request) Successful in 9m5s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-13 11:12:40 +03:00
Denis Kirillov
bb81afc14a [#398] Support retryer 
Add two strategy for PutBucketSettings request retryer:
* exponential backoff (increasing up to `max_backoff` delays with jitter)
* constant backoff (always the same `max_backoff` delay between requests)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-06 13:02:17 +00:00
Marina Biryukova
e25dc90c20 [#399] Add OPTIONS method for object operations 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-06-04 12:59:45 +00:00
Pavel Pogodaev
b5fae316cf [#396] Add user to response 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-06-04 09:37:55 +00:00
Roman Loginov
9152b084ec [#387] Fix typo 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-05-22 15:06:02 +00:00
Roman Loginov
21dbe3ea8e [#387] api: Add tests for middleware 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-05-22 15:06:02 +00:00
Roman Loginov
f4d174e740 [#387] middleware: Extend test coverage 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-05-22 15:06:02 +00:00
Roman Loginov
8a758293b9 [#387] middleware: Delete unused code 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-05-22 15:06:02 +00:00
Denis Kirillov
fb521c7ac6 [#367] policy: Set IAM-MFA property to false by default
All checks were successful
/ DCO (pull_request) Successful in 2m34s
Details
/ Vulncheck (pull_request) Successful in 2m41s
Details
/ Builds (1.20) (pull_request) Successful in 4m26s
Details
/ Builds (1.21) (pull_request) Successful in 4m19s
Details
/ Lint (pull_request) Successful in 5m48s
Details
/ Tests (1.20) (pull_request) Successful in 3m55s
Details
/ Tests (1.21) (pull_request) Successful in 3m53s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-05-22 12:05:42 +03:00
Alex Vanin
87b9e97a80 [#354] Do not proceed on bucket remove error
All checks were successful
/ DCO (pull_request) Successful in 2m22s
Details
/ Vulncheck (pull_request) Successful in 2m37s
Details
/ Builds (1.20) (pull_request) Successful in 3m42s
Details
/ Builds (1.21) (pull_request) Successful in 3m12s
Details
/ Lint (pull_request) Successful in 4m51s
Details
/ Tests (1.20) (pull_request) Successful in 3m13s
Details
/ Tests (1.21) (pull_request) Successful in 3m6s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-17 20:38:39 +03:00
Artem Tataurov
d62d8f3874 [#385] Support the renaming of ObjectRequest and ObjectContainer
All checks were successful
/ DCO (pull_request) Successful in 1m33s
Details
/ Builds (1.20) (pull_request) Successful in 2m7s
Details
/ Builds (1.21) (pull_request) Successful in 1m24s
Details
/ Vulncheck (pull_request) Successful in 1m58s
Details
/ Lint (pull_request) Successful in 4m26s
Details
/ Tests (1.20) (pull_request) Successful in 2m35s
Details
/ Tests (1.21) (pull_request) Successful in 2m42s
Details 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2024-05-14 16:51:36 +03:00
Alex Vanin
6bf6a3b1a3 [#362] Check user and groups during policy check
All checks were successful
/ DCO (pull_request) Successful in 4m8s
Details
/ Vulncheck (pull_request) Successful in 4m10s
Details
/ Builds (1.20) (pull_request) Successful in 5m33s
Details
/ Builds (1.21) (pull_request) Successful in 5m24s
Details
/ Lint (pull_request) Successful in 8m32s
Details
/ Tests (1.20) (pull_request) Successful in 5m9s
Details
/ Tests (1.21) (pull_request) Successful in 4m52s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-08 15:25:14 +03:00
Marina Biryukova
c43ef040dc [#382] Fix request type determination
All checks were successful
/ DCO (pull_request) Successful in 1m36s
Details
/ Builds (1.20) (pull_request) Successful in 2m15s
Details
/ Builds (1.21) (pull_request) Successful in 2m9s
Details
/ Lint (pull_request) Successful in 3m22s
Details
/ Tests (1.20) (pull_request) Successful in 2m18s
Details
/ Tests (1.21) (pull_request) Successful in 2m6s
Details
/ Vulncheck (pull_request) Successful in 57s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-05-07 15:17:22 +03:00
Marina Biryukova
2ab655b909 [#380] Add test for credentials versioning 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-05-03 07:24:13 +00:00
Pavel Pogodaev
db05021786 [#379] Add Iana CharsetReader for Oracle integration
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Builds (1.20) (pull_request) Successful in 2m23s
Details
/ Builds (1.21) (pull_request) Successful in 2m0s
Details
/ Vulncheck (pull_request) Successful in 2m7s
Details
/ Lint (pull_request) Successful in 4m16s
Details
/ Tests (1.20) (pull_request) Successful in 2m38s
Details
/ Tests (1.21) (pull_request) Successful in 2m29s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-04-25 17:44:38 +03:00
Marina Biryukova
034396d554 [#377] Add check of Source IP
All checks were successful
/ DCO (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m16s
Details
/ Builds (1.21) (pull_request) Successful in 2m26s
Details
/ Vulncheck (pull_request) Successful in 2m24s
Details
/ Lint (pull_request) Successful in 4m17s
Details
/ Tests (1.20) (pull_request) Successful in 2m42s
Details
/ Tests (1.21) (pull_request) Successful in 2m32s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-22 15:29:18 +03:00
Pavel Pogodaev
3c436d8de9 [#365] Include iam user tags in query
All checks were successful
/ Vulncheck (pull_request) Successful in 1m48s
Details
/ Builds (1.20) (pull_request) Successful in 2m30s
Details
/ Builds (1.21) (pull_request) Successful in 1m25s
Details
/ Lint (pull_request) Successful in 3m52s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m22s
Details
/ DCO (pull_request) Successful in 45s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-04-22 10:47:43 +03:00
Marina Biryukova
45f77de8c8 [#371] Add custom Source IP header configuration 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-22 07:42:45 +00:00
Marina Biryukova
e22ff52165 [#367] Add check of AccessBox attributes 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-19 06:25:26 +00:00
Denis Kirillov
5315f7b733 [#269] Create frostfsid wrapper with cache
All checks were successful
/ DCO (pull_request) Successful in 2m10s
Details
/ Vulncheck (pull_request) Successful in 2m0s
Details
/ Builds (1.20) (pull_request) Successful in 2m31s
Details
/ Builds (1.21) (pull_request) Successful in 1m31s
Details
/ Lint (pull_request) Successful in 3m34s
Details
/ Tests (1.20) (pull_request) Successful in 2m26s
Details
/ Tests (1.21) (pull_request) Successful in 2m21s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-18 09:32:30 +03:00
Denis Kirillov
fec3b3f31e [#269] Add frostfsid cache configuration 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-17 12:11:23 +03:00
Marina Biryukova
3ff027587c [#357] Add check of request and resource tags 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-17 07:06:58 +00:00
Denis Kirillov
8307c73fef [#364] Fix removing combined object
Some checks failed
/ Vulncheck (pull_request) Failing after 3m8s
Details
/ DCO (pull_request) Successful in 3m49s
Details
/ Builds (1.20) (pull_request) Successful in 5m35s
Details
/ Builds (1.21) (pull_request) Successful in 4m16s
Details
/ Lint (pull_request) Successful in 6m55s
Details
/ Tests (1.20) (pull_request) Successful in 5m14s
Details
/ Tests (1.21) (pull_request) Successful in 4m29s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-12 14:56:38 +03:00
Roman Loginov
d8889fca56 [#340] Fix encode object acl 
In the process of encode the acl of an object,
we use a map. As a result, when traversing the
map, we can get a different sequence of permissions
each time. Therefore, a list is used instead of a map.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-04-11 09:28:30 +00:00
Alex Vanin
61ff4702a2 [#360] Reuse single target during policy check
Some checks failed
/ DCO (pull_request) Successful in 1m38s
Details
/ Vulncheck (pull_request) Failing after 2m4s
Details
/ Builds (1.20) (pull_request) Successful in 2m33s
Details
/ Builds (1.21) (pull_request) Successful in 2m12s
Details
/ Lint (pull_request) Successful in 3m6s
Details
/ Tests (1.20) (pull_request) Successful in 2m57s
Details
/ Tests (1.21) (pull_request) Successful in 2m6s
Details 
Policy engine library is able to manage multiple
targets and resolve different status results.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-10 17:56:47 +03:00
Alex Vanin
6da1acc554 [#360] Use 'c' prefix for bucket policies instead of 'n' 
With 'c' prefix, acl chains become shorter, thus gateway
receives shorter results and avoids sessions to neo-go.

There is still issue with many IAM rules.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-10 17:56:47 +03:00
Denis Kirillov
9c012d0a66 [#355] Remove policies when delete bucket 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-09 15:49:46 +00:00
Marina Biryukova
37d05dcefd [#353] Add check of listing parameters and versionID
Some checks failed
/ DCO (pull_request) Successful in 1m36s
Details
/ Vulncheck (pull_request) Failing after 2m17s
Details
/ Builds (1.20) (pull_request) Successful in 3m27s
Details
/ Builds (1.21) (pull_request) Successful in 3m22s
Details
/ Lint (pull_request) Successful in 5m4s
Details
/ Tests (1.20) (pull_request) Successful in 2m53s
Details
/ Tests (1.21) (pull_request) Successful in 2m47s
Details 
Add properties in policy check:
* s3:delimiter
* s3:prefix
* s3:max-keys
* s3:VersionId

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-08 17:57:55 +03:00
Denis Kirillov
8669bf6b50 [#346] acl: Update APE and fix using
All checks were successful
/ DCO (pull_request) Successful in 2m57s
Details
/ Vulncheck (pull_request) Successful in 3m33s
Details
/ Lint (pull_request) Successful in 4m44s
Details
/ Tests (1.20) (pull_request) Successful in 3m38s
Details
/ Tests (1.21) (pull_request) Successful in 3m29s
Details
/ Builds (1.20) (pull_request) Successful in 1m12s
Details
/ Builds (1.21) (pull_request) Successful in 3m23s
Details 
* Remove native policy when remove bucket policy
* Allow policies that contain only s3 compatible statements
(now deny rules cannot be converted to native rules)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-02 12:43:04 +00:00
Denis Kirillov
fbe7a784e8 [#301] Support GetBucketPolicyStatus 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-28 09:13:25 +03:00
Denis Kirillov
80c7b73eb9 [#306] In APE buckets forbid canned acl except private
Some checks failed
/ DCO (pull_request) Successful in 2m50s
Details
/ Vulncheck (pull_request) Failing after 3m15s
Details
/ Builds (1.20) (pull_request) Successful in 3m39s
Details
/ Builds (1.21) (pull_request) Successful in 3m41s
Details
/ Lint (pull_request) Successful in 5m48s
Details
/ Tests (1.20) (pull_request) Successful in 4m0s
Details
/ Tests (1.21) (pull_request) Successful in 3m53s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-19 16:57:26 +03:00
Denis Kirillov
62cc5a04a7 [#328] Log error on failed response writing
Some checks failed
/ DCO (pull_request) Successful in 3m34s
Details
/ Vulncheck (pull_request) Failing after 4m18s
Details
/ Builds (1.20) (pull_request) Successful in 4m58s
Details
/ Builds (1.21) (pull_request) Successful in 4m24s
Details
/ Lint (pull_request) Successful in 7m27s
Details
/ Tests (1.20) (pull_request) Successful in 5m24s
Details
/ Tests (1.21) (pull_request) Successful in 5m0s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-15 11:02:26 +03:00
Denis Kirillov
4ee3648183 [#328] Log invalid lock enabled header 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 15:09:51 +03:00
Denis Kirillov
ee48d1dc85 [#325] Log error on failed request id generation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
f958eef2b3 [#325] Use default empty data.LockInfo in get/head in case of error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
81b44ab3d3 [#325] Fix mutex usage in controller 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
8050ca2d51 [#306] Use session token for container read operations
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Vulncheck (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m59s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m19s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 18:14:33 +03:00
Denis Kirillov
c12e264697 [#306] Simplify cid resolver for metrics 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 17:46:16 +03:00
Denis Kirillov
e9f38a49e4 [#306] Fix forming key for bucket cache 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 16:09:40 +03:00
Denis Kirillov
fabb4134bc [#318] Use log msg from constants
All checks were successful
/ DCO (pull_request) Successful in 1m44s
Details
/ Builds (1.20) (pull_request) Successful in 2m24s
Details
/ Builds (1.21) (pull_request) Successful in 2m18s
Details
/ Vulncheck (pull_request) Successful in 2m17s
Details
/ Lint (pull_request) Successful in 2m36s
Details
/ Tests (1.20) (pull_request) Successful in 1m42s
Details
/ Tests (1.21) (pull_request) Successful in 1m32s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
e1ee36b979 [#318] Fix tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
937367caaf [#318] Fix panic on invalid multipart form 
Previously, simple 'curl -X POST http://localhost:8084/test' leads to panic because of wrong handle matching

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
7b86bac6ee [#318] Log unmatched requests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
529ec7e0b9 [#318] Don't log empty bucket/name 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
4741e74210 [#318] Log successfully authenticated accessKeyIDs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
f1470bab4a [#318] auth: Add context for logged errors 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
6e5bcaef97 [#318] Log policy request checking 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
1522db05c5 [#318] Log namespace for requests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Marina Biryukova
31da31862a [#300] Update error logging in DeleteMultipleObjects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-02-29 14:24:32 +00:00
Denis Kirillov
7de1ffdbe9 [#306] Fix billing tests
All checks were successful
/ DCO (pull_request) Successful in 1m43s
Details
/ Vulncheck (pull_request) Successful in 1m42s
Details
/ Builds (1.20) (pull_request) Successful in 2m34s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m50s
Details
/ Tests (1.20) (pull_request) Successful in 2m20s
Details
/ Tests (1.21) (pull_request) Successful in 2m9s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 18:00:27 +03:00
Denis Kirillov
3285a2e105 [#306] policy: Change default access strategy 
Use access strategy based on bucket type and/or config flags.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:53:13 +03:00
Denis Kirillov
56b50f2075 [#306] Remove flag to disable policy contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
8f89f275bd [#306] Save bucket policy as native chain 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
ff15f9f28a [#306] Fix update settings for buckets without owner key in tree 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
c868af8a62 [#306] Add flag to enable old ACL bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
bac1b3fb2d [#306] Use zero basic acl to mark APE containers 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
c452d58ce2 [#306] Reduce number of policy contract invocations 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
d9d12debc3 [#306] Add tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
3d0d2032c6 [#306] acl: Handle put/get acl for APE buckets 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
1f2cf0ed67 [#306] Use APE instead of eACL on bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
37be8851b3 [#306] Simplify namespaces configuration 
Resolve ns alias at the beginning of the request just once.
Keep in ns map only one default ns key.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
c4c199defe [#306] Use s3 as chain id prefix to be consistent 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Marina Biryukova
2981a47e99 [#321] Use correct owner id in billing metrics
All checks were successful
/ DCO (pull_request) Successful in 1m20s
Details
/ Vulncheck (pull_request) Successful in 1m54s
Details
/ Builds (1.20) (pull_request) Successful in 2m22s
Details
/ Builds (1.21) (pull_request) Successful in 2m8s
Details
/ Lint (pull_request) Successful in 4m32s
Details
/ Tests (1.20) (pull_request) Successful in 2m27s
Details
/ Tests (1.21) (pull_request) Successful in 2m13s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-02-28 14:52:44 +03:00
Alex Vanin
563c1d9bd7 [#308] Fix linter issues
All checks were successful
/ DCO (pull_request) Successful in 1m31s
Details
/ Builds (1.20) (pull_request) Successful in 1m59s
Details
/ Builds (1.21) (pull_request) Successful in 1m37s
Details
/ Vulncheck (pull_request) Successful in 2m14s
Details
/ Lint (pull_request) Successful in 4m7s
Details
/ Tests (1.20) (pull_request) Successful in 2m28s
Details
/ Tests (1.21) (pull_request) Successful in 1m58s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-02-16 18:25:06 +03:00
Denis Kirillov
924e87face [#305] Support checking if accessbox was removed 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-09 09:41:48 +03:00
Denis Kirillov
c334adeb6d [#165] Sort nodes in ServiceClientMemory
Some checks failed
/ DCO (pull_request) Successful in 1m22s
Details
/ Builds (1.20) (pull_request) Successful in 1m52s
Details
/ Builds (1.21) (pull_request) Successful in 1m17s
Details
/ Vulncheck (pull_request) Successful in 1m51s
Details
/ Lint (pull_request) Failing after 1m55s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m24s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:53:12 +03:00