TrueCloudLab/frostfs-s3-gw
17
3
Fork 19
Code Issues 30 Pull requests 9 Projects Releases 18 Packages Activity Actions
1317 commits 5 branches 43 tags 21 MiB
Commit graph

418 commits

Author SHA1 Message Date
Denis Kirillov
8307c73fef [#364] Fix removing combined object
Some checks failed
/ Vulncheck (pull_request) Failing after 3m8s
Details
/ DCO (pull_request) Successful in 3m49s
Details
/ Builds (1.20) (pull_request) Successful in 5m35s
Details
/ Builds (1.21) (pull_request) Successful in 4m16s
Details
/ Lint (pull_request) Successful in 6m55s
Details
/ Tests (1.20) (pull_request) Successful in 5m14s
Details
/ Tests (1.21) (pull_request) Successful in 4m29s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-12 14:56:38 +03:00
Roman Loginov
d8889fca56 [#340] Fix encode object acl 
In the process of encode the acl of an object,
we use a map. As a result, when traversing the
map, we can get a different sequence of permissions
each time. Therefore, a list is used instead of a map.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-04-11 09:28:30 +00:00
Alex Vanin
6da1acc554 [#360] Use 'c' prefix for bucket policies instead of 'n' 
With 'c' prefix, acl chains become shorter, thus gateway
receives shorter results and avoids sessions to neo-go.

There is still issue with many IAM rules.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-10 17:56:47 +03:00
Denis Kirillov
9c012d0a66 [#355] Remove policies when delete bucket 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-09 15:49:46 +00:00
Denis Kirillov
8669bf6b50 [#346] acl: Update APE and fix using
All checks were successful
/ DCO (pull_request) Successful in 2m57s
Details
/ Vulncheck (pull_request) Successful in 3m33s
Details
/ Lint (pull_request) Successful in 4m44s
Details
/ Tests (1.20) (pull_request) Successful in 3m38s
Details
/ Tests (1.21) (pull_request) Successful in 3m29s
Details
/ Builds (1.20) (pull_request) Successful in 1m12s
Details
/ Builds (1.21) (pull_request) Successful in 3m23s
Details 
* Remove native policy when remove bucket policy
* Allow policies that contain only s3 compatible statements
(now deny rules cannot be converted to native rules)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-02 12:43:04 +00:00
Denis Kirillov
fbe7a784e8 [#301] Support GetBucketPolicyStatus 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-28 09:13:25 +03:00
Denis Kirillov
80c7b73eb9 [#306] In APE buckets forbid canned acl except private
Some checks failed
/ DCO (pull_request) Successful in 2m50s
Details
/ Vulncheck (pull_request) Failing after 3m15s
Details
/ Builds (1.20) (pull_request) Successful in 3m39s
Details
/ Builds (1.21) (pull_request) Successful in 3m41s
Details
/ Lint (pull_request) Successful in 5m48s
Details
/ Tests (1.20) (pull_request) Successful in 4m0s
Details
/ Tests (1.21) (pull_request) Successful in 3m53s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-19 16:57:26 +03:00
Denis Kirillov
62cc5a04a7 [#328] Log error on failed response writing
Some checks failed
/ DCO (pull_request) Successful in 3m34s
Details
/ Vulncheck (pull_request) Failing after 4m18s
Details
/ Builds (1.20) (pull_request) Successful in 4m58s
Details
/ Builds (1.21) (pull_request) Successful in 4m24s
Details
/ Lint (pull_request) Successful in 7m27s
Details
/ Tests (1.20) (pull_request) Successful in 5m24s
Details
/ Tests (1.21) (pull_request) Successful in 5m0s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-15 11:02:26 +03:00
Denis Kirillov
4ee3648183 [#328] Log invalid lock enabled header 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 15:09:51 +03:00
Denis Kirillov
f958eef2b3 [#325] Use default empty data.LockInfo in get/head in case of error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
8050ca2d51 [#306] Use session token for container read operations
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Vulncheck (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m59s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m19s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 18:14:33 +03:00
Denis Kirillov
c12e264697 [#306] Simplify cid resolver for metrics 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 17:46:16 +03:00
Denis Kirillov
937367caaf [#318] Fix panic on invalid multipart form 
Previously, simple 'curl -X POST http://localhost:8084/test' leads to panic because of wrong handle matching

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Marina Biryukova
31da31862a [#300] Update error logging in DeleteMultipleObjects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-02-29 14:24:32 +00:00
Denis Kirillov
56b50f2075 [#306] Remove flag to disable policy contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
8f89f275bd [#306] Save bucket policy as native chain 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
ff15f9f28a [#306] Fix update settings for buckets without owner key in tree 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
c868af8a62 [#306] Add flag to enable old ACL bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
bac1b3fb2d [#306] Use zero basic acl to mark APE containers 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
c452d58ce2 [#306] Reduce number of policy contract invocations 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
d9d12debc3 [#306] Add tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
3d0d2032c6 [#306] acl: Handle put/get acl for APE buckets 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
1f2cf0ed67 [#306] Use APE instead of eACL on bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
37be8851b3 [#306] Simplify namespaces configuration 
Resolve ns alias at the beginning of the request just once.
Keep in ns map only one default ns key.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
c4c199defe [#306] Use s3 as chain id prefix to be consistent 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Alex Vanin
563c1d9bd7 [#308] Fix linter issues
All checks were successful
/ DCO (pull_request) Successful in 1m31s
Details
/ Builds (1.20) (pull_request) Successful in 1m59s
Details
/ Builds (1.21) (pull_request) Successful in 1m37s
Details
/ Vulncheck (pull_request) Successful in 2m14s
Details
/ Lint (pull_request) Successful in 4m7s
Details
/ Tests (1.20) (pull_request) Successful in 2m28s
Details
/ Tests (1.21) (pull_request) Successful in 1m58s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-02-16 18:25:06 +03:00
Denis Kirillov
c334adeb6d [#165] Sort nodes in ServiceClientMemory
Some checks failed
/ DCO (pull_request) Successful in 1m22s
Details
/ Builds (1.20) (pull_request) Successful in 1m52s
Details
/ Builds (1.21) (pull_request) Successful in 1m17s
Details
/ Vulncheck (pull_request) Successful in 1m51s
Details
/ Lint (pull_request) Failing after 1m55s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m24s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:53:12 +03:00
Denis Kirillov
a74d498df2 [#165] Return sort after HEAD in listing
Some checks failed
/ DCO (pull_request) Successful in 1m16s
Details
/ Builds (1.20) (pull_request) Successful in 2m9s
Details
/ Builds (1.21) (pull_request) Successful in 1m39s
Details
/ Vulncheck (pull_request) Successful in 1m52s
Details
/ Lint (pull_request) Failing after 1m47s
Details
/ Tests (1.20) (pull_request) Successful in 2m27s
Details
/ Tests (1.21) (pull_request) Successful in 2m13s
Details 
We have to sort object after HEAD because we make request in different goroutines,
so the order is not deterministic.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:08 +03:00
Denis Kirillov
69297a4a38 [#165] Delete object from tree in case of storage error 
Extend storage node errors in case of which we continue deleting from tree
with 'object not found' error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:08 +03:00
Denis Kirillov
71d82d1cc8 [#165] Fix lint issues 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:08 +03:00
Denis Kirillov
fafe4af529 [#165] Fix real object size in listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:15:03 +03:00
Denis Kirillov
88f1acbdfc [#165] Cancel context in outdated list session 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
4e15452853 [#165] Fix lint errors 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
da642a498a [#165] Listing fix data race 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
093de13f54 [#165] Add stream listing tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
cf4fc3b602 [#165] Extend error on getting listing containers not in current namespace 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
739a6ec9df [#165] Support latest only stream listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
c7ee628ab0 [#165] Fix versions listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
4ad84b9b94 [#165] listing: Use NodeVersion instead of ObjectInfo 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
29ac91dfd5 [#165] Support streaming listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
84af85ed67 [#302] Update APE to support chain id as bytes
All checks were successful
/ DCO (pull_request) Successful in 1m52s
Details
/ Vulncheck (pull_request) Successful in 2m0s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m59s
Details
/ Lint (pull_request) Successful in 2m44s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m24s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 12:53:45 +03:00
Denis Kirillov
43cae9ee04 [#248] Correct object versions response markers
All checks were successful
/ DCO (pull_request) Successful in 1m40s
Details
/ Vulncheck (pull_request) Successful in 1m42s
Details
/ Builds (1.20) (pull_request) Successful in 2m29s
Details
/ Builds (1.21) (pull_request) Successful in 1m37s
Details
/ Lint (pull_request) Successful in 3m55s
Details
/ Tests (1.20) (pull_request) Successful in 2m23s
Details
/ Tests (1.21) (pull_request) Successful in 2m16s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-21 10:53:50 +03:00
Denis Kirillov
a17ff66975 [#282] policy: Use prefixes to distinguish s3/iam actions/resources
All checks were successful
/ DCO (pull_request) Successful in 1m37s
Details
/ Vulncheck (pull_request) Successful in 1m50s
Details
/ Builds (1.20) (pull_request) Successful in 2m24s
Details
/ Builds (1.21) (pull_request) Successful in 2m2s
Details
/ Lint (pull_request) Successful in 4m26s
Details
/ Tests (1.20) (pull_request) Successful in 2m28s
Details
/ Tests (1.21) (pull_request) Successful in 1m58s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-20 10:41:15 +03:00
Denis Kirillov
38c5503a02 [#261] alc: Remove unused
All checks were successful
/ Vulncheck (pull_request) Successful in 2m1s
Details
/ DCO (pull_request) Successful in 1m49s
Details
/ Builds (1.20) (pull_request) Successful in 2m21s
Details
/ Builds (1.21) (pull_request) Successful in 2m4s
Details
/ Lint (pull_request) Successful in 2m56s
Details
/ Tests (1.20) (pull_request) Successful in 2m17s
Details
/ Tests (1.21) (pull_request) Successful in 1m46s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-18 15:49:58 +03:00
Denis Kirillov
8273af8bf8 [#261] Make PutBucketPolicy handler use policy contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-18 15:49:54 +03:00
Denis Kirillov
28c6bb4cb8 [#266] Support per namespace placement policies configuration
All checks were successful
/ DCO (pull_request) Successful in 1m34s
Details
/ Builds (1.20) (pull_request) Successful in 3m15s
Details
/ Builds (1.21) (pull_request) Successful in 2m55s
Details
/ Vulncheck (pull_request) Successful in 2m51s
Details
/ Lint (pull_request) Successful in 5m12s
Details
/ Tests (1.20) (pull_request) Successful in 2m57s
Details
/ Tests (1.21) (pull_request) Successful in 2m48s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-28 16:12:42 +03:00
Denis Kirillov
ff1ec56d24 [#260] Use namespace as domain when create bucket
All checks were successful
/ DCO (pull_request) Successful in 1m26s
Details
/ Vulncheck (pull_request) Successful in 2m6s
Details
/ Builds (1.20) (pull_request) Successful in 2m20s
Details
/ Builds (1.21) (pull_request) Successful in 2m21s
Details
/ Lint (pull_request) Successful in 3m15s
Details
/ Tests (1.20) (pull_request) Successful in 2m28s
Details
/ Tests (1.21) (pull_request) Successful in 2m17s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-27 15:30:12 +03:00
Denis Kirillov
cf7254f8cd [#260] Refactor api/auth/center.go 
Move the Center interface to middleware package where it's used

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:09 +03:00
Roman Loginov
861454e499 [#218] Add check content sha256 header 
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-11-22 11:33:52 +00:00
Marina Biryukova
b28ecef43b [#219] Return ETag in quotes
All checks were successful
/ Vulncheck (pull_request) Successful in 9m5s
Details
/ Lint (pull_request) Successful in 11m1s
Details
/ Tests (1.20) (pull_request) Successful in 9m59s
Details
/ Tests (1.21) (pull_request) Successful in 9m53s
Details
/ DCO (pull_request) Successful in 11m34s
Details
/ Builds (1.20) (pull_request) Successful in 12m24s
Details
/ Builds (1.21) (pull_request) Successful in 8m59s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-11-22 11:12:32 +00:00