TrueCloudLab/frostfs-s3-gw
15
3
Fork 19
Code Issues 28 Pull requests 6 Projects Releases 17 Packages Activity Actions
1331 commits 5 branches 41 tags 21 MiB
Commit graph

1331 commits

This branch
This branch
All branches
Author SHA1 Message Date
Denis Kirillov
c5f61d8a4e [#431] tree: Support removing old split system nodes 
It's need to fit user expectation on deleting CORs for example.
Previously after removing cors (that was uploaded in split manner)
we can still get some data (from other node)
because deletion worked only for latest node version.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-18 16:50:48 +03:00
Denis Kirillov
0d1131965e [#431] acl: Fix test backward compatibility
All checks were successful
/ Builds (1.21) (pull_request) Successful in 1m44s
Details
/ Builds (1.22) (pull_request) Successful in 2m16s
Details
/ DCO (pull_request) Successful in 2m28s
Details
/ Vulncheck (pull_request) Successful in 2m55s
Details
/ Lint (pull_request) Successful in 6m1s
Details
/ Tests (1.21) (pull_request) Successful in 3m15s
Details
/ Tests (1.22) (pull_request) Successful in 3m5s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-17 09:53:33 +03:00
Denis Kirillov
12c38d2819 [#431] ci: Update go version 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-17 09:53:28 +03:00
Denis Kirillov
dfc0b5d6b4 [#431] tree: Fix multipart having system name 
Previously if multipart key has the same name as some system node
(e.g. bucket-settings, bucket-cors etc.) it shadows real system node
and bucket started to be unversioned again for example.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-17 09:53:24 +03:00
Denis Kirillov
9971585839 [#431] Fix split tree 
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated, so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-17 09:53:16 +03:00
Alex Vanin
85a5b65807 Release v0.29.2
Some checks failed
/ DCO (pull_request) Successful in 1m51s
Details
/ Builds (1.20) (pull_request) Successful in 2m18s
Details
/ Builds (1.21) (pull_request) Successful in 1m42s
Details
/ Vulncheck (pull_request) Failing after 2m15s
Details
/ Lint (pull_request) Successful in 4m1s
Details
/ Tests (1.20) (pull_request) Successful in 2m48s
Details
/ Tests (1.21) (pull_request) Successful in 1m20s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-03 14:25:15 +03:00
Alex Vanin
0e3dd2ff19 [#398] Fix parameter parsing in bucket retryer
Some checks failed
/ DCO (pull_request) Successful in 2m0s
Details
/ Builds (1.20) (pull_request) Successful in 2m16s
Details
/ Builds (1.21) (pull_request) Successful in 1m39s
Details
/ Vulncheck (pull_request) Failing after 2m7s
Details
/ Lint (pull_request) Successful in 4m25s
Details
/ Tests (1.20) (pull_request) Successful in 1m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m34s
Details 
RetryStrategyExponential should use jitter backoff
instead of constant delay function

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-03 13:43:58 +03:00
Alex Vanin
caea6e4208 Release v0.29.1
Some checks failed
/ DCO (pull_request) Successful in 4m52s
Details
/ Vulncheck (pull_request) Failing after 5m27s
Details
/ Builds (1.20) (pull_request) Successful in 6m16s
Details
/ Builds (1.21) (pull_request) Successful in 5m57s
Details
/ Lint (pull_request) Successful in 9m35s
Details
/ Tests (1.20) (pull_request) Successful in 5m46s
Details
/ Tests (1.21) (pull_request) Successful in 5m34s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-06-20 17:49:00 +03:00
Alex Vanin
94a21f5351 [#398] Update sdk-go to avoid unnecessary retries 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-06-20 17:49:00 +03:00
Denis Kirillov
5d6d5e41d0 [#398] Support retryer
Some checks failed
/ DCO (pull_request) Successful in 1m40s
Details
/ Builds (1.20) (pull_request) Successful in 2m22s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Vulncheck (pull_request) Failing after 2m40s
Details
/ Lint (pull_request) Successful in 4m31s
Details
/ Tests (1.20) (pull_request) Successful in 2m45s
Details
/ Tests (1.21) (pull_request) Successful in 2m38s
Details 
Add two strategy for PutBucketSettings request retryer:
* exponential backoff (increasing up to `max_backoff` delays with jitter)
* constant backoff (always the same `max_backoff` delay between requests)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>

(cherry picked from commit bb81afc14a)
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-06-19 08:55:45 +03:00
Marina Biryukova
7d86b816a1 [#399] Add OPTIONS method for object operations 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>

(cherry picked from commit e25dc90c20)
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-06-19 08:55:45 +03:00
Alex Vanin
dea79631bb Release v0.29.0
Some checks failed
/ DCO (pull_request) Successful in 1m57s
Details
/ Vulncheck (pull_request) Failing after 2m26s
Details
/ Builds (1.20) (pull_request) Successful in 2m38s
Details
/ Builds (1.21) (pull_request) Successful in 2m39s
Details
/ Lint (pull_request) Successful in 4m19s
Details
/ Tests (1.20) (pull_request) Successful in 1m20s
Details
/ Tests (1.21) (pull_request) Successful in 2m0s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-27 13:50:24 +03:00
Alex Vanin
618ea318ba [#394] Port release v0.28.2 changelog 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-27 13:50:24 +03:00
Alex Vanin
7ae4323116 [#354] Do not proceed on bucket remove error
Some checks failed
/ DCO (pull_request) Successful in 1m59s
Details
/ Builds (1.20) (pull_request) Successful in 2m18s
Details
/ Builds (1.21) (pull_request) Successful in 2m12s
Details
/ Vulncheck (pull_request) Failing after 3m3s
Details
/ Lint (pull_request) Successful in 4m56s
Details
/ Tests (1.20) (pull_request) Successful in 3m18s
Details
/ Tests (1.21) (pull_request) Successful in 3m15s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-17 20:35:57 +03:00
Marina Biryukova
c0f959ece7 [#383] Fix request type determination
Some checks failed
/ Vulncheck (pull_request) Failing after 1m38s
Details
/ DCO (pull_request) Successful in 1m47s
Details
/ Builds (1.20) (pull_request) Successful in 2m18s
Details
/ Builds (1.21) (pull_request) Successful in 2m10s
Details
/ Lint (pull_request) Successful in 2m52s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m14s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-05-07 15:21:19 +03:00
Denis Kirillov
0cf19d24ee [#269] Create frostfsid wrapper with cache
Some checks failed
/ DCO (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m36s
Details
/ Builds (1.21) (pull_request) Successful in 2m28s
Details
/ Vulncheck (pull_request) Failing after 2m17s
Details
/ Lint (pull_request) Successful in 4m21s
Details
/ Tests (1.20) (pull_request) Successful in 2m42s
Details
/ Tests (1.21) (pull_request) Successful in 2m37s
Details 
(cherry picked from commit 5315f7b733)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-26 12:00:37 +03:00
Denis Kirillov
188e0cfd01 [#269] authmate: Update frostfsid using 
(cherry picked from commit 43a687b572)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-26 12:00:32 +03:00
Denis Kirillov
50aeba6a4e [#269] Move frostfsid client to separate package 
(cherry picked from commit 29a2dae40c)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-26 12:00:24 +03:00
Denis Kirillov
c6d0e5fc44 [#269] Add frostfsid cache configuration 
(cherry picked from commit fec3b3f31e)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-26 11:59:45 +03:00
Denis Kirillov
377fa127b5 [#370] Fix fetching attributes from tree
Some checks failed
/ DCO (pull_request) Successful in 1m45s
Details
/ Builds (1.20) (pull_request) Successful in 2m19s
Details
/ Builds (1.21) (pull_request) Successful in 1m49s
Details
/ Vulncheck (pull_request) Failing after 2m29s
Details
/ Lint (pull_request) Successful in 5m1s
Details
/ Tests (1.20) (pull_request) Successful in 3m18s
Details
/ Tests (1.21) (pull_request) Successful in 3m13s
Details 
Port #374

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-19 17:33:19 +03:00
Alex Vanin
f02bad65a8 [#362] Check user and groups during policy check
Some checks failed
/ DCO (pull_request) Successful in 1m56s
Details
/ Builds (1.20) (pull_request) Successful in 2m11s
Details
/ Builds (1.21) (pull_request) Successful in 1m52s
Details
/ Vulncheck (pull_request) Failing after 2m7s
Details
/ Lint (pull_request) Successful in 4m21s
Details
/ Tests (1.20) (pull_request) Successful in 2m37s
Details
/ Tests (1.21) (pull_request) Successful in 2m31s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-12 16:52:08 +03:00
Alex Vanin
13d00dd7ce [#362] Expand control service 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-12 16:52:08 +03:00
Alex Vanin
02122892ca [#362] Update policy-engine to support 'u' and 'g' targets 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-12 16:52:08 +03:00
Denis Kirillov
272e3a0f03 [#363] Fix removing combined object
Some checks failed
/ DCO (pull_request) Successful in 1m18s
Details
/ Builds (1.20) (pull_request) Successful in 2m20s
Details
/ Builds (1.21) (pull_request) Successful in 1m33s
Details
/ Vulncheck (pull_request) Failing after 4m46s
Details
/ Lint (pull_request) Successful in 6m10s
Details
/ Tests (1.20) (pull_request) Successful in 5m12s
Details
/ Tests (1.21) (pull_request) Successful in 3m40s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-12 14:56:06 +03:00
Alex Vanin
65a8e2dadc [#360] Reuse single target during policy check
Some checks failed
/ DCO (pull_request) Successful in 1m40s
Details
/ Vulncheck (pull_request) Failing after 1m51s
Details
/ Builds (1.20) (pull_request) Successful in 2m29s
Details
/ Builds (1.21) (pull_request) Successful in 1m44s
Details
/ Lint (pull_request) Successful in 3m57s
Details
/ Tests (1.20) (pull_request) Successful in 2m26s
Details
/ Tests (1.21) (pull_request) Successful in 2m18s
Details 
Policy engine library is able to manage multiple
targets and resolve different status results.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-10 17:40:25 +03:00
Alex Vanin
b7e15402a1 [#360] Use 'c' prefix for bucket policies instead of 'n' 
With 'c' prefix, acl chains become shorter, thus gateway
receives shorter results and avoids sessions to neo-go.

There is still issue with many IAM rules.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-10 17:40:25 +03:00
Denis Kirillov
0cc6b41b2d [#358] Update APE to allow put tombstone on delete object
Some checks failed
/ Builds (1.20) (pull_request) Successful in 4m17s
Details
/ Builds (1.21) (pull_request) Successful in 3m58s
Details
/ DCO (pull_request) Successful in 4m8s
Details
/ Vulncheck (pull_request) Failing after 4m36s
Details
/ Lint (pull_request) Successful in 6m50s
Details
/ Tests (1.20) (pull_request) Successful in 5m13s
Details
/ Tests (1.21) (pull_request) Successful in 5m7s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-10 15:11:25 +03:00
Denis Kirillov
51be9d9778 [#354] Remove policies when delete bucket
Some checks failed
/ DCO (pull_request) Successful in 2m10s
Details
/ Builds (1.20) (pull_request) Successful in 3m30s
Details
/ Builds (1.21) (pull_request) Successful in 2m29s
Details
/ Vulncheck (pull_request) Failing after 3m10s
Details
/ Lint (pull_request) Successful in 4m59s
Details
/ Tests (1.20) (pull_request) Successful in 3m12s
Details
/ Tests (1.21) (pull_request) Successful in 1m34s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-09 17:30:20 +03:00
Denis Kirillov
1cad101609 [#354] Update frostfs-contract to terminate session iterator 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-09 17:30:10 +03:00
Denis Kirillov
d631ee55c9 [#351] policy: Use iterators to list chains
All checks were successful
/ DCO (pull_request) Successful in 3m49s
Details
/ Vulncheck (pull_request) Successful in 4m44s
Details
/ Builds (1.20) (pull_request) Successful in 5m14s
Details
/ Builds (1.21) (pull_request) Successful in 4m47s
Details
/ Lint (pull_request) Successful in 7m10s
Details
/ Tests (1.20) (pull_request) Successful in 4m53s
Details
/ Tests (1.21) (pull_request) Successful in 4m17s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-03 11:32:43 +03:00
Alex Vanin
85c8210ffd [#347] Explicitly specify sorting order of subtree for object listing
All checks were successful
/ DCO (pull_request) Successful in 54s
Details
/ Vulncheck (pull_request) Successful in 1m56s
Details
/ Builds (1.20) (pull_request) Successful in 3m16s
Details
/ Builds (1.21) (pull_request) Successful in 3m18s
Details
/ Lint (pull_request) Successful in 8m45s
Details
/ Tests (1.20) (pull_request) Successful in 2m56s
Details
/ Tests (1.21) (pull_request) Successful in 2m25s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-02 18:14:43 +03:00
Denis Kirillov
bcfbcdc82f [#345] acl: Update APE and fix using
Some checks failed
/ Lint (pull_request) Successful in 2m53s
Details
/ Tests (1.20) (pull_request) Successful in 2m43s
Details
/ Tests (1.21) (pull_request) Successful in 2m44s
Details
/ DCO (pull_request) Successful in 2m40s
Details
/ Vulncheck (pull_request) Failing after 3m10s
Details
/ Builds (1.20) (pull_request) Successful in 3m18s
Details
/ Builds (1.21) (pull_request) Successful in 3m33s
Details 
* Remove native policy when remove bucket policy
* Allow policies that contain only s3 compatible statements
(now deny rules cannot be converted to native rules)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-02 12:42:43 +00:00
Denis Kirillov
fd310f5e9f [#343] docs: Actualize s3 compatibility table
Some checks failed
/ Vulncheck (pull_request) Failing after 9s
Details
/ DCO (pull_request) Successful in 1m30s
Details
/ Builds (1.20) (pull_request) Successful in 2m11s
Details
/ Builds (1.21) (pull_request) Successful in 1m49s
Details
/ Lint (pull_request) Successful in 4m5s
Details
/ Tests (1.20) (pull_request) Successful in 2m26s
Details
/ Tests (1.21) (pull_request) Successful in 2m20s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-02 10:00:44 +03:00
Denis Kirillov
1f5f2bd3d5 [#306] In APE buckets forbid canned acl except private
Some checks failed
/ DCO (pull_request) Successful in 1m35s
Details
/ Builds (1.20) (pull_request) Successful in 2m9s
Details
/ Builds (1.21) (pull_request) Successful in 2m3s
Details
/ Vulncheck (pull_request) Failing after 3m30s
Details
/ Lint (pull_request) Successful in 6m1s
Details
/ Tests (1.20) (pull_request) Successful in 3m13s
Details
/ Tests (1.21) (pull_request) Successful in 2m52s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-19 16:56:32 +03:00
Denis Kirillov
a32b41716f [#328] Log error on failed response writing
Some checks failed
/ DCO (pull_request) Successful in 3m48s
Details
/ Builds (1.20) (pull_request) Successful in 3m58s
Details
/ Builds (1.21) (pull_request) Successful in 3m59s
Details
/ Vulncheck (pull_request) Failing after 3m44s
Details
/ Lint (pull_request) Successful in 2m43s
Details
/ Tests (1.20) (pull_request) Successful in 3m58s
Details
/ Tests (1.21) (pull_request) Successful in 3m53s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-15 11:04:05 +03:00
Denis Kirillov
6788306998 [#328] Log invalid tree service KVs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 15:35:23 +03:00
Denis Kirillov
4ee3648183 [#328] Log invalid lock enabled header 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 15:09:51 +03:00
Denis Kirillov
ee48d1dc85 [#325] Log error on failed request id generation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
f958eef2b3 [#325] Use default empty data.LockInfo in get/head in case of error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
81b44ab3d3 [#325] Fix mutex usage in controller 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Denis Kirillov
623001c403 [#325] Close listener on error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Marina Biryukova
70043c4800 [#324] Close nns resolver after use 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-03-04 09:06:26 +00:00
Denis Kirillov
8050ca2d51 [#306] Use session token for container read operations
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Vulncheck (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m59s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m19s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 18:14:33 +03:00
Denis Kirillov
c12e264697 [#306] Simplify cid resolver for metrics 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 17:46:16 +03:00
Denis Kirillov
e9f38a49e4 [#306] Fix forming key for bucket cache 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 16:09:40 +03:00
Denis Kirillov
fabb4134bc [#318] Use log msg from constants
All checks were successful
/ DCO (pull_request) Successful in 1m44s
Details
/ Builds (1.20) (pull_request) Successful in 2m24s
Details
/ Builds (1.21) (pull_request) Successful in 2m18s
Details
/ Vulncheck (pull_request) Successful in 2m17s
Details
/ Lint (pull_request) Successful in 2m36s
Details
/ Tests (1.20) (pull_request) Successful in 1m42s
Details
/ Tests (1.21) (pull_request) Successful in 1m32s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
e1ee36b979 [#318] Fix tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
937367caaf [#318] Fix panic on invalid multipart form 
Previously, simple 'curl -X POST http://localhost:8084/test' leads to panic because of wrong handle matching

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
7b86bac6ee [#318] Log unmatched requests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
529ec7e0b9 [#318] Don't log empty bucket/name 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00