TrueCloudLab/frostfs-s3-gw
15
3
Fork 18
Code Issues 19 Pull requests 6 Projects Releases 17 Packages Activity Actions
1441 commits 5 branches 40 tags 20 MiB
Commit graph

1441 commits

This branch
This branch
All branches
Author SHA1 Message Date
Denis Kirillov
6d52f46012 [#165] Fix v1 listing bookmark 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
0ae49eaab0 [#165] Generalize allObjectListingParams 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
093de13f54 [#165] Add stream listing tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
cf4fc3b602 [#165] Extend error on getting listing containers not in current namespace 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
739a6ec9df [#165] Support latest only stream listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
c7ee628ab0 [#165] Fix versions listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
4ad84b9b94 [#165] listing: Use NodeVersion instead of ObjectInfo 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
3e20f736a6 [#165] Move listing function to one file 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
b52552e8c2 [#165] Add batching in streamin listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
6e8960b2ab [#165] Add list session cache 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
29ac91dfd5 [#165] Support streaming listing 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Denis Kirillov
84af85ed67 [#302] Update APE to support chain id as bytes
All checks were successful
/ DCO (pull_request) Successful in 1m52s
Details
/ Vulncheck (pull_request) Successful in 2m0s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m59s
Details
/ Lint (pull_request) Successful in 2m44s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m24s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 12:53:45 +03:00
Pavel Pogodaev
4804904d9d [#298] journald update version 
We want to have less useless fields in logs

Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-01-29 12:49:22 +00:00
Denis Kirillov
e23cc43824 [#299] Drop unused legacy minio code
All checks were successful
/ DCO (pull_request) Successful in 2m36s
Details
/ Vulncheck (pull_request) Successful in 2m59s
Details
/ Lint (pull_request) Successful in 5m23s
Details
/ Tests (1.20) (pull_request) Successful in 3m24s
Details
/ Tests (1.21) (pull_request) Successful in 3m3s
Details
/ Builds (1.20) (pull_request) Successful in 1m16s
Details
/ Builds (1.21) (pull_request) Successful in 2m53s
Details 
736d8cbac4 (diff-f5a8931b4d5f3b7f583e4cd719bfd2904980518a6f338d463ec76aea814db772)
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-01-29 12:57:18 +03:00
Evgenii Stratonikov
12434d5f4d [#297] .forgejo: Check only PR commits in dco-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-01-26 15:11:52 +00:00
Evgenii Stratonikov
2e870e99c7 [#297] .forgejo: Update dco-go to v3 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-01-26 15:11:52 +00:00
Alex Vanin
45e025320f [#296] Port v0.28.1 release changelog
All checks were successful
/ Vulncheck (pull_request) Successful in 2m9s
Details
/ DCO (pull_request) Successful in 2m41s
Details
/ Builds (1.20) (pull_request) Successful in 2m57s
Details
/ Builds (1.21) (pull_request) Successful in 2m28s
Details
/ Lint (pull_request) Successful in 4m44s
Details
/ Tests (1.20) (pull_request) Successful in 2m7s
Details
/ Tests (1.21) (pull_request) Successful in 1m52s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-01-24 17:50:24 +03:00
Denis Kirillov
eae49908da [#292] authmate: Support custom attributes
All checks were successful
/ DCO (pull_request) Successful in 1m36s
Details
/ Builds (1.20) (pull_request) Successful in 2m2s
Details
/ Builds (1.21) (pull_request) Successful in 1m28s
Details
/ Vulncheck (pull_request) Successful in 1m53s
Details
/ Lint (pull_request) Successful in 3m19s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m11s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-01-22 10:02:43 +03:00
Marina Biryukova
c32220762f [#288] Fix possibility of panic during SIGHUP
All checks were successful
/ DCO (pull_request) Successful in 1m22s
Details
/ Builds (1.20) (pull_request) Successful in 1m59s
Details
/ Builds (1.21) (pull_request) Successful in 1m10s
Details
/ Vulncheck (pull_request) Successful in 1m51s
Details
/ Lint (pull_request) Successful in 4m30s
Details
/ Tests (1.20) (pull_request) Successful in 2m20s
Details
/ Tests (1.21) (pull_request) Successful in 2m11s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-01-09 10:53:54 +03:00
Roman Loginov
6f9ee3da76 [#275] Change logic delete multipart upload
All checks were successful
/ Vulncheck (pull_request) Successful in 1m13s
Details
/ DCO (pull_request) Successful in 1m43s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m36s
Details
/ Lint (pull_request) Successful in 3m33s
Details
/ Tests (1.20) (pull_request) Successful in 2m34s
Details
/ Tests (1.21) (pull_request) Successful in 2m29s
Details 
In order not to accidentally take outdated
information about downloaded parts from other
nodes, now when the multipart is abort or complete,
the root node of the multipart upload with the
finish flag remains in the tree.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-12-27 13:06:45 +03:00
Marina Biryukova
08019f1574 [#280] Add put requests to duration metric
All checks were successful
/ DCO (pull_request) Successful in 1m48s
Details
/ Builds (1.20) (pull_request) Successful in 2m10s
Details
/ Builds (1.21) (pull_request) Successful in 1m24s
Details
/ Vulncheck (pull_request) Successful in 2m0s
Details
/ Lint (pull_request) Successful in 4m24s
Details
/ Tests (1.20) (pull_request) Successful in 2m16s
Details
/ Tests (1.21) (pull_request) Successful in 2m6s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-12-22 16:33:05 +03:00
Denis Kirillov
899213b3f3 [#287] Support proxy for frostfsid and policy contracts
All checks were successful
/ Builds (1.20) (pull_request) Successful in 1m52s
Details
/ Builds (1.21) (pull_request) Successful in 1m12s
Details
/ DCO (pull_request) Successful in 1m46s
Details
/ Vulncheck (pull_request) Successful in 1m43s
Details
/ Lint (pull_request) Successful in 4m3s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m23s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-22 15:45:10 +03:00
Denis Kirillov
3b6d2bc522 [#287] authmate: Support frostfsid proxy and namespace 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-22 15:43:14 +03:00
Denis Kirillov
6509639540 [#283] control: Changed type of chainID to bytes
All checks were successful
/ DCO (pull_request) Successful in 1m38s
Details
/ Builds (1.20) (pull_request) Successful in 2m11s
Details
/ Builds (1.21) (pull_request) Successful in 2m7s
Details
/ Vulncheck (pull_request) Successful in 1m54s
Details
/ Lint (pull_request) Successful in 4m2s
Details
/ Tests (1.20) (pull_request) Successful in 2m23s
Details
/ Tests (1.21) (pull_request) Successful in 2m13s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-21 18:07:17 +03:00
Denis Kirillov
5698d5844e [#283] Support frostfsid groups in policy request checking 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-21 14:21:36 +03:00
Denis Kirillov
43cae9ee04 [#248] Correct object versions response markers
All checks were successful
/ DCO (pull_request) Successful in 1m40s
Details
/ Vulncheck (pull_request) Successful in 1m42s
Details
/ Builds (1.20) (pull_request) Successful in 2m29s
Details
/ Builds (1.21) (pull_request) Successful in 1m37s
Details
/ Lint (pull_request) Successful in 3m55s
Details
/ Tests (1.20) (pull_request) Successful in 2m23s
Details
/ Tests (1.21) (pull_request) Successful in 2m16s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-21 10:53:50 +03:00
Denis Kirillov
a17ff66975 [#282] policy: Use prefixes to distinguish s3/iam actions/resources
All checks were successful
/ DCO (pull_request) Successful in 1m37s
Details
/ Vulncheck (pull_request) Successful in 1m50s
Details
/ Builds (1.20) (pull_request) Successful in 2m24s
Details
/ Builds (1.21) (pull_request) Successful in 2m2s
Details
/ Lint (pull_request) Successful in 4m26s
Details
/ Tests (1.20) (pull_request) Successful in 2m28s
Details
/ Tests (1.21) (pull_request) Successful in 1m58s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-20 10:41:15 +03:00
Denis Kirillov
38c5503a02 [#261] alc: Remove unused
All checks were successful
/ Vulncheck (pull_request) Successful in 2m1s
Details
/ DCO (pull_request) Successful in 1m49s
Details
/ Builds (1.20) (pull_request) Successful in 2m21s
Details
/ Builds (1.21) (pull_request) Successful in 2m4s
Details
/ Lint (pull_request) Successful in 2m56s
Details
/ Tests (1.20) (pull_request) Successful in 2m17s
Details
/ Tests (1.21) (pull_request) Successful in 1m46s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-18 15:49:58 +03:00
Denis Kirillov
8273af8bf8 [#261] Make PutBucketPolicy handler use policy contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-18 15:49:54 +03:00
Denis Kirillov
6dbb07f0fa [#261] Update policy-engine dependency 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-14 17:52:13 +03:00
Denis Kirillov
340e6b807b [#266] Update params that being reloaded in one place
All checks were successful
/ DCO (pull_request) Successful in 1m33s
Details
/ Builds (1.20) (pull_request) Successful in 1m57s
Details
/ Builds (1.21) (pull_request) Successful in 1m39s
Details
/ Vulncheck (pull_request) Successful in 1m55s
Details
/ Lint (pull_request) Successful in 4m0s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m22s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-14 16:21:09 +03:00
Denis Kirillov
0850d21ff3 [#266] Move trimming namespaces name into fetch function 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-14 16:12:10 +03:00
Denis Kirillov
9272f4e108 [#259] Support contract based policies
All checks were successful
/ DCO (pull_request) Successful in 1m21s
Details
/ Vulncheck (pull_request) Successful in 1m41s
Details
/ Builds (1.20) (pull_request) Successful in 2m19s
Details
/ Builds (1.21) (pull_request) Successful in 2m1s
Details
/ Lint (pull_request) Successful in 3m20s
Details
/ Tests (1.20) (pull_request) Successful in 2m14s
Details
/ Tests (1.21) (pull_request) Successful in 2m10s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-11 10:01:46 +03:00
Denis Kirillov
be6a37ada5 [#262] Support configuring max tree request attempts
All checks were successful
/ DCO (pull_request) Successful in 1m15s
Details
/ Vulncheck (pull_request) Successful in 1m39s
Details
/ Builds (1.20) (pull_request) Successful in 1m19s
Details
/ Builds (1.21) (pull_request) Successful in 2m27s
Details
/ Lint (pull_request) Successful in 4m24s
Details
/ Tests (1.20) (pull_request) Successful in 2m17s
Details
/ Tests (1.21) (pull_request) Successful in 2m36s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-12-07 16:41:21 +03:00
Denis Kirillov
836874a761 [#262] Set tree request id 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-07 16:40:56 +03:00
Alex Vanin
7a285d1464 [#267] Update api-go to fix stable marshal of empty structs 
Newer version of api-go does not ignore non-nil empty
structures in protobuf messages, so compatibility with
previous version is preserved.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-12-07 16:40:56 +03:00
Alex Vanin
f58a0d04ff [#233] Clean tag node in the tree service instead of removal 
With new retry policy of tree service pool, gateway should avoid
deletion of system nodes from tree. Absence of node in the tree
will trigger retry. Other storage in the network may return already
deleted node while tree is not completely synced, and client will
get unexpected result.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-12-07 16:40:56 +03:00
Alex Vanin
94c2674f44 [#233] Update tree service pool in SDK 
Tree service pool now produces retries if
tree service returns 'not found' errors
or empty result on 'GetNodeByPath'

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-12-07 16:40:53 +03:00
Alex Vanin
6b1b43a364 Release v0.28.0
All checks were successful
/ Vulncheck (pull_request) Successful in 4m23s
Details
/ DCO (pull_request) Successful in 4m25s
Details
/ Builds (1.20) (pull_request) Successful in 6m12s
Details
/ Builds (1.21) (pull_request) Successful in 5m58s
Details
/ Lint (pull_request) Successful in 6m49s
Details
/ Tests (1.20) (pull_request) Successful in 5m45s
Details
/ Tests (1.21) (pull_request) Successful in 5m26s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-12-07 15:47:31 +03:00
Denis Kirillov
43abf58068 [#257] Support flag to deny access if policy rules not found
All checks were successful
/ DCO (pull_request) Successful in 1m13s
Details
/ Vulncheck (pull_request) Successful in 2m2s
Details
/ Builds (1.20) (pull_request) Successful in 2m22s
Details
/ Builds (1.21) (pull_request) Successful in 2m16s
Details
/ Lint (pull_request) Successful in 3m26s
Details
/ Tests (1.20) (pull_request) Successful in 2m21s
Details
/ Tests (1.21) (pull_request) Successful in 1m37s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-06 17:47:58 +03:00
Denis Kirillov
ca15acf1d3 [#257] router: Use named constants 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-06 17:47:58 +03:00
Denis Kirillov
473239bf36 [#257] Add policy checker 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-06 17:47:51 +03:00
Marina Biryukova
93cf7c462b [#271] Add namespace label to billing metrics
All checks were successful
/ DCO (pull_request) Successful in 2m35s
Details
/ Vulncheck (pull_request) Successful in 3m3s
Details
/ Builds (1.20) (pull_request) Successful in 3m34s
Details
/ Builds (1.21) (pull_request) Successful in 2m20s
Details
/ Lint (pull_request) Successful in 5m27s
Details
/ Tests (1.20) (pull_request) Successful in 3m25s
Details
/ Tests (1.21) (pull_request) Successful in 3m12s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-12-04 14:54:40 +03:00
Denis Kirillov
6c5f9b2764 [#266] Fix namespace config initialization 
Don't use nil Namespaces map in case when file isn't provided or invalid

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-04 09:40:47 +00:00
Denis Kirillov
42862fd69e [#258] Support policy management in control svc
All checks were successful
/ Vulncheck (pull_request) Successful in 3m20s
Details
/ DCO (pull_request) Successful in 3m56s
Details
/ Builds (1.20) (pull_request) Successful in 4m13s
Details
/ Builds (1.21) (pull_request) Successful in 4m9s
Details
/ Lint (pull_request) Successful in 3m9s
Details
/ Tests (1.20) (pull_request) Successful in 4m14s
Details
/ Tests (1.21) (pull_request) Successful in 3m59s
Details 
Add PutPolicies, RemovePolicies, GetPolicy, ListPolicies methods

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-01 15:47:12 +03:00
Denis Kirillov
c7a65bd075 [#258] Add control service 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-01 14:17:06 +03:00
Denis Kirillov
28c6bb4cb8 [#266] Support per namespace placement policies configuration
All checks were successful
/ DCO (pull_request) Successful in 1m34s
Details
/ Builds (1.20) (pull_request) Successful in 3m15s
Details
/ Builds (1.21) (pull_request) Successful in 2m55s
Details
/ Vulncheck (pull_request) Successful in 2m51s
Details
/ Lint (pull_request) Successful in 5m12s
Details
/ Tests (1.20) (pull_request) Successful in 2m57s
Details
/ Tests (1.21) (pull_request) Successful in 2m48s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-28 16:12:42 +03:00
Alex Vanin
0db6cd6727 [#268] Add dirty version check 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-11-28 10:42:49 +00:00
Denis Kirillov
ff1ec56d24 [#260] Use namespace as domain when create bucket
All checks were successful
/ DCO (pull_request) Successful in 1m26s
Details
/ Vulncheck (pull_request) Successful in 2m6s
Details
/ Builds (1.20) (pull_request) Successful in 2m20s
Details
/ Builds (1.21) (pull_request) Successful in 2m21s
Details
/ Lint (pull_request) Successful in 3m15s
Details
/ Tests (1.20) (pull_request) Successful in 2m28s
Details
/ Tests (1.21) (pull_request) Successful in 2m17s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-27 15:30:12 +03:00
Denis Kirillov
9ebfca654b [#260] Update CHANGELOG.md 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:19 +03:00