TrueCloudLab/frostfs-s3-gw
18
3
Fork 20
Code Issues 27 Pull requests 9 Projects Releases 21 Packages Activity Actions
1474 commits 6 branches 46 tags 24 MiB
Commit graph

1474 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nikita Zinkevich
2c002b657e
[#570] Update frostfs-sdk-go with new tree service client
All checks were successful
/ DCO (pull_request) Successful in 1m37s
Details
/ Vulncheck (pull_request) Successful in 2m22s
Details
/ Builds (pull_request) Successful in 4m12s
Details
/ Lint (pull_request) Successful in 5m16s
Details
/ Tests (pull_request) Successful in 4m13s
Details 
Fix imports in order to apply new sdk changes caused by merging with frostfs-api-go and reimplementing tree service client

Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-12-09 15:08:48 +03:00
Marina Biryukova
f215d200e8 [#559] Remove multipart objects using tombstones
All checks were successful
/ DCO (pull_request) Successful in 1m33s
Details
/ Vulncheck (pull_request) Successful in 2m38s
Details
/ Builds (pull_request) Successful in 2m48s
Details
/ Lint (pull_request) Successful in 4m6s
Details
/ Tests (pull_request) Successful in 2m48s
Details
/ Vulncheck (push) Successful in 1m19s
Details
/ Builds (push) Successful in 2m14s
Details
/ Lint (push) Successful in 3m40s
Details
/ Tests (push) Successful in 2m21s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-12-04 11:03:01 +03:00
Pavel Pogodaev
51322cccdf [#502] Add Dropped logs (by sampling) metric
All checks were successful
/ DCO (pull_request) Successful in 2m9s
Details
/ Vulncheck (pull_request) Successful in 2m22s
Details
/ Builds (pull_request) Successful in 2m0s
Details
/ Lint (pull_request) Successful in 3m6s
Details
/ Tests (pull_request) Successful in 2m2s
Details
/ Vulncheck (push) Successful in 1m13s
Details
/ Builds (push) Successful in 1m58s
Details
/ Lint (push) Successful in 3m33s
Details
/ Tests (push) Successful in 2m19s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-12-03 12:16:56 +00:00
Alex Vanin
3cd88d6204 Release v0.31.1
All checks were successful
/ Vulncheck (push) Successful in 1m23s
Details
/ Builds (push) Successful in 1m34s
Details
/ Lint (push) Successful in 2m52s
Details
/ Tests (push) Successful in 2m43s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-11-29 13:35:58 +00:00
Pavel Pogodaev
e71ba5e22a [#543] Add md5 sse-c S3Tests compatability
All checks were successful
/ Vulncheck (push) Successful in 1m28s
Details
/ Builds (push) Successful in 2m13s
Details
/ Lint (push) Successful in 3m23s
Details
/ Tests (push) Successful in 2m13s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-11-28 06:06:55 +00:00
Marina Biryukova
e3141fc8e3 [#563] Ignore precondition headers with invalid date format
Some checks failed
/ DCO (pull_request) Successful in 1m43s
Details
/ Vulncheck (pull_request) Successful in 1m51s
Details
/ Builds (pull_request) Successful in 2m6s
Details
/ Lint (pull_request) Successful in 3m14s
Details
/ Tests (pull_request) Successful in 2m8s
Details
/ Builds (push) Has been cancelled
Details
/ Lint (push) Has been cancelled
Details
/ Tests (push) Has been cancelled
Details
/ Vulncheck (push) Has been cancelled
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-27 12:07:04 +03:00
Alex Vanin
a12fea8a5b Release v0.31.0
All checks were successful
/ DCO (pull_request) Successful in 1m36s
Details
/ Vulncheck (pull_request) Successful in 1m50s
Details
/ Builds (pull_request) Successful in 1m54s
Details
/ Lint (pull_request) Successful in 2m24s
Details
/ Tests (pull_request) Successful in 1m55s
Details
/ Vulncheck (push) Successful in 1m36s
Details
/ Builds (push) Successful in 1m53s
Details
/ Lint (push) Successful in 3m33s
Details
/ Tests (push) Successful in 2m26s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-11-20 15:45:07 +03:00
Denis Kirillov
9875307c9b [#556] Check bucket name not only during creation
All checks were successful
/ Vulncheck (push) Successful in 2m3s
Details
/ Builds (push) Successful in 1m36s
Details
/ Lint (push) Successful in 2m47s
Details
/ Tests (push) Successful in 2m56s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-20 08:13:27 +00:00
Denis Kirillov
b1775f9478 [#553] authmate: Add retryer to create access box
Some checks failed
/ DCO (pull_request) Successful in 1m20s
Details
/ Vulncheck (pull_request) Successful in 1m34s
Details
/ Builds (pull_request) Successful in 1m45s
Details
/ Lint (pull_request) Successful in 2m17s
Details
/ Tests (pull_request) Successful in 1m56s
Details
/ Builds (push) Has been cancelled
Details
/ Lint (push) Has been cancelled
Details
/ Tests (push) Has been cancelled
Details
/ Vulncheck (push) Has been cancelled
Details 
After using AddChain to provide access to container we have to wait:
* tx with APE chain be accepted by blockchain
* cache in storage node be updated

it takes a while. So we add retry
 (the same as when we add bucket settings during bucket creation)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-19 15:46:00 +03:00
Denis Kirillov
4fa45bdac2 [#553] authmate: Don't use basic acl 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-19 15:45:54 +03:00
Roman Loginov
368c7d2acd [#549] Add tracing attributes
All checks were successful
/ Vulncheck (push) Successful in 1m4s
Details
/ Builds (push) Successful in 1m44s
Details
/ Lint (push) Successful in 2m42s
Details
/ Tests (push) Successful in 2m0s
Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-11-18 11:55:31 +00:00
Vitaliy Potyarkin
31076796ce [#550] Execute CI on push to master
All checks were successful
/ Vulncheck (pull_request) Successful in 2m43s
Details
/ DCO (pull_request) Successful in 2m50s
Details
/ Builds (pull_request) Successful in 2m58s
Details
/ Lint (pull_request) Successful in 4m5s
Details
/ Tests (pull_request) Successful in 2m51s
Details
/ Vulncheck (push) Successful in 1m40s
Details
/ Builds (push) Successful in 1m22s
Details
/ Lint (push) Successful in 3m0s
Details
/ Tests (push) Successful in 2m9s
Details 
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2024-11-15 14:31:11 +03:00
Marina Biryukova
eff0de43d5 [#538] Return headers with 304 Not Modified 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-13 13:49:09 +00:00
Pavel Pogodaev
fb00dff83b [#540] Add md5 S3Tests compatability
All checks were successful
/ DCO (pull_request) Successful in 2m14s
Details
/ Vulncheck (pull_request) Successful in 2m20s
Details
/ Builds (pull_request) Successful in 2m46s
Details
/ Lint (pull_request) Successful in 4m4s
Details
/ Tests (pull_request) Successful in 3m31s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-11-13 14:50:16 +03:00
Denis Kirillov
d8f126b339 [#539] Fix listing v1 bookmark marker 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-12 12:58:09 +00:00
Marina Biryukova
7ab902d8d2 [#536] Add rule ID generation 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:51:02 +00:00
Marina Biryukova
0792fcf456 [#536] Fix error codes in lifecycle configuration check 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:51:02 +00:00
Marina Biryukova
c46ffa8146 [#536] Add prefix to lifecycle rule 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:51:02 +00:00
Marina Biryukova
3260308cc0 [#528] Check owner ID before deleting bucket 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:47:43 +00:00
Vitaliy Potyarkin
d6e6a13576 [#542] Stop using obsolete .github directory
All checks were successful
/ Builds (pull_request) Successful in 1m30s
Details
/ DCO (pull_request) Successful in 1m33s
Details
/ Vulncheck (pull_request) Successful in 1m49s
Details
/ Lint (pull_request) Successful in 3m18s
Details
/ Tests (pull_request) Successful in 2m17s
Details 
This commit is a part of multi-repo cleanup effort:
TrueCloudLab/frostfs-infra#136

Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2024-11-06 15:31:16 +03:00
Marina Biryukova
17d40245de [#505] docs: Add example of uploading file using presigned URL 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-02 08:53:54 +00:00
Marina Biryukova
979d85b046 [#505] authmate: Add flag for headers in generate-presigned-url cmd 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-02 08:53:54 +00:00
Roman Loginov
539dab8680 [#501] Add the trace id to the logs of middlware 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-11-02 08:51:48 +00:00
Roman Loginov
76008d4ba1 [#501] Consider using request logger in logAndSendError 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-11-02 08:51:48 +00:00
Alex Vanin
8bc19725ba [#521] Add documentation for multinet settings
All checks were successful
/ DCO (pull_request) Successful in 2m55s
Details
/ Vulncheck (pull_request) Successful in 3m35s
Details
/ Builds (pull_request) Successful in 1m59s
Details
/ Lint (pull_request) Successful in 3m25s
Details
/ Tests (pull_request) Successful in 2m0s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:27 +03:00
Alex Vanin
9e64304499 [#521] Use handler to register dial events 
While frostfs-node uses dial handler to udpate metric
value, gateway starts with simple event logging.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:27 +03:00
Alex Vanin
94504e9746 [#521] Use source dialer for gRPC connection to storage 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:27 +03:00
Alex Vanin
a8458dbc27 [#521] Add internal/net package with multinet dialer source 
Code is taken from frostfs-node#1422
Author: Dmitrii Stepanov (dstepanov-yadro)

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:26 +03:00
Alex Vanin
424038de6c [#524] Update pool to treat maintenance mode differently 
Contains these changes for pool component in SDK:
* frostfs-sdk-go#279 fix mm error counting during search operation
* frostfs-sdk-go#283 immediately mark mm node as unhealthy
* frostfs-sdk-go#278 do not reconnect to mm node

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-23 12:17:20 +00:00
Denis Kirillov
3cf27d281d [#509] Support fallback address when getting box
All checks were successful
/ DCO (pull_request) Successful in 1m26s
Details
/ Builds (pull_request) Successful in 1m36s
Details
/ Vulncheck (pull_request) Successful in 2m17s
Details
/ Lint (pull_request) Successful in 3m10s
Details
/ Tests (pull_request) Successful in 2m18s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
3c7cb82553 [#509] Init resolvers before first resolving 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
57b7e83380 [#509] Save isCustom flag into accessbox 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
6a90f4e624 [#509] Update docs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
cb3753f286 [#509] Add tests for custom credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
81209e308c [#509] Fix tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
b78e55e101 [#509] Support custom AWS credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Vladimir Domnich
25c24f5ce6 [#522] Add waiter to contract clients
All checks were successful
/ DCO (pull_request) Successful in 1m19s
Details
/ Vulncheck (pull_request) Successful in 1m37s
Details
/ Builds (pull_request) Successful in 1m42s
Details
/ Lint (pull_request) Successful in 2m54s
Details
/ Tests (pull_request) Successful in 1m49s
Details 
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2024-10-23 09:22:19 +03:00
Marina Biryukova
09c11262c6 [#516] Check Content-Md5 of lifecycle configuration 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-22 14:21:49 +00:00
Marina Biryukova
f120715a37 [#516] Convert expiration date to epoch 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-22 14:21:49 +00:00
Roman Loginov
aaed083d82 [#520] Support the continuous use of interceptors
All checks were successful
/ DCO (pull_request) Successful in 3m18s
Details
/ Builds (pull_request) Successful in 3m28s
Details
/ Vulncheck (pull_request) Successful in 3m25s
Details
/ Lint (pull_request) Successful in 3m51s
Details
/ Tests (pull_request) Successful in 3m29s
Details 
We can always add interceptors to the grpc
connection to the storage, since the actual
use will be controlled by the configuration
from the frostfs-observability library.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-10-21 11:49:22 +03:00
Marina Biryukova
e35b582fe2 [#506] Deny bucket names with dot
All checks were successful
/ DCO (pull_request) Successful in 1m14s
Details
/ Vulncheck (pull_request) Successful in 1m49s
Details
/ Builds (pull_request) Successful in 1m37s
Details
/ Lint (pull_request) Successful in 3m3s
Details
/ Tests (pull_request) Successful in 1m34s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 12:50:22 +03:00
Marina Biryukova
39fc7aa3ee [#404] Fix using encoding type in multipart upload listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 09:44:05 +00:00
Marina Biryukova
da41f47826 [#404] Add check of encoding type in object listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 09:44:05 +00:00
Marina Biryukova
9e5fb4be95 [#507] Return not implemented by default in bucket router 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-07 05:38:35 +00:00
Denis Kirillov
346243b159 [#450] Fix getPutPayloadSize
All checks were successful
/ DCO (pull_request) Successful in 1m17s
Details
/ Vulncheck (pull_request) Successful in 1m20s
Details
/ Builds (pull_request) Successful in 1m24s
Details
/ Lint (pull_request) Successful in 2m23s
Details
/ Tests (pull_request) Successful in 1m25s
Details 
If X-Amz-Decoded-Content-Length explicitly set use it even if value is 0

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-03 14:23:06 +03:00
Denis Kirillov
03481274f0 [#467] authmate: Add sign command 
Support singing arbitrary data using aws sigv4 algorithm

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-02 13:42:22 +00:00
Nikita Zinkevich
c2adbd758a [#488] middleware/auth: Add frostfs-to-s3 error transformation
All checks were successful
/ Vulncheck (pull_request) Successful in 1m3s
Details
/ DCO (pull_request) Successful in 1m27s
Details
/ Builds (pull_request) Successful in 1m27s
Details
/ Lint (pull_request) Successful in 3m5s
Details
/ Tests (pull_request) Successful in 1m56s
Details 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 13:18:25 +03:00
Nikita Zinkevich
bc17ab5e47 [#488] middleware/policy: Add frostfs-to-s3 error transformation 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
9fadfbbc2f [#488] Renamed api/errors, layer/frostfs and layer/tree package names 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
827ea1a41e [#488] Move layer/frostfs.go to layer/frostfs/frostfs.go 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00