TrueCloudLab/frostfs-s3-gw
17
3
Fork 20
Code Issues 25 Pull requests 6 Projects Releases 24 Packages 1 Activity Actions
1475 commits 6 branches 50 tags 25 MiB
Commit graph

1475 commits

This branch
This branch
All branches
Author SHA1 Message Date
Roman Loginov
a990fb13d1 [#583] Fix list-buckets vhs routing 
The problem is that with VHS requests,
the list-buckets operation does not work
because the request is filtered on
list-objects-v1. Since list-buckets can
also have query parameters, in the end it
is necessary to distinguish list-buckets
from list-objects-v1 only by the presence
of the bucket name in the URL (provided
that the request is in VHS style).

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-12-17 13:02:28 +03:00
Roman Loginov
c1f3110dcc [#584] Return BucketAlreadyExists when global domain taken 
A situation may occur when the global
domain is already occupied when
creating the container. The error
comes from the nns smart contract.
This error actually means that the
bucket has already been created.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-12-16 17:06:43 +03:00
Roman Loginov
3f599f6bc2 Release v0.31.2
All checks were successful
/ DCO (pull_request) Successful in 2m57s
Details
/ Vulncheck (pull_request) Successful in 3m12s
Details
/ Builds (pull_request) Successful in 3m57s
Details
/ Lint (pull_request) Successful in 4m17s
Details
/ Tests (pull_request) Successful in 4m1s
Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-12-13 10:32:48 +03:00
Roman Loginov
ca5536d6d5 [#576] Update frostfs-sdk-go version and drop frostfs-api-go
All checks were successful
/ DCO (pull_request) Successful in 4m15s
Details
/ Vulncheck (pull_request) Successful in 5m5s
Details
/ Builds (pull_request) Successful in 3m23s
Details
/ Lint (pull_request) Successful in 4m47s
Details
/ Tests (pull_request) Successful in 5m43s
Details 
The new version of frostfs-sdk-go
contains a fix to solve the problem
of not being able to delete an EC object.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-12-13 05:05:48 +03:00
Alex Vanin
3cd88d6204 Release v0.31.1
All checks were successful
/ Vulncheck (push) Successful in 1m23s
Details
/ Builds (push) Successful in 1m34s
Details
/ Lint (push) Successful in 2m52s
Details
/ Tests (push) Successful in 2m43s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-11-29 13:35:58 +00:00
Pavel Pogodaev
e71ba5e22a [#543] Add md5 sse-c S3Tests compatability
All checks were successful
/ Vulncheck (push) Successful in 1m28s
Details
/ Builds (push) Successful in 2m13s
Details
/ Lint (push) Successful in 3m23s
Details
/ Tests (push) Successful in 2m13s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-11-28 06:06:55 +00:00
Marina Biryukova
e3141fc8e3 [#563] Ignore precondition headers with invalid date format
Some checks failed
/ DCO (pull_request) Successful in 1m43s
Details
/ Vulncheck (pull_request) Successful in 1m51s
Details
/ Builds (pull_request) Successful in 2m6s
Details
/ Lint (pull_request) Successful in 3m14s
Details
/ Tests (pull_request) Successful in 2m8s
Details
/ Builds (push) Has been cancelled
Details
/ Lint (push) Has been cancelled
Details
/ Tests (push) Has been cancelled
Details
/ Vulncheck (push) Has been cancelled
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-27 12:07:04 +03:00
Alex Vanin
a12fea8a5b Release v0.31.0
All checks were successful
/ DCO (pull_request) Successful in 1m36s
Details
/ Vulncheck (pull_request) Successful in 1m50s
Details
/ Builds (pull_request) Successful in 1m54s
Details
/ Lint (pull_request) Successful in 2m24s
Details
/ Tests (pull_request) Successful in 1m55s
Details
/ Vulncheck (push) Successful in 1m36s
Details
/ Builds (push) Successful in 1m53s
Details
/ Lint (push) Successful in 3m33s
Details
/ Tests (push) Successful in 2m26s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-11-20 15:45:07 +03:00
Denis Kirillov
9875307c9b [#556] Check bucket name not only during creation
All checks were successful
/ Vulncheck (push) Successful in 2m3s
Details
/ Builds (push) Successful in 1m36s
Details
/ Lint (push) Successful in 2m47s
Details
/ Tests (push) Successful in 2m56s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-20 08:13:27 +00:00
Denis Kirillov
b1775f9478 [#553] authmate: Add retryer to create access box
Some checks failed
/ DCO (pull_request) Successful in 1m20s
Details
/ Vulncheck (pull_request) Successful in 1m34s
Details
/ Builds (pull_request) Successful in 1m45s
Details
/ Lint (pull_request) Successful in 2m17s
Details
/ Tests (pull_request) Successful in 1m56s
Details
/ Builds (push) Has been cancelled
Details
/ Lint (push) Has been cancelled
Details
/ Tests (push) Has been cancelled
Details
/ Vulncheck (push) Has been cancelled
Details 
After using AddChain to provide access to container we have to wait:
* tx with APE chain be accepted by blockchain
* cache in storage node be updated

it takes a while. So we add retry
 (the same as when we add bucket settings during bucket creation)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-19 15:46:00 +03:00
Denis Kirillov
4fa45bdac2 [#553] authmate: Don't use basic acl 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-19 15:45:54 +03:00
Roman Loginov
368c7d2acd [#549] Add tracing attributes
All checks were successful
/ Vulncheck (push) Successful in 1m4s
Details
/ Builds (push) Successful in 1m44s
Details
/ Lint (push) Successful in 2m42s
Details
/ Tests (push) Successful in 2m0s
Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-11-18 11:55:31 +00:00
Vitaliy Potyarkin
31076796ce [#550] Execute CI on push to master
All checks were successful
/ Vulncheck (pull_request) Successful in 2m43s
Details
/ DCO (pull_request) Successful in 2m50s
Details
/ Builds (pull_request) Successful in 2m58s
Details
/ Lint (pull_request) Successful in 4m5s
Details
/ Tests (pull_request) Successful in 2m51s
Details
/ Vulncheck (push) Successful in 1m40s
Details
/ Builds (push) Successful in 1m22s
Details
/ Lint (push) Successful in 3m0s
Details
/ Tests (push) Successful in 2m9s
Details 
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2024-11-15 14:31:11 +03:00
Marina Biryukova
eff0de43d5 [#538] Return headers with 304 Not Modified 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-13 13:49:09 +00:00
Pavel Pogodaev
fb00dff83b [#540] Add md5 S3Tests compatability
All checks were successful
/ DCO (pull_request) Successful in 2m14s
Details
/ Vulncheck (pull_request) Successful in 2m20s
Details
/ Builds (pull_request) Successful in 2m46s
Details
/ Lint (pull_request) Successful in 4m4s
Details
/ Tests (pull_request) Successful in 3m31s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-11-13 14:50:16 +03:00
Denis Kirillov
d8f126b339 [#539] Fix listing v1 bookmark marker 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-11-12 12:58:09 +00:00
Marina Biryukova
7ab902d8d2 [#536] Add rule ID generation 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:51:02 +00:00
Marina Biryukova
0792fcf456 [#536] Fix error codes in lifecycle configuration check 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:51:02 +00:00
Marina Biryukova
c46ffa8146 [#536] Add prefix to lifecycle rule 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:51:02 +00:00
Marina Biryukova
3260308cc0 [#528] Check owner ID before deleting bucket 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-12 12:47:43 +00:00
Vitaliy Potyarkin
d6e6a13576 [#542] Stop using obsolete .github directory
All checks were successful
/ Builds (pull_request) Successful in 1m30s
Details
/ DCO (pull_request) Successful in 1m33s
Details
/ Vulncheck (pull_request) Successful in 1m49s
Details
/ Lint (pull_request) Successful in 3m18s
Details
/ Tests (pull_request) Successful in 2m17s
Details 
This commit is a part of multi-repo cleanup effort:
TrueCloudLab/frostfs-infra#136

Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2024-11-06 15:31:16 +03:00
Marina Biryukova
17d40245de [#505] docs: Add example of uploading file using presigned URL 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-02 08:53:54 +00:00
Marina Biryukova
979d85b046 [#505] authmate: Add flag for headers in generate-presigned-url cmd 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-11-02 08:53:54 +00:00
Roman Loginov
539dab8680 [#501] Add the trace id to the logs of middlware 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-11-02 08:51:48 +00:00
Roman Loginov
76008d4ba1 [#501] Consider using request logger in logAndSendError 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-11-02 08:51:48 +00:00
Alex Vanin
8bc19725ba [#521] Add documentation for multinet settings
All checks were successful
/ DCO (pull_request) Successful in 2m55s
Details
/ Vulncheck (pull_request) Successful in 3m35s
Details
/ Builds (pull_request) Successful in 1m59s
Details
/ Lint (pull_request) Successful in 3m25s
Details
/ Tests (pull_request) Successful in 2m0s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:27 +03:00
Alex Vanin
9e64304499 [#521] Use handler to register dial events 
While frostfs-node uses dial handler to udpate metric
value, gateway starts with simple event logging.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:27 +03:00
Alex Vanin
94504e9746 [#521] Use source dialer for gRPC connection to storage 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:27 +03:00
Alex Vanin
a8458dbc27 [#521] Add internal/net package with multinet dialer source 
Code is taken from frostfs-node#1422
Author: Dmitrii Stepanov (dstepanov-yadro)

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-29 15:55:26 +03:00
Alex Vanin
424038de6c [#524] Update pool to treat maintenance mode differently 
Contains these changes for pool component in SDK:
* frostfs-sdk-go#279 fix mm error counting during search operation
* frostfs-sdk-go#283 immediately mark mm node as unhealthy
* frostfs-sdk-go#278 do not reconnect to mm node

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-10-23 12:17:20 +00:00
Denis Kirillov
3cf27d281d [#509] Support fallback address when getting box
All checks were successful
/ DCO (pull_request) Successful in 1m26s
Details
/ Builds (pull_request) Successful in 1m36s
Details
/ Vulncheck (pull_request) Successful in 2m17s
Details
/ Lint (pull_request) Successful in 3m10s
Details
/ Tests (pull_request) Successful in 2m18s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
3c7cb82553 [#509] Init resolvers before first resolving 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
57b7e83380 [#509] Save isCustom flag into accessbox 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
6a90f4e624 [#509] Update docs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
cb3753f286 [#509] Add tests for custom credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
81209e308c [#509] Fix tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
b78e55e101 [#509] Support custom AWS credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Vladimir Domnich
25c24f5ce6 [#522] Add waiter to contract clients
All checks were successful
/ DCO (pull_request) Successful in 1m19s
Details
/ Vulncheck (pull_request) Successful in 1m37s
Details
/ Builds (pull_request) Successful in 1m42s
Details
/ Lint (pull_request) Successful in 2m54s
Details
/ Tests (pull_request) Successful in 1m49s
Details 
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2024-10-23 09:22:19 +03:00
Marina Biryukova
09c11262c6 [#516] Check Content-Md5 of lifecycle configuration 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-22 14:21:49 +00:00
Marina Biryukova
f120715a37 [#516] Convert expiration date to epoch 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-22 14:21:49 +00:00
Roman Loginov
aaed083d82 [#520] Support the continuous use of interceptors
All checks were successful
/ DCO (pull_request) Successful in 3m18s
Details
/ Builds (pull_request) Successful in 3m28s
Details
/ Vulncheck (pull_request) Successful in 3m25s
Details
/ Lint (pull_request) Successful in 3m51s
Details
/ Tests (pull_request) Successful in 3m29s
Details 
We can always add interceptors to the grpc
connection to the storage, since the actual
use will be controlled by the configuration
from the frostfs-observability library.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-10-21 11:49:22 +03:00
Marina Biryukova
e35b582fe2 [#506] Deny bucket names with dot
All checks were successful
/ DCO (pull_request) Successful in 1m14s
Details
/ Vulncheck (pull_request) Successful in 1m49s
Details
/ Builds (pull_request) Successful in 1m37s
Details
/ Lint (pull_request) Successful in 3m3s
Details
/ Tests (pull_request) Successful in 1m34s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 12:50:22 +03:00
Marina Biryukova
39fc7aa3ee [#404] Fix using encoding type in multipart upload listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 09:44:05 +00:00
Marina Biryukova
da41f47826 [#404] Add check of encoding type in object listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 09:44:05 +00:00
Marina Biryukova
9e5fb4be95 [#507] Return not implemented by default in bucket router 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-07 05:38:35 +00:00
Denis Kirillov
346243b159 [#450] Fix getPutPayloadSize
All checks were successful
/ DCO (pull_request) Successful in 1m17s
Details
/ Vulncheck (pull_request) Successful in 1m20s
Details
/ Builds (pull_request) Successful in 1m24s
Details
/ Lint (pull_request) Successful in 2m23s
Details
/ Tests (pull_request) Successful in 1m25s
Details 
If X-Amz-Decoded-Content-Length explicitly set use it even if value is 0

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-03 14:23:06 +03:00
Denis Kirillov
03481274f0 [#467] authmate: Add sign command 
Support singing arbitrary data using aws sigv4 algorithm

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-02 13:42:22 +00:00
Nikita Zinkevich
c2adbd758a [#488] middleware/auth: Add frostfs-to-s3 error transformation
All checks were successful
/ Vulncheck (pull_request) Successful in 1m3s
Details
/ DCO (pull_request) Successful in 1m27s
Details
/ Builds (pull_request) Successful in 1m27s
Details
/ Lint (pull_request) Successful in 3m5s
Details
/ Tests (pull_request) Successful in 1m56s
Details 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 13:18:25 +03:00
Nikita Zinkevich
bc17ab5e47 [#488] middleware/policy: Add frostfs-to-s3 error transformation 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
9fadfbbc2f [#488] Renamed api/errors, layer/frostfs and layer/tree package names 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00