TrueCloudLab/frostfs-s3-gw
15
3
Fork 18
Code Issues 19 Pull requests 6 Projects Releases 17 Packages Activity Actions
1443 commits 5 branches 40 tags 20 MiB
Commit graph

1443 commits

This branch
This branch
All branches
Author SHA1 Message Date
Denis Kirillov
57b7e83380 [#509] Save isCustom flag into accessbox 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
6a90f4e624 [#509] Update docs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
cb3753f286 [#509] Add tests for custom credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
81209e308c [#509] Fix tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Denis Kirillov
b78e55e101 [#509] Support custom AWS credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-23 15:01:31 +03:00
Vladimir Domnich
25c24f5ce6 [#522] Add waiter to contract clients
All checks were successful
/ DCO (pull_request) Successful in 1m19s
Details
/ Vulncheck (pull_request) Successful in 1m37s
Details
/ Builds (pull_request) Successful in 1m42s
Details
/ Lint (pull_request) Successful in 2m54s
Details
/ Tests (pull_request) Successful in 1m49s
Details 
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2024-10-23 09:22:19 +03:00
Marina Biryukova
09c11262c6 [#516] Check Content-Md5 of lifecycle configuration 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-22 14:21:49 +00:00
Marina Biryukova
f120715a37 [#516] Convert expiration date to epoch 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-22 14:21:49 +00:00
Roman Loginov
aaed083d82 [#520] Support the continuous use of interceptors
All checks were successful
/ DCO (pull_request) Successful in 3m18s
Details
/ Builds (pull_request) Successful in 3m28s
Details
/ Vulncheck (pull_request) Successful in 3m25s
Details
/ Lint (pull_request) Successful in 3m51s
Details
/ Tests (pull_request) Successful in 3m29s
Details 
We can always add interceptors to the grpc
connection to the storage, since the actual
use will be controlled by the configuration
from the frostfs-observability library.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-10-21 11:49:22 +03:00
Marina Biryukova
e35b582fe2 [#506] Deny bucket names with dot
All checks were successful
/ DCO (pull_request) Successful in 1m14s
Details
/ Vulncheck (pull_request) Successful in 1m49s
Details
/ Builds (pull_request) Successful in 1m37s
Details
/ Lint (pull_request) Successful in 3m3s
Details
/ Tests (pull_request) Successful in 1m34s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 12:50:22 +03:00
Marina Biryukova
39fc7aa3ee [#404] Fix using encoding type in multipart upload listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 09:44:05 +00:00
Marina Biryukova
da41f47826 [#404] Add check of encoding type in object listing 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-08 09:44:05 +00:00
Marina Biryukova
9e5fb4be95 [#507] Return not implemented by default in bucket router 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-10-07 05:38:35 +00:00
Denis Kirillov
346243b159 [#450] Fix getPutPayloadSize
All checks were successful
/ DCO (pull_request) Successful in 1m17s
Details
/ Vulncheck (pull_request) Successful in 1m20s
Details
/ Builds (pull_request) Successful in 1m24s
Details
/ Lint (pull_request) Successful in 2m23s
Details
/ Tests (pull_request) Successful in 1m25s
Details 
If X-Amz-Decoded-Content-Length explicitly set use it even if value is 0

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-03 14:23:06 +03:00
Denis Kirillov
03481274f0 [#467] authmate: Add sign command 
Support singing arbitrary data using aws sigv4 algorithm

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-02 13:42:22 +00:00
Nikita Zinkevich
c2adbd758a [#488] middleware/auth: Add frostfs-to-s3 error transformation
All checks were successful
/ Vulncheck (pull_request) Successful in 1m3s
Details
/ DCO (pull_request) Successful in 1m27s
Details
/ Builds (pull_request) Successful in 1m27s
Details
/ Lint (pull_request) Successful in 3m5s
Details
/ Tests (pull_request) Successful in 1m56s
Details 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 13:18:25 +03:00
Nikita Zinkevich
bc17ab5e47 [#488] middleware/policy: Add frostfs-to-s3 error transformation 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
9fadfbbc2f [#488] Renamed api/errors, layer/frostfs and layer/tree package names 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
827ea1a41e [#488] Move layer/frostfs.go to layer/frostfs/frostfs.go 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
968f10a72f [#488] Move layer/tree_service.go to layer/tree/tree_service.go 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Marina Biryukova
582e6ac642 [#503] Update SDK to fix error counting
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Vulncheck (pull_request) Successful in 2m15s
Details
/ Builds (pull_request) Successful in 2m5s
Details
/ Lint (pull_request) Successful in 3m45s
Details
/ Tests (pull_request) Successful in 2m1s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-27 10:36:48 +03:00
Pavel Pogodaev
99f273f9af [#461] Configure logger sampling policy
All checks were successful
/ DCO (pull_request) Successful in 57s
Details
/ Vulncheck (pull_request) Successful in 1m17s
Details
/ Builds (pull_request) Successful in 2m3s
Details
/ Lint (pull_request) Successful in 2m32s
Details
/ Tests (pull_request) Successful in 2m4s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-09-26 10:34:44 +03:00
Roman Loginov
cd96adef36 [#499] Fix of sighup traicing docs
All checks were successful
/ DCO (pull_request) Successful in 50s
Details
/ Vulncheck (pull_request) Successful in 1m14s
Details
/ Builds (pull_request) Successful in 1m22s
Details
/ Lint (pull_request) Successful in 1m50s
Details
/ Tests (pull_request) Successful in 1m27s
Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-09-25 14:34:18 +03:00
Denis Kirillov
738ce14f50 [#434] Remove container on failed bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-09-25 07:15:24 +00:00
Nikita Zinkevich
5358e39f71 [#496] Update frostfs-go-sdk for in-flight requests support
All checks were successful
/ DCO (pull_request) Successful in 1m12s
Details
/ Builds (pull_request) Successful in 1m3s
Details
/ Vulncheck (pull_request) Successful in 1m44s
Details
/ Lint (pull_request) Successful in 2m41s
Details
/ Tests (pull_request) Successful in 1m59s
Details 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-09-23 08:57:40 +03:00
Aleksey Savaitan
34c1426b9f [#484] Add root ca cert for telemetry configuration 
Signed-off-by: Aleksey Savaitan <a.savaitan@yadro.com>
 2024-09-19 11:07:13 +00:00
Roman Loginov
8ca73e2079 [#493] Fix of receiving VHS namespaces map
All checks were successful
/ Vulncheck (pull_request) Successful in 1m17s
Details
/ Builds (pull_request) Successful in 1m35s
Details
/ Lint (pull_request) Successful in 2m34s
Details
/ Tests (pull_request) Successful in 1m36s
Details
/ DCO (pull_request) Successful in 36s
Details 
In the process of forming a map with namespaces
for which VHS is enabled, we resolve the alias
of the namespace. The problem is that to resolve,
we need default namespace names, which in turn do
not have time to decide by this time. Therefore,
now the check for the default name takes place
directly in the prepareVHSNamespaces function
based on previously read default names.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-09-17 16:57:05 +03:00
Roman Loginov
a87c636b4c [#493] Fix X-Frostfs-S3-VHS header processing 
It is assumed that the X-Frostfs-S3-VHS header
will have the value enabled/disabled instead
of true/false.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-09-16 14:28:19 +03:00
Roman Ognev
26baf8a94e [#492] Add panic catchers to fuzzing tests 
Signed-off-by: Roman Ognev <r.ognev@yadro.com>
 2024-09-16 10:36:30 +00:00
Marina Biryukova
f187141ae5 [#486] Fix PUT object with negative Content-Length 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-16 08:45:46 +00:00
Marina Biryukova
3cffc782e9 [#486] Fix PUT encrypted object 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-16 08:45:46 +00:00
Marina Biryukova
d0e4d55772 [#460] Add network info cache 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-13 09:56:24 +00:00
Alex Vanin
42e72889a5 [#450] Add test for chunk-encoded object size
All checks were successful
/ DCO (pull_request) Successful in 1m9s
Details
/ Vulncheck (pull_request) Successful in 1m20s
Details
/ Builds (pull_request) Successful in 1m52s
Details
/ Lint (pull_request) Successful in 2m47s
Details
/ Tests (pull_request) Successful in 1m49s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-09-13 12:00:24 +03:00
Pavel Pogodaev
98815d5473 [#450] Fix aws-chunked header workflow 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-09-13 11:59:07 +03:00
Nikita Zinkevich
62615d7ab7 [#369] Request reproducer
All checks were successful
/ DCO (pull_request) Successful in 47s
Details
/ Builds (pull_request) Successful in 59s
Details
/ Vulncheck (pull_request) Successful in 1m15s
Details
/ Lint (pull_request) Successful in 2m15s
Details
/ Tests (pull_request) Successful in 1m20s
Details 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-09-11 15:25:09 +03:00
Nikita Zinkevich
575ab4d294 [#369] Enhanced http requests logging 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-09-11 15:25:09 +03:00
Marina Biryukova
d919e6cce2 [#482] Fix containers resolving
All checks were successful
/ DCO (pull_request) Successful in 1m8s
Details
/ Builds (pull_request) Successful in 1m7s
Details
/ Vulncheck (pull_request) Successful in 1m12s
Details
/ Lint (pull_request) Successful in 2m24s
Details
/ Tests (pull_request) Successful in 1m24s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-05 12:33:14 +03:00
Denis Kirillov
056f168d77 [#448] multipart: Support removing duplicated parts 
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-09-03 13:20:38 +00:00
Marina Biryukova
9bdfe2a016 [#479] Update APE to support s3:PatchObject action 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
d6b506f6d9 [#466] Implement PATCH for multipart objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
a2e0b92575 [#473] Add PATCH to extensions doc 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
b08f476ea7 [#462] Implement PATCH for regular objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Pavel Pogodaev
f4275d837a [#413] Add SECURITY.md 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-09-03 11:45:05 +00:00
Roman Ognev
664f83b2b7 [#480] Add fuzzing tests
All checks were successful
/ DCO (pull_request) Successful in 1m34s
Details
/ Vulncheck (pull_request) Successful in 1m45s
Details
/ Builds (1.22) (pull_request) Successful in 1m51s
Details
/ Builds (1.23) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m3s
Details
/ Tests (1.22) (pull_request) Successful in 1m52s
Details
/ Tests (1.23) (pull_request) Successful in 1m51s
Details 
Signed-off-by: Roman Ognev <r.ognev@yadro.com>
 2024-09-02 15:59:07 +03:00
Denis Kirillov
136b5521fe [#475] Support graceful_close_on_switch_timeout param 
This allows in-flight requests finish during rebalance

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-29 13:22:08 +00:00
Roman Loginov
a5f670d904 [#329] Reduce using mutex when update app settings 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-29 12:03:26 +00:00
Denis Kirillov
d76c4fe2a2 [#472] tree: Don't use sorted GetSubTree for nodes without FileName 
Sorted GetSubTree doesn't return nodes without FileName attribute
if there are more than 1000 nodes in subtree

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-27 09:43:29 +00:00
Nikita Zinkevich
0637133c61 [#470] Update Go version to 1.22
All checks were successful
/ DCO (pull_request) Successful in 1m7s
Details
/ Vulncheck (pull_request) Successful in 1m13s
Details
/ Builds (1.22) (pull_request) Successful in 1m32s
Details
/ Builds (1.23) (pull_request) Successful in 1m34s
Details
/ Lint (pull_request) Successful in 2m51s
Details
/ Tests (1.22) (pull_request) Successful in 1m39s
Details
/ Tests (1.23) (pull_request) Successful in 1m41s
Details 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-08-23 12:18:42 +03:00
Roman Loginov
bf00fa6aa9 [#449] Add support headers for vhs and servername 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-23 08:35:05 +00:00
Roman Loginov
ff690ce996 [#446] Add tests for address style middleware 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-23 08:35:05 +00:00