TrueCloudLab/frostfs-s3-gw
15
3
Fork 18
Code Issues 19 Pull requests 6 Projects Releases 17 Packages Activity Actions
1434 commits 5 branches 40 tags 20 MiB
Commit graph

217 commits

Author SHA1 Message Date
Denis Kirillov
b456e6a287 [#509] Add tests for custom credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-11 15:41:35 +03:00
Denis Kirillov
4e73270b81 [#509] Fix tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-11 15:41:35 +03:00
Denis Kirillov
07c8923614 [#509] Support custom AWS credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-10-11 15:41:30 +03:00
Nikita Zinkevich
9fadfbbc2f [#488] Renamed api/errors, layer/frostfs and layer/tree package names 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Nikita Zinkevich
827ea1a41e [#488] Move layer/frostfs.go to layer/frostfs/frostfs.go 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-10-02 12:35:04 +03:00
Marina Biryukova
d0e4d55772 [#460] Add network info cache 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-13 09:56:24 +00:00
Nikita Zinkevich
575ab4d294 [#369] Enhanced http requests logging 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-09-11 15:25:09 +03:00
Denis Kirillov
056f168d77 [#448] multipart: Support removing duplicated parts 
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-09-03 13:20:38 +00:00
Marina Biryukova
d6b506f6d9 [#466] Implement PATCH for multipart objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Marina Biryukova
b08f476ea7 [#462] Implement PATCH for regular objects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-09-03 11:57:59 +00:00
Denis Kirillov
d76c4fe2a2 [#472] tree: Don't use sorted GetSubTree for nodes without FileName 
Sorted GetSubTree doesn't return nodes without FileName attribute
if there are more than 1000 nodes in subtree

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-27 09:43:29 +00:00
Roman Loginov
534ae7f0f1 [#446] Add support virtual-hosted-style 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2024-08-23 08:35:05 +00:00
Ekaterina Lebedeva
77673797f9 [#474] lint: Update golangci-lint to v1.60 and fix issues
All checks were successful
/ DCO (pull_request) Successful in 55s
Details
/ Vulncheck (pull_request) Successful in 1m17s
Details
/ Builds (1.21) (pull_request) Successful in 1m34s
Details
/ Builds (1.22) (pull_request) Successful in 1m32s
Details
/ Lint (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 1m44s
Details
/ Tests (1.22) (pull_request) Successful in 1m42s
Details 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-08-22 19:13:52 +03:00
Marina Biryukova
0644067496 [#412] Store creation epoch in tree service 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Marina Biryukova
481520705a [#42] Support expiration lifecycle 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-21 10:38:35 +03:00
Denis Kirillov
f5326b9f04 [#437] tree: Support removing old split system nodes 
It's need to fit user expectation on deleting CORS for example.
Previously after removing cors (that was uploaded in split manner)
we can still get some data (from other node)
because deletion worked only for latest node version.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-26 06:54:10 +00:00
Marina Biryukova
c506620199 [#439] Update SDK version 
New SDK version supports extended log records in pool component during inner nodes health check.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-24 12:16:37 +00:00
Denis Kirillov
6cb0026007 [#427] layer: Split FrostFS ReadObject to separate methods
All checks were successful
/ Builds (1.21) (pull_request) Successful in 1m20s
Details
/ Builds (1.22) (pull_request) Successful in 1m15s
Details
/ Vulncheck (pull_request) Successful in 1m6s
Details
/ DCO (pull_request) Successful in 57s
Details
/ Lint (pull_request) Successful in 2m59s
Details
/ Tests (1.21) (pull_request) Successful in 1m9s
Details
/ Tests (1.22) (pull_request) Successful in 1m23s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-23 16:53:59 +03:00
Marina Biryukova
971006a28c [#422] Support separate container for CORS 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-23 12:33:29 +00:00
Alex Vanin
ec349e4523 [#430] Adopt compatibility workarounds in Tree API 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-07-19 14:47:47 +03:00
Denis Kirillov
977a20760b [#430] Delete all split version at once
All checks were successful
/ DCO (pull_request) Successful in 8m40s
Details
/ Vulncheck (pull_request) Successful in 13m10s
Details
/ Builds (1.21) (pull_request) Successful in 14m11s
Details
/ Builds (1.22) (pull_request) Successful in 14m18s
Details
/ Lint (pull_request) Successful in 16m31s
Details
/ Tests (1.21) (pull_request) Successful in 10m48s
Details
/ Tests (1.22) (pull_request) Successful in 14m31s
Details 
Previously after split we can get two `null` versioned object with the same key
and deleting such key removes only one node/object.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-19 11:26:51 +03:00
Denis Kirillov
456319d2f1 [#430] Fix split tree 
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated, so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-19 11:24:46 +03:00
Denis Kirillov
70eedfc077 [#414] authmate: Add register-user command
All checks were successful
/ Builds (1.20) (pull_request) Successful in 14m30s
Details
/ Builds (1.21) (pull_request) Successful in 14m25s
Details
/ DCO (pull_request) Successful in 6m47s
Details
/ Vulncheck (pull_request) Successful in 6m59s
Details
/ Lint (pull_request) Successful in 11m52s
Details
/ Tests (1.20) (pull_request) Successful in 8m24s
Details
/ Tests (1.21) (pull_request) Successful in 8m19s
Details 
New command allows register user in frostfsid and
set allowed rules in policy contract

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-08 14:13:00 +03:00
Denis Kirillov
465eaa816a [#372] Drop [e]ACL related code
All checks were successful
/ DCO (pull_request) Successful in 2m15s
Details
/ Vulncheck (pull_request) Successful in 2m55s
Details
/ Builds (1.20) (pull_request) Successful in 3m46s
Details
/ Builds (1.21) (pull_request) Successful in 3m48s
Details
/ Lint (pull_request) Successful in 5m26s
Details
/ Tests (1.20) (pull_request) Successful in 3m34s
Details
/ Tests (1.21) (pull_request) Successful in 3m18s
Details 
Always consider buckets as APE compatible

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-01 16:58:44 +03:00
Denis Kirillov
9432782ce6 [#401] Drop notifications
All checks were successful
/ DCO (pull_request) Successful in 2m5s
Details
/ Builds (1.20) (pull_request) Successful in 2m40s
Details
/ Builds (1.21) (pull_request) Successful in 2m33s
Details
/ Vulncheck (pull_request) Successful in 2m22s
Details
/ Lint (pull_request) Successful in 4m24s
Details
/ Tests (1.20) (pull_request) Successful in 2m48s
Details
/ Tests (1.21) (pull_request) Successful in 2m45s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-25 15:49:37 +03:00
Denis Kirillov
2b04fcb5ec [#406] Remove control api 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-06-21 06:36:56 +00:00
Anoke
58850f590e [#335] Improve determining AccessBox latest version 
Signed-off-by: Anoke <rustamgta1011@gmail.com>
 2024-06-06 12:35:48 +00:00
Alex Vanin
6bf6a3b1a3 [#362] Check user and groups during policy check
All checks were successful
/ DCO (pull_request) Successful in 4m8s
Details
/ Vulncheck (pull_request) Successful in 4m10s
Details
/ Builds (1.20) (pull_request) Successful in 5m33s
Details
/ Builds (1.21) (pull_request) Successful in 5m24s
Details
/ Lint (pull_request) Successful in 8m32s
Details
/ Tests (1.20) (pull_request) Successful in 5m9s
Details
/ Tests (1.21) (pull_request) Successful in 4m52s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-08 15:25:14 +03:00
Marina Biryukova
2ab655b909 [#380] Add test for credentials versioning 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-05-03 07:24:13 +00:00
Pavel Pogodaev
3c436d8de9 [#365] Include iam user tags in query
All checks were successful
/ Vulncheck (pull_request) Successful in 1m48s
Details
/ Builds (1.20) (pull_request) Successful in 2m30s
Details
/ Builds (1.21) (pull_request) Successful in 1m25s
Details
/ Lint (pull_request) Successful in 3m52s
Details
/ Tests (1.20) (pull_request) Successful in 2m24s
Details
/ Tests (1.21) (pull_request) Successful in 2m22s
Details
/ DCO (pull_request) Successful in 45s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-04-22 10:47:43 +03:00
Marina Biryukova
e22ff52165 [#367] Add check of AccessBox attributes 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-04-19 06:25:26 +00:00
Denis Kirillov
5315f7b733 [#269] Create frostfsid wrapper with cache
All checks were successful
/ DCO (pull_request) Successful in 2m10s
Details
/ Vulncheck (pull_request) Successful in 2m0s
Details
/ Builds (1.20) (pull_request) Successful in 2m31s
Details
/ Builds (1.21) (pull_request) Successful in 1m31s
Details
/ Lint (pull_request) Successful in 3m34s
Details
/ Tests (1.20) (pull_request) Successful in 2m26s
Details
/ Tests (1.21) (pull_request) Successful in 2m21s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-18 09:32:30 +03:00
Denis Kirillov
29a2dae40c [#269] Move frostfsid client to separate package 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-17 12:11:23 +03:00
Alex Vanin
6da1acc554 [#360] Use 'c' prefix for bucket policies instead of 'n' 
With 'c' prefix, acl chains become shorter, thus gateway
receives shorter results and avoids sessions to neo-go.

There is still issue with many IAM rules.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-10 17:56:47 +03:00
Alex Vanin
cb83f7646f [#347] port: Explicitly specify sorting order of subtree for object listing
Some checks failed
/ DCO (pull_request) Successful in 1m56s
Details
/ Vulncheck (pull_request) Failing after 4m57s
Details
/ Builds (1.20) (pull_request) Successful in 5m54s
Details
/ Builds (1.21) (pull_request) Successful in 5m56s
Details
/ Lint (pull_request) Successful in 13m10s
Details
/ Tests (1.20) (pull_request) Successful in 5m34s
Details
/ Tests (1.21) (pull_request) Successful in 3m22s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-04-09 18:57:47 +03:00
Denis Kirillov
8407b3ea4c [#352] policy: Use iterators to list chains 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-04 12:51:12 +00:00
Denis Kirillov
8669bf6b50 [#346] acl: Update APE and fix using
All checks were successful
/ DCO (pull_request) Successful in 2m57s
Details
/ Vulncheck (pull_request) Successful in 3m33s
Details
/ Lint (pull_request) Successful in 4m44s
Details
/ Tests (1.20) (pull_request) Successful in 3m38s
Details
/ Tests (1.21) (pull_request) Successful in 3m29s
Details
/ Builds (1.20) (pull_request) Successful in 1m12s
Details
/ Builds (1.21) (pull_request) Successful in 3m23s
Details 
* Remove native policy when remove bucket policy
* Allow policies that contain only s3 compatible statements
(now deny rules cannot be converted to native rules)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-04-02 12:43:04 +00:00
Pavel Pogodaev
bfcde09f07 [#291] server auto re-binding
Some checks failed
/ Vulncheck (pull_request) Failing after 1m38s
Details
/ DCO (pull_request) Successful in 1m43s
Details
/ Builds (1.20) (pull_request) Successful in 2m17s
Details
/ Builds (1.21) (pull_request) Successful in 1m57s
Details
/ Lint (pull_request) Successful in 5m7s
Details
/ Tests (1.20) (pull_request) Successful in 2m32s
Details
/ Tests (1.21) (pull_request) Successful in 2m8s
Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-03-27 14:28:50 +03:00
Denis Kirillov
80c7b73eb9 [#306] In APE buckets forbid canned acl except private
Some checks failed
/ DCO (pull_request) Successful in 2m50s
Details
/ Vulncheck (pull_request) Failing after 3m15s
Details
/ Builds (1.20) (pull_request) Successful in 3m39s
Details
/ Builds (1.21) (pull_request) Successful in 3m41s
Details
/ Lint (pull_request) Successful in 5m48s
Details
/ Tests (1.20) (pull_request) Successful in 4m0s
Details
/ Tests (1.21) (pull_request) Successful in 3m53s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-19 16:57:26 +03:00
Denis Kirillov
62cc5a04a7 [#328] Log error on failed response writing
Some checks failed
/ DCO (pull_request) Successful in 3m34s
Details
/ Vulncheck (pull_request) Failing after 4m18s
Details
/ Builds (1.20) (pull_request) Successful in 4m58s
Details
/ Builds (1.21) (pull_request) Successful in 4m24s
Details
/ Lint (pull_request) Successful in 7m27s
Details
/ Tests (1.20) (pull_request) Successful in 5m24s
Details
/ Tests (1.21) (pull_request) Successful in 5m0s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-15 11:02:26 +03:00
Denis Kirillov
6788306998 [#328] Log invalid tree service KVs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 15:35:23 +03:00
Denis Kirillov
4ee3648183 [#328] Log invalid lock enabled header 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 15:09:51 +03:00
Denis Kirillov
ee48d1dc85 [#325] Log error on failed request id generation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-04 09:49:41 +00:00
Marina Biryukova
70043c4800 [#324] Close nns resolver after use 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-03-04 09:06:26 +00:00
Denis Kirillov
8050ca2d51 [#306] Use session token for container read operations
All checks were successful
/ DCO (pull_request) Successful in 1m54s
Details
/ Vulncheck (pull_request) Successful in 1m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m49s
Details
/ Builds (1.21) (pull_request) Successful in 1m56s
Details
/ Lint (pull_request) Successful in 3m59s
Details
/ Tests (1.20) (pull_request) Successful in 2m30s
Details
/ Tests (1.21) (pull_request) Successful in 2m19s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 18:14:33 +03:00
Denis Kirillov
fabb4134bc [#318] Use log msg from constants
All checks were successful
/ DCO (pull_request) Successful in 1m44s
Details
/ Builds (1.20) (pull_request) Successful in 2m24s
Details
/ Builds (1.21) (pull_request) Successful in 2m18s
Details
/ Vulncheck (pull_request) Successful in 2m17s
Details
/ Lint (pull_request) Successful in 2m36s
Details
/ Tests (1.20) (pull_request) Successful in 1m42s
Details
/ Tests (1.21) (pull_request) Successful in 1m32s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Denis Kirillov
4741e74210 [#318] Log successfully authenticated accessKeyIDs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-29 17:30:28 +03:00
Marina Biryukova
31da31862a [#300] Update error logging in DeleteMultipleObjects 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-02-29 14:24:32 +00:00
Denis Kirillov
56b50f2075 [#306] Remove flag to disable policy contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov
8f89f275bd [#306] Save bucket policy as native chain 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00