TrueCloudLab/frostfs-s3-gw
18
3
Fork 20
Code Issues 30 Pull requests 9 Projects Releases 23 Packages Activity Actions
1319 commits 6 branches 48 tags 24 MiB
Commit graph

184 commits

Author SHA1 Message Date
Marina Biryukova
b8c93ed391 [#172] Convert handler config to interface 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-04 11:01:27 +00:00
Roman Loginov
c66c09765d [#196] Support soft memory limit setting 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-09-05 13:13:56 +00:00
Marina Biryukova
2fc328a6d2 [#195] Add log constants linter
All checks were successful
/ DCO (pull_request) Successful in 1m36s
Details
/ Vulncheck (pull_request) Successful in 1m41s
Details
/ Lint (pull_request) Successful in 8m31s
Details
/ Tests (1.19) (pull_request) Successful in 2m6s
Details
/ Tests (1.20) (pull_request) Successful in 1m57s
Details
/ Builds (1.19) (pull_request) Successful in 2m21s
Details
/ Builds (1.20) (pull_request) Successful in 9m3s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-08-28 12:58:44 +03:00
Denis Kirillov
b5fce5c8d2 [#168] Skip only invalid policies and copies instead of ignoring all of them
All checks were successful
/ DCO (pull_request) Successful in 2m48s
Details
/ Vulncheck (pull_request) Successful in 3m30s
Details
/ Builds (1.19) (pull_request) Successful in 3m39s
Details
/ Builds (1.20) (pull_request) Successful in 3m28s
Details
/ Lint (pull_request) Successful in 4m14s
Details
/ Tests (1.19) (pull_request) Successful in 3m3s
Details
/ Tests (1.20) (pull_request) Successful in 2m45s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-25 12:05:30 +03:00
Denis Kirillov
631d9d83b6 [#185] Fix payload reader 
When we use io.CopyBuffer it check for exact io.EOF matching,
so we need keep original EOF error otherwise io.CopyBuffer returns error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-24 18:01:08 +03:00
Roman Loginov
8efcc957ea [#96] Move log messages to constants
All checks were successful
/ DCO (pull_request) Successful in 1m35s
Details
/ Builds (1.19) (pull_request) Successful in 2m14s
Details
/ Builds (1.20) (pull_request) Successful in 2m9s
Details
/ Vulncheck (pull_request) Successful in 5m39s
Details
/ Lint (pull_request) Successful in 2m49s
Details
/ Tests (1.19) (pull_request) Successful in 7m34s
Details
/ Tests (1.20) (pull_request) Successful in 1m44s
Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-08-23 18:32:31 +03:00
Denis Kirillov
6b1f365e65 [#192] Support client cut 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-23 06:26:55 +00:00
Marina Biryukova
012ece40bb [#180] Fix linter issues
All checks were successful
/ Builds (1.19) (pull_request) Successful in 2m55s
Details
/ Builds (1.20) (pull_request) Successful in 2m47s
Details
/ DCO (pull_request) Successful in 3m4s
Details
/ Vulncheck (pull_request) Successful in 3m39s
Details
/ Lint (pull_request) Successful in 4m19s
Details
/ Tests (1.19) (pull_request) Successful in 3m52s
Details
/ Tests (1.20) (pull_request) Successful in 3m18s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-08-21 17:23:24 +03:00
Artem Tataurov
94a42fa273 [#51] Update frostfs-sdk-go 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-08-17 14:26:25 +03:00
Roman Loginov
40d7f844e3 [#137] Refactor context data retrievers 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-08-16 14:05:38 +00:00
Denis Kirillov
18878b66d3 [#175] Use gate owner as object owner
Some checks failed
/ DCO (pull_request) Failing after 3s
Details
/ Vulncheck (pull_request) Failing after 2s
Details
/ Builds (1.19) (pull_request) Failing after 3s
Details
/ Builds (1.20) (pull_request) Successful in 7m47s
Details
/ Lint (pull_request) Failing after 4s
Details
/ Tests (1.19) (pull_request) Successful in 8m8s
Details
/ Tests (1.20) (pull_request) Failing after 3s
Details 
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-08 12:22:33 +03:00
Denis Kirillov
fc90981c03 [#149] Update inner imports after moving middlewares 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-07-11 17:25:09 +03:00
Denis Kirillov
84358f6742 [#135] authmate: Support CRDT GSet for credentials 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-16 09:42:49 +03:00
Denis Kirillov
7a380fa46c [#135] frostfs: Add SEARCH operation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-16 09:42:41 +03:00
Denis Kirillov
0590f84d68 [#135] crdt: Add GSet 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-16 09:39:43 +03:00
Denis Kirillov
462589fc0c [#103] Return 504 http code on timeout errors
All checks were successful
/ Vulncheck (pull_request) Successful in 1m42s
Details
/ Lint (pull_request) Successful in 3m31s
Details
/ Tests (1.19) (pull_request) Successful in 2m57s
Details
/ Tests (1.20) (pull_request) Successful in 3m4s
Details
/ Builds (1.19) (pull_request) Successful in 2m35s
Details
/ Builds (1.20) (pull_request) Successful in 2m39s
Details
/ DCO (pull_request) Successful in 1m7s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-14 09:16:21 +03:00
Denis Kirillov
0bcda6ea37 [#133] Drop sync-tree 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-09 09:34:36 +03:00
Denis Kirillov
9dabaf6ecd [#133] Use tree pool from SDK 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-09 09:34:04 +03:00
Denis Kirillov
11f30a037b [#114] tree: Fix retry tests 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-05-23 11:35:44 +03:00
Denis Kirillov
136a186c14 [#114] tree: Don't ignore unhealthy endpoints 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-05-19 17:22:39 +03:00
Denis Kirillov
24390fdec8 [#110] tree: Add more logs for switching tree endpoints 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-05-19 17:20:38 +03:00
Denis Kirillov
1fdbfb0dab [#110] tree: Update errors to switch endpoint 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-05-19 17:20:34 +03:00
Alex Vanin
1406f57bba [#1] Update comment lines 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-05-19 07:30:04 +00:00
Denis Kirillov
b366e75366 [#81] Use impersonate bearer token 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-05-03 17:22:52 +03:00
Artem Tataurov
e487ee5b7d [#70] Add arrays of copies numbers for location constraints 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-03 13:48:26 +03:00
Denis Kirillov
69d8779daf [#74] tree: Simplify retry 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-26 16:39:57 +03:00
Denis Kirillov
9da77667f3 [#74] Add round tree retry 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-26 16:39:57 +03:00
Denis Kirillov
ca8791a5fd [#74] Support multiple tree endpoints 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-26 16:39:57 +03:00
Denis Kirillov
01afa1cae4 [#75] Make grpc tree client implementation internal 
Since we have pkg 'internal/frostfs/services/tree' that is downloading
during build we cannot export any package that is depended on it.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-03 14:04:35 +03:00
Denis Kirillov
a025f2e9c5 [#59] tree: Make interface for tree service client 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-29 09:34:10 +03:00
Denis Kirillov
a0937126cb [#64] Support new system attributes 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-22 11:46:58 +03:00
Denis Kirillov
0af06c3bd9 [TrueCloudLab#40] Add param to configure xml decoder 
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/" attribute
for CompleteMultipartUpload requests

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 15:44:13 +03:00
Alex Vanin
813aa2f173 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:38:08 +03:00
Denis Kirillov
388482e230 [#2] Rename internals 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-21 11:17:45 +03:00
Denis Kirillov
765fd28a49 [#2] Update build 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-20 18:24:30 +03:00
Alex Vanin
96dff367db [#1] Build S3 Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 12:43:52 +03:00
Denis Kirillov
094eb12578 [#726] Use client time on regular requests 
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-14 14:53:55 +03:00
Denis Kirillov
f59bf91a67 [#699] neofs/tree: Add healthcheck 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-15 13:59:54 +03:00
Denis Kirillov
3d08562843 [#505] Handle access denied from tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov
80beedf13e [#504] Use bktInfo in tree service to check owner 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov
5184ed66c6 [#657] Don't use <empty> to handle trailing slash 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 15:50:43 +03:00
Denis Kirillov
9cd4ef1ac4 [#657] Replace FileName with FilePath attribute 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 15:50:43 +03:00
Denis Kirillov
6108f8d929 [#624] Update tree service revision 
Remove extra request to tree service to know parentID

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 12:17:51 +03:00
Denis Kirillov
e33d6fceda [#624] Get full subTree for listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 12:17:51 +03:00
Alex Vanin
9590b39333 [#682] Return added node id in AddVersion 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-29 18:24:00 +03:00
Alex Vanin
a28d6b64b9 [#682] Return added node id in addNodeByPath 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-29 18:24:00 +03:00
Denis Kirillov
7b165d3f18 [#671] Fix using nested name in multipart upload 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 12:01:49 +03:00
Denis Kirillov
24e1f7e1dd [#545] Drop object search 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 11:49:08 +03:00
Denis Kirillov
e4ec983213 [#678] Update SDK to fix panic on go1.19 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 11:33:17 +03:00
Angira Kekteeva
5307211398 [#634] Add CopiesNumber in NeoFS requests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-16 17:49:15 +03:00
Angira Kekteeva
8eb1afbcb8 [#645] Replace tagging when PutObject 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-10 09:21:27 +03:00
Angira Kekteeva
f66e4a77cd [#577] Add IsDeleteMarker() to NodeVersion 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Denis Kirillov
08898f4fb4 [#615] Expose pool metrics 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-02 16:04:31 +03:00
Alex Vanin
5219965105 [#498] Remove log string sanitizer 
`zap` lib does all the job anyway.
Suppress CWE-117 from CodeQL.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-29 09:32:37 +03:00
Denis Kirillov
ec6039c53b [#627] Correct listing object same as directory 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 09:32:23 +03:00
Alex Vanin
685a5f0ce8 [#540] Do not set sticky bit in authmate container 
In public containers sticky bit allows to limit
ownership of the objects by request authors.

In private or public-read containers it doesn't
make any sense so sticky bit is redundant there.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-28 09:09:24 +03:00
Denis Kirillov
75afc0e4a6 [#601] Fix GetLatestVersionsByPrefix 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
0057f6b7db [#546] Add size and etag in nodeVersionInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
56eb2dc3dc [#569] Rename versioning consts 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
72013e08ec [#569] Refactor versioning in tree service 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
e46763e429 [#533] Fix typos 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
c88a9842db [#488] Sync using oid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
85c203e157 [#488] Sync using cid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
12a2060dd0 [#524] Optimize listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
d8ab1b4799 [#520] Fix saving two lock types simultaneously 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
87d19f09f7 [#519] Fix transform of AcessDenied error 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
cb5237fb82 [#510] Fix listing with prefix 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
64931c77f4 [#503] Handle leading slash in names 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
41c815441b [#502] Fix list empty bucket 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
f8496973b8 [#477] Aggregate fetching tags and lock in layer.TreeService 
New method aggregates GetObjectTagging and GetLock
methods in single RPC to the tree service.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
5304e68b9a [#477] Add getTreeNodes method 
Find multiple treeNodes from
subtree in single request.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9b1ccd39be [#475] Add extra attributes to partInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
5b9a5cadef [#452] Fix lint issues 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9fb3fb1274 [#452] Fix versions tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
dd534e8738 [#451] Handle lock objects using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
6fe46a5944 [#453] Use only FileName as path attribute 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
70957d75fd [#417] Complete multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
6b2ddcadd0 [#417] Abort multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
58f2bf44b1 [#417] List parts using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
bc0bdc7767 [#417] Upload part using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
e1b9a4432a [#417] List multipart uploads using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
13e01164d7 [#417] Create multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
24bad60048 [#449] Add prefix for user tags in tree service 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
385a2102f0 [#449] Add tree service for bucket tagging 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
99feb1d936 [#449] Add tree service for object tagging 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9c74cca9af [#420] Using tree service to list object versions 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
55c38e73e6 [#444] Handle trailing slash in names 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
cfa6098919 [#416] Make emptyOID a local variable 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
25477cdaf8 [#416] Use tree service to list objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
7e8b57605a [#413] Fix delete object without version 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
332de834ae [#413] Add stream reading 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
1e86413667 [#413] Add signature for tree requests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
37b1baed41 [#413] Refactor 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
977f176713 [#413] Use tree service to delete object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
49bd77d9cf [#413] Use tree service to head objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
36f3c43af5 [#413] Use tree service to put object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
ab5c44ac14 [#429] Refactor TreeService for notif,cors,settings 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
8eff857e41 [#429] Add tree service for cors 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
7520952792 [#429] Add tree service for notifications 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
b03ae827fb [#429] Refactor tree service 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00