TrueCloudLab
/
frostfs-s3-gw
15
2
Fork 16
Code Issues 21 Pull Requests 9 Actions Packages Projects Releases 9 Activity
1,089 Commits
6 Branches
28 Tags
19 MiB
Commit Graph

324 Commits (751d66bde0bf94c7a59b68c188ef11a289f68bc3)

Author SHA1 Message Date
Denis Kirillov 462589fc0c [#103] Return 504 http code on timeout errors
/ Vulncheck (pull_request) Successful in 1m42s Details
/ Lint (pull_request) Successful in 3m31s Details
/ Tests (1.19) (pull_request) Successful in 2m57s Details
/ Tests (1.20) (pull_request) Successful in 3m4s Details
/ Builds (1.19) (pull_request) Successful in 2m35s Details
/ Builds (1.20) (pull_request) Successful in 2m39s Details
/ DCO (pull_request) Successful in 1m7s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-14 09:16:21 +03:00
Denis Kirillov 4a6e3a19ce [#125] Handle negative Content-Length on put 
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-02 14:16:19 +00:00
Alexey Vanin 868edfdb31 [#78] Add test of bucket removal with object not found error
DCO Details
Builds (1.20) Details
Tests (1.19) Details
Tests (1.20) Details
Builds (1.19) Details
Lint Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-06-01 16:13:28 +03:00
Artem Tataurov a0f0d792b8 [#78] layer: Clean up already removed object from tree 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-24 17:42:46 +03:00
Artem Tataurov 9f186d9aba [#104] app: Reload copies numbers on SIGHUP 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-23 13:19:58 +03:00
Artem Tataurov e24bc3f2ce [#101] app: Refactor the default copies number setting 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-17 11:36:28 +03:00
Artem Tataurov e487ee5b7d [#70] Add arrays of copies numbers for location constraints 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-03 13:48:26 +03:00
Denis Kirillov ee8cce662b [#74] service/tree: Add logger 
Log error instead of failing when multiple unversioned nodes are found

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-26 16:39:57 +03:00
Denis Kirillov 70ec5a0a5b [#83] Don't create extra delete marker 
We shouldn't create delete marker if:
1. object doesn't exist at all
2. last version is already a delete marker

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-19 17:56:11 +03:00
Artem Tataurov a5c89b78bc [#79] Return X-Owner-Id in head-bucket response 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-04-17 10:12:25 +03:00
Denis Kirillov a025f2e9c5 [#59] tree: Make interface for tree service client 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-29 09:34:10 +03:00
Denis Kirillov bd3164c57f [#68] Fix pre-commit issues 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-24 16:22:06 +03:00
Alexey Vanin ef556bd8ac [#60] Use session token to set eACL during Complete Multipart Upload 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-22 11:22:11 +03:00
Alexey Vanin 5104683f68 [#60] Refactor start of periodic XML writer 
Reduce code duplication for error handling

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-22 11:02:16 +03:00
Alexey Vanin 8151753eeb [#60] Use periodic white space XML writer in Complete Multipart Upload 
This mechanism is used by Amazon S3 to keep client's
connection alive while object is being constructed from
the upload parts.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 13:34:22 +03:00
Alexey Vanin 2282c32822 [#60] Add *NoHeader functions 
Such functions should be used together with periodic white space
XML writer.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 12:07:36 +03:00
Alexey Vanin cf18158da4 [#60] Implement periodic white space XML writer 
Periodic white space XML writer sends XML header
and white spaces to the io.Writer.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 12:04:34 +03:00
Denis Kirillov 0af06c3bd9 [TrueCloudLab#40] Add param to configure xml decoder 
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/" attribute
for CompleteMultipartUpload requests

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 15:44:13 +03:00
Denis Kirillov 64e7356acc [TrueCloudLab#32] Add custom policy unmarshaler 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 11:19:01 +00:00
Denis Kirillov 32bf915502 [TrueCloudLab#37] Limit number of objects to delete 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 11:16:22 +00:00
Alexey Vanin 813aa2f173 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:38:08 +03:00
Denis Kirillov 740acadd37 [TrueCloudLab#36] Fix cors object payload 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-20 19:22:48 +03:00
Alexey Vanin aadefd98b6 [TrueCloudLab#25] Process allow and deny lists of zones in bucket head requests 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-02-17 13:47:48 +03:00
Denis Kirillov 9dcacc230e [TrueCloudLab#23] Return error on unknown LocationConstraint 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-08 15:04:12 +03:00
Artem Tataurov 5ee4bf80ae [#18] Return container name in head-bucket response 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-02-01 14:25:21 +03:00
Denis Kirillov fc5c09c084 [TrueCloudLab#5] Request metrics per user 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-25 11:02:23 +03:00
Artem Tataurov 8e14ccad9f [#754] Remove duplicating CID/OID logging calls 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7)
 2022-12-28 17:06:32 +03:00
Artem Tataurov af7fbd6f32 [#754] Add CID and OID to logs 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2022-12-28 17:06:32 +03:00
Denis Kirillov 388482e230 [#2] Rename internals 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-21 11:17:45 +03:00
Alexey Vanin 96dff367db [#1] Build S3 Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 12:43:52 +03:00
Denis Kirillov dd4f66712c [#742] Add multiple listeners 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-24 17:48:26 +03:00
Denis Kirillov d2587b21af [#747] Reload policies on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-18 13:14:25 +03:00
Denis Kirillov 094eb12578 [#726] Use client time on regular requests 
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-14 14:53:55 +03:00
Denis Kirillov d47840f137 [#568] Add configuration for region to policy map 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-11 13:12:49 +03:00
Denis Kirillov 2886ac161c [#740] Fix forming policy by ast 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-10 15:33:03 +03:00
Denis Kirillov 18a6aca4b4 [#617] api/handler: Simplify tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-25 19:04:37 +03:00
Angira Kekteeva 712740b668 [#728] Remove redundant log message 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-10-25 19:04:11 +03:00
Denis Kirillov 821df3d648 [#669] Optimize getNodeVersion for tags operation 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-20 17:56:09 +03:00
Denis Kirillov 90eb4f0188 [#713] Add check access control for system caches 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 15:26:42 +03:00
Denis Kirillov cb55d36063 [#713] Update tests 
Add bearer token to test context

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 15:26:42 +03:00
Denis Kirillov c6098e53f6 [#719] Add tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 11:13:20 +03:00
Denis Kirillov b3d96e6460 [#719] Improve lock error handling 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 11:13:20 +03:00
Denis Kirillov 2a41929be3 [#702] Reload resolvers and TLS certs on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-28 15:07:31 +03:00
Denis Kirillov 3d08562843 [#505] Handle access denied from tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov 80beedf13e [#504] Use bktInfo in tree service to check owner 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov 4082cd6b54 [#606] Keep eacl records order on conflict 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-08 12:21:47 +03:00
Denis Kirillov 9cd4ef1ac4 [#657] Replace FileName with FilePath attribute 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 15:50:43 +03:00
Denis Kirillov 96c7b79d1c [#683] Forbid copy to itself in unversioned bucket 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-30 13:31:48 +03:00
Denis Kirillov 44d5878a80 [#656] Read full body on upload error 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-30 09:39:20 +03:00
Denis Kirillov 5c7a509cc9 [#684] Fix continuation token for directories 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-29 16:10:01 +03:00