TrueCloudLab
/
frostfs-s3-gw
15
2
Fork 14
Code Issues 20 Pull Requests 5 Actions Packages Releases 6 Activity
1,302 Commits
4 Branches
23 Tags
17 MiB
Commit Graph

79 Commits (v0.29.0-rc.5)

Author SHA1 Message Date
Denis Kirillov 56b50f2075 [#306] Remove flag to disable policy contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov c868af8a62 [#306] Add flag to enable old ACL bucket creation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Denis Kirillov 37be8851b3 [#306] Simplify namespaces configuration 
Resolve ns alias at the beginning of the request just once.
Keep in ns map only one default ns key.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-28 17:50:08 +03:00
Artem Tataurov bd8d2d00ba [#313] logger: Fix logging level changing for journald
/ DCO (pull_request) Successful in 1m21s Details
/ Builds (1.20) (pull_request) Successful in 2m26s Details
/ Builds (1.21) (pull_request) Successful in 1m14s Details
/ Vulncheck (pull_request) Successful in 2m19s Details
/ Lint (pull_request) Failing after 2m41s Details
/ Tests (1.20) (pull_request) Successful in 2m30s Details
/ Tests (1.21) (pull_request) Successful in 2m17s Details 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2024-02-16 17:44:16 +03:00
Denis Kirillov 924e87face [#305] Support checking if accessbox was removed 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-09 09:41:48 +03:00
Denis Kirillov 6e8960b2ab [#165] Add list session cache 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-02-02 16:09:48 +03:00
Pavel Pogodaev 4804904d9d [#298] journald update version 
We want to have less useless fields in logs

Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2024-01-29 12:49:22 +00:00
Denis Kirillov 899213b3f3 [#287] Support proxy for frostfsid and policy contracts
/ Builds (1.20) (pull_request) Successful in 1m52s Details
/ Builds (1.21) (pull_request) Successful in 1m12s Details
/ DCO (pull_request) Successful in 1m46s Details
/ Vulncheck (pull_request) Successful in 1m43s Details
/ Lint (pull_request) Successful in 4m3s Details
/ Tests (1.20) (pull_request) Successful in 2m24s Details
/ Tests (1.21) (pull_request) Successful in 2m23s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-22 15:45:10 +03:00
Denis Kirillov 5698d5844e [#283] Support frostfsid groups in policy request checking 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-21 14:21:36 +03:00
Denis Kirillov 0850d21ff3 [#266] Move trimming namespaces name into fetch function 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-14 16:12:10 +03:00
Denis Kirillov 9272f4e108 [#259] Support contract based policies
/ DCO (pull_request) Successful in 1m21s Details
/ Vulncheck (pull_request) Successful in 1m41s Details
/ Builds (1.20) (pull_request) Successful in 2m19s Details
/ Builds (1.21) (pull_request) Successful in 2m1s Details
/ Lint (pull_request) Successful in 3m20s Details
/ Tests (1.20) (pull_request) Successful in 2m14s Details
/ Tests (1.21) (pull_request) Successful in 2m10s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-11 10:01:46 +03:00
Denis Kirillov be6a37ada5 [#262] Support configuring max tree request attempts
/ DCO (pull_request) Successful in 1m15s Details
/ Vulncheck (pull_request) Successful in 1m39s Details
/ Builds (1.20) (pull_request) Successful in 1m19s Details
/ Builds (1.21) (pull_request) Successful in 2m27s Details
/ Lint (pull_request) Successful in 4m24s Details
/ Tests (1.20) (pull_request) Successful in 2m17s Details
/ Tests (1.21) (pull_request) Successful in 2m36s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-12-07 16:41:21 +03:00
Denis Kirillov 43abf58068 [#257] Support flag to deny access if policy rules not found
/ DCO (pull_request) Successful in 1m13s Details
/ Vulncheck (pull_request) Successful in 2m2s Details
/ Builds (1.20) (pull_request) Successful in 2m22s Details
/ Builds (1.21) (pull_request) Successful in 2m16s Details
/ Lint (pull_request) Successful in 3m26s Details
/ Tests (1.20) (pull_request) Successful in 2m21s Details
/ Tests (1.21) (pull_request) Successful in 1m37s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-06 17:47:58 +03:00
Denis Kirillov 473239bf36 [#257] Add policy checker 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-06 17:47:51 +03:00
Marina Biryukova 93cf7c462b [#271] Add namespace label to billing metrics
/ DCO (pull_request) Successful in 2m35s Details
/ Vulncheck (pull_request) Successful in 3m3s Details
/ Builds (1.20) (pull_request) Successful in 3m34s Details
/ Builds (1.21) (pull_request) Successful in 2m20s Details
/ Lint (pull_request) Successful in 5m27s Details
/ Tests (1.20) (pull_request) Successful in 3m25s Details
/ Tests (1.21) (pull_request) Successful in 3m12s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-12-04 14:54:40 +03:00
Denis Kirillov 6c5f9b2764 [#266] Fix namespace config initialization 
Don't use nil Namespaces map in case when file isn't provided or invalid

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-04 09:40:47 +00:00
Denis Kirillov c7a65bd075 [#258] Add control service 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-01 14:17:06 +03:00
Denis Kirillov 28c6bb4cb8 [#266] Support per namespace placement policies configuration
/ DCO (pull_request) Successful in 1m34s Details
/ Builds (1.20) (pull_request) Successful in 3m15s Details
/ Builds (1.21) (pull_request) Successful in 2m55s Details
/ Vulncheck (pull_request) Successful in 2m51s Details
/ Lint (pull_request) Successful in 5m12s Details
/ Tests (1.20) (pull_request) Successful in 2m57s Details
/ Tests (1.21) (pull_request) Successful in 2m48s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-28 16:12:42 +03:00
Denis Kirillov 055cc6a22a [#260] Use namespace as domain when resolve bucket 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:11 +03:00
Denis Kirillov 6304d7bfda [#260] Support frostfsid validation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:11 +03:00
Roman Loginov 406075aebb [#236] Add support zapjournald logger configuration
/ DCO (pull_request) Successful in 1m34s Details
/ Vulncheck (pull_request) Successful in 2m6s Details
/ Builds (1.20) (pull_request) Successful in 2m15s Details
/ Builds (1.21) (pull_request) Successful in 2m2s Details
/ Lint (pull_request) Successful in 2m59s Details
/ Tests (1.20) (pull_request) Successful in 2m22s Details
/ Tests (1.21) (pull_request) Successful in 1m43s Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-11-13 16:31:11 +03:00
Marina Biryukova 890a8ed237 [#227] Add versionID header after complete multipart 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-31 14:07:08 +00:00
Denis Kirillov 122af0b5a7 [#220] Support configuring web server timeout params 
Set IdleTimeout and ReadHeaderTimeout to `30s`.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-31 13:48:08 +00:00
Marina Biryukova 25bb581fee [#205] Add md5 checksum in header
/ Vulncheck (pull_request) Successful in 1m50s Details
/ DCO (pull_request) Successful in 3m29s Details
/ Lint (pull_request) Successful in 3m37s Details
/ Tests (1.20) (pull_request) Successful in 2m9s Details
/ Tests (1.21) (pull_request) Successful in 2m5s Details
/ Builds (1.20) (pull_request) Successful in 5m47s Details
/ Builds (1.21) (pull_request) Successful in 1m29s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-25 11:04:19 +03:00
Marina Biryukova 298662df9d [#221] Expand xmlns field ignore
/ Vulncheck (pull_request) Successful in 1m38s Details
/ Lint (pull_request) Successful in 2m49s Details
/ Tests (1.20) (pull_request) Successful in 1m56s Details
/ Tests (1.21) (pull_request) Successful in 1m44s Details
/ DCO (pull_request) Successful in 3m56s Details
/ Builds (1.20) (pull_request) Successful in 6m25s Details
/ Builds (1.21) (pull_request) Successful in 1m36s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-13 16:21:13 +03:00
Denis Kirillov 65412ce1d3 [#197] Configure buffer max size for PUT 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-11 12:32:48 +00:00
Roman Loginov c66c09765d [#196] Support soft memory limit setting 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-09-05 13:13:56 +00:00
Marina Biryukova 2fc328a6d2 [#195] Add log constants linter
/ DCO (pull_request) Successful in 1m36s Details
/ Vulncheck (pull_request) Successful in 1m41s Details
/ Lint (pull_request) Successful in 8m31s Details
/ Tests (1.19) (pull_request) Successful in 2m6s Details
/ Tests (1.20) (pull_request) Successful in 1m57s Details
/ Builds (1.19) (pull_request) Successful in 2m21s Details
/ Builds (1.20) (pull_request) Successful in 9m3s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-08-28 12:58:44 +03:00
Denis Kirillov b5fce5c8d2 [#168] Skip only invalid policies and copies instead of ignoring all of them
/ DCO (pull_request) Successful in 2m48s Details
/ Vulncheck (pull_request) Successful in 3m30s Details
/ Builds (1.19) (pull_request) Successful in 3m39s Details
/ Builds (1.20) (pull_request) Successful in 3m28s Details
/ Lint (pull_request) Successful in 4m14s Details
/ Tests (1.19) (pull_request) Successful in 3m3s Details
/ Tests (1.20) (pull_request) Successful in 2m45s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-25 12:05:30 +03:00
Roman Loginov 8efcc957ea [#96] Move log messages to constants
/ DCO (pull_request) Successful in 1m35s Details
/ Builds (1.19) (pull_request) Successful in 2m14s Details
/ Builds (1.20) (pull_request) Successful in 2m9s Details
/ Vulncheck (pull_request) Successful in 5m39s Details
/ Lint (pull_request) Successful in 2m49s Details
/ Tests (1.19) (pull_request) Successful in 7m34s Details
/ Tests (1.20) (pull_request) Successful in 1m44s Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-08-23 18:32:31 +03:00
Denis Kirillov 6b1f365e65 [#192] Support client cut 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-23 06:26:55 +00:00
Roman Loginov 6a9d3261a7 [#117] Refactor fetch/parse config parameters functions
/ Vulncheck (pull_request) Successful in 2m0s Details
/ Lint (pull_request) Successful in 2m53s Details
/ Tests (1.19) (pull_request) Successful in 2m14s Details
/ Tests (1.20) (pull_request) Successful in 1m51s Details
/ DCO (pull_request) Successful in 4m0s Details
/ Builds (1.19) (pull_request) Successful in 6m30s Details
/ Builds (1.20) (pull_request) Successful in 2m7s Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-08-22 08:05:26 +00:00
Denis Kirillov b59aa06637 [#146] Add kludge.bypass_content_encoding_check_in_chunks flag
/ Builds (1.19) (pull_request) Successful in 3m0s Details
/ Builds (1.20) (pull_request) Successful in 2m50s Details
/ DCO (pull_request) Failing after 1m9s Details
/ Vulncheck (pull_request) Successful in 1m20s Details
/ Lint (pull_request) Successful in 7m50s Details
/ Tests (1.19) (pull_request) Successful in 3m1s Details
/ Tests (1.20) (pull_request) Successful in 3m21s Details 
Flag allows to skip checking `Content-Encoding` for `aws-chunked` value

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-07-18 14:49:52 +03:00
Denis Kirillov 9dabaf6ecd [#133] Use tree pool from SDK 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-09 09:34:04 +03:00
Pavel Pogodaev 4e1fd9589b [#84] add tracing support
Builds (1.20) Details
Builds (1.19) Details
DCO Details
Tests (1.19) Details
Tests (1.20) Details
Lint Details 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2023-06-06 10:25:27 +03:00
Alexey Vanin 81e860481d [#122] Fix linter warnings
Builds (1.19) Details
DCO Details
Builds (1.20) Details
Lint Details
Tests (1.19) Details
Tests (1.20) Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-05-31 19:43:03 +03:00
Artem Tataurov 9f186d9aba [#104] app: Reload copies numbers on SIGHUP 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-23 13:19:58 +03:00
Artem Tataurov e24bc3f2ce [#101] app: Refactor the default copies number setting 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-17 11:36:28 +03:00
Artem Tataurov e487ee5b7d [#70] Add arrays of copies numbers for location constraints 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-05-03 13:48:26 +03:00
Alexey Vanin 8151753eeb [#60] Use periodic white space XML writer in Complete Multipart Upload 
This mechanism is used by Amazon S3 to keep client's
connection alive while object is being constructed from
the upload parts.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 13:34:22 +03:00
Denis Kirillov 0af06c3bd9 [TrueCloudLab#40] Add param to configure xml decoder 
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/" attribute
for CompleteMultipartUpload requests

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 15:44:13 +03:00
Alexey Vanin 813aa2f173 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:38:08 +03:00
Alexey Vanin aadefd98b6 [TrueCloudLab#25] Process allow and deny lists of zones in bucket head requests 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-02-17 13:47:48 +03:00
Denis Kirillov 1ce8b8a30d [TrueCloudLab#21] Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-07 12:26:40 +03:00
Denis Kirillov 388482e230 [#2] Rename internals 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-21 11:17:45 +03:00
Denis Kirillov 765fd28a49 [#2] Update build 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-20 18:24:30 +03:00
Alexey Vanin 96dff367db [#1] Build S3 Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 12:43:52 +03:00
Denis Kirillov 5265afe213 [#742] Update docs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-24 17:48:26 +03:00
Denis Kirillov dd4f66712c [#742] Add multiple listeners 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-24 17:48:26 +03:00
Denis Kirillov d1825f97fb [#750] Update SDK to support timeout for stream 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-15 18:10:28 +03:00