TrueCloudLab
/
frostfs-s3-gw
16
2
Fork 14
Code Issues 19 Pull Requests 4 Actions Packages Releases 6 Activity
1,297 Commits
4 Branches
24 Tags
18 MiB
Commit Graph

1297 Commits (v0.29.0-rc.2)

Author SHA1 Message Date
Denis Kirillov 42862fd69e [#258] Support policy management in control svc
/ Vulncheck (pull_request) Successful in 3m20s Details
/ DCO (pull_request) Successful in 3m56s Details
/ Builds (1.20) (pull_request) Successful in 4m13s Details
/ Builds (1.21) (pull_request) Successful in 4m9s Details
/ Lint (pull_request) Successful in 3m9s Details
/ Tests (1.20) (pull_request) Successful in 4m14s Details
/ Tests (1.21) (pull_request) Successful in 3m59s Details 
Add PutPolicies, RemovePolicies, GetPolicy, ListPolicies methods

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-01 15:47:12 +03:00
Denis Kirillov c7a65bd075 [#258] Add control service 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-12-01 14:17:06 +03:00
Denis Kirillov 28c6bb4cb8 [#266] Support per namespace placement policies configuration
/ DCO (pull_request) Successful in 1m34s Details
/ Builds (1.20) (pull_request) Successful in 3m15s Details
/ Builds (1.21) (pull_request) Successful in 2m55s Details
/ Vulncheck (pull_request) Successful in 2m51s Details
/ Lint (pull_request) Successful in 5m12s Details
/ Tests (1.20) (pull_request) Successful in 2m57s Details
/ Tests (1.21) (pull_request) Successful in 2m48s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-28 16:12:42 +03:00
Alexey Vanin 0db6cd6727 [#268] Add dirty version check 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-11-28 10:42:49 +00:00
Denis Kirillov ff1ec56d24 [#260] Use namespace as domain when create bucket
/ DCO (pull_request) Successful in 1m26s Details
/ Vulncheck (pull_request) Successful in 2m6s Details
/ Builds (1.20) (pull_request) Successful in 2m20s Details
/ Builds (1.21) (pull_request) Successful in 2m21s Details
/ Lint (pull_request) Successful in 3m15s Details
/ Tests (1.20) (pull_request) Successful in 2m28s Details
/ Tests (1.21) (pull_request) Successful in 2m17s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-27 15:30:12 +03:00
Denis Kirillov 9ebfca654b [#260] Update CHANGELOG.md 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:19 +03:00
Denis Kirillov 055cc6a22a [#260] Use namespace as domain when resolve bucket 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:11 +03:00
Denis Kirillov a61ff3b8cb [#260] authmate: Support key registration in frostfsid contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:11 +03:00
Denis Kirillov 6304d7bfda [#260] Support frostfsid validation 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:11 +03:00
Denis Kirillov cf7254f8cd [#260] Refactor api/auth/center.go 
Move the Center interface to middleware package where it's used

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-23 11:00:09 +03:00
Roman Loginov 861454e499 [#218] Add check content sha256 header 
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-11-22 11:33:52 +00:00
Marina Biryukova b28ecef43b [#219] Return ETag in quotes
/ Vulncheck (pull_request) Successful in 9m5s Details
/ Lint (pull_request) Successful in 11m1s Details
/ Tests (1.20) (pull_request) Successful in 9m59s Details
/ Tests (1.21) (pull_request) Successful in 9m53s Details
/ DCO (pull_request) Successful in 11m34s Details
/ Builds (1.20) (pull_request) Successful in 12m24s Details
/ Builds (1.21) (pull_request) Successful in 8m59s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-11-22 11:12:32 +00:00
Roman Loginov 406075aebb [#236] Add support zapjournald logger configuration
/ DCO (pull_request) Successful in 1m34s Details
/ Vulncheck (pull_request) Successful in 2m6s Details
/ Builds (1.20) (pull_request) Successful in 2m15s Details
/ Builds (1.21) (pull_request) Successful in 2m2s Details
/ Lint (pull_request) Successful in 2m59s Details
/ Tests (1.20) (pull_request) Successful in 2m22s Details
/ Tests (1.21) (pull_request) Successful in 1m43s Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-11-13 16:31:11 +03:00
Marina Biryukova fe796ba538 [#217] Consider Copy-Source-SSE-* headers during copy 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-11-13 13:22:58 +00:00
Denis Kirillov 5ee73fad6a [#248] Correct NextVersionIDMarker in listing versions
/ DCO (pull_request) Successful in 4m40s Details
/ Vulncheck (pull_request) Successful in 5m43s Details
/ Lint (pull_request) Successful in 10m27s Details
/ Tests (1.20) (pull_request) Successful in 5m41s Details
/ Tests (1.21) (pull_request) Successful in 5m47s Details
/ Builds (1.20) (pull_request) Successful in 1m9s Details
/ Builds (1.21) (pull_request) Successful in 4m7s Details 
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements
says that
"When the number of responses exceeds the value of MaxKeys,
NextVersionIdMarker specifies the first object version not returned
 that satisfies the search criteria. Use this value for the
 version-id-marker request parameter in a subsequent request."
 the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-31 17:36:24 +03:00
Marina Biryukova 890a8ed237 [#227] Add versionID header after complete multipart 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-31 14:07:08 +00:00
Denis Kirillov 0bed25816c [#224] Add conditional escaping for object name 
Chi gives inconsistent results in terms of whether
the strings returned are URL coded or not
See:
* https://github.com/go-chi/chi/issues/641
* https://github.com/go-chi/chi/issues/642

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-31 13:58:51 +00:00
Denis Kirillov b169c5e6c3 [#239] Update test for check goroutines leak 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-31 13:51:23 +00:00
Denis Kirillov 122af0b5a7 [#220] Support configuring web server timeout params 
Set IdleTimeout and ReadHeaderTimeout to `30s`.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-31 13:48:08 +00:00
Marina Biryukova cf13aae342 [#225] Add default storage class to responses 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-31 13:37:07 +00:00
Denis Kirillov 0938d7ee82 [#226] Fix status code in GET/HEAD delete marker
/ Vulncheck (pull_request) Successful in 2m26s Details
/ Lint (pull_request) Successful in 4m0s Details
/ Tests (1.20) (pull_request) Successful in 2m59s Details
/ Tests (1.21) (pull_request) Successful in 1m52s Details
/ DCO (pull_request) Successful in 4m17s Details
/ Builds (1.20) (pull_request) Successful in 6m33s Details
/ Builds (1.21) (pull_request) Successful in 1m28s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-27 10:58:57 +03:00
Denis Kirillov 4f5f5fb5c8 [#222] Fix marshaling errors in `DeleteObjects` method 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-25 14:54:02 +00:00
Marina Biryukova 25bb581fee [#205] Add md5 checksum in header
/ Vulncheck (pull_request) Successful in 1m50s Details
/ DCO (pull_request) Successful in 3m29s Details
/ Lint (pull_request) Successful in 3m37s Details
/ Tests (1.20) (pull_request) Successful in 2m9s Details
/ Tests (1.21) (pull_request) Successful in 2m5s Details
/ Builds (1.20) (pull_request) Successful in 5m47s Details
/ Builds (1.21) (pull_request) Successful in 1m29s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-25 11:04:19 +03:00
Denis Kirillov 8d6aa0d40a [#243] Fix list object versions marker param
/ Vulncheck (pull_request) Successful in 2m8s Details
/ DCO (pull_request) Successful in 2m5s Details
/ Builds (1.20) (pull_request) Successful in 2m39s Details
/ Builds (1.21) (pull_request) Successful in 2m15s Details
/ Lint (pull_request) Successful in 4m23s Details
/ Tests (1.20) (pull_request) Successful in 2m53s Details
/ Tests (1.21) (pull_request) Successful in 2m18s Details 
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-18 10:35:47 +03:00
Roman Loginov 7e91f62c28 [#223] Add store content language 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-10-17 14:42:02 +00:00
Roman Loginov 01323ca8e0 [#216] Add check tag key uniqueness 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-10-17 14:40:29 +00:00
Marina Biryukova 298662df9d [#221] Expand xmlns field ignore
/ Vulncheck (pull_request) Successful in 1m38s Details
/ Lint (pull_request) Successful in 2m49s Details
/ Tests (1.20) (pull_request) Successful in 1m56s Details
/ Tests (1.21) (pull_request) Successful in 1m44s Details
/ DCO (pull_request) Successful in 3m56s Details
/ Builds (1.20) (pull_request) Successful in 6m25s Details
/ Builds (1.21) (pull_request) Successful in 1m36s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-13 16:21:13 +03:00
Denis Kirillov 10a03faeb4 [#197] Update CHANGELOG.md 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-11 12:32:48 +00:00
Denis Kirillov 65412ce1d3 [#197] Configure buffer max size for PUT 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-11 12:32:48 +00:00
Denis Kirillov 7de73f6b73 [#197] Disable homomorphic hash for PUT 
Disable TZ hash for PUT if it's disabled for container itself

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-11 12:32:48 +00:00
Denis Kirillov 8fc9d93f37 [#197] Update SDK 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-11 12:32:48 +00:00
Roman Loginov 7301ca52ab [#154] Rename OwnerPublicKey to SeedKey
/ Vulncheck (pull_request) Successful in 1m30s Details
/ DCO (pull_request) Successful in 1m29s Details
/ Builds (1.20) (pull_request) Successful in 2m13s Details
/ Builds (1.21) (pull_request) Successful in 1m52s Details
/ Lint (pull_request) Successful in 3m59s Details
/ Tests (1.20) (pull_request) Successful in 2m11s Details
/ Tests (1.21) (pull_request) Successful in 2m33s Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-10-06 14:00:37 +03:00
Roman Loginov e1ec61ddfc [#215] Fix get latest version node 
When the object version is received,
the node of the secondary object may return.
Now we choose the right node ourselves.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-10-06 09:21:41 +00:00
Roman Loginov e3f2d59565 [#154] Rename access key to secret key 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-10-06 09:20:39 +00:00
Roman Loginov c4af1dc4ad [#171] Update message error auth header malformed 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-10-04 11:13:12 +00:00
Marina Biryukova b8c93ed391 [#172] Convert handler config to interface 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-10-04 11:01:27 +00:00
Marina Biryukova 51e591877b [#207] Fix list parts with empty list
/ DCO (pull_request) Successful in 1m2s Details
/ Vulncheck (pull_request) Successful in 1m25s Details
/ Builds (1.20) (pull_request) Successful in 1m59s Details
/ Builds (1.21) (pull_request) Successful in 1m52s Details
/ Lint (pull_request) Successful in 8m23s Details
/ Tests (1.20) (pull_request) Successful in 2m1s Details
/ Tests (1.21) (pull_request) Successful in 9m26s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-21 11:27:20 +03:00
Marina Biryukova a4c612614a [#210] Fix multipart object reader
/ Vulncheck (pull_request) Successful in 2m38s Details
/ DCO (pull_request) Successful in 3m40s Details
/ Lint (pull_request) Successful in 5m20s Details
/ Tests (1.20) (pull_request) Successful in 3m19s Details
/ Tests (1.21) (pull_request) Successful in 3m8s Details
/ Builds (1.20) (pull_request) Successful in 6m1s Details
/ Builds (1.21) (pull_request) Successful in 1m33s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-19 16:30:08 +03:00
Marina Biryukova 12cf29aed2 [#207] Fix part-number-marker handling
/ DCO (pull_request) Successful in 1m57s Details
/ Vulncheck (pull_request) Successful in 2m21s Details
/ Builds (1.20) (pull_request) Successful in 1m36s Details
/ Builds (1.21) (pull_request) Successful in 7m55s Details
/ Lint (pull_request) Successful in 13m57s Details
/ Tests (1.20) (pull_request) Successful in 2m34s Details
/ Tests (1.21) (pull_request) Successful in 1m57s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-19 12:43:07 +03:00
Alexey Vanin 16840f1256 [#177] Add release instructions page
/ DCO (pull_request) Successful in 54s Details
/ Vulncheck (pull_request) Successful in 1m14s Details
/ Builds (1.20) (pull_request) Successful in 3m14s Details
/ Builds (1.21) (pull_request) Successful in 8m42s Details
/ Lint (pull_request) Successful in 13m19s Details
/ Tests (1.20) (pull_request) Successful in 1m31s Details
/ Tests (1.21) (pull_request) Successful in 6m15s Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-09-07 12:32:12 +00:00
Marina Biryukova 066b9a0250 [#142] Add trace ID into log when tracing is enabled
/ Vulncheck (pull_request) Successful in 1m29s Details
/ DCO (pull_request) Successful in 2m18s Details
/ Lint (pull_request) Successful in 3m48s Details
/ Tests (1.20) (pull_request) Successful in 2m4s Details
/ Tests (1.21) (pull_request) Successful in 1m42s Details
/ Builds (1.20) (pull_request) Successful in 7m6s Details
/ Builds (1.21) (pull_request) Successful in 1m56s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-07 14:19:37 +03:00
Artem Tataurov 54e1c333a1 [#152] authmate: Add basic error types and exit codes
/ DCO (pull_request) Successful in 1m9s Details
/ Vulncheck (pull_request) Successful in 5m17s Details
/ Builds (1.20) (pull_request) Successful in 2m1s Details
/ Builds (1.21) (pull_request) Successful in 7m18s Details
/ Lint (pull_request) Successful in 15m19s Details
/ Tests (1.20) (pull_request) Successful in 1m14s Details
/ Tests (1.21) (pull_request) Successful in 11m45s Details 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-09-06 23:56:56 +03:00
Marina Biryukova 69227b4845 [#199] Add metrics for HTTP endpoint status 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-05 13:30:27 +00:00
Roman Loginov c66c09765d [#196] Support soft memory limit setting 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-09-05 13:13:56 +00:00
Marina Biryukova 9120e97ac5 [#203] Add go1.21 to CI
/ DCO (pull_request) Successful in 59s Details
/ Vulncheck (pull_request) Successful in 1m35s Details
/ Builds (1.20) (pull_request) Successful in 1m21s Details
/ Builds (1.21) (pull_request) Successful in 7m46s Details
/ Lint (pull_request) Successful in 12m44s Details
/ Tests (1.20) (pull_request) Successful in 2m9s Details
/ Tests (1.21) (pull_request) Successful in 2m0s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-08-31 15:26:07 +03:00
Marina Biryukova 2fc328a6d2 [#195] Add log constants linter
/ DCO (pull_request) Successful in 1m36s Details
/ Vulncheck (pull_request) Successful in 1m41s Details
/ Lint (pull_request) Successful in 8m31s Details
/ Tests (1.19) (pull_request) Successful in 2m6s Details
/ Tests (1.20) (pull_request) Successful in 1m57s Details
/ Builds (1.19) (pull_request) Successful in 2m21s Details
/ Builds (1.20) (pull_request) Successful in 9m3s Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-08-28 12:58:44 +03:00
Denis Kirillov b5fce5c8d2 [#168] Skip only invalid policies and copies instead of ignoring all of them
/ DCO (pull_request) Successful in 2m48s Details
/ Vulncheck (pull_request) Successful in 3m30s Details
/ Builds (1.19) (pull_request) Successful in 3m39s Details
/ Builds (1.20) (pull_request) Successful in 3m28s Details
/ Lint (pull_request) Successful in 4m14s Details
/ Tests (1.19) (pull_request) Successful in 3m3s Details
/ Tests (1.20) (pull_request) Successful in 2m45s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-25 12:05:30 +03:00
Denis Kirillov 41a128b1aa [#185] Update CHANGELOG.md
/ Vulncheck (pull_request) Successful in 1m47s Details
/ Lint (pull_request) Successful in 2m46s Details
/ Tests (1.19) (pull_request) Successful in 2m9s Details
/ Tests (1.20) (pull_request) Successful in 2m6s Details
/ DCO (pull_request) Successful in 4m17s Details
/ Builds (1.19) (pull_request) Successful in 6m36s Details
/ Builds (1.20) (pull_request) Successful in 1m43s Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-24 18:03:11 +03:00
Denis Kirillov 6617adc22b [#185] Use correct object size when object is combined or encrypted 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-24 18:03:08 +03:00
Denis Kirillov 631d9d83b6 [#185] Fix payload reader 
When we use io.CopyBuffer it check for exact io.EOF matching,
so we need keep original EOF error otherwise io.CopyBuffer returns error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-24 18:01:08 +03:00