Commit graph

638 commits

Author SHA1 Message Date
6f9ee3da76 [#275] Change logic delete multipart upload
In order not to accidentally take outdated
information about downloaded parts from other
nodes, now when the multipart is abort or complete,
the root node of the multipart upload with the
finish flag remains in the tree.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-12-27 13:06:45 +03:00
08019f1574 [#280] Add put requests to duration metric
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-12-22 16:33:05 +03:00
5698d5844e [#283] Support frostfsid groups in policy request checking
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-21 14:21:36 +03:00
43cae9ee04 [#248] Correct object versions response markers
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-21 10:53:50 +03:00
a17ff66975 [#282] policy: Use prefixes to distinguish s3/iam actions/resources
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-20 10:41:15 +03:00
38c5503a02 [#261] alc: Remove unused
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-18 15:49:58 +03:00
8273af8bf8 [#261] Make PutBucketPolicy handler use policy contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-18 15:49:54 +03:00
6dbb07f0fa [#261] Update policy-engine dependency
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-14 17:52:13 +03:00
9272f4e108 [#259] Support contract based policies
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-11 10:01:46 +03:00
836874a761 [#262] Set tree request id
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-07 16:40:56 +03:00
f58a0d04ff [#233] Clean tag node in the tree service instead of removal
With new retry policy of tree service pool, gateway should avoid
deletion of system nodes from tree. Absence of node in the tree
will trigger retry. Other storage in the network may return already
deleted node while tree is not completely synced, and client will
get unexpected result.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-12-07 16:40:56 +03:00
43abf58068 [#257] Support flag to deny access if policy rules not found
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-06 17:47:58 +03:00
ca15acf1d3 [#257] router: Use named constants
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-06 17:47:58 +03:00
473239bf36 [#257] Add policy checker
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-06 17:47:51 +03:00
93cf7c462b [#271] Add namespace label to billing metrics
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-12-04 14:54:40 +03:00
28c6bb4cb8 [#266] Support per namespace placement policies configuration
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-28 16:12:42 +03:00
ff1ec56d24 [#260] Use namespace as domain when create bucket
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-27 15:30:12 +03:00
055cc6a22a [#260] Use namespace as domain when resolve bucket
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:11 +03:00
6304d7bfda [#260] Support frostfsid validation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:11 +03:00
cf7254f8cd [#260] Refactor api/auth/center.go
Move the Center interface to middleware package where it's used

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:09 +03:00
861454e499 [#218] Add check content sha256 header
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-11-22 11:33:52 +00:00
b28ecef43b [#219] Return ETag in quotes
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-22 11:12:32 +00:00
fe796ba538 [#217] Consider Copy-Source-SSE-* headers during copy
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-13 13:22:58 +00:00
5ee73fad6a [#248] Correct NextVersionIDMarker in listing versions
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements
says that
"When the number of responses exceeds the value of MaxKeys,
NextVersionIdMarker specifies the first object version not returned
 that satisfies the search criteria. Use this value for the
 version-id-marker request parameter in a subsequent request."
 the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 17:36:24 +03:00
890a8ed237 [#227] Add versionID header after complete multipart
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 14:07:08 +00:00
0bed25816c [#224] Add conditional escaping for object name
Chi gives inconsistent results in terms of whether
the strings returned are URL coded or not
See:
* https://github.com/go-chi/chi/issues/641
* https://github.com/go-chi/chi/issues/642

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:58:51 +00:00
b169c5e6c3 [#239] Update test for check goroutines leak
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:51:23 +00:00
cf13aae342 [#225] Add default storage class to responses
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 13:37:07 +00:00
0938d7ee82 [#226] Fix status code in GET/HEAD delete marker
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-27 10:58:57 +03:00
4f5f5fb5c8 [#222] Fix marshaling errors in DeleteObjects method
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-25 14:54:02 +00:00
25bb581fee [#205] Add md5 checksum in header
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-25 11:04:19 +03:00
8d6aa0d40a [#243] Fix list object versions marker param
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 10:35:47 +03:00
7e91f62c28 [#223] Add store content language
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:42:02 +00:00
01323ca8e0 [#216] Add check tag key uniqueness
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:40:29 +00:00
298662df9d [#221] Expand xmlns field ignore
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-13 16:21:13 +03:00
65412ce1d3 [#197] Configure buffer max size for PUT
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
7de73f6b73 [#197] Disable homomorphic hash for PUT
Disable TZ hash for PUT if it's disabled for container itself

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
e3f2d59565 [#154] Rename access key to secret key
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 09:20:39 +00:00
c4af1dc4ad [#171] Update message error auth header malformed
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-04 11:13:12 +00:00
b8c93ed391 [#172] Convert handler config to interface
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-04 11:01:27 +00:00
51e591877b [#207] Fix list parts with empty list
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-21 11:27:20 +03:00
a4c612614a [#210] Fix multipart object reader
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 16:30:08 +03:00
12cf29aed2 [#207] Fix part-number-marker handling
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
066b9a0250 [#142] Add trace ID into log when tracing is enabled
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-07 14:19:37 +03:00
9120e97ac5 [#203] Add go1.21 to CI
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-31 15:26:07 +03:00
b5fce5c8d2 [#168] Skip only invalid policies and copies instead of ignoring all of them
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
6617adc22b [#185] Use correct object size when object is combined or encrypted
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
8898c2ec08 [#185] Add tests for list multipart uploads
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8efcc957ea [#96] Move log messages to constants
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b728fef87 [#192] Add tests to make sure client_cut flag is passed to sdk
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00