Commit graph

403 commits

Author SHA1 Message Date
8f89f275bd [#306] Save bucket policy as native chain
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
ff15f9f28a [#306] Fix update settings for buckets without owner key in tree
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c868af8a62 [#306] Add flag to enable old ACL bucket creation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
bac1b3fb2d [#306] Use zero basic acl to mark APE containers
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c452d58ce2 [#306] Reduce number of policy contract invocations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
d9d12debc3 [#306] Add tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
3d0d2032c6 [#306] acl: Handle put/get acl for APE buckets
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
1f2cf0ed67 [#306] Use APE instead of eACL on bucket creation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
37be8851b3 [#306] Simplify namespaces configuration
Resolve ns alias at the beginning of the request just once.
Keep in ns map only one default ns key.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c4c199defe [#306] Use s3 as chain id prefix to be consistent
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
563c1d9bd7 [#308] Fix linter issues
All checks were successful
/ DCO (pull_request) Successful in 1m31s
/ Builds (1.20) (pull_request) Successful in 1m59s
/ Builds (1.21) (pull_request) Successful in 1m37s
/ Vulncheck (pull_request) Successful in 2m14s
/ Lint (pull_request) Successful in 4m7s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 1m58s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-02-16 18:25:06 +03:00
c334adeb6d [#165] Sort nodes in ServiceClientMemory
Some checks failed
/ DCO (pull_request) Successful in 1m22s
/ Builds (1.20) (pull_request) Successful in 1m52s
/ Builds (1.21) (pull_request) Successful in 1m17s
/ Vulncheck (pull_request) Successful in 1m51s
/ Lint (pull_request) Failing after 1m55s
/ Tests (1.20) (pull_request) Successful in 2m24s
/ Tests (1.21) (pull_request) Successful in 2m24s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:53:12 +03:00
a74d498df2 [#165] Return sort after HEAD in listing
Some checks failed
/ DCO (pull_request) Successful in 1m16s
/ Builds (1.20) (pull_request) Successful in 2m9s
/ Builds (1.21) (pull_request) Successful in 1m39s
/ Vulncheck (pull_request) Successful in 1m52s
/ Lint (pull_request) Failing after 1m47s
/ Tests (1.20) (pull_request) Successful in 2m27s
/ Tests (1.21) (pull_request) Successful in 2m13s
We have to sort object after HEAD because we make request in different goroutines,
so the order is not deterministic.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
69297a4a38 [#165] Delete object from tree in case of storage error
Extend storage node errors in case of which we continue deleting from tree
with 'object not found' error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
71d82d1cc8 [#165] Fix lint issues
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
fafe4af529 [#165] Fix real object size in listing
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:03 +03:00
88f1acbdfc [#165] Cancel context in outdated list session
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
4e15452853 [#165] Fix lint errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
da642a498a [#165] Listing fix data race
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
093de13f54 [#165] Add stream listing tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
cf4fc3b602 [#165] Extend error on getting listing containers not in current namespace
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
739a6ec9df [#165] Support latest only stream listing
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
c7ee628ab0 [#165] Fix versions listing
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
4ad84b9b94 [#165] listing: Use NodeVersion instead of ObjectInfo
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
29ac91dfd5 [#165] Support streaming listing
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
84af85ed67 [#302] Update APE to support chain id as bytes
All checks were successful
/ DCO (pull_request) Successful in 1m52s
/ Vulncheck (pull_request) Successful in 2m0s
/ Builds (1.20) (pull_request) Successful in 2m49s
/ Builds (1.21) (pull_request) Successful in 1m59s
/ Lint (pull_request) Successful in 2m44s
/ Tests (1.20) (pull_request) Successful in 2m30s
/ Tests (1.21) (pull_request) Successful in 2m24s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 12:53:45 +03:00
43cae9ee04 [#248] Correct object versions response markers
All checks were successful
/ DCO (pull_request) Successful in 1m40s
/ Vulncheck (pull_request) Successful in 1m42s
/ Builds (1.20) (pull_request) Successful in 2m29s
/ Builds (1.21) (pull_request) Successful in 1m37s
/ Lint (pull_request) Successful in 3m55s
/ Tests (1.20) (pull_request) Successful in 2m23s
/ Tests (1.21) (pull_request) Successful in 2m16s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-21 10:53:50 +03:00
a17ff66975 [#282] policy: Use prefixes to distinguish s3/iam actions/resources
All checks were successful
/ DCO (pull_request) Successful in 1m37s
/ Vulncheck (pull_request) Successful in 1m50s
/ Builds (1.20) (pull_request) Successful in 2m24s
/ Builds (1.21) (pull_request) Successful in 2m2s
/ Lint (pull_request) Successful in 4m26s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 1m58s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-20 10:41:15 +03:00
38c5503a02 [#261] alc: Remove unused
All checks were successful
/ Vulncheck (pull_request) Successful in 2m1s
/ DCO (pull_request) Successful in 1m49s
/ Builds (1.20) (pull_request) Successful in 2m21s
/ Builds (1.21) (pull_request) Successful in 2m4s
/ Lint (pull_request) Successful in 2m56s
/ Tests (1.20) (pull_request) Successful in 2m17s
/ Tests (1.21) (pull_request) Successful in 1m46s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-18 15:49:58 +03:00
8273af8bf8 [#261] Make PutBucketPolicy handler use policy contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-18 15:49:54 +03:00
28c6bb4cb8 [#266] Support per namespace placement policies configuration
All checks were successful
/ DCO (pull_request) Successful in 1m34s
/ Builds (1.20) (pull_request) Successful in 3m15s
/ Builds (1.21) (pull_request) Successful in 2m55s
/ Vulncheck (pull_request) Successful in 2m51s
/ Lint (pull_request) Successful in 5m12s
/ Tests (1.20) (pull_request) Successful in 2m57s
/ Tests (1.21) (pull_request) Successful in 2m48s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-28 16:12:42 +03:00
ff1ec56d24 [#260] Use namespace as domain when create bucket
All checks were successful
/ DCO (pull_request) Successful in 1m26s
/ Vulncheck (pull_request) Successful in 2m6s
/ Builds (1.20) (pull_request) Successful in 2m20s
/ Builds (1.21) (pull_request) Successful in 2m21s
/ Lint (pull_request) Successful in 3m15s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 2m17s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-27 15:30:12 +03:00
cf7254f8cd [#260] Refactor api/auth/center.go
Move the Center interface to middleware package where it's used

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:09 +03:00
861454e499 [#218] Add check content sha256 header
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-11-22 11:33:52 +00:00
b28ecef43b [#219] Return ETag in quotes
All checks were successful
/ Vulncheck (pull_request) Successful in 9m5s
/ Lint (pull_request) Successful in 11m1s
/ Tests (1.20) (pull_request) Successful in 9m59s
/ Tests (1.21) (pull_request) Successful in 9m53s
/ DCO (pull_request) Successful in 11m34s
/ Builds (1.20) (pull_request) Successful in 12m24s
/ Builds (1.21) (pull_request) Successful in 8m59s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-22 11:12:32 +00:00
fe796ba538 [#217] Consider Copy-Source-SSE-* headers during copy
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-13 13:22:58 +00:00
5ee73fad6a [#248] Correct NextVersionIDMarker in listing versions
All checks were successful
/ DCO (pull_request) Successful in 4m40s
/ Vulncheck (pull_request) Successful in 5m43s
/ Lint (pull_request) Successful in 10m27s
/ Tests (1.20) (pull_request) Successful in 5m41s
/ Tests (1.21) (pull_request) Successful in 5m47s
/ Builds (1.20) (pull_request) Successful in 1m9s
/ Builds (1.21) (pull_request) Successful in 4m7s
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements
says that
"When the number of responses exceeds the value of MaxKeys,
NextVersionIdMarker specifies the first object version not returned
 that satisfies the search criteria. Use this value for the
 version-id-marker request parameter in a subsequent request."
 the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 17:36:24 +03:00
890a8ed237 [#227] Add versionID header after complete multipart
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 14:07:08 +00:00
0bed25816c [#224] Add conditional escaping for object name
Chi gives inconsistent results in terms of whether
the strings returned are URL coded or not
See:
* https://github.com/go-chi/chi/issues/641
* https://github.com/go-chi/chi/issues/642

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:58:51 +00:00
b169c5e6c3 [#239] Update test for check goroutines leak
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:51:23 +00:00
cf13aae342 [#225] Add default storage class to responses
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 13:37:07 +00:00
0938d7ee82 [#226] Fix status code in GET/HEAD delete marker
All checks were successful
/ Vulncheck (pull_request) Successful in 2m26s
/ Lint (pull_request) Successful in 4m0s
/ Tests (1.20) (pull_request) Successful in 2m59s
/ Tests (1.21) (pull_request) Successful in 1m52s
/ DCO (pull_request) Successful in 4m17s
/ Builds (1.20) (pull_request) Successful in 6m33s
/ Builds (1.21) (pull_request) Successful in 1m28s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-27 10:58:57 +03:00
4f5f5fb5c8 [#222] Fix marshaling errors in DeleteObjects method
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-25 14:54:02 +00:00
25bb581fee [#205] Add md5 checksum in header
All checks were successful
/ Vulncheck (pull_request) Successful in 1m50s
/ DCO (pull_request) Successful in 3m29s
/ Lint (pull_request) Successful in 3m37s
/ Tests (1.20) (pull_request) Successful in 2m9s
/ Tests (1.21) (pull_request) Successful in 2m5s
/ Builds (1.20) (pull_request) Successful in 5m47s
/ Builds (1.21) (pull_request) Successful in 1m29s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-25 11:04:19 +03:00
8d6aa0d40a [#243] Fix list object versions marker param
All checks were successful
/ Vulncheck (pull_request) Successful in 2m8s
/ DCO (pull_request) Successful in 2m5s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ Builds (1.21) (pull_request) Successful in 2m15s
/ Lint (pull_request) Successful in 4m23s
/ Tests (1.20) (pull_request) Successful in 2m53s
/ Tests (1.21) (pull_request) Successful in 2m18s
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 10:35:47 +03:00
7e91f62c28 [#223] Add store content language
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:42:02 +00:00
01323ca8e0 [#216] Add check tag key uniqueness
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:40:29 +00:00
298662df9d [#221] Expand xmlns field ignore
All checks were successful
/ Vulncheck (pull_request) Successful in 1m38s
/ Lint (pull_request) Successful in 2m49s
/ Tests (1.20) (pull_request) Successful in 1m56s
/ Tests (1.21) (pull_request) Successful in 1m44s
/ DCO (pull_request) Successful in 3m56s
/ Builds (1.20) (pull_request) Successful in 6m25s
/ Builds (1.21) (pull_request) Successful in 1m36s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-13 16:21:13 +03:00
7de73f6b73 [#197] Disable homomorphic hash for PUT
Disable TZ hash for PUT if it's disabled for container itself

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
e3f2d59565 [#154] Rename access key to secret key
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 09:20:39 +00:00