c868af8a62
[ #306 ] Add flag to enable old ACL bucket creation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
bac1b3fb2d
[ #306 ] Use zero basic acl to mark APE containers
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c452d58ce2
[ #306 ] Reduce number of policy contract invocations
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
d9d12debc3
[ #306 ] Add tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
3d0d2032c6
[ #306 ] acl: Handle put/get acl for APE buckets
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
1f2cf0ed67
[ #306 ] Use APE instead of eACL on bucket creation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
37be8851b3
[ #306 ] Simplify namespaces configuration
...
Resolve ns alias at the beginning of the request just once.
Keep in ns map only one default ns key.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c4c199defe
[ #306 ] Use s3 as chain id prefix to be consistent
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
563c1d9bd7
[ #308 ] Fix linter issues
...
/ DCO (pull_request) Successful in 1m31s
/ Builds (1.20) (pull_request) Successful in 1m59s
/ Builds (1.21) (pull_request) Successful in 1m37s
/ Vulncheck (pull_request) Successful in 2m14s
/ Lint (pull_request) Successful in 4m7s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 1m58s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-02-16 18:25:06 +03:00
c334adeb6d
[ #165 ] Sort nodes in ServiceClientMemory
...
/ DCO (pull_request) Successful in 1m22s
/ Builds (1.20) (pull_request) Successful in 1m52s
/ Builds (1.21) (pull_request) Successful in 1m17s
/ Vulncheck (pull_request) Successful in 1m51s
/ Lint (pull_request) Failing after 1m55s
/ Tests (1.20) (pull_request) Successful in 2m24s
/ Tests (1.21) (pull_request) Successful in 2m24s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:53:12 +03:00
a74d498df2
[ #165 ] Return sort after HEAD in listing
...
/ DCO (pull_request) Successful in 1m16s
/ Builds (1.20) (pull_request) Successful in 2m9s
/ Builds (1.21) (pull_request) Successful in 1m39s
/ Vulncheck (pull_request) Successful in 1m52s
/ Lint (pull_request) Failing after 1m47s
/ Tests (1.20) (pull_request) Successful in 2m27s
/ Tests (1.21) (pull_request) Successful in 2m13s
We have to sort object after HEAD because we make request in different goroutines,
so the order is not deterministic.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
69297a4a38
[ #165 ] Delete object from tree in case of storage error
...
Extend storage node errors in case of which we continue deleting from tree
with 'object not found' error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
71d82d1cc8
[ #165 ] Fix lint issues
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
fafe4af529
[ #165 ] Fix real object size in listing
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:03 +03:00
88f1acbdfc
[ #165 ] Cancel context in outdated list session
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
4e15452853
[ #165 ] Fix lint errors
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
da642a498a
[ #165 ] Listing fix data race
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
093de13f54
[ #165 ] Add stream listing tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
cf4fc3b602
[ #165 ] Extend error on getting listing containers not in current namespace
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
739a6ec9df
[ #165 ] Support latest only stream listing
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
c7ee628ab0
[ #165 ] Fix versions listing
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
4ad84b9b94
[ #165 ] listing: Use NodeVersion instead of ObjectInfo
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
29ac91dfd5
[ #165 ] Support streaming listing
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:09:48 +03:00
84af85ed67
[ #302 ] Update APE to support chain id as bytes
...
/ DCO (pull_request) Successful in 1m52s
/ Vulncheck (pull_request) Successful in 2m0s
/ Builds (1.20) (pull_request) Successful in 2m49s
/ Builds (1.21) (pull_request) Successful in 1m59s
/ Lint (pull_request) Successful in 2m44s
/ Tests (1.20) (pull_request) Successful in 2m30s
/ Tests (1.21) (pull_request) Successful in 2m24s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 12:53:45 +03:00
43cae9ee04
[ #248 ] Correct object versions response markers
...
/ DCO (pull_request) Successful in 1m40s
/ Vulncheck (pull_request) Successful in 1m42s
/ Builds (1.20) (pull_request) Successful in 2m29s
/ Builds (1.21) (pull_request) Successful in 1m37s
/ Lint (pull_request) Successful in 3m55s
/ Tests (1.20) (pull_request) Successful in 2m23s
/ Tests (1.21) (pull_request) Successful in 2m16s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-21 10:53:50 +03:00
a17ff66975
[ #282 ] policy: Use prefixes to distinguish s3/iam actions/resources
...
/ DCO (pull_request) Successful in 1m37s
/ Vulncheck (pull_request) Successful in 1m50s
/ Builds (1.20) (pull_request) Successful in 2m24s
/ Builds (1.21) (pull_request) Successful in 2m2s
/ Lint (pull_request) Successful in 4m26s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 1m58s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-20 10:41:15 +03:00
38c5503a02
[ #261 ] alc: Remove unused
...
/ Vulncheck (pull_request) Successful in 2m1s
/ DCO (pull_request) Successful in 1m49s
/ Builds (1.20) (pull_request) Successful in 2m21s
/ Builds (1.21) (pull_request) Successful in 2m4s
/ Lint (pull_request) Successful in 2m56s
/ Tests (1.20) (pull_request) Successful in 2m17s
/ Tests (1.21) (pull_request) Successful in 1m46s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-18 15:49:58 +03:00
8273af8bf8
[ #261 ] Make PutBucketPolicy handler use policy contract
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-18 15:49:54 +03:00
28c6bb4cb8
[ #266 ] Support per namespace placement policies configuration
...
/ DCO (pull_request) Successful in 1m34s
/ Builds (1.20) (pull_request) Successful in 3m15s
/ Builds (1.21) (pull_request) Successful in 2m55s
/ Vulncheck (pull_request) Successful in 2m51s
/ Lint (pull_request) Successful in 5m12s
/ Tests (1.20) (pull_request) Successful in 2m57s
/ Tests (1.21) (pull_request) Successful in 2m48s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-28 16:12:42 +03:00
ff1ec56d24
[ #260 ] Use namespace as domain when create bucket
...
/ DCO (pull_request) Successful in 1m26s
/ Vulncheck (pull_request) Successful in 2m6s
/ Builds (1.20) (pull_request) Successful in 2m20s
/ Builds (1.21) (pull_request) Successful in 2m21s
/ Lint (pull_request) Successful in 3m15s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 2m17s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-27 15:30:12 +03:00
cf7254f8cd
[ #260 ] Refactor api/auth/center.go
...
Move the Center interface to middleware package where it's used
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:09 +03:00
861454e499
[ #218 ] Add check content sha256 header
...
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-11-22 11:33:52 +00:00
b28ecef43b
[ #219 ] Return ETag in quotes
...
/ Vulncheck (pull_request) Successful in 9m5s
/ Lint (pull_request) Successful in 11m1s
/ Tests (1.20) (pull_request) Successful in 9m59s
/ Tests (1.21) (pull_request) Successful in 9m53s
/ DCO (pull_request) Successful in 11m34s
/ Builds (1.20) (pull_request) Successful in 12m24s
/ Builds (1.21) (pull_request) Successful in 8m59s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-22 11:12:32 +00:00
fe796ba538
[ #217 ] Consider Copy-Source-SSE-* headers during copy
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-13 13:22:58 +00:00
5ee73fad6a
[ #248 ] Correct NextVersionIDMarker in listing versions
...
/ DCO (pull_request) Successful in 4m40s
/ Vulncheck (pull_request) Successful in 5m43s
/ Lint (pull_request) Successful in 10m27s
/ Tests (1.20) (pull_request) Successful in 5m41s
/ Tests (1.21) (pull_request) Successful in 5m47s
/ Builds (1.20) (pull_request) Successful in 1m9s
/ Builds (1.21) (pull_request) Successful in 4m7s
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements
says that
"When the number of responses exceeds the value of MaxKeys,
NextVersionIdMarker specifies the first object version not returned
that satisfies the search criteria. Use this value for the
version-id-marker request parameter in a subsequent request."
the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 17:36:24 +03:00
890a8ed237
[ #227 ] Add versionID header after complete multipart
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 14:07:08 +00:00
0bed25816c
[ #224 ] Add conditional escaping for object name
...
Chi gives inconsistent results in terms of whether
the strings returned are URL coded or not
See:
* https://github.com/go-chi/chi/issues/641
* https://github.com/go-chi/chi/issues/642
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:58:51 +00:00
b169c5e6c3
[ #239 ] Update test for check goroutines leak
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:51:23 +00:00
cf13aae342
[ #225 ] Add default storage class to responses
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 13:37:07 +00:00
0938d7ee82
[ #226 ] Fix status code in GET/HEAD delete marker
...
/ Vulncheck (pull_request) Successful in 2m26s
/ Lint (pull_request) Successful in 4m0s
/ Tests (1.20) (pull_request) Successful in 2m59s
/ Tests (1.21) (pull_request) Successful in 1m52s
/ DCO (pull_request) Successful in 4m17s
/ Builds (1.20) (pull_request) Successful in 6m33s
/ Builds (1.21) (pull_request) Successful in 1m28s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-27 10:58:57 +03:00
4f5f5fb5c8
[ #222 ] Fix marshaling errors in DeleteObjects method
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-25 14:54:02 +00:00
25bb581fee
[ #205 ] Add md5 checksum in header
...
/ Vulncheck (pull_request) Successful in 1m50s
/ DCO (pull_request) Successful in 3m29s
/ Lint (pull_request) Successful in 3m37s
/ Tests (1.20) (pull_request) Successful in 2m9s
/ Tests (1.21) (pull_request) Successful in 2m5s
/ Builds (1.20) (pull_request) Successful in 5m47s
/ Builds (1.21) (pull_request) Successful in 1m29s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-25 11:04:19 +03:00
8d6aa0d40a
[ #243 ] Fix list object versions marker param
...
/ Vulncheck (pull_request) Successful in 2m8s
/ DCO (pull_request) Successful in 2m5s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ Builds (1.21) (pull_request) Successful in 2m15s
/ Lint (pull_request) Successful in 4m23s
/ Tests (1.20) (pull_request) Successful in 2m53s
/ Tests (1.21) (pull_request) Successful in 2m18s
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 10:35:47 +03:00
7e91f62c28
[ #223 ] Add store content language
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:42:02 +00:00
01323ca8e0
[ #216 ] Add check tag key uniqueness
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:40:29 +00:00
298662df9d
[ #221 ] Expand xmlns field ignore
...
/ Vulncheck (pull_request) Successful in 1m38s
/ Lint (pull_request) Successful in 2m49s
/ Tests (1.20) (pull_request) Successful in 1m56s
/ Tests (1.21) (pull_request) Successful in 1m44s
/ DCO (pull_request) Successful in 3m56s
/ Builds (1.20) (pull_request) Successful in 6m25s
/ Builds (1.21) (pull_request) Successful in 1m36s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-13 16:21:13 +03:00
7de73f6b73
[ #197 ] Disable homomorphic hash for PUT
...
Disable TZ hash for PUT if it's disabled for container itself
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
e3f2d59565
[ #154 ] Rename access key to secret key
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 09:20:39 +00:00
b8c93ed391
[ #172 ] Convert handler config to interface
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-04 11:01:27 +00:00
12cf29aed2
[ #207 ] Fix part-number-marker handling
...
/ DCO (pull_request) Successful in 1m57s
/ Vulncheck (pull_request) Successful in 2m21s
/ Builds (1.20) (pull_request) Successful in 1m36s
/ Builds (1.21) (pull_request) Successful in 7m55s
/ Lint (pull_request) Successful in 13m57s
/ Tests (1.20) (pull_request) Successful in 2m34s
/ Tests (1.21) (pull_request) Successful in 1m57s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
066b9a0250
[ #142 ] Add trace ID into log when tracing is enabled
...
/ Vulncheck (pull_request) Successful in 1m29s
/ DCO (pull_request) Successful in 2m18s
/ Lint (pull_request) Successful in 3m48s
/ Tests (1.20) (pull_request) Successful in 2m4s
/ Tests (1.21) (pull_request) Successful in 1m42s
/ Builds (1.20) (pull_request) Successful in 7m6s
/ Builds (1.21) (pull_request) Successful in 1m56s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-07 14:19:37 +03:00
9120e97ac5
[ #203 ] Add go1.21 to CI
...
/ DCO (pull_request) Successful in 59s
/ Vulncheck (pull_request) Successful in 1m35s
/ Builds (1.20) (pull_request) Successful in 1m21s
/ Builds (1.21) (pull_request) Successful in 7m46s
/ Lint (pull_request) Successful in 12m44s
/ Tests (1.20) (pull_request) Successful in 2m9s
/ Tests (1.21) (pull_request) Successful in 2m0s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-31 15:26:07 +03:00
b5fce5c8d2
[ #168 ] Skip only invalid policies and copies instead of ignoring all of them
...
/ DCO (pull_request) Successful in 2m48s
/ Vulncheck (pull_request) Successful in 3m30s
/ Builds (1.19) (pull_request) Successful in 3m39s
/ Builds (1.20) (pull_request) Successful in 3m28s
/ Lint (pull_request) Successful in 4m14s
/ Tests (1.19) (pull_request) Successful in 3m3s
/ Tests (1.20) (pull_request) Successful in 2m45s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
6617adc22b
[ #185 ] Use correct object size when object is combined or encrypted
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
8898c2ec08
[ #185 ] Add tests for list multipart uploads
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8efcc957ea
[ #96 ] Move log messages to constants
...
/ DCO (pull_request) Successful in 1m35s
/ Builds (1.19) (pull_request) Successful in 2m14s
/ Builds (1.20) (pull_request) Successful in 2m9s
/ Vulncheck (pull_request) Successful in 5m39s
/ Lint (pull_request) Successful in 2m49s
/ Tests (1.19) (pull_request) Successful in 7m34s
/ Tests (1.20) (pull_request) Successful in 1m44s
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b728fef87
[ #192 ] Add tests to make sure client_cut flag is passed to sdk
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
fcf1c45ad2
[ #188 ] Fix url escaping
...
/ Vulncheck (pull_request) Successful in 2m41s
/ Builds (1.19) (pull_request) Successful in 3m51s
/ Builds (1.20) (pull_request) Successful in 3m22s
/ DCO (pull_request) Successful in 5m9s
/ Lint (pull_request) Successful in 5m22s
/ Tests (1.19) (pull_request) Successful in 5m39s
/ Tests (1.20) (pull_request) Successful in 5m18s
Url escaping has already been done in `net/http/request.go`
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-22 11:27:39 +03:00
94a42fa273
[ #51 ] Update frostfs-sdk-go
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
40d7f844e3
[ #137 ] Refactor context data retrievers
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
18878b66d3
[ #175 ] Use gate owner as object owner
...
/ DCO (pull_request) Failing after 3s
/ Vulncheck (pull_request) Failing after 2s
/ Builds (1.19) (pull_request) Failing after 3s
/ Builds (1.20) (pull_request) Successful in 7m47s
/ Lint (pull_request) Failing after 4s
/ Tests (1.19) (pull_request) Successful in 8m8s
/ Tests (1.20) (pull_request) Failing after 3s
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
fe897ec588
[ #178 ] wrapReader: Fix goroutine leak
...
/ Vulncheck (pull_request) Successful in 59s
/ Builds (1.19) (pull_request) Successful in 1m28s
/ Builds (1.20) (pull_request) Successful in 1m19s
/ DCO (pull_request) Successful in 45s
/ Lint (pull_request) Successful in 1m45s
/ Tests (1.19) (pull_request) Successful in 7m51s
/ Tests (1.20) (pull_request) Successful in 1m43s
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-01 17:59:34 +03:00
52931663e1
[ #176 ] multipart: Replace part on re-upload
...
/ Builds (1.19) (pull_request) Successful in 2m57s
/ Builds (1.20) (pull_request) Successful in 2m36s
/ DCO (pull_request) Successful in 1m15s
/ Vulncheck (pull_request) Successful in 5m15s
/ Lint (pull_request) Successful in 3m50s
/ Tests (1.19) (pull_request) Successful in 8m14s
/ Tests (1.20) (pull_request) Successful in 3m17s
We want to have exactly one object and tree node for each part number
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-21 16:20:04 +03:00
631b7f67b4
[ #63 ] multipart: Log upload id for every failed request
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
bcf5a85aab
[ #63 ] multipart: Fix copying
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
ad81b599dd
[ #63 ] Add fast multipart upload
...
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
b59aa06637
[ #146 ] Add kludge.bypass_content_encoding_check_in_chunks flag
...
/ Builds (1.19) (pull_request) Successful in 3m0s
/ Builds (1.20) (pull_request) Successful in 2m50s
/ DCO (pull_request) Failing after 1m9s
/ Vulncheck (pull_request) Successful in 1m20s
/ Lint (pull_request) Successful in 7m50s
/ Tests (1.19) (pull_request) Successful in 3m1s
/ Tests (1.20) (pull_request) Successful in 3m21s
Flag allows to skip checking `Content-Encoding` for `aws-chunked` value
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:52 +03:00
d62aa7b979
[ #146 ] Fix preconditions: trim quotes in etags
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
751a9be7cc
[ #146 ] Move getting chunk payload reader to separate function
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
14ef9ff091
[ #158 ] Separate init object reader from read itself
...
/ Vulncheck (pull_request) Successful in 1m40s
/ Builds (1.19) (pull_request) Successful in 2m54s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ DCO (pull_request) Successful in 3m48s
/ Lint (pull_request) Successful in 3m32s
/ Tests (1.19) (pull_request) Successful in 2m38s
/ Tests (1.20) (pull_request) Successful in 2m55s
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:32:05 +03:00
fc90981c03
[ #149 ] Update inner imports after moving middlewares
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
d531b13866
[ #143 ] Add more context to some s3 errors
...
/ DCO (pull_request) Successful in 1m18s
/ Vulncheck (pull_request) Successful in 1m50s
/ Lint (pull_request) Successful in 3m1s
/ Tests (1.19) (pull_request) Successful in 2m47s
/ Tests (1.20) (pull_request) Successful in 3m2s
/ Builds (1.19) (pull_request) Successful in 2m54s
/ Builds (1.20) (pull_request) Successful in 7m58s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:08:33 +03:00
f921bc8af5
[ #143 ] Fix typo
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
be03c5178f
[ #143 ] Fix NoSuchKey error on get/head
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
9df8695463
[ #143 ] Fix transformToS3Error function
...
/ Builds (1.19) (pull_request) Successful in 3m11s
/ Builds (1.20) (pull_request) Successful in 2m57s
/ DCO (pull_request) Successful in 4m7s
/ Lint (pull_request) Successful in 2m25s
/ Tests (1.19) (pull_request) Successful in 3m9s
/ Tests (1.20) (pull_request) Successful in 3m18s
/ Vulncheck (pull_request) Successful in 1m15s
Unwrap error before checking for s3 error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-21 17:16:40 +03:00
614d703726
[ #106 ] Add chunk uploading
...
/ DCO (pull_request) Successful in 1m4s
/ Vulncheck (pull_request) Successful in 1m50s
/ Builds (1.19) (pull_request) Successful in 9m1s
/ Builds (1.20) (pull_request) Successful in 2m20s
/ Lint (pull_request) Successful in 10m19s
/ Tests (1.19) (pull_request) Successful in 2m45s
/ Tests (1.20) (pull_request) Successful in 3m19s
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-06-21 10:23:57 +03:00
23593eee3d
[ #111 ] Use request scope logger
...
/ Lint (pull_request) Failing after 32s
/ Tests (1.19) (pull_request) Failing after 32s
/ Tests (1.20) (pull_request) Failing after 32s
/ Builds (1.19) (pull_request) Failing after 32s
/ Builds (1.20) (pull_request) Failing after 32s
/ Vulncheck (pull_request) Failing after 57s
/ DCO (pull_request) Failing after 1m0s
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
462589fc0c
[ #103 ] Return 504 http code on timeout errors
...
/ Vulncheck (pull_request) Successful in 1m42s
/ Lint (pull_request) Successful in 3m31s
/ Tests (1.19) (pull_request) Successful in 2m57s
/ Tests (1.20) (pull_request) Successful in 3m4s
/ Builds (1.19) (pull_request) Successful in 2m35s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ DCO (pull_request) Successful in 1m7s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-14 09:16:21 +03:00
4a6e3a19ce
[ #125 ] Handle negative Content-Length on put
...
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
868edfdb31
[ #78 ] Add test of bucket removal with object not found error
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
a0f0d792b8
[ #78 ] layer: Clean up already removed object from tree
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-24 17:42:46 +03:00
9f186d9aba
[ #104 ] app: Reload copies numbers on SIGHUP
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-23 13:19:58 +03:00
e24bc3f2ce
[ #101 ] app: Refactor the default copies number setting
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-17 11:36:28 +03:00
e487ee5b7d
[ #70 ] Add arrays of copies numbers for location constraints
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
ee8cce662b
[ #74 ] service/tree: Add logger
...
Log error instead of failing when multiple unversioned nodes are found
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
70ec5a0a5b
[ #83 ] Don't create extra delete marker
...
We shouldn't create delete marker if:
1. object doesn't exist at all
2. last version is already a delete marker
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-19 17:56:11 +03:00
a5c89b78bc
[ #79 ] Return X-Owner-Id in head-bucket response
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-17 10:12:25 +03:00
a025f2e9c5
[ #59 ] tree: Make interface for tree service client
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 09:34:10 +03:00
bd3164c57f
[ #68 ] Fix pre-commit issues
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-24 16:22:06 +03:00
ef556bd8ac
[ #60 ] Use session token to set eACL during Complete Multipart Upload
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:22:11 +03:00
5104683f68
[ #60 ] Refactor start of periodic XML writer
...
Reduce code duplication for error handling
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:02:16 +03:00
8151753eeb
[ #60 ] Use periodic white space XML writer in Complete Multipart Upload
...
This mechanism is used by Amazon S3 to keep client's
connection alive while object is being constructed from
the upload parts.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 13:34:22 +03:00
2282c32822
[ #60 ] Add *NoHeader functions
...
Such functions should be used together with periodic white space
XML writer.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:07:36 +03:00
cf18158da4
[ #60 ] Implement periodic white space XML writer
...
Periodic white space XML writer sends XML header
and white spaces to the io.Writer.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:04:34 +03:00
0af06c3bd9
[TrueCloudLab#40] Add param to configure xml decoder
...
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/ " attribute
for CompleteMultipartUpload requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 15:44:13 +03:00
64e7356acc
[TrueCloudLab#32] Add custom policy unmarshaler
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 11:19:01 +00:00
32bf915502
[TrueCloudLab#37] Limit number of objects to delete
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 11:16:22 +00:00
813aa2f173
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 17:38:08 +03:00
740acadd37
[TrueCloudLab#36] Fix cors object payload
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-20 19:22:48 +03:00
aadefd98b6
[TrueCloudLab#25] Process allow and deny lists of zones in bucket head requests
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-02-17 13:47:48 +03:00
