Commit graph

1171 commits

Author SHA1 Message Date
298662df9d [#221] Expand xmlns field ignore
All checks were successful
/ Vulncheck (pull_request) Successful in 1m38s
/ Lint (pull_request) Successful in 2m49s
/ Tests (1.20) (pull_request) Successful in 1m56s
/ Tests (1.21) (pull_request) Successful in 1m44s
/ DCO (pull_request) Successful in 3m56s
/ Builds (1.20) (pull_request) Successful in 6m25s
/ Builds (1.21) (pull_request) Successful in 1m36s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-13 16:21:13 +03:00
10a03faeb4 [#197] Update CHANGELOG.md
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
65412ce1d3 [#197] Configure buffer max size for PUT
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
7de73f6b73 [#197] Disable homomorphic hash for PUT
Disable TZ hash for PUT if it's disabled for container itself

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
8fc9d93f37 [#197] Update SDK
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
7301ca52ab [#154] Rename OwnerPublicKey to SeedKey
All checks were successful
/ Vulncheck (pull_request) Successful in 1m30s
/ DCO (pull_request) Successful in 1m29s
/ Builds (1.20) (pull_request) Successful in 2m13s
/ Builds (1.21) (pull_request) Successful in 1m52s
/ Lint (pull_request) Successful in 3m59s
/ Tests (1.20) (pull_request) Successful in 2m11s
/ Tests (1.21) (pull_request) Successful in 2m33s
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 14:00:37 +03:00
e1ec61ddfc [#215] Fix get latest version node
When the object version is received,
the node of the secondary object may return.
Now we choose the right node ourselves.

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 09:21:41 +00:00
e3f2d59565 [#154] Rename access key to secret key
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 09:20:39 +00:00
c4af1dc4ad [#171] Update message error auth header malformed
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-04 11:13:12 +00:00
b8c93ed391 [#172] Convert handler config to interface
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-04 11:01:27 +00:00
51e591877b [#207] Fix list parts with empty list
All checks were successful
/ DCO (pull_request) Successful in 1m2s
/ Vulncheck (pull_request) Successful in 1m25s
/ Builds (1.20) (pull_request) Successful in 1m59s
/ Builds (1.21) (pull_request) Successful in 1m52s
/ Lint (pull_request) Successful in 8m23s
/ Tests (1.20) (pull_request) Successful in 2m1s
/ Tests (1.21) (pull_request) Successful in 9m26s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-21 11:27:20 +03:00
a4c612614a [#210] Fix multipart object reader
All checks were successful
/ Vulncheck (pull_request) Successful in 2m38s
/ DCO (pull_request) Successful in 3m40s
/ Lint (pull_request) Successful in 5m20s
/ Tests (1.20) (pull_request) Successful in 3m19s
/ Tests (1.21) (pull_request) Successful in 3m8s
/ Builds (1.20) (pull_request) Successful in 6m1s
/ Builds (1.21) (pull_request) Successful in 1m33s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 16:30:08 +03:00
12cf29aed2 [#207] Fix part-number-marker handling
All checks were successful
/ DCO (pull_request) Successful in 1m57s
/ Vulncheck (pull_request) Successful in 2m21s
/ Builds (1.20) (pull_request) Successful in 1m36s
/ Builds (1.21) (pull_request) Successful in 7m55s
/ Lint (pull_request) Successful in 13m57s
/ Tests (1.20) (pull_request) Successful in 2m34s
/ Tests (1.21) (pull_request) Successful in 1m57s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
16840f1256 [#177] Add release instructions page
All checks were successful
/ DCO (pull_request) Successful in 54s
/ Vulncheck (pull_request) Successful in 1m14s
/ Builds (1.20) (pull_request) Successful in 3m14s
/ Builds (1.21) (pull_request) Successful in 8m42s
/ Lint (pull_request) Successful in 13m19s
/ Tests (1.20) (pull_request) Successful in 1m31s
/ Tests (1.21) (pull_request) Successful in 6m15s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-09-07 12:32:12 +00:00
066b9a0250 [#142] Add trace ID into log when tracing is enabled
All checks were successful
/ Vulncheck (pull_request) Successful in 1m29s
/ DCO (pull_request) Successful in 2m18s
/ Lint (pull_request) Successful in 3m48s
/ Tests (1.20) (pull_request) Successful in 2m4s
/ Tests (1.21) (pull_request) Successful in 1m42s
/ Builds (1.20) (pull_request) Successful in 7m6s
/ Builds (1.21) (pull_request) Successful in 1m56s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-07 14:19:37 +03:00
54e1c333a1 [#152] authmate: Add basic error types and exit codes
All checks were successful
/ DCO (pull_request) Successful in 1m9s
/ Vulncheck (pull_request) Successful in 5m17s
/ Builds (1.20) (pull_request) Successful in 2m1s
/ Builds (1.21) (pull_request) Successful in 7m18s
/ Lint (pull_request) Successful in 15m19s
/ Tests (1.20) (pull_request) Successful in 1m14s
/ Tests (1.21) (pull_request) Successful in 11m45s
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-09-06 23:56:56 +03:00
69227b4845 [#199] Add metrics for HTTP endpoint status
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-05 13:30:27 +00:00
c66c09765d [#196] Support soft memory limit setting
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-09-05 13:13:56 +00:00
9120e97ac5 [#203] Add go1.21 to CI
All checks were successful
/ DCO (pull_request) Successful in 59s
/ Vulncheck (pull_request) Successful in 1m35s
/ Builds (1.20) (pull_request) Successful in 1m21s
/ Builds (1.21) (pull_request) Successful in 7m46s
/ Lint (pull_request) Successful in 12m44s
/ Tests (1.20) (pull_request) Successful in 2m9s
/ Tests (1.21) (pull_request) Successful in 2m0s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-31 15:26:07 +03:00
2fc328a6d2 [#195] Add log constants linter
All checks were successful
/ DCO (pull_request) Successful in 1m36s
/ Vulncheck (pull_request) Successful in 1m41s
/ Lint (pull_request) Successful in 8m31s
/ Tests (1.19) (pull_request) Successful in 2m6s
/ Tests (1.20) (pull_request) Successful in 1m57s
/ Builds (1.19) (pull_request) Successful in 2m21s
/ Builds (1.20) (pull_request) Successful in 9m3s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-28 12:58:44 +03:00
b5fce5c8d2 [#168] Skip only invalid policies and copies instead of ignoring all of them
All checks were successful
/ DCO (pull_request) Successful in 2m48s
/ Vulncheck (pull_request) Successful in 3m30s
/ Builds (1.19) (pull_request) Successful in 3m39s
/ Builds (1.20) (pull_request) Successful in 3m28s
/ Lint (pull_request) Successful in 4m14s
/ Tests (1.19) (pull_request) Successful in 3m3s
/ Tests (1.20) (pull_request) Successful in 2m45s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
41a128b1aa [#185] Update CHANGELOG.md
All checks were successful
/ Vulncheck (pull_request) Successful in 1m47s
/ Lint (pull_request) Successful in 2m46s
/ Tests (1.19) (pull_request) Successful in 2m9s
/ Tests (1.20) (pull_request) Successful in 2m6s
/ DCO (pull_request) Successful in 4m17s
/ Builds (1.19) (pull_request) Successful in 6m36s
/ Builds (1.20) (pull_request) Successful in 1m43s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:11 +03:00
6617adc22b [#185] Use correct object size when object is combined or encrypted
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
631d9d83b6 [#185] Fix payload reader
When we use io.CopyBuffer it check for exact io.EOF matching,
so we need keep original EOF error otherwise io.CopyBuffer returns error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
adec93af54 [#185] tree: Fix getSubTreeMultipartUploads
Every tree node contains only FileName
but key in multipart info must contain FilePath

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8898c2ec08 [#185] Add tests for list multipart uploads
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8efcc957ea [#96] Move log messages to constants
All checks were successful
/ DCO (pull_request) Successful in 1m35s
/ Builds (1.19) (pull_request) Successful in 2m14s
/ Builds (1.20) (pull_request) Successful in 2m9s
/ Vulncheck (pull_request) Successful in 5m39s
/ Lint (pull_request) Successful in 2m49s
/ Tests (1.19) (pull_request) Successful in 7m34s
/ Tests (1.20) (pull_request) Successful in 1m44s
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b728fef87 [#192] Add tests to make sure client_cut flag is passed to sdk
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
6b1f365e65 [#192] Support client cut
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
fcf1c45ad2 [#188] Fix url escaping
All checks were successful
/ Vulncheck (pull_request) Successful in 2m41s
/ Builds (1.19) (pull_request) Successful in 3m51s
/ Builds (1.20) (pull_request) Successful in 3m22s
/ DCO (pull_request) Successful in 5m9s
/ Lint (pull_request) Successful in 5m22s
/ Tests (1.19) (pull_request) Successful in 5m39s
/ Tests (1.20) (pull_request) Successful in 5m18s
Url escaping has already been done in `net/http/request.go`

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-22 11:27:39 +03:00
6a9d3261a7 [#117] Refactor fetch/parse config parameters functions
All checks were successful
/ Vulncheck (pull_request) Successful in 2m0s
/ Lint (pull_request) Successful in 2m53s
/ Tests (1.19) (pull_request) Successful in 2m14s
/ Tests (1.20) (pull_request) Successful in 1m51s
/ DCO (pull_request) Successful in 4m0s
/ Builds (1.19) (pull_request) Successful in 6m30s
/ Builds (1.20) (pull_request) Successful in 2m7s
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-22 08:05:26 +00:00
012ece40bb [#180] Fix linter issues
All checks were successful
/ Builds (1.19) (pull_request) Successful in 2m55s
/ Builds (1.20) (pull_request) Successful in 2m47s
/ DCO (pull_request) Successful in 3m4s
/ Vulncheck (pull_request) Successful in 3m39s
/ Lint (pull_request) Successful in 4m19s
/ Tests (1.19) (pull_request) Successful in 3m52s
/ Tests (1.20) (pull_request) Successful in 3m18s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-21 17:23:24 +03:00
c750c87a61 [#51] metrics: Add a metric of currently used nodes
Some checks failed
/ DCO (pull_request) Successful in 1m18s
/ Builds (1.19) (pull_request) Successful in 2m23s
/ Builds (1.20) (pull_request) Successful in 2m18s
/ Vulncheck (pull_request) Successful in 5m26s
/ Lint (pull_request) Failing after 2m4s
/ Tests (1.19) (pull_request) Successful in 7m42s
/ Tests (1.20) (pull_request) Successful in 2m11s
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
94a42fa273 [#51] Update frostfs-sdk-go
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
40d7f844e3 [#137] Refactor context data retrievers
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
52b89d3497 [#153] Add labels in metrics of total bytes
All checks were successful
/ DCO (pull_request) Successful in 2m6s
/ Vulncheck (pull_request) Successful in 2m28s
/ Builds (1.19) (pull_request) Successful in 3m1s
/ Builds (1.20) (pull_request) Successful in 2m46s
/ Lint (pull_request) Successful in 2m14s
/ Tests (1.19) (pull_request) Successful in 3m10s
/ Tests (1.20) (pull_request) Successful in 2m27s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-16 10:30:40 +03:00
6b109eee92 [#182] Fix parsing signed headers in presigned urls
Some checks failed
/ Vulncheck (pull_request) Failing after 3s
/ Builds (1.19) (pull_request) Successful in 8m22s
/ Builds (1.20) (pull_request) Failing after 3s
/ Lint (pull_request) Successful in 9m51s
/ Tests (1.19) (pull_request) Failing after 3s
/ Tests (1.20) (pull_request) Failing after 2s
/ DCO (pull_request) Failing after 2s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 13:04:53 +03:00
18878b66d3 [#175] Use gate owner as object owner
Some checks failed
/ DCO (pull_request) Failing after 3s
/ Vulncheck (pull_request) Failing after 2s
/ Builds (1.19) (pull_request) Failing after 3s
/ Builds (1.20) (pull_request) Successful in 7m47s
/ Lint (pull_request) Failing after 4s
/ Tests (1.19) (pull_request) Successful in 8m8s
/ Tests (1.20) (pull_request) Failing after 3s
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
46eae4a356 [#179] Fix GetSubTree failures with updated SDK
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-08-02 14:56:23 +00:00
fe897ec588 [#178] wrapReader: Fix goroutine leak
All checks were successful
/ Vulncheck (pull_request) Successful in 59s
/ Builds (1.19) (pull_request) Successful in 1m28s
/ Builds (1.20) (pull_request) Successful in 1m19s
/ DCO (pull_request) Successful in 45s
/ Lint (pull_request) Successful in 1m45s
/ Tests (1.19) (pull_request) Successful in 7m51s
/ Tests (1.20) (pull_request) Successful in 1m43s
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-01 17:59:34 +03:00
52931663e1 [#176] multipart: Replace part on re-upload
All checks were successful
/ Builds (1.19) (pull_request) Successful in 2m57s
/ Builds (1.20) (pull_request) Successful in 2m36s
/ DCO (pull_request) Successful in 1m15s
/ Vulncheck (pull_request) Successful in 5m15s
/ Lint (pull_request) Successful in 3m50s
/ Tests (1.19) (pull_request) Successful in 8m14s
/ Tests (1.20) (pull_request) Successful in 3m17s
We want to have exactly one object and tree node for each part number

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-21 16:20:04 +03:00
1a09041cd1 [#63] Simplify multiObjectReader and add tests
All checks were successful
/ DCO (pull_request) Successful in 1m19s
/ Vulncheck (pull_request) Successful in 2m31s
/ Builds (1.19) (pull_request) Successful in 2m51s
/ Builds (1.20) (pull_request) Successful in 2m34s
/ Lint (pull_request) Successful in 5m51s
/ Tests (1.19) (pull_request) Successful in 9m36s
/ Tests (1.20) (pull_request) Successful in 10m20s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
631b7f67b4 [#63] multipart: Log upload id for every failed request
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
8ca2998297 [#63] Update CHANGELOG.md
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
bcf5a85aab [#63] multipart: Fix copying
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
ad81b599dd [#63] Add fast multipart upload
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
361d10cc78 [#174] Fix query for listing multipart uploads
All checks were successful
/ Builds (1.19) (pull_request) Successful in 3m2s
/ Builds (1.20) (pull_request) Successful in 2m50s
/ DCO (pull_request) Successful in 3m59s
/ Vulncheck (pull_request) Successful in 2m39s
/ Lint (pull_request) Successful in 3m56s
/ Tests (1.19) (pull_request) Successful in 3m5s
/ Tests (1.20) (pull_request) Successful in 3m12s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
62e6b49254 [#174] Log unmatched requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
80c4982bd4 [#174] Add router tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
73ed3f7782 [#174] Fix router filter query matching
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00