Commit Graph

1169 Commits (962c1201251b13861d2480a53c64b55c09562a1f)

Author SHA1 Message Date
Alexey Vanin 962c120125 Release v0.28.0
/ DCO (pull_request) Successful in 1m50s Details
/ Vulncheck (pull_request) Successful in 2m40s Details
/ Builds (1.20) (pull_request) Successful in 2m41s Details
/ Builds (1.21) (pull_request) Successful in 2m8s Details
/ Lint (pull_request) Successful in 5m8s Details
/ Tests (1.20) (pull_request) Successful in 2m18s Details
/ Tests (1.21) (pull_request) Successful in 2m14s Details
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-12-07 13:48:06 +03:00
Alexey Vanin d1a1d489b1 [#267] Update api-go to fix stable marshal of empty structs
/ DCO (pull_request) Successful in 4m4s Details
/ Builds (1.20) (pull_request) Successful in 4m25s Details
/ Builds (1.21) (pull_request) Successful in 3m42s Details
/ Vulncheck (pull_request) Successful in 4m43s Details
/ Lint (pull_request) Successful in 6m13s Details
/ Tests (1.20) (pull_request) Successful in 4m47s Details
/ Tests (1.21) (pull_request) Successful in 4m43s Details
Newer version of api-go does not ignore non-nil empty
structures in protobuf messages, so compatibility with
previous version is preserved.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-11-21 16:45:32 +03:00
Alexey Vanin 2f29229383 [#249] Skip flaky test
/ Builds (1.20) (pull_request) Successful in 4m27s Details
/ Builds (1.21) (pull_request) Successful in 4m30s Details
/ DCO (pull_request) Successful in 5m11s Details
/ Vulncheck (pull_request) Successful in 7m37s Details
/ Lint (pull_request) Successful in 10m4s Details
/ Tests (1.20) (pull_request) Successful in 7m27s Details
/ Tests (1.21) (pull_request) Successful in 5m18s Details
Test manually passes but it breaks CI/CD sometimes.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-26 10:48:43 +03:00
Alexey Vanin ff5c55e14d [#245] Ignore debian/changelog for clean builds
Changes in debian/changelog produce `-dirty` suffix
in version.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-19 10:49:27 +00:00
Alexey Vanin e87c3715c5 [#233] Clean tag node in the tree service instead of removal
/ DCO (pull_request) Successful in 1m49s Details
/ Vulncheck (pull_request) Successful in 2m10s Details
/ Builds (1.20) (pull_request) Successful in 2m26s Details
/ Builds (1.21) (pull_request) Successful in 2m17s Details
/ Lint (pull_request) Successful in 4m21s Details
/ Tests (1.20) (pull_request) Successful in 2m15s Details
/ Tests (1.21) (pull_request) Successful in 2m10s Details
With new retry policy of tree service pool, gateway should avoid
deletion of system nodes from tree. Absence of node in the tree
will trigger retry. Other storage in the network may return already
deleted node while tree is not completely synced, and client will
get unexpected result.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-18 11:52:48 +03:00
Denis Kirillov 5f9555afad [#237] Fix list object versions marker param
/ DCO (pull_request) Successful in 1m33s Details
/ Builds (1.20) (pull_request) Successful in 1m59s Details
/ Builds (1.21) (pull_request) Successful in 1m33s Details
/ Lint (pull_request) Successful in 4m20s Details
/ Tests (1.20) (pull_request) Successful in 21m13s Details
/ Tests (1.21) (pull_request) Successful in 1m43s Details
/ Vulncheck (pull_request) Successful in 1m4s Details
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-12 11:36:39 +03:00
Alexey Vanin 4f9811fe70 [#233] Update tree service pool in SDK
/ DCO (pull_request) Successful in 1m43s Details
/ Builds (1.20) (pull_request) Successful in 2m0s Details
/ Builds (1.21) (pull_request) Successful in 1m37s Details
/ Vulncheck (pull_request) Successful in 4m31s Details
/ Lint (pull_request) Successful in 3m44s Details
/ Tests (1.20) (pull_request) Successful in 7m33s Details
/ Tests (1.21) (pull_request) Successful in 1m52s Details
Tree service pool now produces retries if
tree service returns 'not found' errors
or empty result on 'GetNodeByPath'

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-11 14:57:03 +03:00
Roman Loginov 670576248a [#231] Fix get latest version node
/ Vulncheck (pull_request) Successful in 1m33s Details
/ DCO (pull_request) Successful in 1m38s Details
/ Builds (1.20) (pull_request) Successful in 2m57s Details
/ Builds (1.21) (pull_request) Successful in 2m48s Details
/ Lint (pull_request) Successful in 4m53s Details
/ Tests (1.20) (pull_request) Successful in 3m19s Details
/ Tests (1.21) (pull_request) Successful in 3m16s Details
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 11:48:22 +03:00
Marina Biryukova 51e591877b [#207] Fix list parts with empty list
/ DCO (pull_request) Successful in 1m2s Details
/ Vulncheck (pull_request) Successful in 1m25s Details
/ Builds (1.20) (pull_request) Successful in 1m59s Details
/ Builds (1.21) (pull_request) Successful in 1m52s Details
/ Lint (pull_request) Successful in 8m23s Details
/ Tests (1.20) (pull_request) Successful in 2m1s Details
/ Tests (1.21) (pull_request) Successful in 9m26s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-21 11:27:20 +03:00
Marina Biryukova a4c612614a [#210] Fix multipart object reader
/ Vulncheck (pull_request) Successful in 2m38s Details
/ DCO (pull_request) Successful in 3m40s Details
/ Lint (pull_request) Successful in 5m20s Details
/ Tests (1.20) (pull_request) Successful in 3m19s Details
/ Tests (1.21) (pull_request) Successful in 3m8s Details
/ Builds (1.20) (pull_request) Successful in 6m1s Details
/ Builds (1.21) (pull_request) Successful in 1m33s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 16:30:08 +03:00
Marina Biryukova 12cf29aed2 [#207] Fix part-number-marker handling
/ DCO (pull_request) Successful in 1m57s Details
/ Vulncheck (pull_request) Successful in 2m21s Details
/ Builds (1.20) (pull_request) Successful in 1m36s Details
/ Builds (1.21) (pull_request) Successful in 7m55s Details
/ Lint (pull_request) Successful in 13m57s Details
/ Tests (1.20) (pull_request) Successful in 2m34s Details
/ Tests (1.21) (pull_request) Successful in 1m57s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
Alexey Vanin 16840f1256 [#177] Add release instructions page
/ DCO (pull_request) Successful in 54s Details
/ Vulncheck (pull_request) Successful in 1m14s Details
/ Builds (1.20) (pull_request) Successful in 3m14s Details
/ Builds (1.21) (pull_request) Successful in 8m42s Details
/ Lint (pull_request) Successful in 13m19s Details
/ Tests (1.20) (pull_request) Successful in 1m31s Details
/ Tests (1.21) (pull_request) Successful in 6m15s Details
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-09-07 12:32:12 +00:00
Marina Biryukova 066b9a0250 [#142] Add trace ID into log when tracing is enabled
/ Vulncheck (pull_request) Successful in 1m29s Details
/ DCO (pull_request) Successful in 2m18s Details
/ Lint (pull_request) Successful in 3m48s Details
/ Tests (1.20) (pull_request) Successful in 2m4s Details
/ Tests (1.21) (pull_request) Successful in 1m42s Details
/ Builds (1.20) (pull_request) Successful in 7m6s Details
/ Builds (1.21) (pull_request) Successful in 1m56s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-07 14:19:37 +03:00
Artem Tataurov 54e1c333a1 [#152] authmate: Add basic error types and exit codes
/ DCO (pull_request) Successful in 1m9s Details
/ Vulncheck (pull_request) Successful in 5m17s Details
/ Builds (1.20) (pull_request) Successful in 2m1s Details
/ Builds (1.21) (pull_request) Successful in 7m18s Details
/ Lint (pull_request) Successful in 15m19s Details
/ Tests (1.20) (pull_request) Successful in 1m14s Details
/ Tests (1.21) (pull_request) Successful in 11m45s Details
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-09-06 23:56:56 +03:00
Marina Biryukova 69227b4845 [#199] Add metrics for HTTP endpoint status
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-05 13:30:27 +00:00
Roman Loginov c66c09765d [#196] Support soft memory limit setting
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-09-05 13:13:56 +00:00
Marina Biryukova 9120e97ac5 [#203] Add go1.21 to CI
/ DCO (pull_request) Successful in 59s Details
/ Vulncheck (pull_request) Successful in 1m35s Details
/ Builds (1.20) (pull_request) Successful in 1m21s Details
/ Builds (1.21) (pull_request) Successful in 7m46s Details
/ Lint (pull_request) Successful in 12m44s Details
/ Tests (1.20) (pull_request) Successful in 2m9s Details
/ Tests (1.21) (pull_request) Successful in 2m0s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-31 15:26:07 +03:00
Marina Biryukova 2fc328a6d2 [#195] Add log constants linter
/ DCO (pull_request) Successful in 1m36s Details
/ Vulncheck (pull_request) Successful in 1m41s Details
/ Lint (pull_request) Successful in 8m31s Details
/ Tests (1.19) (pull_request) Successful in 2m6s Details
/ Tests (1.20) (pull_request) Successful in 1m57s Details
/ Builds (1.19) (pull_request) Successful in 2m21s Details
/ Builds (1.20) (pull_request) Successful in 9m3s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-28 12:58:44 +03:00
Denis Kirillov b5fce5c8d2 [#168] Skip only invalid policies and copies instead of ignoring all of them
/ DCO (pull_request) Successful in 2m48s Details
/ Vulncheck (pull_request) Successful in 3m30s Details
/ Builds (1.19) (pull_request) Successful in 3m39s Details
/ Builds (1.20) (pull_request) Successful in 3m28s Details
/ Lint (pull_request) Successful in 4m14s Details
/ Tests (1.19) (pull_request) Successful in 3m3s Details
/ Tests (1.20) (pull_request) Successful in 2m45s Details
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
Denis Kirillov 41a128b1aa [#185] Update CHANGELOG.md
/ Vulncheck (pull_request) Successful in 1m47s Details
/ Lint (pull_request) Successful in 2m46s Details
/ Tests (1.19) (pull_request) Successful in 2m9s Details
/ Tests (1.20) (pull_request) Successful in 2m6s Details
/ DCO (pull_request) Successful in 4m17s Details
/ Builds (1.19) (pull_request) Successful in 6m36s Details
/ Builds (1.20) (pull_request) Successful in 1m43s Details
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:11 +03:00
Denis Kirillov 6617adc22b [#185] Use correct object size when object is combined or encrypted
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
Denis Kirillov 631d9d83b6 [#185] Fix payload reader
When we use io.CopyBuffer it check for exact io.EOF matching,
so we need keep original EOF error otherwise io.CopyBuffer returns error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
Denis Kirillov adec93af54 [#185] tree: Fix getSubTreeMultipartUploads
Every tree node contains only FileName
but key in multipart info must contain FilePath

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
Denis Kirillov 8898c2ec08 [#185] Add tests for list multipart uploads
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
Roman Loginov 8efcc957ea [#96] Move log messages to constants
/ DCO (pull_request) Successful in 1m35s Details
/ Builds (1.19) (pull_request) Successful in 2m14s Details
/ Builds (1.20) (pull_request) Successful in 2m9s Details
/ Vulncheck (pull_request) Successful in 5m39s Details
/ Lint (pull_request) Successful in 2m49s Details
/ Tests (1.19) (pull_request) Successful in 7m34s Details
/ Tests (1.20) (pull_request) Successful in 1m44s Details
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
Denis Kirillov 6b728fef87 [#192] Add tests to make sure client_cut flag is passed to sdk
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
Denis Kirillov 6b1f365e65 [#192] Support client cut
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
Denis Kirillov fcf1c45ad2 [#188] Fix url escaping
/ Vulncheck (pull_request) Successful in 2m41s Details
/ Builds (1.19) (pull_request) Successful in 3m51s Details
/ Builds (1.20) (pull_request) Successful in 3m22s Details
/ DCO (pull_request) Successful in 5m9s Details
/ Lint (pull_request) Successful in 5m22s Details
/ Tests (1.19) (pull_request) Successful in 5m39s Details
/ Tests (1.20) (pull_request) Successful in 5m18s Details
Url escaping has already been done in `net/http/request.go`

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-22 11:27:39 +03:00
Roman Loginov 6a9d3261a7 [#117] Refactor fetch/parse config parameters functions
/ Vulncheck (pull_request) Successful in 2m0s Details
/ Lint (pull_request) Successful in 2m53s Details
/ Tests (1.19) (pull_request) Successful in 2m14s Details
/ Tests (1.20) (pull_request) Successful in 1m51s Details
/ DCO (pull_request) Successful in 4m0s Details
/ Builds (1.19) (pull_request) Successful in 6m30s Details
/ Builds (1.20) (pull_request) Successful in 2m7s Details
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-22 08:05:26 +00:00
Marina Biryukova 012ece40bb [#180] Fix linter issues
/ Builds (1.19) (pull_request) Successful in 2m55s Details
/ Builds (1.20) (pull_request) Successful in 2m47s Details
/ DCO (pull_request) Successful in 3m4s Details
/ Vulncheck (pull_request) Successful in 3m39s Details
/ Lint (pull_request) Successful in 4m19s Details
/ Tests (1.19) (pull_request) Successful in 3m52s Details
/ Tests (1.20) (pull_request) Successful in 3m18s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-21 17:23:24 +03:00
Artem Tataurov c750c87a61 [#51] metrics: Add a metric of currently used nodes
/ DCO (pull_request) Successful in 1m18s Details
/ Builds (1.19) (pull_request) Successful in 2m23s Details
/ Builds (1.20) (pull_request) Successful in 2m18s Details
/ Vulncheck (pull_request) Successful in 5m26s Details
/ Lint (pull_request) Failing after 2m4s Details
/ Tests (1.19) (pull_request) Successful in 7m42s Details
/ Tests (1.20) (pull_request) Successful in 2m11s Details
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
Artem Tataurov 94a42fa273 [#51] Update frostfs-sdk-go
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
Roman Loginov 40d7f844e3 [#137] Refactor context data retrievers
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
Marina Biryukova 52b89d3497 [#153] Add labels in metrics of total bytes
/ DCO (pull_request) Successful in 2m6s Details
/ Vulncheck (pull_request) Successful in 2m28s Details
/ Builds (1.19) (pull_request) Successful in 3m1s Details
/ Builds (1.20) (pull_request) Successful in 2m46s Details
/ Lint (pull_request) Successful in 2m14s Details
/ Tests (1.19) (pull_request) Successful in 3m10s Details
/ Tests (1.20) (pull_request) Successful in 2m27s Details
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-16 10:30:40 +03:00
Denis Kirillov 6b109eee92 [#182] Fix parsing signed headers in presigned urls
/ Vulncheck (pull_request) Failing after 3s Details
/ Builds (1.19) (pull_request) Successful in 8m22s Details
/ Builds (1.20) (pull_request) Failing after 3s Details
/ Lint (pull_request) Successful in 9m51s Details
/ Tests (1.19) (pull_request) Failing after 3s Details
/ Tests (1.20) (pull_request) Failing after 2s Details
/ DCO (pull_request) Failing after 2s Details
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 13:04:53 +03:00
Denis Kirillov 18878b66d3 [#175] Use gate owner as object owner
/ DCO (pull_request) Failing after 3s Details
/ Vulncheck (pull_request) Failing after 2s Details
/ Builds (1.19) (pull_request) Failing after 3s Details
/ Builds (1.20) (pull_request) Successful in 7m47s Details
/ Lint (pull_request) Failing after 4s Details
/ Tests (1.19) (pull_request) Successful in 8m8s Details
/ Tests (1.20) (pull_request) Failing after 3s Details
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
Alexey Vanin 46eae4a356 [#179] Fix GetSubTree failures with updated SDK
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-08-02 14:56:23 +00:00
Denis Kirillov fe897ec588 [#178] wrapReader: Fix goroutine leak
/ Vulncheck (pull_request) Successful in 59s Details
/ Builds (1.19) (pull_request) Successful in 1m28s Details
/ Builds (1.20) (pull_request) Successful in 1m19s Details
/ DCO (pull_request) Successful in 45s Details
/ Lint (pull_request) Successful in 1m45s Details
/ Tests (1.19) (pull_request) Successful in 7m51s Details
/ Tests (1.20) (pull_request) Successful in 1m43s Details
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-01 17:59:34 +03:00
Denis Kirillov 52931663e1 [#176] multipart: Replace part on re-upload
/ Builds (1.19) (pull_request) Successful in 2m57s Details
/ Builds (1.20) (pull_request) Successful in 2m36s Details
/ DCO (pull_request) Successful in 1m15s Details
/ Vulncheck (pull_request) Successful in 5m15s Details
/ Lint (pull_request) Successful in 3m50s Details
/ Tests (1.19) (pull_request) Successful in 8m14s Details
/ Tests (1.20) (pull_request) Successful in 3m17s Details
We want to have exactly one object and tree node for each part number

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-21 16:20:04 +03:00
Denis Kirillov 1a09041cd1 [#63] Simplify multiObjectReader and add tests
/ DCO (pull_request) Successful in 1m19s Details
/ Vulncheck (pull_request) Successful in 2m31s Details
/ Builds (1.19) (pull_request) Successful in 2m51s Details
/ Builds (1.20) (pull_request) Successful in 2m34s Details
/ Lint (pull_request) Successful in 5m51s Details
/ Tests (1.19) (pull_request) Successful in 9m36s Details
/ Tests (1.20) (pull_request) Successful in 10m20s Details
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
Denis Kirillov 631b7f67b4 [#63] multipart: Log upload id for every failed request
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
Denis Kirillov 8ca2998297 [#63] Update CHANGELOG.md
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
Denis Kirillov bcf5a85aab [#63] multipart: Fix copying
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
Denis Kirillov ad81b599dd [#63] Add fast multipart upload
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
Denis Kirillov 361d10cc78 [#174] Fix query for listing multipart uploads
/ Builds (1.19) (pull_request) Successful in 3m2s Details
/ Builds (1.20) (pull_request) Successful in 2m50s Details
/ DCO (pull_request) Successful in 3m59s Details
/ Vulncheck (pull_request) Successful in 2m39s Details
/ Lint (pull_request) Successful in 3m56s Details
/ Tests (1.19) (pull_request) Successful in 3m5s Details
/ Tests (1.20) (pull_request) Successful in 3m12s Details
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
Denis Kirillov 62e6b49254 [#174] Log unmatched requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
Denis Kirillov 80c4982bd4 [#174] Add router tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
Denis Kirillov 73ed3f7782 [#174] Fix router filter query matching
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
Denis Kirillov 6e3595e35b [#174] Fix object keys with slashes in chi
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
Alexey Vanin 57add29643 [#173] Use forked actions in workflow
/ Builds (1.19) (pull_request) Successful in 2m37s Details
/ Builds (1.20) (pull_request) Successful in 2m39s Details
/ DCO (pull_request) Successful in 3m38s Details
/ Vulncheck (pull_request) Successful in 5m10s Details
/ Lint (pull_request) Successful in 3m30s Details
/ Tests (1.19) (pull_request) Successful in 2m50s Details
/ Tests (1.20) (pull_request) Successful in 8m11s Details
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-18 15:18:34 +03:00