Commit graph

762 commits

Author SHA1 Message Date
c5c6272029 [#221] pool: Make sampler safe for concurrent using
All checks were successful
DCO / DCO (pull_request) Successful in 1m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m31s
Tests and linters / Lint (pull_request) Successful in 2m25s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-20 14:14:14 +03:00
3de256d05e [#223] object: Introduce new fields for ECHeader
All checks were successful
DCO / DCO (pull_request) Successful in 1m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m23s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m31s
Tests and linters / Lint (pull_request) Successful in 2m44s
* Introduce `parentSplitID`, `parentSplitParentID` fields
  for `ECHeader`;
* Fix ECHeader's constructor;
* Fix `Split` and `Reconstruct`;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:16:28 +03:00
09b79d13f3 [#223] object: Introduce ec_parent search filter
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:03:40 +03:00
d4e6f4e125 [#223] go.mod: Update frosts-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:02:44 +03:00
b2ad1f3b3e [#215] client: Introduce apemanager rpc interface
All checks were successful
DCO / DCO (pull_request) Successful in 1m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m28s
Tests and linters / Lint (pull_request) Successful in 2m58s
* Introduce `APEManagerAddChain`, `APEManagerRemoveChain`, `APEManagerListChains`.
* Introduce reqeuest/response types for these handlers (Prm*, Res*).
* Inroduce status type for apemanager `APEManagerAccessDenied`; add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:52 +03:00
32a975a20d [#215] apemanager: Introduce apemanager types
* Introduce `Chain`, `ChainTarget` and `TargetType`.
* Implement api-v2 converters for the introduced types.
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:49 +03:00
eaf36706a2 [#215] go.mod: Update frostfs-api-go/v2 package version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-08 11:18:18 +03:00
02c936f397 [#216] netmap: Add policy decode fuzz test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-07 11:10:37 +00:00
99e02858af [#220] netmap: Fix setters for Replica.DataCount/ParityCount
All checks were successful
Tests and linters / Tests (1.21) (pull_request) Successful in 5m44s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m0s
DCO / DCO (pull_request) Successful in 9m11s
Tests and linters / Lint (pull_request) Successful in 16m8s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 09:34:14 +03:00
12ddefe078 [#218] object: Implement Range\RangeHash requests for EC object
All checks were successful
DCO / DCO (pull_request) Successful in 4m33s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m25s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m34s
Tests and linters / Lint (pull_request) Successful in 6m10s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-02 11:01:21 +03:00
20ab57bf7e [#214] object: Implement Get\Head requests for EC object
All checks were successful
DCO / DCO (pull_request) Successful in 1m3s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m15s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Tests and linters / Lint (pull_request) Successful in 2m13s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 11:07:26 +03:00
3790142b10 [#212] pool: Control sub tree nodes order
All checks were successful
DCO / DCO (pull_request) Successful in 1m37s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m53s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m4s
Tests and linters / Lint (pull_request) Successful in 3m22s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-02 17:15:49 +03:00
ec0cb2169f [#211] object: Fix setIDWithSignature
All checks were successful
DCO / DCO (pull_request) Successful in 1m0s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m7s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s
Tests and linters / Lint (pull_request) Successful in 2m12s
* Calculate and set checksum before ID is calculated.
* Add header verification for parts in unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
425d48f68b [#211] netmap: Introduce ReplicaDescriptor method
* Make ReplicaNumberByIndex deprecated.
* Introduce ReplicaDescriptor method that access i-th replica directly.
* Introduce new getters for ReplicaDescriptor.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
6d0da3f861 [#211] go.mod: Update frostfs-api-go version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
1af9b6d18b [#155] sdk-go: Add buffer support for payloadSizeLimiter
All checks were successful
DCO / DCO (pull_request) Successful in 58s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m18s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Tests and linters / Lint (pull_request) Successful in 2m12s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
bd2d350b09 [#205] object: Initial EC implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
e9be3e6d94 [#205] netmap: Add well-known EC parameters to network config
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
70e9e40c7f [#205] netmap: Add EC statement to placement policy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
d33b54d280 [#205] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
6f248436a5 [#210] client/put_transformer: Fix error handling
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Tests and linters / Lint (pull_request) Successful in 2m39s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-19 15:36:51 +03:00
edd40474e8 [#209] pre-commit: Add unit-test hook
All checks were successful
DCO / DCO (pull_request) Successful in 1m27s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m36s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m5s
Tests and linters / Lint (pull_request) Successful in 3m13s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
d9ec7c1988 [#209] Makefile: Allow to override test flags
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
64b83f8220 [#209] Makefile: Update golangci-lint to 1.56.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
7212f38115 [#209] pre-commit: Remove gitlint
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
8081445ff2 [#208] go.mod: Bump frostfs-api-go version
All checks were successful
DCO / DCO (pull_request) Successful in 1m5s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m34s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m30s
Tests and linters / Lint (pull_request) Successful in 2m56s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:34:04 +03:00
6a7ef9d8c3 [#208] go.mod: Bump protobuf version
Some checks failed
DCO / DCO (pull_request) Failing after 42s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m54s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m51s
Tests and linters / Lint (pull_request) Successful in 4m27s
Found by vulncheck:
Vulnerability #1: GO-2024-2611
    Infinite loop in JSON unmarshaling in google.golang.org/protobuf
  More info: https://pkg.go.dev/vuln/GO-2024-2611
  Module: google.golang.org/protobuf
    Found in: google.golang.org/protobuf@v1.32.0
    Fixed in: google.golang.org/protobuf@v1.33.0

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:23:32 +03:00
6fe4e2541d [#207] netmap: Fix string escape in PlacementPolicy.String()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-01 15:02:05 +00:00
a5fab572ff [#206] Add session tokens for container read operations
All checks were successful
Tests and linters / Tests (1.21) (pull_request) Successful in 1m29s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m37s
Tests and linters / Lint (pull_request) Successful in 2m45s
DCO / DCO (pull_request) Successful in 42s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 15:18:53 +03:00
a86170f53a [#202] object: Reset marshal data on CutPayload
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m26s
Tests and linters / Lint (pull_request) Successful in 2m13s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
aa41f71dcc [#202] eacl: Drop storage group test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
3a00fd51e4 [#202] go.mod: Update api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:43 +03:00
65b4525b3b [#198] object/user: Add ScriptHash method
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-26 14:10:09 +00:00
7efff9d53d [#201] .forgejo: Update dco-go to v3
All checks were successful
DCO / DCO (pull_request) Successful in 8m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m3s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m8s
Tests and linters / Lint (pull_request) Successful in 15m19s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 12:21:42 +03:00
110b7e4170 [#200] pre-commit: Fix linter invocation target
All checks were successful
DCO / DCO (pull_request) Successful in 1m33s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m49s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m56s
Tests and linters / Lint (pull_request) Successful in 3m21s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-17 17:56:20 +03:00
56debcfa56 [#190] sdk-go: Gofumpt fixes
All checks were successful
DCO / DCO (pull_request) Successful in 56s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m20s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
157a9930e8 [#190] sdk-go: Pass user.ID by value
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
1c07098740 [#194] make: Fix make test
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 11:48:51 +00:00
03d35dd1f3 [#166] netmap: Add support YML tests
All checks were successful
DCO / DCO (pull_request) Successful in 1m58s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m10s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m17s
Tests and linters / Lint (pull_request) Successful in 4m4s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 14:43:03 +03:00
dea8759762 [#166] netmap: Move tests from JSON to YML
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 14:42:39 +03:00
3787477133 [#189] client: Make PrmDial fields public
All checks were successful
DCO / DCO (pull_request) Successful in 42s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m19s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Tests and linters / Lint (pull_request) Successful in 2m58s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-14 11:18:00 +03:00
e91d40e250 [#189] client: Make PrmInit fields public for client
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-13 15:18:07 +03:00
ab75edd709 [#191] pool/tree: Support limit request attempts
All checks were successful
DCO / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Tests and linters / Lint (pull_request) Successful in 2m35s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:40 +03:00
8999d2f080 [#191] pool/tree: Support request id
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:35 +03:00
6fbe1595cb [#121] pool: Refactor PrmObjectSearch usage
All checks were successful
DCO / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m24s
Tests and linters / Lint (pull_request) Successful in 2m33s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:15 +03:00
a9237aabd2 [#121] client: Make PrmObjectSearch fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:06 +03:00
a487033505 [#121] client: Nuke out unused prmCommonMeta
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m25s
Tests and linters / Lint (pull_request) Successful in 2m29s
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
51c3618850 [#121] client: Make PrmContainerList fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
665e5807bc [#188] transformer: Allow to provide size hint
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                │     out     │
                                                │   sec/op    │
Transformer/small/no_size_hint-8                  65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8     64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8   58.70µ ± 5%
Transformer/big/no_size_hint-8                    367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8       562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8     385.6m ± 7%
geomean                                           5.197m

                                                │     out      │
                                                │     B/op     │
Transformer/small/no_size_hint-8                  13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8     13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8   13.39Ki ± 0%
Transformer/big/no_size_hint-8                    288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8       1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8     288.0Mi ± 0%
geomean                                           2.533Mi

                                                │    out     │
                                                │ allocs/op  │
Transformer/small/no_size_hint-8                  92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8     92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8   92.00 ± 0%
Transformer/big/no_size_hint-8                    546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8       607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8     545.5 ± 0%
geomean                                           228.1
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-30 12:13:04 +00:00
a02c0bfac8 [#186] netmap: Marshal policy with brackets
All checks were successful
DCO / DCO (pull_request) Successful in 1m14s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m34s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m47s
Tests and linters / Lint (pull_request) Successful in 2m52s
Brackets can be semantically important and must not be omitted,
otherwise the output is plain wrong.
We do not take the responsibility to preserve every bracket, though,
because parser does some optimizations related to grouping long chains
of filters combined with the same operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-27 10:54:45 +03:00