alexvanin/frostfs-sdk-go
1
0
Fork 0
forked from TrueCloudLab/frostfs-sdk-go
Code Issues Pull requests Projects Releases Packages Wiki Activity
748 commits 3 branches 0 tags 5.1 MiB
Commit graph

748 commits

This branch
This branch
All branches
Author SHA1 Message Date
aarifullin
6d0da3f861 [#211] go.mod: Update frostfs-api-go version 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-29 13:48:04 +03:00
Alexander Chuprov
1af9b6d18b [#155] sdk-go: Add buffer support for payloadSizeLimiter 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-03-25 10:47:04 +03:00
Evgenii Stratonikov
bd2d350b09 [#205] object: Initial EC implementation 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-22 10:14:12 +00:00
Evgenii Stratonikov
e9be3e6d94 [#205] netmap: Add well-known EC parameters to network config 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-22 10:14:12 +00:00
Evgenii Stratonikov
70e9e40c7f [#205] netmap: Add EC statement to placement policy 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-22 10:14:12 +00:00
Evgenii Stratonikov
d33b54d280 [#205] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-22 10:14:12 +00:00
Alexander Chuprov
6f248436a5 [#210] client/put_transformer: Fix error handling 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-03-19 15:36:51 +03:00
Evgenii Stratonikov
edd40474e8 [#209] pre-commit: Add unit-test hook 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-14 13:35:48 +03:00
Evgenii Stratonikov
d9ec7c1988 [#209] Makefile: Allow to override test flags 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-14 13:35:48 +03:00
Evgenii Stratonikov
64b83f8220 [#209] Makefile: Update golangci-lint to 1.56.2 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-14 13:35:48 +03:00
Evgenii Stratonikov
7212f38115 [#209] pre-commit: Remove gitlint 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-14 13:35:48 +03:00
Dmitrii Stepanov
8081445ff2 [#208] go.mod: Bump frostfs-api-go version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-06 13:34:04 +03:00
Dmitrii Stepanov
6a7ef9d8c3 [#208] go.mod: Bump protobuf version 
Found by vulncheck:
Vulnerability #1: GO-2024-2611
    Infinite loop in JSON unmarshaling in google.golang.org/protobuf
  More info: https://pkg.go.dev/vuln/GO-2024-2611
  Module: google.golang.org/protobuf
    Found in: google.golang.org/protobuf@v1.32.0
    Fixed in: google.golang.org/protobuf@v1.33.0

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-06 13:23:32 +03:00
Evgenii Stratonikov
6fe4e2541d [#207] netmap: Fix string escape in PlacementPolicy.String() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-01 15:02:05 +00:00
Denis Kirillov
a5fab572ff [#206] Add session tokens for container read operations 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-03-01 15:18:53 +03:00
Dmitrii Stepanov
a86170f53a [#202] object: Reset marshal data on CutPayload 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-15 15:23:47 +03:00
Dmitrii Stepanov
aa41f71dcc [#202] eacl: Drop storage group test 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-15 15:23:47 +03:00
Dmitrii Stepanov
3a00fd51e4 [#202] go.mod: Update api-go version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-15 15:23:43 +03:00
Alexander Chuprov
65b4525b3b [#198] object/user: Add ScriptHash method 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-01-26 14:10:09 +00:00
Evgenii Stratonikov
7efff9d53d [#201] .forgejo: Update dco-go to v3 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-01-26 12:21:42 +03:00
Alexander Chuprov
110b7e4170 [#200] pre-commit: Fix linter invocation target 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-01-17 17:56:20 +03:00
Dmitrii Stepanov
56debcfa56 [#190] sdk-go: Gofumpt fixes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-22 19:21:20 +03:00
Dmitrii Stepanov
157a9930e8 [#190] sdk-go: Pass user.ID by value 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-22 19:21:20 +03:00
Alexander Chuprov
1c07098740 [#194] make: Fix make test 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-17 11:48:51 +00:00
Alexander Chuprov
03d35dd1f3 [#166] netmap: Add support YML tests 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-17 14:43:03 +03:00
Alexander Chuprov
dea8759762 [#166] netmap: Move tests from JSON to YML 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-17 14:42:39 +03:00
aarifullin
3787477133 [#189] client: Make PrmDial fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-14 11:18:00 +03:00
aarifullin
e91d40e250 [#189] client: Make PrmInit fields public for client 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-13 15:18:07 +03:00
Denis Kirillov
ab75edd709 [#191] pool/tree: Support limit request attempts 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-07 14:45:40 +03:00
Denis Kirillov
8999d2f080 [#191] pool/tree: Support request id 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-07 14:45:35 +03:00
aarifullin
6fbe1595cb [#121] pool: Refactor PrmObjectSearch usage 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-01 17:45:15 +03:00
aarifullin
a9237aabd2 [#121] client: Make PrmObjectSearch fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-01 17:45:06 +03:00
aarifullin
a487033505 [#121] client: Nuke out unused prmCommonMeta 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-10-31 22:55:25 +03:00
aarifullin
51c3618850 [#121] client: Make PrmContainerList fields public 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-10-31 22:55:25 +03:00
Evgenii Stratonikov
665e5807bc [#188] transformer: Allow to provide size hint 
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                │     out     │
                                                │   sec/op    │
Transformer/small/no_size_hint-8                  65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8     64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8   58.70µ ± 5%
Transformer/big/no_size_hint-8                    367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8       562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8     385.6m ± 7%
geomean                                           5.197m

                                                │     out      │
                                                │     B/op     │
Transformer/small/no_size_hint-8                  13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8     13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8   13.39Ki ± 0%
Transformer/big/no_size_hint-8                    288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8       1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8     288.0Mi ± 0%
geomean                                           2.533Mi

                                                │    out     │
                                                │ allocs/op  │
Transformer/small/no_size_hint-8                  92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8     92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8   92.00 ± 0%
Transformer/big/no_size_hint-8                    546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8       607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8     545.5 ± 0%
geomean                                           228.1
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-30 12:13:04 +00:00
Evgenii Stratonikov
a02c0bfac8 [#186] netmap: Marshal policy with brackets 
Brackets can be semantically important and must not be omitted,
otherwise the output is plain wrong.
We do not take the responsibility to preserve every bracket, though,
because parser does some optimizations related to grouping long chains
of filters combined with the same operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-27 10:54:45 +03:00
Evgenii Stratonikov
20d325e307 [#167] netmap: Fix reverse min agregator 
The higher the price, the lower reverse min weight should be.
Previously nodes with 0 price had 0 weight which is a bit misleading.

Introduced in d71a0e0755.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-27 07:53:19 +00:00
Egor Olefirenko
670619d242 [#131] client: keep backwards-compatibility, update README.md, fix chore 
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
 2023-10-26 14:35:49 +00:00
Egor Olefirenko
0d79d10482 [#131] client: rename option consistently and fix test 
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
 2023-10-26 14:35:49 +00:00
Egor Olefirenko
9727beb47d [#131] client: Switch ResolveFrostFSFailures to DontResolveFrostFSFailures option 
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
 2023-10-26 14:35:49 +00:00
aarifullin
84315fab6a [#121] client: Make PrmBalanceGet fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-23 18:53:14 +03:00
Dmitrii Stepanov
71335489ae [#183] forgejo: Make linter great again 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-23 18:13:40 +03:00
aarifullin
4c1feaf2cb [#182] pool: Fix linter error about deprecated methods 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-23 17:40:00 +03:00
Airat Arifullin
5804128ff3 [#121] client: Make PrmObjectPutSingle fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-19 08:38:13 +00:00
Denis Kirillov
abd38c918e [#177] pool: Support NetMapSnapshot method 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-18 11:31:53 +03:00
Alex Vanin
fc4551b843 [#172] pool: Use priority of errors in tree pool 
When retry happens, use priority map to decide
which error to return. Consider network errors
less desirable than business logic errors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-11 12:00:34 +03:00
Alex Vanin
eb5288f4a5 [#172] pool: Do more retries on unexpected tree service responses 
1. Try its best by looking for nodes during 'GetNodeByPath'
2. Retry on 'tree not found' and other not found errors

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-06 11:12:48 +03:00
Alex Vanin
60463871db [#171] pool: Add test for healthy status monitor 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-03 19:47:22 +03:00
Alex Vanin
8a04638749 [#171] pool: Close only dialed connections 
To avoid panics during close operation, close
only dialed connections.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-03 19:47:22 +03:00
Alex Vanin
ddbfb758c9 [#171] pool: Use dial status to close connections during restarts 
Every client restart, pool creates new client instance. If client
failed due to dial error, there was no prior connection and go
routine on a server side. If client failed due to communication
or business logic errors, then server side maintains connection and
client should close it to avoid routine and connection leak.

Dialing is a part of healthcheck, so health status is now a enum
of three values:
- unhealthy due to dial fail,
- unhealthy due to transmission fail,
- healthy.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-03 17:00:36 +03:00