TrueCloudLab/frostfs-sdk-go
33
0
Fork 21
Code Issues 34 Pull requests 1 Releases Activity Actions
769 commits 4 branches 0 tags 9.8 MiB
Commit graph

769 commits

This branch
This branch
All branches
Author SHA1 Message Date
Denis Kirillov
8999d2f080 [#191] pool/tree: Support request id 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-11-07 14:45:35 +03:00
aarifullin
6fbe1595cb [#121] pool: Refactor PrmObjectSearch usage
All checks were successful
DCO / DCO (pull_request) Successful in 1m8s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m24s
Details
Tests and linters / Lint (pull_request) Successful in 2m33s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-01 17:45:15 +03:00
aarifullin
a9237aabd2 [#121] client: Make PrmObjectSearch fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-01 17:45:06 +03:00
aarifullin
a487033505 [#121] client: Nuke out unused prmCommonMeta
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m25s
Details
Tests and linters / Lint (pull_request) Successful in 2m29s
Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-10-31 22:55:25 +03:00
aarifullin
51c3618850 [#121] client: Make PrmContainerList fields public 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-10-31 22:55:25 +03:00
Evgenii Stratonikov
665e5807bc [#188] transformer: Allow to provide size hint 
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                │     out     │
                                                │   sec/op    │
Transformer/small/no_size_hint-8                  65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8     64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8   58.70µ ± 5%
Transformer/big/no_size_hint-8                    367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8       562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8     385.6m ± 7%
geomean                                           5.197m

                                                │     out      │
                                                │     B/op     │
Transformer/small/no_size_hint-8                  13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8     13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8   13.39Ki ± 0%
Transformer/big/no_size_hint-8                    288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8       1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8     288.0Mi ± 0%
geomean                                           2.533Mi

                                                │    out     │
                                                │ allocs/op  │
Transformer/small/no_size_hint-8                  92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8     92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8   92.00 ± 0%
Transformer/big/no_size_hint-8                    546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8       607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8     545.5 ± 0%
geomean                                           228.1
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-30 12:13:04 +00:00
Evgenii Stratonikov
a02c0bfac8 [#186] netmap: Marshal policy with brackets
All checks were successful
DCO / DCO (pull_request) Successful in 1m14s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m34s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m47s
Details
Tests and linters / Lint (pull_request) Successful in 2m52s
Details 
Brackets can be semantically important and must not be omitted,
otherwise the output is plain wrong.
We do not take the responsibility to preserve every bracket, though,
because parser does some optimizations related to grouping long chains
of filters combined with the same operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-27 10:54:45 +03:00
Evgenii Stratonikov
20d325e307 [#167] netmap: Fix reverse min agregator 
The higher the price, the lower reverse min weight should be.
Previously nodes with 0 price had 0 weight which is a bit misleading.

Introduced in d71a0e0755.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-27 07:53:19 +00:00
Egor Olefirenko
670619d242 [#131] client: keep backwards-compatibility, update README.md, fix chore 
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
 2023-10-26 14:35:49 +00:00
Egor Olefirenko
0d79d10482 [#131] client: rename option consistently and fix test 
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
 2023-10-26 14:35:49 +00:00
Egor Olefirenko
9727beb47d [#131] client: Switch ResolveFrostFSFailures to DontResolveFrostFSFailures option 
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
 2023-10-26 14:35:49 +00:00
aarifullin
84315fab6a [#121] client: Make PrmBalanceGet fields public
Some checks failed
Tests and linters / Tests (1.21) (pull_request) Successful in 1m1s
Details
Tests and linters / Lint (pull_request) Successful in 2m29s
Details
DCO / DCO (pull_request) Successful in 4m13s
Details
Tests and linters / Tests (1.20) (pull_request) Failing after 5m47s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-23 18:53:14 +03:00
Dmitrii Stepanov
71335489ae [#183] forgejo: Make linter great again
All checks were successful
DCO / DCO (pull_request) Successful in 50s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m8s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m45s
Details
Tests and linters / Lint (pull_request) Successful in 12m49s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-23 18:13:40 +03:00
aarifullin
4c1feaf2cb [#182] pool: Fix linter error about deprecated methods
All checks were successful
DCO / DCO (pull_request) Successful in 1m11s
Details
Tests and linters / Lint (pull_request) Successful in 1m30s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m1s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-23 17:40:00 +03:00
Airat Arifullin
5804128ff3 [#121] client: Make PrmObjectPutSingle fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-19 08:38:13 +00:00
Denis Kirillov
abd38c918e [#177] pool: Support NetMapSnapshot method
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Details
Tests and linters / Lint (pull_request) Successful in 1m20s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m36s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-10-18 11:31:53 +03:00
Alex Vanin
fc4551b843 [#172] pool: Use priority of errors in tree pool
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m8s
Details
Tests and linters / Lint (pull_request) Successful in 1m33s
Details
DCO / DCO (pull_request) Successful in 5m8s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m2s
Details 
When retry happens, use priority map to decide
which error to return. Consider network errors
less desirable than business logic errors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-11 12:00:34 +03:00
Alex Vanin
eb5288f4a5 [#172] pool: Do more retries on unexpected tree service responses
All checks were successful
DCO / DCO (pull_request) Successful in 2m12s
Details
Tests and linters / Lint (pull_request) Successful in 3m33s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m35s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m3s
Details 
1. Try its best by looking for nodes during 'GetNodeByPath'
2. Retry on 'tree not found' and other not found errors

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-06 11:12:48 +03:00
Alex Vanin
60463871db [#171] pool: Add test for healthy status monitor
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m10s
Details
Tests and linters / Lint (pull_request) Successful in 1m31s
Details
DCO / DCO (pull_request) Successful in 4m12s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m9s
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-03 19:47:22 +03:00
Alex Vanin
8a04638749 [#171] pool: Close only dialed connections 
To avoid panics during close operation, close
only dialed connections.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-03 19:47:22 +03:00
Alex Vanin
ddbfb758c9 [#171] pool: Use dial status to close connections during restarts 
Every client restart, pool creates new client instance. If client
failed due to dial error, there was no prior connection and go
routine on a server side. If client failed due to communication
or business logic errors, then server side maintains connection and
client should close it to avoid routine and connection leak.

Dialing is a part of healthcheck, so health status is now a enum
of three values:
- unhealthy due to dial fail,
- unhealthy due to transmission fail,
- healthy.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-10-03 17:00:36 +03:00
andnilin@gmail.com
d71a0e0755 [#88] netmap: use bool, fix hrw_sort tests 
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
 2023-10-03 07:05:03 +00:00
AndrewDanilin
163b3e1961 [#88] netmap: fix min aggregator bug, add tests 
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
 2023-10-03 07:05:03 +00:00
aarifullin
84b9d29fc9 [#170] checksum: Use constant mapping for checksum types
All checks were successful
DCO / DCO (pull_request) Successful in 1m8s
Details
Tests and linters / Lint (pull_request) Successful in 1m22s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m43s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m39s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-28 17:20:24 +03:00
Alex Vanin
99c273f499 [#169] pool: Close inner pools during close routine
All checks were successful
DCO / DCO (pull_request) Successful in 1m9s
Details
Tests and linters / Lint (pull_request) Successful in 1m26s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m44s
Details 
Some apps do not reuse pool instance and expect that
`pool.Close()` free resources. But it didn't actually
close inner SDK clients, so it leads to goroutine leak
in storage.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-09-20 12:16:13 +03:00
Evgenii Stratonikov
555ccc63b2 [#167] netmap: Allow to select insufficient number of nodes
All checks were successful
DCO / DCO (pull_request) Successful in 57s
Details
Tests and linters / Lint (pull_request) Successful in 2m31s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 3m22s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m5s
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-15 14:47:54 +03:00
Evgenii Stratonikov
0550438b53 [#167] netmap/tests: Add replica to invalid tests 
Make sure we fail exactly because of the reason specified.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-15 14:33:44 +03:00
Evgenii Stratonikov
c899163860 [#167] netmap/tests: Add json file name to the test output 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-15 14:33:44 +03:00
Evgenii Stratonikov
ac8fc6d440 [#162] netmap: Allow to parse single unnamed selectors
All checks were successful
DCO / DCO (pull_request) Successful in 1m6s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m38s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m34s
Details
Tests and linters / Lint (pull_request) Successful in 5m27s
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-11 15:22:24 +03:00
Evgenii Stratonikov
0a0b590df3 [#162] Fix pre-commit warnings 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-11 15:21:35 +03:00
Evgenii Stratonikov
4df642e941 [#162] netmap: Fix possible panic 
Placement policy is unvalidated external input.
Under no circumstances should we panic here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-11 15:19:56 +03:00
Evgenii Stratonikov
8bc64e088e [#161] .golangci.yml: Reenable deprecated usage warnings
All checks were successful
DCO / DCO (pull_request) Successful in 1m12s
Details
Tests and linters / Lint (pull_request) Successful in 1m28s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m47s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m57s
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-08 17:17:02 +03:00
Evgenii Stratonikov
49ad985cad [#161] *: Do not use math/rand.Read() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-08 17:17:02 +03:00
Airat Arifullin
aa12d8c6a6 [#121] client: Make PrmObjectHash fields public 
* Introduce buildRequest for PrmObjectHash

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-08 13:36:45 +00:00
Airat Arifullin
303508328a [#121] pool: Refactor PrmSessionCreate usage
All checks were successful
Tests and linters / Lint (pull_request) Successful in 1m2s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m8s
Details
DCO / DCO (pull_request) Successful in 3m55s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m3s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-05 17:32:03 +03:00
Airat Arifullin
55699d1480 [#121] client: Make PrmSessionCreate fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-05 17:30:42 +03:00
Airat Arifullin
55a1f23e71 [#121] client: Make PrmEndpointInfo, PrmNetworkInfo fields public
All checks were successful
DCO / DCO (pull_request) Successful in 44s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m21s
Details
Tests and linters / Lint (pull_request) Successful in 3m11s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m29s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-04 19:55:23 +03:00
Airat Arifullin
291a71ba84 [#121] client: Make PrmAnnounceSpace fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-04 19:53:34 +03:00
Airat Arifullin
5a471e5002 [#121] client: Make PrmObjectDelete fields public 
* Introduce buildRequest for PrmObjectDelete
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-09-04 14:14:22 +00:00
Anton Nikiforov
b5fe52d6bd [#150] policy: Check for redundant selectors and filters
All checks were successful
DCO / DCO (pull_request) Successful in 43s
Details
Tests and linters / Lint (pull_request) Successful in 1m17s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m21s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m22s
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-29 14:16:57 +03:00
Airat Arifullin
84e7e69f98 [#121] client: Make PrmObjectGet/Head/GetRange fields public
All checks were successful
DCO / DCO (pull_request) Successful in 45s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m10s
Details
Tests and linters / Lint (pull_request) Successful in 2m13s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m49s
Details 
* Remove common PrmObjectRead structure
* Introduce buildRequest for PrmObjectGet/Head/GetRange
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-28 11:26:57 +03:00
Denis Kirillov
46a214d065 [#149] pool: Configure homomorphic hash and buffer size
All checks were successful
DCO / DCO (pull_request) Successful in 2m39s
Details
Tests and linters / Lint (pull_request) Successful in 3m23s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 4m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m25s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-25 09:45:15 +03:00
Denis Kirillov
202412230a [#115] pool: Drop part buffer pool
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m6s
Details
Tests and linters / Lint (pull_request) Successful in 1m28s
Details
DCO / DCO (pull_request) Successful in 4m29s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m32s
Details 
Tests showed that using part buffer pool doesn't save memory a lot.
Especially on big parts.
Probably we can use pool only for small parts
after adding buffer in payloadSizeLimiter

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-21 12:03:03 +03:00
Denis Kirillov
3cb3841073 [#115] pool: Try putSingle if possible 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-21 12:03:03 +03:00
Denis Kirillov
faeeeab87a [#114] pool: Don't use part buffers when client cut is off 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-21 12:02:40 +03:00
Denis Kirillov
cae215534f [#114] pool: Fix linter errors 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-21 12:02:40 +03:00
Denis Kirillov
518fb79bc0 [#114] pool: Support client cut with memory limiter 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-08-21 12:02:40 +03:00
Airat Arifullin
342524159a [#121] pool: Make PrmContainerSetEACL fields public
All checks were successful
DCO / DCO (pull_request) Successful in 54s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m18s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m28s
Details
Tests and linters / Lint (pull_request) Successful in 3m7s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-21 10:33:19 +03:00
Airat Arifullin
22978303f8 [#121] clientt: Make PrmContainerSetEACL fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-18 18:05:47 +03:00
Airat Arifullin
6fdbe75517 [#121] pool: Make PrmContainerEACL fields public 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-16 09:53:47 +00:00