a-savchuk/frostfs-sdk-go
1
0
Fork 0
forked from TrueCloudLab/frostfs-sdk-go
Code Pull requests Activity
763 commits 8 branches 0 tags 6.9 MiB
Commit graph

763 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dmitrii Stepanov
1dc3b77ac7 [#270] pool: Replace deprecated DialContext 
`Healthcheck` request performed after client creation, so no extra RPC required.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-16 15:09:59 +03:00
Dmitrii Stepanov
88c6556c37 [#270] go.mod: Upgrade google.golang.org/grpc 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-16 15:09:54 +03:00
Airat Arifullin
d342c0bc16 [#268] client: Make PayloadPatch correctly receive empty patch payload 
* Make the method `PatchPayload` send a patch with empty
  payload patch if range's length is non-zero and if it's the
  first call.
* Empty payload patches just cut original object payload. So, these
  patches are also valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-09-12 15:12:52 +03:00
Airat Arifullin
f0c599d06d [#268] client: Fix sequential PayloadPatch calls 
* The flag 'firstPayloadPatch' keeps its state after first
  `PatchPayload` that make other calls incorrectly set patch
  ranges.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-09-12 15:09:01 +03:00
Aleksey Savchuk
7d84d104fb [#260] *: Fix linter warnings 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:55:32 +00:00
Aleksey Savchuk
812126a8ff [#260] .golangci.yml: Add protogetter linter 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:55:32 +00:00
Aleksey Savchuk
d86223ed56 [#260] Makefile: Add pre-commit targets 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:55:32 +00:00
Aleksey Savchuk
76a0cfdadb [#217] netmap: Return node netmap state directly 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:51:51 +00:00
Ekaterina Lebedeva
46ee543899 [#265] go.mod: Use range over int 
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-09-04 12:37:46 +03:00
Aleksey Savchuk
8f751d9dd0
[#263] go.mod: Update api-go 
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-03 12:36:28 +03:00
Nikita Zinkevich
3c00f4eeac [#203] Add pool docs 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-08-30 08:24:31 +03:00
Evgenii Stratonikov
f0b9493ce3 [#261] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-27 11:33:09 +03:00
Denis Kirillov
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout 
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-22 08:02:51 +00:00
Ekaterina Lebedeva
9115d3f281 [#1316] lint: Fix warnings 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-08-21 17:59:10 +03:00
Ekaterina Lebedeva
cf225be0df [#1316] go.mod: Bump go version to 1.22 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-08-21 16:42:58 +03:00
Denis Kirillov
338d1ef254 [#258] pool/tree: Add node address to error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-20 14:19:58 +03:00
Evgenii Stratonikov
6dd7be11d1 [#256] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-20 10:20:28 +03:00
Denis Kirillov
203bba65a0 [#253] pool: Don't count regular FrostFS errors 
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-14 16:48:51 +03:00
Airat Arifullin
98aabc45a7 [#252] patcher: Fix applying patch from the same offset 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-13 18:58:21 +03:00
Airat Arifullin
908c96a94d [#251] pool: Fix handlerError panic for objectPatch 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-13 15:28:39 +03:00
Anton Nikiforov
2077b35736 [#231] netmap: Add LIKE operation for filter 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-08-12 09:49:12 +03:00
Anton Nikiforov
92c7596157 [#231] go.mod: Update version for frostfs-api-go/v2 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-08-12 09:49:04 +03:00
Anton Nikiforov
a15b1264f5 [#231] policy: Bump jre version to 17 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-08-12 09:44:30 +03:00
Airat Arifullin
5d58519253 [#249] pool: Introduce objectPatch method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-08 18:26:40 +03:00
Airat Arifullin
93171b3319 [#249] session: Support patch verb 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-08 18:26:40 +03:00
Airat Arifullin
3ba7446157 [#249] client: Introduce ObjectPatch 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-08 18:26:35 +03:00
Airat Arifullin
335aa18dc6 [#249] go.mod: Bump frostfs-api-go/v2 version 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-06 16:57:42 +03:00
Airat Arifullin
361739e860 [#247] object: Introduce patcher package 
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-05 12:32:14 +00:00
Airat Arifullin
6dd500def9 [#247] object: Introduce Patch type 
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-05 12:32:14 +00:00
Marina Biryukova
e83d6b7c6a [#244] pool/tree: Collect request duration statistic 
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-02 13:01:14 +03:00
Evgenii Stratonikov
9da46f566f [#243] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-26 14:13:49 +03:00
Marina Biryukova
fa89999d91 [#242] pool: Log error that caused healthy status change 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-22 15:12:27 +03:00
Marina Biryukova
7e94a6adf2 [#237] pool: Return creation epoch from object put 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-22 06:15:23 +00:00
Denis Kirillov
ce8270568d [#239] go.mod: Update frostfs-contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-18 17:17:40 +03:00
Denis Kirillov
7c06cdff2d [#239] pool/tree: Update tree service client 
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-17 14:32:04 +03:00
Denis Kirillov
e18b916231 [#238] go.mod: Update frostfs-contract to v0.19.3 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-16 11:36:21 +03:00
Evgenii Stratonikov
c4ff8a6cda [#236] go.mod: Update neo-go to v0.106.2 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 15:43:11 +03:00
Evgenii Stratonikov
fc7c524fcb [#236] *: Replace slice.Copy() with bytes.Clone() 
The former is deprecated.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 15:43:07 +03:00
Evgenii Stratonikov
e977b8a94c [#236] netmap: Remove unused field from meanIQRAgg 
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00
Evgenii Stratonikov
6729f54c4e [#236] netmap: Reuse slice for weights in ContainerNodes() 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00
Evgenii Stratonikov
159a50fcf0 [#236] netmap: Reduce allocations in getSelection() 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00
Evgenii Stratonikov
a69f00903c [#236] netmap: Replace sort.Slice() with slices.Sort() 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   9.227µ ± 13%  -11.24% (p=0.015 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   9.189µ ±  7%        ~ (p=0.105 n=10)
geomean                                                          10.25µ         9.208µ        -10.18%

                                                              │     old      │                 new                 │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   8.320Ki ± 0%  -4.31% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.742Ki ± 0%  -4.62% (p=0.000 n=10)
geomean                                                         8.401Ki        8.026Ki       -4.47%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
geomean                                                         138.0        122.0       -11.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00
Evgenii Stratonikov
9d89f08c7b [#236] go.mod: Bump min go version to go1.21 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:06 +03:00
Alexander Chuprov
51cefd4908 [#232] netmap: Allow empty values for unknown parameters in network config 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-07-09 12:19:25 +03:00
Marina Biryukova
560cbbd1f1 [#234] pool: Update token expiration check in cache 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-05 12:36:17 +03:00
Airat Arifullin
27e965007d [#233] pool: Introduce ape-manager methods 
* Remove GetEACL, SetEACL methods as they are deprecated;
* Fix mock;
* Introduce add/remove/list methods to request ape-manager from
  the client.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-07-04 01:29:35 +03:00
Ekaterina Lebedeva
1a5886e776 [#228] client: Move isClientErrMaintenance from node 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-06-17 17:07:30 +03:00
Airat Arifullin
ebd8fcd168 [#224] object: Introduce parent attributes in EC-header 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 16:20:48 +03:00
Airat Arifullin
717a7d00ef [#225] bearer: Introduce APEOverride field for the token 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 12:14:42 +00:00
Airat Arifullin
dd23c6fd2b [#225] apemanager: Move apemanager to ape package 
* Update go.mod;
* Fix packages;
* Fix client's `apemanager` methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 12:14:42 +00:00