aarifullin/frostfs-sdk-go
1
0
Fork 0
forked from TrueCloudLab/frostfs-sdk-go
Code Pull requests Activity
772 commits 44 branches 0 tags 7.8 MiB
Commit graph

772 commits

This branch
This branch
All branches
Author SHA1 Message Date
Aleksey Savchuk
e50838a33d [#269] netmap: Regenerate policy parser 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-20 11:09:16 +00:00
Aleksey Savchuk
97cf56ba41 [#269] netmap: Support non-ascii attributes in SELECT IN 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-20 11:09:16 +00:00
Aleksey Savchuk
07625e3bd1 [#269] Update ANTLR version 4.13.0 -> 4.13.1 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-20 11:09:16 +00:00
Aleksey Savchuk
da2f0e7532 [#269] .gitignore: Ignore ANTLR jar file 
The previous wildcard failed to properly match the ANTLR jar file.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-20 11:09:16 +00:00
Aleksey Savchuk
114b4c14b5 [#269] Makefile: Update policy target 
The previous policy target generated device-specific comments
(e.g., `/home/john_doe/repos/<...>`), which could result in
unnecessary file changes This behavior would confuse git and
require manual changes to resolve. The update ensures comments
are now device-agnostic, preventing unwanted changes.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-20 11:09:16 +00:00
Aleksey Savchuk
e580ee991d [#271] Drop handling of system attributes with NeoFS prefix 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-18 09:59:38 +00:00
Aleksey Savchuk
6821fe6fb2 [#271] object: Add UserAttributes method 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-18 09:59:38 +00:00
Dmitrii Stepanov
6009d089fc [#270] client: Use RPC call instead of Dial 
After api-go upgrade created client doesn't establish connection after created,
so RPC call is required to establish and check connection.
RPC call returns status error, so conversion from status error to context error
is required to satisfy Dial contract and unit tests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-16 15:31:51 +03:00
Dmitrii Stepanov
3e455777fd [#270] go.mod: Upgrade api-go version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-16 15:10:02 +03:00
Dmitrii Stepanov
1dc3b77ac7 [#270] pool: Replace deprecated DialContext 
`Healthcheck` request performed after client creation, so no extra RPC required.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-16 15:09:59 +03:00
Dmitrii Stepanov
88c6556c37 [#270] go.mod: Upgrade google.golang.org/grpc 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-16 15:09:54 +03:00
Airat Arifullin
d342c0bc16 [#268] client: Make PayloadPatch correctly receive empty patch payload 
* Make the method `PatchPayload` send a patch with empty
  payload patch if range's length is non-zero and if it's the
  first call.
* Empty payload patches just cut original object payload. So, these
  patches are also valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-09-12 15:12:52 +03:00
Airat Arifullin
f0c599d06d [#268] client: Fix sequential PayloadPatch calls 
* The flag 'firstPayloadPatch' keeps its state after first
  `PatchPayload` that make other calls incorrectly set patch
  ranges.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-09-12 15:09:01 +03:00
Aleksey Savchuk
7d84d104fb [#260] *: Fix linter warnings 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:55:32 +00:00
Aleksey Savchuk
812126a8ff [#260] .golangci.yml: Add protogetter linter 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:55:32 +00:00
Aleksey Savchuk
d86223ed56 [#260] Makefile: Add pre-commit targets 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:55:32 +00:00
Aleksey Savchuk
76a0cfdadb [#217] netmap: Return node netmap state directly 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-09 08:51:51 +00:00
Ekaterina Lebedeva
46ee543899 [#265] go.mod: Use range over int 
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-09-04 12:37:46 +03:00
Aleksey Savchuk
8f751d9dd0
[#263] go.mod: Update api-go 
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-09-03 12:36:28 +03:00
Nikita Zinkevich
3c00f4eeac [#203] Add pool docs 
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
 2024-08-30 08:24:31 +03:00
Evgenii Stratonikov
f0b9493ce3 [#261] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-27 11:33:09 +03:00
Denis Kirillov
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout 
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-22 08:02:51 +00:00
Ekaterina Lebedeva
9115d3f281 [#1316] lint: Fix warnings 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-08-21 17:59:10 +03:00
Ekaterina Lebedeva
cf225be0df [#1316] go.mod: Bump go version to 1.22 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-08-21 16:42:58 +03:00
Denis Kirillov
338d1ef254 [#258] pool/tree: Add node address to error 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-20 14:19:58 +03:00
Evgenii Stratonikov
6dd7be11d1 [#256] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-20 10:20:28 +03:00
Denis Kirillov
203bba65a0 [#253] pool: Don't count regular FrostFS errors 
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-08-14 16:48:51 +03:00
Airat Arifullin
98aabc45a7 [#252] patcher: Fix applying patch from the same offset 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-13 18:58:21 +03:00
Airat Arifullin
908c96a94d [#251] pool: Fix handlerError panic for objectPatch 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-13 15:28:39 +03:00
Anton Nikiforov
2077b35736 [#231] netmap: Add LIKE operation for filter 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-08-12 09:49:12 +03:00
Anton Nikiforov
92c7596157 [#231] go.mod: Update version for frostfs-api-go/v2 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-08-12 09:49:04 +03:00
Anton Nikiforov
a15b1264f5 [#231] policy: Bump jre version to 17 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-08-12 09:44:30 +03:00
Airat Arifullin
5d58519253 [#249] pool: Introduce objectPatch method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-08 18:26:40 +03:00
Airat Arifullin
93171b3319 [#249] session: Support patch verb 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-08 18:26:40 +03:00
Airat Arifullin
3ba7446157 [#249] client: Introduce ObjectPatch 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-08 18:26:35 +03:00
Airat Arifullin
335aa18dc6 [#249] go.mod: Bump frostfs-api-go/v2 version 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-06 16:57:42 +03:00
Airat Arifullin
361739e860 [#247] object: Introduce patcher package 
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-05 12:32:14 +00:00
Airat Arifullin
6dd500def9 [#247] object: Introduce Patch type 
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-05 12:32:14 +00:00
Marina Biryukova
e83d6b7c6a [#244] pool/tree: Collect request duration statistic 
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-08-02 13:01:14 +03:00
Evgenii Stratonikov
9da46f566f [#243] go.mod: Update api-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-26 14:13:49 +03:00
Marina Biryukova
fa89999d91 [#242] pool: Log error that caused healthy status change 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-22 15:12:27 +03:00
Marina Biryukova
7e94a6adf2 [#237] pool: Return creation epoch from object put 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-22 06:15:23 +00:00
Denis Kirillov
ce8270568d [#239] go.mod: Update frostfs-contract 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-18 17:17:40 +03:00
Denis Kirillov
7c06cdff2d [#239] pool/tree: Update tree service client 
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-17 14:32:04 +03:00
Denis Kirillov
e18b916231 [#238] go.mod: Update frostfs-contract to v0.19.3 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-07-16 11:36:21 +03:00
Evgenii Stratonikov
c4ff8a6cda [#236] go.mod: Update neo-go to v0.106.2 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 15:43:11 +03:00
Evgenii Stratonikov
fc7c524fcb [#236] *: Replace slice.Copy() with bytes.Clone() 
The former is deprecated.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 15:43:07 +03:00
Evgenii Stratonikov
e977b8a94c [#236] netmap: Remove unused field from meanIQRAgg 
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00
Evgenii Stratonikov
6729f54c4e [#236] netmap: Reuse slice for weights in ContainerNodes() 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00
Evgenii Stratonikov
159a50fcf0 [#236] netmap: Reduce allocations in getSelection() 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-12 14:25:07 +03:00