Commit graph

767 commits

Author SHA1 Message Date
e580ee991d [#271] Drop handling of system attributes with NeoFS prefix
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-18 09:59:38 +00:00
6821fe6fb2 [#271] object: Add UserAttributes method
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-18 09:59:38 +00:00
6009d089fc [#270] client: Use RPC call instead of Dial
After api-go upgrade created client doesn't establish connection after created,
so RPC call is required to establish and check connection.
RPC call returns status error, so conversion from status error to context error
is required to satisfy Dial contract and unit tests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:31:51 +03:00
3e455777fd [#270] go.mod: Upgrade api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:10:02 +03:00
1dc3b77ac7 [#270] pool: Replace deprecated DialContext
`Healthcheck` request performed after client creation, so no extra RPC required.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:09:59 +03:00
88c6556c37 [#270] go.mod: Upgrade google.golang.org/grpc
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:09:54 +03:00
d342c0bc16 [#268] client: Make PayloadPatch correctly receive empty patch payload
* Make the method `PatchPayload` send a patch with empty
  payload patch if range's length is non-zero and if it's the
  first call.
* Empty payload patches just cut original object payload. So, these
  patches are also valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:12:52 +03:00
f0c599d06d [#268] client: Fix sequential PayloadPatch calls
* The flag 'firstPayloadPatch' keeps its state after first
  `PatchPayload` that make other calls incorrectly set patch
  ranges.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:09:01 +03:00
7d84d104fb [#260] *: Fix linter warnings
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
812126a8ff [#260] .golangci.yml: Add protogetter linter
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
d86223ed56 [#260] Makefile: Add pre-commit targets
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
76a0cfdadb [#217] netmap: Return node netmap state directly
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:51:51 +00:00
46ee543899 [#265] go.mod: Use range over int
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
8f751d9dd0
[#263] go.mod: Update api-go
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-03 12:36:28 +03:00
3c00f4eeac [#203] Add pool docs
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-08-30 08:24:31 +03:00
f0b9493ce3 [#261] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:33:09 +03:00
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-22 08:02:51 +00:00
9115d3f281 [#1316] lint: Fix warnings
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 17:59:10 +03:00
cf225be0df [#1316] go.mod: Bump go version to 1.22
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 16:42:58 +03:00
338d1ef254 [#258] pool/tree: Add node address to error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-20 14:19:58 +03:00
6dd7be11d1 [#256] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-20 10:20:28 +03:00
203bba65a0 [#253] pool: Don't count regular FrostFS errors
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-14 16:48:51 +03:00
98aabc45a7 [#252] patcher: Fix applying patch from the same offset
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 18:58:21 +03:00
908c96a94d [#251] pool: Fix handlerError panic for objectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 15:28:39 +03:00
2077b35736 [#231] netmap: Add LIKE operation for filter
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:12 +03:00
92c7596157 [#231] go.mod: Update version for frostfs-api-go/v2
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:04 +03:00
a15b1264f5 [#231] policy: Bump jre version to 17
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:44:30 +03:00
5d58519253 [#249] pool: Introduce objectPatch method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
93171b3319 [#249] session: Support patch verb
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
3ba7446157 [#249] client: Introduce ObjectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:35 +03:00
335aa18dc6 [#249] go.mod: Bump frostfs-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-06 16:57:42 +03:00
361739e860 [#247] object: Introduce patcher package
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
6dd500def9 [#247] object: Introduce Patch type
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
e83d6b7c6a [#244] pool/tree: Collect request duration statistic
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-02 13:01:14 +03:00
9da46f566f [#243] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 14:13:49 +03:00
fa89999d91 [#242] pool: Log error that caused healthy status change
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 15:12:27 +03:00
7e94a6adf2 [#237] pool: Return creation epoch from object put
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
ce8270568d [#239] go.mod: Update frostfs-contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-18 17:17:40 +03:00
7c06cdff2d [#239] pool/tree: Update tree service client
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-17 14:32:04 +03:00
e18b916231 [#238] go.mod: Update frostfs-contract to v0.19.3
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-16 11:36:21 +03:00
c4ff8a6cda [#236] go.mod: Update neo-go to v0.106.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:11 +03:00
fc7c524fcb [#236] *: Replace slice.Copy() with bytes.Clone()
The former is deprecated.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:07 +03:00
e977b8a94c [#236] netmap: Remove unused field from meanIQRAgg
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
6729f54c4e [#236] netmap: Reuse slice for weights in ContainerNodes()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
159a50fcf0 [#236] netmap: Reduce allocations in getSelection()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
a69f00903c [#236] netmap: Replace sort.Slice() with slices.Sort()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   9.227µ ± 13%  -11.24% (p=0.015 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   9.189µ ±  7%        ~ (p=0.105 n=10)
geomean                                                          10.25µ         9.208µ        -10.18%

                                                              │     old      │                 new                 │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   8.320Ki ± 0%  -4.31% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.742Ki ± 0%  -4.62% (p=0.000 n=10)
geomean                                                         8.401Ki        8.026Ki       -4.47%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
geomean                                                         138.0        122.0       -11.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
9d89f08c7b [#236] go.mod: Bump min go version to go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:06 +03:00
51cefd4908 [#232] netmap: Allow empty values for unknown parameters in network config
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-09 12:19:25 +03:00
560cbbd1f1 [#234] pool: Update token expiration check in cache
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-05 12:36:17 +03:00
27e965007d [#233] pool: Introduce ape-manager methods
* Remove GetEACL, SetEACL methods as they are deprecated;
* Fix mock;
* Introduce add/remove/list methods to request ape-manager from
  the client.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-04 01:29:35 +03:00