8088063195
[ #787 ] netmap: Refactor NewEpoch method
...
DCO action / DCO (pull_request) Successful in 3m58s
Vulncheck / Vulncheck (pull_request) Successful in 4m51s
Build / Build Components (1.21) (pull_request) Successful in 5m47s
Build / Build Components (1.20) (pull_request) Successful in 5m57s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m25s
Tests and linters / Staticcheck (pull_request) Successful in 6m31s
Tests and linters / Lint (pull_request) Successful in 7m15s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m16s
Tests and linters / Tests with -race (pull_request) Successful in 8m30s
Split for user and control methods.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:22:31 +03:00
c8a62ffedd
[ #787 ] morph: Calculate VUB and nonce when hash is nil
...
DCO action / DCO (pull_request) Successful in 6m41s
Vulncheck / Vulncheck (pull_request) Successful in 7m54s
Build / Build Components (1.21) (pull_request) Successful in 9m29s
Build / Build Components (1.20) (pull_request) Successful in 9m46s
Tests and linters / Staticcheck (pull_request) Successful in 12m42s
Tests and linters / Tests (1.20) (pull_request) Successful in 13m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m50s
Tests and linters / Tests with -race (pull_request) Successful in 14m26s
Tests and linters / Lint (pull_request) Successful in 15m21s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
2393d13e4d
[ #787 ] morph: Return VUB for IR service calls
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
518f3baf41
[ #787 ] morph: Return VUB from Invoke method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
bdfa523487
[ #787 ] proto: Add VUB field for IR service
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
0f75e48138
[ #796 ] policer: Fix tombstone objects replication
...
Tombstone objects must be replicated to all container nodes.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 13:39:33 +03:00
7cdae4f660
[ #792 ] proto: Regenerate with fixed version
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:09:13 +00:00
9133b4389e
[ #788 ] objectsvc: Fix formatting (gofumpt)
...
DCO action / DCO (pull_request) Successful in 3m19s
Vulncheck / Vulncheck (pull_request) Successful in 3m40s
Build / Build Components (1.21) (pull_request) Successful in 4m17s
Build / Build Components (1.20) (pull_request) Successful in 4m32s
Tests and linters / Staticcheck (pull_request) Successful in 4m46s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 5m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m24s
Tests and linters / Tests with -race (pull_request) Successful in 7m38s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:27:32 +03:00
1b22801eed
[ #788 ] engine: Fix flaky tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:25:46 +03:00
3534d6d05b
[ #794 ] objectsvc: Return accidentally removed acl checks for Head
...
DCO action / DCO (pull_request) Successful in 1m42s
Vulncheck / Vulncheck (pull_request) Successful in 3m23s
Build / Build Components (1.21) (pull_request) Successful in 4m22s
Build / Build Components (1.20) (pull_request) Successful in 5m44s
Tests and linters / Staticcheck (pull_request) Successful in 6m3s
Tests and linters / Lint (pull_request) Successful in 6m35s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m32s
Tests and linters / Tests with -race (pull_request) Successful in 8m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m54s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 17:13:58 +03:00
66848d3288
[ #770 ] cli: Add methods to work with APE rules via control svc
...
* Add methods to frostfs-cli
* Implement rpc in control service
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
8e11ef46b8
[ #770 ] object: Introduce ape chain checker for object svc
...
* Introduce Request type converted from RequestInfo type
to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
permitted to be performed
* Make put handlers use APE checker instead EACL
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
5ec73fe8a0
[ #770 ] node: Introduce ape chain source
...
* Provide methods to access rule chains with access
policy engine (APE) chain source
* Initialize apeChainSource within object service
initialization
* Share apeChainSource with control service
* Implement dummy apeChainSource instance based on
in-memory implementation
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
3a2c319b87
[ #770 ] control: Generate gRPC methods to manipulate APE chains
...
* Define new types and gRPC methods to manipulate APE chains
in control service.
* Stub gRPC handlers for the generated methods.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
70ab1ebd54
[ #763 ] metrics: Add container_objects_total metric
...
DCO action / DCO (pull_request) Successful in 3m54s
Build / Build Components (1.20) (pull_request) Successful in 4m58s
Build / Build Components (1.21) (pull_request) Successful in 5m16s
Vulncheck / Vulncheck (pull_request) Successful in 9m54s
Tests and linters / Lint (pull_request) Successful in 10m57s
Tests and linters / Tests (1.21) (pull_request) Successful in 12m40s
Tests and linters / Staticcheck (pull_request) Successful in 12m34s
Tests and linters / Tests with -race (pull_request) Successful in 12m48s
Tests and linters / Tests (1.20) (pull_request) Successful in 13m19s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
9c98fa6152
[ #763 ] metabase: Add container objects counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
226e84d782
[ #684 ] node: Add skipped objects count to evacuation result
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
523fb3ca51
[ #684 ] proto: Add skipped count to evacuation status response
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
74c91eeef5
[ #777 ] client: Refactor PrmContainerList, PrmObjectSearch usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-06 06:50:11 +00:00
20d6132f31
[ #531 ] signSvc: Add SetMarshaledData method call
...
Vulncheck / Vulncheck (pull_request) Successful in 1m25s
DCO action / DCO (pull_request) Successful in 1m36s
Build / Build Components (1.21) (pull_request) Successful in 2m40s
Build / Build Components (1.20) (pull_request) Successful in 3m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m41s
Tests and linters / Staticcheck (pull_request) Successful in 5m32s
Tests and linters / Lint (pull_request) Successful in 5m59s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m3s
Tests and linters / Tests with -race (pull_request) Successful in 6m6s
To reduce memory allocations add `SetMarshaledData` method call
to return already marshalled data in next `StableMarshal` calls.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-02 17:34:33 +03:00
a26483fc30
[ #749 ] morph: Fix panic when closing morph client
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-11-01 10:48:10 +00:00
c80b46fad3
[ #754 ] blobstor: Estimate compressability
...
DCO action / DCO (pull_request) Successful in 1m59s
Vulncheck / Vulncheck (pull_request) Successful in 3m31s
Build / Build Components (1.20) (pull_request) Successful in 4m37s
Build / Build Components (1.21) (pull_request) Successful in 4m33s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m54s
Tests and linters / Staticcheck (pull_request) Successful in 4m49s
Tests and linters / Tests with -race (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 6m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m9s
Now it is possible to enable compressability estimation.
If data is likely uncompressable, it should reduce CPU time and memory.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 11:24:32 +03:00
05b508f79a
[ #772 ] proto: Fix file ending
...
DCO action / DCO (pull_request) Successful in 3m47s
Vulncheck / Vulncheck (pull_request) Successful in 4m47s
Build / Build Components (1.21) (pull_request) Successful in 6m5s
Build / Build Components (1.20) (pull_request) Successful in 6m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 13m9s
Tests and linters / Staticcheck (pull_request) Successful in 13m17s
Tests and linters / Lint (pull_request) Successful in 13m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 14m12s
Tests and linters / Tests with -race (pull_request) Successful in 4m30s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:04 +03:00
79088baa06
[ #772 ] node: Apply gofumpt
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
00aa6d9749
[ #633 ] shard/test: Fix TestCounters()
...
Introduced in 362f24953a
2023-10-31 12:53:28 +00:00
b8f79f4227
[ #633 ] shard/test: Fix race conditions in TestCounters()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 12:53:28 +00:00
261d281154
[ #762 ] go.mod: Update sdk-go
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 11:22:29 +00:00
869518be0a
[ #728 ] writecache: Fix Badger writecache race.
...
Build / Build Components (1.21) (pull_request) Successful in 1m47s
DCO action / DCO (pull_request) Successful in 2m1s
Vulncheck / Vulncheck (pull_request) Successful in 3m5s
Build / Build Components (1.20) (pull_request) Successful in 3m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m43s
Tests and linters / Staticcheck (pull_request) Successful in 5m49s
Tests and linters / Lint (pull_request) Successful in 6m20s
Tests and linters / Tests with -race (pull_request) Successful in 6m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m6s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-30 18:36:41 +03:00
d4b6ebe7e7
[ #725 ] writecache: Fix metric values
...
DCO action / DCO (pull_request) Successful in 1m33s
Build / Build Components (1.21) (pull_request) Successful in 3m11s
Build / Build Components (1.20) (pull_request) Successful in 3m22s
Vulncheck / Vulncheck (pull_request) Successful in 3m34s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m0s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m1s
Tests and linters / Staticcheck (pull_request) Successful in 6m4s
Tests and linters / Tests with -race (pull_request) Successful in 3m58s
Tests and linters / Lint (pull_request) Successful in 11m43s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-27 12:22:29 +03:00
121f5c4dd8
[ #757 ] ir: Do not exclude node in maintenance mode from netmap
...
Build / Build Components (1.20) (pull_request) Successful in 6m20s
DCO action / DCO (pull_request) Successful in 6m45s
Vulncheck / Vulncheck (pull_request) Successful in 6m20s
Build / Build Components (1.21) (pull_request) Successful in 8m34s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m21s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m17s
Tests and linters / Staticcheck (pull_request) Successful in 9m15s
Tests and linters / Lint (pull_request) Successful in 9m42s
Tests and linters / Tests with -race (pull_request) Successful in 11m34s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-26 10:50:32 +03:00
9f7c2d8810
[ #752 ] innerring: Simplify keyPosition()
...
DCO action / DCO (pull_request) Successful in 3m0s
Vulncheck / Vulncheck (pull_request) Successful in 3m10s
Build / Build Components (1.20) (pull_request) Successful in 4m3s
Build / Build Components (1.21) (pull_request) Successful in 4m8s
Tests and linters / Staticcheck (pull_request) Successful in 5m13s
Tests and linters / Lint (pull_request) Successful in 5m44s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m0s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m7s
Tests and linters / Tests with -race (pull_request) Successful in 7m50s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
cddc58ace2
[ #752 ] innerring: Optimize keyPosition()
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
KeyPosition-8 2771.50n ± 10% 40.32n ± 4% -98.55% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
KeyPosition-8 1.531Ki ± 0% 0.000Ki ± 0% -100.00% (p=0.000 n=10)
│ old │ new │
│ allocs/op │ allocs/op vs base │
KeyPosition-8 21.00 ± 0% 0.00 ± 0% -100.00% (p=0.000 n=10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
0a9830564f
[ #752 ] morph: Adopt neo-go RPC statuses
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
6950312967
[ #752 ] morph: Drop loop copy kludges
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
fe1acf9e9a
[ #750 ] morph: Remove deprecated channel use
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 07:57:05 +00:00
559ad58ab1
[ #642 ] writecache: Remove usage of close channel in bbolt
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:57:50 +00:00
c0b86f2d93
[ #642 ] writecache: Remove usage of close channel in badger
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:57:50 +00:00
b0cf100427
[ #49 ] node: React on SIGHUP only when node in READY state
...
Add more info in logs when node is going to shut down,
but initialization process still in progress.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:55:29 +00:00
58b6224dd8
[ #747 ] client: Refactor PrmObjectPutInit usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
12b7cf2533
[ #747 ] client: Refactor PrmObjectPutSingle usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
dc4d27201b
[ #733 ] morph: Fix delete container signature check
...
DCO action / DCO (pull_request) Successful in 3m26s
Vulncheck / Vulncheck (pull_request) Successful in 4m22s
Build / Build Components (1.21) (pull_request) Successful in 5m30s
Build / Build Components (1.20) (pull_request) Successful in 6m4s
Tests and linters / Staticcheck (pull_request) Successful in 6m52s
Tests and linters / Tests with -race (pull_request) Failing after 7m16s
Tests and linters / Lint (pull_request) Successful in 9m46s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m44s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m41s
Committed invalid condition, it was just for debug.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 18:07:37 +03:00
189dbb01be
[ #733 ] frostfs-cli: Add control ir remove-container
...
DCO action / DCO (pull_request) Successful in 3m51s
Vulncheck / Vulncheck (pull_request) Successful in 3m49s
Build / Build Components (1.21) (pull_request) Successful in 4m34s
Build / Build Components (1.20) (pull_request) Successful in 6m21s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m12s
Tests and linters / Staticcheck (pull_request) Successful in 6m40s
Tests and linters / Tests with -race (pull_request) Successful in 9m21s
Tests and linters / Lint (pull_request) Successful in 9m39s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 16:22:18 +03:00
f2437f7ae9
[ #734 ] shard: Fix Delete method
...
DCO action / DCO (pull_request) Successful in 1m36s
Build / Build Components (1.20) (pull_request) Successful in 3m46s
Vulncheck / Vulncheck (pull_request) Successful in 3m16s
Tests and linters / Staticcheck (pull_request) Successful in 4m42s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m27s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m35s
Tests and linters / Tests with -race (pull_request) Successful in 6m33s
Build / Build Components (1.21) (pull_request) Successful in 13m7s
Tests and linters / Lint (pull_request) Successful in 19m14s
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
f26233b47a
[ #734 ] metabase: Include UpdateStorageID in metrics and traces
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
7e0c5a55de
[ #734 ] writecache: Fix flush
...
Now UpdateStorageID doesn't return error in case of logical error.
If object is in graveyard or GC market, it is still required to
update storage ID.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:17 +03:00
3a997d1207
[ #680 ] metrics: Initialize log metrics together with services
...
DCO action / DCO (pull_request) Successful in 3m31s
Vulncheck / Vulncheck (pull_request) Successful in 3m17s
Build / Build Components (1.21) (pull_request) Successful in 3m56s
Tests and linters / Staticcheck (pull_request) Successful in 5m13s
Tests and linters / Lint (pull_request) Successful in 5m50s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m31s
Build / Build Components (1.20) (pull_request) Successful in 7m44s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m32s
Tests and linters / Tests with -race (pull_request) Successful in 7m33s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-11 17:08:03 +03:00
994f48f8bb
[ #680 ] metrics: Export log and morph with script
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-11 17:08:03 +03:00
aca11d7474
[ #735 ] policer: Allow to provide metrics from the outside
...
DCO action / DCO (pull_request) Successful in 3m9s
Vulncheck / Vulncheck (pull_request) Successful in 3m28s
Build / Build Components (1.21) (pull_request) Successful in 4m17s
Build / Build Components (1.20) (pull_request) Successful in 4m35s
Tests and linters / Lint (pull_request) Successful in 7m19s
Tests and linters / Tests with -race (pull_request) Failing after 9m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m0s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m19s
Tests and linters / Staticcheck (pull_request) Successful in 11m4s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-11 15:14:13 +03:00
5e229dc248
[ #701 ] metrics: add metric to evaluate policer performance
...
Add processed objects counter in policerMetrics,
add policer field to NodeMetrics
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-10-09 19:02:08 +00:00
4caa934eea
[ #729 ] containersvc: Remove load announcement
...
IR code was removed in 8879c6ea
2023-10-09 19:01:13 +00:00
