Commit graph

2428 commits

Author SHA1 Message Date
dbccc429cf WIP
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 20:20:01 +03:00
05996103ec goodstor: initial implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 17:58:09 +03:00
884b77211c uringstor: fixes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 17:58:09 +03:00
ea2918d405 blobstortest: temporary skip some tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 17:57:56 +03:00
32f3268819 uringstor: initial implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 17:57:20 +03:00
Roman Khimov
d29b13454f [#239] morph/client: Simplify code interacting with magic numbers
It can't be uint64 in fact, but this error is buried deeply in the NetworkInfo
API structure, so we're not touching MagicNumber() for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 13:15:30 +00:00
Roman Khimov
f41ad9d419 [#239] morph/client: Recreate actor/wrappers in SetGroupSignerScope
That's the reason #2230 and #2263 were not detected earlier, we actually had
Global scope being used before reconnection to RPC node.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 13:15:30 +00:00
Roman Khimov
be4df989e5 [#239] morph/client: Deduplicate signers in Client a bit
One signer in the cfg is enough.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 13:15:30 +00:00
Roman Khimov
96b38f7e86 [#239] morph/client: Add CalledByEntry into the "grouped" scope
Fixes #2230, fixes #2263. CustomGroups are nice while we're only calling NeoFS
contracts, but it doesn't work at all for standard ones like GAS or Notary.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 13:15:30 +00:00
04727ce1d6 Revert "[#135] signature: Add tracing"
This reverts commit 5778980252.
2023-04-12 16:57:02 +03:00
08769f413f Revert "[#135] acl: Add tracing spans"
This reverts commit b2ca730547.
2023-04-12 16:54:13 +03:00
5d2affa5cd testutil: Fix linter warning
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-12 11:00:02 +03:00
5778980252 [#135] signature: Add tracing
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
b2ca730547 [#135] acl: Add tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
0920d848d0 [#135] get-object: Add tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
c4865783fc [#236] blobstor/test: Prefill storage in parallel in read benchmark
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-11 20:52:11 +03:00
6ad5c38225 [#236] testutil: Use random object id in RandObjGenerator
Before this commit it was like this:
```
BenchmarkSubstorageWritePerf/memstore-rand10-8            227425              4859 ns/op
BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8     --- FAIL: BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
BenchmarkSubstorageWritePerf/fstree-rand10-8            --- FAIL: BenchmarkSubstorageWritePerf/fstree-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-11 20:52:04 +03:00
c85a0bc866 [#236] blobstor/test: Reduce test descriptions
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-11 20:52:04 +03:00
93eba19a8e [#223] objectsvc: Refactor split-tree traverse
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
2ed9fd3f94 [#223] objectsvc: Refactor request parameters
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
ccf8463e69 [#223] controlsvc: Drop unnecessary nolint
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
ae86cda58c [#223] sg: Refactor storage group parameters
Resolve containedctx linter for SearchSGPrm and GetSGPrm  structs.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
6016d78a45 [#223] core: Refactor object format validator
Resolve funlen linter for FormatValidator.ValidateContent method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 13:56:09 +03:00
02831d427b [#213] metrics: Refactor object metrics
Resolve funlen linter for newObjectServiceMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 09:43:24 +03:00
38ae71cc7d [#213] metrics: Refactor engine metrics
Resolve funlen linter for newEngineMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 09:43:24 +03:00
0b9622c418 [#191] control: Add Doctor RPC
Doctor RPC performs complex operations on the storage engine.
Currently only duplicate removal is supported.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 17:25:50 +00:00
dbc3811ff4 [#191] engine: Allow to remove redundant object copies
RemoveDuplicates() removes all duplicate object copies stored on
multiple shards. All shards are processed and the command tries to leave
a copy on the best shard according to HRW.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 17:25:50 +00:00
cb172e73a6 [#228] node: Use uber atomic package instead standard
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-04-07 15:37:27 +00:00
c236b54a65 [#212] reputationsvc: Resolve funlen linter
Resolve funlen linter for Calculator.iterateDaughter method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
7ebbfa3358 [#212] reputationsvc: Resolve linters and rename
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
469e8a6e59 [#212] reputationsvc: Resolve containedctx linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
e2f13d03d7 [#222] auditsvc: Refactor PoR audit
Resolve funlen linter for Context.checkStorageGroupPoR method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 17:29:13 +03:00
e8d340287f [#222] auditsvc: Refactor audit task
Resolve containedctx linter. Cancel task by listen cancel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 17:29:13 +03:00
3dbff0a478 [#222] auditsvc: Resolve containedctx linter
Resolve containedctx linter for commonCommunicatorPrm type.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 17:29:13 +03:00
fe87735073 [#219] morph: Refactor notary preparator
Resolve funlen linter for Preparator.Prepare method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:43:20 +03:00
d07e40d6fe [#219] morph: Refactor moprh event listener
Resolve funlen and gocognit linters for listener.listenLoop method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:43:20 +03:00
775179f823 [#219] morph: Refactor notary invoke
Resolve funlen linter for Client.notaryInvoke method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:43:20 +03:00
e815b19101 [#219] morph: Resolve containedctx linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:43:20 +03:00
56282edf02 [#166] node: Parallelize background tree service sync
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-04-07 04:43:32 +00:00
4ec69cbbf8 [#203] control: Fix test
Fix shard compare loop.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
f32f61df87 [#203] pilorama: Refactor tests
Do not pass 0 as channel capacity.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
8908798f59 [#203] node: Resolve unused vars
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
bab11492ad [#203] node: Resolve never used errors
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
68a2f36636 [#203] morph: Mark depracated methods
Skip staticcheck for depracated methods. Will be fixed soon.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
9e2df4b7c7 [#203] node: Fix double imports
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
ab891517de [#116] node: Fix bug with extra generated files in TestReload
* Create testNewEngine in engineWithShards without default opts

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-04-05 18:49:34 +03:00
c58ab0c369 [#193] getsvc: Reduce private key requests
Get private key only once for request forwaring.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
89924071cd [#193] getsvc: Edit request forwarder signature
Pass context to forwarder direct, without closure.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
6c7b708a98 [#193] getsvc: Refactor get range params creation
Resolve funlen linter for toRangePrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00