TrueCloudLab/frostfs-node
20
1
Fork 27
Code Issues 95 Pull requests 4 Projects Releases 1 Activity Actions
3884 commits 8 branches 85 tags 54 MiB
Commit graph

2559 commits

Author SHA1 Message Date
Evgenii Stratonikov
f07e2d4812 [#285] lint: Fix revive/unused-parameter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-26 17:35:04 +03:00
Airat Arifullin
9d01029733 [#166] node: Parallelize background tree service sync by batching
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
* Merge operations

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-26 10:17:56 +00:00
Airat Arifullin
299b24b974 [#166] node: Parallelize background tree service sync by batching 
* Concurrently dispatch TreeApply operations for batching in forest

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-26 10:17:56 +00:00
Denis Kirillov
dce5924a89 [#229] services/tree: Use bearer owner as signer 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-26 10:23:33 +03:00
Alex Vanin
89530534a1 [#229] service/tree: Disable container owner check in tree service 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-04-26 10:23:33 +03:00
Alex Vanin
c04f6c5e59 [#229] acl: Allow Impersonate 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-04-26 10:23:33 +03:00
Evgenii Stratonikov
6fef2726b8 [#164] metrics: Allow to export metrics description 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-24 09:01:13 +00:00
Evgenii Stratonikov
4ade5339da [#164] metrics: Rename metrics.go -> node.go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-24 09:01:13 +00:00
Evgenii Stratonikov
015d62425b [#164] metrics: Fill local registry explicitly 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-24 09:01:13 +00:00
Pavel Karpy
09938a9841 Revert "[#262] meta: Do not return old expired objects"
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
This reverts commit 3d23b087

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-20 19:22:55 +03:00
Pavel Karpy
e9461686b8 [#274] wc: Resolve possible deadlock 
If operation with WC are _fast enough_ (e.g. `Init` failed and `Close` is
called immediately) there is a race and a deadlock that do not allow finish
(and start, in fact) an initialization routine because of taken `modeMtx`
and also do not allow finish `Close` call because of awaiting initialization
finish. So do stop initialization _before_ any mutex is taken.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-20 19:22:54 +03:00
Pavel Karpy
6b6f33ed71 [#274] wc: Make wait groups work more explicit 
Do not run routine that calls `wg.Done()` inside, it is hard to read.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-20 19:22:54 +03:00
Pavel Karpy
4f5f832137 [#268] notary_preparator: Actualize notary requests parsing
Some checks failed
ci/woodpecker/pr/pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline failed
Details 
After 75d7891ca1
`neo-go` does claim that an empty invocation script is the only way to
fill missing signature for unsigned notary requests. The new notary actor
does it that way and, therefore, breaks notary request parsing by the
Alphabet because of skipping any request that is not filled with a dummy (64
zeros) invocation script. Support both way. The "Dummy" approach will be
dropped later.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-20 10:29:34 +03:00
Airat Arifullin
6c90bb87f1 [#118] node: add ctx for unit tests for blobstor
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-19 18:09:33 +03:00
Pavel Karpy
3d23b08773 [#262] meta: Do not return old expired objects
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-19 13:37:38 +00:00
Airat Arifullin
13c8afcb02 [#118] node: add unit concurrent tests for blobstor
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-19 10:22:50 +00:00
Pavel Karpy
20cd080323 [#255] write-cache: Fix init race condition
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Do not use WC's internals in the initialization routines without mode
protection. WC should be able to change its mode even if the initialization
is not finished yet.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-19 09:29:07 +00:00
Dmitrii Stepanov
3d43b0f7f9 [#265] node: Fix after SDK & API-Go version up
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-18 12:09:19 +03:00
Dmitrii Stepanov
a358255c1b [#265] node: Drop unused 
Resolve unused linter.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-18 12:09:16 +03:00
Dmitrii Stepanov
f07d4158f5 [#249] node: Drop subnet from IR and morph 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-17 09:46:02 +03:00
Pavel Karpy
262c9c2b93 [#256] blobovniczaTree: Make Exists test stable
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Corrupt and request _the same_ file.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 16:46:24 +03:00
Evgenii Stratonikov
0b42a00a60 [#254] innerring: Remove unused TimersHandlers() method from processors
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 15:34:15 +03:00
Evgenii Stratonikov
8466894fdf [#250] control: remove DumpShard and RestoreShard RPC
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
We have `Evacuate` with a cleaner interface.
Also, remove them from CLI and engine.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 12:28:49 +00:00
Pavel Karpy
070154d506 [#247] client: Drop reputation related RPCs
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:54:28 +03:00
Pavel Karpy
beabed788c [#247] network_config: Drop reputation 
Drop the code that was expected to work with global reputation network
parameters.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:53:22 +03:00
Pavel Karpy
8799138fcb [#247] morph: Drop reputation contract 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:44:14 +03:00
Pavel Karpy
960e3c219e [#247] config, doc: Drop reputation references 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:44:14 +03:00
Pavel Karpy
560f73ab7e [#247] node, ir: Drop reputation related code 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:44:14 +03:00
Dmitrii Stepanov
6121b541b5 [#242] treesvc: Add tracing spans
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Dmitrii Stepanov
d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Dmitrii Stepanov
200fc8b882 [#242] put: Pass context to relay function 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Anton Nikiforov
995db117d0 [#238] node: Read cfg from dir even if cfg file not set
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-04-14 10:19:10 +00:00
Alejandro Lopez
41eb3129ae [#139] Refactor blobovnicza exist test to not use chmod
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-04-14 12:16:14 +03:00
Anton Nikiforov
299b6a6938 [#100] adm: Use netmap constants from pkg
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-04-14 05:12:50 +00:00
Evgenii Stratonikov
cffcc7745e [#240] logs: Factor out common service log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Evgenii Stratonikov
0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Roman Khimov
d29b13454f [#239] morph/client: Simplify code interacting with magic numbers
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
It can't be uint64 in fact, but this error is buried deeply in the NetworkInfo
API structure, so we're not touching MagicNumber() for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-13 13:15:30 +00:00
Roman Khimov
f41ad9d419 [#239] morph/client: Recreate actor/wrappers in SetGroupSignerScope 
That's the reason #2230 and #2263 were not detected earlier, we actually had
Global scope being used before reconnection to RPC node.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-13 13:15:30 +00:00
Roman Khimov
be4df989e5 [#239] morph/client: Deduplicate signers in Client a bit 
One signer in the cfg is enough.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-13 13:15:30 +00:00
Roman Khimov
96b38f7e86 [#239] morph/client: Add CalledByEntry into the "grouped" scope 
Fixes #2230, fixes #2263. CustomGroups are nice while we're only calling NeoFS
contracts, but it doesn't work at all for standard ones like GAS or Notary.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-13 13:15:30 +00:00
Evgenii Stratonikov
04727ce1d6 Revert "[#135] signature: Add tracing" 
This reverts commit 5778980252.
 2023-04-12 16:57:02 +03:00
Evgenii Stratonikov
08769f413f Revert "[#135] acl: Add tracing spans" 
This reverts commit b2ca730547.
 2023-04-12 16:54:13 +03:00
Evgenii Stratonikov
5d2affa5cd testutil: Fix linter warning 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-12 11:00:02 +03:00
Dmitrii Stepanov
5778980252 [#135] signature: Add tracing
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
b2ca730547 [#135] acl: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Evgenii Stratonikov
c4865783fc [#236] blobstor/test: Prefill storage in parallel in read benchmark
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:11 +03:00
Evgenii Stratonikov
6ad5c38225 [#236] testutil: Use random object id in RandObjGenerator 
Before this commit it was like this:
```
BenchmarkSubstorageWritePerf/memstore-rand10-8            227425              4859 ns/op
BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8     --- FAIL: BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
BenchmarkSubstorageWritePerf/fstree-rand10-8            --- FAIL: BenchmarkSubstorageWritePerf/fstree-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:04 +03:00
Evgenii Stratonikov
c85a0bc866 [#236] blobstor/test: Reduce test descriptions 
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:04 +03:00
Dmitrii Stepanov
93eba19a8e [#223] objectsvc: Refactor split-tree traverse 
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov
2ed9fd3f94 [#223] objectsvc: Refactor request parameters 
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov
ccf8463e69 [#223] controlsvc: Drop unnecessary nolint 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov
ae86cda58c [#223] sg: Refactor storage group parameters 
Resolve containedctx linter for SearchSGPrm and GetSGPrm  structs.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov
6016d78a45 [#223] core: Refactor object format validator 
Resolve funlen linter for FormatValidator.ValidateContent method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 13:56:09 +03:00
Dmitrii Stepanov
02831d427b [#213] metrics: Refactor object metrics
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for newObjectServiceMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 09:43:24 +03:00
Dmitrii Stepanov
38ae71cc7d [#213] metrics: Refactor engine metrics 
Resolve funlen linter for newEngineMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 09:43:24 +03:00
Evgenii Stratonikov
0b9622c418 [#191] control: Add Doctor RPC 
Doctor RPC performs complex operations on the storage engine.
Currently only duplicate removal is supported.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-07 17:25:50 +00:00
Evgenii Stratonikov
dbc3811ff4 [#191] engine: Allow to remove redundant object copies 
RemoveDuplicates() removes all duplicate object copies stored on
multiple shards. All shards are processed and the command tries to leave
a copy on the best shard according to HRW.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-07 17:25:50 +00:00
Airat Arifullin
cb172e73a6 [#228] node: Use uber atomic package instead standard
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-07 15:37:27 +00:00
Dmitrii Stepanov
c236b54a65 [#212] reputationsvc: Resolve funlen linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for Calculator.iterateDaughter method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:35:57 +00:00
Dmitrii Stepanov
7ebbfa3358 [#212] reputationsvc: Resolve linters and rename 
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:35:57 +00:00
Dmitrii Stepanov
469e8a6e59 [#212] reputationsvc: Resolve containedctx linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:35:57 +00:00
Dmitrii Stepanov
e2f13d03d7 [#222] auditsvc: Refactor PoR audit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for Context.checkStorageGroupPoR method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 17:29:13 +03:00
Dmitrii Stepanov
e8d340287f [#222] auditsvc: Refactor audit task 
Resolve containedctx linter. Cancel task by listen cancel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 17:29:13 +03:00
Dmitrii Stepanov
3dbff0a478 [#222] auditsvc: Resolve containedctx linter 
Resolve containedctx linter for commonCommunicatorPrm type.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 17:29:13 +03:00
Dmitrii Stepanov
fe87735073 [#219] morph: Refactor notary preparator
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for Preparator.Prepare method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:43:20 +03:00
Dmitrii Stepanov
d07e40d6fe [#219] morph: Refactor moprh event listener 
Resolve funlen and gocognit linters for listener.listenLoop method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:43:20 +03:00
Dmitrii Stepanov
775179f823 [#219] morph: Refactor notary invoke 
Resolve funlen linter for Client.notaryInvoke method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:43:20 +03:00
Dmitrii Stepanov
e815b19101 [#219] morph: Resolve containedctx linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:43:20 +03:00
Airat Arifullin
56282edf02 [#166] node: Parallelize background tree service sync
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-07 04:43:32 +00:00
Dmitrii Stepanov
4ec69cbbf8 [#203] control: Fix test 
Fix shard compare loop.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov
f32f61df87 [#203] pilorama: Refactor tests 
Do not pass 0 as channel capacity.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov
8908798f59 [#203] node: Resolve unused vars 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov
bab11492ad [#203] node: Resolve never used errors 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov
68a2f36636 [#203] morph: Mark depracated methods 
Skip staticcheck for depracated methods. Will be fixed soon.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov
9e2df4b7c7 [#203] node: Fix double imports 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Airat Arifullin
ab891517de [#116] node: Fix bug with extra generated files in TestReload
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
* Create testNewEngine in engineWithShards without default opts

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-05 18:49:34 +03:00
Dmitrii Stepanov
c58ab0c369 [#193] getsvc: Reduce private key requests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Get private key only once for request forwaring.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
89924071cd [#193] getsvc: Edit request forwarder signature 
Pass context to forwarder direct, without closure.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
6c7b708a98 [#193] getsvc: Refactor get range params creation 
Resolve funlen linter for toRangePrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
b0786d2e5c [#193] getsvc: Refactor get params creation 
Resolve funlen linter for toPrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
f889893216 [#193] getsvc: Refactor head param creation 
Resolve funlen linter for toHeadPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
91ead04fa4 [#193] getsvc: Resolve funlen linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
1bf21dbb47 [#193] getsvc: Resolve context linters 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov
206458c841 [#217] containersvc: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:00 +00:00
Dmitrii Stepanov
279261ace3 [#217] containersvc: Refactor route passing 
Resolve containedctx for routeCtx.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:00 +00:00
Airat Arifullin
6f7b6a8813 [#116] node: Improve shard/engine construction in tests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
* Introduce testEngineWrapper that can be constructed with different options

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-05 14:36:40 +00:00
Dmitrii Stepanov
d6486d172e [#210] policier: Refactor nodes processing
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for processNodes method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov
080be5cfcd [#210] policier: Refactor object placement 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov
23575e1ac0 [#210] policier: Resolve contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Evgenii Stratonikov
9098d0eec0 [#211] engine: Unify shard mode checks for tree operations
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
All operations must ensure the shard is not in a degraded mode.
Write operations must also ensure the shard is not in a read-only mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-05 11:10:39 +00:00
Evgenii Stratonikov
760af6b912 [#211] fstree: Consider ENOSPC a logical error 
We already do this for file writing, however directory creation can also
fail.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-05 11:10:39 +00:00
Dmitrii Stepanov
d85703a963 [#208] searchsvc: Refactor request forwarding
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen & gocognit linters for toPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:55 +03:00
Dmitrii Stepanov
0b38419fbf [#208] searchsvc: Resolve context linters 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:50 +03:00
Dmitrii Stepanov
5f2a1531fe [#208] deletesvc: Resolve containedctx linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:40 +03:00
Dmitrii Stepanov
4941926c9d [#207] aclsvc: Drop outdated tag
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:14 +00:00
Dmitrii Stepanov
585415fa92 [#207] aclsvc: Refactor send checker 
Resolve funlen linter for putStreamBasicChecker.Send method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:14 +00:00
Dmitrii Stepanov
9ef790f782 [#207] aclsvc: Refactor object headers read 
Resolve funlen linter for readObjectHeaders method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:13 +00:00
Dmitrii Stepanov
cd33a57f44 [#207] aclsvc: Refactor EACL check 
Resolve funlen linter for CheckEACL method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:13 +00:00
Dmitrii Stepanov
1f1aed87be [#188] metabase: Refactor object inhume
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for db.Inhume method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
5a66db80c5 [#188] engine: Refactor shard evacuation 
Resolve funlen and gocognit linter for StorageEngine.Evacuate method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
456bc097f7 [#188] engine: Refactor get range from engine 
Resolve funlen linter for StorageEngine.getRange method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
3010ca2649 [#188] engine: Refactor get object from engine 
Resolve funlen linter for StorageEngine.get method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
0739c36a3b [#188] metabase: Refactor object put to metabase 
Resolve funlen linter for db.put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
8273a3dfb2 [#188] blobstor: Refactor blobstor test 
Resolve funlen linter for TestIterate function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
594b5821ed [#188] blobstor: Refactor put data to blobovniczas 
Resolve funlen linter for Blobovniczas.Put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Evgenii Stratonikov
ee7468daa7 [#205] innerring: Provide alphabetState param to epochTimer
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Fix NPE, introduced in f09ee27a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-04 10:47:31 +00:00
Dmitrii Stepanov
8e5a0dcf27 [#204] gc: Fix GC handlers start
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 06:48:27 +00:00
Evgenii Stratonikov
0948a280fa [#195] morph: use blocking unlimited pool for notifications
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
Details 
With non-blocking pool restricted by 10 in capacity, the probability of
dropping events is unexpectedly big. Notifications are an essential part of the FrostFS,
we should not drop anything, especially new epochs.
```
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        debug        subscriber/subscriber.go:154        new notification event from sidechain        {"name": "NewEpoch"}
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        warn        event/listener.go:248        listener worker pool drained        {"chain": "morph", "capacity": 10}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-03 15:58:16 +00:00
Dmitrii Stepanov
ece6c820e7 [#199] putsvc: Refactor streamer initialization
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov
27bdddc48f [#199] putsvc: Refactor put object 
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov
cecea8053a [#199] putsvc: Refactor streamer pool 
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov
14d894178e [#199] putsvc: Refactor placement iterator 
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Pavel Karpy
a69c6d1ec9 [#2272] morph: Do not subscribe to events without listening
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
It led to a neo-go dead-lock in the `subscriber` component. Subscribing to
notifications is the same RPC as any others, so it could also be blocked
forever if no async listening (reading the notification channel) routine
exists. If a number of subscriptions is big enough (or a caller is lucky
enough) subscribing loop might have not finished subscribing before the
first notification is received and then: subscribing RPC is blocked by
received notification (non)handling and listening notifications routine is
blocked by not finished subscription loop.
That commit starts listening notification channel _before_ any subscription
actions.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-03 08:42:41 +00:00
Pavel Karpy
aa92f977ef [#181] ir: Do not pay for audit by non-alphabet nodes 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-31 12:53:37 +03:00
Pavel Karpy
f09ee27af9 [#181] ir: Do not process container estimations by non-alphabet nodes 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-31 12:53:09 +03:00
Pavel Karpy
db5321309d [#181] ir: Do not sync the Alphabet by non-alphabet nodes 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-31 12:52:16 +03:00
Pavel Karpy
44d5412e10 [#181] ir: Do not deposit notary GAS by non-alphabet nodes 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-31 12:51:12 +03:00
Dmitrii Stepanov
56161d39b4 [#168] node: Refactor container services 
Resolve containedctx for remoteLoadAnnounceWriter struct

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-31 09:32:59 +03:00
Dmitrii Stepanov
a7c79c773a [#168] node: Refactor node config 
Resolve containedctx linter for cfg

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-31 09:32:59 +03:00
Dmitrii Stepanov
8426d25f4b [#185] ir: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
c1cbe6ff2d [#185] ir: Refactor signature verification 
Resolve funlen linter for verifySignature method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
aeb4bbc51e [#185] ir: Refactor alphabet update 
Resolve funlen linter for processAlphabetSync method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
30c18d46cc [#185] ir: Refactor audit 
Resolve funlen linter for processStartAudit method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
6be2688fb4 [#185] ir: Refactor gas emit 
Resolve funlen linter for processEmit method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
4d160bd4ab [#185] ir: Refactor ir service creation 
Resolve funlen linter for New function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
c8a6978563 [#185] ir: Refactor ir start 
Resolve funlen linter for Server.Start method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Dmitrii Stepanov
85fb9e77c4 [#185] ir: Resolve funlen linter 
Resolve funlen linter for GlagoliticLetter.String method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 19:10:33 +00:00
Airat Arifullin
7be5a0fd79 [#189] node: Fix bug with sync/atomic package
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-30 14:50:43 +00:00
Dmitrii Stepanov
ab32067152 [#183] gc: Fix drop expired locked complex objects
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Do not delete bucket keys during iteration

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 15:33:42 +03:00
Dmitrii Stepanov
9f0bce5c15 [#183] gc: Fix drop expired locked simple objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 15:33:42 +03:00
Alejandro Lopez
341fe1688f [#139] test: Add test storage implementation
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-29 14:28:49 +00:00
Pavel Karpy
e843e7f090 [#184] ir: Delete unused config reader param
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Not used since notary environments do endless and non-configurable
deposits.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-29 10:34:54 +00:00
Pavel Karpy
ba58a77f8c [#184] node, ir: Do not wait for already made notary deposits 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-29 10:34:54 +00:00
Pavel Karpy
3646723ae3 [#184] morph: Make deposit logs INFO 
We have already had and solved plenty of deposit issues and notary balance
is a really important thing. Deserves to be INFO even before the huge logs
severity refactor, happens on an app start only.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-29 10:34:54 +00:00
Pavel Karpy
97e201993b [#184] *: Unify done contexts handling 
If `ctx.Done()`, return `ctx.Err()` in every function that returns an error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-29 10:34:54 +00:00
Airat Arifullin
221203beeb [#180] node: Refactor panics in unit test
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Dmitrii Stepanov
91717d4b98 [#176] morph: Resolve funlen linter
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 18:49:55 +03:00
Evgenii Stratonikov
382ecae96a [#172] Use ContainersOf() for container list fetching
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Previously we were limited by ~2048 containers because of
neo-go VM limits.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-27 14:32:55 +00:00
Evgenii Stratonikov
9e54646248 [#172] client/container: Support listing containers with containersOf 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-27 14:32:55 +00:00
Evgenii Stratonikov
bf7d80f44b [#172] morph/client: Support iterators via neo-go session API 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-27 14:32:55 +00:00
Pavel Karpy
f3ff9fd251 [#73] morph: Rename vars that collide with package names
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-24 09:42:30 +00:00
Pavel Karpy
533e9f8b75 [#59] morph: Adopt updated neo-go client API for subs 
It does not use deprecated methods anymore but also adds more code that
removes. Future refactor that will affect more components will optimize
usage of the updated API.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-24 09:42:30 +00:00
aarifullin
34329d67ff [#86] node: Fix unit test and linter errors
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 12:42:58 +03:00
Ayrat Arifullin
9808dec591 [#86] node: Move testing utils to one package
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
Details 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 08:19:15 +00:00
Denis Kirillov
342e571d89 [#159] Add handle __SYSTEM__ sys attributes
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-22 17:35:20 +03:00
Pavel Karpy
da8da1c63a [#98] fstree: Do not fail iteration over just removed files
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
A directory is read and files are saved to a local variable. The iteration
over such files may lead to a non-existing files reading due to a normal SN
operation cycle and, therefore, may lead to a returning the OS error to a
caller. Skip just removed (or lost) files as the golang std library does in
similar situations:
5f1a0320b9/src/os/dir_unix.go (L128-L133).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-22 07:26:27 +00:00
Dmitrii Stepanov
49234b915e [#155] search-service: Fix search with ST
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Search should return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-22 07:21:09 +00:00
Dmitrii Stepanov
1637a3edce [#155] search-service: Add search with ST test 
In case of session token (ST) with object IDs search should
return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-22 07:21:09 +00:00