Commit graph

852 commits

Author SHA1 Message Date
25bedab91a [#425] object: Do not store large slices in pool
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Dynamically growing an unbounded buffers can cause a large
amount of memory to be pinned and never be freed.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-02 17:48:24 +03:00
5e2fcec60f [#396] treesvc: properly remember last height on shutdown
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Previously `newHeight` was updated in parallel, so that applying
operation at height H did not imply successful TreeApply() for H-1.
And because we have no context in TreeUpdateLastSyncHeight(), invalid
starting height could be written if the context was canceled.

In this commit we return the new height only if all operations were
successfully applied.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-29 10:25:26 +00:00
493cafc62a [#355] Increase tree svc client cache size to test hypotheses
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-17 15:17:35 +03:00
Pavel Karpy
c60029d3b0 [#323] node: Fix tree svc panic
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
If a connection has not been established earlier, it stores `nil` in LRU
cache. Cache eviction tries to close every connection (even a `nil` one) and
panics but not crash the app because we are using pools.
That ugly bug also leads to a deadlock where `Unlock` is not called via
`defer` func (and that is the way I found it).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-05-04 20:04:30 +03:00
Pavel Karpy
0beb7ccf5c [#284] node: Use copy_number on server side
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-26 10:57:34 +03:00
79d59e4ed2 [#266] services/tree: Do not accept requests until initial sync is finished
Some checks failed
ci/woodpecker/pr/pre-commit Pipeline failed
ci/woodpecker/push/pre-commit Pipeline failed
`Apply` is deliberately left out -- we don't want to miss anything new.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 13:30:45 +03:00
364b4ac572 [#266] services/tree: Batch operations on the service level
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 13:30:45 +03:00
f7679a8168 [#266] services/tree: Return operation log up to some height
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 13:30:45 +03:00
04727ce1d6 Revert "[#135] signature: Add tracing"
This reverts commit 5778980252.
2023-04-12 16:57:02 +03:00
08769f413f Revert "[#135] acl: Add tracing spans"
This reverts commit b2ca730547.
2023-04-12 16:54:13 +03:00
5778980252 [#135] signature: Add tracing
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
b2ca730547 [#135] acl: Add tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
0920d848d0 [#135] get-object: Add tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
93eba19a8e [#223] objectsvc: Refactor split-tree traverse
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
2ed9fd3f94 [#223] objectsvc: Refactor request parameters
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
ccf8463e69 [#223] controlsvc: Drop unnecessary nolint
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
0b9622c418 [#191] control: Add Doctor RPC
Doctor RPC performs complex operations on the storage engine.
Currently only duplicate removal is supported.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 17:25:50 +00:00
c236b54a65 [#212] reputationsvc: Resolve funlen linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Resolve funlen linter for Calculator.iterateDaughter method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
7ebbfa3358 [#212] reputationsvc: Resolve linters and rename
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
469e8a6e59 [#212] reputationsvc: Resolve containedctx linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
e2f13d03d7 [#222] auditsvc: Refactor PoR audit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Resolve funlen linter for Context.checkStorageGroupPoR method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 17:29:13 +03:00
e8d340287f [#222] auditsvc: Refactor audit task
Resolve containedctx linter. Cancel task by listen cancel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 17:29:13 +03:00
3dbff0a478 [#222] auditsvc: Resolve containedctx linter
Resolve containedctx linter for commonCommunicatorPrm type.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 17:29:13 +03:00
56282edf02 [#166] node: Parallelize background tree service sync
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-04-07 04:43:32 +00:00
4ec69cbbf8 [#203] control: Fix test
Fix shard compare loop.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
9e2df4b7c7 [#203] node: Fix double imports
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
c58ab0c369 [#193] getsvc: Reduce private key requests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Get private key only once for request forwaring.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
89924071cd [#193] getsvc: Edit request forwarder signature
Pass context to forwarder direct, without closure.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
6c7b708a98 [#193] getsvc: Refactor get range params creation
Resolve funlen linter for toRangePrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
b0786d2e5c [#193] getsvc: Refactor get params creation
Resolve funlen linter for toPrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
f889893216 [#193] getsvc: Refactor head param creation
Resolve funlen linter for toHeadPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
91ead04fa4 [#193] getsvc: Resolve funlen linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
1bf21dbb47 [#193] getsvc: Resolve context linters
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:48 +00:00
206458c841 [#217] containersvc: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:00 +00:00
279261ace3 [#217] containersvc: Refactor route passing
Resolve containedctx for routeCtx.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:00 +00:00
d6486d172e [#210] policier: Refactor nodes processing
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
Resolve funlen linter for processNodes method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:55:52 +03:00
080be5cfcd [#210] policier: Refactor object placement
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:55:52 +03:00
23575e1ac0 [#210] policier: Resolve contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:55:52 +03:00
d85703a963 [#208] searchsvc: Refactor request forwarding
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Resolve funlen & gocognit linters for toPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 16:24:55 +03:00
0b38419fbf [#208] searchsvc: Resolve context linters
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 16:24:50 +03:00
5f2a1531fe [#208] deletesvc: Resolve containedctx linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 16:24:40 +03:00
4941926c9d [#207] aclsvc: Drop outdated tag
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 13:22:14 +00:00
585415fa92 [#207] aclsvc: Refactor send checker
Resolve funlen linter for putStreamBasicChecker.Send method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 13:22:14 +00:00
9ef790f782 [#207] aclsvc: Refactor object headers read
Resolve funlen linter for readObjectHeaders method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 13:22:13 +00:00
cd33a57f44 [#207] aclsvc: Refactor EACL check
Resolve funlen linter for CheckEACL method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-04 13:22:13 +00:00
ece6c820e7 [#199] putsvc: Refactor streamer initialization
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-03 15:58:11 +00:00
27bdddc48f [#199] putsvc: Refactor put object
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-03 15:58:11 +00:00
cecea8053a [#199] putsvc: Refactor streamer pool
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-03 15:58:11 +00:00
14d894178e [#199] putsvc: Refactor placement iterator
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-03 15:58:11 +00:00