TrueCloudLab
/
frostfs-node
18
1
Fork 21
Code Issues 89 Pull Requests 8 Actions Releases Activity
4,603 Commits
7 Branches
23 Tags
48 MiB
Commit Graph

1084 Commits (1eb47ab2cec34b7674bbdcfc6b1b32b2e1f0cdf8)

Author SHA1 Message Date
Evgenii Stratonikov b8bcfac531 [#510] treesvc: Fix panic in bearer token processing 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-12 10:23:21 +00:00
Dmitrii Stepanov 24eb988897 [#294] deletesvc: Drop cast
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov c83e7c875f [#294] searchsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov e8091101c7 [#294] searchsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov ec9b738465 [#294] putsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 800a685e84 [#294] putsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 1420b8b9ea [#294] deletesvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov a476d8285a [#294] deletesvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 70a1081988 [#294] aclsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 18d8898b00 [#294] aclsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 61541eaec2 [#294] aclsvc: Refactor checker constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Evgenii Stratonikov 7da284f3e8 [#509] go.mod: Update sdk-go
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Lint (pull_request) Successful in 2m15s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m7s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m50s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details
Build / Build Components (1.19) (pull_request) Successful in 3m47s Details
Build / Build Components (1.20) (pull_request) Successful in 3m30s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 20m39s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-11 17:38:00 +03:00
Evgenii Stratonikov 80481c015c [#509] putsvc: Omit AccessIdentifiers from iteratePlacement() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-11 17:38:00 +03:00
Dmitrii Stepanov a65e26878b [#486] put service: Fix error typo
ci/woodpecker/pr/pre-commit Pipeline failed Details
Build / Build Components (1.19) (pull_request) Successful in 4m12s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m52s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m49s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m13s Details
Tests and linters / Lint (pull_request) Successful in 10m50s Details
Build / Build Components (1.20) (pull_request) Successful in 11m28s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m54s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Dmitrii Stepanov fcbf90d31b [#486] node: Add PutSingle implemetation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Dmitrii Stepanov 7b76527759 [#486] node: Add PutSingle wrappers 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Evgenii Stratonikov 140d970a95 [#498] policer: Fix objectSDK import
ci/woodpecker/pr/pre-commit Pipeline failed Details
Build / Build Components (1.19) (pull_request) Successful in 4m23s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 3m27s Details
Tests and linters / Lint (pull_request) Successful in 4m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Build / Build Components (1.20) (pull_request) Successful in 10m18s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m41s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m33s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-07 18:52:40 +03:00
Evgenii Stratonikov 2310a5c7ba [#498] policer: Allow to set sleep duration between iterations
ci/woodpecker/push/pre-commit Pipeline failed Details 
Speed up tests on CI.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-07 14:52:33 +00:00
Evgenii Stratonikov e858479a74 [#498] policer: Explicitly Rewind() iterator after finish 
Previously, we can continue to return `EndOfListing` infinitely.
Reflect iterator reuse via Rewind() method.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-07 14:52:33 +00:00
Alexander Chuprov 033eaf77e1 [#496] node: Fix linter importas
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m1s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s Details
Tests and linters / Lint (pull_request) Successful in 14m40s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Evgenii Stratonikov 6eefe9747e treesvc: Do not provide credentials unless TLS is used
ci/woodpecker/push/pre-commit Pipeline failed Details 
4f413fe86e was perfect, except it did the opposite of what we needed.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-06 08:04:28 +00:00
Alexey Vanin f354b8a270 [#473] Add tests for rem values of traverser
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m8s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Lint (pull_request) Successful in 9m3s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m27s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m18s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 13m54s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-07-05 20:13:23 +03:00
Alexey Vanin c6df6c84ae [#473] Do not modify traverse plan when `[0]` copies number vector is provided 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-07-05 19:54:43 +03:00
Anton Nikiforov 90e9a85acc [#483] Update dependencies 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-07-03 15:44:59 +03:00
Alejandro Lopez 26acf5689e [#92] Ensure policer objects cannot be worked on concurrently
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-03 07:05:31 +00:00
Alejandro Lopez 3223402c90 [#92] Embed policer's objectsInWork mutex 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-03 07:05:31 +00:00
Alejandro Lopez f9730f090d [#92] Refactor policer and add some unit tests 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-03 07:05:31 +00:00
Airat Arifullin f437ab8f15 [#197] object: Make Delete method return correct status
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-06-30 12:58:56 +00:00
Evgenii Stratonikov d0ab552a90 [#478] *: Fix funlen linter warnings
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-28 15:01:49 +00:00
Evgenii Stratonikov 8a4e250dae [#468] *: replace outdated TODO crypto-related links
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-28 12:13:20 +00:00
Evgenii Stratonikov 4f413fe86e [#1] treesvc: Properly check for secure transport 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-28 12:13:20 +00:00
Evgenii Stratonikov f4c71cea65 [#1] *: Replace outdated FIXME/TODO links 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-28 12:13:20 +00:00
Evgenii Stratonikov 0c866f62c5 [#473] placement: Fix backwards compatibility for `copies_number`
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-27 11:13:16 +03:00
Anton Nikiforov 71889234b7 [#449] tree: Allow reading requests signed by keys from allow list 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-06-23 14:44:19 +03:00
Evgenii Stratonikov 40eae22109 [#460] services/util: Remove `CreateRequestStreamer`
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-21 17:08:04 +03:00
Evgenii Stratonikov a64dc9ad70 [#460] services/util: Remove `HandleServerStreamRequest` 
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-21 17:08:01 +03:00
Evgenii Stratonikov 167a67f0b8 [#460] services/util: Remove `HandleUnaryRequest` 
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-21 17:07:56 +03:00
Evgenii Stratonikov 785d81a68a [#460] services/object: Reduce `distibutedTarget` memory footprint 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-21 17:07:50 +03:00
Dmitrii Stepanov 059e9e88a2 [#373] metabase: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Anton Nikiforov 50caa388b0 [#303] ir: Use pub key when validate container deletion
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-06-20 08:02:48 +00:00
Dmitrii Stepanov c348ae35b0 [#424] metrics: Drop embedded metrics 
It was not obvious where metrics are used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-14 14:53:32 +03:00
Alejandro Lopez 4887f489a1 [#17] Add morph client metrics
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-06-13 14:06:04 +03:00
Airat Arifullin 90e9247b69 [#371] ir: Add morph cache metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-06-13 10:10:13 +00:00
Dmitrii Stepanov 957a43a124 [#266] services/tree: Add sync check
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Do not accept requests until initial sync is finished.
`Apply` is deliberately left out -- we don't want to miss anything new.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 10:00:45 +00:00
Dmitrii Stepanov e69a1e8482 [#266] services/tree: Return operation log up to some height 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 10:00:45 +00:00
Dmitrii Stepanov 0400153b7d [#424] object: Do not store large slices in pool 
Dynamically growing an unbounded buffers can cause a large
amount of memory to be pinned and never be freed.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 07:19:55 +00:00
Dmitrii Stepanov dcdfb6ed41 [#412] node: Use observability interceptors 
Use metrics and tracing interceptors.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Dmitrii Stepanov 74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Alejandro Lopez faca861451 [#411] Remove unnecessary pointers for sync objects
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-31 10:19:14 +00:00
Alejandro Lopez bc34fee6a7 [#370] Add tree service metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-26 13:39:12 +00:00
Dmitrii Stepanov 9119199f6e [#397] cli: Fix evacuation method names
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-26 11:28:30 +03:00
Dmitrii Stepanov f7c0b50d70 [#329] cli: Add async evacuate commands 
Add start, stop evacuate and evacuate status commands.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-19 08:43:52 +00:00
Dmitrii Stepanov e4889e06ba [#329] node: Make evacuate async 
Now it's possible to run evacuate shard in async.
Also only one evacuate process can be in progress.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-19 08:43:52 +00:00
Dmitrii Stepanov 100b1b5128 [#329] node: Add async evacuate proto methods 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-19 08:43:52 +00:00
Alejandro Lopez 13a7a90101 [#355] Increase tree svc client cache size to test hypotheses
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-17 14:11:46 +03:00
Alejandro Lopez a6ee7a3087 [#324] Add replicator metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-12 11:00:17 +00:00
Alejandro Lopez 90799497d3 [#114] Add remove-node IR control command
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-10 14:31:44 +00:00
Evgenii Stratonikov b2ffd7df53 [#291] object: Use PayloadSizeLimiter from SDK
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-05 19:07:06 +03:00
Evgenii Stratonikov 35ea207df6 [#291] object: Split validating target in two 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-05 19:07:06 +03:00
Pavel Karpy ee58b390bb [#221] node: Allow using vector `copies_number`
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Also, take into account that value in general (it was not used before at
all).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-05-05 16:07:13 +00:00
Pavel Karpy 479c5a65e1 [#322] node: Fix tree svc panic
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
If a connection has not been established earlier, it stores `nil` in LRU
cache. Cache eviction tries to close every connection (even a `nil` one) and
panics but not crash the app because we are using pools.
That ugly bug also leads to a deadlock where `Unlock` is not called via
`defer` func (and that is the way I found it).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-05-04 19:48:51 +03:00
Dmitrii Stepanov 529d0bc710 [#302] tree: Drop unused ctx
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-02 16:54:06 +03:00
Dmitrii Stepanov eca5c210dd [#299] evacuate: Add context cancel checks
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-02 15:55:30 +03:00
Dmitrii Stepanov 22d47376a6 [#277] getsvc: Refactor errors
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Move errors to separate files.
Use zap.Error for error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Dmitrii Stepanov 45438e7b06 [#277] getsvc: Rename and reorder code 
Rename execCtx to request.
Move code to appropriate files.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Dmitrii Stepanov 1440450606 [#277] getsvc: Drop cyclic struct dependency 
Drop cyclic dependency between execCtx and Service.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Dmitrii Stepanov 591c4e7d50 [#277] getsvc: Move headOnly to request params 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Dmitrii Stepanov 265d2326a0 [#277] getsvc: Extract remote storage 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Dmitrii Stepanov 30e1b62b67 [#277] getsvc: Fix service deps 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Dmitrii Stepanov 8fc082b688 [#277] getsvc: Do not return status error 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-28 14:03:12 +00:00
Evgenii Stratonikov f1ea8fec93 [#248] object_manager: Remove storage group 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-28 17:00:43 +03:00
Evgenii Stratonikov 7f49f07255 [#248] services: Remove audit service 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-28 17:00:43 +03:00
Evgenii Stratonikov f73ac6e02d [#290] control: Use generics for response wrappers
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-28 10:57:24 +03:00
Alejandro Lopez ff25521204 [#270] Add IR epoch tick control call
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-04-28 07:57:00 +00:00
Evgenii Stratonikov dfe4ada838 [#285] lint: Resolve revive/if-return 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-26 17:35:08 +03:00
Evgenii Stratonikov f07e2d4812 [#285] lint: Fix revive/unused-parameter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-26 17:35:04 +03:00
Airat Arifullin 9d01029733 [#166] node: Parallelize background tree service sync by batching
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Merge operations

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-26 10:17:56 +00:00
Airat Arifullin 299b24b974 [#166] node: Parallelize background tree service sync by batching 
* Concurrently dispatch TreeApply operations for batching in forest

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-26 10:17:56 +00:00
Denis Kirillov dce5924a89 [#229] services/tree: Use bearer owner as signer 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-04-26 10:23:33 +03:00
Alexey Vanin 89530534a1 [#229] service/tree: Disable container owner check in tree service 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-04-26 10:23:33 +03:00
Alexey Vanin c04f6c5e59 [#229] acl: Allow Impersonate 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-04-26 10:23:33 +03:00
Dmitrii Stepanov 3d43b0f7f9 [#265] node: Fix after SDK & API-Go version up
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-18 12:09:19 +03:00
Dmitrii Stepanov f07d4158f5 [#249] node: Drop subnet from IR and morph 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-17 09:46:02 +03:00
Evgenii Stratonikov 8466894fdf [#250] control: remove `DumpShard` and `RestoreShard` RPC
ci/woodpecker/push/pre-commit Pipeline was successful Details 
We have `Evacuate` with a cleaner interface.
Also, remove them from CLI and engine.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 12:28:49 +00:00
Pavel Karpy 560f73ab7e [#247] node, ir: Drop reputation related code 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:44:14 +03:00
Dmitrii Stepanov 6121b541b5 [#242] treesvc: Add tracing spans
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Dmitrii Stepanov d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Dmitrii Stepanov 200fc8b882 [#242] put: Pass context to relay function 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Evgenii Stratonikov cffcc7745e [#240] logs: Factor out common service log messages
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Evgenii Stratonikov 0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Evgenii Stratonikov 04727ce1d6 Revert "[#135] signature: Add tracing" 
This reverts commit 5778980252.
 2023-04-12 16:57:02 +03:00
Evgenii Stratonikov 08769f413f Revert "[#135] acl: Add tracing spans" 
This reverts commit b2ca730547.
 2023-04-12 16:54:13 +03:00
Dmitrii Stepanov 5778980252 [#135] signature: Add tracing
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov b2ca730547 [#135] acl: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov 0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov 5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov 93eba19a8e [#223] objectsvc: Refactor split-tree traverse 
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov 2ed9fd3f94 [#223] objectsvc: Refactor request parameters 
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov ccf8463e69 [#223] controlsvc: Drop unnecessary nolint 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Evgenii Stratonikov 0b9622c418 [#191] control: Add Doctor RPC 
Doctor RPC performs complex operations on the storage engine.
Currently only duplicate removal is supported.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-07 17:25:50 +00:00
Dmitrii Stepanov c236b54a65 [#212] reputationsvc: Resolve funlen linter
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Resolve funlen linter for Calculator.iterateDaughter method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:35:57 +00:00
Dmitrii Stepanov 7ebbfa3358 [#212] reputationsvc: Resolve linters and rename 
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:35:57 +00:00
Dmitrii Stepanov 469e8a6e59 [#212] reputationsvc: Resolve containedctx linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 15:35:57 +00:00
Dmitrii Stepanov e2f13d03d7 [#222] auditsvc: Refactor PoR audit
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Resolve funlen linter for Context.checkStorageGroupPoR method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 17:29:13 +03:00
Dmitrii Stepanov e8d340287f [#222] auditsvc: Refactor audit task 
Resolve containedctx linter. Cancel task by listen cancel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 17:29:13 +03:00
Dmitrii Stepanov 3dbff0a478 [#222] auditsvc: Resolve containedctx linter 
Resolve containedctx linter for commonCommunicatorPrm type.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-07 17:29:13 +03:00
Airat Arifullin 56282edf02 [#166] node: Parallelize background tree service sync
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-07 04:43:32 +00:00
Dmitrii Stepanov 4ec69cbbf8 [#203] control: Fix test 
Fix shard compare loop.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov 9e2df4b7c7 [#203] node: Fix double imports 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Dmitrii Stepanov c58ab0c369 [#193] getsvc: Reduce private key requests
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Get private key only once for request forwaring.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov 89924071cd [#193] getsvc: Edit request forwarder signature 
Pass context to forwarder direct, without closure.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov 6c7b708a98 [#193] getsvc: Refactor get range params creation 
Resolve funlen linter for toRangePrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov b0786d2e5c [#193] getsvc: Refactor get params creation 
Resolve funlen linter for toPrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov f889893216 [#193] getsvc: Refactor head param creation 
Resolve funlen linter for toHeadPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov 91ead04fa4 [#193] getsvc: Resolve funlen linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov 1bf21dbb47 [#193] getsvc: Resolve context linters 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:48 +00:00
Dmitrii Stepanov 206458c841 [#217] containersvc: Resolve containedctx linter
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:00 +00:00
Dmitrii Stepanov 279261ace3 [#217] containersvc: Refactor route passing 
Resolve containedctx for routeCtx.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:38:00 +00:00
Dmitrii Stepanov d6486d172e [#210] policier: Refactor nodes processing
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Resolve funlen linter for processNodes method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov 080be5cfcd [#210] policier: Refactor object placement 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov 23575e1ac0 [#210] policier: Resolve contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov d85703a963 [#208] searchsvc: Refactor request forwarding
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Resolve funlen & gocognit linters for toPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:55 +03:00
Dmitrii Stepanov 0b38419fbf [#208] searchsvc: Resolve context linters 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:50 +03:00
Dmitrii Stepanov 5f2a1531fe [#208] deletesvc: Resolve containedctx linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:40 +03:00
Dmitrii Stepanov 4941926c9d [#207] aclsvc: Drop outdated tag
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:14 +00:00
Dmitrii Stepanov 585415fa92 [#207] aclsvc: Refactor send checker 
Resolve funlen linter for putStreamBasicChecker.Send method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:14 +00:00
Dmitrii Stepanov 9ef790f782 [#207] aclsvc: Refactor object headers read 
Resolve funlen linter for readObjectHeaders method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:13 +00:00
Dmitrii Stepanov cd33a57f44 [#207] aclsvc: Refactor EACL check 
Resolve funlen linter for CheckEACL method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:13 +00:00
Dmitrii Stepanov ece6c820e7 [#199] putsvc: Refactor streamer initialization
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov 27bdddc48f [#199] putsvc: Refactor put object 
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov cecea8053a [#199] putsvc: Refactor streamer pool 
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov 14d894178e [#199] putsvc: Refactor placement iterator 
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov 56161d39b4 [#168] node: Refactor container services 
Resolve containedctx for remoteLoadAnnounceWriter struct

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-31 09:32:59 +03:00
Airat Arifullin 221203beeb [#180] node: Refactor panics in unit test
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Denis Kirillov 342e571d89 [#159] Add handle __SYSTEM__ sys attributes
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-22 17:35:20 +03:00
Dmitrii Stepanov 49234b915e [#155] search-service: Fix search with ST
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Search should return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-22 07:21:09 +00:00
Dmitrii Stepanov 1637a3edce [#155] search-service: Add search with ST test 
In case of session token (ST) with object IDs search should
return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-22 07:21:09 +00:00
Evgenii Stratonikov cbc2efb1d6 [#156] object/get: Make toHeadPrm() pass gocognit
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Evgenii Stratonikov 3f6b962349 [#156] services/tree: Pass context to replicationWorker() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Evgenii Stratonikov 5368c4207a [#156] services/tree: Split syncLoop() in functions 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Evgenii Stratonikov 47e8c5bf23 [#156] pilorama: Remove CIDDescriptor from TreeApply() 
Initially it was there to check whether an update is being initiated by
a proper node. It is now obsolete for 2 reasons:
1. Background synchronization fetches all operations from a single node.
2. There are a lot more problems with trust in the tree service, it is
   only used in controlled environments.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Dmitrii Stepanov 44b86bac5a [#148] linter: Add contextcheck linter
ci/woodpecker/push/full-pre-commit Pipeline failed Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov 481a1ca6f3 [#148] linter: Add gocognit linter 
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov 97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov cc8ff015b4 [#148] linter: Add containedctx linter 
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:52:39 +03:00
Pavel Karpy abd21f8099 [#136] Revert "[#2260] services/object: Do not assemble object with TTL=1" 
This reverts commit 2567f8020e. It assumes
that assembling logic could break some failover scenarios if request
forwarding is done. However, it also breaks requesting big objects via a
non-container node with TTL=2. Failover has been rechecked without that
commit and no problems were found. Any (if found) other bugs related to
the forwarding and object assembling must be solved more carefully.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-20 10:52:18 +03:00
Pavel Karpy 64bde68fb9 [#67] node: Accept expired locked objects 
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-16 16:22:19 +03:00
Evgenii Stratonikov 22be532cbd object/put: Persist session token till the end of a session 
Previously a token could've expired in the middle of an object.PUT
stream, leading to upload being interrupted. This is bad, because user
doesn't always now what is the right values for the session token
lifetime. More than that, setting it to a very high value will
eventually blow up the session token database.

In this commit we read the session token once and reuse it for the whole
stream duration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-16 06:45:50 +00:00
Dmitrii Stepanov ac0a278a05 [#85] get-service: Drop unused assemble flag 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov b8e93d4c08 [#85] get-service: Use assembler to assemble LOB 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov 07de839f18 [#85] get-service: Fix corrupted chain logic 
Should return an error in case of a broken LOB reference chain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov 2886b1581b [#85] get-service: Add unit tests 
Add unit tests to cover all assemble statements

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov 8b9e40a848 [#85] get-service: Add assembler 
Extract assemble logic to assembler

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Evgenii Stratonikov 6cd806f998 [#82] services/tree: Save last synchronized height in a persistent storage 
Remember the last synchronized height and use it after service restart.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-13 11:25:44 +00:00
Evgenii Stratonikov 6226c3ba86 [#129] policer: Use safer defaults 
If `processNodes` exits earlier for some reason, `needLocalCopy` could
be false.
See https://github.com/nspcc-dev/neofs-node/issues/2267

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-10 10:59:15 +00:00
Evgenii Stratonikov f2250a316f [#129] tree: Do not remove tree if the netmap is empty 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-10 10:59:15 +00:00
Alexey Vanin 20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Dmitrii Stepanov 6925fb4c59 [TrueCloudLab/hrw#2] node: Use typed HRW methods 
Update HRW lib and use typed HRW methods to sort shards and nodes

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-02-28 13:36:25 +03:00
Alejandro Lopez 73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-22 13:43:19 +03:00
Alejandro Lopez cb5468abb8 [#66] node: Replace interface{} with any 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-21 16:47:07 +03:00
Anton Nikiforov 22f3c7d080 [#1868] Reload config for pprof and metrics on SIGHUP 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov 2567f8020e [#2260] services/object: Do not assemble object with TTL=1 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Pavel Karpy 901d62567d [#57] node: Broadcast link objects 
It boosts object assembling by an _average_ container node.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-17 11:58:27 +03:00
Stanislav Bogatyrev cb016d53a6 [#1] Fix comments and error messages 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev c761a95eef [#1] Fix project name in control service 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Evgenii Stratonikov d65a95a2c6 [#28] pilorama: Remove `LogMove` struct 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 46c62be7e8 [#28] Fix linter issues 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 67c97c6804 [#2210] services/tree: Drop messages not in queue 
Currently, under high load clients are blocked on channel send
and the number of goroutines can increase indefinitely.
In this commit we drop replication messages if send/recv queue is full
and rely on a background synchronization.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 0d8366f475 [#2207] object/acl: Return status error for expired session token 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 6efa93be0a [#1621] services/tree: Return `Apply` result asyncronously 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 554b85411f [#2190] services/object: Log service error with INFO level 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 3d1d2ee7b1 [#11] Regenerate proto files 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-12 08:55:47 +03:00
Evgenii Stratonikov f0be0befc5 [#5] services/object_manager: Use generic LRU cache 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-31 23:04:06 +03:00
Evgenii Stratonikov 1b3374ac7f [#5] services/tree: User generic LRU cache 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-31 23:04:06 +03:00
Evgenii Stratonikov 8f61cc1dcc [#5] policer: Use generic LRU client 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-31 23:04:06 +03:00
Evgenii Stratonikov 6f5edac730 [#2164] network/cache: Do not reconnect to failed clients immediately 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov b4e90cdf51 [#2165] pilorama: Optimize `TreeApply` when used for synchronization 
Because synchronization _most likely_ will have apply already existing
operations, it is much faster to check their presence in a read
transaction. However, always doing this will degrade the perfomance
for normal `Apply`. And, let's be honest, it is already not good.
Thus we add a separate parameter which specifies whether this logic is
enabled.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov f9fcd85363 [#2165] services/tree: Remember starting height for the synchronization 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 06137dbf8e [#2165] services/tree: Do not export `synchronizeAllTrees` 
It is used only privately.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov c299b98afe [#2165] services/tree: Parallelize synchronization 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov de9957e076 [#2165] services/tree: Always synchronize all containers 
In case of split-brain we must synchronize everything.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 6a4e5e6f0a [#2144] node: Try node's private key if dynamic token fetching failed 
`GETRANGEHASH` request spawns `GETRANGE` requests if an object could not be
found locally. If the original request contains session, it can be static
and, therefore, fetching session key can not be performed successfully.
As the best effort a node could request object's range with its own key.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 86a4fba571 [#2144] node: Clarify `KeyStorage.GetKey` method 
Actualize the doc, fix API status error return.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 04b5ec759b [#2139] object/put: Use `sync.Pool` for temporary payloads 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 9e0decd12d [#2162] services/tree: Close connection after the syncronization 
There was a goroutine leak here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 306609030a [#2159] node: Add tree replication timeout configuration 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 3bb5a320d7 [#2154] services/tree: Do not log an error when synchronizing container of 1 node 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 387d1e2977 [#2127] services/tree: Randomize node order for synchronization 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov b207dc424f [#2158] policer: Reduce default cache size 
We use cache to avoid policing the same object multiple times in a short
time span (< 30 seconds). If we have 200_000 objects in a blobstor, it is a bit useless
-- if it takes 1 second to process an object and we have `replicator.pool_size: 20`
in config, the next iteration will happen in 10_000 second which is much
larger than 30 second. However we still consume a lot of memory, so it
makes sense to use saner default.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy b413094704 [#2095] node: Fix collecting child objects 
Stop child objects collection if the last returned object (the most "left"
object in the collected chain) starts exactly from the `GETRANGE`'s `from`
value.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 350eecfa13 [#2095] node: Do not allow `GETRANGE` requests with zero length 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Pavel Karpy d54022eacc [#2047] node: Do not send chunk twice on request forwarding 
That could happen if a node forwards request to a node that closed the
connection during the original object stream.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-02 11:27:48 +03:00
Evgenii Stratonikov bd25db5d4a [#1984] metrics: Use separate metrics for success/failed requests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-12-01 14:18:10 +03:00
Evgenii Stratonikov e21c472dc7 [#1984] services/object: Increase put_req_count after the request is processed 
As it is specified in metrics description.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-12-01 14:18:10 +03:00
Pavel Karpy 51963abce7 [#1972] node: Fix errors comments in the Put service 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 16:58:52 +03:00
Evgenii Stratonikov 660c38d07e [#2062] services/policer: Use a proper key for object cache 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov 1779664644 [#2058] services/policer: Fix panic in `shardPolicyWorker` 
```
2022/11/15 08:40:56 worker exits from a panic: runtime error: index out of range [0] with length 0
2022/11/15 08:40:56 worker exits from panic: goroutine 1188 [running]:
github.com/panjf2000/ants/v2.(*goWorker).run.func1.1()
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:58 +0x10c
panic({0x1042b60, 0xc0015ae018})
	runtime/panic.go:1038 +0x215
github.com/nspcc-dev/neofs-node/pkg/services/policer.(*Policer).shardPolicyWorker.func1()
	github.com/nspcc-dev/neofs-node/pkg/services/policer/process.go:65 +0x366
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
created by github.com/panjf2000/ants/v2.(*goWorker).run
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x68
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Anton Nikiforov 9a20498f34 [#1940] Removing all trees by container ID if tree ID is empty in `pilorama.Forest.TreeDrop` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-19 11:01:04 +03:00
Pavel Karpy 634792077e [#1502] node: Store lock object on every container node 
Includes extending listing methods in the Storage Engine with object types.
It allows tuning replication/policer algorithms: container nodes do
not remove `LOCK` objects as redundant and try to fulfill `LOCK` placement
on the ohter container nodes.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00