TrueCloudLab
/
frostfs-node
19
1
Fork 20
Code Issues 93 Pull Requests 9 Actions Releases Activity
4,609 Commits
8 Branches
14 Tags
44 MiB
Commit Graph

50 Commits (v0.38.5)

Author SHA1 Message Date
Dmitrii Stepanov d6534fd755 [#1016] frostfs-node: Fix gopls issues 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-01 12:13:43 +03:00
Evgenii Stratonikov dad56d2e98 Revert "[#972] Use min/max builtins" 
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 89784b2e0a [#972] Use min/max builtins 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 13:13:09 +00:00
Dmitrii Stepanov 11add38e87 [#857] golangci: Add protogetter linter
DCO action / DCO (pull_request) Successful in 1m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m2s Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s Details
Build / Build Components (1.20) (pull_request) Successful in 4m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m17s Details
Tests and linters / Lint (pull_request) Successful in 6m11s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m22s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 26e4f7005c [#741] treesvc: Refactor tree sync 
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov b21be1abdd [#741] treesvc: Do not update sync height if some node is unavailable 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov b215817e14 [#741] treesvc: Remove unused height variables 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov 79088baa06 [#772] node: Apply gofumpt 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:03 +03:00
Airat Arifullin 4ea0df77d0 [#574] policer: Check if the container was really removed 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-28 14:21:38 +00:00
Airat Arifullin 554ff2c06b [#574] core: Extend Source interface with DeletionInfo method 
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-28 14:21:38 +00:00
Airat Arifullin 23be3eb627 [#574] tree: Check if container is really removed
Vulncheck / Vulncheck (pull_request) Successful in 2m37s Details
Build / Build Components (1.21) (pull_request) Successful in 3m36s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m36s Details
DCO action / DCO (pull_request) Successful in 4m16s Details
Tests and linters / Lint (pull_request) Successful in 5m34s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m1s Details 
* Use DeletionInfo method from morph client to check if
  the container has been really removed from neo-go

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-08-21 12:50:20 +03:00
Dmitrii Stepanov 957a43a124 [#266] services/tree: Add sync check
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Do not accept requests until initial sync is finished.
`Apply` is deliberately left out -- we don't want to miss anything new.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 10:00:45 +00:00
Dmitrii Stepanov dcdfb6ed41 [#412] node: Use observability interceptors 
Use metrics and tracing interceptors.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Alejandro Lopez bc34fee6a7 [#370] Add tree service metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-26 13:39:12 +00:00
Dmitrii Stepanov 529d0bc710 [#302] tree: Drop unused ctx
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-02 16:54:06 +03:00
Airat Arifullin 9d01029733 [#166] node: Parallelize background tree service sync by batching
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Merge operations

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-26 10:17:56 +00:00
Airat Arifullin 299b24b974 [#166] node: Parallelize background tree service sync by batching 
* Concurrently dispatch TreeApply operations for batching in forest

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-26 10:17:56 +00:00
Dmitrii Stepanov 6121b541b5 [#242] treesvc: Add tracing spans
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Evgenii Stratonikov 0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Dmitrii Stepanov 5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Airat Arifullin 56282edf02 [#166] node: Parallelize background tree service sync
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-07 04:43:32 +00:00
Dmitrii Stepanov 9e2df4b7c7 [#203] node: Fix double imports 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-06 16:33:36 +03:00
Evgenii Stratonikov 5368c4207a [#156] services/tree: Split syncLoop() in functions 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Evgenii Stratonikov 47e8c5bf23 [#156] pilorama: Remove CIDDescriptor from TreeApply() 
Initially it was there to check whether an update is being initiated by
a proper node. It is now obsolete for 2 reasons:
1. Background synchronization fetches all operations from a single node.
2. There are a lot more problems with trust in the tree service, it is
   only used in controlled environments.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Dmitrii Stepanov 481a1ca6f3 [#148] linter: Add gocognit linter 
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov 97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Evgenii Stratonikov 6cd806f998 [#82] services/tree: Save last synchronized height in a persistent storage 
Remember the last synchronized height and use it after service restart.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-13 11:25:44 +00:00
Evgenii Stratonikov f2250a316f [#129] tree: Do not remove tree if the netmap is empty 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-10 10:59:15 +00:00
Alexey Vanin 20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Evgenii Stratonikov 46c62be7e8 [#28] Fix linter issues 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov b4e90cdf51 [#2165] pilorama: Optimize `TreeApply` when used for synchronization 
Because synchronization _most likely_ will have apply already existing
operations, it is much faster to check their presence in a read
transaction. However, always doing this will degrade the perfomance
for normal `Apply`. And, let's be honest, it is already not good.
Thus we add a separate parameter which specifies whether this logic is
enabled.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov f9fcd85363 [#2165] services/tree: Remember starting height for the synchronization 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 06137dbf8e [#2165] services/tree: Do not export `synchronizeAllTrees` 
It is used only privately.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov c299b98afe [#2165] services/tree: Parallelize synchronization 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov de9957e076 [#2165] services/tree: Always synchronize all containers 
In case of split-brain we must synchronize everything.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 9e0decd12d [#2162] services/tree: Close connection after the syncronization 
There was a goroutine leak here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 3bb5a320d7 [#2154] services/tree: Do not log an error when synchronizing container of 1 node 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 387d1e2977 [#2127] services/tree: Randomize node order for synchronization 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Anton Nikiforov 9a20498f34 [#1940] Removing all trees by container ID if tree ID is empty in `pilorama.Forest.TreeDrop` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-19 11:01:04 +03:00
Pavel Karpy e1be0180f6 [#1329] tree: Sync trees when a node first time appears in a container 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-21 18:47:56 +03:00
Pavel Karpy 1766ca2039 [#1902] tree: Allow synchronize all the container trees 
Add `SynchronizeAllTrees` method of the Tree service. It allows fetching
tree IDs and sync all of them. Share common logic b/w the new method and
the `SynchronizeTree`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy bc67f77d86 [#1329] tree: Fix sync applying 
Add the node position in a container and the container size to the CID
descriptor that is passed to the `TreeApply`. Previously, `checkValid` does
not allow any log operarations.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 20:53:34 +03:00
Pavel Karpy f5f560d903 [#1329] tree: Do not sync trees that are from external containers 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 20:53:34 +03:00
Pavel Karpy 13c4a9f4b8 [#1332] tree: Make `SignMessage` public 
It will allow reusing signing routine in other components
(e.g. `neofs-cli`).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Evgenii Stratonikov 8a47f9ba12 [#1753] services/tree: Remove MaxGetSubTreeCount constant 
It was used only in `GetOpLog` and ignored on the server side.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 10:52:37 +03:00
Evgenii Stratonikov 4558f30575 [#1644] services/tree: Do not use deprecated gRPC options 
Get rid of `grpc.WithInsecure`.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-30 18:39:08 +03:00
Evgenii Stratonikov 39f47f61c6 [#1445] services/tree: Cache the list of container nodes 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov 4437cd7113 [#1442] pilorama: Generate timestamp based on node position in the container 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov f4bc1c601a [#1329] services/tree: Synchronize from the last stored height 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00