TrueCloudLab
/
frostfs-node
17
1
Fork 21
Code Issues 86 Pull Requests 6 Actions Projects Releases Activity
4,047 Commits
7 Branches
24 Tags
49 MiB
Commit Graph

269 Commits (9ca63ac8c35f15075597eb7264b7331a72d6dc0e)

Author SHA1 Message Date
Dmitrii Stepanov b4e72a2dfd [#335] treesvc: Sort nodes by Filename in GetSubTree 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-20 10:14:10 +03:00
Dmitrii Stepanov 3e8de14e7d [#382] evacuate: Fix unit tests
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-17 10:24:05 +00:00
Alexander Chuprov 033eaf77e1 [#496] node: Fix linter importas
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m1s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s Details
Tests and linters / Lint (pull_request) Successful in 14m40s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Dmitrii Stepanov 43d263c3d5 [#428] linter: Fix unkeyed assignment
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Thanks to gopls.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-26 13:29:39 +00:00
Dmitrii Stepanov cac4ed93d6 [#428] engine: Add low_mem config parameter 
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-26 13:29:39 +00:00
Dmitrii Stepanov 059e9e88a2 [#373] metabase: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 028d4a8058 [#373] blobovnicza: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 01a0c97760 [#453] engine: Set Disabled mode to deleted shard
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-20 12:04:07 +03:00
Evgenii Stratonikov 69df0d21c2 [#446] engine: Move to read-only on blobstor errors
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-16 14:53:32 +03:00
Evgenii Stratonikov 20b84f183a [#446] engine: Simplify logs for shard mode change 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-16 14:51:29 +03:00
Dmitrii Stepanov 4449006862 [#424] metrics: Use mode value as metric value for shard 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-14 18:26:19 +03:00
Dmitrii Stepanov 1b364d8cf4 [#424] metrics: Refactor engine metrics 
Use histogram vector to measure request duration.
Fix naming like in Prometheus best practice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-14 14:53:32 +03:00
Dmitrii Stepanov 85deb12f4d [#424] writecache: Drop metrics when close 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-14 14:52:40 +03:00
Dmitrii Stepanov 07f155ac77 [#424] metrics: Use labels for writecache methods and operations 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-14 14:52:40 +03:00
Dmitrii Stepanov 2541d319de [#266] pilorama: Allow to get current tree height 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 10:00:45 +00:00
Evgenii Stratonikov 41ab4d070e [#423] *: Use hrw.StringHash() where possible
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-13 07:18:25 +00:00
Anton Nikiforov 263c6fdc50 [#372] node: Add metrics for the error counter in the engine
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-06-07 13:04:47 +00:00
Dmitrii Stepanov 74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Dmitrii Stepanov dbf41391b5 [#401] engine: Extend evacuation logs
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Add operation-tag to logger.
Log evacuation results.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-31 13:24:30 +03:00
Dmitrii Stepanov 3220c4df9f [#376] metrics: Add GC metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-31 10:22:12 +00:00
Alejandro Lopez faca861451 [#411] Remove unnecessary pointers for sync objects
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-31 10:19:14 +00:00
Dmitrii Stepanov 271a56c2ab [#395] metrics: Drop redundant metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details 
HistogramVec already has labeled counter.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-26 13:39:37 +00:00
Dmitrii Stepanov 2ce43935f9 [#312] metrics: Add writecache metrcis 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-24 10:18:39 +00:00
Evgenii Stratonikov 4b768fd115 [#381] *: Move to sync/atomic 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-23 08:18:01 +03:00
Dmitrii Stepanov e4889e06ba [#329] node: Make evacuate async 
Now it's possible to run evacuate shard in async.
Also only one evacuate process can be in progress.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-19 08:43:52 +00:00
Evgenii Stratonikov 945454f60c [#321] engine/test: Execute tests in parallel 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-12 09:45:03 +00:00
Alejandro Lopez 973af12854 [#327] tests: replace os.MkdirTemp with t.TempDir
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-05 11:24:51 +00:00
Dmitrii Stepanov eca5c210dd [#299] evacuate: Add context cancel checks
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-02 15:55:30 +03:00
Airat Arifullin ada081dfd5 [#19] node: Make policier read shards concurrently
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Introduce ListWithMultiCursor that simultaneously reads objects
  from different shards

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-27 11:11:47 +03:00
Evgenii Stratonikov 8466894fdf [#250] control: remove `DumpShard` and `RestoreShard` RPC
ci/woodpecker/push/pre-commit Pipeline was successful Details 
We have `Evacuate` with a cleaner interface.
Also, remove them from CLI and engine.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 12:28:49 +00:00
Dmitrii Stepanov 6121b541b5 [#242] treesvc: Add tracing spans
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Dmitrii Stepanov d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Evgenii Stratonikov 0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Dmitrii Stepanov 0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Evgenii Stratonikov dbc3811ff4 [#191] engine: Allow to remove redundant object copies 
RemoveDuplicates() removes all duplicate object copies stored on
multiple shards. All shards are processed and the command tries to leave
a copy on the best shard according to HRW.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-07 17:25:50 +00:00
Airat Arifullin ab891517de [#116] node: Fix bug with extra generated files in TestReload
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Create testNewEngine in engineWithShards without default opts

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-05 18:49:34 +03:00
Airat Arifullin 6f7b6a8813 [#116] node: Improve shard/engine construction in tests
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Introduce testEngineWrapper that can be constructed with different options

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-05 14:36:40 +00:00
Dmitrii Stepanov 23575e1ac0 [#210] policier: Resolve contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov 5a66db80c5 [#188] engine: Refactor shard evacuation 
Resolve funlen and gocognit linter for StorageEngine.Evacuate method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov 456bc097f7 [#188] engine: Refactor get range from engine 
Resolve funlen linter for StorageEngine.getRange method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov 3010ca2649 [#188] engine: Refactor get object from engine 
Resolve funlen linter for StorageEngine.get method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov a7c79c773a [#168] node: Refactor node config 
Resolve containedctx linter for cfg

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-31 09:32:59 +03:00
Airat Arifullin 7be5a0fd79 [#189] node: Fix bug with sync/atomic package
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-30 14:50:43 +00:00
Dmitrii Stepanov 9f0bce5c15 [#183] gc: Fix drop expired locked simple objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-30 15:33:42 +03:00
Alejandro Lopez 341fe1688f [#139] test: Add test storage implementation
ci/woodpecker/push/pre-commit Pipeline was successful Details 
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-29 14:28:49 +00:00
Pavel Karpy 97e201993b [#184] *: Unify done contexts handling 
If `ctx.Done()`, return `ctx.Err()` in every function that returns an error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-29 10:34:54 +00:00
Airat Arifullin 221203beeb [#180] node: Refactor panics in unit test
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Airat Arifullin 9808dec591 [#86] node: Move testing utils to one package
ci/woodpecker/push/pre-commit Pipeline failed Details 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 08:19:15 +00:00
Evgenii Stratonikov 47e8c5bf23 [#156] pilorama: Remove CIDDescriptor from TreeApply() 
Initially it was there to check whether an update is being initiated by
a proper node. It is now obsolete for 2 reasons:
1. Background synchronization fetches all operations from a single node.
2. There are a lot more problems with trust in the tree service, it is
   only used in controlled environments.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Dmitrii Stepanov 481a1ca6f3 [#148] linter: Add gocognit linter 
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00