acid-ant/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
4606 commits 110 branches 26 tags 20 MiB
Commit graph

325 commits

Author SHA1 Message Date
Evgenii Stratonikov
c1d90f018b [#1074] pilorama: Allow empty filenames in SortedByFilename() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-04 10:27:45 +00:00
Ekaterina Lebedeva
d5194ab2a6 [#949] metabase: fix shard.UpdateID() 
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-04-01 17:27:34 +03:00
Evgenii Stratonikov
e12fcc041d [#1059] services/tree: Fast sorted listing 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-01 12:37:34 +00:00
Evgenii Stratonikov
f23e38c285 Revert "[#446] engine: Move to read-only on blobstor errors" 
This reverts commit 69df0d21c2.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-01 12:48:30 +03:00
Dmitrii Stepanov
942d83611b [#874] engine: Revert Check object existance concurrently 
This reverts commit f526f49995.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-01 08:42:34 +00:00
Dmitrii Stepanov
31e2396a5f [#1043] control: Add ResetEvacuationStatus implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-13 10:29:45 +00:00
Dmitrii Stepanov
d6534fd755 [#1016] frostfs-node: Fix gopls issues 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-01 12:13:43 +03:00
Dmitrii Stepanov
2ad433dbcb [#1005] engine: Drop shards weights 
Unused.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-26 17:25:05 +03:00
Dmitrii Stepanov
7470c383dd [#997] metabase: Drop toMoveIt bucket 
It is not used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-21 10:06:05 +03:00
Evgenii Stratonikov
dad56d2e98 Revert "[#972] Use min/max builtins" 
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
7627d08914 Revert "[#972] Use require.ElementsMatch() where possible" 
This reverts commit 6d9707ff1f.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
6d9707ff1f [#972] Use require.ElementsMatch() where possible 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
89784b2e0a [#972] Use min/max builtins 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 13:13:09 +00:00
Anton Nikiforov
0bd030507e [#948] metrics: Set actual value for shard_id after restart 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-13 09:43:21 +03:00
Dmitrii Stepanov
db67c21d55 [#947] engine: Evacuate trees to remote nodes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
728150d1d2 [#947] engine: Evacuate trees to local shards 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
15d853ea22 [#947] controlSvc: Return tree evacuation stat 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:39 +03:00
Dmitrii Stepanov
b3f3505ada [#947] cli: Allow to specify evacuation scope 
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
a6eb66bf9c [#947] evacuate: Refactor evacuate parameters 
Drop methods to make it easier to extend.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
9ba48c582d [#917] engine: Allow to detach shards 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-06 14:49:47 +03:00
Anton Nikiforov
d0eadf7ea2 [#799] engine: Skip put when object removed from shard 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-01 17:49:22 +00:00
Anton Nikiforov
e3573de6db [#930] gc: Stop internal activity by context 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-01-31 08:30:34 +00:00
Dmitrii Stepanov
c681354afd [#938] engine: Fix container count removal 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-30 12:37:48 +03:00
Dmitrii Stepanov
931a5e9aaf [#918] engine: Move shard to degraded mode if metabase open failed 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-23 11:16:40 +03:00
Dmitrii Stepanov
f526f49995 [#874] engine: Check object existance concurrently 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-23 09:28:29 +03:00
Dmitrii Stepanov
f5160b27fc [#920] tests: Fix data races 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-19 14:06:05 +03:00
Dmitrii Stepanov
63d3ed1ad8 [#904] tests: Close test engine after test 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-17 19:04:39 +03:00
Dmitrii Stepanov
a2ab373a0a [#895] metabase: Do not delete GC mark for virtual objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
7166e77c2b [#895] test: Add logger to test shard 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
47dcfa20f3 [#895] test: Use t.Cleanup only for external resources 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
f1b2b8bffa [#895] test: Fix NewLogger arguments list 
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
4b8b4da681 [#864] engine: Drop container count metric if container removed 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-10 10:45:32 +03:00
Dmitrii Stepanov
d75e7e9a21 [#864] engine: Drop container size metric if container deleted 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-10 10:44:54 +03:00
Dmitrii Stepanov
581887148a [#569] cli: Add control shards writecache seal command 
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov
0cb0fc1735 [#569] writecache: Allow to seal writecache after flush 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov
8180a0664f [#887] node: Drop badger writecache implementation 
Badger implementation isn't tested and works not well,
but requires human resources to maintain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-22 13:00:54 +03:00
Dmitrii Stepanov
f1c7905263 [#661] blobovniczatree: Make Rebuild concurrent 
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov
d30ab5f29e [#838] metabase: Count user objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Alexander Chuprov
b2c63e57ba [#651] engine/test: Speedup StorageEngine_Inhume 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-30 13:19:43 +00:00
Dmitrii Stepanov
1b22801eed [#788] engine: Fix flaky tests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-09 10:25:46 +03:00
Dmitrii Stepanov
70ab1ebd54 [#763] metrics: Add container_objects_total metric 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-08 12:30:57 +03:00
Dmitrii Stepanov
226e84d782 [#684] node: Add skipped objects count to evacuation result 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-07 12:17:11 +00:00
Dmitrii Stepanov
79088baa06 [#772] node: Apply gofumpt 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:03 +03:00
Dmitrii Stepanov
d4b6ebe7e7 [#725] writecache: Fix metric values 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-27 12:22:29 +03:00
Alexander Chuprov
c1e4130020 [#146] node: Add trace_id to logs 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-09-27 11:05:27 +03:00
Evgenii Stratonikov
c6af4a3ec8 [#679] engine: Do not increase error counter on meta mismatch 
It was introduced in 69e1e6ca to help node determine faulty shards.
However, the situation is possible in a real-life scenario:
1. Object O is evacuated from shard A to B.
2. Shard A is unmounted because of lower-level errors.
3. We now have object in meta on A and in blobstor on B. Technically we
   have it in meta on shard B too, but we still got the error if B goes
   to a degraded mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-14 10:39:18 +03:00
Ekaterina Lebedeva
8a81af5a3b [#653] Add context parameter to Open functions 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-09-07 18:03:29 +03:00
Evgenii Stratonikov
96e690883f [#638] Unify test loggers 
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
322c1dc273 [#638] Use test.NewLogger() in tests 
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Alexander Chuprov
4d9a6c07fb [#618] core: Replace fmt.Sprintf with strconv.FormatUint 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-21 07:14:50 +00:00