dstepanov-yadro
/
frostfs-node
forked from TrueCloudLab/frostfs-node
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,614 Commits
216 Branches
26 Tags
30 MiB
Commit Graph

169 Commits (feat/resync_speedup)

Author SHA1 Message Date
Anton Nikiforov 3195142d67 [#959] writecache: Avoid manipulation with cache in `DEGRADED` mode 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-11 18:35:41 +00:00
Dmitrii Stepanov 66a26b7775 [#973] node: Resolve revive: unused-parameter linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-11 17:11:49 +03:00
Anton Nikiforov 05b5f5ca85 [#959] writecache: Fix panic on `Get` when it is not initialized 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-14 16:10:33 +03:00
Anton Nikiforov 0bd030507e [#948] metrics: Set actual value for `shard_id` after restart 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-13 09:43:21 +03:00
Anton Nikiforov d19ade23c8 [#959] node: Set mode to shard's components when open it 
Avoid opening database for `metabase` and `cache` in `Degraded` mode.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-09 14:04:01 +00:00
Dmitrii Stepanov 47dcfa20f3 [#895] test: Use t.Cleanup only for external resources 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov f1b2b8bffa [#895] test: Fix NewLogger arguments list 
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov 581887148a [#569] cli: Add `control shards writecache seal` command 
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 7a9db5bcdd [#569] writecache: Do not wait modeMtx if mode changes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 32c282ca10 [#569] writecache: Refactor flush 
Make single RUnlock call instead of two.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 0cb0fc1735 [#569] writecache: Allow to seal writecache after flush 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 8180a0664f [#887] node: Drop badger writecache implementation 
Badger implementation isn't tested and works not well,
but requires human resources to maintain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-22 13:00:54 +03:00
Dmitrii Stepanov 94ffe8bb45 [#857] golangci: Add testifylint linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 8d18fa159e [#667] writecache: Fix flush test 
Allow to disable background flusher for testing purposes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 17:45:43 +03:00
Dmitrii Stepanov 76ff26039c [#96] node: Drop neo-go's `slices` package 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 13:24:04 +03:00
Dmitrii Stepanov 137e987a4e [#655] storage: Drop `LazyHandler` 
LazyHandler is implemented and used incorrectly.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-16 17:27:38 +03:00
Dmitrii Stepanov 79088baa06 [#772] node: Apply gofumpt 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:03 +03:00
Dmitrii Stepanov 869518be0a [#728] writecache: Fix Badger writecache race. 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-30 18:36:41 +03:00
Dmitrii Stepanov d4b6ebe7e7 [#725] writecache: Fix metric values 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-27 12:22:29 +03:00
Anton Nikiforov 559ad58ab1 [#642] writecache: Remove usage of close channel in bbolt 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-24 15:57:50 +00:00
Anton Nikiforov c0b86f2d93 [#642] writecache: Remove usage of close channel in badger 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-24 15:57:50 +00:00
Dmitrii Stepanov f26233b47a [#734] metabase: Include UpdateStorageID in metrics and traces 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-16 17:00:18 +03:00
Dmitrii Stepanov d07afd803c [#726] writecache: Fix small object flush for Badger 
Do not marshal object twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-06 11:32:50 +03:00
Dmitrii Stepanov 997ac7cd8d [#726] writecache: Fix small object flush for BBolt 
Do not marshal object twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-06 11:32:44 +03:00
Airat Arifullin 3889e829e6 [#667] writecache: Add logs for report error func in tests 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-09-14 17:02:54 +00:00
Ekaterina Lebedeva 8a81af5a3b [#653] Add context parameter to Open functions 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-09-07 18:03:29 +03:00
Anton Nikiforov 4dff9555f1 [#568] writecache: Improve flushing scheme for badger 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-30 17:22:28 +00:00
Alejandro Lopez 1daef2ceeb [#660] writecache: Fix remaining addr2key uses 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-30 17:12:33 +00:00
Evgenii Stratonikov 96e690883f [#638] Unify test loggers 
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov 322c1dc273 [#638] Use test.NewLogger() in tests 
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Alejandro Lopez 345a1a69a2 [#635] Use internal key type when deleting from badger wc 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-22 10:53:19 +03:00
Evgenii Stratonikov f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L() 
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:51:10 +00:00
Dmitrii Stepanov 2efe9cc1be [#585] writecache: Fix DB counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov 58c8722c81 [#585] fstree: Add optional file counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov eec97d177e [#585] writecache: Count items periodically 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Alejandro Lopez 88b6755c5e [#598] Fix use-after-close bug in badger writecache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-15 07:04:06 +00:00
Alejandro Lopez 376f03a445 [#598] Hold mode mutex when setting mode 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-14 07:12:27 +00:00
Alexander Chuprov 21800e9fcc [#162] core: Move literals to constants 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-11 15:48:42 +03:00
Alejandro Lopez 8f994163ee [#586] Fix writecache benchmarks and refactor hacky NeedsCompression 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-10 08:05:18 +00:00
Alejandro Lopez 023b90342c [#584] Disable compression in badger writecache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-10 10:14:06 +03:00
Alejandro Lopez 5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 10:26:53 +00:00
Alejandro Lopez ae322e9f73 [#576] Set SyncWrites for badger writecache by default 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 11:28:26 +03:00
Alejandro Lopez 7da4306e38 [#575] writecache: Fix log level for badger writecache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-08 08:16:36 +00:00
Dmitrii Stepanov 0e697266c3 [#563] writecache: Fix metrics and bolt delete 
Estimate cache size after delete objects to update metric.
Update counters on small object deletion.
Do not count bbolt DB file as FSTree object.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-07 12:53:28 +00:00
Alejandro Lopez 1a0cb0f34a [#421] Try using badger for the write-cache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-07 08:16:57 +00:00
Evgenii Stratonikov ec8b4fdc48 [#541] writecache/test: Close writecache on exit 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-29 10:57:25 +00:00
Evgenii Stratonikov ad5f527bd3 [#541] writecache/test: Remove initWC() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-29 10:57:25 +00:00
Alejandro Lopez 11027945d8 [#479] writecache: Fix writecache fstree flush premature ctx cancel 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-14 10:25:52 +03:00
Alexander Chuprov 033eaf77e1 [#496] node: Fix linter importas 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Evgenii Stratonikov fe01781811 [#446] los: Wrap SSD errors in a separate type 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-16 14:53:31 +03:00