TrueCloudLab
/
frostfs-node
19
1
Fork 21
Code Issues 88 Pull Requests 8 Actions Releases Activity
4,597 Commits
7 Branches
22 Tags
48 MiB
Commit Graph

169 Commits (v0.38.3)

Author SHA1 Message Date
Anton Nikiforov 3195142d67 [#959] writecache: Avoid manipulation with cache in `DEGRADED` mode 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-11 18:35:41 +00:00
Dmitrii Stepanov 66a26b7775 [#973] node: Resolve revive: unused-parameter linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-11 17:11:49 +03:00
Anton Nikiforov 05b5f5ca85 [#959] writecache: Fix panic on `Get` when it is not initialized
Vulncheck / Vulncheck (pull_request) Successful in 2m41s Details
DCO action / DCO (pull_request) Successful in 2m43s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s Details
Build / Build Components (1.21) (pull_request) Successful in 3m45s Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s Details
Tests and linters / Lint (pull_request) Successful in 5m30s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m57s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m39s Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-14 16:10:33 +03:00
Anton Nikiforov 0bd030507e [#948] metrics: Set actual value for `shard_id` after restart
DCO action / DCO (pull_request) Successful in 2m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m56s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m18s Details
Build / Build Components (1.21) (pull_request) Successful in 4m11s Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s Details
Tests and linters / Lint (pull_request) Successful in 6m3s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m20s Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-13 09:43:21 +03:00
Anton Nikiforov d19ade23c8 [#959] node: Set mode to shard's components when open it 
Avoid opening database for `metabase` and `cache` in `Degraded` mode.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-09 14:04:01 +00:00
Dmitrii Stepanov 47dcfa20f3 [#895] test: Use t.Cleanup only for external resources 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov f1b2b8bffa [#895] test: Fix NewLogger arguments list 
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov 581887148a [#569] cli: Add `control shards writecache seal` command
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 3m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m0s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details
Tests and linters / Lint (pull_request) Successful in 6m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m12s Details 
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 7a9db5bcdd [#569] writecache: Do not wait modeMtx if mode changes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 32c282ca10 [#569] writecache: Refactor flush 
Make single RUnlock call instead of two.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 0cb0fc1735 [#569] writecache: Allow to seal writecache after flush 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 8180a0664f [#887] node: Drop badger writecache implementation
Vulncheck / Vulncheck (pull_request) Successful in 6m1s Details
Build / Build Components (1.21) (pull_request) Successful in 7m37s Details
Build / Build Components (1.20) (pull_request) Successful in 7m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m56s Details
Tests and linters / Lint (pull_request) Successful in 9m26s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s Details
Tests and linters / Tests with -race (pull_request) Successful in 15m7s Details
DCO action / DCO (pull_request) Successful in 1m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s Details 
Badger implementation isn't tested and works not well,
but requires human resources to maintain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-22 13:00:54 +03:00
Dmitrii Stepanov 94ffe8bb45 [#857] golangci: Add testifylint linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 8d18fa159e [#667] writecache: Fix flush test
DCO action / DCO (pull_request) Successful in 1m28s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m55s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details
Build / Build Components (1.21) (pull_request) Successful in 4m40s Details
Build / Build Components (1.20) (pull_request) Successful in 4m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m32s Details
Tests and linters / Lint (pull_request) Successful in 5m59s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m59s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m57s Details 
Allow to disable background flusher for testing purposes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 17:45:43 +03:00
Dmitrii Stepanov 76ff26039c [#96] node: Drop neo-go's `slices` package
DCO action / DCO (pull_request) Successful in 2m13s Details
Build / Build Components (1.20) (pull_request) Successful in 5m48s Details
Build / Build Components (1.21) (pull_request) Successful in 5m47s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m33s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m37s Details
Tests and linters / Lint (pull_request) Successful in 8m8s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m38s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m22s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 13:24:04 +03:00
Dmitrii Stepanov 137e987a4e [#655] storage: Drop `LazyHandler` 
LazyHandler is implemented and used incorrectly.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-16 17:27:38 +03:00
Dmitrii Stepanov 79088baa06 [#772] node: Apply gofumpt 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:03 +03:00
Dmitrii Stepanov 869518be0a [#728] writecache: Fix Badger writecache race.
Build / Build Components (1.21) (pull_request) Successful in 1m47s Details
DCO action / DCO (pull_request) Successful in 2m1s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m5s Details
Build / Build Components (1.20) (pull_request) Successful in 3m58s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m43s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details
Tests and linters / Lint (pull_request) Successful in 6m20s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m8s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 3m6s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-30 18:36:41 +03:00
Dmitrii Stepanov d4b6ebe7e7 [#725] writecache: Fix metric values
DCO action / DCO (pull_request) Successful in 1m33s Details
Build / Build Components (1.21) (pull_request) Successful in 3m11s Details
Build / Build Components (1.20) (pull_request) Successful in 3m22s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m34s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m0s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m1s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 3m58s Details
Tests and linters / Lint (pull_request) Successful in 11m43s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-27 12:22:29 +03:00
Anton Nikiforov 559ad58ab1 [#642] writecache: Remove usage of close channel in bbolt 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-24 15:57:50 +00:00
Anton Nikiforov c0b86f2d93 [#642] writecache: Remove usage of close channel in badger 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-24 15:57:50 +00:00
Dmitrii Stepanov f26233b47a [#734] metabase: Include UpdateStorageID in metrics and traces 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-16 17:00:18 +03:00
Dmitrii Stepanov d07afd803c [#726] writecache: Fix small object flush for Badger
DCO action / DCO (pull_request) Successful in 2m25s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m41s Details
Build / Build Components (1.21) (pull_request) Successful in 3m2s Details
Build / Build Components (1.20) (pull_request) Successful in 3m17s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m9s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 18m15s Details
Tests and linters / Lint (pull_request) Successful in 20m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 18m2s Details 
Do not marshal object twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-06 11:32:50 +03:00
Dmitrii Stepanov 997ac7cd8d [#726] writecache: Fix small object flush for BBolt 
Do not marshal object twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-06 11:32:44 +03:00
Airat Arifullin 3889e829e6 [#667] writecache: Add logs for report error func in tests 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-09-14 17:02:54 +00:00
Ekaterina Lebedeva 8a81af5a3b [#653] Add context parameter to Open functions
DCO action / DCO (pull_request) Successful in 1m38s Details
Build / Build Components (1.20) (pull_request) Successful in 4m22s Details
Build / Build Components (1.21) (pull_request) Successful in 4m25s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m56s Details
Tests and linters / Lint (pull_request) Successful in 6m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m43s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m1s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m32s Details 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-09-07 18:03:29 +03:00
Anton Nikiforov 4dff9555f1 [#568] writecache: Improve flushing scheme for badger 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-30 17:22:28 +00:00
Alejandro Lopez 1daef2ceeb [#660] writecache: Fix remaining addr2key uses 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-30 17:12:33 +00:00
Evgenii Stratonikov 96e690883f [#638] Unify test loggers 
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov 322c1dc273 [#638] Use test.NewLogger() in tests 
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Alejandro Lopez 345a1a69a2 [#635] Use internal key type when deleting from badger wc
DCO action / DCO (pull_request) Successful in 2m48s Details
Build / Build Components (1.21) (pull_request) Successful in 4m59s Details
Build / Build Components (1.20) (pull_request) Successful in 5m5s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m55s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m21s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m53s Details
Tests and linters / Lint (pull_request) Successful in 8m35s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m57s Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-22 10:53:19 +03:00
Evgenii Stratonikov f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L() 
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:51:10 +00:00
Dmitrii Stepanov 2efe9cc1be [#585] writecache: Fix DB counter
DCO action / DCO (pull_request) Successful in 2m14s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m44s Details
Build / Build Components (1.21) (pull_request) Successful in 2m54s Details
Build / Build Components (1.20) (pull_request) Successful in 3m5s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m43s Details
Tests and linters / Lint (pull_request) Successful in 5m12s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m44s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 21m5s Details
Tests and linters / Tests with -race (pull_request) Failing after 26m26s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov 58c8722c81 [#585] fstree: Add optional file counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov eec97d177e [#585] writecache: Count items periodically 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Alejandro Lopez 88b6755c5e [#598] Fix use-after-close bug in badger writecache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-15 07:04:06 +00:00
Alejandro Lopez 376f03a445 [#598] Hold mode mutex when setting mode 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-14 07:12:27 +00:00
Alexander Chuprov 21800e9fcc [#162] core: Move literals to constants 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-11 15:48:42 +03:00
Alejandro Lopez 8f994163ee [#586] Fix writecache benchmarks and refactor hacky NeedsCompression 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-10 08:05:18 +00:00
Alejandro Lopez 023b90342c [#584] Disable compression in badger writecache
Build / Build Components (1.20) (pull_request) Failing after 3s Details
Build / Build Components (1.21) (pull_request) Failing after 2s Details
Tests and linters / Lint (pull_request) Failing after 4s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 3s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3s Details
Tests and linters / Tests with -race (pull_request) Failing after 3s Details
Tests and linters / Staticcheck (pull_request) Failing after 3s Details
Vulncheck / Vulncheck (pull_request) Failing after 3s Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-10 10:14:06 +03:00
Alejandro Lopez 5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 10:26:53 +00:00
Alejandro Lopez ae322e9f73 [#576] Set SyncWrites for badger writecache by default
Build / Build Components (1.19) (pull_request) Failing after 3s Details
Build / Build Components (1.20) (pull_request) Failing after 2s Details
Tests and linters / Tests (1.19) (pull_request) Failing after 3s Details
Tests and linters / Tests with -race (pull_request) Failing after 3s Details
Tests and linters / Lint (pull_request) Failing after 12s Details
Tests and linters / Staticcheck (pull_request) Failing after 3s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m46s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m37s Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 11:28:26 +03:00
Alejandro Lopez 7da4306e38 [#575] writecache: Fix log level for badger writecache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-08 08:16:36 +00:00
Dmitrii Stepanov 0e697266c3 [#563] writecache: Fix metrics and bolt delete 
Estimate cache size after delete objects to update metric.
Update counters on small object deletion.
Do not count bbolt DB file as FSTree object.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-07 12:53:28 +00:00
Alejandro Lopez 1a0cb0f34a [#421] Try using badger for the write-cache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-07 08:16:57 +00:00
Evgenii Stratonikov ec8b4fdc48 [#541] writecache/test: Close writecache on exit 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-29 10:57:25 +00:00
Evgenii Stratonikov ad5f527bd3 [#541] writecache/test: Remove initWC() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-29 10:57:25 +00:00
Alejandro Lopez 11027945d8 [#479] writecache: Fix writecache fstree flush premature ctx cancel
Build / Build Components (1.20) (pull_request) Successful in 3m49s Details
Build / Build Components (1.19) (pull_request) Successful in 4m2s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m13s Details
Tests and linters / Lint (pull_request) Successful in 11m25s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m4s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m0s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 25m11s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-14 10:25:52 +03:00
Alexander Chuprov 033eaf77e1 [#496] node: Fix linter importas
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m1s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s Details
Tests and linters / Lint (pull_request) Successful in 14m40s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Evgenii Stratonikov fe01781811 [#446] los: Wrap SSD errors in a separate type 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-16 14:53:31 +03:00