7470c383dd
[ #997 ] metabase: Drop toMoveIt bucket
...
DCO action / DCO (pull_request) Successful in 2m9s
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Build / Build Components (1.20) (pull_request) Successful in 3m56s
Build / Build Components (1.21) (pull_request) Successful in 3m54s
Tests and linters / Staticcheck (pull_request) Successful in 5m26s
Tests and linters / Lint (pull_request) Successful in 6m19s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m6s
Tests and linters / Tests with -race (pull_request) Successful in 8m20s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s
It is not used.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
dad56d2e98
Revert "[ #972 ] Use min/max builtins"
...
This reverts commit 89784b2e0a
2024-02-19 15:36:01 +00:00
89784b2e0a
[ #972 ] Use min/max builtins
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
2429508ac5
[ #959 ] shard: Skip rebuild in DEGRADED mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
0bd030507e
[ #948 ] metrics: Set actual value for shard_id after restart
...
DCO action / DCO (pull_request) Successful in 2m26s
Vulncheck / Vulncheck (pull_request) Successful in 3m56s
Tests and linters / Staticcheck (pull_request) Successful in 4m18s
Build / Build Components (1.21) (pull_request) Successful in 4m11s
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Tests and linters / Lint (pull_request) Successful in 6m3s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s
Tests and linters / Tests with -race (pull_request) Successful in 9m20s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:43:21 +03:00
7f692409cf
[ #970 ] fstree: Handle unsupported O_TMPFILE
...
Metabase test relied on this behaviour, so fix the test too.
Cherry-picking was hard and did too many conflicts,
here is an original PR:
https://github.com/nspcc-dev/neofs-node/pull/2624
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
d19ade23c8
[ #959 ] node: Set mode to shard's components when open it
...
Avoid opening database for `metabase` and `cache` in `Degraded` mode.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-09 14:04:01 +00:00
728150d1d2
[ #947 ] engine: Evacuate trees to local shards
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
8e2a0611f4
[ #947 ] tree: Add method to list all trees
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
e3573de6db
[ #930 ] gc: Stop internal activity by context
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
675eec91f3
[ #938 ] shard: Update only changed counters
...
DCO action / DCO (pull_request) Successful in 5m40s
Vulncheck / Vulncheck (pull_request) Successful in 5m33s
Tests and linters / Staticcheck (pull_request) Successful in 7m12s
Build / Build Components (1.21) (pull_request) Successful in 7m25s
Build / Build Components (1.20) (pull_request) Successful in 7m29s
Tests and linters / Lint (pull_request) Successful in 8m26s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m48s
Tests and linters / Tests with -race (pull_request) Successful in 9m56s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m10s
If metric value hasn't changed, but we update metric, then
non existed metric will apear with zero value.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
931a5e9aaf
[ #918 ] engine: Move shard to degraded mode if metabase open failed
...
DCO action / DCO (pull_request) Successful in 1m55s
Build / Build Components (1.20) (pull_request) Successful in 2m14s
Vulncheck / Vulncheck (pull_request) Successful in 2m15s
Build / Build Components (1.21) (pull_request) Successful in 4m6s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m24s
Tests and linters / Staticcheck (pull_request) Successful in 6m11s
Tests and linters / Lint (pull_request) Successful in 6m37s
Tests and linters / Tests with -race (pull_request) Successful in 6m35s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 11:16:40 +03:00
f526f49995
[ #874 ] engine: Check object existance concurrently
...
DCO action / DCO (pull_request) Successful in 1m36s
Vulncheck / Vulncheck (pull_request) Successful in 2m33s
Build / Build Components (1.21) (pull_request) Successful in 3m9s
Build / Build Components (1.20) (pull_request) Successful in 4m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m5s
Tests and linters / Staticcheck (pull_request) Successful in 6m6s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m39s
Tests and linters / Lint (pull_request) Successful in 7m0s
Tests and linters / Tests with -race (pull_request) Successful in 6m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 09:28:29 +03:00
57171907e3
[ #904 ] metabase: Return if object was actuall inserted
...
This requires to count metrics properly.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 19:04:39 +03:00
a2ab373a0a
[ #895 ] metabase: Do not delete GC mark for virtual objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
47dcfa20f3
[ #895 ] test: Use t.Cleanup only for external resources
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
f1b2b8bffa
[ #895 ] test: Fix NewLogger arguments list
...
`debug` is always true.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
4b8b4da681
[ #864 ] engine: Drop container count metric if container removed
...
DCO action / DCO (pull_request) Successful in 1m30s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Build / Build Components (1.20) (pull_request) Successful in 3m53s
Tests and linters / Lint (pull_request) Successful in 4m31s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m1s
Tests and linters / Staticcheck (pull_request) Successful in 4m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m13s
Tests and linters / Tests with -race (pull_request) Successful in 8m34s
Vulncheck / Vulncheck (pull_request) Successful in 58s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:45:32 +03:00
d75e7e9a21
[ #864 ] engine: Drop container size metric if container deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:44:54 +03:00
dfd62ca6b1
[ #864 ] metabase: Refactor delete/inhume
...
Available -> Logic, Raw -> Phy for delete/inhume results.
Use single counter instead of vectors.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-09 09:59:42 +03:00
581887148a
[ #569 ] cli: Add control shards writecache seal command
...
DCO action / DCO (pull_request) Successful in 2m46s
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Build / Build Components (1.21) (pull_request) Successful in 3m53s
Build / Build Components (1.20) (pull_request) Successful in 4m0s
Tests and linters / Staticcheck (pull_request) Successful in 5m31s
Tests and linters / Lint (pull_request) Successful in 6m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s
Tests and linters / Tests with -race (pull_request) Successful in 13m12s
It does the same as `control shards flush-writecache --seal`, but
has better name.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
0cb0fc1735
[ #569 ] writecache: Allow to seal writecache after flush
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
8180a0664f
[ #887 ] node: Drop badger writecache implementation
...
Vulncheck / Vulncheck (pull_request) Successful in 6m1s
Build / Build Components (1.21) (pull_request) Successful in 7m37s
Build / Build Components (1.20) (pull_request) Successful in 7m52s
Tests and linters / Staticcheck (pull_request) Successful in 8m56s
Tests and linters / Lint (pull_request) Successful in 9m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s
Tests and linters / Tests with -race (pull_request) Successful in 15m7s
DCO action / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s
Badger implementation isn't tested and works not well,
but requires human resources to maintain.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-22 13:00:54 +03:00
3b7c0362a8
[ #861 ] shard: Fix Delete object
...
DCO action / DCO (pull_request) Successful in 12m22s
Build / Build Components (1.20) (pull_request) Successful in 12m44s
Build / Build Components (1.21) (pull_request) Successful in 13m2s
Vulncheck / Vulncheck (pull_request) Successful in 13m3s
Tests and linters / Staticcheck (pull_request) Successful in 15m32s
Tests and linters / Lint (pull_request) Successful in 16m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 26m11s
Tests and linters / Tests with -race (pull_request) Successful in 28m29s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m28s
It is possible that object doesn't exist in metabase.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 14:25:40 +03:00
f1c7905263
[ #661 ] blobovniczatree: Make Rebuild concurrent
...
Different DBs can be rebuild concurrently.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
422226da18
[ #661 ] blobovniczatree: Add Rebuild implementation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
44552a849b
[ #661 ] shard: Add blobstor rebuilder
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
d30ab5f29e
[ #838 ] metabase: Count user objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
f314da4af3
[ #838 ] metabase: Add user object type counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
29550fe600
[ #838 ] shard: Refactor updateMetrics method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
445ebcc0e7
[ #651 ] shard/test: Speedup Shard_Delete
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
2302e5d342
[ #651 ] shard/test: Refactor Shard_Delete
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
07390ad4e3
[ #715 ] node: Unify config parameter names
...
DCO action / DCO (pull_request) Successful in 2m54s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s
Tests and linters / Lint (pull_request) Successful in 5m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
76ff26039c
[ #96 ] node: Drop neo-go's slices package
...
DCO action / DCO (pull_request) Successful in 2m13s
Build / Build Components (1.20) (pull_request) Successful in 5m48s
Build / Build Components (1.21) (pull_request) Successful in 5m47s
Vulncheck / Vulncheck (pull_request) Successful in 5m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m33s
Tests and linters / Staticcheck (pull_request) Successful in 7m37s
Tests and linters / Lint (pull_request) Successful in 8m8s
Tests and linters / Tests (1.20) (pull_request) Successful in 10m38s
Tests and linters / Tests with -race (pull_request) Successful in 11m22s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 13:24:04 +03:00
70ab1ebd54
[ #763 ] metrics: Add container_objects_total metric
...
DCO action / DCO (pull_request) Successful in 3m54s
Build / Build Components (1.20) (pull_request) Successful in 4m58s
Build / Build Components (1.21) (pull_request) Successful in 5m16s
Vulncheck / Vulncheck (pull_request) Successful in 9m54s
Tests and linters / Lint (pull_request) Successful in 10m57s
Tests and linters / Tests (1.21) (pull_request) Successful in 12m40s
Tests and linters / Staticcheck (pull_request) Successful in 12m34s
Tests and linters / Tests with -race (pull_request) Successful in 12m48s
Tests and linters / Tests (1.20) (pull_request) Successful in 13m19s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
79088baa06
[ #772 ] node: Apply gofumpt
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
00aa6d9749
[ #633 ] shard/test: Fix TestCounters()
...
Introduced in 362f24953a
2023-10-31 12:53:28 +00:00
b8f79f4227
[ #633 ] shard/test: Fix race conditions in TestCounters()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 12:53:28 +00:00
f2437f7ae9
[ #734 ] shard: Fix Delete method
...
DCO action / DCO (pull_request) Successful in 1m36s
Build / Build Components (1.20) (pull_request) Successful in 3m46s
Vulncheck / Vulncheck (pull_request) Successful in 3m16s
Tests and linters / Staticcheck (pull_request) Successful in 4m42s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m27s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m35s
Tests and linters / Tests with -race (pull_request) Successful in 6m33s
Build / Build Components (1.21) (pull_request) Successful in 13m7s
Tests and linters / Lint (pull_request) Successful in 19m14s
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
c1e4130020
[ #146 ] node: Add trace_id to logs
...
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
DCO action / DCO (pull_request) Successful in 3m36s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Build / Build Components (1.20) (pull_request) Successful in 3m37s
Tests and linters / Staticcheck (pull_request) Successful in 4m39s
Tests and linters / Lint (pull_request) Successful in 5m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m27s
Tests and linters / Tests with -race (pull_request) Successful in 6m29s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m19s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
8a81af5a3b
[ #653 ] Add context parameter to Open functions
...
DCO action / DCO (pull_request) Successful in 1m38s
Build / Build Components (1.20) (pull_request) Successful in 4m22s
Build / Build Components (1.21) (pull_request) Successful in 4m25s
Vulncheck / Vulncheck (pull_request) Successful in 4m56s
Tests and linters / Lint (pull_request) Successful in 6m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m43s
Tests and linters / Staticcheck (pull_request) Successful in 8m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m14s
Tests and linters / Tests with -race (pull_request) Successful in 8m32s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
a716db99db
[ #668 ] shard/test: Do not alter rootPath option
...
Supposedly, this was added to allow creating 2 different shards without
subtest. Now we use t.TempDir() everywhere, so this should not be a
problem.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
36759f8434
[ #668 ] shard/test: Properly check event processing
...
See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
39879fa868
[ #668 ] shard/test: Add dontRelease options
...
Most of the time we would like to close shard with minor exceptions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
c661ba1312
[ #668 ] shard/test: Use sane defaults in the test constructor
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
268adb79cb
[ #668 ] shard/test: Simplify shard construction
...
newCustomShard() has many parameters but only the first is obligatory.
`enableWriteCache` is left as-is, because it directly affects the
functionality.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
429f941cda
[ #668 ] shard/test: Release shard in t.Cleanup()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
382eb8a485
[ #668 ] shard/test: Disable GC where it is not needed
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
42696016de
[ #668 ] shard: Close stopChannel in GC
...
It is done once, but now we could read it from multiple places.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
bdecfbc1be
[ #668 ] shard/test: Move tests to the main package
...
Semantic patch (also, duplicate definitions are removed):
```
@@
var e identifier
@@
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/shard"
-shard.e
+e
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
