2b3fc50681
[ #1320 ] shard: Fix TestGCDropsObjectInhumedFromWritecache flaky test
...
DCO action / DCO (pull_request) Successful in 1m27s
Tests and linters / Run gofumpt (pull_request) Successful in 2m25s
Build / Build Components (1.23) (pull_request) Successful in 2m46s
Vulncheck / Vulncheck (pull_request) Successful in 2m37s
Build / Build Components (1.22) (pull_request) Successful in 2m55s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m50s
Tests and linters / Staticcheck (pull_request) Successful in 3m33s
Tests and linters / Lint (pull_request) Successful in 3m45s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m45s
Tests and linters / Tests (1.23) (pull_request) Successful in 3m42s
Tests and linters / Tests with -race (pull_request) Successful in 4m22s
Tests and linters / gopls check (pull_request) Successful in 4m25s
The `TestGCDropsObjectInhumedFromWritecache` test was flaky because a
running asynchronous rebuild operation prevented GC from deleting the
object. A test-only shard option `WithDisabledRebuild` has been added
to fix this.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-02 10:26:53 +03:00
882c068410
[ #1334 ] metabase: Store upgrade flag
...
DCO action / DCO (pull_request) Successful in 1m35s
Tests and linters / Run gofumpt (pull_request) Successful in 1m52s
Build / Build Components (1.22) (pull_request) Successful in 2m40s
Build / Build Components (1.23) (pull_request) Successful in 2m39s
Vulncheck / Vulncheck (pull_request) Successful in 2m20s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m53s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m11s
Tests and linters / Tests (1.23) (pull_request) Successful in 3m13s
Tests and linters / Staticcheck (pull_request) Successful in 3m46s
Tests and linters / Tests with -race (pull_request) Successful in 4m16s
Tests and linters / Lint (pull_request) Successful in 4m23s
Tests and linters / gopls check (pull_request) Successful in 4m14s
This allows to check if metabase upgrade was not completed.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-30 09:06:20 +03:00
7bf20c9f1f
[ #1323 ] metabase: Add expiration epoch buckets
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-22 08:21:40 +00:00
80ce7c3a00
[ #1284 ] shard: Resolve funlen linter
...
DCO action / DCO (pull_request) Successful in 1m34s
Vulncheck / Vulncheck (pull_request) Successful in 2m0s
Tests and linters / Run gofumpt (pull_request) Successful in 2m14s
Build / Build Components (1.22) (pull_request) Successful in 2m41s
Build / Build Components (1.21) (pull_request) Successful in 2m43s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m50s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m51s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m51s
Tests and linters / Tests with -race (pull_request) Successful in 2m56s
Tests and linters / Staticcheck (pull_request) Successful in 2m57s
Tests and linters / Lint (pull_request) Successful in 3m27s
Tests and linters / gopls check (pull_request) Successful in 3m56s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 13:23:35 +03:00
93d63e1632
[ #1284 ] writecache: Allow to seal writecache async
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 13:23:35 +03:00
36efccd862
[ #1298 ] writecache: Add shrink flag for Seal command
...
DCO action / DCO (pull_request) Successful in 1m23s
Vulncheck / Vulncheck (pull_request) Successful in 1m48s
Tests and linters / Run gofumpt (pull_request) Successful in 1m54s
Build / Build Components (1.22) (pull_request) Successful in 2m23s
Build / Build Components (1.21) (pull_request) Successful in 2m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m53s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m53s
Tests and linters / Staticcheck (pull_request) Successful in 2m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m54s
Tests and linters / Tests with -race (pull_request) Successful in 2m58s
Tests and linters / Lint (pull_request) Successful in 3m24s
Tests and linters / gopls check (pull_request) Successful in 3m36s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-08 16:32:29 +03:00
5c01bd5be8
[ #1298 ] writecache: Add restore-mode flag for Seal command
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-07 11:20:14 +03:00
18182e578e
[ #1261 ] shard: Fix delete objects from FSTree
...
DCO action / DCO (pull_request) Successful in 6m56s
Vulncheck / Vulncheck (pull_request) Successful in 7m38s
Build / Build Components (1.22) (pull_request) Successful in 9m57s
Build / Build Components (1.21) (pull_request) Successful in 10m38s
Tests and linters / gopls check (pull_request) Successful in 10m43s
Tests and linters / Staticcheck (pull_request) Successful in 11m1s
Tests and linters / Lint (pull_request) Successful in 13m35s
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m53s
Tests and linters / Tests (1.21) (pull_request) Successful in 15m55s
Tests and linters / Tests with -race (pull_request) Successful in 15m49s
Tests and linters / Tests (1.22) (pull_request) Successful in 16m10s
Replace nil storageID with empty like by shard.Get.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-22 14:38:09 +03:00
3940bc17c1
[ #1251 ] pilorama: Allow traversing multiple branches in parallel
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
7085723c6b
[ #1074 ] pilorama: Allow empty filenames in SortedByFilename()
...
DCO action / DCO (pull_request) Successful in 5m38s
Vulncheck / Vulncheck (pull_request) Successful in 5m57s
Tests and linters / gopls check (pull_request) Successful in 6m57s
Build / Build Components (1.22) (pull_request) Successful in 6m48s
Build / Build Components (1.21) (pull_request) Successful in 6m52s
Tests and linters / Lint (pull_request) Successful in 8m26s
Tests and linters / Staticcheck (pull_request) Successful in 8m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m39s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m47s
Tests and linters / Tests with -race (pull_request) Successful in 13m49s
Pre-commit hooks / Pre-commit (pull_request) Successful in 17m24s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-28 17:46:24 +03:00
9ac74efc41
[ #1173 ] shard: Use mode from config on reload
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-20 11:29:10 +00:00
68ac490729
[ #1174 ] shard: Update metric mode_info on Init
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-13 08:32:59 +00:00
6a39c3d15e
[ #1086 ] engine: Do not use metabase if shard looks bad
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-13 07:35:22 +00:00
806236da78
[ #1121 ] node: Change mode of shard components
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-06-05 05:55:24 +00:00
6f2187a420
[ #1121 ] node: Refactor mods of shard
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-06-05 05:55:24 +00:00
92e19feb57
[ #1147 ] node: Use public fields for shard.ExistsPrm
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
6130650bb6
[ #1147 ] node: Implement Lock\Delete requests for EC object
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
a82c8cc5b8
[ #1147 ] gc: Execute callback for expired tombstones when they exists
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
d45d086acd
[ #1129 ] policer: Add EC chunk replication
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:48 +03:00
c9efaa5819
[ #966 ] node: Add path of the write_cache to metric labels
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
411a8d0245
[ #1004 ] blobovnicza: Use TTL for blobovnicza tree cache
...
DCO action / DCO (pull_request) Successful in 12m46s
Vulncheck / Vulncheck (pull_request) Successful in 14m10s
Build / Build Components (1.22) (pull_request) Successful in 15m23s
Build / Build Components (1.21) (pull_request) Successful in 15m38s
Tests and linters / Staticcheck (pull_request) Successful in 3m36s
Tests and linters / gopls check (pull_request) Successful in 4m0s
Tests and linters / Lint (pull_request) Successful in 5m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m41s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Tests and linters / Tests (1.22) (pull_request) Successful in 7m58s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-26 19:54:29 +03:00
3dc81cb4fc
Reapply "[ #972 ] Use min/max builtins"
...
This reverts commit dad56d2e98
2024-04-10 12:09:34 +00:00
1b17258c04
[ #1029 ] metabase: Add refill metrics
...
DCO action / DCO (pull_request) Successful in 1m22s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Build / Build Components (1.21) (pull_request) Successful in 3m56s
Build / Build Components (1.20) (pull_request) Successful in 3m59s
Tests and linters / Staticcheck (pull_request) Successful in 5m31s
Tests and linters / gopls check (pull_request) Successful in 5m26s
Tests and linters / Lint (pull_request) Successful in 6m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m54s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m13s
Tests and linters / Tests with -race (pull_request) Successful in 9m30s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 13:05:44 +03:00
57466594fb
[ #1024 ] shard: Resync metabase concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
1005bf4f56
[ #1024 ] shard: Add refill metabase benchmark
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
d5194ab2a6
[ #949 ] metabase: fix shard.UpdateID()
...
Vulncheck / Vulncheck (pull_request) Successful in 1m20s
DCO action / DCO (pull_request) Successful in 1m59s
Build / Build Components (1.21) (pull_request) Successful in 3m25s
Build / Build Components (1.20) (pull_request) Successful in 4m46s
Tests and linters / Staticcheck (pull_request) Successful in 6m5s
Tests and linters / gopls check (pull_request) Successful in 6m17s
Tests and linters / Lint (pull_request) Successful in 7m7s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m38s
Tests and linters / Tests with -race (pull_request) Successful in 8m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m56s
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
e12fcc041d
[ #1059 ] services/tree: Fast sorted listing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:37:34 +00:00
942d83611b
[ #874 ] engine: Revert Check object existance concurrently
...
This reverts commit f526f49995
2024-04-01 08:42:34 +00:00
66a26b7775
[ #973 ] node: Resolve revive: unused-parameter linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:11:49 +03:00
0882840bf5
[ #634 ] shard: Add writecache inhume tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:12:34 +03:00
2ad433dbcb
[ #1005 ] engine: Drop shards weights
...
DCO action / DCO (pull_request) Successful in 4m7s
Vulncheck / Vulncheck (pull_request) Successful in 4m53s
Build / Build Components (1.21) (pull_request) Successful in 5m46s
Build / Build Components (1.20) (pull_request) Successful in 6m21s
Tests and linters / Staticcheck (pull_request) Successful in 7m45s
Tests and linters / Lint (pull_request) Successful in 8m44s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 15m42s
Tests and linters / Tests with -race (pull_request) Successful in 16m10s
Unused.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-26 17:25:05 +03:00
7470c383dd
[ #997 ] metabase: Drop toMoveIt bucket
...
DCO action / DCO (pull_request) Successful in 2m9s
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Build / Build Components (1.20) (pull_request) Successful in 3m56s
Build / Build Components (1.21) (pull_request) Successful in 3m54s
Tests and linters / Staticcheck (pull_request) Successful in 5m26s
Tests and linters / Lint (pull_request) Successful in 6m19s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m6s
Tests and linters / Tests with -race (pull_request) Successful in 8m20s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s
It is not used.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
dad56d2e98
Revert "[ #972 ] Use min/max builtins"
...
This reverts commit 89784b2e0a
2024-02-19 15:36:01 +00:00
89784b2e0a
[ #972 ] Use min/max builtins
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
2429508ac5
[ #959 ] shard: Skip rebuild in DEGRADED mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
0bd030507e
[ #948 ] metrics: Set actual value for shard_id after restart
...
DCO action / DCO (pull_request) Successful in 2m26s
Vulncheck / Vulncheck (pull_request) Successful in 3m56s
Tests and linters / Staticcheck (pull_request) Successful in 4m18s
Build / Build Components (1.21) (pull_request) Successful in 4m11s
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Tests and linters / Lint (pull_request) Successful in 6m3s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s
Tests and linters / Tests with -race (pull_request) Successful in 9m20s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:43:21 +03:00
7f692409cf
[ #970 ] fstree: Handle unsupported O_TMPFILE
...
Metabase test relied on this behaviour, so fix the test too.
Cherry-picking was hard and did too many conflicts,
here is an original PR:
https://github.com/nspcc-dev/neofs-node/pull/2624
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
d19ade23c8
[ #959 ] node: Set mode to shard's components when open it
...
Avoid opening database for `metabase` and `cache` in `Degraded` mode.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-09 14:04:01 +00:00
728150d1d2
[ #947 ] engine: Evacuate trees to local shards
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
8e2a0611f4
[ #947 ] tree: Add method to list all trees
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
e3573de6db
[ #930 ] gc: Stop internal activity by context
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
675eec91f3
[ #938 ] shard: Update only changed counters
...
DCO action / DCO (pull_request) Successful in 5m40s
Vulncheck / Vulncheck (pull_request) Successful in 5m33s
Tests and linters / Staticcheck (pull_request) Successful in 7m12s
Build / Build Components (1.21) (pull_request) Successful in 7m25s
Build / Build Components (1.20) (pull_request) Successful in 7m29s
Tests and linters / Lint (pull_request) Successful in 8m26s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m48s
Tests and linters / Tests with -race (pull_request) Successful in 9m56s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m10s
If metric value hasn't changed, but we update metric, then
non existed metric will apear with zero value.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
931a5e9aaf
[ #918 ] engine: Move shard to degraded mode if metabase open failed
...
DCO action / DCO (pull_request) Successful in 1m55s
Build / Build Components (1.20) (pull_request) Successful in 2m14s
Vulncheck / Vulncheck (pull_request) Successful in 2m15s
Build / Build Components (1.21) (pull_request) Successful in 4m6s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m24s
Tests and linters / Staticcheck (pull_request) Successful in 6m11s
Tests and linters / Lint (pull_request) Successful in 6m37s
Tests and linters / Tests with -race (pull_request) Successful in 6m35s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 11:16:40 +03:00
f526f49995
[ #874 ] engine: Check object existance concurrently
...
DCO action / DCO (pull_request) Successful in 1m36s
Vulncheck / Vulncheck (pull_request) Successful in 2m33s
Build / Build Components (1.21) (pull_request) Successful in 3m9s
Build / Build Components (1.20) (pull_request) Successful in 4m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m5s
Tests and linters / Staticcheck (pull_request) Successful in 6m6s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m39s
Tests and linters / Lint (pull_request) Successful in 7m0s
Tests and linters / Tests with -race (pull_request) Successful in 6m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 09:28:29 +03:00
57171907e3
[ #904 ] metabase: Return if object was actuall inserted
...
This requires to count metrics properly.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 19:04:39 +03:00
a2ab373a0a
[ #895 ] metabase: Do not delete GC mark for virtual objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
47dcfa20f3
[ #895 ] test: Use t.Cleanup only for external resources
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
f1b2b8bffa
[ #895 ] test: Fix NewLogger arguments list
...
`debug` is always true.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
4b8b4da681
[ #864 ] engine: Drop container count metric if container removed
...
DCO action / DCO (pull_request) Successful in 1m30s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Build / Build Components (1.20) (pull_request) Successful in 3m53s
Tests and linters / Lint (pull_request) Successful in 4m31s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m1s
Tests and linters / Staticcheck (pull_request) Successful in 4m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m13s
Tests and linters / Tests with -race (pull_request) Successful in 8m34s
Vulncheck / Vulncheck (pull_request) Successful in 58s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:45:32 +03:00
d75e7e9a21
[ #864 ] engine: Drop container size metric if container deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:44:54 +03:00
