Commit graph

10 commits

Author SHA1 Message Date
f3c141de41
[#xx] writecache: Fix race condition when reporting cache size metrics
Some checks failed
DCO action / DCO (pull_request) Failing after 34s
Vulncheck / Vulncheck (pull_request) Successful in 1m0s
Build / Build Components (pull_request) Successful in 1m30s
Tests and linters / Run gofumpt (pull_request) Successful in 1m22s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m40s
Tests and linters / Staticcheck (pull_request) Successful in 2m30s
Tests and linters / Tests (pull_request) Successful in 2m33s
Tests and linters / gopls check (pull_request) Successful in 2m54s
Tests and linters / Tests with -race (pull_request) Successful in 3m9s
Tests and linters / Lint (pull_request) Successful in 3m26s
There is a race condition when multiple cache operation try to report
the cache size metrics simultaneously. Consider the following example:
- the initial total size of objects stored in the cache size is 2
- worker X deletes an object and reads the cache size, which is 1
- worker Y deletes an object and reads the cache size, which is 0
- worker Y reports the cache size it learnt, which is 0
- worker X reports the cache size it learnt, which is 1

As a result, the observed cache size is 1 (i. e. one object remains
in the cache), which is incorrect because the actual cache size is 0.

To fix this, a separate worker for reporting the cache size metric has
been created. All operations should use a queue (a buffered channel) to
request the reporter worker to report the metrics. Currently, all queue
writes are non-blocking.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-02-18 13:06:55 +03:00
fb928616cc
[#1598] golangci: Enable unparam linter
All checks were successful
DCO action / DCO (pull_request) Successful in 35s
Vulncheck / Vulncheck (pull_request) Successful in 1m6s
Build / Build Components (pull_request) Successful in 1m45s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m44s
Tests and linters / Staticcheck (pull_request) Successful in 1m59s
Tests and linters / gopls check (pull_request) Successful in 2m19s
Tests and linters / Run gofumpt (pull_request) Successful in 2m35s
Tests and linters / Lint (pull_request) Successful in 2m56s
Tests and linters / Tests with -race (pull_request) Successful in 3m48s
Tests and linters / Tests (pull_request) Successful in 3m58s
To drop unnecessary parameters and return values.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-14 09:06:47 +03:00
7429553266
[#1437] node: Fix contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
6db46257c0
[#1437] node: Use ctx for logging
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:07 +03:00
f345fe9a58 [#1367] writecache: Move DB related code to upgrade.go
This is done to drop this file in the future.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
25d2ae8aaf [#1367] writecache: Drop BBolt related config variables
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
e39378b1c3 [#1367] writecache: Add background flushing objects limiter
To limit memory usage by background flush.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
8a6e3025a0 [#1367] writecache: Flush from FSTree concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
b142b6f48e [#1367] fstree: Add size to file counter
FSTree file counter used by writecache. As writecache has now only one
storage, so it is required to use real object size to get writecache
size more accurate than `count * max_object_size`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
5f6c7cbdb1 [#1367] writecache: Drop bbolt DB
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 14:22:29 +03:00
Renamed from pkg/local_object_storage/writecache/cachebbolt.go (Browse further)