3f4717a37f
[ #1692 ] metabase: Do not allocate map in cache unless needed
...
Vulncheck / Vulncheck (push) Successful in 1m5s
Pre-commit hooks / Pre-commit (push) Successful in 1m31s
Build / Build Components (push) Successful in 2m4s
Tests and linters / gopls check (push) Successful in 3m59s
OCI image / Build container images (push) Successful in 5m24s
Tests and linters / Run gofumpt (push) Successful in 6m12s
Tests and linters / Tests with -race (push) Successful in 6m41s
Tests and linters / Staticcheck (push) Successful in 6m49s
Tests and linters / Lint (push) Successful in 7m9s
Tests and linters / Tests (push) Successful in 7m7s
Change-Id: I8b1015a8c7c3df4153a08fdb788117d9f0d6c333
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-21 08:56:32 +00:00
60cea8c714
[ #1692 ] metabase/test: Fix end of iteration error check
...
This is not good:
```
BenchmarkListWithCursor/1_item-8 --- FAIL: BenchmarkListWithCursor/1_item-8
list_test.go:63: error: end of object listing
```
Change-Id: I61b70937ce30fefaf16ebeb0cdb51bdd39096061
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-21 08:56:18 +00:00
e8801dbf49
[ #1691 ] metabase: Move cheaper conditions to the front in ListWithCursor()
...
Vulncheck / Vulncheck (push) Successful in 1m15s
Build / Build Components (push) Successful in 2m10s
Pre-commit hooks / Pre-commit (push) Successful in 2m44s
Tests and linters / gopls check (push) Successful in 3m46s
OCI image / Build container images (push) Successful in 4m36s
Tests and linters / Run gofumpt (push) Successful in 4m24s
Tests and linters / Lint (push) Successful in 5m11s
Tests and linters / Staticcheck (push) Successful in 5m7s
Tests and linters / Tests (push) Successful in 5m13s
Tests and linters / Tests with -race (push) Has been cancelled
`objectLocked` call is expensive, it does IO. We may omit it if object
is not expired.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-20 12:52:36 +00:00
eb9df85b98
[ #1685 ] metabase: Cache primary bucket
...
Build / Build Components (push) Has been cancelled
OCI image / Build container images (push) Has been cancelled
Pre-commit hooks / Pre-commit (push) Has been cancelled
Tests and linters / Lint (push) Has been cancelled
Tests and linters / Tests (push) Has been cancelled
Tests and linters / Tests with -race (push) Has been cancelled
Tests and linters / Staticcheck (push) Has been cancelled
Tests and linters / gopls check (push) Has been cancelled
Tests and linters / Run gofumpt (push) Has been cancelled
Vulncheck / Vulncheck (push) Successful in 1m32s
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ expired │ primary │
│ sec/op │ sec/op vs base │
Select/string_equal-8 3.529m ± 11% 3.689m ± 7% +4.55% (p=0.023 n=10)
Select/string_not_equal-8 3.440m ± 7% 3.543m ± 13% ~ (p=0.190 n=10)
Select/common_prefix-8 3.240m ± 6% 3.050m ± 5% -5.85% (p=0.005 n=10)
Select/unknown-8 3.198m ± 6% 2.928m ± 8% -8.44% (p=0.003 n=10)
geomean 3.349m 3.287m -1.84%
│ expired │ primary │
│ B/op │ B/op vs base │
Select/string_equal-8 1.885Mi ± 0% 1.786Mi ± 0% -5.23% (p=0.000 n=10)
Select/string_not_equal-8 1.885Mi ± 0% 1.786Mi ± 0% -5.23% (p=0.000 n=10)
Select/common_prefix-8 1.885Mi ± 0% 1.786Mi ± 0% -5.23% (p=0.000 n=10)
Select/unknown-8 1.877Mi ± 0% 1.779Mi ± 0% -5.26% (p=0.000 n=10)
geomean 1.883Mi 1.784Mi -5.24%
│ expired │ primary │
│ allocs/op │ allocs/op vs base │
Select/string_equal-8 46.04k ± 0% 43.04k ± 0% -6.50% (p=0.000 n=10)
Select/string_not_equal-8 46.04k ± 0% 43.04k ± 0% -6.50% (p=0.000 n=10)
Select/common_prefix-8 46.04k ± 0% 43.04k ± 0% -6.50% (p=0.000 n=10)
Select/unknown-8 45.05k ± 0% 42.05k ± 0% -6.65% (p=0.000 n=10)
geomean 45.79k 42.79k -6.54%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-20 12:52:01 +00:00
21bed3362c
[ #1685 ] metabase: Cache expired bucket
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ master │ expired │
│ sec/op │ sec/op vs base │
Select/string_equal-8 4.007m ± 10% 3.529m ± 11% -11.94% (p=0.000 n=10)
Select/string_not_equal-8 3.834m ± 12% 3.440m ± 7% -10.29% (p=0.029 n=10)
Select/common_prefix-8 3.470m ± 9% 3.240m ± 6% ~ (p=0.105 n=10)
Select/unknown-8 3.156m ± 3% 3.198m ± 6% ~ (p=0.631 n=10)
geomean 3.602m 3.349m -7.03%
│ master │ expired │
│ B/op │ B/op vs base │
Select/string_equal-8 1.907Mi ± 0% 1.885Mi ± 0% -1.18% (p=0.000 n=10)
Select/string_not_equal-8 1.907Mi ± 0% 1.885Mi ± 0% -1.18% (p=0.000 n=10)
Select/common_prefix-8 1.907Mi ± 0% 1.885Mi ± 0% -1.18% (p=0.000 n=10)
Select/unknown-8 1.900Mi ± 0% 1.877Mi ± 0% -1.18% (p=0.000 n=10)
geomean 1.905Mi 1.883Mi -1.18%
│ master │ expired │
│ allocs/op │ allocs/op vs base │
Select/string_equal-8 47.03k ± 0% 46.04k ± 0% -2.12% (p=0.000 n=10)
Select/string_not_equal-8 47.03k ± 0% 46.04k ± 0% -2.12% (p=0.000 n=10)
Select/common_prefix-8 47.03k ± 0% 46.04k ± 0% -2.12% (p=0.000 n=10)
Select/unknown-8 46.04k ± 0% 45.05k ± 0% -2.16% (p=0.000 n=10)
geomean 46.78k 45.79k -2.13%
```
Change-Id: I9c7a5e1f5c8b9eb3f25a563fd74c6ad2a9d1b92e
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-20 12:52:01 +00:00
a49f0717b3
[ #1685 ] metabase: Cache frequently accessed singleton buckets
...
Vulncheck / Vulncheck (push) Successful in 1m9s
Build / Build Components (push) Successful in 1m47s
Pre-commit hooks / Pre-commit (push) Successful in 1m45s
Tests and linters / Run gofumpt (push) Successful in 4m17s
Tests and linters / Lint (push) Successful in 4m44s
Tests and linters / Staticcheck (push) Successful in 4m44s
OCI image / Build container images (push) Successful in 5m0s
Tests and linters / gopls check (push) Successful in 5m38s
Tests and linters / Tests with -race (push) Successful in 5m55s
Tests and linters / Tests (push) Successful in 8m19s
There are some buckets we access almost always, to check whether an
object is alive. In search we also iterate over lots of objects, and
`tx.Bucket()` shows itself a lot in pprof.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 1 │ 2 │
│ sec/op │ sec/op vs base │
Select/string_equal-8 4.753m ± 6% 3.969m ± 14% -16.50% (p=0.000 n=10)
Select/string_not_equal-8 4.247m ± 9% 3.486m ± 11% -17.93% (p=0.000 n=10)
Select/common_prefix-8 4.163m ± 5% 3.323m ± 5% -20.18% (p=0.000 n=10)
Select/unknown-8 3.557m ± 3% 3.064m ± 8% -13.85% (p=0.001 n=10)
geomean 4.158m 3.445m -17.15%
│ 1 │ 2 │
│ B/op │ B/op vs base │
Select/string_equal-8 2.250Mi ± 0% 1.907Mi ± 0% -15.24% (p=0.000 n=10)
Select/string_not_equal-8 2.250Mi ± 0% 1.907Mi ± 0% -15.24% (p=0.000 n=10)
Select/common_prefix-8 2.250Mi ± 0% 1.907Mi ± 0% -15.24% (p=0.000 n=10)
Select/unknown-8 2.243Mi ± 0% 1.900Mi ± 0% -15.29% (p=0.000 n=10)
geomean 2.248Mi 1.905Mi -15.26%
│ 1 │ 2 │
│ allocs/op │ allocs/op vs base │
Select/string_equal-8 56.02k ± 0% 47.03k ± 0% -16.05% (p=0.000 n=10)
Select/string_not_equal-8 56.02k ± 0% 47.03k ± 0% -16.05% (p=0.000 n=10)
Select/common_prefix-8 56.02k ± 0% 47.03k ± 0% -16.05% (p=0.000 n=10)
Select/unknown-8 55.03k ± 0% 46.04k ± 0% -16.34% (p=0.000 n=10)
geomean 55.78k 46.78k -16.12%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-20 10:17:42 +00:00
39f549a7ab
[ #1642 ] tree: Intoduce a helper LastChild
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-03-20 10:12:49 +00:00
760b6a44ea
[ #1642 ] tree: Fix sorted getSubtree for multiversion filenames
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-03-20 10:12:49 +00:00
a11b2d27e4
[ #1642 ] tree: Introduce Cursor type
...
* Use `Cursor` as parameter for `TreeSortedByFilename`
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-03-20 10:12:49 +00:00
a405fb1f39
[ #1683 ] metabase: Check object status once in Select()
...
Vulncheck / Vulncheck (push) Successful in 1m4s
Pre-commit hooks / Pre-commit (push) Successful in 1m28s
Build / Build Components (push) Successful in 2m3s
Tests and linters / Run gofumpt (push) Successful in 3m33s
Tests and linters / Staticcheck (push) Successful in 3m42s
Tests and linters / Lint (push) Successful in 3m49s
Tests and linters / Tests (push) Successful in 4m13s
Tests and linters / Tests with -race (push) Successful in 4m18s
OCI image / Build container images (push) Successful in 4m43s
Tests and linters / gopls check (push) Successful in 6m9s
objectStatus() is called twice for the same object:
First, in selectObject() to filter removed objects.
Then, again, in getObjectForSlowFilters() via db.get().
The second call will return the same result, so remove useless branch.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ status │
│ sec/op │ sec/op vs base │
Select/string_equal-8 5.022m ± 7% 3.968m ± 8% -20.98% (p=0.000 n=10)
Select/string_not_equal-8 4.953m ± 9% 3.990m ± 10% -19.44% (p=0.000 n=10)
Select/common_prefix-8 4.962m ± 8% 3.971m ± 9% -19.98% (p=0.000 n=10)
Select/unknown-8 5.246m ± 9% 3.548m ± 5% -32.37% (p=0.000 n=10)
geomean 5.045m 3.865m -23.39%
│ old │ status │
│ B/op │ B/op vs base │
Select/string_equal-8 2.685Mi ± 0% 2.250Mi ± 0% -16.20% (p=0.000 n=10)
Select/string_not_equal-8 2.685Mi ± 0% 2.250Mi ± 0% -16.20% (p=0.000 n=10)
Select/common_prefix-8 2.685Mi ± 0% 2.250Mi ± 0% -16.20% (p=0.000 n=10)
Select/unknown-8 2.677Mi ± 0% 2.243Mi ± 0% -16.24% (p=0.000 n=10)
geomean 2.683Mi 2.248Mi -16.21%
│ old │ status │
│ allocs/op │ allocs/op vs base │
Select/string_equal-8 69.03k ± 0% 56.02k ± 0% -18.84% (p=0.000 n=10)
Select/string_not_equal-8 69.03k ± 0% 56.02k ± 0% -18.84% (p=0.000 n=10)
Select/common_prefix-8 69.03k ± 0% 56.02k ± 0% -18.84% (p=0.000 n=10)
Select/unknown-8 68.03k ± 0% 55.03k ± 0% -19.11% (p=0.000 n=10)
geomean 68.78k 55.77k -18.90%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-18 11:48:51 +00:00
a7319bc979
[ #1683 ] metabase/test: Report allocs in benchmarkSelect()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-18 11:48:51 +00:00
07a660fbc4
[ #1677 ] writecache: Add QoS limiter usage
...
DCO action / DCO (pull_request) Successful in 33s
Vulncheck / Vulncheck (pull_request) Successful in 1m14s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m22s
Build / Build Components (pull_request) Successful in 1m31s
Tests and linters / Run gofumpt (pull_request) Successful in 3m16s
Tests and linters / Lint (pull_request) Successful in 3m48s
Tests and linters / Staticcheck (pull_request) Successful in 3m45s
Tests and linters / gopls check (pull_request) Successful in 4m11s
Tests and linters / Tests with -race (pull_request) Successful in 6m9s
Tests and linters / Tests (pull_request) Successful in 4m6s
Vulncheck / Vulncheck (push) Successful in 1m11s
Pre-commit hooks / Pre-commit (push) Successful in 1m23s
Build / Build Components (push) Successful in 1m50s
Tests and linters / Run gofumpt (push) Successful in 3m30s
Tests and linters / Lint (push) Successful in 3m49s
Tests and linters / Tests with -race (push) Successful in 3m53s
Tests and linters / Staticcheck (push) Successful in 3m51s
Tests and linters / gopls check (push) Successful in 4m23s
Tests and linters / Tests (push) Successful in 4m30s
OCI image / Build container images (push) Successful in 5m18s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-03-14 16:23:33 +03:00
460e5cbccf
[ #1671 ] Use slices.Delete() where possible
...
gopatch is missing for this one, because
https://github.com/uber-go/gopatch/issues/179
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-13 08:12:20 +00:00
155d3ddb6e
[ #1671 ] Use min builtin where possible
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-13 08:12:20 +00:00
40536d8a06
[ #1671 ] Use fmt.Appendf where warranted
...
Fix gopls warnings:
```
cmd/frostfs-adm/internal/modules/morph/config/config.go:68:20-64: Replace []byte(fmt.Sprintf...) with fmt.Appendf
````
gopatch:
```
@@
var f expression
@@
-[]byte(fmt.Sprintf(f, ...))
+fmt.Appendf(nil, f, ...)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-13 08:12:20 +00:00
bcc84c85a0
[ #1671 ] Replace interface{} with any
...
gopatch:
```
@@
@@
-interface{}
+any
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-13 08:12:20 +00:00
2005fdda09
[ #1667 ] shard: Drop shard pool
...
DCO action / DCO (pull_request) Successful in 47s
Vulncheck / Vulncheck (pull_request) Successful in 1m4s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m25s
Build / Build Components (pull_request) Successful in 1m53s
Tests and linters / Run gofumpt (pull_request) Successful in 2m43s
Tests and linters / Lint (pull_request) Successful in 3m13s
Tests and linters / Tests (pull_request) Successful in 3m24s
Tests and linters / gopls check (pull_request) Successful in 3m30s
Tests and linters / Staticcheck (pull_request) Successful in 3m33s
Tests and linters / Tests with -race (pull_request) Successful in 3m35s
Vulncheck / Vulncheck (push) Successful in 1m1s
Pre-commit hooks / Pre-commit (push) Successful in 1m27s
Build / Build Components (push) Successful in 1m58s
Tests and linters / Run gofumpt (push) Successful in 2m40s
Tests and linters / Staticcheck (push) Successful in 3m3s
Tests and linters / Lint (push) Successful in 3m7s
Tests and linters / Tests (push) Successful in 3m23s
Tests and linters / gopls check (push) Successful in 3m41s
Tests and linters / Tests with -race (push) Successful in 3m47s
OCI image / Build container images (push) Successful in 4m45s
After adding an ops limiter, shard's `put` pool is redundant.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-03-11 13:59:51 +03:00
3727d60331
[ #1653 ] qos: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-03-11 10:57:47 +00:00
8643e0abc5
[ #1668 ] writecache: Use object size to check free space
...
DCO action / DCO (pull_request) Successful in 1m7s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m30s
Build / Build Components (pull_request) Successful in 1m56s
Vulncheck / Vulncheck (pull_request) Successful in 1m47s
Tests and linters / Run gofumpt (pull_request) Successful in 3m16s
Tests and linters / Lint (pull_request) Successful in 3m49s
Tests and linters / Staticcheck (pull_request) Successful in 3m49s
Tests and linters / Tests (pull_request) Successful in 3m59s
Tests and linters / gopls check (pull_request) Successful in 4m9s
Tests and linters / Tests with -race (pull_request) Successful in 4m40s
Pre-commit hooks / Pre-commit (push) Successful in 1m17s
Vulncheck / Vulncheck (push) Successful in 1m24s
Build / Build Components (push) Successful in 1m57s
Tests and linters / Run gofumpt (push) Successful in 2m37s
Tests and linters / Tests (push) Successful in 3m1s
Tests and linters / Staticcheck (push) Successful in 3m7s
Tests and linters / Lint (push) Successful in 3m11s
Tests and linters / gopls check (push) Successful in 3m14s
Tests and linters / Tests with -race (push) Successful in 3m25s
OCI image / Build container images (push) Successful in 4m54s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-03-10 17:52:57 +03:00
9e31cb249f
[ #1635 ] control: Add method to search shards by object
...
Added method `ListShardsForObject` to ControlService and to
StorageEngine. It returns information about shards storing
object on the node.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-03-07 14:32:01 +03:00
4685afb1dc
[ #1636 ] engine: Validate limiter release in unit tests
...
DCO action / DCO (pull_request) Successful in 33s
Vulncheck / Vulncheck (pull_request) Successful in 1m2s
Build / Build Components (pull_request) Successful in 1m24s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m26s
Tests and linters / Run gofumpt (pull_request) Successful in 2m53s
Tests and linters / Lint (pull_request) Successful in 3m4s
Tests and linters / Staticcheck (pull_request) Successful in 3m9s
Tests and linters / Tests (pull_request) Successful in 3m11s
Tests and linters / gopls check (pull_request) Successful in 3m43s
Tests and linters / Tests with -race (pull_request) Successful in 3m53s
Tests and linters / Run gofumpt (push) Successful in 34s
Vulncheck / Vulncheck (push) Successful in 1m7s
Pre-commit hooks / Pre-commit (push) Successful in 1m56s
Build / Build Components (push) Successful in 2m9s
Tests and linters / Staticcheck (push) Successful in 2m39s
Tests and linters / gopls check (push) Successful in 2m51s
Tests and linters / Tests (push) Successful in 3m17s
Tests and linters / Lint (push) Successful in 3m41s
Tests and linters / Tests with -race (push) Successful in 4m33s
OCI image / Build container images (push) Successful in 4m30s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:30 +03:00
eb8b9b2b3b
[ #1636 ] blobovniczatree: Validate limiter release in rebuild unit tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:30 +03:00
6c6e463b73
[ #1636 ] shard: Change ops limiter on shard reload
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:29 +03:00
c2d855aedd
[ #1636 ] qos: Return Resource Exhausted error
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:28 +03:00
b9360be1dc
[ #1636 ] blobovniczatree: Use RebuildLimiter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:27 +03:00
ceff5e1f6a
[ #1636 ] storage: Refactor shard rebuild
...
Drop redundant interfaces.
Rename fields.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:27 +03:00
e0dc3c3d0c
[ #1636 ] shard: Add limiter usage
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 17:25:26 +03:00
0991077cb3
[ #1657 ] engine: Fix data race in evacuation tests
...
Pre-commit hooks / Pre-commit (push) Successful in 1m23s
Vulncheck / Vulncheck (push) Successful in 1m3s
Build / Build Components (push) Successful in 1m52s
Tests and linters / Run gofumpt (push) Successful in 2m29s
Tests and linters / Tests with -race (push) Successful in 2m47s
Tests and linters / Lint (push) Successful in 3m4s
Tests and linters / Staticcheck (push) Successful in 3m0s
Tests and linters / gopls check (push) Successful in 3m11s
Tests and linters / Tests (push) Successful in 3m30s
OCI image / Build container images (push) Successful in 4m41s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-28 14:14:12 +00:00
02f3a7f65c
[ #1648 ] writecache: Fix race condition when reporting cache size metrics
...
Vulncheck / Vulncheck (push) Successful in 1m2s
Build / Build Components (push) Successful in 1m53s
Pre-commit hooks / Pre-commit (push) Successful in 2m7s
Tests and linters / Run gofumpt (push) Successful in 2m45s
Tests and linters / Tests with -race (push) Successful in 3m17s
Tests and linters / Lint (push) Successful in 3m20s
Tests and linters / gopls check (push) Successful in 3m15s
Tests and linters / Staticcheck (push) Successful in 3m21s
Tests and linters / Tests (push) Successful in 3m37s
OCI image / Build container images (push) Successful in 4m37s
DCO action / DCO (pull_request) Successful in 28s
Vulncheck / Vulncheck (pull_request) Successful in 50s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m28s
Build / Build Components (pull_request) Successful in 1m38s
Tests and linters / Run gofumpt (pull_request) Successful in 3m0s
Tests and linters / Tests with -race (pull_request) Successful in 3m10s
Tests and linters / Tests (pull_request) Successful in 3m14s
Tests and linters / Lint (pull_request) Successful in 3m21s
Tests and linters / Staticcheck (pull_request) Successful in 3m21s
Tests and linters / gopls check (pull_request) Successful in 3m34s
There is a race condition when multiple cache operation try to report
the cache size metrics simultaneously. Consider the following example:
- the initial total size of objects stored in the cache size is 2
- worker X deletes an object and reads the cache size, which is 1
- worker Y deletes an object and reads the cache size, which is 0
- worker Y reports the cache size it learnt, which is 0
- worker X reports the cache size it learnt, which is 1
As a result, the observed cache size is 1 (i. e. one object remains
in the cache), which is incorrect because the actual cache size is 0.
To fix this, let's report the metrics periodically in the flush loop.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-02-19 17:05:40 +03:00
5d79abe523
[ #1608 ] shard: Add IO tag for rebuild
...
DCO action / DCO (pull_request) Successful in 44s
Vulncheck / Vulncheck (pull_request) Successful in 1m3s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m32s
Build / Build Components (pull_request) Successful in 1m40s
Tests and linters / Staticcheck (pull_request) Successful in 2m53s
Tests and linters / Tests with -race (pull_request) Successful in 3m10s
Tests and linters / Run gofumpt (pull_request) Successful in 3m10s
Tests and linters / Lint (pull_request) Successful in 3m42s
Tests and linters / Tests (pull_request) Successful in 4m13s
Tests and linters / gopls check (pull_request) Successful in 4m21s
Vulncheck / Vulncheck (push) Successful in 1m6s
Pre-commit hooks / Pre-commit (push) Successful in 1m30s
Build / Build Components (push) Successful in 2m8s
Tests and linters / Staticcheck (push) Successful in 2m54s
Tests and linters / Run gofumpt (push) Successful in 3m42s
Tests and linters / Lint (push) Successful in 3m53s
Tests and linters / gopls check (push) Successful in 4m3s
Tests and linters / Tests (push) Successful in 3m28s
OCI image / Build container images (push) Successful in 4m24s
Tests and linters / Tests with -race (push) Successful in 4m59s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-07 15:43:07 +03:00
92450a76ba
[ #1608 ] shard: Add IO tag to GC
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-07 15:43:06 +03:00
abba5b2089
[ #1608 ] writecache: Add IO tag to flush worker
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-07 15:43:06 +03:00
f6b3f79e89
[ #1608 ] qos: Add qos service to adjust incoming IO tags
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-07 15:43:03 +03:00
9b113c3156
[ #1613 ] morph: Add tracing for morph queries to neo-go
...
DCO action / DCO (pull_request) Successful in 59s
Vulncheck / Vulncheck (pull_request) Successful in 1m4s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m55s
Build / Build Components (pull_request) Successful in 2m4s
Tests and linters / Staticcheck (pull_request) Successful in 2m38s
Tests and linters / Lint (pull_request) Successful in 3m16s
Tests and linters / Run gofumpt (pull_request) Successful in 3m54s
Tests and linters / Tests (pull_request) Successful in 4m12s
Tests and linters / gopls check (pull_request) Successful in 4m31s
Tests and linters / Tests with -race (pull_request) Successful in 4m38s
OCI image / Build container images (push) Failing after 18s
Vulncheck / Vulncheck (push) Successful in 1m2s
Pre-commit hooks / Pre-commit (push) Successful in 1m39s
Build / Build Components (push) Successful in 1m45s
Tests and linters / Staticcheck (push) Successful in 2m18s
Tests and linters / Run gofumpt (push) Successful in 2m46s
Tests and linters / Lint (push) Successful in 3m5s
Tests and linters / Tests with -race (push) Successful in 3m23s
Tests and linters / Tests (push) Successful in 3m52s
Tests and linters / gopls check (push) Successful in 4m18s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-02-05 16:38:20 +03:00
4de5fca547
[ #1624 ] blobovniczatree: Return source error from Init()
...
DCO action / DCO (pull_request) Successful in 39s
Vulncheck / Vulncheck (pull_request) Successful in 54s
Build / Build Components (pull_request) Successful in 1m23s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m24s
Tests and linters / Run gofumpt (pull_request) Successful in 1m32s
Tests and linters / Staticcheck (pull_request) Successful in 1m56s
Tests and linters / Tests (pull_request) Successful in 2m28s
Tests and linters / Lint (pull_request) Successful in 2m38s
Tests and linters / Tests with -race (pull_request) Successful in 3m47s
Tests and linters / gopls check (pull_request) Successful in 3m49s
Vulncheck / Vulncheck (push) Successful in 1m7s
Pre-commit hooks / Pre-commit (push) Successful in 1m28s
Tests and linters / Run gofumpt (push) Successful in 1m42s
Build / Build Components (push) Successful in 2m9s
Tests and linters / Staticcheck (push) Successful in 2m16s
Tests and linters / Tests (push) Successful in 2m32s
Tests and linters / Lint (push) Successful in 3m6s
Tests and linters / Tests with -race (push) Successful in 3m36s
Tests and linters / gopls check (push) Successful in 3m47s
OCI image / Build container images (push) Successful in 4m28s
As it was before: when the database was opened, an error returned,
but along with the original error, the `context cancelled`` error returned,
because `iterateIncompletedRebuildDBPaths` method has `ctx.Done()` check
and egCtx passed to `iterateIncompletedRebuildDBPaths` method.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-03 17:36:21 +03:00
e1a984e9d8
[ #1620 ] logs: Drop redundant trace_id fields
...
DCO action / DCO (pull_request) Successful in 39s
Tests and linters / Run gofumpt (pull_request) Successful in 38s
Vulncheck / Vulncheck (pull_request) Successful in 52s
Tests and linters / Staticcheck (pull_request) Successful in 2m4s
Build / Build Components (pull_request) Successful in 2m21s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m23s
Tests and linters / Tests with -race (pull_request) Successful in 4m8s
Tests and linters / Lint (pull_request) Successful in 5m47s
Tests and linters / Tests (pull_request) Successful in 6m31s
Tests and linters / gopls check (pull_request) Successful in 2m37s
Tests and linters / Run gofumpt (push) Successful in 32s
Vulncheck / Vulncheck (push) Successful in 58s
Pre-commit hooks / Pre-commit (push) Successful in 1m23s
Build / Build Components (push) Successful in 1m43s
Tests and linters / Staticcheck (push) Successful in 2m24s
Tests and linters / Lint (push) Successful in 3m43s
OCI image / Build container images (push) Successful in 4m34s
Tests and linters / Tests (push) Successful in 4m37s
Tests and linters / Tests with -race (push) Successful in 5m21s
Tests and linters / gopls check (push) Successful in 5m29s
`trace_id` is taken from context.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-29 16:13:51 +03:00
c98357606b
[ #1606 ] Use slices.Clone()/bytes.Clone() where possible
...
Tests and linters / Run gofumpt (pull_request) Successful in 28s
DCO action / DCO (pull_request) Successful in 42s
Vulncheck / Vulncheck (pull_request) Successful in 1m0s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m33s
Build / Build Components (pull_request) Successful in 1m46s
Tests and linters / Staticcheck (pull_request) Successful in 2m0s
Tests and linters / Lint (pull_request) Successful in 2m55s
Tests and linters / Tests (pull_request) Successful in 4m17s
Tests and linters / Tests with -race (pull_request) Successful in 5m14s
Tests and linters / gopls check (pull_request) Successful in 5m17s
Tests and linters / Run gofumpt (push) Successful in 27s
Vulncheck / Vulncheck (push) Successful in 51s
Pre-commit hooks / Pre-commit (push) Successful in 1m25s
Build / Build Components (push) Successful in 2m10s
Tests and linters / Staticcheck (push) Successful in 2m25s
Tests and linters / Lint (push) Successful in 3m24s
Tests and linters / Tests (push) Successful in 4m4s
OCI image / Build container images (push) Successful in 4m21s
Tests and linters / Tests with -race (push) Successful in 4m25s
Tests and linters / gopls check (push) Successful in 4m49s
gopatch:
```
@@
var from, to expression
@@
+import "bytes"
-to := make([]byte, len(from))
-copy(to, from)
+to := bytes.Clone(from)
@@
var from, to expression
@@
+import "bytes"
-to = make([]byte, len(from))
-copy(to, from)
+to = bytes.Clone(from)
@@
var from, to, typ expression
@@
+import "slices"
-to := make([]typ, len(from))
-copy(to, from)
+to := slices.Clone(from)
@@
var from, to, typ expression
@@
+import "slices"
-to = make([]typ, len(from))
-copy(to, from)
+to = slices.Clone(from)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-17 14:50:14 +03:00
05fd999162
[ #1600 ] fstree: Handle incomplete writes
...
DCO action / DCO (pull_request) Successful in 37s
Vulncheck / Vulncheck (pull_request) Successful in 59s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m22s
Build / Build Components (pull_request) Successful in 1m40s
Tests and linters / gopls check (pull_request) Successful in 2m16s
Tests and linters / Run gofumpt (pull_request) Successful in 2m51s
Tests and linters / Tests (pull_request) Successful in 3m23s
Tests and linters / Lint (pull_request) Successful in 3m40s
Tests and linters / Staticcheck (pull_request) Successful in 3m36s
Tests and linters / Tests with -race (pull_request) Successful in 4m14s
Tests and linters / Run gofumpt (push) Successful in 33s
Vulncheck / Vulncheck (push) Successful in 50s
Pre-commit hooks / Pre-commit (push) Successful in 1m28s
Build / Build Components (push) Successful in 1m37s
Tests and linters / Staticcheck (push) Successful in 3m12s
Tests and linters / Lint (push) Successful in 3m23s
Tests and linters / Tests with -race (push) Successful in 4m55s
Tests and linters / Tests (push) Successful in 1m46s
Tests and linters / gopls check (push) Successful in 2m23s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-14 14:52:35 +03:00
eff95bd632
[ #1598 ] engine: Drop unnecessary result structs
...
DCO action / DCO (pull_request) Successful in 31s
Tests and linters / Run gofumpt (pull_request) Successful in 29s
Vulncheck / Vulncheck (pull_request) Successful in 54s
Build / Build Components (pull_request) Successful in 1m29s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m28s
Tests and linters / gopls check (pull_request) Successful in 2m15s
Tests and linters / Lint (pull_request) Successful in 2m41s
Tests and linters / Staticcheck (pull_request) Successful in 2m49s
Tests and linters / Tests (pull_request) Successful in 3m52s
Tests and linters / Tests with -race (pull_request) Successful in 5m1s
Vulncheck / Vulncheck (push) Successful in 1m5s
Pre-commit hooks / Pre-commit (push) Successful in 1m19s
Build / Build Components (push) Successful in 1m36s
Tests and linters / gopls check (push) Successful in 2m16s
Tests and linters / Run gofumpt (push) Successful in 2m32s
Tests and linters / Staticcheck (push) Successful in 2m49s
Tests and linters / Lint (push) Successful in 3m10s
Tests and linters / Tests (push) Successful in 3m12s
Tests and linters / Tests with -race (push) Successful in 4m15s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-14 11:15:21 +03:00
fb928616cc
[ #1598 ] golangci: Enable unparam linter
...
DCO action / DCO (pull_request) Successful in 35s
Vulncheck / Vulncheck (pull_request) Successful in 1m6s
Build / Build Components (pull_request) Successful in 1m45s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m44s
Tests and linters / Staticcheck (pull_request) Successful in 1m59s
Tests and linters / gopls check (pull_request) Successful in 2m19s
Tests and linters / Run gofumpt (pull_request) Successful in 2m35s
Tests and linters / Lint (pull_request) Successful in 2m56s
Tests and linters / Tests with -race (pull_request) Successful in 3m48s
Tests and linters / Tests (pull_request) Successful in 3m58s
To drop unnecessary parameters and return values.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-14 09:06:47 +03:00
4d5ae59a52
[ #1598 ] golangci: Enable unconvert linters
...
To drop unnecessary conversions.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-14 09:06:40 +03:00
6c51f48aab
[ #1596 ] metrics: Create public aliases for internal engine metrics
...
Vulncheck / Vulncheck (push) Successful in 58s
Pre-commit hooks / Pre-commit (push) Successful in 1m21s
Build / Build Components (push) Successful in 1m33s
Tests and linters / gopls check (push) Successful in 2m14s
Tests and linters / Run gofumpt (push) Successful in 2m55s
Tests and linters / Tests (push) Successful in 2m59s
Tests and linters / Staticcheck (push) Successful in 3m7s
Tests and linters / Lint (push) Successful in 3m15s
Tests and linters / Tests with -race (push) Successful in 3m44s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-01-13 10:05:01 +00:00
3900b92927
Revert "[ #1492 ] metabase: Ensure Unmarshal() is called on a cloned slice"
...
DCO action / DCO (pull_request) Successful in 42s
Tests and linters / Run gofumpt (pull_request) Successful in 44s
Vulncheck / Vulncheck (pull_request) Successful in 1m20s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m38s
Build / Build Components (pull_request) Successful in 1m55s
Tests and linters / Tests (pull_request) Successful in 2m32s
Tests and linters / Staticcheck (pull_request) Successful in 2m36s
Tests and linters / gopls check (pull_request) Successful in 2m51s
Tests and linters / Lint (pull_request) Successful in 3m5s
Tests and linters / Tests with -race (pull_request) Successful in 3m57s
Build / Build Components (push) Has been cancelled
Pre-commit hooks / Pre-commit (push) Has been cancelled
Tests and linters / Lint (push) Has been cancelled
Tests and linters / Tests (push) Has been cancelled
Tests and linters / Tests with -race (push) Has been cancelled
Tests and linters / Staticcheck (push) Has been cancelled
Tests and linters / gopls check (push) Has been cancelled
Tests and linters / Run gofumpt (push) Has been cancelled
Vulncheck / Vulncheck (push) Has been cancelled
This reverts commit 8ed7a676d5
2025-01-09 14:34:20 +03:00
fa08bfa553
[ #1583 ] metabase/test: Update TestLisObjectsWithCursor
...
DCO action / DCO (pull_request) Successful in 4m37s
Vulncheck / Vulncheck (pull_request) Successful in 5m1s
Tests and linters / Run gofumpt (pull_request) Successful in 5m36s
Build / Build Components (pull_request) Successful in 6m3s
Pre-commit hooks / Pre-commit (pull_request) Successful in 6m0s
Tests and linters / Staticcheck (pull_request) Successful in 6m29s
Tests and linters / gopls check (pull_request) Successful in 6m41s
Tests and linters / Tests with -race (pull_request) Successful in 6m56s
Tests and linters / Tests (pull_request) Successful in 7m5s
Tests and linters / Lint (pull_request) Successful in 7m13s
Vulncheck / Vulncheck (push) Successful in 5m9s
Tests and linters / Run gofumpt (push) Successful in 5m18s
Pre-commit hooks / Pre-commit (push) Successful in 5m37s
Build / Build Components (push) Successful in 6m6s
Tests and linters / Staticcheck (push) Successful in 6m30s
Tests and linters / gopls check (push) Successful in 6m33s
Tests and linters / Tests (push) Successful in 7m6s
Tests and linters / Tests with -race (push) Successful in 7m7s
Tests and linters / Lint (push) Successful in 7m26s
Update this test following recent changes to ensure
that `(*DB).ListWithCursor` skips expired objects.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-26 14:39:50 +03:00
0da998ef50
[ #1583 ] metabase: Skip expired objects in ListWithCursor
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-26 14:39:49 +03:00
226dd25dd0
[ #1568 ] pilorama: Replace "containerID" with "container ID" in the error message
...
Tests and linters / Run gofumpt (push) Successful in 2m46s
Tests and linters / Staticcheck (push) Successful in 3m31s
Tests and linters / Lint (push) Successful in 5m0s
Vulncheck / Vulncheck (push) Successful in 7m2s
Build / Build Components (push) Successful in 7m38s
Pre-commit hooks / Pre-commit (push) Successful in 7m54s
Tests and linters / Tests (push) Successful in 8m11s
Tests and linters / Tests with -race (push) Successful in 8m39s
Tests and linters / gopls check (push) Successful in 8m57s
It is "container ID" in every other place.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-18 15:52:26 +00:00
bd0197eaa8
[ #1568 ] storage: Remove "could not/can't/failed to" from error messages
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-18 15:52:26 +00:00
f0c43c8d80
[ #1502 ] Use zap.Error for logging errors
...
Vulncheck / Vulncheck (pull_request) Successful in 3m1s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m29s
Tests and linters / gopls check (pull_request) Successful in 3m50s
Tests and linters / Lint (pull_request) Successful in 4m35s
DCO action / DCO (pull_request) Successful in 5m12s
Tests and linters / Run gofumpt (pull_request) Successful in 5m33s
Build / Build Components (pull_request) Successful in 5m45s
Tests and linters / Tests with -race (pull_request) Successful in 6m37s
Tests and linters / Tests (pull_request) Successful in 7m17s
Tests and linters / Staticcheck (pull_request) Successful in 7m36s
Tests and linters / Run gofumpt (push) Successful in 1m22s
Tests and linters / Staticcheck (push) Successful in 3m19s
Tests and linters / Lint (push) Successful in 4m35s
Vulncheck / Vulncheck (push) Successful in 5m20s
Build / Build Components (push) Successful in 6m16s
Pre-commit hooks / Pre-commit (push) Successful in 6m37s
Tests and linters / Tests (push) Successful in 6m48s
Tests and linters / Tests with -race (push) Successful in 7m15s
Tests and linters / gopls check (push) Successful in 7m27s
Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like
`zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`.
Leave similar expressions with other messages unchanged, for example,
`zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`.
This change was made by applying the following patch:
```diff
@@
var err expression
@@
-zap.String("error", err.Error())
+zap.Error(err)
@@
var err expression
@@
-zap.String("err", err.Error())
+zap.Error(err)
```
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-16 11:13:42 +03:00
8ba9f31fca
[ #1510 ] metabase/test: Fix BenchmarkListWithCursor
...
DCO action / DCO (pull_request) Successful in 4m10s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m53s
Tests and linters / gopls check (pull_request) Successful in 5m9s
Build / Build Components (pull_request) Successful in 5m36s
Tests and linters / Lint (pull_request) Successful in 6m0s
Tests and linters / Tests (pull_request) Successful in 6m4s
Tests and linters / Run gofumpt (pull_request) Successful in 6m5s
Vulncheck / Vulncheck (pull_request) Successful in 6m48s
Tests and linters / Tests with -race (pull_request) Successful in 7m35s
Tests and linters / Staticcheck (pull_request) Successful in 7m42s
Tests and linters / Staticcheck (push) Successful in 3m50s
Tests and linters / gopls check (push) Successful in 3m57s
Build / Build Components (push) Successful in 4m30s
Tests and linters / Lint (push) Successful in 4m50s
Tests and linters / Run gofumpt (push) Successful in 5m38s
Vulncheck / Vulncheck (push) Successful in 6m51s
Pre-commit hooks / Pre-commit (push) Successful in 7m27s
Tests and linters / Tests (push) Successful in 7m45s
Tests and linters / Tests with -race (push) Successful in 8m26s
- Fix misplaced `(*DB).Close` (broken after 47dcfa20f3fcdbf5e509
2024-12-13 13:19:15 +03:00
2af3409d39
[ #1510 ] metabase/test: Fix BenchmarkGet
...
Fix misplaced `(*DB).Close` (broken after 47dcfa20f3
2024-12-13 13:18:43 +03:00
