Speedup metabase.Select(), part 1 #1683

Merged
fyrchik merged 2 commits from fyrchik/frostfs-node:speedup-search-1 into master 2025-03-18 11:48:52 +00:00
Owner

objectStatus() is called twice for the same object:
First, in selectObject() to filter removed objects.
Then, again, in getObjectForSlowFilters() via db.get().
The second call will return the same result, so remove useless branch.

goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                          │     old     │                status                │
                          │   sec/op    │    sec/op     vs base                │
Select/string_equal-8       5.022m ± 7%   3.968m ±  8%  -20.98% (p=0.000 n=10)
Select/string_not_equal-8   4.953m ± 9%   3.990m ± 10%  -19.44% (p=0.000 n=10)
Select/common_prefix-8      4.962m ± 8%   3.971m ±  9%  -19.98% (p=0.000 n=10)
Select/unknown-8            5.246m ± 9%   3.548m ±  5%  -32.37% (p=0.000 n=10)
geomean                     5.045m        3.865m        -23.39%

                              │     old      │                status                │
                              │     B/op     │     B/op      vs base                │
Select/string_equal-8       2.685Mi ± 0%   2.250Mi ± 0%  -16.20% (p=0.000 n=10)
Select/string_not_equal-8   2.685Mi ± 0%   2.250Mi ± 0%  -16.20% (p=0.000 n=10)
Select/common_prefix-8      2.685Mi ± 0%   2.250Mi ± 0%  -16.20% (p=0.000 n=10)
Select/unknown-8            2.677Mi ± 0%   2.243Mi ± 0%  -16.24% (p=0.000 n=10)
geomean                     2.683Mi        2.248Mi       -16.21%

                          │     old     │               status                │
                          │  allocs/op  │  allocs/op   vs base                │
Select/string_equal-8       69.03k ± 0%   56.02k ± 0%  -18.84% (p=0.000 n=10)
Select/string_not_equal-8   69.03k ± 0%   56.02k ± 0%  -18.84% (p=0.000 n=10)
Select/common_prefix-8      69.03k ± 0%   56.02k ± 0%  -18.84% (p=0.000 n=10)
Select/unknown-8            68.03k ± 0%   55.03k ± 0%  -19.11% (p=0.000 n=10)
geomean                     68.78k        55.77k       -18.90%
objectStatus() is called twice for the same object: First, in selectObject() to filter removed objects. Then, again, in getObjectForSlowFilters() via db.get(). The second call will return the same result, so remove useless branch. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ status │ │ sec/op │ sec/op vs base │ Select/string_equal-8 5.022m ± 7% 3.968m ± 8% -20.98% (p=0.000 n=10) Select/string_not_equal-8 4.953m ± 9% 3.990m ± 10% -19.44% (p=0.000 n=10) Select/common_prefix-8 4.962m ± 8% 3.971m ± 9% -19.98% (p=0.000 n=10) Select/unknown-8 5.246m ± 9% 3.548m ± 5% -32.37% (p=0.000 n=10) geomean 5.045m 3.865m -23.39% │ old │ status │ │ B/op │ B/op vs base │ Select/string_equal-8 2.685Mi ± 0% 2.250Mi ± 0% -16.20% (p=0.000 n=10) Select/string_not_equal-8 2.685Mi ± 0% 2.250Mi ± 0% -16.20% (p=0.000 n=10) Select/common_prefix-8 2.685Mi ± 0% 2.250Mi ± 0% -16.20% (p=0.000 n=10) Select/unknown-8 2.677Mi ± 0% 2.243Mi ± 0% -16.24% (p=0.000 n=10) geomean 2.683Mi 2.248Mi -16.21% │ old │ status │ │ allocs/op │ allocs/op vs base │ Select/string_equal-8 69.03k ± 0% 56.02k ± 0% -18.84% (p=0.000 n=10) Select/string_not_equal-8 69.03k ± 0% 56.02k ± 0% -18.84% (p=0.000 n=10) Select/common_prefix-8 69.03k ± 0% 56.02k ± 0% -18.84% (p=0.000 n=10) Select/unknown-8 68.03k ± 0% 55.03k ± 0% -19.11% (p=0.000 n=10) geomean 68.78k 55.77k -18.90% ```
fyrchik added 2 commits 2025-03-18 07:19:40 +00:00
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
metabase: Check object status once in Select()
Some checks failed
DCO action / DCO (pull_request) Failing after 36s
Vulncheck / Vulncheck (pull_request) Successful in 1m5s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m33s
Build / Build Components (pull_request) Successful in 1m40s
Tests and linters / Run gofumpt (pull_request) Successful in 3m37s
Tests and linters / Lint (pull_request) Successful in 3m52s
Tests and linters / Staticcheck (pull_request) Successful in 4m0s
Tests and linters / Tests with -race (pull_request) Successful in 4m16s
Tests and linters / Tests (pull_request) Successful in 4m24s
Tests and linters / gopls check (pull_request) Successful in 5m26s
3753f0a178
objectStatus() is called twice for the same object:
First, in selectObject() to filter removed objects.
Then, again, in getObjectForSlowFilters() via db.get().
The second call will return the same result, so remove useless branch.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                          │     old     │                status                │
                          │   sec/op    │    sec/op     vs base                │
Select/string_equal-8       5.022m ± 7%   3.968m ±  8%  -20.98% (p=0.000 n=10)
Select/string_not_equal-8   4.953m ± 9%   3.990m ± 10%  -19.44% (p=0.000 n=10)
Select/common_prefix-8      4.962m ± 8%   3.971m ±  9%  -19.98% (p=0.000 n=10)
Select/unknown-8            5.246m ± 9%   3.548m ±  5%  -32.37% (p=0.000 n=10)
geomean                     5.045m        3.865m        -23.39%

                          │     old      │                status                │
                          │     B/op     │     B/op      vs base                │
Select/string_equal-8       2.685Mi ± 0%   2.250Mi ± 0%  -16.20% (p=0.000 n=10)
Select/string_not_equal-8   2.685Mi ± 0%   2.250Mi ± 0%  -16.20% (p=0.000 n=10)
Select/common_prefix-8      2.685Mi ± 0%   2.250Mi ± 0%  -16.20% (p=0.000 n=10)
Select/unknown-8            2.677Mi ± 0%   2.243Mi ± 0%  -16.24% (p=0.000 n=10)
geomean                     2.683Mi        2.248Mi       -16.21%

                          │     old     │               status                │
                          │  allocs/op  │  allocs/op   vs base                │
Select/string_equal-8       69.03k ± 0%   56.02k ± 0%  -18.84% (p=0.000 n=10)
Select/string_not_equal-8   69.03k ± 0%   56.02k ± 0%  -18.84% (p=0.000 n=10)
Select/common_prefix-8      69.03k ± 0%   56.02k ± 0%  -18.84% (p=0.000 n=10)
Select/unknown-8            68.03k ± 0%   55.03k ± 0%  -19.11% (p=0.000 n=10)
geomean                     68.78k        55.77k       -18.90%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
requested reviews from storage-core-committers, storage-core-developers 2025-03-18 07:19:41 +00:00
dstepanov-yadro approved these changes 2025-03-18 07:37:41 +00:00
acid-ant approved these changes 2025-03-18 07:48:46 +00:00
fyrchik force-pushed speedup-search-1 from 3753f0a178 to b362823558 2025-03-18 11:21:46 +00:00 Compare
aarifullin approved these changes 2025-03-18 11:47:23 +00:00
fyrchik added this to the v0.45.0 milestone 2025-03-18 11:48:15 +00:00
fyrchik added the
frostfs-node
internal
perfomance
labels 2025-03-18 11:48:47 +00:00
fyrchik merged commit a405fb1f39 into master 2025-03-18 11:48:52 +00:00
fyrchik deleted branch speedup-search-1 2025-03-18 11:48:54 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1683
No description provided.