Small perfomance optimizations, cleanup #236

Merged
fyrchik merged 1 commit from fyrchik/frostfs-sdk-go:sort-faster into master 2024-09-04 19:51:16 +00:00
Owner

Total improvement:

goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │               weights                │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   8.384µ ± 10%  -19.35% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   7.998µ ±  6%  -20.89% (p=0.000 n=10)
geomean                                                          10.25µ         8.189µ        -20.12%

                                                              │     old      │               weights                │
                                                              │     B/op     │     B/op      vs base                │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   7.617Ki ± 0%  -12.40% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.039Ki ± 0%  -13.28% (p=0.000 n=10)
geomean                                                         8.401Ki        7.322Ki       -12.84%

                                                              │     old     │              weights               │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.00 ± 0%   77.00 ± 0%  -44.20% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.00 ± 0%   77.00 ± 0%  -44.20% (p=0.000 n=10)
geomean                                                          138.0        77.00       -44.20%
Total improvement: ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ weights │ │ sec/op │ sec/op vs base │ Netmap_ContainerNodes/REP_2-8 10.395µ ± 14% 8.384µ ± 10% -19.35% (p=0.000 n=10) Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8 10.110µ ± 16% 7.998µ ± 6% -20.89% (p=0.000 n=10) geomean 10.25µ 8.189µ -20.12% │ old │ weights │ │ B/op │ B/op vs base │ Netmap_ContainerNodes/REP_2-8 8.695Ki ± 0% 7.617Ki ± 0% -12.40% (p=0.000 n=10) Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8 8.117Ki ± 0% 7.039Ki ± 0% -13.28% (p=0.000 n=10) geomean 8.401Ki 7.322Ki -12.84% │ old │ weights │ │ allocs/op │ allocs/op vs base │ Netmap_ContainerNodes/REP_2-8 138.00 ± 0% 77.00 ± 0% -44.20% (p=0.000 n=10) Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8 138.00 ± 0% 77.00 ± 0% -44.20% (p=0.000 n=10) geomean 138.0 77.00 -44.20% ```
fyrchik added 5 commits 2024-07-12 10:48:43 +00:00
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   9.227µ ± 13%  -11.24% (p=0.015 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   9.189µ ±  7%        ~ (p=0.105 n=10)
geomean                                                          10.25µ         9.208µ        -10.18%

                                                              │     old      │                 new                 │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   8.320Ki ± 0%  -4.31% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.742Ki ± 0%  -4.62% (p=0.000 n=10)
geomean                                                         8.401Ki        8.026Ki       -4.47%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
geomean                                                         138.0        122.0       -11.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
netmap: Remove unused field from meanIQRAgg
Some checks failed
Tests and linters / Tests (1.20) (pull_request) Failing after 58s
DCO / DCO (pull_request) Failing after 1m9s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m48s
1453fb21ce
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
fyrchik added the
internal
perfomance
labels 2024-07-12 10:48:58 +00:00
fyrchik requested review from storage-core-committers 2024-07-12 10:49:52 +00:00
fyrchik requested review from storage-core-developers 2024-07-12 10:49:52 +00:00
fyrchik force-pushed sort-faster from 1453fb21ce to 917ceecc5d 2024-07-12 10:59:55 +00:00 Compare
acid-ant approved these changes 2024-07-12 11:08:40 +00:00
fyrchik force-pushed sort-faster from 917ceecc5d to 0ad31ef05f 2024-07-12 11:22:30 +00:00 Compare
fyrchik force-pushed sort-faster from 0ad31ef05f to ccae4bcace 2024-07-12 11:25:18 +00:00 Compare
fyrchik force-pushed sort-faster from ccae4bcace to 1decccc542 2024-07-12 12:42:14 +00:00 Compare
fyrchik force-pushed sort-faster from 1decccc542 to c4ff8a6cda 2024-07-12 12:43:21 +00:00 Compare
fyrchik requested review from storage-sdk-committers 2024-07-12 13:19:51 +00:00
fyrchik requested review from storage-sdk-developers 2024-07-12 13:19:56 +00:00
fyrchik requested review from storage-services-committers 2024-07-12 13:19:57 +00:00
fyrchik requested review from storage-services-developers 2024-07-12 13:19:59 +00:00
fyrchik merged commit c4ff8a6cda into master 2024-07-15 10:16:53 +00:00
fyrchik deleted branch sort-faster 2024-07-15 10:17:01 +00:00
aarifullin reviewed 2024-07-15 10:26:15 +00:00
@ -118,1 +116,3 @@
w := make([]float64, 0, len(n))
func (n nodes) appendWeightsTo(wf weightFunc, w []float64) []float64 {
if cap(w) < len(n) {
w = make([]float64, 0, len(n))
Member

The method's name is appendWeightsTo. Here you pass w, then just reallocate w but you don't save previous values. We need to fix that?

The method's name is `appendWeightsTo`. Here you pass `w`, then just reallocate `w` but you don't save previous values. We need to fix that?
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-core-developers
TrueCloudLab/storage-sdk-committers
TrueCloudLab/storage-sdk-developers
TrueCloudLab/storage-services-committers
TrueCloudLab/storage-services-developers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-sdk-go#236
No description provided.