Evgenii Stratonikov
16a7740ccd
[ #8 ] hrw: Introduce StringHash() for hashing strings directly
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 6 │ 7 │
│ sec/op │ sec/op vs base │
SortHashersByValue_Typed_fnv_10-8 248.8n ± 1% 166.9n ± 9% -32.93% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 2.195µ ± 6% 1.297µ ± 6% -40.93% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 22.47µ ± 4% 12.42µ ± 10% -44.73% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 301.7n ± 6% 180.8n ± 4% -40.09% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 2.526µ ± 1% 1.378µ ± 5% -45.47% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 24.37µ ± 2% 12.80µ ± 4% -47.48% (p=0.000 n=10)
geomean 2.472µ 1.430µ -42.13%
│ 6 │ 7 │
│ B/op │ B/op vs base │
SortHashersByValue_Typed_fnv_10-8 144.0 ± 0% 144.0 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByValue_Typed_fnv_100-8 960.0 ± 0% 960.0 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByValue_Typed_fnv_1000-8 8.062Ki ± 0% 8.062Ki ± 0% ~ (p=1.000 n=10) ¹
SortHashersByWeightValueTyped_fnv_10-8 144.0 ± 0% 144.0 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByWeightValueTyped_fnv_100-8 960.0 ± 0% 960.0 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByWeightValueTyped_fnv_1000-8 8.062Ki ± 0% 8.062Ki ± 0% ~ (p=1.000 n=10) ¹
geomean 1.021Ki 1.021Ki +0.00%
¹ all samples are equal
│ 6 │ 7 │
│ allocs/op │ allocs/op vs base │
SortHashersByValue_Typed_fnv_10-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByValue_Typed_fnv_100-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByValue_Typed_fnv_1000-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByWeightValueTyped_fnv_10-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByWeightValueTyped_fnv_100-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
SortHashersByWeightValueTyped_fnv_1000-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
geomean 2.000 2.000 +0.00%
¹ all samples are equal
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 10:56:24 +03:00
Evgenii Stratonikov
266da7c69a
[ #8 ] hrw: Do not allocate for swap()/less() helpers
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 4 │ 5 │
│ sec/op │ sec/op vs base │
SortHashersByValue_Typed_fnv_10-8 309.2n ± 2% 294.4n ± 1% -4.75% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 2.306µ ± 1% 2.549µ ± 1% +10.54% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 21.73µ ± 1% 24.80µ ± 3% +14.14% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 347.1n ± 1% 334.8n ± 2% -3.56% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 2.668µ ± 1% 2.954µ ± 3% +10.72% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 2.673µ ± 1% 2.957µ ± 4% +10.63% (p=0.000 n=10)
geomean 1.836µ 1.947µ +6.01%
│ 4 │ 5 │
│ B/op │ B/op vs base │
SortHashersByValue_Typed_fnv_10-8 216.0 ± 0% 144.0 ± 0% -33.33% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 1032.0 ± 0% 960.0 ± 0% -6.98% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 8.133Ki ± 0% 8.062Ki ± 0% -0.86% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 216.0 ± 0% 144.0 ± 0% -33.33% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 1032.0 ± 0% 960.0 ± 0% -6.98% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 1032.0 ± 0% 960.0 ± 0% -6.98% (p=0.000 n=10)
geomean 867.8 730.1 -15.87%
│ 4 │ 5 │
│ allocs/op │ allocs/op vs base │
SortHashersByValue_Typed_fnv_10-8 4.000 ± 0% 2.000 ± 0% -50.00% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 4.000 ± 0% 2.000 ± 0% -50.00% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 4.000 ± 0% 2.000 ± 0% -50.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 4.000 ± 0% 2.000 ± 0% -50.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 4.000 ± 0% 2.000 ± 0% -50.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 4.000 ± 0% 2.000 ± 0% -50.00% (p=0.000 n=10)
geomean 4.000 2.000 -50.00%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 10:56:23 +03:00
Evgenii Stratonikov
c52f74d8e1
[ #8 ] hrw: Do not allocate 2 slices for sort
...
Currently we allocate `rule` and then create `dist` which depends on it.
In this commit we create `dist` directly.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 3 │ 4 │
│ sec/op │ sec/op vs base │
SortHashersByValue_Typed_fnv_10-8 336.3n ± 3% 309.2n ± 2% -8.06% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 2.424µ ± 3% 2.306µ ± 1% -4.87% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 22.35µ ± 1% 21.73µ ± 1% -2.75% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 346.6n ± 3% 347.1n ± 1% ~ (p=0.631 n=10)
SortHashersByWeightValueTyped_fnv_100-8 2.637µ ± 6% 2.668µ ± 1% ~ (p=0.481 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 2.609µ ± 4% 2.673µ ± 1% +2.43% (p=0.000 n=10)
geomean 1.875µ 1.836µ -2.06%
│ 3 │ 4 │
│ B/op │ B/op vs base │
SortHashersByValue_Typed_fnv_10-8 296.0 ± 0% 216.0 ± 0% -27.03% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 1.883Ki ± 0% 1.008Ki ± 0% -46.47% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 16.133Ki ± 0% 8.133Ki ± 0% -49.59% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 296.0 ± 0% 216.0 ± 0% -27.03% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 1.883Ki ± 0% 1.008Ki ± 0% -46.47% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 1.883Ki ± 0% 1.008Ki ± 0% -46.47% (p=0.000 n=10)
geomean 1.442Ki 867.8 -41.24%
│ 3 │ 4 │
│ allocs/op │ allocs/op vs base │
SortHashersByValue_Typed_fnv_10-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
geomean 5.000 4.000 -20.00%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 10:53:44 +03:00
Evgenii Stratonikov
895ecf150f
[ #8 ] hrw: Inline swap() when slice is known
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 2 │ 3 │
│ sec/op │ sec/op vs base │
SortHashersByValue_Typed_fnv_10-8 368.5n ± 2% 336.3n ± 3% -8.75% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 2.411µ ± 4% 2.424µ ± 3% ~ (p=0.853 n=10)
SortHashersByValue_Typed_fnv_1000-8 22.19µ ± 2% 22.35µ ± 1% ~ (p=0.247 n=10)
SortHashersByWeightValueTyped_fnv_10-8 364.3n ± 2% 346.6n ± 3% -4.86% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 2.541µ ± 3% 2.637µ ± 6% ~ (p=0.055 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 2.483µ ± 1% 2.609µ ± 4% +5.07% (p=0.003 n=10)
geomean 1.888µ 1.875µ -0.71%
│ 2 │ 3 │
│ B/op │ B/op vs base │
SortHashersByValue_Typed_fnv_10-8 312.0 ± 0% 296.0 ± 0% -5.13% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 1.898Ki ± 0% 1.883Ki ± 0% -0.82% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 16.15Ki ± 0% 16.13Ki ± 0% -0.10% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 312.0 ± 0% 296.0 ± 0% -5.13% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 1.898Ki ± 0% 1.883Ki ± 0% -0.82% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 1.898Ki ± 0% 1.883Ki ± 0% -0.82% (p=0.000 n=10)
geomean 1.474Ki 1.442Ki -2.16%
│ 2 │ 3 │
│ allocs/op │ allocs/op vs base │
SortHashersByValue_Typed_fnv_10-8 6.000 ± 0% 5.000 ± 0% -16.67% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 6.000 ± 0% 5.000 ± 0% -16.67% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 6.000 ± 0% 5.000 ± 0% -16.67% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 6.000 ± 0% 5.000 ± 0% -16.67% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 6.000 ± 0% 5.000 ± 0% -16.67% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 6.000 ± 0% 5.000 ± 0% -16.67% (p=0.000 n=10)
geomean 6.000 5.000 -16.67%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-01 21:41:16 +03:00
Evgenii Stratonikov
213c105ac1
[ #8 ] go.mod: Use faster murmur3 lib
...
Specifically, this line became possible, because of noescape annotations
for assembly.
```
./hrw.go:307:14: make([]byte, 8) does not escape
```
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 1 │ 2 │
│ sec/op │ sec/op vs base │
SortHashersByValue_Typed_fnv_10-8 580.1n ± 1% 368.5n ± 2% -36.47% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 4.215µ ± 2% 2.411µ ± 4% -42.79% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 39.40µ ± 1% 22.19µ ± 2% -43.68% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 599.6n ± 2% 364.3n ± 2% -39.25% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 4.337µ ± 5% 2.541µ ± 3% -41.41% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 4.344µ ± 3% 2.483µ ± 1% -42.84% (p=0.000 n=10)
geomean 4.400µ 1.888µ -41.13% ¹
¹ benchmark set differs from baseline; geomeans may not be comparable
│ 1 │ 2 │
│ B/op │ B/op vs base │
SortHashersByValue_Typed_fnv_10-8 472.0 ± 0% 312.0 ± 0% -33.90% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 3.461Ki ± 0% 1.898Ki ± 0% -45.15% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 31.77Ki ± 0% 16.15Ki ± 0% -49.18% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 472.0 ± 0% 312.0 ± 0% -33.90% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 3.461Ki ± 0% 1.898Ki ± 0% -45.15% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 3.461Ki ± 0% 1.898Ki ± 0% -45.15% (p=0.000 n=10)
geomean 3.070Ki 1.474Ki -42.37% ¹
¹ benchmark set differs from baseline; geomeans may not be comparable
│ 1 │ 2 │
│ allocs/op │ allocs/op vs base │
SortHashersByValue_Typed_fnv_10-8 16.000 ± 0% 6.000 ± 0% -62.50% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 106.000 ± 0% 6.000 ± 0% -94.34% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 1006.000 ± 0% 6.000 ± 0% -99.40% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 16.000 ± 0% 6.000 ± 0% -62.50% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 106.000 ± 0% 6.000 ± 0% -94.34% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 106.000 ± 0% 6.000 ± 0% -94.34% (p=0.000 n=10)
geomean 113.0 6.000 -92.69% ¹
¹ benchmark set differs from baseline; geomeans may not be comparable
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-01 21:41:00 +03:00
Evgenii Stratonikov
c175ef4099
[ #8 ] hrw: Do not create index slice for sorter
...
`ind` is only needed to index dist or weights, swap them directly.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ 0 │ 1 │
│ sec/op │ sec/op vs base │
SortHashersByValue_Typed_fnv_10-8 596.2n ± 4% 580.1n ± 1% -2.72% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 4.453µ ± 2% 4.215µ ± 2% -5.35% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 41.58µ ± 4% 39.40µ ± 1% -5.23% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 624.5n ± 2% 599.6n ± 2% -3.99% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 4.593µ ± 2% 4.337µ ± 5% -5.56% (p=0.003 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 4.896µ ± 8% 4.344µ ± 3% -11.27% (p=0.000 n=10)
geomean 4.668µ 4.400µ -5.75%
│ 0 │ 1 │
│ B/op │ B/op vs base │
SortHashersByValue_Typed_fnv_10-8 584.0 ± 0% 472.0 ± 0% -19.18% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 4.367Ki ± 0% 3.461Ki ± 0% -20.75% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 39.80Ki ± 0% 31.77Ki ± 0% -20.18% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 600.0 ± 0% 472.0 ± 0% -21.33% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 4.383Ki ± 0% 3.461Ki ± 0% -21.03% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 4.383Ki ± 0% 3.461Ki ± 0% -21.03% (p=0.000 n=10)
geomean 3.742Ki 3.070Ki -17.96%
│ 0 │ 1 │
│ allocs/op │ allocs/op vs base │
SortHashersByValue_Typed_fnv_10-8 17.00 ± 0% 16.00 ± 0% -5.88% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8 107.0 ± 0% 106.0 ± 0% -0.93% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8 1.007k ± 0% 1.006k ± 0% -0.10% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8 17.00 ± 0% 16.00 ± 0% -5.88% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8 107.0 ± 0% 106.0 ± 0% -0.93% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8 107.0 ± 0% 106.0 ± 0% -0.93% (p=0.000 n=10)
geomean 115.3 113.0 -1.94%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-01 21:40:46 +03:00
Dmitrii Stepanov
5671632658
[ #2 ] hrw: Add typed methods for hashers
...
Add generic methods for sort-methods used in node
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-27 14:18:58 +03:00
Evgenii Stratonikov
22b833d972
Panic if provided value is not a slice
...
Signed-off-by: Evgenii Stratonikov <stratonikov@runbox.com>
2021-12-28 14:37:17 +03:00
Evgenii Stratonikov
dddcfc8fc5
Simplify SortByValue/Weight a bit
...
Get rid of unneeded types.
2020-09-11 18:30:18 +03:00
f52ea8fb21
Handle NaN values in weight validation function ( #7 )
...
NaN values can pass the validation since NaN values don't work
with float arithmetic and comparisons are always true. Now validation
function has check for NaN values.
2019-08-01 12:16:03 +03:00
aa230933d1
Move normalization routine out of hrw library ( #6 )
...
HRW library supports weighted sorting. Weights must be normalized
before applying. Since there could be different types of normalization
for multiple criteria, there is no point to perform simple
normalization in this library. Pass a slice of normalized weights
to the `SortByWeight` functions.
This commit proposes to:
- remove normalization routine from `SortByWeight` function;
- add `ValidateWeights` function to check if weights are normalized;
- rename `weight` -> `distance` to avoid naming confusion between
hash distance and actual weights;
- use testify lib in the tests;
2019-07-05 09:49:24 +03:00
58a8ce4e47
Added weighted HRW sorting ( #5 )
...
* Added weighted HRW sorting
This commit proposes renaming of old `SortByWeight` functions to `Sort`
and implementation of `SortByWeight` function with explicit weights in
arguments. `SortByWeight` function calculates normalized hashes of
nodes and normalized input weights. Then multiplies these values to
obtain node's actual weight for later sorting.
- renamed `SortByWeight` function to `Sort`
- added `SortByWeight`, `SortSliceByWeightValue` and
`SortSliceBeWeightIndex` functions
- moved code with reflection processing into `prepareRule` function
- added tests and benchmarks for new weighted functions
- added benchmark results into README
* Fixed comments
2019-05-27 10:45:29 +03:00
d32f396204
✌️ Fixed in SortSliceByValue issue when weight calculated twice ( #4 )
...
- fixes for SortSliceByValue
- fixes for tests
2019-05-24 12:51:39 +03:00
9a6fcdb1f8
🐌 hot fix types
2019-04-12 14:19:18 +03:00
e06640668d
🐿 Extend SortSliceByValue with more types
2019-04-12 14:09:49 +03:00
e8ba64d915
Small refactoring ( #2 )
...
- Use val.Index(0).Interface().(type) in switch-case of SortSliceByValue
- add go1.12 to TravisCI
2019-03-14 15:48:58 +03:00
098fd24704
Support for int32 slices ( #4 )
2019-02-01 12:57:05 +03:00
e5d4b47881
Work around Hashing and tests ( #2 )
...
* Work around Hashing and tests
* fixes
* add TravisCI
* fix lint
* so slow
* add badges
* work on README
* Improved tests and get rid of fnv (#3 )
2019-01-31 12:54:02 +03:00
cf9a7e5b89
Removed redundant m64 variable and tweaked uniform distribution test. ( #1 )
2019-01-30 12:32:34 +03:00
01e85efd7e
Base HRW implementation in golang
2019-01-30 01:58:30 +03:00
Evgenii Stratonikov
Dmitrii Stepanov
Alex Vanin
Evgeniy Kulikov
Evgeniy Kulikov