TrueCloudLab/hrw
22
0
Fork 6
Code Issues 2 Pull requests Releases 2 Activity
33 commits 2 branches 13 tags 1.6 MiB
Commit graph

33 commits

This branch
This branch
All branches
Author SHA1 Message Date
Evgenii Stratonikov
c52f74d8e1 [#8] hrw: Do not allocate 2 slices for sort 
Currently we allocate `rule` and then create `dist` which depends on it.
In this commit we create `dist` directly.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                         │      3      │                 4                  │
                                         │   sec/op    │   sec/op     vs base               │
SortHashersByValue_Typed_fnv_10-8          336.3n ± 3%   309.2n ± 2%  -8.06% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8         2.424µ ± 3%   2.306µ ± 1%  -4.87% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8        22.35µ ± 1%   21.73µ ± 1%  -2.75% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8     346.6n ± 3%   347.1n ± 1%       ~ (p=0.631 n=10)
SortHashersByWeightValueTyped_fnv_100-8    2.637µ ± 6%   2.668µ ± 1%       ~ (p=0.481 n=10)
SortHashersByWeightValueTyped_fnv_1000-8   2.609µ ± 4%   2.673µ ± 1%  +2.43% (p=0.000 n=10)
geomean                                    1.875µ        1.836µ       -2.06%

                                         │       3       │                  4                   │
                                         │     B/op      │     B/op      vs base                │
SortHashersByValue_Typed_fnv_10-8             296.0 ± 0%     216.0 ± 0%  -27.03% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8          1.883Ki ± 0%   1.008Ki ± 0%  -46.47% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8        16.133Ki ± 0%   8.133Ki ± 0%  -49.59% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8        296.0 ± 0%     216.0 ± 0%  -27.03% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8     1.883Ki ± 0%   1.008Ki ± 0%  -46.47% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8    1.883Ki ± 0%   1.008Ki ± 0%  -46.47% (p=0.000 n=10)
geomean                                     1.442Ki          867.8       -41.24%

                                         │     3      │                 4                  │
                                         │ allocs/op  │ allocs/op   vs base                │
SortHashersByValue_Typed_fnv_10-8          5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8         5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8        5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8     5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8    5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8   5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
geomean                                    5.000        4.000       -20.00%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-02 10:53:44 +03:00
Evgenii Stratonikov
895ecf150f [#8] hrw: Inline swap() when slice is known 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                         │      2      │                 3                  │
                                         │   sec/op    │   sec/op     vs base               │
SortHashersByValue_Typed_fnv_10-8          368.5n ± 2%   336.3n ± 3%  -8.75% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8         2.411µ ± 4%   2.424µ ± 3%       ~ (p=0.853 n=10)
SortHashersByValue_Typed_fnv_1000-8        22.19µ ± 2%   22.35µ ± 1%       ~ (p=0.247 n=10)
SortHashersByWeightValueTyped_fnv_10-8     364.3n ± 2%   346.6n ± 3%  -4.86% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8    2.541µ ± 3%   2.637µ ± 6%       ~ (p=0.055 n=10)
SortHashersByWeightValueTyped_fnv_1000-8   2.483µ ± 1%   2.609µ ± 4%  +5.07% (p=0.003 n=10)
geomean                                    1.888µ        1.875µ       -0.71%

                                         │      2       │                  3                  │
                                         │     B/op     │     B/op      vs base               │
SortHashersByValue_Typed_fnv_10-8            312.0 ± 0%     296.0 ± 0%  -5.13% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8         1.898Ki ± 0%   1.883Ki ± 0%  -0.82% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8        16.15Ki ± 0%   16.13Ki ± 0%  -0.10% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8       312.0 ± 0%     296.0 ± 0%  -5.13% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8    1.898Ki ± 0%   1.883Ki ± 0%  -0.82% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8   1.898Ki ± 0%   1.883Ki ± 0%  -0.82% (p=0.000 n=10)
geomean                                    1.474Ki        1.442Ki       -2.16%

                                         │     2      │                 3                  │
                                         │ allocs/op  │ allocs/op   vs base                │
SortHashersByValue_Typed_fnv_10-8          6.000 ± 0%   5.000 ± 0%  -16.67% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8         6.000 ± 0%   5.000 ± 0%  -16.67% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8        6.000 ± 0%   5.000 ± 0%  -16.67% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8     6.000 ± 0%   5.000 ± 0%  -16.67% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8    6.000 ± 0%   5.000 ± 0%  -16.67% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8   6.000 ± 0%   5.000 ± 0%  -16.67% (p=0.000 n=10)
geomean                                    6.000        5.000       -16.67%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-01 21:41:16 +03:00
Evgenii Stratonikov
213c105ac1 [#8] go.mod: Use faster murmur3 lib 
Specifically, this line became possible, because of noescape annotations
for assembly.
```
./hrw.go:307:14: make([]byte, 8) does not escape
```

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                              │      1       │                   2                   │
                                              │    sec/op    │   sec/op     vs base                  │
SortHashersByValue_Typed_fnv_10-8               580.1n ±  1%   368.5n ± 2%  -36.47% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8              4.215µ ±  2%   2.411µ ± 4%  -42.79% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8             39.40µ ±  1%   22.19µ ± 2%  -43.68% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8          599.6n ±  2%   364.3n ± 2%  -39.25% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8         4.337µ ±  5%   2.541µ ± 3%  -41.41% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8        4.344µ ±  3%   2.483µ ± 1%  -42.84% (p=0.000 n=10)
geomean                                         4.400µ         1.888µ       -41.13%                ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                                              │      1       │                   2                    │
                                              │     B/op     │     B/op      vs base                  │
SortHashersByValue_Typed_fnv_10-8                 472.0 ± 0%     312.0 ± 0%  -33.90% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8              3.461Ki ± 0%   1.898Ki ± 0%  -45.15% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8             31.77Ki ± 0%   16.15Ki ± 0%  -49.18% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8            472.0 ± 0%     312.0 ± 0%  -33.90% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8         3.461Ki ± 0%   1.898Ki ± 0%  -45.15% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8        3.461Ki ± 0%   1.898Ki ± 0%  -45.15% (p=0.000 n=10)
geomean                                         3.070Ki        1.474Ki       -42.37%                ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                                              │       1       │                  2                   │
                                              │   allocs/op   │ allocs/op   vs base                  │
SortHashersByValue_Typed_fnv_10-8                 16.000 ± 0%   6.000 ± 0%  -62.50% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8               106.000 ± 0%   6.000 ± 0%  -94.34% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8             1006.000 ± 0%   6.000 ± 0%  -99.40% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8            16.000 ± 0%   6.000 ± 0%  -62.50% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8          106.000 ± 0%   6.000 ± 0%  -94.34% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8         106.000 ± 0%   6.000 ± 0%  -94.34% (p=0.000 n=10)
geomean                                            113.0        6.000       -92.69%                ¹
¹ benchmark set differs from baseline; geomeans may not be comparable
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-01 21:41:00 +03:00
Evgenii Stratonikov
c175ef4099 [#8] hrw: Do not create index slice for sorter 
`ind` is only needed to index dist or weights, swap them directly.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/hrw
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                              │      0      │                  1                   │
                                              │   sec/op    │    sec/op     vs base                │
SortHashersByValue_Typed_fnv_10-8               596.2n ± 4%   580.1n ±  1%   -2.72% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8              4.453µ ± 2%   4.215µ ±  2%   -5.35% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8             41.58µ ± 4%   39.40µ ±  1%   -5.23% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8          624.5n ± 2%   599.6n ±  2%   -3.99% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8         4.593µ ± 2%   4.337µ ±  5%   -5.56% (p=0.003 n=10)
SortHashersByWeightValueTyped_fnv_1000-8        4.896µ ± 8%   4.344µ ±  3%  -11.27% (p=0.000 n=10)
geomean                                         4.668µ        4.400µ         -5.75%

                                              │      0       │                  1                   │
                                              │     B/op     │     B/op      vs base                │
SortHashersByValue_Typed_fnv_10-8                 584.0 ± 0%     472.0 ± 0%  -19.18% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8              4.367Ki ± 0%   3.461Ki ± 0%  -20.75% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8             39.80Ki ± 0%   31.77Ki ± 0%  -20.18% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8            600.0 ± 0%     472.0 ± 0%  -21.33% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8         4.383Ki ± 0%   3.461Ki ± 0%  -21.03% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8        4.383Ki ± 0%   3.461Ki ± 0%  -21.03% (p=0.000 n=10)
geomean                                         3.742Ki        3.070Ki       -17.96%

                                              │      0      │                 1                  │
                                              │  allocs/op  │  allocs/op   vs base               │
SortHashersByValue_Typed_fnv_10-8                17.00 ± 0%    16.00 ± 0%  -5.88% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_100-8               107.0 ± 0%    106.0 ± 0%  -0.93% (p=0.000 n=10)
SortHashersByValue_Typed_fnv_1000-8             1.007k ± 0%   1.006k ± 0%  -0.10% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_10-8           17.00 ± 0%    16.00 ± 0%  -5.88% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_100-8          107.0 ± 0%    106.0 ± 0%  -0.93% (p=0.000 n=10)
SortHashersByWeightValueTyped_fnv_1000-8         107.0 ± 0%    106.0 ± 0%  -0.93% (p=0.000 n=10)
geomean                                          115.3         113.0       -1.94%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-01 21:40:46 +03:00
Evgenii Stratonikov
2c085708de [#6] .github: Remove CODEOWNERS 
Not supported by Gitea/Forgejo.
See https://github.com/go-gitea/gitea/issues/10161 .

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-09 22:50:15 +03:00
Evgenii Stratonikov
15b3800347 [#6] Remove .travis.yml 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-09 22:50:15 +03:00
Evgenii Stratonikov
2e205cf1ca [#6] pre-commit: Add gitlint hook 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-09 22:50:15 +03:00
Evgenii Stratonikov
ebca2848ad [#6] pre-commit: Add golangci-lint hook 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-09 22:50:15 +03:00
Evgenii Stratonikov
0ad932400c [#6] pre-commit: Add initial configuration 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-09 22:50:14 +03:00
Alex Vanin
08e14caaf3 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 14:00:00 +03:00
Evgenii Stratonikov
7e33833933 [#4] .github: Add CODEOWNERS 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-27 17:25:20 +03:00
Dmitrii Stepanov
79b208bebf [#3] hrw: Update README 
Update benchmark results

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-02-27 14:18:58 +03:00
Dmitrii Stepanov
5671632658 [#2] hrw: Add typed methods for hashers 
Add generic methods for sort-methods used in node

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-02-27 14:18:58 +03:00
Dmitrii Stepanov
9e9fc653e5 [#2] hrw: Go version update 
Go version updated 1.14 -> 1.18

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-02-27 14:18:58 +03:00
Evgenii Stratonikov
997b540432 Move from nspcc-dev to TrueCloudLab 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-12 18:02:07 +03:00
Evgenii Stratonikov
22b833d972 Panic if provided value is not a slice 
Signed-off-by: Evgenii Stratonikov <stratonikov@runbox.com>
 2021-12-28 14:37:17 +03:00
Evgenii Stratonikov
fad35bbd3b Specify go.mod version 2020-09-11 18:30:18 +03:00
Evgenii Stratonikov
dddcfc8fc5 Simplify SortByValue/Weight a bit 
Get rid of unneeded types.
 2020-09-11 18:30:18 +03:00
Alex Vanin
f52ea8fb21
Handle NaN values in weight validation function (#7) 
NaN values can pass the validation since NaN values don't work
with float arithmetic and comparisons are always true. Now validation
function has check for NaN values.
 2019-08-01 12:16:03 +03:00
Alex Vanin
aa230933d1
Move normalization routine out of hrw library (#6) 
HRW library supports weighted sorting. Weights must be normalized
before applying. Since there could be different types of normalization
for multiple criteria, there is no point to perform simple
normalization in this library. Pass a slice of normalized weights
to the `SortByWeight` functions.

This commit proposes to:
- remove normalization routine from `SortByWeight` function;
- add `ValidateWeights` function to check if weights are normalized;
- rename `weight` -> `distance` to avoid naming confusion between
  hash distance and actual weights;
- use testify lib in the tests;
 2019-07-05 09:49:24 +03:00
Alex Vanin
58a8ce4e47
Added weighted HRW sorting (#5) 
* Added weighted HRW sorting

This commit proposes renaming of old `SortByWeight` functions to `Sort`
and implementation of `SortByWeight` function with explicit weights in
arguments. `SortByWeight` function calculates normalized hashes of
nodes and normalized input weights. Then multiplies these values to
obtain node's actual weight for later sorting.

- renamed `SortByWeight` function to `Sort`
- added `SortByWeight`, `SortSliceByWeightValue` and
  `SortSliceBeWeightIndex` functions
- moved code with reflection processing into `prepareRule` function
- added tests and benchmarks for new weighted functions
- added benchmark results into README

* Fixed comments
 2019-05-27 10:45:29 +03:00
Evgeniy Kulikov
d32f396204
✌️ Fixed in SortSliceByValue issue when weight calculated twice (#4) 
- fixes for SortSliceByValue
- fixes for tests
 2019-05-24 12:51:39 +03:00
Evgeniy Kulikov
9a6fcdb1f8
🐌 hot fix types 2019-04-12 14:19:18 +03:00
Evgeniy Kulikov
e06640668d
🐿 Extend SortSliceByValue with more types 2019-04-12 14:09:49 +03:00
Evgeniy Kulikov
7462e26df6
Update to release version after [this issue](https://github.com/spaolacci/murmur3/issues/24) (#3) 2019-03-18 14:49:34 +03:00
Evgeniy Kulikov
e8ba64d915
Small refactoring (#2) 
- Use val.Index(0).Interface().(type) in switch-case of SortSliceByValue
- add go1.12 to TravisCI
 2019-03-14 15:48:58 +03:00
Evgeniy Kulikov
29dac15c96
Move repo to NSPCC (#1) 2019-02-01 14:30:34 +03:00
Alex Vanin
098fd24704 Support for int32 slices (#4) 2019-02-01 12:57:05 +03:00
Evgeniy Kulikov
e5d4b47881
Work around Hashing and tests (#2) 
* Work around Hashing and tests

* fixes

* add TravisCI

* fix lint

* so slow

* add badges

* work on README

* Improved tests and get rid of fnv (#3)
 2019-01-31 12:54:02 +03:00
Alex Vanin
cf9a7e5b89 Removed redundant m64 variable and tweaked uniform distribution test. (#1) 2019-01-30 12:32:34 +03:00
Evgeniy Kulikov
9b6321f99c
remove binaries from repo 2019-01-30 02:01:54 +03:00
Evgeniy Kulikov
01e85efd7e
Base HRW implementation in golang 2019-01-30 01:58:30 +03:00
Evgeniy Kulikov
ca071432c4
Initial commit 2019-01-29 22:43:14 +03:00