Commit graph

31 commits

Author SHA1 Message Date
Roman Khimov
357bc76882 *: use slices.Concat where appropriate
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-08-30 17:00:11 +03:00
Roman Khimov
d9ee31fb52 *: use clear() to clear maps
Supposedly more efficient since we can avoid some memory management dances.

Memory pool agrees:

goos: linux
goarch: amd64
pkg: github.com/nspcc-dev/neo-go/pkg/core/mempool
cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics
                       │  pool.old   │              pool.new               │
                       │   sec/op    │   sec/op     vs base                │
Pool/one,_incr_fee-16    12.44m ± 1%   12.51m ± 1%   +0.55% (p=0.029 n=10)
Pool/many,_same_fee-16   4.960m ± 2%   3.100m ± 1%  -37.50% (p=0.000 n=10)
Pool/many,_incr_fee-16   16.03m ± 2%   14.11m ± 1%  -12.00% (p=0.000 n=10)
Pool/one,_same_fee-16                  1.742m ± 1%
geomean                  9.964m        5.556m       -17.92%

                       │    pool.old     │                pool.new                │
                       │      B/op       │      B/op       vs base                │
Pool/one,_incr_fee-16     8.117Ki ± 120%   7.101Ki ± 128%  -12.52% (p=0.022 n=10)
Pool/many,_same_fee-16   3941.2Ki ±   0%   805.4Ki ±   0%  -79.56% (p=0.000 n=10)
Pool/many,_incr_fee-16   3936.2Ki ±   0%   829.8Ki ±   0%  -78.92% (p=0.000 n=10)
Pool/one,_same_fee-16                      12.98Ki ±  10%
geomean                   501.2Ki          88.59Ki         -66.47%

                       │   pool.old   │               pool.new               │
                       │  allocs/op   │  allocs/op    vs base                │
Pool/one,_incr_fee-16     28.00 ± 21%    24.00 ± 21%  -14.29% (p=0.002 n=10)
Pool/many,_same_fee-16   40.38k ±  0%   40.03k ±  0%   -0.86% (p=0.000 n=10)
Pool/many,_incr_fee-16   40.38k ±  0%   40.04k ±  0%   -0.85% (p=0.000 n=10)
Pool/one,_same_fee-16                    23.00 ±  4%
geomean                  3.574k          969.8         -5.55%

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-08-24 22:41:48 +03:00
Anna Shaleva
c06543cf50 *: fix linter issues
Linter is updated up to v1.60.1, the following issue is fixed:
```
predeclared  variable max has same name as predeclared identifier
```

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-08-14 12:36:15 +03:00
Anna Shaleva
8162e9033d *: replace slice.Copy with bytes.Clone
And refactor some code a bit, don't use bytes.Clone where type-specific
helpers may be used instead.

Close #2907.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-03-05 13:54:10 +03:00
Elizaveta Chichindaeva
28908aa3cf [#2442] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-04 19:48:27 +03:00
Anna Shaleva
0cf525d62e core: add ability to traverse backwards for Billet 2022-04-29 16:10:04 +03:00
Roman Khimov
9d2ef775cf storage: simplify (*MemCachedStore).Put/Delete interface
They never return errors, so their interface should reflect that. This allows
to remove quite a lot of useless and never tested code.

Notice that Get still does return an error. It can be made not to do that, but
usually we need to differentiate between successful/unsuccessful accesses
anyway, so this doesn't help much.
2022-02-16 18:24:20 +03:00
Roman Khimov
c5f1e2fbcd mpt: don't allocate for every key
Microoptimization.
2022-02-11 14:19:54 +03:00
Roman Khimov
c4ee310e85 mpt: modify refcounted storage scheme to make GC possible
Add "active" flag into the node data and make the remainder modal, for active
nodes it's a reference counter, for inactive ones the deactivation height is
stored.

Technically, refcounted chains storing just one trie don't need a flag, but
it's a bit simpler this way.
2022-02-10 21:56:20 +03:00
Roman Khimov
86cb4ed80f mpt: add the notion of MPT mode
It directly affects the storage format, so it's important. ModeGC is not used
at the moment, but defined for future extensibility.
2022-02-10 19:15:18 +03:00
Roman Khimov
2bc493a839 mpt: simplify makeStorageKey() interface 2022-02-08 12:38:03 +03:00
Evgeniy Stratonikov
856385b106 dao: use custom storage prefix
We use 2 prefixes for storing items because of state synchronization.
This commit allows to parametrize dao with the default prefix.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:07 +03:00
Anna Shaleva
892eadf86d core: mandate passing from as a subprefix for (*Trie).Find
However, we need to distinguish empty subprefix and nil subprefix (no
start specified) to match the C# behaviour.
2021-10-13 14:47:41 +03:00
Anna Shaleva
8e7c76827b core: optimise (*Billet).Traverse and (*Trie).Find
(*Billet).Traverse changes:
1. Get rid of the `offset` argument. We can cut `from` and pass just the
part that remains. This implies that node with path matching `from` will
also be included in the result, so additional check needs to be added to
the callback function.
2. Pass `path` and `from` without search prefix. Append prefix to the
result inside the callback.
3. Remove duplicating code.

(*Trie).Find changes:
1. Properly prepare `from` argument for traversing function. It closly
depends on the `path` argument.
2021-10-13 13:52:33 +03:00
Anna Shaleva
43ac4e1517 rpc: implement findstates RPC handler 2021-10-13 11:41:05 +03:00
Anna Shaleva
c8120a139d core: add MaxKeyLength restrictions to MPT's operations 2021-10-06 16:37:23 +03:00
Anna Shaleva
f721384ead core: allow empty MPT Leaf values
Allow it for (*Trie).Put. And distinguish empty value and nil value for
(*Trie).PutBatch, because batch is already capable of handling both nil
and empty value. For (*Trie).PutBatch putting nil value means deletion,
while putting empty value means just putting LeafNode with an empty
value.
2021-09-03 13:46:48 +03:00
Evgeniy Stratonikov
db80ef28df mpt: move empty hash node in a separate type
We use them quite frequently (consider children for a new branch
node) and it is better to get rid of unneeded allocations.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 12:01:16 +03:00
Roman Khimov
19717dd9a8 slice: introduce common Copy helper
It's a bit more convenient to use.
2021-07-19 22:57:55 +03:00
Evgeniy Stratonikov
25428d53f1 mpt: strip branch node in a simple implementation
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-07 15:47:07 +03:00
Evgeniy Stratonikov
e833d333fe mpt: disallow empty keys
This is not a problem in practice, as all keys are prefixed
by a contract ID. However in theory it can lead to a different
state root after new portion of changes thus this fix.
2021-06-04 11:09:17 +03:00
Evgeniy Stratonikov
0cb6ec7345 mpt: allow to remove non-existent keys in batch
This bug was here before batch were intoduced.
`Delete` is allowed to be called on missing keys with
HALT result, MPT needs to take this into account.
2021-02-17 12:37:44 +03:00
Evgenii Stratonikov
c2f70a179b Revert "mpt: do not allocate new buffer when updating dirty node"
This reverts commit 168ba7960c.

It seems, there are some problems with it:
`2020-12-25T18:13:07.476+0300    WARN    blockQueue: failed adding block into the blockchain     {"error": "error while trying to apply MPT changes: unexpected EOF", "blockHeight": 9729, "nextIndex": 9730}`
2020-12-25 18:42:47 +03:00
Evgenii Stratonikov
168ba7960c mpt: do not allocate new buffer when updating dirty node
Running time becomes faster under high load while staying the same in
the average case.
Memory allocation done in `Trie` goes down by about ~10% (even more,
actually).
2020-12-25 11:33:49 +03:00
Evgenii Stratonikov
85f927d892 mpt: implement reference counting
Also postpone MPT initialization until `storeBlock`
because we need to read-and-check or save info about refcounting
depending on starting height.
2020-11-20 16:50:30 +03:00
Evgenii Stratonikov
1c559634aa mpt: fill cached fields when getting node from store
Node which has been got from store shouldn't be flushed again.
2020-11-19 12:25:43 +03:00
Roman Khimov
caea6d6ca8 mpt: fix extension node cache invalidation
It should always be invalidated if something changes in the `next` (below the
extension node).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
2b53877dff mpt: don't flush nodes already present in the DB
It's just a waste of time.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
475bf2445a mpt: restructure nodes a bit, implement serialization and hash cache
It drastically reduces the number of allocations and hash calculations.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
6ca22027d5 mpt: implement (*Trie).Collapse()
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
dc6741bce7 mpt: implement MPT trie
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00