Commit graph

38 commits

Author SHA1 Message Date
Anna Shaleva
c95f2079d5 core: adjust comments on behaviour defferences for MPT TestCompatibility
C# node does not return empty proof enymore in case if path is bad. C#
node also throws an exception on bad Put.

Our node does not return an error on delete if the key is empty.
2021-09-03 13:46:52 +03:00
Anna Shaleva
f721384ead core: allow empty MPT Leaf values
Allow it for (*Trie).Put. And distinguish empty value and nil value for
(*Trie).PutBatch, because batch is already capable of handling both nil
and empty value. For (*Trie).PutBatch putting nil value means deletion,
while putting empty value means just putting LeafNode with an empty
value.
2021-09-03 13:46:48 +03:00
Evgeniy Stratonikov
73e4040628 mpt: use BinWriter.Grow() instead of custom buffer
Also add benchmarks.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-10 09:34:05 +03:00
Evgeniy Stratonikov
bd2b1a0521 mpt: add Size method to trie nodes
Knowing serialized size of the node is useful for
preallocating byte-slice in advance.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 12:01:16 +03:00
Evgeniy Stratonikov
db80ef28df mpt: move empty hash node in a separate type
We use them quite frequently (consider children for a new branch
node) and it is better to get rid of unneeded allocations.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 12:01:16 +03:00
Evgeniy Stratonikov
43ee671f36 mpt: do not allocate NodeObject for serialization
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 10:28:19 +03:00
Roman Khimov
19717dd9a8 slice: introduce common Copy helper
It's a bit more convenient to use.
2021-07-19 22:57:55 +03:00
Evgeniy Stratonikov
a33e422b79 mpt: do not create an Extension for the last child of a Branch
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-07 15:49:53 +03:00
Evgeniy Stratonikov
25428d53f1 mpt: strip branch node in a simple implementation
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-07 15:47:07 +03:00
Evgeniy Stratonikov
9691eee10c mpt: strip branch if 1 child is left
If the child left is a hash node, we should retrieve it from store.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-05 11:04:20 +03:00
Evgeniy Stratonikov
f3d02b9fef mpt/tests: add compatibility tests 2021-06-04 11:09:17 +03:00
Evgeniy Stratonikov
e833d333fe mpt: disallow empty keys
This is not a problem in practice, as all keys are prefixed
by a contract ID. However in theory it can lead to a different
state root after new portion of changes thus this fix.
2021-06-04 11:09:17 +03:00
Evgeniy Stratonikov
5b1f6207de mpt: reduce max key length
The constant used is actually a nibbled-path restriction.
Clarify this and make them private.
2021-06-04 11:09:17 +03:00
Roman Khimov
9d2712573f *: enable godot linter and fix all its warnings
It's important for NeoGo to have clean documentation. No functional changes.
2021-05-12 23:17:03 +03:00
Roman Khimov
c4e084b0d8 *: fix whitespace errors
leading/trailing newlines
2021-05-12 22:51:41 +03:00
Roman Khimov
92dbb3c4b9 *: fix all unused warnings
From golangci-lint.
2021-05-12 18:53:12 +03:00
Anna Shaleva
8c358aa556 mpt: change MaxKeyLength for LeafNode 2021-04-01 10:20:16 +03:00
Anna Shaleva
7f038bd465 mpt: split HashNode in two types
First type is non-empty HashNode, and the second one is an Empty node.
2021-04-01 10:19:57 +03:00
Anna Shaleva
b9927c39ee mpt: refactor nodes serialisation
It should be serialised with type in case if it's a children node.
The type can be either HashT or EmptyT.
2021-04-01 10:19:57 +03:00
Anna Shaleva
6e836d325e mpt: increase max leaf value length 2021-04-01 10:17:08 +03:00
Anna Shaleva
7dc9f0bde2 mpt: swap Leaf and Node types 2021-04-01 10:17:08 +03:00
Evgeniy Stratonikov
0cb6ec7345 mpt: allow to remove non-existent keys in batch
This bug was here before batch were intoduced.
`Delete` is allowed to be called on missing keys with
HALT result, MPT needs to take this into account.
2021-02-17 12:37:44 +03:00
Evgenii Stratonikov
fb88d4f3a0 mpt: support put in batches 2021-01-13 12:25:27 +03:00
Evgenii Stratonikov
c2f70a179b Revert "mpt: do not allocate new buffer when updating dirty node"
This reverts commit 168ba7960c.

It seems, there are some problems with it:
`2020-12-25T18:13:07.476+0300    WARN    blockQueue: failed adding block into the blockchain     {"error": "error while trying to apply MPT changes: unexpected EOF", "blockHeight": 9729, "nextIndex": 9730}`
2020-12-25 18:42:47 +03:00
Evgenii Stratonikov
168ba7960c mpt: do not allocate new buffer when updating dirty node
Running time becomes faster under high load while staying the same in
the average case.
Memory allocation done in `Trie` goes down by about ~10% (even more,
actually).
2020-12-25 11:33:49 +03:00
Evgenii Stratonikov
7d91a3a89e pkg: move internal/ package to the root directory
This way we can use it in scripts and cli.
2020-11-24 16:39:56 +03:00
Evgenii Stratonikov
85f927d892 mpt: implement reference counting
Also postpone MPT initialization until `storeBlock`
because we need to read-and-check or save info about refcounting
depending on starting height.
2020-11-20 16:50:30 +03:00
Evgenii Stratonikov
1c559634aa mpt: fill cached fields when getting node from store
Node which has been got from store shouldn't be flushed again.
2020-11-19 12:25:43 +03:00
Evgenii Stratonikov
fd9ff4102a mpt: export func for decoding node with type
`NodeObject` can contain auxilliary fields and shouldn't be used from outside.
2020-11-19 12:25:43 +03:00
Roman Khimov
0c264b1486 mpt: fix comment typo 2020-09-21 17:54:11 +03:00
Roman Khimov
e7d13e6db2 *: fix misspellings found in Go Report Card 2020-08-14 12:16:24 +03:00
Roman Khimov
caea6d6ca8 mpt: fix extension node cache invalidation
It should always be invalidated if something changes in the `next` (below the
extension node).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
2b53877dff mpt: don't flush nodes already present in the DB
It's just a waste of time.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
475bf2445a mpt: restructure nodes a bit, implement serialization and hash cache
It drastically reduces the number of allocations and hash calculations.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
6ca22027d5 mpt: implement (*Trie).Collapse()
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
f0b85f8af7 mpt: implement JSON marshaling/unmarshaling
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
9b328240dd mpt: implement MPT proof Get and Verify
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
dc6741bce7 mpt: implement MPT trie
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00