Commit graph

2131 commits

Author SHA1 Message Date
Roman Khimov
e3af560d11 dao: optimize storage cache flushing
Items were serialized several times if there were several successful
transactions in a block, prevent that by using State field as a bitfield (as
it almost was intended to) and adding one more bit. It also eliminates useless
duplicate MPT traversions.

Confirmed to not break storage changes up to 3.3M on testnet.
2020-06-04 17:21:58 +03:00
Roman Khimov
69ccca675d core: fix PrevHash calculation for MPTRoot
This was differing from C# notion of PrevHash. It's not a previous root, but
rather a hash of the previous serialized MPTRoot structure (that is to be
signed by CNs).
2020-06-04 17:19:30 +03:00
Roman Khimov
685d3eb870 dao: prevent double serialization of StorageItems
Converting to MPT value serializes the StorageItem, so it makes no sense doing
it again.
2020-06-04 17:18:15 +03:00
Roman Khimov
f77c239296 mpt: fix extension node cache invalidation
It should always be invalidated if something changes in the `next` (below the
extension node).
2020-06-04 17:16:32 +03:00
Evgenii Stratonikov
5794bdb169 state: implement JSON marshaling for MPT* items 2020-06-03 18:09:28 +03:00
Roman Khimov
a6ffa90ceb
Merge pull request #1009 from nspcc-dev/update-dbft-2.x
consensus: update dbft to include CountFailed and RecoveryMessage changes
2020-06-03 16:12:05 +03:00
Roman Khimov
24785f1f50
Merge pull request #1008 from nspcc-dev/feature/mpt
core: update MPT during block processing
2020-06-03 16:07:04 +03:00
Roman Khimov
baafa30266
Merge pull request #1006 from nspcc-dev/interop/crypto
core: implement key recover interops
2020-06-03 14:39:44 +03:00
Anna Shaleva
6c06bc57cc core: implement key recover interops
Implement secp256k1 and secp256r1 recover interops, closes #1003.

Note:

We have to implement Koblitz-related math to recover keys properly
with Neo.Cryptography.Secp256k1Recover interop as far as standard
go elliptic package supports short-form Weierstrass curve with a=-3
only (see https://github.com/golang/go/issues/26776 for details).
However, it's not the best choise to have a lot of such math in our
project, so it would be better to use ready-made solution for
Koblitz-related cryptography.
2020-06-03 14:36:04 +03:00
Evgenii Stratonikov
10189b6ab3 consensus: extend payloads with StateRoot info
Create and verify witness after block processing.
2020-06-03 13:33:44 +03:00
Evgenii Stratonikov
edcfdb3bde network: add MPT-related P2P payloads 2020-06-03 13:33:44 +03:00
Evgenii Stratonikov
20f190ef69 core: update MPT during block processing 2020-06-03 13:33:44 +03:00
Roman Khimov
7d24fc5500
Merge pull request #1011 from nspcc-dev/optimize-mpt-2.x-neox
Optimize mpt 2.x neox
2020-06-03 11:36:17 +03:00
Roman Khimov
029fecbb71 mpt: don't flush nodes already present in the DB
It's just a waste of time.
2020-06-03 00:37:30 +03:00
Roman Khimov
1b6aee42d5 mpt: restructure nodes a bit, implement serialization and hash cache
It drastically reduces the number of allocations and hash calculations.
2020-06-03 00:37:21 +03:00
Roman Khimov
fd73310a1c
Merge pull request #1007 from nspcc-dev/update-dbft-2.x
consensus: update dbft to include CountFailed and RecoveryMessage changes
2020-06-02 22:44:46 +03:00
Roman Khimov
e4f413aa29 consensus: update dbft to include CountFailed and RecoveryMessage changes
dbft interface changed a little also because of Neo 3 changes, but it still is
compatible with Neo 2.
2020-06-02 11:12:47 +03:00
Roman Khimov
2f90a06db3
Merge pull request #990 from nspcc-dev/feature/mpt
Initial MPT implementation (2.x)
2020-06-01 18:58:35 +03:00
Evgenii Stratonikov
314430be1d mpt: implement NEO storage key conversion 2020-06-01 18:15:13 +03:00
Evgenii Stratonikov
103c45850a mpt: implement (*Trie).Collapse()
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.
2020-06-01 18:15:13 +03:00
Evgenii Stratonikov
9c478378e1 mpt: implement JSON marshaling/unmarshaling
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.
2020-06-01 18:15:13 +03:00
Evgenii Stratonikov
31d9aeddd2 mpt: implement MPT proof Get and Verify 2020-06-01 18:14:19 +03:00
Evgenii Stratonikov
861a1638e8 mpt: implement MPT trie
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.
2020-06-01 18:14:19 +03:00
Roman Khimov
806b28aab7 Makefile: add neox suffix to neox-2.x branch 2020-05-30 16:01:35 +03:00
Roman Khimov
9a0d7d3254 transaction: gofmt -s 2020-05-30 15:48:57 +03:00
Roman Khimov
0f6d01faa1 CHANGELOG: release v0.75.0 2020-05-28 18:54:14 +03:00
Roman Khimov
90536d533d
Merge pull request #996 from nspcc-dev/dont-change-cached-until-persist-2.x
dao: split GetStorageItem for Cached into external and internal versions
2020-05-28 11:50:09 +03:00
Roman Khimov
06bed2b4bf dao: split GetStorageItem for Cached into external and internal versions
Lower Cached state shouldn't be changed until Persist. Fixes #994.
2020-05-27 18:34:41 +03:00
Roman Khimov
0da1935ebb Merge pull request #992 from nspcc-dev/fix/state
vm,dao: return storage iterator from DAO in Storage.Find interop
2020-05-27 11:53:54 +03:00
Evgenii Stratonikov
b0d07c3031 vm: make Iterator interface public
There is nothing wrong with iterators being implemented in other parts
of code (e.g. Storage.Find). In this case type assertions can
prevent bugs at compile-time.
2020-05-27 11:40:46 +03:00
Evgenii Stratonikov
503442a60d dao: restrict GetStorageItems by prefix
All storage items can still be retrived via zero-length prefix.
2020-05-27 11:40:46 +03:00
Evgenii Stratonikov
776bd85ded vm,dao: return storage iterator from DAO in Storage.Find interop
Reproduce behavior of the reference realization:
- if item was Put in cache after it was encountered during
  Storage.Find, it must appear twice
- checking if item is in cache must be performed in real-time
  during `Iterator.Next()`
2020-05-27 11:40:46 +03:00
Roman Khimov
79c87ca8a5
Merge pull request #991 from nspcc-dev/fix/restore
cli: support DB restore from diff
2020-05-27 11:36:44 +03:00
Evgenii Stratonikov
78bd01db57 cli: support DB restore from diff
Diff dumps provided by NGD contain index of the first block.
Also do proper error handling.
2020-05-27 10:46:12 +03:00
Roman Khimov
7a783b64d7
Merge pull request #984 from nspcc-dev/fix/state
core: fix a typo in GetUnspentCoins interop
2020-05-25 12:28:39 +03:00
Evgenii Stratonikov
86ce234d5f core: fix a typo in GetUnspentCoins interop
Fix a bug introduced in 8ed77f0d.
2020-05-25 11:14:44 +03:00
Roman Khimov
ff0241ed07
Merge pull request #981 from aprasolova/feature/dockerfile
Clean build in Dockerfile
2020-05-22 17:35:36 +03:00
anastasia prasolova
7431263903 added make version 2020-05-22 14:59:51 +03:00
anastasia prasolova
fe2e501488 clean docker build 2020-05-22 13:56:26 +03:00
Roman Khimov
3675502927
Merge pull request #979 from nspcc-dev/fix/state
core: make GetUnspentCoins interop return array, fix #978.
2020-05-22 13:33:32 +03:00
Evgenii Stratonikov
eeeb05fc2c compiler: restore support for GetUnspentCoins
Revert a587274.
2020-05-22 13:24:44 +03:00
Evgenii Stratonikov
8ed77f0d25 core: make GetUnspentCoins interop return array, fix #978. 2020-05-22 13:24:43 +03:00
Roman Khimov
fb9b2ae982
Merge pull request #975 from nspcc-dev/clone-struct-in-setitem-2.x
vm: clone struct item on SETITEM, close #972
2020-05-21 18:02:06 +03:00
Roman Khimov
4fda492873
Merge pull request #974 from nspcc-dev/fix/state
vm: allow to convert Map item to bool

Fixes
2020-05-21T17:45:51.808+0300 WARN contract invocation failed {"tx": "71c43cc32ae5336622d7105d1c96387c8cca6c948beac29edfef46712ba5ffeb", "block": 3960417, "error": "error encountered at instruction 14802 (JMPIFNOT): can't convert to bool: Map"}
2020-05-21 17:59:19 +03:00
Roman Khimov
f8b41e486c vm: clone struct item on SETITEM, close #972
It's the same as with APPEND and VALUES.
2020-05-21 17:56:26 +03:00
Evgenii Stratonikov
a8fa68914a vm: allow to convert Map item to bool 2020-05-21 17:53:19 +03:00
Roman Khimov
e07fd4f9f0
Merge pull request #973 from nspcc-dev/fix/refcount
vm: update stack size in SETITEM properly
2020-05-21 16:47:27 +03:00
Roman Khimov
61c595909a
Merge pull request #971 from nspcc-dev/add-key-size-checks-2.x
Add key size checks 2.x
2020-05-21 15:38:40 +03:00
Evgenii Stratonikov
cd8445aa59 vm: update stack size in SETITEM properly 2020-05-21 15:20:37 +03:00
Roman Khimov
60bca03577 crypto: add input data length check in (*PublicKey).DecodeBytes
DecodeBinary works with streams, so it can't do that, but DecodeBytes can and
should. Also fix unmarshalled binary buffer that this check exposed.
2020-05-21 14:28:16 +03:00