Commit graph

497 commits

Author SHA1 Message Date
Roman Khimov
685d3eb870 dao: prevent double serialization of StorageItems
Converting to MPT value serializes the StorageItem, so it makes no sense doing
it again.
2020-06-04 17:18:15 +03:00
Roman Khimov
f77c239296 mpt: fix extension node cache invalidation
It should always be invalidated if something changes in the `next` (below the
extension node).
2020-06-04 17:16:32 +03:00
Roman Khimov
24785f1f50
Merge pull request #1008 from nspcc-dev/feature/mpt
core: update MPT during block processing
2020-06-03 16:07:04 +03:00
Roman Khimov
baafa30266
Merge pull request #1006 from nspcc-dev/interop/crypto
core: implement key recover interops
2020-06-03 14:39:44 +03:00
Anna Shaleva
6c06bc57cc core: implement key recover interops
Implement secp256k1 and secp256r1 recover interops, closes #1003.

Note:

We have to implement Koblitz-related math to recover keys properly
with Neo.Cryptography.Secp256k1Recover interop as far as standard
go elliptic package supports short-form Weierstrass curve with a=-3
only (see https://github.com/golang/go/issues/26776 for details).
However, it's not the best choise to have a lot of such math in our
project, so it would be better to use ready-made solution for
Koblitz-related cryptography.
2020-06-03 14:36:04 +03:00
Evgenii Stratonikov
20f190ef69 core: update MPT during block processing 2020-06-03 13:33:44 +03:00
Roman Khimov
029fecbb71 mpt: don't flush nodes already present in the DB
It's just a waste of time.
2020-06-03 00:37:30 +03:00
Roman Khimov
1b6aee42d5 mpt: restructure nodes a bit, implement serialization and hash cache
It drastically reduces the number of allocations and hash calculations.
2020-06-03 00:37:21 +03:00
Roman Khimov
2f90a06db3
Merge pull request #990 from nspcc-dev/feature/mpt
Initial MPT implementation (2.x)
2020-06-01 18:58:35 +03:00
Evgenii Stratonikov
314430be1d mpt: implement NEO storage key conversion 2020-06-01 18:15:13 +03:00
Evgenii Stratonikov
103c45850a mpt: implement (*Trie).Collapse()
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.
2020-06-01 18:15:13 +03:00
Evgenii Stratonikov
9c478378e1 mpt: implement JSON marshaling/unmarshaling
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.
2020-06-01 18:15:13 +03:00
Evgenii Stratonikov
31d9aeddd2 mpt: implement MPT proof Get and Verify 2020-06-01 18:14:19 +03:00
Evgenii Stratonikov
861a1638e8 mpt: implement MPT trie
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.
2020-06-01 18:14:19 +03:00
Roman Khimov
9a0d7d3254 transaction: gofmt -s 2020-05-30 15:48:57 +03:00
Roman Khimov
06bed2b4bf dao: split GetStorageItem for Cached into external and internal versions
Lower Cached state shouldn't be changed until Persist. Fixes #994.
2020-05-27 18:34:41 +03:00
Evgenii Stratonikov
b0d07c3031 vm: make Iterator interface public
There is nothing wrong with iterators being implemented in other parts
of code (e.g. Storage.Find). In this case type assertions can
prevent bugs at compile-time.
2020-05-27 11:40:46 +03:00
Evgenii Stratonikov
503442a60d dao: restrict GetStorageItems by prefix
All storage items can still be retrived via zero-length prefix.
2020-05-27 11:40:46 +03:00
Evgenii Stratonikov
776bd85ded vm,dao: return storage iterator from DAO in Storage.Find interop
Reproduce behavior of the reference realization:
- if item was Put in cache after it was encountered during
  Storage.Find, it must appear twice
- checking if item is in cache must be performed in real-time
  during `Iterator.Next()`
2020-05-27 11:40:46 +03:00
Evgenii Stratonikov
86ce234d5f core: fix a typo in GetUnspentCoins interop
Fix a bug introduced in 8ed77f0d.
2020-05-25 11:14:44 +03:00
Evgenii Stratonikov
8ed77f0d25 core: make GetUnspentCoins interop return array, fix #978. 2020-05-22 13:24:43 +03:00
Roman Khimov
0ce3a12e87 core: check for key length in CheckWitness, fix #968
Only one type of keys is allowed here.
2020-05-21 14:27:41 +03:00
Roman Khimov
5ec70b9fc2
Merge pull request #930 from nspcc-dev/fix/keys
Cache storage operations across same block tx executions
2020-05-21 11:08:42 +03:00
Evgenii Stratonikov
b96fe8173c core,dao: implement Block-level storage caching
The order in which storage.Find items are returns depends on what items
were processed in previous transactions of the same block.
The easiest way to implement this sort of caching is to cache operations
with storage, flushing the only in `Persist()`.
2020-05-19 17:19:51 +03:00
Roman Khimov
a43f2234dd core: fix Neo.Contract.GetStorageContext security check
This syscall should only work for contracts created by current transaction and
that is what is supposed to be checked here. Do so by looking at the
differences between ic.dao and original lower DAO.
2020-05-19 13:13:15 +03:00
Roman Khimov
80b8b50f02 core: fix Neo.Blockchain.GetValidators implementation
It should return keys, attempting to push []*state.Validator to the stack
would probably lead to failure.
2020-05-19 13:13:15 +03:00
Roman Khimov
9546e021a9 rpc/block: rework the way Block is JSONized
Our block.Block was JSONized in a bit different fashion than result.Block in
its Nonce and NextConsensus fields. It's not good for notifications because
third-party clients would probably expect to see the same format. Also, using
completely different Block representation in result is probably making our
client a bit weaker as this representation is harder to use with other neo-go
components.

So use the same approach we took for Transactions and wrap block.Base which is
to be serialized in proper way.
2020-05-14 17:28:14 +03:00
Roman Khimov
83febead59 transaction: add json.Unmarshaler to Attribute
It actually was missing and it might affect Transaction conversion to/from
JSON.
2020-05-14 17:28:14 +03:00
Roman Khimov
9454ef5c28 core: improve locking in storeBlock
Getting batch, updating Prometheus metrics and pushing events doesn't require
any locking: batch is a local cache batch that no one outside cares about,
Prometheus metrics are not critical to be in perfect sync and events are
asynchronous anyway.
2020-05-13 17:17:41 +03:00
Roman Khimov
1ac4f8528d core: add Blockchain event subscription mechanism
A deep internal part of #895. Blockchainer interface is also extended for
various uses of these methods.
2020-05-12 17:41:15 +03:00
Roman Khimov
2e58a14978 core: improve documentation a little 2020-05-12 17:20:31 +03:00
Roman Khimov
943d435cd2 core: ensure we produce correct blocks for tests
Blocks must have at least one transaction and we should check for correct
merkle root generation.
2020-05-07 21:46:28 +03:00
Anna Shaleva
54cdfe4a23 storage: add support of BadgerDB
closes #820
2020-04-09 13:55:59 +03:00
Evgenii Stratonikov
ee0ba9b1b4 core: make SpawnVM a method of context
spawnVMWithInterops is rather long too type and
it doesn't use Blockchain in any way.
2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
a71cd0961e core/dao: remove unnecessary slice type
It is used only once, so a simple `sort.Slice`
invocation will suffice.
2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
030b7754ad core: move DAO to a separate package 2020-04-08 08:38:44 +03:00
Roman Khimov
9997661998
Merge pull request #821 from nspcc-dev/fix-cache-propagation-to-invocations
core: wrap cached dao properly, don't miss cached data
2020-04-06 11:52:41 +03:00
Evgenii Stratonikov
1fcc019bf3 rpc: update test chain
Also provide info for getblockheader RPC while
generating test chain.
2020-04-06 09:31:09 +03:00
Roman Khimov
5f09381cf4 core: wrap cached dao properly, don't miss cached data
Fixes #817 where invoked contract missed updated account information because
it got it one layer below cachedDao used to process the block.
2020-04-03 10:15:11 +03:00
Roman Khimov
2d0ad30fcf vm: rework Map with internal slice representation
Which makes iterating over map stable which is important for serialization and
and even fixes occasional test failures. We use the same ordering here as
NEO 3.0 uses, but it should also be fine for NEO 2.0 because it has no
defined order.
2020-04-01 19:33:53 +03:00
Roman Khimov
9b5dab57e8
Merge pull request #810 from nspcc-dev/refactor/config
config: move config.go out of config
2020-04-01 10:29:57 +03:00
Anna Shaleva
5a984fdf88 config: move config.go out of config/
closes #423
2020-03-31 17:55:59 +03:00
Roman Khimov
e41853d0a4
Merge pull request #789 from nspcc-dev/fix/tx_specific_data_marshalling
rpc: fix marshalling of type-specific tx data
2020-03-30 16:10:25 +03:00
Anna Shaleva
9c09ad9c89 rpc: fix marshalling of type-specific tx data
closes #585
2020-03-30 15:48:50 +03:00
Roman Khimov
237de73dec
Merge pull request #807 from nspcc-dev/optimize-memcached-persist
storage: optimize (*MemCachedStore).Persist for memory-backed ps
2020-03-29 13:41:21 +03:00
Roman Khimov
4758de71ec storage: optimize (*MemCachedStore).Persist for memory-backed ps
Most of the time it's persisted into the MemoryStore or MemCachedStore, when
that's the case there is no real need to go through the Batch mechanism as it
incurs multiple copies of the data.

Importing 1.5M mainnet blocks with verification turned off, before:
real    12m39,484s
user    20m48,300s
sys     2m25,022s

After:
real    11m15,053s
user    18m2,755s
sys     2m4,162s

So it's around 10% improvement which looks good enough.
2020-03-28 17:21:50 +03:00
Roman Khimov
c738975b7b
Merge pull request #803 from nspcc-dev/feature/appcall
emit: implement AppCallWithOperationAndArgs
2020-03-27 13:30:48 +03:00
Evgenii Stratonikov
db2dccf7cb emit: implement AppCallWithOperationAndArgs
It is nice to have a typical task of calling contract method
with specific arguments incapsulated inside some function.
2020-03-27 11:05:36 +03:00
Evgenii Stratonikov
0036b3e52b random: make use or random package in tests
Also implement Bytes/Fill routines for generating byte slices.
2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
9abda40171 testserdes: implement helpers for encode/decode routines
Frequently one needs to check if struct serializes/deserializes
properly. This commit implements helpers for such cases including:
1. JSON
2. io.Serializable interface
2020-03-27 10:27:46 +03:00