We need to compact our in-memory MPT from time to time, otherwise it quickly
fills up all available memory. This raises two obvious quesions --- when to do
that and to what level do that.
As for 'when', I think it's quite easy to use our regular persistence interval
as an anchor (and it also frees up some memory), but we can't do that in the
persistence routine itself because of synchronization issues (adding some
synchronization primitives would add some cost that I'd also like to avoid),
so do it indirectly by comparing persisted and current height in `storeBlock`.
Choosing proper level is another problem, but if we're to roughly estimate one
full branch node to use 1K of memory (usually it's way less than that) then we
can easily store 1K of these nodes and that gives us a depth of 10 for our
trie.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
This was differing from C# notion of PrevHash. It's not a previous root, but
rather a hash of the previous serialized MPTRoot structure (that is to be
signed by CNs).
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Get 2 contracts in pair which is useful everytime we need to test
syscall with one contract calling the other.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Invoke `_initialize` method on every call if present.
In NEO3 there is no entrypoint and methods are invoked by offset,
thus `Main` function is no longer required.
We still have special `Main` method in tests to simplify them.
Allow to invoke methods by offset:
1. Every invoked contract must have manifest.
2. Check arguments count on invocation.
3. Change AppCall to a regular syscall.
4. Add test suite for `System.Contract.Call`.
Disallow costly verification methods. We put this limit in policy
contract as it may be a subject to change in future.
In fact this value also overrides gas limit for header verification.
Close#1202.
When providing public key as a subslice, it still can be
decoded as a valid key, thus interop will not return an error
but rather push `false` on stack. This test is about providing
invalid key, so ensure this via setting invalid prefix.
Part of #1055.
It should have `AllowStates` flag.
Also removed unreachable code: we can't have such situation when
script container is not a transaction in the scope of `CheckWitness`
method because:
1. Blocks have their own implementation of CheckWitness for
internal usage (it's (bc *Blockchain) verifyHeaderWitnesses method).
2. For the outside calls of System.Runtime.CheckWitness interop (e.g.
calls from smart-contract) script container is always a transaction.
Part of #1055.
Split methods, as they have a lot of common code. This also fixex nil
error of storageGetReadOnlyContext in case when contract does not have
storage.
Fixes#1144. It's quite simple approach, we just update balance info right
upon contract migration. It will slow down migration transactions, but it
takes about 1-2 seconds to Seek through balances at mainnet's 3.8M, so the
approach should still work good enough. The other idea was to make lazy
updates (maintaining contract migration map), but it's more complicated to
implement (and implies that a balance get might also do a write).
There also is a concern about memory usage, it can give a spike of some tens
of megabytes, but that also is considered to be acceptable.
Part of #1055.
We should check contract scripthash against the one provided in manifest
and manifest groups. We shouldn't put on stack anything after return.
And ofcourse, we mast not destroy the old contract at the end, as
`contractDestroy` removes all storage items associated with the
old contract ID (which equals to the new contract ID). We just remove
old contract state - it's enough.
We were accepting transactions with zero system fee, but we shouldn't do
that. Also, transaction's verification execution has to be limited by network
fee.
GetValidators without parameter is called upon DBFT initialization and it
should receive validators for the next block (that will create it),
parameterized GetValidators is used for NextConsensus calculation where we
need a list for the current state of the chain.
NextBlockValidators are updated before the new block persist, so we need to
use GetValidators to get the list corresponding to the current state of the
chain.
Part of #1133
It will help us to use big.Int to store amount of NEP5 tokens. As far as
big.Int doesn't have constant size, we shouldn't use `NEP5TransferSize`
constant anymore.
Even if the value is zero, the GAS distribution updates the balance height, so
storage item must be updated too. Fixes the followin on preview2 testnet:
block 74227: value mismatch for key ffffffff1454a6cb279fbcedc66162ad4ad5d1d910202b92743e000000000000000000000005: 1041032104809fd5002103f3210128010000 vs 1041032104809fd50021033f110128010000
They make no sense. Fixes preview2 testnet state problem:
file BlockStorage_100000/dump-block-70000.json: block 69935: state mismatch for key ffffffff1454a6cb279fbcedc66162ad4ad5d1d910202b92743e000000000000000000000005: Deleted vs Added
And fix contract create to really update the ID, eliminating this difference
in the storage (preview2 testnet):
file BlockStorage_100000/dump-block-39000.json: block 38043: key mismatch: 0c000000617373657464df4ebe92334d1fc7e64b10f1d1e33942d9905e510000000000000009 vs 00000000617373657464df4ebe92334d1fc7e64b10f1d1e33942d9905e510000000000000009
Preview2 testnet:
file BlockStorage_100000/dump-block-12000.json: block 11562: key mismatch: feffffff1454a6cb279fbcedc66162ad4ad5d1d910202b92743e000000000000000000000005 vs feffffff1431b7e7aea5131f74721e002c6a56b610885813f79e000000000000000000000005
Originally this code was written to run after transactions processing, but
after 0fa4c49735 it works in different manner.
ValidatorsCount is not initialized at block 0 with C# node (the first voter
initializes it) and until that initialization happens the standby validators
list is being returned as is without sorting.
Fixes state mismatch for the key ffffffff0e00000000000000000000000000000001 in
the first blocks.
It also affects tests as now the first validator is different and it receives
the network fees.
After block was stored it's possible to have new FeePerByte constraint,
so we should remove all transactions which do not meet this requirement.
Also caching of FeePerByte was added in order not to re-verify
transactions each time mempool needs to be updated.
MarshalJSON should be defined on structure (not pointer), as we use
structures to marshal parameters (e.g. in NotificationEvent and
Invoke of RPC result package) and never use pointers for that purpose.
Also added marshalling of nil array into `[]` instead of `null` to
follow C# implementation.
part of #904
1. We now have MaxTransactionsPerBlock set in native Policy contract,
so this value should be used in (dbft).GetVerified method instead
of passing it as an argument.
2. Removed (dbft).WithTxPerBlock.
2. DBFT API has changed, so update it's version.
3. Removed MaxTransactionsPerBlock from node configuration, as we
have it set in native Policy contract.
C# implementation uses NEWARRAY for creating arguments.
Don't change our implementation in `emit`, because PACK is cheaper and
this script must not depend on the internal details of `emit` package anyway.
There is no such thing as high/low priority transactions, as there are
no free transactions anymore and they are ordered by fees contained
in transaction itself.
Closes#1063.
It's just JSON, io.Serializable is only used for DB storage where the length
should be obtained from the stream. Fixes:
2020-06-18T22:14:10.571+0300 WARN contract invocation failed {"tx": "1ffd475a9c246495d6206cb80a9a78e9d14a433ded60cd37aa87d897655606e1", "block": 25893, "error": "error encountered at instruction 3696 (SYSCALL): failed to invoke syscall: invalid character ':' after top-level value"}
We make it explicit in the appropriate Block/Transaction structures, not via a
singleton as C# node does. I think this approach has a bit more potential and
allows better packages reuse for different purposes.
