Evgenii Stratonikov
54b177cf40
dao: store blocks/txransactions by big-endian hash
...
There is no need for additional allocations.
2020-11-25 14:03:43 +03:00
Anna Shaleva
b00eb51c55
core: add P2PNotary designated role
2020-11-24 18:47:09 +03:00
Evgenii Stratonikov
31eca342eb
*: replace all NEP5 occurences to NEP17
2020-11-24 13:08:24 +03:00
Evgenii Stratonikov
b97dfae8d8
native: replace NEP-5 with NEP-17
2020-11-24 13:08:23 +03:00
Evgenii Stratonikov
a5914f89fa
core: allow to provide block in GetTestVM()
...
Sometimes amount of GAS consumed depends on block height.
2020-11-24 12:17:29 +03:00
Evgenii Stratonikov
e38e8aa48a
rpc: implement getproof RPC
2020-11-20 18:06:22 +03:00
Evgenii Stratonikov
1869d6d460
core: allow to use state root in header
2020-11-20 17:16:32 +03:00
Evgenii Stratonikov
85f927d892
mpt: implement reference counting
...
Also postpone MPT initialization until `storeBlock`
because we need to read-and-check or save info about refcounting
depending on starting height.
2020-11-20 16:50:30 +03:00
Anna Shaleva
083879838c
rpc: adjust getrawtransaction and gettransactionheight RPC call
...
We should return verbose transaction in case if it is in the mempool
from `getrawtransaction`. We also shouldn't return height from
`gettransactionheight` in case if transaction is in the mempool.
2020-11-16 13:25:42 +03:00
Anna Shaleva
7ca93e76ac
core, rpc: allow to store several AppExecResult for a single hash
...
It is required for we have several executions per block.
2020-11-12 16:24:39 +03:00
Evgenii Stratonikov
54992ad4f3
core: provide account in calculate claimable
...
`getunclaimedgas` RPC should return all GAS available to claim.
2020-11-10 16:08:21 +03:00
Evgenii Stratonikov
af583c14ea
native: add GAS rewards for voters
...
Close #1348 .
2020-11-10 15:30:30 +03:00
Roman Khimov
bdd073aad7
Merge pull request #1520 from nspcc-dev/tune-some-limits
...
Tune some limits
2020-11-06 15:35:44 +03:00
Roman Khimov
0ee377792e
Merge pull request #1519 from nspcc-dev/designate-role-history
...
native: implement designate contract history retention
2020-11-06 15:35:18 +03:00
Roman Khimov
9e781bff47
native: implement designate contract history retention
...
Follow neo-project/neo#2007 . Fix getDesignatedByRole price along the way.
2020-11-06 13:46:40 +03:00
Roman Khimov
b327308df8
core/config: move MaxTraceableBlocks to configuration file
...
Follow neo-project/neo#2042 , use 2102400 setting for mainnet/testnet and
200000 for private networks.
2020-11-06 13:41:46 +03:00
Anna Shaleva
15a939b1da
rpc: allow to getcontractstate by address, id or name
...
close #1423
2020-11-03 17:23:49 +03:00
Roman Khimov
eaa260474f
trigger/core: split System trigger into OnPerist and PostPersist
...
Follow neo-project/neo#2022 .
2020-10-29 19:17:07 +03:00
Roman Khimov
044786b995
core: use Application trigger in CalculateClaimable()
...
It doesn't need a System one.
2020-10-29 19:17:07 +03:00
Anna Shaleva
ec63d5c456
core: add conflicts attribute
...
Close #1491
2020-10-29 10:57:31 +03:00
Anna Shaleva
95630b72e8
core: verify transaction attributes before adding it into mempool
...
Attributes check should be done before adding transaction to
the pool, otherwise there might be a case when transaction with invalid
attributes is in the pool.
2020-10-23 11:05:03 +03:00
Anna Shaleva
09b8b8de73
core: reserve attributes range for experimantal purposes
2020-10-23 11:04:59 +03:00
Anna Shaleva
368ff820b3
core: add NotValidBefore transaction attribute
...
Close #1490
2020-10-23 11:02:46 +03:00
Evgenii Stratonikov
aee02ff2e3
core: allow to invoke verify of native contracts
2020-10-14 11:44:27 +03:00
Roman Khimov
a154481860
core: drop stale transactions from the mempool
...
They're no longer valid if `ValidUntilBlock == curheight`.
2020-10-12 14:58:40 +03:00
Roman Khimov
8c2fd91c5c
Merge pull request #1462 from nspcc-dev/rpc/exceptions
...
rpc, core: add FaultException to AppExecResult and Invoke* results
2020-10-08 16:20:38 +03:00
Anna Shaleva
0f71088246
core: add FaultException to AppExecResult
2020-10-08 16:12:19 +03:00
Anna Shaleva
9fe2b04db9
core: add cache to Policy blocked accounts
...
Close #1441
2020-10-08 13:47:21 +03:00
Roman Khimov
8e146d19b3
Merge pull request #1447 from nspcc-dev/core/fix_neo_cache
...
core: fix cache for (NEO).nextValidators and (NEO).committee
2020-10-02 17:09:43 +03:00
Anna Shaleva
eccf639803
core: add InitializeCache method to NEO native contracts
...
There might be a case when cached contract values store nil (e.g.
after restoring chain from dump). We should always initialize cached
values irrespective to the (NEO).Initialize method.
This commit fixes a bug introduced in 83e94d3
2020-10-02 17:03:06 +03:00
Roman Khimov
b5d138b150
Merge pull request #1451 from nspcc-dev/native/designative
...
native: implement Designate contract
2020-10-02 16:09:33 +03:00
Evgenii Stratonikov
c468c02ef5
native: implement Designate contract
2020-10-02 11:03:25 +03:00
Roman Khimov
991089593f
Merge pull request #1450 from nspcc-dev/fix/witnessscope
...
transaction: rename FeeOnly to None
2020-10-01 16:46:07 +03:00
Evgenii Stratonikov
fa09b9af7b
transaction: rename FeeOnly to None
...
Follow missed change from neo-project/neo#1816 .
`None` may be used for any signer. Currently it is used
for sender to only pay fees, or to sign tx attributes.
2020-10-01 15:28:19 +03:00
Anna Shaleva
e34e367a7b
core: take into account size fee during verifyTxWitnesses
...
GasLimit for transaction scripts verification should not include fee for
transaction size.
2020-09-30 21:40:04 +03:00
Anna Shaleva
a2bfd16136
core: take into account gasConsumed during tx witnesses verification
...
We should pay attention to the previously consumed gas during tx
witnesses verification.
2020-09-29 17:43:35 +03:00
Evgenii Stratonikov
e91d13c615
core: implement oracle tx verification
2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
bd1790bddf
native: cache oracle nodes
2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
ef7c0dbd78
core: use GetCommitteeAddress to get committee address
2020-09-25 17:02:31 +03:00
Evgenii Stratonikov
c5cdaae87a
native: support postPersist method
...
It should be called for NEO contract to distribute
committee bounties.
2020-09-23 14:47:09 +03:00
Evgenii Stratonikov
81a11c629a
native: remove getValidators method
...
Follow https://github.com/neo-project/neo/pull/1920 .
2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
83e94d3bbc
native: cache committee members
2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
7d90d79ae6
core: update claimable GAS calculation
2020-09-23 14:12:42 +03:00
Roman Khimov
e4b52d3947
core/rpc: add continue flag to iterating functions
...
Most of the time we don't need to get all transfers from the DB and
deserialize them.
2020-09-21 22:23:34 +03:00
Roman Khimov
373c669c6a
core/state: reverse the order of ForEachTransfer
...
When using limits we're usually concerned about the most recent
transfers. Returning 3 transfers from the middle of the chain isn't very
helpful.
2020-09-21 22:05:15 +03:00
Anna Shaleva
770c8d774c
core, rpc: add GetCommittee method
...
Closes #1414
2020-09-21 15:56:25 +03:00
Anna Shaleva
c6f099294b
core: do not marshal block hash in application log
...
Closes #1388
2020-09-21 14:08:15 +03:00
Roman Khimov
1608fbff87
Merge pull request #1378 from nspcc-dev/tests/cli
...
Implement tests for CLI
2020-09-19 17:02:34 +03:00
Evgenii Stratonikov
bff67c921a
core: wrap dao in GetTestVM
...
MPT must not be shared to the test VM.
Fix DATA RACE between `Collapse()` and RPC calls.
2020-09-18 12:07:02 +03:00
Roman Khimov
61ad82160b
core: remove named return variable from addHeaders()
2020-09-16 17:45:12 +03:00
Roman Khimov
6bffa811d4
core: remove (*block.Block) checks for topBlock
...
If it's non-nil, it has *block.Block inside. If it doesn't --- tell everyone
about it with a nice panic message.
2020-09-16 17:40:27 +03:00
Roman Khimov
33ea179f6e
core: remove forward index check from addHeaders()
...
It can't ever happen. We're guaranteed to have a consistent chain of headers
(we're verifying them above, if we're not verifying --- it's not our fault)
that starts at HeaderHeight that was actual when we were asking for it
previously. HeaderHeight can only move forward, so if that happened that would
be filtered out by the condition below and the first one can't happen. Though
to be absolutely sure change the second check to only pass "+1" headers (which
is what we want).
2020-09-16 16:33:42 +03:00
Roman Khimov
7cbb660082
core: rework headerList
...
It's a contention point as all accesses to it are serialized and they compete
with persisting logic at the same time.
2020-09-16 16:30:40 +03:00
Roman Khimov
ce09c82b25
block: remove Verify()
...
It's used in two places now:
* Blockchain.AddBlock()
This one does transaction duplication check of its own, doing it in
Verify() is just a waste of time. Merkle tree root hash value check is
still relevant though
* Block.DecodeBinary()
We're decoding blocks for the following purposes:
- on restore from dump
The block will be added to the chain via AddBlock() and that will do a
full check of it (if configured to do so)
- on retrieving the block from the DB (DAO)
We trust the DB, if it's gone wild, this check won't really help
- on receiving the block via P2P
It's gonna be put into block queue and then end up in AddBlock() which
will check it
- on receiving the block via RPC (submitblock)
It is to be passed into AddBlock()
- on receiving the block via RPC in a client
That's the only problematic case probably, but RPC client has to trust
the server and it can check for the signature if it really
cares. Or a separate in-client check might be added.
As we can see nothing really requires this verification to be done the way it
is now, AddBlock can just have a Merkle check and DecodeBinary can do fine
without it at all.
2020-09-16 12:50:13 +03:00
Roman Khimov
2e876b5593
block: remove Base.Verify()
...
It's a no-op and there is nothing we can do about it, header contents could
only be checked against chain state, there is nothing to check for internal
consistency.
2020-09-16 12:50:13 +03:00
Roman Khimov
19c69618c5
transaction: cache tx size, don't serialize it over and over again
2020-09-11 18:55:19 +03:00
Roman Khimov
7310e748e3
core: reuse mempool from AddBlock() for bc.isTxStillRelevant()
...
It's already there most of the time and creating another slice is just a waste
of time. Checking for presence with map is also a little faster.
2020-09-10 15:02:03 +03:00
Roman Khimov
26339c75dc
vm/core: drop old key caching system
...
Obsoleted by f5f58a7e91
2020-09-10 14:43:24 +03:00
Roman Khimov
a2d9b89964
dao: reuse buffers when storing blocks, txes and aers
...
Reduce memory allocation pressure.
2020-09-09 20:46:31 +03:00
Evgenii Stratonikov
2661ebd295
transaction: add HighPriority attribute
...
HighPriority attributes specifies that transaction was
signed by a committee.
2020-08-23 09:39:46 +03:00
Roman Khimov
e5813ae8cd
Merge pull request #1346 from nspcc-dev/string-tryboolean
...
Fail converting long strings to boolean
2020-08-22 17:04:11 +03:00
Evgenii Stratonikov
7e34072519
core: implement (*Blockchain).VerifyWitness
...
`ScriptFromWitness` is no longer useful, because we support
contract verification.
2020-08-22 12:45:20 +03:00
Roman Khimov
a7670303e8
stackitem: change Bool() to TryBool(), prepare for its failures
2020-08-21 20:55:20 +03:00
Roman Khimov
55b2cbb74d
core: refactor and improve verification and pooling
...
Now we have VerifyTx() and PoolTx() APIs that either verify transaction in
isolation or verify it against the mempool (either the primary one or the one
given) and then add it there. There is no possibility to check against the
mempool, but not add a transaction to it, but I doubt we really need it.
It allows to remove some duplication between old PoolTx and verifyTx where
they both tried to check transaction against mempool (verifying first and then
adding it). It also saves us utility token balance check because it's done by
the mempool anyway and we no longer need to do that explicitly in verifyTx.
It makes AddBlock() and verifyBlock() transaction's checks more correct,
because previously they could miss that even though sender S has enough
balance to pay for A, B or C, he can't pay for all of them.
Caveats:
* consensus is running concurrently to other processes, so things could
change while verifyBlock() is iterating over transactions, this will be
mitigated in subsequent commits
Improves TPS value for single node by at least 11%.
Fixes #667 , fixes #668 .
2020-08-20 18:50:18 +03:00
Roman Khimov
e998c102ca
mempool: rename NewMemPool into New
...
"mempool" is a package name already.
2020-08-20 18:50:18 +03:00
Roman Khimov
c7032022f8
core: don't search through the whole DAO in isTxStillRelevant
...
New transactions are added to the chain with blocks. If there is no
transaction X at height N in DAO, it could only be added with block N+1, so
it has to be present there. Therefore we can replace `dao.HasTransaction()`
check with a search through in-block transactions. HasTransaction() is nasty
in that it may add useless load the DB and this code is being run with a big
Blockchain lock held, so we don't want to be delayed here at all.
Improves single-node TPS by ~2%.
2020-08-20 18:50:18 +03:00
Roman Khimov
3a379e4f3e
core: don't reverify mempooled transactions in AddBlock
...
The end effect is almost as if `VerifyTransactions: false` was set in the
config, but without actually compromising the guarantees provided by it.
It almost doubles performance for single-mode benchmarks and makes block
processing smoother (more smaller blocks are being produced).
2020-08-19 15:16:19 +03:00
Roman Khimov
d0c29f52c9
core: verification script must return exactly one value
...
C# node is quite picky as it expects there to be exactly one value returned,
but our testchain actually adds 4 signatures for multisig cases instead of 3
which makes it technically incompatible with C# node.
2020-08-17 22:02:15 +03:00
Roman Khimov
8e619cc671
Merge pull request #1318 from nspcc-dev/fix/verifytests
...
Add tests for `verifyTx` and `verifyHeader`
2020-08-14 17:24:03 +03:00
Roman Khimov
e7d13e6db2
*: fix misspellings found in Go Report Card
2020-08-14 12:16:24 +03:00
Evgenii Stratonikov
1eb9a4c6c6
core: allow to use verification contracts
...
In NEO3 we can't just appcall hash, as verification script has no access
to state. Instead we use `verify` method of an arbitrary contract.
2020-08-14 09:40:34 +03:00
Evgenii Stratonikov
7f2a931fb6
core: add tests for (*Blockchain).verifyHeader
2020-08-14 09:37:32 +03:00
Evgenii Stratonikov
e8cf4d96ce
core: add tests for (*Blockchain).verifyTx
2020-08-14 09:37:30 +03:00
Roman Khimov
ca542f11fe
core: drop unused secondsPerBlock() method
2020-08-11 21:37:18 +03:00
Roman Khimov
b1034d8ed6
state: drop Neo 2 Account and everything related
...
It's substituted with NEP5Balances now.
2020-08-11 20:42:02 +03:00
Evgenii Stratonikov
807338f97e
core: do not store NEP5 transfer log in memory
...
Traversing transfer log instead of accumulating and returning it
is faster and takes less memory.
2020-08-07 18:21:06 +03:00
Roman Khimov
c3c88a57cd
Merge pull request #1281 from nspcc-dev/drop-go-1.12-and-fix-some-things
...
Drop go 1.12 and fix some things
2020-08-07 13:34:54 +03:00
Roman Khimov
d5a9d80c12
core: refactor out policy check for transaction
...
We were checking blocked accounts twice which is obviously excessive. We also
have our accounts sorted, so we can rely on that in CheckPolicy(). It also
doesn't make much sense to check MaxBlockSystemFee in Blockchain code, policy
contract can handle that.
2020-08-07 12:21:52 +03:00
Roman Khimov
90180c6fb6
native: pass DAO to CheckPolicy(), it doesn't need interop context
...
Simplify things a bit.
2020-08-07 12:21:52 +03:00
Roman Khimov
791c983304
core: drop GetScriptHashesForVerifying
...
It no longer depends on blockchain state and there can't ever be an error, in
fact we can always iterate over signers, so copying these hashes doesn't make
much sense at all as well as sorting arrays in verifyTxWitnesses (witnesses
order must match signers order).
2020-08-07 12:21:52 +03:00
Roman Khimov
c19838ea67
core: use errors.Is in PoolTx
...
Just in case.
2020-08-07 12:21:52 +03:00
Roman Khimov
4c38fae54c
core: fix mempool error check in PoolTx
...
ErrAlreadyExists should be returned for ErrDup, not ErrConflict.
2020-08-07 12:21:52 +03:00
Roman Khimov
0e2784cd2c
always wrap errors when creating new ones with fmt.Errorf()
...
It doesn't really change anything in most of the cases, but it's a useful
habit anyway.
Fix #350 .
2020-08-07 12:21:52 +03:00
Roman Khimov
205f52c563
core: use error wrapping to provide more details
2020-08-07 12:21:52 +03:00
Roman Khimov
5ef08f60ae
remove github.com/pkg/errors from dependencies
...
It's not needed any more with Go 1.13 as we have wrapping/unwrapping in base
packages. All errors.Wrap calls are replaced with fmt.Errorf, some strings are
improved along the way.
2020-08-07 12:21:52 +03:00
Evgenii Stratonikov
27169d140f
core: implement (*Blockchain).GetStandByCommitee()
2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
3e39f0f211
native: rename Validator to Candidate
2020-08-06 14:26:24 +03:00
Roman Khimov
b7f89b0544
core: restrict verification scripts with CallFlags.None
...
Follow neo-project/neo#1776 . Looks very suspicious to me as it severely limits
verification scripts, but we have to be compatible at this point.
2020-08-05 19:17:48 +03:00
Anna Shaleva
90825efa16
core: move transaction's sender to cosigners
...
Closes #1184
Ported changes from https://github.com/neo-project/neo/pull/1752
2020-08-04 17:33:50 +03:00
Roman Khimov
62f5aa8eb4
Merge pull request #1254 from nspcc-dev/native/policy/maxblocksystemfee
...
core, consensus: add maxBlockSystemFee to native Policy
2020-08-04 13:13:06 +03:00
Anna Shaleva
db65ed04d9
consensus: apply policy during verifyBlock
...
To follow C# implementation we should also check proposed block on
policy matching.
2020-08-04 10:59:09 +03:00
Anna Shaleva
4a1c8464f9
core: use maxBlockSize to apply policy to tx set
...
We should also take into account maxBlockSize in
(bc *Blockchain).ApplyPolicyToTxSet
2020-08-04 10:59:09 +03:00
Anna Shaleva
c647f8e4ed
core: add maxBlockSystemFee to native Policy
...
Closes #1195
2020-08-04 10:59:00 +03:00
Evgenii Stratonikov
e5d538ed21
core: use raw stack items in application logs
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
fyrchik
e4fc655115
Merge pull request #1239 from nspcc-dev/store_nep5_with_id
...
core: store contract IDs instead of hashes for NEP5Balances and Transfers
2020-07-30 12:56:19 +03:00
Roman Khimov
e21c65c59f
core: add state height to prometheus metrics
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
5ee3ecf381
core: update verified state root height
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
236438d799
core: do MPT compaction every once in a while
...
We need to compact our in-memory MPT from time to time, otherwise it quickly
fills up all available memory. This raises two obvious quesions --- when to do
that and to what level do that.
As for 'when', I think it's quite easy to use our regular persistence interval
as an anchor (and it also frees up some memory), but we can't do that in the
persistence routine itself because of synchronization issues (adding some
synchronization primitives would add some cost that I'd also like to avoid),
so do it indirectly by comparing persisted and current height in `storeBlock`.
Choosing proper level is another problem, but if we're to roughly estimate one
full branch node to use 1K of memory (usually it's way less than that) then we
can easily store 1K of these nodes and that gives us a depth of 10 for our
trie.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
58b7e16e0e
core: fix PrevHash calculation for MPTRoot
...
This was differing from C# notion of PrevHash. It's not a previous root, but
rather a hash of the previous serialized MPTRoot structure (that is to be
signed by CNs).
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
0e29382035
core: update MPT during block processing
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
