Roman Khimov
2ea29924c4
Merge pull request #1454 from nspcc-dev/core/policy_checks
...
core: add validation to policy methods
2020-10-05 19:09:59 +03:00
Anna Shaleva
560e470484
core: add validation to native Policy methods
...
close #1442
2020-10-05 18:08:37 +03:00
Anna Shaleva
e1e586f18b
core: restrict the muximum number of contents per block
2020-10-05 18:08:33 +03:00
Roman Khimov
0e82d4cbd1
Merge pull request #1458 from nspcc-dev/fix/storageput
...
core: calculate gas for `System.Storage.Put` correctly
2020-10-05 14:26:17 +03:00
Evgenii Stratonikov
833bbb1d35
core: calculate gas for System.Storage.Put correctly
...
If `Put` creates new key, its length should contribute
to a GAS cost of the syscall.
2020-10-05 13:03:44 +03:00
Roman Khimov
8e146d19b3
Merge pull request #1447 from nspcc-dev/core/fix_neo_cache
...
core: fix cache for (NEO).nextValidators and (NEO).committee
2020-10-02 17:09:43 +03:00
Anna Shaleva
eccf639803
core: add InitializeCache method to NEO native contracts
...
There might be a case when cached contract values store nil (e.g.
after restoring chain from dump). We should always initialize cached
values irrespective to the (NEO).Initialize method.
This commit fixes a bug introduced in 83e94d3
2020-10-02 17:03:06 +03:00
Roman Khimov
15e459feea
core: fix designate test
...
Merge conflict between #1450 and #1451 .
2020-10-02 16:23:48 +03:00
Roman Khimov
b5d138b150
Merge pull request #1451 from nspcc-dev/native/designative
...
native: implement Designate contract
2020-10-02 16:09:33 +03:00
Anna Shaleva
6b201893fa
core: unexport NewNEO() and NewGAS() methods
2020-10-02 15:45:49 +03:00
Evgenii Stratonikov
c468c02ef5
native: implement Designate contract
2020-10-02 11:03:25 +03:00
Roman Khimov
991089593f
Merge pull request #1450 from nspcc-dev/fix/witnessscope
...
transaction: rename FeeOnly to None
2020-10-01 16:46:07 +03:00
Evgenii Stratonikov
fa09b9af7b
transaction: rename FeeOnly to None
...
Follow missed change from neo-project/neo#1816 .
`None` may be used for any signer. Currently it is used
for sender to only pay fees, or to sign tx attributes.
2020-10-01 15:28:19 +03:00
Roman Khimov
08aa773dac
Merge pull request #1446 from nspcc-dev/core/network_fee_fixes
...
core: fix CalculateNetworkFee and verifyTxWitnesses
2020-09-30 21:46:33 +03:00
Anna Shaleva
e34e367a7b
core: take into account size fee during verifyTxWitnesses
...
GasLimit for transaction scripts verification should not include fee for
transaction size.
2020-09-30 21:40:04 +03:00
Anna Shaleva
2b11e99225
core: fix CalculateNetworkFee for signature contracts
...
First PUSHDATA1 is from invocation script, the second PUSHDATA1 is
from verification script. E.g.:
Invocation script:
INDEX OPCODE PARAMETER
0 PUSHDATA1 035913b9588da23a5c3ce14b2886a6b8ebb6a0eb92bdaa948510dfb5ae5194d6cb <<
35 PUSHNULL
36 SYSCALL Neo.Crypto.VerifyWithECDsaSecp256r1 (95440d78)
Verification script:
INDEX OPCODE PARAMETER
0 PUSHDATA1 3930fe5a9b44682f37741955df4a5f2585ed5aa438fa6e17ae51083673b1d64253e5a859c0cf168be67971e53a23c1c40582777d94a8e391db23ff613849627d <<
2020-09-30 21:31:43 +03:00
Roman Khimov
5365da9860
native: remove unregistered candidate with 0 votes on NEO transfer
...
See neo-project/neo#1935 .
2020-09-29 22:38:38 +03:00
Roman Khimov
adcbb2287f
Merge pull request #1439 from nspcc-dev/core/verify_tx_witnesses_fix
...
core: take into account gasConsumed during tx witnesses verification
2020-09-29 18:11:39 +03:00
Anna Shaleva
ff6aa0fd45
core: allow empty callingScriptHash during CheckWitness
...
CallingScriptHash can be empty in the entry transaction script.
2020-09-29 17:43:39 +03:00
Anna Shaleva
a2bfd16136
core: take into account gasConsumed during tx witnesses verification
...
We should pay attention to the previously consumed gas during tx
witnesses verification.
2020-09-29 17:43:35 +03:00
Roman Khimov
c21f699ffc
Merge pull request #1432 from nspcc-dev/fix/postpersist
...
Initialize cache in native contracts properly
2020-09-29 14:21:15 +03:00
Roman Khimov
e4be60c3a8
Merge pull request #1435 from nspcc-dev/checkwitness-calling-script-hash
...
interop/runtime: allow calling script hash to pass CheckWitness
2020-09-29 12:49:09 +03:00
Evgenii Stratonikov
26dc807a2d
native: initialize cache in OnPersist
...
If node started from non-zero height, `Initialize` is
not called thus cache is empty. Thus we should initialize
it before any tx is executed.
2020-09-29 10:17:47 +03:00
Roman Khimov
53c9690bdc
interop/runtime: allow calling script hash to pass CheckWitness
...
See neo-project/neo#1924 and neo-project/neo#1925 .
2020-09-29 09:56:19 +03:00
Evgenii Stratonikov
05118e96f4
native: store cache during Oracle contract creation
...
`Initialize` is not called during restore from dump.
2020-09-28 15:34:04 +03:00
Evgenii Stratonikov
e0e7fd5367
native: cache GASPerBlock values
...
Close #1421 .
2020-09-28 10:53:48 +03:00
Evgenii Stratonikov
e91d13c615
core: implement oracle tx verification
2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
f084acc339
native: cache oracle script hash
2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
bd1790bddf
native: cache oracle nodes
2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
141d6e325f
native: implement basis for oracle contract
2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
ef7c0dbd78
core: use GetCommitteeAddress to get committee address
2020-09-25 17:02:31 +03:00
Evgenii Stratonikov
5b205ffa7d
native: cache committee script hash
2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
ca1b8a7df0
state: implement marshalers for oracle-related state
2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
30526046e7
native: don't push void result in Call
...
This was done in https://github.com/neo-project/neo/pull/1693
for native calls. `OnPersist` script still uses `DROP` though
as value is pushed via `CheckReturn` logic for regular calls.
2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
877b8ece63
core: move contractCall* to a separate package
2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
c5cdaae87a
native: support postPersist method
...
It should be called for NEO contract to distribute
committee bounties.
2020-09-23 14:47:09 +03:00
Evgenii Stratonikov
af16519413
native: do not update committee every block
...
Update frequency depends on committee size and
amount of validators. For mainnet it is 28.
2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
81a11c629a
native: remove getValidators method
...
Follow https://github.com/neo-project/neo/pull/1920 .
2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
83e94d3bbc
native: cache committee members
2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
43b3e15330
native: send GAS to a committee member on persist
2020-09-23 14:42:12 +03:00
Evgenii Stratonikov
5a38208361
native: implement NEO.Get/SetMaxGasPerBlock()
2020-09-23 14:12:42 +03:00
Evgenii Stratonikov
7d90d79ae6
core: update claimable GAS calculation
2020-09-23 14:12:42 +03:00
Roman Khimov
41af738c1b
state: drop (*NEP5TransferLog).DecodeBinaryReturnCount
...
It's no longer needed.
2020-09-22 19:21:12 +03:00
Roman Khimov
e4b52d3947
core/rpc: add continue flag to iterating functions
...
Most of the time we don't need to get all transfers from the DB and
deserialize them.
2020-09-21 22:23:34 +03:00
Roman Khimov
373c669c6a
core/state: reverse the order of ForEachTransfer
...
When using limits we're usually concerned about the most recent
transfers. Returning 3 transfers from the middle of the chain isn't very
helpful.
2020-09-21 22:05:15 +03:00
Roman Khimov
ff11a5f990
state: use more efficient encoding for amount
...
It's variable-length anyway, so wasting 8 bytes for what typically is 1 byte
makes no sense.
2020-09-21 21:51:33 +03:00
Roman Khimov
806b89db76
state: store the length of NEP5TransferLog in the first byte
...
We lose `size` field after serialization/deserialization which can lead to
adding more than NEP5TransferBatchSize elements into the NEP5TransferLog.
2020-09-21 21:51:11 +03:00
Roman Khimov
0c264b1486
mpt: fix comment typo
2020-09-21 17:54:11 +03:00
Anna Shaleva
770c8d774c
core, rpc: add GetCommittee method
...
Closes #1414
2020-09-21 15:56:25 +03:00
Anna Shaleva
c6f099294b
core: do not marshal block hash in application log
...
Closes #1388
2020-09-21 14:08:15 +03:00
Roman Khimov
a439941a71
Merge pull request #1407 from nspcc-dev/core/oracleattr
...
Implement OracleResponse transaction attribute
2020-09-21 12:55:10 +03:00
Roman Khimov
1608fbff87
Merge pull request #1378 from nspcc-dev/tests/cli
...
Implement tests for CLI
2020-09-19 17:02:34 +03:00
Evgenii Stratonikov
bff67c921a
core: wrap dao in GetTestVM
...
MPT must not be shared to the test VM.
Fix DATA RACE between `Collapse()` and RPC calls.
2020-09-18 12:07:02 +03:00
Roman Khimov
61ad82160b
core: remove named return variable from addHeaders()
2020-09-16 17:45:12 +03:00
Roman Khimov
6bffa811d4
core: remove (*block.Block) checks for topBlock
...
If it's non-nil, it has *block.Block inside. If it doesn't --- tell everyone
about it with a nice panic message.
2020-09-16 17:40:27 +03:00
Roman Khimov
33ea179f6e
core: remove forward index check from addHeaders()
...
It can't ever happen. We're guaranteed to have a consistent chain of headers
(we're verifying them above, if we're not verifying --- it's not our fault)
that starts at HeaderHeight that was actual when we were asking for it
previously. HeaderHeight can only move forward, so if that happened that would
be filtered out by the condition below and the first one can't happen. Though
to be absolutely sure change the second check to only pass "+1" headers (which
is what we want).
2020-09-16 16:33:42 +03:00
Roman Khimov
7cbb660082
core: rework headerList
...
It's a contention point as all accesses to it are serialized and they compete
with persisting logic at the same time.
2020-09-16 16:30:40 +03:00
Evgenii Stratonikov
1625689316
transaction: implement OracleResponse attribute
2020-09-16 14:50:31 +03:00
Evgenii Stratonikov
06b29e409c
transaction: remove Attribute.Data field
2020-09-16 13:39:53 +03:00
Roman Khimov
ce09c82b25
block: remove Verify()
...
It's used in two places now:
* Blockchain.AddBlock()
This one does transaction duplication check of its own, doing it in
Verify() is just a waste of time. Merkle tree root hash value check is
still relevant though
* Block.DecodeBinary()
We're decoding blocks for the following purposes:
- on restore from dump
The block will be added to the chain via AddBlock() and that will do a
full check of it (if configured to do so)
- on retrieving the block from the DB (DAO)
We trust the DB, if it's gone wild, this check won't really help
- on receiving the block via P2P
It's gonna be put into block queue and then end up in AddBlock() which
will check it
- on receiving the block via RPC (submitblock)
It is to be passed into AddBlock()
- on receiving the block via RPC in a client
That's the only problematic case probably, but RPC client has to trust
the server and it can check for the signature if it really
cares. Or a separate in-client check might be added.
As we can see nothing really requires this verification to be done the way it
is now, AddBlock can just have a Merkle check and DecodeBinary can do fine
without it at all.
2020-09-16 12:50:13 +03:00
Roman Khimov
2e876b5593
block: remove Base.Verify()
...
It's a no-op and there is nothing we can do about it, header contents could
only be checked against chain state, there is nothing to check for internal
consistency.
2020-09-16 12:50:13 +03:00
Roman Khimov
d52e79668b
hash: introduce memory-optimized merkle root hash calculation routine
...
NewMerkleTree is a memory hog, we can do better than that:
BenchmarkMerkle/NewMerkleTree-8 13 88434670 ns/op 20828207 B/op 300035 allocs/op
BenchmarkMerkle/CalcMerkleRoot-8 15 69264150 ns/op 0 B/op 0 allocs/op
2020-09-15 18:38:15 +03:00
Roman Khimov
19c69618c5
transaction: cache tx size, don't serialize it over and over again
2020-09-11 18:55:19 +03:00
Roman Khimov
83fc38ae3a
mempool: don't create new big.Int in tryAddSendersFee() if possible
...
Do a little less allocations.
2020-09-10 15:35:19 +03:00
Roman Khimov
fc7ea6217d
mempool: avoid reassigning utilityBalanceAndFees value
...
It's not needed, we're either creating a new one and assigning it 6 lines
above or we're changing already existing big.Int via a pointer, so no update
is needed.
2020-09-10 15:20:04 +03:00
Roman Khimov
7310e748e3
core: reuse mempool from AddBlock() for bc.isTxStillRelevant()
...
It's already there most of the time and creating another slice is just a waste
of time. Checking for presence with map is also a little faster.
2020-09-10 15:02:03 +03:00
Roman Khimov
26339c75dc
vm/core: drop old key caching system
...
Obsoleted by f5f58a7e91
2020-09-10 14:43:24 +03:00
Roman Khimov
fe1f1d19be
mempool: store only pointer in the verifiedMap
...
It's only used for presence checks, there is no need for metadata here.
2020-09-09 20:46:31 +03:00
Roman Khimov
a2d9b89964
dao: reuse buffers when storing blocks, txes and aers
...
Reduce memory allocation pressure.
2020-09-09 20:46:31 +03:00
Roman Khimov
9591d64e53
mempool: don't sort items by hash
...
There is nothing requiring us to do so. It also is bad because it allows for
new transaction to replace some already existing one with the same fee
parameters just because it has "better" hash.
But the other thing is that for transactions with equal fees it's always
better for us to append them to the end of the list, instead of inserting them
in the middle, so this change allows to reduce slice item movements and gain
some 6-7% increase for single-node TPS.
2020-09-09 20:46:31 +03:00
Roman Khimov
5df726db68
mempool: replace timeStamp with blockStamp
...
Time is not really relevant for us here and we don't use this timestamp in any
way. Yet it occupies 24 bytes and we do two clock_gettime calls to get it.
Replace it with blockStamp which is going to be used in the future for
transaction retransmissions.
It allows to improve single-node TPS by another 3%.
2020-09-09 20:46:31 +03:00
Roman Khimov
bc31ab3d2c
storage: add bloom filter to leveldb
...
We're constantly checking for transactions there and most of the time this
check is not successful (meaning that the transaction in question is
new). Bloom filter easily reduces the need to search over the DB in 99% of
these cases and gives some 13% increase in single-node TPS.
2020-09-09 20:46:31 +03:00
Roman Khimov
0ea8c8ba67
mempool: drop a level of indirection
...
`item` is so small that it makes no sense bothering memory allocator with
every instance of it.
2020-09-09 20:46:31 +03:00
Roman Khimov
53c014a0bb
crypto/consensus: sign hashes and cache them for consensus payloads
...
Avoid serializing payload again and again for various purposes. To sign it, we
only need a hash.
Some 2.4% gain in TPS could be achieved with this.
2020-09-09 20:46:31 +03:00
Roman Khimov
49e9c1aa0f
Merge pull request #1387 from nspcc-dev/rpc/notifications_state_serialisation
...
rpc: replace result.ApplicationLog with state.AppExecResult
2020-09-07 17:48:08 +03:00
Anna Shaleva
acacac1b24
rpc: use state.AppExecResult for ApplicationLog marshalling
...
Closes #1371
2020-09-07 13:38:32 +03:00
Evgenii Stratonikov
230700ee81
core: do not persist Policy contract
...
It has nothing to persist.
2020-09-05 10:54:47 +03:00
Evgenii Stratonikov
1788cf02a1
core: fix VerifyTX test
...
HighPriority attribute was lost during intermediate code changes.
2020-08-27 18:40:37 +03:00
Evgenii Stratonikov
dce456f77f
native: fix a bug in GetCommitteeMembers
...
Return standby committee instead of validators.
2020-08-27 18:40:37 +03:00
Roman Khimov
9e0c13b69d
Merge pull request #1362 from nspcc-dev/feature/verification
...
Allow to use account with contracts on client
2020-08-27 17:30:29 +03:00
Evgenii Stratonikov
9c7168e4e8
rpc/client: allow to use contract accounts in AddNetworkFee
2020-08-27 11:32:55 +03:00
Evgenii Stratonikov
f6319f80e8
interop: allow to call CheckWitness without AllowStates
...
State access is needed only in specific circumstances.
2020-08-27 10:29:29 +03:00
Anna Shaleva
fdf73597af
core: return error for unknown storage configuration
2020-08-25 16:11:44 +03:00
Evgenii Stratonikov
4782a94200
native: do not rewrite NextValidators in (*NEO).OnPersist()
...
There is no need in writing the same value again and again.
Closes #1337 .
2020-08-24 16:45:09 +03:00
Roman Khimov
6c27c2c13a
Merge pull request #1350 from nspcc-dev/some-post-preview3-core-changes
...
Some post-preview3 core changes
2020-08-24 16:26:12 +03:00
Roman Khimov
9c72ea1d64
core/interop: add base58 encoding/decoding syscalls
...
Follow neo-project/neo#1833 .
2020-08-23 17:19:56 +03:00
Roman Khimov
7cba3a0342
core: require AllowStates flag for IsStandard
...
Follow neo-project/neo#1830 .
2020-08-23 15:41:55 +03:00
Evgenii Stratonikov
7ee1ddff61
transaction: add tests for (*Transaction).isValid()
2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
890c752b3b
mempool: add tests for items ordering
2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
2661ebd295
transaction: add HighPriority attribute
...
HighPriority attributes specifies that transaction was
signed by a committee.
2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
e0104679b5
core/tests: make verityTx test more verbose
2020-08-23 09:39:46 +03:00
Roman Khimov
e5813ae8cd
Merge pull request #1346 from nspcc-dev/string-tryboolean
...
Fail converting long strings to boolean
2020-08-22 17:04:11 +03:00
Evgenii Stratonikov
7e34072519
core: implement (*Blockchain).VerifyWitness
...
`ScriptFromWitness` is no longer useful, because we support
contract verification.
2020-08-22 12:45:20 +03:00
Roman Khimov
93f51f922a
stackitem: return error in TryBytes() for big byte strings
...
Follow neo-project/neo-vm#349 .
2020-08-21 21:05:47 +03:00
Roman Khimov
a7670303e8
stackitem: change Bool() to TryBool(), prepare for its failures
2020-08-21 20:55:20 +03:00
Roman Khimov
0e086d61ac
mempool: store feeSum as big.Int
...
Prevent (very) potential overflow.
2020-08-20 19:06:59 +03:00
Roman Khimov
55b2cbb74d
core: refactor and improve verification and pooling
...
Now we have VerifyTx() and PoolTx() APIs that either verify transaction in
isolation or verify it against the mempool (either the primary one or the one
given) and then add it there. There is no possibility to check against the
mempool, but not add a transaction to it, but I doubt we really need it.
It allows to remove some duplication between old PoolTx and verifyTx where
they both tried to check transaction against mempool (verifying first and then
adding it). It also saves us utility token balance check because it's done by
the mempool anyway and we no longer need to do that explicitly in verifyTx.
It makes AddBlock() and verifyBlock() transaction's checks more correct,
because previously they could miss that even though sender S has enough
balance to pay for A, B or C, he can't pay for all of them.
Caveats:
* consensus is running concurrently to other processes, so things could
change while verifyBlock() is iterating over transactions, this will be
mitigated in subsequent commits
Improves TPS value for single node by at least 11%.
Fixes #667 , fixes #668 .
2020-08-20 18:50:18 +03:00
Roman Khimov
0d8cc437fe
mempool: swap checks in Add, fail fast
...
Checking for duplicates is easier than checking the balance, so it should be
done first.
2020-08-20 18:50:18 +03:00
Roman Khimov
e998c102ca
mempool: rename NewMemPool into New
...
"mempool" is a package name already.
2020-08-20 18:50:18 +03:00
Roman Khimov
c7032022f8
core: don't search through the whole DAO in isTxStillRelevant
...
New transactions are added to the chain with blocks. If there is no
transaction X at height N in DAO, it could only be added with block N+1, so
it has to be present there. Therefore we can replace `dao.HasTransaction()`
check with a search through in-block transactions. HasTransaction() is nasty
in that it may add useless load the DB and this code is being run with a big
Blockchain lock held, so we don't want to be delayed here at all.
Improves single-node TPS by ~2%.
2020-08-20 18:50:18 +03:00
