TrueCloudLab/neo-go
8
0
Fork 0
mirror of https://github.com/nspcc-dev/neo-go.git synced 2024-12-12 11:10:35 +00:00
Code Issues Projects Releases Packages Wiki Activity
6386 commits 40 branches 138 tags 45 MiB
Commit graph

2015 commits

Author SHA1 Message Date
Evgenii Stratonikov
e4c3339c91 util: move Fixed8 to encoding/fixedn package 2020-12-09 11:18:18 +03:00
Evgenii Stratonikov
b7e86fa6a3 manifest: add Safe flag to method descriptor 
`interop.Contex.AddMethod` sets `Safe` flag for native
contracts. This allows not to forget to change manifest
when changing call flags.
Also fixed invalid `Safe` flags for `Notary` and `Designate` contracts.
 2020-12-08 13:27:43 +03:00
Anna Shaleva
74a143cee3 core: allow to change deposit's till for owner only 2020-12-08 10:39:20 +03:00
Evgenii Stratonikov
cd5219086a wallet: export NewAccountFromPrivateKey() 
Don't perform back-and-forth conversion, don't handle error
which never occur.
 2020-12-04 12:45:53 +03:00
Evgenii Stratonikov
6e749f4977 core: add tests for crypto interops 
Also move related test from `core/`.
 2020-12-03 14:14:35 +03:00
Evgenii Stratonikov
5bd8ca9597 core/tests: extend test suite 2020-12-03 14:02:58 +03:00
Evgenii Stratonikov
dcc58b7c44 core: validate manifest before incrementing ID in Contract.Create 
Also add more tests.
 2020-12-02 15:54:03 +03:00
Evgenii Stratonikov
b203c23515 core: cover enumerator/iterator interops 2020-12-02 15:54:03 +03:00
Evgenii Stratonikov
d136569ac8 core: move System.Binary.* interops to binary/ package 
Also extend test suite.
 2020-12-02 15:54:03 +03:00
Evgenii Stratonikov
2eb256014e core: move System.Runtime.* interops to runtime/ package 
Also extend test suite.
 2020-12-02 15:54:03 +03:00
Roman Khimov
9211cb636c
Merge pull request #1555 from nspcc-dev/constant-contract-hashes 
Constant contract hashes
 2020-12-01 14:12:26 +03:00
Evgenii Stratonikov
1db27c004e native: do not fail on missing request in Oracle.PostPersist() 2020-12-01 12:29:18 +03:00
Evgenii Stratonikov
cba117352c mempool: correctly handle tx with oracle response 
If tx with the same oracle response ID is already in mempool,
replace it if network fee of added transaction is higher and
return error otherwise.
 2020-12-01 12:29:18 +03:00
Evgenii Stratonikov
3085710e9b native: call onPayment only during transfer 
OnPayment method should be called during GAS distribution
and NEO transfer.
 2020-11-30 13:05:44 +03:00
Roman Khimov
d93aa745bb contract: avoid going to the DB for entry scripts 
This optimizes out DB access for non-deployed contracts under the assumption
that deployed ones are always loaded via `LoadScriptWithHash` (and if they're
not --- it's a bug anyway with the new hashing model) which actually is a very
popular case (every entry script does that).
 2020-11-27 21:47:08 +03:00
Roman Khimov
49f6b33eae core: fix contract-based verification script hash 
When using contract-based verification it's important to load contract's hash
along with the script, otherwise it won't be valid.

Simplify things along the way.
 2020-11-27 21:47:08 +03:00
Roman Khimov
1cf1fe5d74 *: introduce stable contract hashes 
Follow neo-project/neo#2044.
 2020-11-27 21:47:08 +03:00
Roman Khimov
7044e9be40
Merge pull request #1549 from nspcc-dev/core/fix_committee_update 
core: update committee every [committee length] blocks
 2020-11-27 11:04:40 +03:00
Roman Khimov
b92ea2a48a manifest/compiler: drop hashes from ABI and debug info 
See neo-project/neo-devpack-dotnet#391 and neo-project/neo#2044.
 2020-11-26 18:30:49 +03:00
Roman Khimov
586dedecad
Merge pull request #1567 from nspcc-dev/tests/stillrelevant 
core: add tests for `Blockchain.isTxStillRelevant()`
 2020-11-26 17:28:21 +03:00
Evgenii Stratonikov
9f26d7b249 core: add tests for Blockchain.isTxStillRelevant() 2020-11-26 16:04:43 +03:00
Roman Khimov
f8957f10d7
Merge pull request #1566 from nspcc-dev/contractjson 
rpc: marshal fees and GAS as Fixed8 decimal
 2020-11-26 15:12:35 +03:00
Evgenii Stratonikov
a79b12b4d4 rpc: marshal fees and GAS as Fixed8 decimal 2020-11-26 15:04:30 +03:00
Evgenii Stratonikov
3a986d8635 core: add test for RemoveUntraceableBlocks setting 2020-11-26 12:33:34 +03:00
Roman Khimov
882c214646
Merge pull request #1561 from nspcc-dev/removeold 
core: remove old blocks and transactions
 2020-11-25 19:14:32 +03:00
Anna Shaleva
9faa63453c core: refactore some tests 
They have a lot of common code.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
17842dabd6 core: implement native Notary contract 2020-11-25 18:37:29 +03:00
Anna Shaleva
c013522296 core: remove native contracts' name method 
We have `name` in contract manifest.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
0f68528095 core: add callback to VM context 2020-11-25 18:37:29 +03:00
Anna Shaleva
c9acc43023 core: invoke contract verification script with AllowStates flag 
We should call contract's `verify` with AllowStates flag.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
eca27055b8 core: fix NEP17 Transfer event 
`Transfer` event declaration was placed at the wrong part of
`newNEP17Native`, that's why it had incorrect parameters.

Fixed.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
2fee69f26f core: add missing onPersist and postPersist methods to natives 
Although not every contract is persisted (see
https://github.com/neo-project/neo/blob/master/src/neo/Ledger/Blockchain.cs#L94)
we have to add `onPersist` and `postPersist` methods to every
native contract in order to display them in manifest for users and
follow C# behaviour. In C# there are `onPersist` and `postPersist`
methods in base native contract class, see
https://github.com/neo-project/neo/blob/master/src/neo/SmartContract/Native/NativeContract.cs#L141
and
https://github.com/neo-project/neo/blob/master/src/neo/SmartContract/Native/NativeContract.cs#L148
 2020-11-25 18:37:29 +03:00
Anna Shaleva
97069a05d5 core: fix locking in storeBlock 2020-11-25 18:37:29 +03:00
Anna Shaleva
52cf328296 core: add NotaryAssisted transaction attribute 2020-11-25 18:37:29 +03:00
Anna Shaleva
fc9f0034c9 core: adjust verifyTxAttributes for HighPriority attributes 
The fact that they have hight priority does not mean that cheks for other
attributes should be skipped.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
8cdf2d3464 core: unify verifyTxAttributes errors 
We already have pretty ErrInvalidAttribute error, so I think that all
other `verifyTxAttributes` errors should be wrappers around ErrInvalidAttr.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
31aa66a4a4 core: check the length of NotValidBefore attr while decoding 
DecodeBinary throws panic otherwise.
 2020-11-25 18:37:29 +03:00
Anna Shaleva
8548786444 core: do not rewrite binreader error for bad Conflicts attr 
We should keep the original BinReader error untouched in case if
it is exists.
 2020-11-25 18:37:24 +03:00
Evgenii Stratonikov
28b4d4e2f8 core: remove old blocks and transactions 
Remove blocks with `height <= current height - MaxTraceableBlocks`
together with transactions.
 2020-11-25 16:38:20 +03:00
Evgenii Stratonikov
54b177cf40 dao: store blocks/txransactions by big-endian hash 
There is no need for additional allocations.
 2020-11-25 14:03:43 +03:00
Anna Shaleva
b00eb51c55 core: add P2PNotary designated role 2020-11-24 18:47:09 +03:00
Evgenii Stratonikov
67f26859a8 scripts: implement script for creating dumps 
This is useful for creating dumps providing various load
to benchmark restore or check compatibility with C# nodes.

Related #1472.
 2020-11-24 16:47:33 +03:00
Evgenii Stratonikov
6f7284906a blockchainer: allow to dump/restore chain 2020-11-24 16:43:11 +03:00
Evgenii Stratonikov
966b50f2ae core: save StateRootInHeader in genesis block 2020-11-24 16:39:56 +03:00
Evgenii Stratonikov
7d91a3a89e pkg: move internal/ package to the root directory 
This way we can use it in scripts and cli.
 2020-11-24 16:39:56 +03:00
Evgenii Stratonikov
31eca342eb *: replace all NEP5 occurences to NEP17 2020-11-24 13:08:24 +03:00
Evgenii Stratonikov
b97dfae8d8 native: replace NEP-5 with NEP-17 2020-11-24 13:08:23 +03:00
Evgenii Stratonikov
a5914f89fa core: allow to provide block in GetTestVM() 
Sometimes amount of GAS consumed depends on block height.
 2020-11-24 12:17:29 +03:00
Evgenii Stratonikov
42ae226f9e native: use proper stack for result 
When native method calls other contract result should be put
on the stack of current context. With oracles this problem wasn't
noticed because of void return type.
 2020-11-24 12:17:28 +03:00
Evgenii Stratonikov
c849176be7 manifest: include contract Name 2020-11-24 11:23:44 +03:00
Evgenii Stratonikov
e38e8aa48a rpc: implement getproof RPC 2020-11-20 18:06:22 +03:00
Evgenii Stratonikov
1869d6d460 core: allow to use state root in header 2020-11-20 17:16:32 +03:00
Evgenii Stratonikov
85f927d892 mpt: implement reference counting 
Also postpone MPT initialization until `storeBlock`
because we need to read-and-check or save info about refcounting
depending on starting height.
 2020-11-20 16:50:30 +03:00
Evgenii Stratonikov
1c559634aa mpt: fill cached fields when getting node from store 
Node which has been got from store shouldn't be flushed again.
 2020-11-19 12:25:43 +03:00
Evgenii Stratonikov
fd9ff4102a mpt: export func for decoding node with type 
`NodeObject` can contain auxilliary fields and shouldn't be used from outside.
 2020-11-19 12:25:43 +03:00
Roman Khimov
2f824c590a
Merge pull request #1556 from nspcc-dev/allow-null-oracle-filters 
Allow null oracle filters
 2020-11-19 10:30:22 +03:00
Roman Khimov
eef921b8e0 native: allow NULL filter in oracle requests 
Follow neo-project/neo#2067
 2020-11-18 23:59:13 +03:00
Anna Shaleva
ec8ebc292c core: restrict notification size for System.Runtime.Notify 2020-11-17 16:27:51 +03:00
Anna Shaleva
933bb4ca75 core: update committee every [committee length] blocks 
Was changed in https://github.com/neo-project/neo/pull/1848. Affects
storage dumps.
 2020-11-17 11:22:32 +03:00
Anna Shaleva
054e60c0cf core: remove NodeList from Oracle storage 
The reference implementation does not store NodeList. This commit fixes
state difference for genesis block.
 2020-11-16 19:09:34 +03:00
Roman Khimov
88cee80a58
Merge pull request #1543 from nspcc-dev/compatibility-fixes 
rpc, cli: tiny compatibility fixes
 2020-11-16 15:53:42 +03:00
Anna Shaleva
083879838c rpc: adjust getrawtransaction and gettransactionheight RPC call 
We should return verbose transaction in case if it is in the mempool
from `getrawtransaction`. We also shouldn't return height from
`gettransactionheight` in case if transaction is in the mempool.
 2020-11-16 13:25:42 +03:00
Roman Khimov
3cb945f022 manifest: simplify marshaling 2020-11-13 21:46:26 +03:00
Roman Khimov
286d9185f4 smartcontract: remove contract features 
We're featureless now, all contracts have access to storage and payable status
is to be determined via new NEP. Follow neo-project/neo#2060.
 2020-11-13 21:26:23 +03:00
Evgenii Stratonikov
adf403666f native: decode storage item in Seek 2020-11-13 17:33:00 +03:00
Roman Khimov
0f827ee6ba
Merge pull request #1531 from nspcc-dev/core/applicationlog_with_multiple_triggers 
core, rpc: store multiple execution results for single hash
 2020-11-12 19:05:22 +03:00
Anna Shaleva
d6992cb5c4 core: marshal block hash for AppExecResult [NotificationsSubsystem] 2020-11-12 17:43:20 +03:00
Anna Shaleva
7ca93e76ac core, rpc: allow to store several AppExecResult for a single hash 
It is required for we have several executions per block.
 2020-11-12 16:24:39 +03:00
Roman Khimov
9142906abe
Merge pull request #1539 from nspcc-dev/core/fix_mempool_test 
core: fix failing mempool test
 2020-11-12 15:48:28 +03:00
Roman Khimov
11a224057b
Merge pull request #1536 from nspcc-dev/feature/retransmit3 
network: retransmit stale transactions (master)
 2020-11-12 15:47:00 +03:00
Anna Shaleva
a84e4c1e89 core: make some checks in TestMempoolAddRemoveConflicts non-critical 
It will help to investigate test failures.
 2020-11-12 15:14:32 +03:00
Anna Shaleva
54e6bcad12 core: fix failing mempool test 
There might be a case when identical nonces are generated for tx6, tx7 or
tx8 (they are not in mempool, so each of them pass mempool-presence
check). In this case test fails due to the lack of hashes into mp.conflicts
map (two of tx6, tx7 or tx8 have identical hashes) with the following
error:

```
=== RUN   TestMempoolAddRemoveConflicts
--- FAIL: TestMempoolAddRemoveConflicts (0.00s)
    mem_pool_test.go:376:
        	Error Trace:	mem_pool_test.go:376
        	Error:      	Not equal:
        	            	expected: 4
        	            	actual  : 3
        	Test:       	TestMempoolAddRemoveConflicts
```

Fixed by maling the nonce non-random.
 2020-11-12 15:05:01 +03:00
Anna Shaleva
b51eb7f5ef core: update opcodes prices 
Changes ported from https://github.com/neo-project/neo/pull/2020.
 2020-11-12 14:20:26 +03:00
Evgenii Stratonikov
3e5b84348d network: retransmit stale transactions 2020-11-12 13:51:44 +03:00
Anna Shaleva
01296bda5f core: remove policy default values initialisation 2020-11-10 18:05:40 +03:00
Roman Khimov
62d6f3ba22
Merge pull request #1530 from nspcc-dev/syscall/atoi 
core: implement `System.Binary.Atoi/Itoa` syscalls
 2020-11-10 16:39:02 +03:00
Evgenii Stratonikov
e4bf531e3e core: implement System.Binary.Atoi/Itoa syscalls 
They follow C# conversion rules, but differ from our `bigint` module
conversions:
1. String must be big-endian.
2. Sign extension is 4-bit in size (single hex character)
   and not 8-byte.
 2020-11-10 16:15:10 +03:00
Evgenii Stratonikov
54992ad4f3 core: provide account in calculate claimable 
`getunclaimedgas` RPC should return all GAS available to claim.
 2020-11-10 16:08:21 +03:00
Evgenii Stratonikov
860c146260 native: increase committee GAS bounty 
Follow neo-project/neo#2049 .
 2020-11-10 15:30:31 +03:00
Evgenii Stratonikov
af583c14ea native: add GAS rewards for voters 
Close #1348.
 2020-11-10 15:30:30 +03:00
Evgenii Stratonikov
a3c7130ab2 native: cache committee together with votes 2020-11-10 15:23:06 +03:00
Roman Khimov
ede2b05f29
Merge pull request #1529 from nspcc-dev/fix/nextconsensus 
consensus: update NextConsensus only when committee is recalculated
 2020-11-10 10:24:35 +03:00
Evgenii Stratonikov
c30d891aa9 consensus: update NextConsensus only when committee is recalculated 2020-11-09 17:35:32 +03:00
Roman Khimov
a1ce3d2db2 transaction: add new oracle response codes 
Follow neo-project/neo#2037.
 2020-11-09 16:54:09 +03:00
Roman Khimov
bdd073aad7
Merge pull request #1520 from nspcc-dev/tune-some-limits 
Tune some limits
 2020-11-06 15:35:44 +03:00
Roman Khimov
0ee377792e
Merge pull request #1519 from nspcc-dev/designate-role-history 
native: implement designate contract history retention
 2020-11-06 15:35:18 +03:00
Roman Khimov
47421f23f4 native: add events to oracle contract 
Follow neo-project/neo#2036.
 2020-11-06 13:49:07 +03:00
Roman Khimov
9e781bff47 native: implement designate contract history retention 
Follow neo-project/neo#2007. Fix getDesignatedByRole price along the way.
 2020-11-06 13:46:40 +03:00
Roman Khimov
b233158c9f transaction: lower MaxValidUntilBlockIncrement 
Follow neo-project/neo#2042.
 2020-11-06 13:41:46 +03:00
Roman Khimov
b327308df8 core/config: move MaxTraceableBlocks to configuration file 
Follow neo-project/neo#2042, use 2102400 setting for mainnet/testnet and
200000 for private networks.
 2020-11-06 13:41:46 +03:00
Roman Khimov
39a38dc5f5 transaction: raise max oracle response size 
Follow neo-project/neo#1984.
 2020-11-06 13:41:46 +03:00
Roman Khimov
679846c1a1
Merge pull request #1426 from nspcc-dev/rpc/getcontractstate_by_id_or_name 
rpc: allow to `getcontractstate` by id or name
 2020-11-05 13:17:42 +03:00
Roman Khimov
d7502debba core: adjust storage update fees 
Follow neo-project/neo#2033.
 2020-11-03 22:50:40 +03:00
Anna Shaleva
15a939b1da rpc: allow to getcontractstate by address, id or name 
close #1423
 2020-11-03 17:23:49 +03:00
Roman Khimov
eaa260474f trigger/core: split System trigger into OnPerist and PostPersist 
Follow neo-project/neo#2022.
 2020-10-29 19:17:07 +03:00
Roman Khimov
044786b995 core: use Application trigger in CalculateClaimable() 
It doesn't need a System one.
 2020-10-29 19:17:07 +03:00
Roman Khimov
2924ecd453 core: use ic.SpawnVM() in tests, not vm.New() 
vm.New() sets some arbitrary trigger while interop context always sets
something appropriate according to its state.
 2020-10-29 19:17:07 +03:00
Roman Khimov
d4da811d12
Merge pull request #1507 from nspcc-dev/conflicts_attr 
core: implement Conflicts transaction attribute
 2020-10-29 16:54:58 +03:00
Anna Shaleva
ec63d5c456 core: add conflicts attribute 
Close #1491
 2020-10-29 10:57:31 +03:00
Roman Khimov
0eb4ad8564
Merge pull request #1508 from nspcc-dev/core/check_witness_tests 
core: add CheckWitness tests
 2020-10-27 15:56:35 +03:00
Roman Khimov
c4a4a84515 native: lower voting fee 
Follow neo-project/neo#2010.
 2020-10-26 22:12:56 +03:00
Anna Shaleva
5a15e6b54c core: add tests for CheckWitness 
Close #1366
 2020-10-26 20:30:45 +03:00
Anna Shaleva
204d7f1c6a core: refactor runtime.CheckWitness interop 
Removed unreachable code, see
8fed383523

runtime.CheckHashedWitness can only be used for transaction
verification, the other two options from reference implementation (block
and consensus payload) have separate methods for verification.
 2020-10-26 20:26:21 +03:00
Roman Khimov
eca1221d22
Merge pull request #1504 from nspcc-dev/core/native_serialisation 
core: native serialisation
 2020-10-23 18:38:35 +03:00
Anna Shaleva
f259a614de core: add transaction.HasSigner method 2020-10-23 16:32:26 +03:00
Anna Shaleva
0232bbcb0c core: refactor GASRecord handling 
We should store each GAS record as
{
    Key: ContractID + RecordsPrefix + RecordID,
    Value: RecordValue,
}
So don't use state.GASRecord for storing anymore. However, it's still
useful to cache GasRecord values by using state.GASRecords, because we
have to keep GASIndexPairs sorted by indexes.
 2020-10-23 12:49:12 +03:00
Anna Shaleva
0da01fde7f core: refactor blocked accounts logic 2020-10-23 11:42:30 +03:00
Roman Khimov
1956f2c079
Merge pull request #1496 from nspcc-dev/signatures-collection 
core: implement NotValidBefore and Reserved tx attributes
 2020-10-23 11:26:49 +03:00
Anna Shaleva
7947e99a1e core: add transaction.GetAttributes 2020-10-23 11:05:03 +03:00
Anna Shaleva
95630b72e8 core: verify transaction attributes before adding it into mempool 
Attributes check should be done before adding transaction to
the pool, otherwise there might be a case when transaction with invalid
attributes is in the pool.
 2020-10-23 11:05:03 +03:00
Anna Shaleva
a6579a05ac core: refactor transaction.Attribute unmarshalling 2020-10-23 11:05:03 +03:00
Anna Shaleva
09b8b8de73 core: reserve attributes range for experimantal purposes 2020-10-23 11:04:59 +03:00
Anna Shaleva
368ff820b3 core: add NotValidBefore transaction attribute 
Close #1490
 2020-10-23 11:02:46 +03:00
Anna Shaleva
6685f8eba9 core: restrict the maximum number of requests per URL 2020-10-22 11:32:02 +03:00
Anna Shaleva
7c232e2ddc core: add max nodes count restriction to designate contract 2020-10-22 11:32:02 +03:00
Anna Shaleva
64d1946fbb rpc: fix Matches and marshalling for notification events 
Close #1494. Marshalling went wrong due to the incorrect pointers usage.
Reproduced and fixed.
 2020-10-19 13:44:20 +03:00
Roman Khimov
99e0e346c6
Merge pull request #1476 from nspcc-dev/fix/verify 
core: allow to invoke `verify` of native contracts
 2020-10-14 12:49:01 +03:00
Roman Khimov
7808ab2dde
Merge pull request #1479 from nspcc-dev/examples/deploy 
examples: add _deploy usage examples
 2020-10-14 12:32:21 +03:00
Evgenii Stratonikov
aee02ff2e3 core: allow to invoke verify of native contracts 2020-10-14 11:44:27 +03:00
Anna Shaleva
fe1f0a7245 core: introduce CheckReturnState constants 
At the moment we should have 3 possible options to check return state
during vm context unloading:
	* no check
	* ensure the stack is empty
	* ensure the stack is not empty

It is necessary to distinguish them because new _deploy method shouldn't
left anything on stack. Example: if we use _deploy method before some
ordinary contract method which returns one value. Without these changes
the contract invocation will fail due to 2 elements on stack left after
invocation (the first `null` element is from _deploy, the second element
is return-value from the ordinary contract method).
 2020-10-13 19:14:44 +03:00
Roman Khimov
0f39da19d0 storage: fix Get for BoltDB, fix #1482 2020-10-13 19:05:11 +03:00
Roman Khimov
c72ecd1be4
Merge pull request #1480 from nspcc-dev/fix-reverification-of-stale-txes 
core: drop stale transactions from the mempool
 2020-10-12 15:54:28 +03:00
Roman Khimov
a154481860 core: drop stale transactions from the mempool 
They're no longer valid if `ValidUntilBlock == curheight`.
 2020-10-12 14:58:40 +03:00
Anna Shaleva
097ef2a731 core: copy storage item key in simple.GetStorageItems 
Related to #1468, ported from #1475.

We should copy the key to avoid bytes substitution. Otherwise there's a
chance that at the end of dao.Store.Seek(...) execution some keys won't
be the same as the original keys found inside saveToMap function because
storage.Seek can guarantee that provided key and value are only valid
until the next `f` call.
 2020-10-10 12:37:46 +03:00
Evgenii Stratonikov
42ff4a8fc7 native: provide name method for all contracts 2020-10-09 15:06:28 +03:00
Roman Khimov
eb9b1cae36
Merge pull request #1474 from nspcc-dev/core/fixes 
Some small improvements
 2020-10-09 10:03:52 +03:00
Roman Khimov
124ce9d247
Merge pull request #1455 from nspcc-dev/get_invocation_counter 
core: remove error from runtime.GetInvocationCounter
 2020-10-08 16:34:40 +03:00
Roman Khimov
8c2fd91c5c
Merge pull request #1462 from nspcc-dev/rpc/exceptions 
rpc, core: add FaultException to AppExecResult and Invoke* results
 2020-10-08 16:20:38 +03:00
Anna Shaleva
0f71088246 core: add FaultException to AppExecResult 2020-10-08 16:12:19 +03:00
Roman Khimov
70e9e329cd
Merge pull request #1463 from nspcc-dev/core/blocked_accounts_cache 
core: add cache to Policy blocked accounts
 2020-10-08 14:35:19 +03:00
Evgenii Stratonikov
037cecf1ac native: move OracleRequest to state package 
It is used by multiple modules outside native
and produces unneeded dependencies.
 2020-10-08 13:48:14 +03:00
Anna Shaleva
9fe2b04db9 core: add cache to Policy blocked accounts 
Close #1441
 2020-10-08 13:47:21 +03:00
Anna Shaleva
6ce00fde82 vm, core: move invocation counter from InteropContext to VM 2020-10-08 11:33:26 +03:00
Anna Shaleva
45bfce60a5 core: remove error from runtime.GetInvocationCounter 
close #1444
 2020-10-08 11:32:00 +03:00
Roman Khimov
705941a800 transaction: add script length limit 
As it is implemented in C# code.
 2020-10-07 18:23:10 +03:00
Roman Khimov
d029f5c0d8 transaction: fix witness script length limits 
See neo-project/neo#1958.
 2020-10-07 18:05:23 +03:00
Evgenii Stratonikov
17922038dc native: add GetOracleInvokeScript() 2020-10-07 10:04:19 +03:00
Evgenii Stratonikov
9733a6f394 core: move CalculateNetworkFee to a separate package 2020-10-07 10:04:19 +03:00
Roman Khimov
d6a1a22afa
Merge pull request #1452 from nspcc-dev/contract/deploy 
Support `_deploy` method
 2020-10-06 19:54:14 +03:00
Evgenii Stratonikov
2d9ef9219a core: call _deploy method during create/update 2020-10-06 18:03:25 +03:00
Evgenii Stratonikov
b71f9e296c core/tests: refactor contract creation a bit 
Make it easier to add new methods.
 2020-10-06 18:03:25 +03:00
Evgenii Stratonikov
b2a3a0851e emit: accept multiple opcodes in Opcode() 2020-10-06 18:03:25 +03:00
Roman Khimov
2de146125c
Merge pull request #1460 from nspcc-dev/contract/update 
core: allow Null in `System.Contract.Update`
 2020-10-06 16:54:21 +03:00
Evgenii Stratonikov
0b76f875c7 core: allow Null in System.Contract.Update 
Null means absense of script or manifest, empty
byte-slice is an error.

Related #1459.
 2020-10-06 10:34:45 +03:00
Roman Khimov
2ea29924c4
Merge pull request #1454 from nspcc-dev/core/policy_checks 
core: add validation to policy methods
 2020-10-05 19:09:59 +03:00
Anna Shaleva
560e470484 core: add validation to native Policy methods 
close #1442
 2020-10-05 18:08:37 +03:00
Anna Shaleva
e1e586f18b core: restrict the muximum number of contents per block 2020-10-05 18:08:33 +03:00
Roman Khimov
0e82d4cbd1
Merge pull request #1458 from nspcc-dev/fix/storageput 
core: calculate gas for `System.Storage.Put` correctly
 2020-10-05 14:26:17 +03:00
Evgenii Stratonikov
833bbb1d35 core: calculate gas for System.Storage.Put correctly 
If `Put` creates new key, its length should contribute
to a GAS cost of the syscall.
 2020-10-05 13:03:44 +03:00
Roman Khimov
8e146d19b3
Merge pull request #1447 from nspcc-dev/core/fix_neo_cache 
core: fix cache for (NEO).nextValidators and (NEO).committee
 2020-10-02 17:09:43 +03:00
Anna Shaleva
eccf639803 core: add InitializeCache method to NEO native contracts 
There might be a case when cached contract values store nil (e.g.
after restoring chain from dump). We should always initialize cached
values irrespective to the (NEO).Initialize method.

This commit fixes a bug introduced in 83e94d3
when 4-nodes privnet is failing after restoring from dump:
```
$ docker logs neo_go_node_one
=> Try to restore blocks before running node
2020-09-30T11:55:49.122Z	INFO	no storage version found! creating genesis block
2020-09-30T11:55:49.124Z	INFO	service hasn't started since it's disabled	{"service": "Pprof"}
2020-09-30T11:55:49.124Z	INFO	service hasn't started since it's disabled	{"service": "Prometheus"}
2020-09-30T11:55:49.124Z	INFO	skipped genesis block	{"hash": "3792eaa22c196399a114666fd491c4b9ac52491d9abb1f633a8036a8ac81e4db"}
2020-09-30T11:55:49.141Z	INFO	shutting down service	{"service": "Pprof", "endpoint": ":30001"}
2020-09-30T11:55:49.141Z	INFO	shutting down service	{"service": "Prometheus", "endpoint": ":40001"}
2020-09-30T11:55:49.141Z	INFO	blockchain persist completed	{"persistedBlocks": 3, "persistedKeys": 146, "headerHeight": 3, "blockHeight": 3, "took": "324.27µs"}
2020-09-30T11:55:49.150Z	INFO	restoring blockchain	{"version": "0.1.0"}
2020-09-30T11:55:49.150Z	INFO	service hasn't started since it's disabled	{"service": "Prometheus"}
2020-09-30T11:55:49.151Z	INFO	service hasn't started since it's disabled	{"service": "Pprof"}
2020-09-30T11:55:49.443Z	INFO	starting rpc-server	{"endpoint": ":30333"}
2020-09-30T11:55:49.443Z	INFO	node started	{"blockHeight": 3, "headerHeight": 3}

    _   ____________        __________
   / | / / ____/ __ \      / ____/ __ \
  /  |/ / __/ / / / /_____/ / __/ / / /
 / /|  / /___/ /_/ /_____/ /_/ / /_/ /
/_/ |_/_____/\____/      \____/\____/

/NEO-GO:/

2020-09-30T11:55:49.444Z	INFO	new peer connected	{"addr": "172.23.0.5:39638", "peerCount": 1}
2020-09-30T11:55:49.444Z	INFO	new peer connected	{"addr": "172.23.0.5:20333", "peerCount": 2}
2020-09-30T11:55:49.444Z	WARN	peer disconnected	{"addr": "172.23.0.5:20333", "reason": "identical node id", "peerCount": 1}
2020-09-30T11:55:49.445Z	WARN	peer disconnected	{"addr": "172.23.0.5:39638", "reason": "identical node id", "peerCount": 0}
2020-09-30T11:55:49.445Z	INFO	new peer connected	{"addr": "172.23.0.3:20335", "peerCount": 1}
2020-09-30T11:55:49.445Z	INFO	new peer connected	{"addr": "172.23.0.2:20334", "peerCount": 2}
2020-09-30T11:55:49.445Z	INFO	started protocol	{"addr": "172.23.0.3:20335", "userAgent": "/NEO-GO:/", "startHeight": 3, "id": 1339919829}
2020-09-30T11:55:49.445Z	INFO	new peer connected	{"addr": "172.23.0.4:20336", "peerCount": 3}
2020-09-30T11:55:49.445Z	INFO	started protocol	{"addr": "172.23.0.4:20336", "userAgent": "/NEO-GO:/", "startHeight": 3, "id": 4036722359}
2020-09-30T11:55:49.445Z	INFO	node reached synchronized state, starting consensus
2020-09-30T11:55:49.445Z	INFO	started protocol	{"addr": "172.23.0.2:20334", "userAgent": "/NEO-GO:/", "startHeight": 3, "id": 1557367037}
panic: runtime error: integer divide by zero

goroutine 132 [running]:
github.com/nspcc-dev/dbft.(*Context).GetPrimaryIndex(...)
	github.com/nspcc-dev/dbft@v0.0.0-20200925163137-8f3b9ab3b720/context.go:83
github.com/nspcc-dev/dbft.(*Context).reset(0xc0000e0780, 0x0)
	github.com/nspcc-dev/dbft@v0.0.0-20200925163137-8f3b9ab3b720/context.go:208 +0x64b
github.com/nspcc-dev/dbft.(*DBFT).InitializeConsensus(0xc0000e0780, 0x964800)
	github.com/nspcc-dev/dbft@v0.0.0-20200925163137-8f3b9ab3b720/dbft.go:87 +0x51
github.com/nspcc-dev/dbft.(*DBFT).Start(0xc0000e0780)
	github.com/nspcc-dev/dbft@v0.0.0-20200925163137-8f3b9ab3b720/dbft.go:81 +0x4b
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).Start(0xc0001a2160)
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:206 +0x56
github.com/nspcc-dev/neo-go/pkg/network.(*Server).tryStartConsensus(0xc0000ec500)
	github.com/nspcc-dev/neo-go/pkg/network/server.go:311 +0xda
github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ec500, 0x104d800, 0xc000222090, 0xc0000a6f10, 0x0, 0x0)
	github.com/nspcc-dev/neo-go/pkg/network/server.go:781 +0xa7a
github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc000222090)
	github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:162 +0x2e7
created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
	github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:40 +0x1ac
```
 2020-10-02 17:03:06 +03:00
Roman Khimov
15e459feea core: fix designate test 
Merge conflict between #1450 and #1451.
 2020-10-02 16:23:48 +03:00
Roman Khimov
b5d138b150
Merge pull request #1451 from nspcc-dev/native/designative 
native: implement Designate contract
 2020-10-02 16:09:33 +03:00
Anna Shaleva
6b201893fa core: unexport NewNEO() and NewGAS() methods 2020-10-02 15:45:49 +03:00
Evgenii Stratonikov
c468c02ef5 native: implement Designate contract 2020-10-02 11:03:25 +03:00
Roman Khimov
991089593f
Merge pull request #1450 from nspcc-dev/fix/witnessscope 
transaction: rename FeeOnly to None
 2020-10-01 16:46:07 +03:00
Evgenii Stratonikov
fa09b9af7b transaction: rename FeeOnly to None 
Follow missed change from neo-project/neo#1816 .
`None` may be used for any signer. Currently it is used
for sender to only pay fees, or to sign tx attributes.
 2020-10-01 15:28:19 +03:00
Roman Khimov
08aa773dac
Merge pull request #1446 from nspcc-dev/core/network_fee_fixes 
core: fix CalculateNetworkFee and verifyTxWitnesses
 2020-09-30 21:46:33 +03:00
Anna Shaleva
e34e367a7b core: take into account size fee during verifyTxWitnesses 
GasLimit for transaction scripts verification should not include fee for
transaction size.
 2020-09-30 21:40:04 +03:00
Anna Shaleva
2b11e99225 core: fix CalculateNetworkFee for signature contracts 
First PUSHDATA1 is from invocation script, the second PUSHDATA1 is
from verification script. E.g.:

Invocation script:

INDEX    OPCODE       PARAMETER
0        PUSHDATA1    035913b9588da23a5c3ce14b2886a6b8ebb6a0eb92bdaa948510dfb5ae5194d6cb    <<
35       PUSHNULL
36       SYSCALL      Neo.Crypto.VerifyWithECDsaSecp256r1 (95440d78)

Verification script:

INDEX    OPCODE       PARAMETER
0        PUSHDATA1    3930fe5a9b44682f37741955df4a5f2585ed5aa438fa6e17ae51083673b1d64253e5a859c0cf168be67971e53a23c1c40582777d94a8e391db23ff613849627d    <<
 2020-09-30 21:31:43 +03:00
Roman Khimov
5365da9860 native: remove unregistered candidate with 0 votes on NEO transfer 
See neo-project/neo#1935.
 2020-09-29 22:38:38 +03:00
Roman Khimov
adcbb2287f
Merge pull request #1439 from nspcc-dev/core/verify_tx_witnesses_fix 
core: take into account gasConsumed during tx witnesses verification
 2020-09-29 18:11:39 +03:00
Anna Shaleva
ff6aa0fd45 core: allow empty callingScriptHash during CheckWitness 
CallingScriptHash can be empty in the entry transaction script.
 2020-09-29 17:43:39 +03:00
Anna Shaleva
a2bfd16136 core: take into account gasConsumed during tx witnesses verification 
We should pay attention to the previously consumed gas during tx
witnesses verification.
 2020-09-29 17:43:35 +03:00
Roman Khimov
c21f699ffc
Merge pull request #1432 from nspcc-dev/fix/postpersist 
Initialize cache in native contracts properly
 2020-09-29 14:21:15 +03:00
Roman Khimov
e4be60c3a8
Merge pull request #1435 from nspcc-dev/checkwitness-calling-script-hash 
interop/runtime: allow calling script hash to pass CheckWitness
 2020-09-29 12:49:09 +03:00
Evgenii Stratonikov
26dc807a2d native: initialize cache in OnPersist 
If node started from non-zero height, `Initialize` is
not called thus cache is empty. Thus we should initialize
it before any tx is executed.
 2020-09-29 10:17:47 +03:00
Roman Khimov
53c9690bdc interop/runtime: allow calling script hash to pass CheckWitness 
See neo-project/neo#1924 and neo-project/neo#1925.
 2020-09-29 09:56:19 +03:00
Evgenii Stratonikov
05118e96f4 native: store cache during Oracle contract creation 
`Initialize` is not called during restore from dump.
 2020-09-28 15:34:04 +03:00
Evgenii Stratonikov
e0e7fd5367 native: cache GASPerBlock values 
Close #1421.
 2020-09-28 10:53:48 +03:00
Evgenii Stratonikov
e91d13c615 core: implement oracle tx verification 2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
f084acc339 native: cache oracle script hash 2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
bd1790bddf native: cache oracle nodes 2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
141d6e325f native: implement basis for oracle contract 2020-09-25 17:34:11 +03:00
Evgenii Stratonikov
ef7c0dbd78 core: use GetCommitteeAddress to get committee address 2020-09-25 17:02:31 +03:00
Evgenii Stratonikov
5b205ffa7d native: cache committee script hash 2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
ca1b8a7df0 state: implement marshalers for oracle-related state 2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
30526046e7 native: don't push void result in Call 
This was done in https://github.com/neo-project/neo/pull/1693
for native calls. `OnPersist` script still uses `DROP` though
as value is pushed via `CheckReturn` logic for regular calls.
 2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
877b8ece63 core: move contractCall* to a separate package 2020-09-25 16:32:10 +03:00
Evgenii Stratonikov
c5cdaae87a native: support postPersist method 
It should be called for NEO contract to distribute
committee bounties.
 2020-09-23 14:47:09 +03:00
Evgenii Stratonikov
af16519413 native: do not update committee every block 
Update frequency depends on committee size and
amount of validators. For mainnet it is 28.
 2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
81a11c629a native: remove getValidators method 
Follow https://github.com/neo-project/neo/pull/1920 .
 2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
83e94d3bbc native: cache committee members 2020-09-23 14:42:13 +03:00
Evgenii Stratonikov
43b3e15330 native: send GAS to a committee member on persist 2020-09-23 14:42:12 +03:00
Evgenii Stratonikov
5a38208361 native: implement NEO.Get/SetMaxGasPerBlock() 2020-09-23 14:12:42 +03:00
Evgenii Stratonikov
7d90d79ae6 core: update claimable GAS calculation 2020-09-23 14:12:42 +03:00
Roman Khimov
41af738c1b state: drop (*NEP5TransferLog).DecodeBinaryReturnCount 
It's no longer needed.
 2020-09-22 19:21:12 +03:00
Roman Khimov
e4b52d3947 core/rpc: add continue flag to iterating functions 
Most of the time we don't need to get all transfers from the DB and
deserialize them.
 2020-09-21 22:23:34 +03:00
Roman Khimov
373c669c6a core/state: reverse the order of ForEachTransfer 
When using limits we're usually concerned about the most recent
transfers. Returning 3 transfers from the middle of the chain isn't very
helpful.
 2020-09-21 22:05:15 +03:00
Roman Khimov
ff11a5f990 state: use more efficient encoding for amount 
It's variable-length anyway, so wasting 8 bytes for what typically is 1 byte
makes no sense.
 2020-09-21 21:51:33 +03:00
Roman Khimov
806b89db76 state: store the length of NEP5TransferLog in the first byte 
We lose `size` field after serialization/deserialization which can lead to
adding more than NEP5TransferBatchSize elements into the NEP5TransferLog.
 2020-09-21 21:51:11 +03:00
Roman Khimov
0c264b1486 mpt: fix comment typo 2020-09-21 17:54:11 +03:00
Anna Shaleva
770c8d774c core, rpc: add GetCommittee method 
Closes #1414
 2020-09-21 15:56:25 +03:00
Anna Shaleva
c6f099294b core: do not marshal block hash in application log 
Closes #1388
 2020-09-21 14:08:15 +03:00
Roman Khimov
a439941a71
Merge pull request #1407 from nspcc-dev/core/oracleattr 
Implement OracleResponse transaction attribute
 2020-09-21 12:55:10 +03:00
Roman Khimov
1608fbff87
Merge pull request #1378 from nspcc-dev/tests/cli 
Implement tests for CLI
 2020-09-19 17:02:34 +03:00
Evgenii Stratonikov
bff67c921a core: wrap dao in GetTestVM 
MPT must not be shared to the test VM.
Fix DATA RACE between `Collapse()` and RPC calls.
 2020-09-18 12:07:02 +03:00
Roman Khimov
61ad82160b core: remove named return variable from addHeaders() 2020-09-16 17:45:12 +03:00
Roman Khimov
6bffa811d4 core: remove (*block.Block) checks for topBlock 
If it's non-nil, it has *block.Block inside. If it doesn't --- tell everyone
about it with a nice panic message.
 2020-09-16 17:40:27 +03:00
Roman Khimov
33ea179f6e core: remove forward index check from addHeaders() 
It can't ever happen. We're guaranteed to have a consistent chain of headers
(we're verifying them above, if we're not verifying --- it's not our fault)
that starts at HeaderHeight that was actual when we were asking for it
previously. HeaderHeight can only move forward, so if that happened that would
be filtered out by the condition below and the first one can't happen. Though
to be absolutely sure change the second check to only pass "+1" headers (which
is what we want).
 2020-09-16 16:33:42 +03:00
Roman Khimov
7cbb660082 core: rework headerList 
It's a contention point as all accesses to it are serialized and they compete
with persisting logic at the same time.
 2020-09-16 16:30:40 +03:00
Evgenii Stratonikov
1625689316 transaction: implement OracleResponse attribute 2020-09-16 14:50:31 +03:00
Evgenii Stratonikov
06b29e409c transaction: remove Attribute.Data field 2020-09-16 13:39:53 +03:00
Roman Khimov
ce09c82b25 block: remove Verify() 
It's used in two places now:
 * Blockchain.AddBlock()
   This one does transaction duplication check of its own, doing it in
   Verify() is just a waste of time. Merkle tree root hash value check is
   still relevant though
 * Block.DecodeBinary()
   We're decoding blocks for the following purposes:
     - on restore from dump
       The block will be added to the chain via AddBlock() and that will do a
       full check of it (if configured to do so)
     - on retrieving the block from the DB (DAO)
       We trust the DB, if it's gone wild, this check won't really help
     - on receiving the block via P2P
       It's gonna be put into block queue and then end up in AddBlock() which
       will check it
     - on receiving the block via RPC (submitblock)
       It is to be passed into AddBlock()
     - on receiving the block via RPC in a client
       That's the only problematic case probably, but RPC client has to trust
       the server and it can check for the signature if it really
       cares. Or a separate in-client check might be added.

As we can see nothing really requires this verification to be done the way it
is now, AddBlock can just have a Merkle check and DecodeBinary can do fine
without it at all.
 2020-09-16 12:50:13 +03:00
Roman Khimov
2e876b5593 block: remove Base.Verify() 
It's a no-op and there is nothing we can do about it, header contents could
only be checked against chain state, there is nothing to check for internal
consistency.
 2020-09-16 12:50:13 +03:00
Roman Khimov
d52e79668b hash: introduce memory-optimized merkle root hash calculation routine 
NewMerkleTree is a memory hog, we can do better than that:

BenchmarkMerkle/NewMerkleTree-8                       13          88434670 ns/op        20828207 B/op     300035 allocs/op
BenchmarkMerkle/CalcMerkleRoot-8                      15          69264150 ns/op               0 B/op          0 allocs/op
 2020-09-15 18:38:15 +03:00
Roman Khimov
19c69618c5 transaction: cache tx size, don't serialize it over and over again 2020-09-11 18:55:19 +03:00
Roman Khimov
83fc38ae3a mempool: don't create new big.Int in tryAddSendersFee() if possible 
Do a little less allocations.
 2020-09-10 15:35:19 +03:00
Roman Khimov
fc7ea6217d mempool: avoid reassigning utilityBalanceAndFees value 
It's not needed, we're either creating a new one and assigning it 6 lines
above or we're changing already existing big.Int via a pointer, so no update
is needed.
 2020-09-10 15:20:04 +03:00
Roman Khimov
7310e748e3 core: reuse mempool from AddBlock() for bc.isTxStillRelevant() 
It's already there most of the time and creating another slice is just a waste
of time. Checking for presence with map is also a little faster.
 2020-09-10 15:02:03 +03:00
Roman Khimov
26339c75dc vm/core: drop old key caching system 
Obsoleted by f5f58a7e91.
 2020-09-10 14:43:24 +03:00
Roman Khimov
fe1f1d19be mempool: store only pointer in the verifiedMap 
It's only used for presence checks, there is no need for metadata here.
 2020-09-09 20:46:31 +03:00
Roman Khimov
a2d9b89964 dao: reuse buffers when storing blocks, txes and aers 
Reduce memory allocation pressure.
 2020-09-09 20:46:31 +03:00
Roman Khimov
9591d64e53 mempool: don't sort items by hash 
There is nothing requiring us to do so. It also is bad because it allows for
new transaction to replace some already existing one with the same fee
parameters just because it has "better" hash.

But the other thing is that for transactions with equal fees it's always
better for us to append them to the end of the list, instead of inserting them
in the middle, so this change allows to reduce slice item movements and gain
some 6-7% increase for single-node TPS.
 2020-09-09 20:46:31 +03:00
Roman Khimov
5df726db68 mempool: replace timeStamp with blockStamp 
Time is not really relevant for us here and we don't use this timestamp in any
way. Yet it occupies 24 bytes and we do two clock_gettime calls to get it.

Replace it with blockStamp which is going to be used in the future for
transaction retransmissions.

It allows to improve single-node TPS by another 3%.
 2020-09-09 20:46:31 +03:00
Roman Khimov
bc31ab3d2c storage: add bloom filter to leveldb 
We're constantly checking for transactions there and most of the time this
check is not successful (meaning that the transaction in question is
new). Bloom filter easily reduces the need to search over the DB in 99% of
these cases and gives some 13% increase in single-node TPS.
 2020-09-09 20:46:31 +03:00
Roman Khimov
0ea8c8ba67 mempool: drop a level of indirection 
`item` is so small that it makes no sense bothering memory allocator with
every instance of it.
 2020-09-09 20:46:31 +03:00
Roman Khimov
53c014a0bb crypto/consensus: sign hashes and cache them for consensus payloads 
Avoid serializing payload again and again for various purposes. To sign it, we
only need a hash.

Some 2.4% gain in TPS could be achieved with this.
 2020-09-09 20:46:31 +03:00
Roman Khimov
49e9c1aa0f
Merge pull request #1387 from nspcc-dev/rpc/notifications_state_serialisation 
rpc: replace result.ApplicationLog with state.AppExecResult
 2020-09-07 17:48:08 +03:00
Anna Shaleva
acacac1b24 rpc: use state.AppExecResult for ApplicationLog marshalling 
Closes #1371
 2020-09-07 13:38:32 +03:00
Evgenii Stratonikov
230700ee81 core: do not persist Policy contract 
It has nothing to persist.
 2020-09-05 10:54:47 +03:00
Evgenii Stratonikov
1788cf02a1 core: fix VerifyTX test 
HighPriority attribute was lost during intermediate code changes.
 2020-08-27 18:40:37 +03:00
Evgenii Stratonikov
dce456f77f native: fix a bug in GetCommitteeMembers 
Return standby committee instead of validators.
 2020-08-27 18:40:37 +03:00
Roman Khimov
9e0c13b69d
Merge pull request #1362 from nspcc-dev/feature/verification 
Allow to use account with contracts on client
 2020-08-27 17:30:29 +03:00
Evgenii Stratonikov
9c7168e4e8 rpc/client: allow to use contract accounts in AddNetworkFee 2020-08-27 11:32:55 +03:00
Evgenii Stratonikov
f6319f80e8 interop: allow to call CheckWitness without AllowStates 
State access is needed only in specific circumstances.
 2020-08-27 10:29:29 +03:00
Anna Shaleva
fdf73597af core: return error for unknown storage configuration 2020-08-25 16:11:44 +03:00
Evgenii Stratonikov
4782a94200 native: do not rewrite NextValidators in (*NEO).OnPersist() 
There is no need in writing the same value again and again.
Closes #1337.
 2020-08-24 16:45:09 +03:00
Roman Khimov
6c27c2c13a
Merge pull request #1350 from nspcc-dev/some-post-preview3-core-changes 
Some post-preview3 core changes
 2020-08-24 16:26:12 +03:00
Roman Khimov
9c72ea1d64 core/interop: add base58 encoding/decoding syscalls 
Follow neo-project/neo#1833.
 2020-08-23 17:19:56 +03:00
Roman Khimov
7cba3a0342 core: require AllowStates flag for IsStandard 
Follow neo-project/neo#1830.
 2020-08-23 15:41:55 +03:00
Evgenii Stratonikov
7ee1ddff61 transaction: add tests for (*Transaction).isValid() 2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
890c752b3b mempool: add tests for items ordering 2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
2661ebd295 transaction: add HighPriority attribute 
HighPriority attributes specifies that transaction was
signed by a committee.
 2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
e0104679b5 core/tests: make verityTx test more verbose 2020-08-23 09:39:46 +03:00
Roman Khimov
e5813ae8cd
Merge pull request #1346 from nspcc-dev/string-tryboolean 
Fail converting long strings to boolean
 2020-08-22 17:04:11 +03:00
Evgenii Stratonikov
7e34072519 core: implement (*Blockchain).VerifyWitness 
`ScriptFromWitness` is no longer useful, because we support
contract verification.
 2020-08-22 12:45:20 +03:00
Roman Khimov
93f51f922a stackitem: return error in TryBytes() for big byte strings 
Follow neo-project/neo-vm#349.
 2020-08-21 21:05:47 +03:00
Roman Khimov
a7670303e8 stackitem: change Bool() to TryBool(), prepare for its failures 2020-08-21 20:55:20 +03:00
Roman Khimov
0e086d61ac mempool: store feeSum as big.Int 
Prevent (very) potential overflow.
 2020-08-20 19:06:59 +03:00
Roman Khimov
55b2cbb74d core: refactor and improve verification and pooling 
Now we have VerifyTx() and PoolTx() APIs that either verify transaction in
isolation or verify it against the mempool (either the primary one or the one
given) and then add it there. There is no possibility to check against the
mempool, but not add a transaction to it, but I doubt we really need it.

It allows to remove some duplication between old PoolTx and verifyTx where
they both tried to check transaction against mempool (verifying first and then
adding it). It also saves us utility token balance check because it's done by
the mempool anyway and we no longer need to do that explicitly in verifyTx.

It makes AddBlock() and verifyBlock() transaction's checks more correct,
because previously they could miss that even though sender S has enough
balance to pay for A, B or C, he can't pay for all of them.

Caveats:
 * consensus is running concurrently to other processes, so things could
   change while verifyBlock() is iterating over transactions, this will be
   mitigated in subsequent commits

Improves TPS value for single node by at least 11%.

Fixes #667, fixes #668.
 2020-08-20 18:50:18 +03:00
Roman Khimov
0d8cc437fe mempool: swap checks in Add, fail fast 
Checking for duplicates is easier than checking the balance, so it should be
done first.
 2020-08-20 18:50:18 +03:00
Roman Khimov
e998c102ca mempool: rename NewMemPool into New 
"mempool" is a package name already.
 2020-08-20 18:50:18 +03:00
Roman Khimov
c7032022f8 core: don't search through the whole DAO in isTxStillRelevant 
New transactions are added to the chain with blocks. If there is no
transaction X at height N in DAO, it could only be added with block N+1, so
it has to be present there. Therefore we can replace `dao.HasTransaction()`
check with a search through in-block transactions. HasTransaction() is nasty
in that it may add useless load the DB and this code is being run with a big
Blockchain lock held, so we don't want to be delayed here at all.

Improves single-node TPS by ~2%.
 2020-08-20 18:50:18 +03:00
Roman Khimov
3a379e4f3e core: don't reverify mempooled transactions in AddBlock 
The end effect is almost as if `VerifyTransactions: false` was set in the
config, but without actually compromising the guarantees provided by it.

It almost doubles performance for single-mode benchmarks and makes block
processing smoother (more smaller blocks are being produced).
 2020-08-19 15:16:19 +03:00
Roman Khimov
eb8122837e core: fix failing TestGetTransaction 
3677cc0e2 ruined it:
    TestGetTransaction: blockchain_test.go:413:
                Error Trace:    blockchain_test.go:413
                Error:          Not equal:
                                expected: 467
                                actual  : 399
                Test:           TestGetTransaction
 2020-08-19 13:22:31 +03:00
Roman Khimov
6dffd4967a
Merge pull request #1335 from nspcc-dev/tests/binary 
Update binary test data
 2020-08-19 11:33:47 +03:00
Evgenii Stratonikov
3677cc0e24 core: update GetTransaction test 2020-08-19 09:41:11 +03:00
Evgenii Stratonikov
04bff62d65 transaction: update binary test data 2020-08-18 15:09:30 +03:00
Evgenii Stratonikov
054b77dbde block: update binary test data 2020-08-18 14:52:30 +03:00
Roman Khimov
52b002e079
Merge pull request #1332 from nspcc-dev/check-for-excessive-verification-results 
core: verification script must return exactly one value
 2020-08-18 12:15:10 +03:00
Anna Shaleva
591f639ad6 core: fix getOnPersistWrapper for native contracts 
According to manifest, OnPersist.ReturnType is void, so we shouldn't
return anything from it. It's not so important, as we drop this value at
the end of OnPersist invocation.
 2020-08-18 09:52:46 +03:00
Roman Khimov
d0c29f52c9 core: verification script must return exactly one value 
C# node is quite picky as it expects there to be exactly one value returned,
but our testchain actually adds 4 signatures for multisig cases instead of 3
which makes it technically incompatible with C# node.
 2020-08-17 22:02:15 +03:00
Roman Khimov
8e619cc671
Merge pull request #1318 from nspcc-dev/fix/verifytests 
Add tests for `verifyTx` and `verifyHeader`
 2020-08-14 17:24:03 +03:00
Roman Khimov
92f37a5d36
Merge pull request #1324 from nspcc-dev/fix-goreportcard-issues 
Fix goreportcard issues
 2020-08-14 14:50:35 +03:00
Evgenii Stratonikov
a796f2b61d names: implement FromID 
Allow to convert interop id to it's name.
 2020-08-14 14:22:45 +03:00
Evgenii Stratonikov
7854dcfd8f core: replace interop names with named constants 2020-08-14 14:21:54 +03:00
Evgenii Stratonikov
f3650e20b0 vm: move InteropNameToID to a separate package 2020-08-14 13:54:11 +03:00
Roman Khimov
e7d13e6db2 *: fix misspellings found in Go Report Card 2020-08-14 12:16:24 +03:00
Roman Khimov
f6a308f7f6 *: fix ineffassign where the value should really be used 
Found with GoReportCard.
 2020-08-14 12:08:57 +03:00
Roman Khimov
ec2a2f3fb9 transaction: fix ineffassign GoReportCard issue 
s is not used purposefuly.
 2020-08-14 12:08:16 +03:00
Evgenii Stratonikov
cadebdfc19 core: add tests for (*Blockchain).verifyHashAgainstScript 2020-08-14 09:40:36 +03:00
Evgenii Stratonikov
1eb9a4c6c6 core: allow to use verification contracts 
In NEO3 we can't just appcall hash, as verification script has no access
to state. Instead we use `verify` method of an arbitrary contract.
 2020-08-14 09:40:34 +03:00
Evgenii Stratonikov
7f2a931fb6 core: add tests for (*Blockchain).verifyHeader 2020-08-14 09:37:32 +03:00
Evgenii Stratonikov
e8cf4d96ce core: add tests for (*Blockchain).verifyTx 2020-08-14 09:37:30 +03:00
Roman Khimov
ca542f11fe core: drop unused secondsPerBlock() method 2020-08-11 21:37:18 +03:00
Roman Khimov
acc2265169 transaction: attribute type is now type, not usage 2020-08-11 21:10:50 +03:00
Roman Khimov
fb70c82157 transaction: there are no valid attributes defined for preview3 2020-08-11 21:10:12 +03:00
Roman Khimov
b1034d8ed6 state: drop Neo 2 Account and everything related 
It's substituted with NEP5Balances now.
 2020-08-11 20:42:02 +03:00
Roman Khimov
34156ef78d core: fix (*HeaderHashList).Write comment 2020-08-11 20:33:16 +03:00
Roman Khimov
2354af463a core: drop some unused HeaderHashList functions 2020-08-11 20:33:16 +03:00
Roman Khimov
08fd6180b2 native: drop validators count structure 
It's not used since #1252.
 2020-08-11 20:19:55 +03:00
Roman Khimov
c3f7a419a0
Merge pull request #1296 from nspcc-dev/smartcontract/examples 
examples: update examples
 2020-08-11 19:09:13 +03:00
Anna Shaleva
b5494320f9 compiler, cli: support events from .yml config file 
We currently can't process events in codegen, so we have to provide
them via .yml config file. Do not delete the rest of the code connected
with conversion of MethodDebugInfo.Event into manifest.Event as we have
issue #1038.
 2020-08-11 13:42:06 +03:00
Evgenii Stratonikov
057e1c6e3c compiler: provide filename to Compile() 2020-08-11 11:10:45 +03:00
Roman Khimov
c16040aecc native: sort GetValidators result 
As it's returned sorted now. Fixes state change mismatch for
NextValidators. It also partially reverts
2f8e7e4d33 and significantly changes the test
chain as the fees are no longer being sent to the same account.
 2020-08-10 19:49:09 +03:00
Roman Khimov
dba248236c smartcontract: add CreateDefaultMultiSigRedeemScript 
And use it where appropriate. Some of our code was just plain wrong (like the
one in GAS contract) and unification is always useful here.
 2020-08-10 18:58:11 +03:00
Roman Khimov
fb97ea9458 native: don't register standby validators on initialization 
C# doesn't do that since neo-project/neo#1762.
 2020-08-10 17:51:46 +03:00
Roman Khimov
f287681fa7 core/native: fix VotersCount zero value encoding 
Empty byte array is enough to be bigint value of zero. Fixes state differences
with C# node.
 2020-08-10 17:07:14 +03:00
Evgenii Stratonikov
bf01599430 vm: check return value on context unload 
When calling external contracts we expect exactly 1 value to be on
stack. For methods returning nothing, `Null` value is pushed, otherwise
it is an error.`
 2020-08-10 11:52:33 +03:00
Roman Khimov
4aeaf05f90
Merge pull request #1289 from nspcc-dev/fix/nep5 
Port `getnep5transfers` changes to master
 2020-08-10 11:11:24 +03:00
Evgenii Stratonikov
14aba7fc80 core: fix failing tests after merge 
Related f5131491b .
 2020-08-10 09:20:34 +03:00
Roman Khimov
f5131491b7
Merge pull request #1271 from nspcc-dev/core/call_from_native 
core: contractCall from native contracts
 2020-08-07 21:24:50 +03:00
Anna Shaleva
c87e2262c3 core: remove ScriptHashGetter from interop context 
We have VM inside the context, so don't need ScriptHashGetter anymore.
 2020-08-07 20:31:20 +03:00
Anna Shaleva
42736095a1 core: add test to call contract from native 
Closes #1200
 2020-08-07 20:31:16 +03:00
Evgenii Stratonikov
807338f97e core: do not store NEP5 transfer log in memory 
Traversing transfer log instead of accumulating and returning it
is faster and takes less memory.
 2020-08-07 18:21:06 +03:00
Anna Shaleva
995053f2eb core: add VM into interop context 2020-08-07 16:15:24 +03:00
Roman Khimov
c3c88a57cd
Merge pull request #1281 from nspcc-dev/drop-go-1.12-and-fix-some-things 
Drop go 1.12 and fix some things
 2020-08-07 13:34:54 +03:00
Roman Khimov
d5a9d80c12 core: refactor out policy check for transaction 
We were checking blocked accounts twice which is obviously excessive. We also
have our accounts sorted, so we can rely on that in CheckPolicy(). It also
doesn't make much sense to check MaxBlockSystemFee in Blockchain code, policy
contract can handle that.
 2020-08-07 12:21:52 +03:00
Roman Khimov
90180c6fb6 native: pass DAO to CheckPolicy(), it doesn't need interop context 
Simplify things a bit.
 2020-08-07 12:21:52 +03:00
Roman Khimov
791c983304 core: drop GetScriptHashesForVerifying 
It no longer depends on blockchain state and there can't ever be an error, in
fact we can always iterate over signers, so copying these hashes doesn't make
much sense at all as well as sorting arrays in verifyTxWitnesses (witnesses
order must match signers order).
 2020-08-07 12:21:52 +03:00
Roman Khimov
c19838ea67 core: use errors.Is in PoolTx 
Just in case.
 2020-08-07 12:21:52 +03:00
Roman Khimov
4c38fae54c core: fix mempool error check in PoolTx 
ErrAlreadyExists should be returned for ErrDup, not ErrConflict.
 2020-08-07 12:21:52 +03:00
Roman Khimov
0e2784cd2c always wrap errors when creating new ones with fmt.Errorf() 
It doesn't really change anything in most of the cases, but it's a useful
habit anyway.

Fix #350.
 2020-08-07 12:21:52 +03:00
Roman Khimov
205f52c563 core: use error wrapping to provide more details 2020-08-07 12:21:52 +03:00
Roman Khimov
5ef08f60ae remove github.com/pkg/errors from dependencies 
It's not needed any more with Go 1.13 as we have wrapping/unwrapping in base
packages. All errors.Wrap calls are replaced with fmt.Errorf, some strings are
improved along the way.
 2020-08-07 12:21:52 +03:00
Evgenii Stratonikov
bfda60c683 native: add missing witness checks to NEO.(Un)registerCandidate 2020-08-07 09:17:02 +03:00
Evgenii Stratonikov
8af3f05358 native: implement NEO.UnregisterCandidate method 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
38a92323c9 native: fill votes when registering unregistered candidate 
If a candidate was registered, then unregistered and then again
registered, it's votes must not be lost.
 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
27169d140f core: implement (*Blockchain).GetStandByCommitee() 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
e5d973d3a4 config: rename StandbyValidators to StandbyCommittee 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
b14b047c78 native: add tests for delegated voting 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
9bc731b3b1 native: implement delegated voting 
Close #867.
 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
f0b62cdaa6 native: make *totalSupply accept DAO 
There is no need to provide full interop context.
 2020-08-06 14:26:24 +03:00
Evgenii Stratonikov
3e39f0f211 native: rename Validator to Candidate 2020-08-06 14:26:24 +03:00
Roman Khimov
5e53aceb0f
Merge pull request #1277 from nspcc-dev/restrict-verification-call-flags 
core: restrict verification scripts with CallFlags.None
 2020-08-06 13:52:14 +03:00
Roman Khimov
57ee8b80e5
Merge pull request #1246 from nspcc-dev/protocol/getblockbyindex 
protocol: request blocks by index
 2020-08-05 23:21:07 +03:00
Roman Khimov
b7f89b0544 core: restrict verification scripts with CallFlags.None 
Follow neo-project/neo#1776. Looks very suspicious to me as it severely limits
verification scripts, but we have to be compatible at this point.
 2020-08-05 19:17:48 +03:00
Evgenii Stratonikov
194da64975 core: deepcopy notification in System.Runtime.Notify 
Make it impossible to change already emitted notifications via
`System.Runtime.GetNotifications`.
 2020-08-05 11:19:41 +03:00
Roman Khimov
ef53a45e7a
Merge pull request #1264 from nspcc-dev/smartcontract/manifest/supported_standards 
smartcontract: add list of supported standards to manifest
 2020-08-04 22:17:00 +03:00
Anna Shaleva
7c6cdcbcc9 core: fix bug with policy.MaxVerificationGas 
In 121c9664b we should take into account isValid flag of
NativePolicy contract while retrieving MaxVerificationGas native
policy value. Otherwise we won't be able to get MaxVerificationGas
after the node was restarted, because this value is not truly
stored along with the other native policy values.

This commit fixes bug with headers verification after the  node
restarting with an existing storage:

```
2020-08-03T12:52:56.158+0300	WARN	failed processing headers	{"error": "vm failed to execute the script with error: error encountered at instruction 0 (PUSHDATA1): gas limit is exceeded", "errorVerbose": "vm failed to execute the script with error: error encountered at instruction 0 (PUSHDATA1): gas limit is exceeded\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHashAgainstScript\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1454\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeaderWitnesses\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1517\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeader\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1175\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).addHeaders\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:484\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddHeaders\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:453\ngithub.com/nspcc-dev/neo-go/pkg/network.(*Server).handleHeadersCmd\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/network/server.go:454\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373"}
```
 2020-08-04 17:52:34 +03:00
Anna Shaleva
ba08a9b6ad core: move all transaction.Decode checks to separate method 
We should perform the same checks during UnmarshalJSON.
 2020-08-04 17:34:06 +03:00
Anna Shaleva
ead6bdb25e core: remove unnecessary comment 2020-08-04 17:34:06 +03:00
Anna Shaleva
90825efa16 core: move transaction's sender to cosigners 
Closes #1184

Ported changes from https://github.com/neo-project/neo/pull/1752
 2020-08-04 17:33:50 +03:00
Anna Shaleva
66ceaa6b75 smartcontract: add list of supported standards to manifest 
Closes #1204
 2020-08-04 17:29:44 +03:00
Anna Shaleva
8697582b23 core: add FeeOnly witness scope 2020-08-04 15:08:59 +03:00
Anna Shaleva
75f1c2d6f2 core: add scopes check to cosigners in DecodeBinary 2020-08-04 15:08:59 +03:00
Anna Shaleva
7865bc5925 core: fix native policy tests 
That was a merge problem.
 2020-08-04 15:05:31 +03:00
Roman Khimov
62f5aa8eb4
Merge pull request #1254 from nspcc-dev/native/policy/maxblocksystemfee 
core, consensus: add maxBlockSystemFee to native Policy
 2020-08-04 13:13:06 +03:00
Roman Khimov
fa5ff8dd95
Merge pull request #1242 from nspcc-dev/fix/json 
Implement lossless stackitem to json conversion
 2020-08-04 12:59:47 +03:00
Anna Shaleva
db65ed04d9 consensus: apply policy during verifyBlock 
To follow C# implementation we should also check proposed block on
policy matching.
 2020-08-04 10:59:09 +03:00
Anna Shaleva
4a1c8464f9 core: use maxBlockSize to apply policy to tx set 
We should also take into account maxBlockSize in
(bc *Blockchain).ApplyPolicyToTxSet
 2020-08-04 10:59:09 +03:00
Anna Shaleva
c647f8e4ed core: add maxBlockSystemFee to native Policy 
Closes #1195
 2020-08-04 10:59:00 +03:00
Roman Khimov
00671deb8f
Merge pull request #1241 from nspcc-dev/fix/string 
Ensure strings are valid UTF-8 where appropriate
 2020-08-03 18:10:27 +03:00
Anna Shaleva
13336f9dba core: adjust calling flags of native Policy methods 2020-08-03 16:09:10 +03:00
Anna Shaleva
60f9abae40 core: chenge maxBlockSizeKey of native Policy 2020-08-03 16:00:15 +03:00
Evgenii Stratonikov
e5d538ed21 core: use raw stack items in application logs 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-31 16:07:16 +03:00
fyrchik
e4fc655115
Merge pull request #1239 from nspcc-dev/store_nep5_with_id 
core: store contract IDs instead of hashes for NEP5Balances and Transfers
 2020-07-30 12:56:19 +03:00
Roman Khimov
e21c65c59f core: add state height to prometheus metrics 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
5ee3ecf381 core: update verified state root height 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
ab802cdd5f state: implement JSON marshaling for MPT* items 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Roman Khimov
236438d799 core: do MPT compaction every once in a while 
We need to compact our in-memory MPT from time to time, otherwise it quickly
fills up all available memory. This raises two obvious quesions --- when to do
that and to what level do that.

As for 'when', I think it's quite easy to use our regular persistence interval
as an anchor (and it also frees up some memory), but we can't do that in the
persistence routine itself because of synchronization issues (adding some
synchronization primitives would add some cost that I'd also like to avoid),
so do it indirectly by comparing persisted and current height in `storeBlock`.

Choosing proper level is another problem, but if we're to roughly estimate one
full branch node to use 1K of memory (usually it's way less than that) then we
can easily store 1K of these nodes and that gives us a depth of 10 for our
trie.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Roman Khimov
58b7e16e0e core: fix PrevHash calculation for MPTRoot 
This was differing from C# notion of PrevHash. It's not a previous root, but
rather a hash of the previous serialized MPTRoot structure (that is to be
signed by CNs).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Roman Khimov
caea6d6ca8 mpt: fix extension node cache invalidation 
It should always be invalidated if something changes in the `next` (below the
extension node).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
0e29382035 core: update MPT during block processing 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Roman Khimov
2b53877dff mpt: don't flush nodes already present in the DB 
It's just a waste of time.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Roman Khimov
475bf2445a mpt: restructure nodes a bit, implement serialization and hash cache 
It drastically reduces the number of allocations and hash calculations.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
6ca22027d5 mpt: implement (*Trie).Collapse() 
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
f0b85f8af7 mpt: implement JSON marshaling/unmarshaling 
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
9b328240dd mpt: implement MPT proof Get and Verify 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
dc6741bce7 mpt: implement MPT trie 
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
3c99393bef core: simplify System.Contract.Call* parameter handling 2020-07-30 12:37:32 +03:00
Evgenii Stratonikov
f40aba4cd0 vm: convert items to UTF-8 strings 
Add `stackitem.ToString` for seamless string conversion.
 2020-07-30 12:37:31 +03:00
Anna Shaleva
0dd1730632 core: store NEP5Transfer asset by ID instead of hash 
To avoid problems with retrieving decimals for migrated contracts.
 2020-07-29 15:15:48 +03:00
Anna Shaleva
b9bdab8ec8 core: store nep5balances using contract id 
Closes #1194
 2020-07-29 15:15:48 +03:00
Anna Shaleva
dbd460d883 core: retrieve contract hash by ID 
We'll need this ability further to retrieve contracts hashes for Nep5Balances.
 2020-07-29 15:14:24 +03:00
Evgenii Stratonikov
9a992865b1 interop: add Secp2k to the list of crypto interops 
Allow to use them during verification.
 2020-07-29 13:43:33 +03:00
Evgenii Stratonikov
23a1430395 core: allow to restrict creating callbacks from syscalls 
Specify DisallowCallback flag if syscall is not allowed to be used in a
callback.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
99b0397a61 core: implement System.Callback.CreateFromSyscall interop 
Allow to create callbacks from syscalls.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
c54b45e76d core: implement System.Callback.CreateFromMethod interop 
Support creating callbacks from contract methods.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
f96c217aba core/tests: refactor getting contract state 
Get 2 contracts in pair which is useful everytime we need to test
syscall with one contract calling the other.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
e654d22991 core/tests: introduce loadScript helper 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
382a7f5b3e core: implement System.Callback.* interops 
Support creating callbacks from pointers.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
c09ea04df3 core: add prices for TRY* opcodes 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Evgenii Stratonikov
51ae12e4fd *: move syscall handling out of VM 
Remove interop-related structures from the `vm` package.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-07-29 13:41:08 +03:00
Anna Shaleva
4bf88ba6b0 core: decouple native contracts from interop service 
Closes #1191.
 2020-07-29 10:33:18 +03:00
fyrchik
b187dfe3ce
Merge pull request #1236 from nspcc-dev/rpc/getrawmempool 
rpc: update `getrawmempool` and `getrawtransaction` RPC-calls
 2020-07-29 10:18:51 +03:00
Anna Shaleva
70ef733ce7 core, vm: store VMState as byte instead of string 
Part of #1183
 2020-07-29 10:14:08 +03:00
Evgenii Stratonikov
685d44dbc1 *: support _initialize method in contracts 
Invoke `_initialize` method on every call if present.
In NEO3 there is no entrypoint and methods are invoked by offset,
thus `Main` function is no longer required.
We still have special `Main` method in tests to simplify them.
 2020-07-27 13:00:35 +03:00
Evgenii Stratonikov
466af55dea vm: isolate stack in LoadScript 
When calling contract it must be provided with a new stack containing
only it's arguments. The result is then copied back on RET.

Fix #1220.
 2020-07-27 13:00:35 +03:00
Evgenii Stratonikov
d2ddf7b7cb *: support invoking methods by offset 
Allow to invoke methods by offset:
1. Every invoked contract must have manifest.
2. Check arguments count on invocation.
3. Change AppCall to a regular syscall.
4. Add test suite for `System.Contract.Call`.
 2020-07-27 13:00:35 +03:00
Evgenii Stratonikov
54d7882acf core: compiler contract on-the-fly in helper test 
Generate proper manifest file too.
 2020-07-27 13:00:34 +03:00
Evgenii Stratonikov
7d8fead1fd native: change onPersist return type to Void 2020-07-27 11:08:01 +03:00
Evgenii Stratonikov
e52c39ae7e manifest: remove EntryPoint from manifest 2020-07-27 11:08:01 +03:00
Roman Khimov
d2c823daa6
Merge pull request #1233 from nspcc-dev/fix/gasverify 
core: restrict maximum gas allow for verification
 2020-07-26 22:40:13 +03:00
Evgenii Stratonikov
121c9664b4 core: restrict maximum gas allow for verification 
Disallow costly verification methods. We put this limit in policy
contract as it may be a subject to change in future.
In fact this value also overrides gas limit for header verification.
Close #1202.
 2020-07-25 14:33:38 +03:00
Evgenii Stratonikov
8931c9a794 core/interop: increase coverate for crypto interops 2020-07-25 12:16:56 +03:00
Evgenii Stratonikov
83e53fab24 core/test: ensure public key is invalid in test 
When providing public key as a subslice, it still can be
decoded as a valid key, thus interop will not return an error
but rather push `false` on stack. This test is about providing
invalid key, so ensure this via setting invalid prefix.
 2020-07-25 12:01:16 +03:00
Anna Shaleva
8fde41001e core, vm: remove allowed triggers from syscalls 
Closes #1205.
 2020-07-23 20:44:39 +03:00
Roman Khimov
d8a1c3de46
Merge pull request #1221 from nspcc-dev/neo3/interop/post-preview2_adjustment3 
interop: post-preview2 adjustment, part 3
 2020-07-23 20:35:53 +03:00
Anna Shaleva
ef8de3b2dc core: adjust System.Runtime.GetNotifications interop 
Part of #1055.

Forgot to add this pretty interop to the list of system interops.
 2020-07-23 07:54:51 +03:00
Anna Shaleva
120eff92f7 core: adjust System.Runtime.Notify interop 
Part of #1198.

Notification name should be UTF8-encoded.
 2020-07-23 07:55:12 +03:00
Anna Shaleva
2c41b7b254 core: adjust System.Runtime.Log interop 
Part of #1055 and #1198.

It should check the message length and encoding.
 2020-07-23 07:54:51 +03:00
Anna Shaleva
88e003d219 core: adjust System.Runtime.GetTime interop 
Part of #1055.

It should have AllowStates flag.
 2020-07-23 07:51:24 +03:00
Anna Shaleva
990ef5525c core: adjust System.Runtime.GasLeft interop 
Part of #1055.

In test mode it should return -1.
 2020-07-23 07:51:24 +03:00
Anna Shaleva
8fed383523 core: adjust System.Runtime.CheckWitness interop 
Part of #1055.

It should have `AllowStates` flag.

Also removed unreachable code: we can't have such situation when
script container is not a transaction in the scope of `CheckWitness`
method because:
1. Blocks have their own implementation of CheckWitness for
internal usage (it's (bc *Blockchain) verifyHeaderWitnesses method).
2. For the outside calls of System.Runtime.CheckWitness interop (e.g.
calls from smart-contract) script container is always a transaction.
 2020-07-23 07:51:24 +03:00
Anna Shaleva
459ac34839 core: adjust System.Enumerator.Create interop 
Part of #1201.

It should be able to create enumerator from primitive byte-array-like
stack items too.
 2020-07-23 07:51:24 +03:00
Anna Shaleva
6e44499cec core: adjust Neo.Native.Deploy interop 
Part of #1055.

It could be that context.Block is nill.
 2020-07-22 16:58:32 +03:00
Anna Shaleva
b8d82b49ec core: add Neo.Crypto.RIPEMD160 interop 
Closes #1193.
 2020-07-22 16:58:32 +03:00
Anna Shaleva
84bf87df52 core: adjust System.Storage.PutEx interop 
Part of #1055.

Added System.Storage.PutEx to compiler. Added StorageFlag in order to
denote whether item is constant or not.
 2020-07-22 16:58:23 +03:00
Anna Shaleva
47eadcdf2a core: adjust System.Storage.Put interop 
Part of #1055.

Maximum storage key len has been changed. Also added maximum storage
value len restriction.
 2020-07-22 11:05:10 +03:00
Anna Shaleva
b5185d5d1a core: refactor System.Storage.Get[ReadOnly]Context interops 
Part of #1055.

Split methods, as they have a lot of common code. This also fixex nil
error of storageGetReadOnlyContext in case when contract does not have
storage.
 2020-07-22 10:51:47 +03:00
Evgenii Stratonikov
3d7fa9de93 *: make Notify interop accept event name 2020-07-20 13:33:32 +03:00
Roman Khimov
ae9658a108 dao: migrate nep5 balances with the contract 
Fixes #1144. It's quite simple approach, we just update balance info right
upon contract migration. It will slow down migration transactions, but it
takes about 1-2 seconds to Seek through balances at mainnet's 3.8M, so the
approach should still work good enough. The other idea was to make lazy
updates (maintaining contract migration map), but it's more complicated to
implement (and implies that a balance get might also do a write).

There also is a concern about memory usage, it can give a spike of some tens
of megabytes, but that also is considered to be acceptable.
 2020-07-17 19:03:12 +03:00
Roman Khimov
3f2f0be2c2
Merge pull request #1187 from nspcc-dev/neo3/interop/post-preview2_adjustment 
interop: post-preview2 adjustment
 2020-07-17 13:33:29 +03:00
Anna Shaleva
74d2f437f4 core: add System.Binary.Base64Encode(Decode) interops 
Part of #1055
 2020-07-17 12:36:18 +03:00
Anna Shaleva
f31ce9289d core: add System.Contract.GetCallFlags interop 
Part of #1055.

It returns calling flags of the current context.
 2020-07-17 12:35:02 +03:00
Anna Shaleva
842feb2533 core: adjust System.Contract.Update interop 
Part of #1055.

We should check contract scripthash against the one provided in manifest
and manifest groups. We shouldn't put on stack anything after return.
And ofcourse, we mast not destroy the old contract at the end, as
`contractDestroy` removes all storage items associated with the
old contract ID (which equals to the new contract ID). We just remove
old contract state - it's enough.
 2020-07-17 12:33:44 +03:00
Anna Shaleva
fddad0b475 core: adjust System.Contract.Create interop 
Part of #1055.

It should check given scripthash against manifest groups and return the
contract state as a struct (not interop interface).
 2020-07-17 09:28:52 +03:00
Anna Shaleva
d2ec0fed3d core: adjust System.Blockchain.GetContract interop 
Part of #1055.

It should put on stack an array instead of interop interface.
 2020-07-17 09:28:52 +03:00
Anna Shaleva
1a5fb01e61 core: adjust System.Contract.IsStandard interop 
Part of #1055.

It should check not only stored contracts, but also interop context
script container in case if it's a transaction.
 2020-07-17 08:19:43 +03:00
Anna Shaleva
d2f452c240 core: adjust System.Contract.CallEx interop 
Part of #1055
 2020-07-17 08:19:43 +03:00
Anna Shaleva
74ffde8367 core: adjust System.Blockchain.GetTransactionFromBlock interop 
To match C# implementation, we should pick all arguments from stack first.
 2020-07-17 08:19:43 +03:00
Roman Khimov
38f0432f36 core: remove stale storeBlock comment, fix #365 
After transaction unification and UTXO removal storeBlock became quite simple
and doesn't require any refactoring any more.
 2020-07-16 23:06:17 +03:00
Roman Khimov
acfded7f45
Merge pull request #1180 from nspcc-dev/fix-calling-scripthash-check-in-nep5 
Add calling scripthash check to native nep5 transfers
 2020-07-16 07:29:14 +03:00
Roman Khimov
59be6bcd24 native: add calling script hash to NEP5 transfer witness check 
Fix transfers from contract's address.
 2020-07-15 22:52:35 +03:00
Roman Khimov
d4c3a17883 interop/native: always use proper ScriptHashGetter, fix #924 
All scripts are run in VM, so it's there to tell us about script hashes
involved and it must be used instead of nep5ScriptHash kludge.
 2020-07-15 22:43:30 +03:00
Roman Khimov
f46ed798f0
Merge pull request #1169 from nspcc-dev/neo3/rpc/fields_names_adjustment 
rpc: adjust RPC calls JSON fields
 2020-07-15 18:24:54 +03:00
Anna Shaleva
538616e9f8 rpc: adjust getrawtransaction RPC-call JSON fields names 
Part of #1130
 2020-07-15 14:44:13 +03:00
Anna Shaleva
a3e306ff78 core: implement Secp256k1 Verify and CheckMultisig interops 
Closes #918.
 2020-07-14 16:21:34 +03:00
Anna Shaleva
5326fc587a core: rename Neo.Crypto.CheckMultisig to Neo.Crypto.CheckMultisigWithECDsaSecp256r1 
Part of #918
 2020-07-14 16:19:12 +03:00
Anna Shaleva
17233e1d8e core: rename Neo.Crypto.Verify to Neo.Crypto.VerifyWithECDsaSecp256r1 
Part of #918
 2020-07-14 16:19:12 +03:00
Roman Khimov
4c23aa1d7c
Merge pull request #1174 from nspcc-dev/neo3/interop/getscriptcontainer 
core, compiler: return tx from GetScriptContainer interop
 2020-07-14 08:58:52 +03:00
Roman Khimov
dafb9eea4d
Merge pull request #1177 from nspcc-dev/no-free-vm-runs 
No free VM runs
 2020-07-14 08:42:13 +03:00
Roman Khimov
db027ad9c5 vm: zero GAS means no GAS, use fee data to properly limit execution 
We were accepting transactions with zero system fee, but we shouldn't do
that. Also, transaction's verification execution has to be limited by network
fee.
 2020-07-14 08:37:29 +03:00
Anna Shaleva
c4f7b06974 core, compiler: return struct from GetScriptContainer interop 
Closes #1173
 2020-07-14 06:04:48 +03:00
Roman Khimov
419d68329c core: limit GAS available for block verification 2020-07-13 18:24:58 +03:00
Roman Khimov
3134e364b2 core: fix CalculateNetworkFee() for multisig contracts 
We return m from the vm.ParseMultiSigContract and n is the length of pubs,
invocation script then pushes m signatures for n keys.
 2020-07-13 18:05:46 +03:00
Roman Khimov
afc5ee1ded core: fix ECDSA verifiation price, it's 1000000 and it's defined in crypto 2020-07-13 18:04:50 +03:00
Roman Khimov
579630a3fa native: don't expose internal slices to the outside world 
They should be hidden.
 2020-07-11 19:54:50 +03:00
Roman Khimov
7eef895061 consensus: use GetNextBlockValidators where appropriate 
GetValidators without parameter is called upon DBFT initialization and it
should receive validators for the next block (that will create it),
parameterized GetValidators is used for NextConsensus calculation where we
need a list for the current state of the chain.
 2020-07-11 19:54:50 +03:00
Roman Khimov
ae497228f0 core: use native NEO GetValidators for bc.GetValidators 
NextBlockValidators are updated before the new block persist, so we need to
use GetValidators to get the list corresponding to the current state of the
chain.
 2020-07-11 19:54:50 +03:00
Roman Khimov
fedcc6b6fc transaction: drop old attributes 
They're not supported in Neo 3. Also change data encoding to base64 following
Neo 3 changes.
 2020-07-10 20:40:27 +03:00
Anna Shaleva
789ee2d3c1 core: do not update mempool while verifying tx 
Closes #1168
 2020-07-09 20:45:59 +03:00
Anna Shaleva
4be1009def core: refactor checkBalanceAndUpdate 2020-07-09 18:27:20 +03:00
Anna Shaleva
e81ccb7deb rpc: adjust getblock RPC-call JSON fields names 
Part of #1130
 2020-07-09 17:34:53 +03:00
Anna Shaleva
abe3c94b95 core: use big.Int to store NEP5 balances 
closes #1133
 2020-07-09 13:26:39 +03:00
Anna Shaleva
43b28ffa06 core: get rid of NEP5TransferSize 
Part of #1133

It will help us to use big.Int to store amount of NEP5 tokens. As far as
big.Int doesn't have constant size, we shouldn't use `NEP5TransferSize`
constant anymore.
 2020-07-09 13:25:03 +03:00
Evgenii Stratonikov
5787d7775f transaction: fail creating tx if there is additional data present 2020-07-03 18:03:57 +03:00
Evgenii Stratonikov
27b3054df4 transaction: set feePerByte on tx construction 2020-07-03 17:52:13 +03:00
Evgenii Stratonikov
3097dc60e5 transaction: cache feePerByte 2020-07-03 17:37:01 +03:00
Anna Shaleva
c421805141 core: fixed typo in method description 2020-06-29 21:40:59 +03:00
Anna Shaleva
0aaaf7f787 *: switch from fixed8 to int64 in (Blockchain).CalculateClaimable 2020-06-29 21:40:54 +03:00
Anna Shaleva
73b630db9b *: switch from fixed8 to int64 
Follow C# implementation, we have to marshall JSON Fixed8 fields without
taking into account decimals.
 2020-06-29 21:39:27 +03:00
Roman Khimov
8ad355b444
Merge pull request #1124 from nspcc-dev/forward-port-from-2.x 
Forward port from 2.x
 2020-06-29 11:52:45 +03:00
Evgenii Stratonikov
f007cca80b native: hide native contract methods 
Copy public keys when returning them to the outside and hide unused
`GetValidators` method.
 2020-06-29 10:48:37 +03:00
Evgenii Stratonikov
8c18142e8a keys: implement PublicKeys.Copy() 
Implement convenient wrapper over explicit allocation and copying.
 2020-06-29 10:44:35 +03:00
Evgenii Stratonikov
bbe02ac584 native: store typed nil in validators cache 
Fix `error encountered at instruction 0 (SYSCALL): sync/atomic: store of nil value into Value"`.
 2020-06-29 10:37:52 +03:00
Roman Khimov
cab437284d core: fix TestGetClaimable occasional failures 2020-06-27 12:36:43 +03:00
Roman Khimov
efa8ae5be4 core: fix TestSubscriptions occasional failures 
panic: Log in goroutine after TestSubscriptions has completed

goroutine 1079 [running]:
testing.(*common).logDepth(0xc00057a100, 0xc00039e210, 0xa4, 0x3)
	/usr/local/go/src/testing/testing.go:634 +0x51a
testing.(*common).log(...)
	/usr/local/go/src/testing/testing.go:614
testing.(*common).Logf(0xc00057a100, 0xe32eaa, 0x2, 0xc0009560e0, 0x1, 0x1)
	/usr/local/go/src/testing/testing.go:649 +0x91
go.uber.org/zap/zaptest.testingWriter.Write(0xf64120, 0xc00057a100, 0x0, 0xc0003fe400, 0xa5, 0x400, 0xc000958e40, 0xc0009560d0, 0xc000958e60)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x120
go.uber.org/zap/zapcore.(*ioCore).Write(0xc0005cd050, 0x0, 0xbfb54ffc0626aba2, 0x916de700, 0x1485500, 0x0, 0x0, 0xe43fb0, 0x1c, 0x0, ...)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c5
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000102d10, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e8
go.uber.org/zap.(*Logger).Info(0xc00035eba0, 0xe43fb0, 0x1c, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:187 +0x96
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).persist(0xc00000cb40, 0xc00017c2c0, 0xbe8a00)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:839 +0x6c9
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run.func2(0xc00000cb40, 0xc0005c6c30)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:302 +0x54
created by github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:301 +0x25d
FAIL	github.com/nspcc-dev/neo-go/pkg/core	2.463s
 2020-06-25 19:22:38 +03:00
Evgenii Stratonikov
978b3c96fe native: cache GetValidators result 2020-06-24 19:42:50 +03:00
Roman Khimov
5251607fb7 transaction: s/txid/hash/ for JSON to match C# implementation 
It uses `hash` for transactions now, but `txid` for application logs.
 2020-06-24 10:43:58 +03:00
Roman Khimov
d81d826bfc core: fix Storage.Get to return Null when there is no value 
Match C# implementation and fix state inconsistency at block 249920 of
preview2 testnet. Make our Go Storage.Get return nil and adapt
examples/tests.
 2020-06-24 10:43:58 +03:00
Roman Khimov
7987cdadc0 native: update storage item value after gas distribution 
Even if the value is zero, the GAS distribution updates the balance height, so
storage item must be updated too. Fixes the followin on preview2 testnet:
block 74227: value mismatch for key ffffffff1454a6cb279fbcedc66162ad4ad5d1d910202b92743e000000000000000000000005: 1041032104809fd5002103f3210128010000 vs 1041032104809fd50021033f110128010000
 2020-06-24 10:22:21 +03:00
Roman Khimov
48fac6f87d native: unify some transfer code 2020-06-24 10:22:21 +03:00
Roman Khimov
fccad11716 native: drop accounts with zero balance 
They make no sense. Fixes preview2 testnet state problem:

file BlockStorage_100000/dump-block-70000.json: block 69935: state mismatch for key ffffffff1454a6cb279fbcedc66162ad4ad5d1d910202b92743e000000000000000000000005: Deleted vs Added
 2020-06-24 10:22:21 +03:00
Roman Khimov
d5c9449a43 dao: combine GetNextContractID and PutNextContractID 
And fix contract create to really update the ID, eliminating this difference
in the storage (preview2 testnet):
file BlockStorage_100000/dump-block-39000.json: block 38043: key mismatch: 0c000000617373657464df4ebe92334d1fc7e64b10f1d1e33942d9905e510000000000000009 vs 00000000617373657464df4ebe92334d1fc7e64b10f1d1e33942d9905e510000000000000009
 2020-06-24 10:22:21 +03:00
Roman Khimov
5e56c9db29 native: fix voting accounting in transfers processing 
We should modify votes count exactly by the amount of NEO balance change, it
might be negative here.
 2020-06-24 10:22:21 +03:00
Roman Khimov
37d44b94e1 native: fix netfee gas redistribution 
Preview2 testnet:
file BlockStorage_100000/dump-block-12000.json: block 11562: key mismatch: feffffff1454a6cb279fbcedc66162ad4ad5d1d910202b92743e000000000000000000000005 vs feffffff1431b7e7aea5131f74721e002c6a56b610885813f79e000000000000000000000005

Originally this code was written to run after transactions processing, but
after 0fa4c49735 it works in different manner.
 2020-06-24 10:22:21 +03:00
Roman Khimov
d0331bf21b native: simplify votes saving in NEO contract 
Follow C# behavior and fix state mismatch at block 11561 of preview2 testnet.
 2020-06-24 10:22:21 +03:00
Roman Khimov
e5f05790d5 core: cache standby validators in the Blockchain 
They never change, so it makes no sense parsing the keys over and over
again. It also simplifies the interface a little.
 2020-06-24 10:22:17 +03:00
Roman Khimov
cd2dca0259 state: fix the way NEO balance is being serialized 
Actually, our format is way better, but for preview2 compatibility we need to
use this one.
 2020-06-24 10:21:24 +03:00
Roman Khimov
c124d2bcdf core: fix gas generation coefficients to match Neo 3 
Follow neo-project/neo#911. Fixes state differences at block 4528 of preview2
testnet compared to C# node.
 2020-06-24 10:21:24 +03:00
Roman Khimov
2f8e7e4d33 native: fix getvalidators to match C# implementation 
ValidatorsCount is not initialized at block 0 with C# node (the first voter
initializes it) and until that initialization happens the standby validators
list is being returned as is without sorting.

Fixes state mismatch for the key ffffffff0e00000000000000000000000000000001 in
the first blocks.

It also affects tests as now the first validator is different and it receives
the network fees.
 2020-06-24 10:21:24 +03:00
Anna Shaleva
9f11a55bd6 core: add policy check to mempool 
After block was stored it's possible to have new FeePerByte constraint,
so we should remove all transactions which do not meet this requirement.
Also caching of FeePerByte was added in order not to re-verify
transactions each time mempool needs to be updated.
 2020-06-24 07:58:09 +03:00
Anna Shaleva
ce402a70d2 core: add policy check to (*Blockchain).verifyTx method 
If any of transaction's script hashes for verifying are included into
Policy blocked accounts list, transaction is invalid.
 2020-06-24 07:58:09 +03:00
Anna Shaleva
9097a1a23d smartcontract: update (Parameter).MarshalJSON method 
MarshalJSON should be defined on structure (not pointer), as we use
structures to marshal parameters (e.g. in NotificationEvent and
Invoke of RPC result package) and never use pointers for that purpose.

Also added marshalling of nil array into `[]` instead of `null` to
follow C# implementation.
 2020-06-24 07:58:09 +03:00
Anna Shaleva
8de0332107 *: use MaxTransactionsPerBlock from Policy native contract 
part of #904

1. We now have MaxTransactionsPerBlock set in native Policy contract,
   so this value should be used in (dbft).GetVerified method instead
   of passing it as an argument.
2. Removed (dbft).WithTxPerBlock.
2. DBFT API has changed, so update it's version.
3. Removed MaxTransactionsPerBlock from node configuration, as we
   have it set in native Policy contract.
 2020-06-24 07:58:05 +03:00
Anna Shaleva
c2735a4569 core: add cache to native Policy contract 
We can cache maxBlockSize, maxTransactionsPerBlock and feePerByte
in order to reduce the number of storage requests.
 2020-06-24 07:36:11 +03:00
Anna Shaleva
08cc04c3d5 core: add native policy contract 
part of #904
 2020-06-24 07:35:07 +03:00
Roman Khimov
36a65e3847
Merge pull request #1077 from nspcc-dev/neo3/smartcontract/id 
core: store smartcontract items by id
 2020-06-20 00:12:07 +03:00
Roman Khimov
a2cd2264a8 core: fix HasStorage check in storageGetContext 
It was returning nil which is wrong.
 2020-06-20 00:04:28 +03:00
Anna Shaleva
f1cdcbc99c core: store smartcontract items by id 
closes #1037
 2020-06-19 20:48:45 +03:00
Anna Shaleva
c767ac2973 core: fix NEO native contract id 
It should be set to -1 as in c# implementation.
 2020-06-19 20:44:52 +03:00
Evgenii Stratonikov
a584c5a6a4 native: fix NEP5 contract method prices 
Closes #1073.
 2020-06-19 12:58:33 +03:00
Evgenii Stratonikov
4b31c18316 core: fix Neo.Native.Deploy price 2020-06-19 12:58:33 +03:00
Evgenii Stratonikov
540ac23ca8 core: calculate prices of Neo.Crypt.* opcodes correctly 2020-06-19 12:58:33 +03:00
Evgenii Stratonikov
05dad10e82 native: fix native contract onPersist script 
C# implementation uses NEWARRAY for creating arguments.
Don't change our implementation in `emit`, because PACK is cheaper and
this script must not depend on the internal details of `emit` package anyway.
 2020-06-19 11:58:38 +03:00
Evgenii Stratonikov
81adb7ce83 core: set price getter for native contract persist routine 2020-06-19 11:38:56 +03:00
Roman Khimov
93b0011988
Merge pull request #1069 from nspcc-dev/addr-locking-and-other-fixes 
Addr, locking and other fixes
 2020-06-18 23:29:19 +03:00
Evgenii Stratonikov
e7d8bb92f6 core/tests: extend native contract test 
Check if native contract's method price is taken into account.
 2020-06-18 22:44:10 +03:00
Evgenii Stratonikov
5354352d63 core: remove transaction priority 
There is no such thing as high/low priority transactions, as there are
no free transactions anymore and they are ordered by fees contained
in transaction itself.

Closes #1063.
 2020-06-18 22:44:10 +03:00
Evgenii Stratonikov
2f724b792c core: remove config fields related to free transactions 
In NEO3 every transaction must have some gas attached.

Closes #1064.
 2020-06-18 22:44:08 +03:00
Roman Khimov
1081791c68 core: fix contract manifest unpacking in System.Contract.Create 
It's just JSON, io.Serializable is only used for DB storage where the length
should be obtained from the stream. Fixes:

2020-06-18T22:14:10.571+0300    WARN    contract invocation failed      {"tx": "1ffd475a9c246495d6206cb80a9a78e9d14a433ded60cd37aa87d897655606e1", "block": 25893, "error": "error encountered at instruction 3696 (SYSCALL): failed to invoke syscall: invalid character ':' after top-level value"}
 2020-06-18 22:38:34 +03:00
Evgenii Stratonikov
84387b2416 core: set gas limit to transaction.SystemFee for all transactions 2020-06-18 22:17:48 +03:00
Roman Khimov
90d3bcbdc3 state: initialize PublicKey before decoding into it, fix #1061 
These are pointers and decoding into a nil pointer is a bad idea.
 2020-06-18 21:59:58 +03:00
Roman Khimov
dd6895b103 native: optimize GAS onPersist for empty blocks 
There is no need to get a list of validators when there are no transactions in
the block.
 2020-06-18 21:51:29 +03:00
Roman Khimov
f0fbb2d8ff core: prevent panic on forced exit 
If we're to close the Blockchain while it's storing a block this might happen:

panic: assignment to entry in nil map

goroutine 63 [running]:
github.com/nspcc-dev/neo-go/pkg/core/storage.(*MemoryStore).put(...)
        /home/rik/dev/neo-go/pkg/core/storage/memory_store.go:53
github.com/nspcc-dev/neo-go/pkg/core/storage.(*MemCachedStore).Persist(0xc000673a40, 0x0, 0x0, 0x0)
        /home/rik/dev/neo-go/pkg/core/storage/memcached_store.go:118 +0x245
github.com/nspcc-dev/neo-go/pkg/core/dao.(*Simple).Persist(0xc003dbdcc0, 0xc003b18900, 0xc002505538, 0xc001c510e0)
        /home/rik/dev/neo-go/pkg/core/dao/dao.go:543 +0x2e
github.com/nspcc-dev/neo-go/pkg/core/dao.(*Cached).Persist(0xc003b189c0, 0xc000240000, 0x0, 0x0)
        /home/rik/dev/neo-go/pkg/core/dao/cacheddao.go:169 +0x756
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).storeBlock(0xc0001f4000, 0xc003611040, 0x0, 0x10)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:647 +0xfa2
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddBlock(0xc0001f4000, 0xc003611040, 0x0, 0x0)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:447 +0xee
github.com/nspcc-dev/neo-go/pkg/network.(*blockQueue).run(0xc0006f4120)
        /home/rik/dev/neo-go/pkg/network/blockqueue.go:48 +0x158
created by github.com/nspcc-dev/neo-go/pkg/network.(*Server).Start
        /home/rik/dev/neo-go/pkg/network/server.go:179 +0x2b5
 2020-06-18 21:35:17 +03:00
Evgenii Stratonikov
8a0b2be285 core: process NEP5 transfers emitted by Native.onPersist 
All GAS is minted/burnt during `onPersist` call.
 2020-06-18 16:38:08 +03:00
Evgenii Stratonikov
3e76b9a9bf native: fix Transfer event in burn 
Tokens should subtract, but transfer amount is positive.
 2020-06-18 16:19:09 +03:00
Evgenii Stratonikov
295d8fc70e core: save application logs for native persist 2020-06-18 15:32:27 +03:00
Evgenii Stratonikov
3894b6db9f native: take into account native contract method prices 
Burn gas right before function invoke.
 2020-06-18 15:32:27 +03:00
Evgenii Stratonikov
031fd93172 native: fix native contract prices 2020-06-18 15:32:27 +03:00
Evgenii Stratonikov
0fa4c49735 core,native: persist native contract via VM 
After native contracts are deployed, single persist script is created
and executed at the start of every block persisting.
 2020-06-18 15:32:27 +03:00
Roman Khimov
b483c38593 block/transaction: add network magic into the hash 
We make it explicit in the appropriate Block/Transaction structures, not via a
singleton as C# node does. I think this approach has a bit more potential and
allows better packages reuse for different purposes.
 2020-06-18 12:39:50 +03:00
Roman Khimov
8fda6a3407 block: fix ConsensusData hashing 
It's DoubleSha256 as with the rest of the structures.
 2020-06-18 12:22:49 +03:00
Roman Khimov
a7cce3f894 smartcontract: use new VerifiableDecodable for ParameterContext 
And implement it for Transaction, the only user of ParameterContext for
now. Which make correct signing/verifying possible for cases when
serialization for general transmission and signing differ.
 2020-06-18 12:12:56 +03:00
Roman Khimov
0002b65238 block: it's not a padding, but a witness count in the base 
At least that's the interpretation C# codebase has, so be a bit more correct
in error message.
 2020-06-18 12:12:00 +03:00
Roman Khimov
b9a66c64e1 block: drop paddings from structures 
It makes no sense at all storing them in a structure, they're just a part of
encoding/decoding process.
 2020-06-18 12:11:41 +03:00
Roman Khimov
26f11a52d9 config: move NetMode into its own micropackage 
It's going to be used a bit more and pulling whole config just for one type is
a bit wrong.
 2020-06-18 12:09:57 +03:00
Roman Khimov
5f276de003
Merge pull request #1057 from nspcc-dev/fix/newaddress 
Update addresses to NEO3 format
 2020-06-17 16:25:32 +03:00
Evgenii Stratonikov
7b4ca57e33 *: change address to the new format 
NEO3 uses new prefix for address (53 = 0x35), thus string representations as
well as encrypted WIFs should be changed.
 2020-06-17 15:58:21 +03:00
Evgenii Stratonikov
3762ebdd08 core: implement System.Runtime.GetInvocationCounter syscall 2020-06-17 11:24:11 +03:00
Evgenii Stratonikov
75e597f880 core: implement System.Runtime.GetNotifications syscall 2020-06-17 11:24:11 +03:00
Evgenii Stratonikov
a4e4439967 core,vm: implement System.Runtime.GasLeft syscall 2020-06-17 11:24:11 +03:00
Evgenii Stratonikov
a7d4fff897 vm: make (*VM).GasLimit public 
VM is inherently single-threaded and replacing setter/getter methods
with public field simplifies code a bit.
 2020-06-16 15:13:20 +03:00
Evgenii Stratonikov
1f97f3abd8 core: adjust prices and flags for System.Runtime.* interops 
Related #1055.
 2020-06-16 12:30:55 +03:00
Evgenii Stratonikov
ad2a75a500 core: move System.ExecutionEngine.* interops to System.Runtime.* 2020-06-16 12:30:55 +03:00
Evgenii Stratonikov
52a27e9be4 core: implement System.Contract.CreateStandardAccount syscall 2020-06-16 12:12:29 +03:00
Evgenii Stratonikov
5b4f38d717 core: implement System.Contract.IsStandard syscall 2020-06-16 12:12:27 +03:00
Evgenii Stratonikov
b1d2d70c02 core: adjust prices for blockchain/contract syscalls 2020-06-16 12:04:57 +03:00
Evgenii Stratonikov
6339efac11 core: implement System.Json.* interops 2020-06-16 11:35:06 +03:00
Evgenii Stratonikov
aae312d408 core,vm: adjust prices for iterator/enumerator syscalls 
Closes #1027.
 2020-06-16 11:05:26 +03:00
Evgenii Stratonikov
18066143bd core,vm: adjust binary (de-)serialization syscalls 
Related #1027.

1. Move System.Runtime.(De)serialize to System.Binary.*
2. Rename compiler stubs.
3. Adjust opcode prices.
 2020-06-16 11:00:38 +03:00
Roman Khimov
b3507d490d
Merge pull request #1051 from nspcc-dev/neo3/rpc/scopes 
core: add json marshaller for Cosigner's scopes
 2020-06-15 16:13:48 +03:00
Anna Shaleva
3568ab3d6d core: add json marshaller for Cosigner's scopes 
Following C# implementation, we should marshal Scopes as a set of
strings instead of single byte.
 2020-06-15 13:48:15 +03:00
Evgenii Stratonikov
2e996ea57d core: adjust System.Storage.* interop prices 2020-06-15 12:02:28 +03:00
Evgenii Stratonikov
61bba1a39c core: store precise gas price in interop descriptions 
In NEO3 sycalls can have prices less than 10^5, we need to store them
precisely. Adjusting actual prices is to be done in the following tasks.
 2020-06-15 12:02:28 +03:00
Evgenii Stratonikov
fd2cd291e7 core: calculate opcode prices correctly 2020-06-15 12:02:22 +03:00
Evgenii Stratonikov
8f20a70969 core: calculate a price of System.Storage.Put correctly 
It also depends on the item already stored by key.
 2020-06-15 12:02:21 +03:00
Roman Khimov
fe31c7ed2d
Merge pull request #1047 from nspcc-dev/neo3/rpc/invoke 
rpc: update invoke* RPC-calls
 2020-06-11 21:39:20 +03:00
Anna Shaleva
9e7fca013e rpc: update CLI and RPC client invoke* calls 
part of #1036
 2020-06-11 20:32:05 +03:00
Anna Shaleva
d5355acfa9 rpc: update RPC server invoke* calls 
part of #1036
 2020-06-11 19:34:13 +03:00
Evgenii Stratonikov
971ab0646f core: specify allowed triggers for interops 
Related #1026, #1027, #1028, #1031.
 2020-06-11 13:16:07 +03:00
Evgenii Stratonikov
b12add5a78 smartcontract: rewrite trigger types for NEO3 2020-06-11 13:16:07 +03:00
Evgenii Stratonikov
a3b876d55c native: specify call flags for native contract methods 2020-06-11 13:16:07 +03:00
Evgenii Stratonikov
0dd00a49f5 core: specify require call flags in interop description 
Related #1026, #1027, #1028, #1031.
 2020-06-11 13:16:07 +03:00
Evgenii Stratonikov
bda94c74c3 vm: check calling flags on syscall invocation 2020-06-11 13:16:07 +03:00
Evgenii Stratonikov
55ab7535be core: optimize contractCallEx 
Provide script hash upon script load.
 2020-06-11 13:06:10 +03:00
Evgenii Stratonikov
a86e1910e1 core: remove unused interops 
Missed in #1040.
 2020-06-11 10:51:19 +03:00
Evgenii Stratonikov
f8a11f61b6 core: update *.Contract.* interops 
1. Remove GetScript, IsPayable, GetStorageContext.
2. Revert 82319538 related to GetStorageContext.
3. Rename Migrate to Update.
4. Move remaining to System.Contract.*.

Related #1031.
 2020-06-11 10:50:35 +03:00
Evgenii Stratonikov
c69f8a2fa3 core: check for permission in System.Contract.Call(Ex) 2020-06-11 10:45:25 +03:00
Evgenii Stratonikov
5514b3f52f smartcontract,vm: remove DynamicInvoke feature 
It doesn't exist in NEO3.
 2020-06-11 10:45:25 +03:00
Evgenii Stratonikov
a03af55732 core: provide monotonically increasing ID for contracts 
ID is a contract identifier which doesn't change during migration.
It is also used as a part of storage context instead of contract hash.
 2020-06-11 10:45:25 +03:00
Evgenii Stratonikov
20616cc97f interop: perform contract checks in CheckWitness 2020-06-11 10:45:25 +03:00
Evgenii Stratonikov
7256efd1ed vm: implement (*VM).AddGas 
Calculating interop prices can be tricky. It is more convenitent to burn
gas inside interops where necessary parameters are popped.
 2020-06-11 10:45:25 +03:00
Evgenii Stratonikov
df958caf93 core: add Manifest to state.Contract 2020-06-11 10:45:24 +03:00
Evgenii Stratonikov
425277098c native: set ContractID for NEO & GAS contracts 2020-06-10 14:23:25 +03:00
Evgenii Stratonikov
38d020d1a2 core: move Neo.Storage.* interops to System.* 2020-06-10 12:13:35 +03:00
Evgenii Stratonikov
0472a0b0b1 core: move Neo.Runtime/Enumerator/Iterator.* interops to System.* 2020-06-10 12:13:35 +03:00
Evgenii Stratonikov
97c6ac7a73 core: move Neo.Blockchain.* syscalls to System.* 2020-06-10 12:13:18 +03:00
Evgenii Stratonikov
3d3fe9398e core: remove Neo.Header/Witness/Account.* interops 
They are not present in NEO3.
 2020-06-10 12:13:18 +03:00
Anna Shaleva
8b7abd36c9 core: remove Block.GetTransactions, Block.GetTransactionsCount interops 
Updated System.Blockchain.GetBlock interop replaced the functionality of
the following interops:
	System.Block.GetTransactions
	System.Block.GetTransactionCount
	Neo.Block.GetTransactions
	Neo.Block.GetTransactionsCount
 2020-06-09 23:24:11 +03:00
Anna Shaleva
7a2d37cf7e core: update System.Blockchain.GetBlock interop 
closes #1025

Now we put on stack stackitem.Array instead of Interop, so we're able to
use all available block properties without extra interop getters.
Removed Neo.Blockchain.GetBlock interop as we don't need it anymore.
 2020-06-09 23:24:04 +03:00
Anna Shaleva
e2187c0a96 core: remove Block.GetTransaction interops 
Removed Neo.Block.GetTransaction and System.Block.GetTransaction
interops. These interops were replaced by new
System.Blockchain.GetTransactionFromBlock interop.
 2020-06-09 22:54:21 +03:00
Anna Shaleva
d692de5ea4 core: add System.Blockchain.GetTransactionFromBlock interop 2020-06-09 22:54:16 +03:00
Anna Shaleva
1e63ae4b3f core: update System.Blockchain.GetTransactionHeight interop 
Update System.Blockchain.GetTransactionHeight and removed
Neo.Blockchain.GetTransactionHeight interop as we don't need it.
 2020-06-09 22:04:18 +03:00
Anna Shaleva
53655c5ac2 core: implement new System.Blockchain.GetTransaction interop 
closes #1023

Now we put on stack stackitem.Array instead of Interop, so we don't
need old transaction-related interops anymore. Removed the following
interops:
	System.Transaction.GetHash
	Neo.Transaction.GetAttributes
	Neo.Transaction.GetHash
	Neo.Transaction.GetWitnesses
	Neo.Attribute.GetData
	Neo.Attribute.GetUsage

Also removed the following duplicated NEO interop:
	Neo.Blockchain.GetTransaction
 2020-06-09 22:04:13 +03:00
Anna Shaleva
f2e3be6fef core: remove old AntShares interops 
We don't need them anymore.
 2020-06-09 20:54:00 +03:00
Roman Khimov
795523f5cd
Merge pull request #1013 from nspcc-dev/neo3/vm/stackitem_refactoring 
vm: move StackItem to a separate package
 2020-06-08 15:30:44 +03:00
Anna Shaleva
4bf103ca12 core: update native contract data serialisation format 
closes #908
 2020-06-08 13:29:35 +03:00
Anna Shaleva
7ca2807875 vm/encoding: move bigint from vm to encoding package 2020-06-08 13:27:13 +03:00
Anna Shaleva
783f5ecb01 vm: move StackItem to a separate package 
closes #912
 2020-06-08 13:27:08 +03:00
Evgenii Stratonikov
138385e512 transaction: encode Script in base64 2020-06-08 10:31:51 +03:00
Evgenii Stratonikov
1fa25efa79 transaction: encode witness scripts in base64 2020-06-08 10:23:56 +03:00
Roman Khimov
63f4f34659 mempool: drop TxWithFee type 
All the fees are in transaction, this makes no sense.
 2020-06-05 19:21:40 +03:00
Roman Khimov
a986e2a064 *: drop support for old on-chain assets 
You no longer can transfer them, so creating/renewing/storing doesn't make
much sense.
 2020-06-05 19:21:37 +03:00
Roman Khimov
bf6435eeaa storage: drop STValidator prefix, it's not used 2020-06-05 19:20:16 +03:00
Roman Khimov
709146f295 transaction: drop Inputs and Outputs, forget UTXO 2020-06-05 19:20:16 +03:00
Roman Khimov
21efccd300 transaction: remove type field, set Version to 0 
Two changes being done here, because they require a lot of updates to
tests. Now we're back into version 0 and we only have one type of
transaction.

It also removes GetType and GetScript interops, both are obsolete in Neo 3.
 2020-06-05 19:20:16 +03:00
Roman Khimov
dfb26f9ab2 block: don't attempt verification if there was a decoding error 2020-06-05 19:20:16 +03:00
Roman Khimov
f63b286c35 core: update genesis block to store timestamp in ms 2020-06-05 19:20:16 +03:00
Roman Khimov
35b30ccfdf core: drop OLDPUSH1, it's no longer needed 
Fixes #927.
 2020-06-05 19:20:16 +03:00
Roman Khimov
0a09a20900 transaction: drop Register transaction type 
And everything associated like SystemFee configuration.
 2020-06-05 19:20:16 +03:00
Roman Khimov
169c5ae775 transaction: drop Issue TX support 2020-06-05 19:20:16 +03:00
Roman Khimov
f445f7c602 transaction: drop Contract transaction type 2020-06-05 19:20:16 +03:00
Roman Khimov
6853470603 block: precompute ConsensusData hash on UnmarshalJSON 
That's what we usually do for block or transaction hashes.
 2020-06-05 19:20:16 +03:00
Roman Khimov
dfc7a9bfd1 transaction: drop Claim TX type 2020-06-05 19:20:16 +03:00
Roman Khimov
3d18f09def core: fix CalculateClaimable for NEP5 NEO 
It's not stored as Fixed8, so calculations need to be adjusted for that.
 2020-06-05 19:20:16 +03:00
Roman Khimov
39dfebccc4 core: no longer treat sysfee as claimable 
As it's not on Neo 3, it just gets burned and that's it. Only network fee is
being redistributed to CNs.
 2020-06-05 19:20:16 +03:00
Roman Khimov
50ed4c5967 core: don't return an error from CalculateClaimable 
As it never returns one.
 2020-06-05 19:20:16 +03:00
Roman Khimov
1bc6d29c0d core: fix potential NPE in GetUtilityTokenBalance 2020-06-05 19:20:16 +03:00
Roman Khimov
9f5fd21160 core: drop UXTO NEO and GAS transactions from the genesis block 
They're no longer required. Adjust test chain for that, now it only has
invocations, transfers NEO/GAS and pays the fees with NEP5 GAS.
 2020-06-05 19:20:16 +03:00
Roman Khimov
a9955719da native: make mint/burn for 0 no-ops 
Prevent emitting useless transfer events and doing useless gets/puts.
 2020-06-05 19:20:16 +03:00
Roman Khimov
657bb7575e rpc: change getunclaimed to getunclaimedgas 
getunclaimed doesn't exist on Neo 3 and getunclaimedgas works for NEP5 GAS.
 2020-06-05 19:20:16 +03:00
Roman Khimov
31ef0c4339 core: don't check systemfee with UTXO outputs 
SystemFee is already being checked by verifyTx against Sender's balance.
 2020-06-05 19:20:16 +03:00
Roman Khimov
d9400800e3 transaction: drop Gas field from InvocationTX 
It essentialy is the new SystemFee, so use that. Had to increase GAS transfer
in test chain to 1000 to pay for deployment.
 2020-06-05 19:20:16 +03:00
Roman Khimov
30c74152ce core: add balance checks for NEP5 transfer to TestCreateBasicChain 2020-06-05 19:20:16 +03:00
Roman Khimov
0a35ee00c6 core: print txes in LE in TestCreateBasicChain 
LE is human and RPC compatible representation, BE hex is almost unusable.
 2020-06-05 19:20:16 +03:00
Roman Khimov
bf6525e54d core: fix GetUtilityTokenBalance 
The value stored in NEP5 tracker for GAS is Fixed8 already, so we need to
type-cast it, but not multiply by 10⁸ again.
 2020-06-05 19:20:16 +03:00
Roman Khimov
7e586cc6ab native: deduce transferred assets from sender on transfer 
When money is being sent usually they go away from someone's pocket, so that
there is a little less money left there. Not in our case as it turns out, we
actually were adding money both to sender and receiver which is nice, but a
bit different from usual economic's expectations.
 2020-06-05 19:20:16 +03:00
Anna Shaleva
8fb67615f8 core: sort results of Storage.Find and NEO.getRegisterValidators 
Closes #909
We need to match elements order with C# implementation.
 2020-05-29 17:33:01 +03:00
Evgenii Stratonikov
44709cf6fe core: make GetUnspentCoins interop return array, fix #978. 2020-05-27 12:01:50 +03:00
Roman Khimov
8231953842 core: fix Neo.Contract.GetStorageContext security check 
This syscall should only work for contracts created by current transaction and
that is what is supposed to be checked here. Do so by looking at the
differences between ic.dao and original lower DAO.
 2020-05-26 19:58:49 +03:00
Roman Khimov
7633439845 rpc/block: rework the way Block is JSONized 
Our block.Block was JSONized in a bit different fashion than result.Block in
its NextConsensus and Index fields. It's not good for notifications because
third-party clients would probably expect to see the same format. Also, using
completely different Block representation is probably making our client a bit
weaker as this representation is harder to use with other neo-go components.

So use the same approach we took for Transactions and wrap block.Block which is
to be serialized in proper way.

Fix `Script` JSONization along the way, 3.0 node wraps it within `witnesses`.
 2020-05-26 11:36:47 +03:00
Roman Khimov
2dc16c0694 transaction: add json.Unmarshaler to Attribute 
It actually was missing and it might affect Transaction conversion to/from
JSON.
 2020-05-26 11:36:47 +03:00
Roman Khimov
44ae9086b6 core: improve and fix locking in storeBlock 
Getting batch, updating Prometheus metrics and pushing events doesn't require
any locking: batch is a local cache batch that no one outside cares about,
Prometheus metrics are not critical to be in perfect sync and events are
asynchronous anyway.

Native contracts also don't require any locks and they should be processed
before dumping storage changes.
 2020-05-26 11:36:47 +03:00
Roman Khimov
3ca71d5e8d core: add Blockchain event subscription mechanism 
A deep internal part of #895. Blockchainer interface is also extended for
various uses of these methods.
 2020-05-25 00:27:39 +03:00
Roman Khimov
ac40d357f0 core: improve documentation a little 2020-05-24 23:54:01 +03:00
Roman Khimov
c9b1d359d8 core: ensure we produce correct blocks for tests 
Check for correct merkle root generation.
 2020-05-24 23:52:54 +03:00
Anna Shaleva
9f2cb1726b core: add witness check to nep5 transfer method 
We have to check, whether transaction's sender is authorised to transfer
nep5 assets.
 2020-05-20 23:34:27 +03:00
Anna Shaleva
1afd630169 core: add senders' fees monitor to mempool 
In order to simplify maintainance of overall current senders` fees
in the mempool we are to keep them in a separate map.
 2020-05-20 23:34:25 +03:00
Anna Shaleva
7acf5b2841 core: add SystemFee and NetworkFee to transaction 
closes #831
 2020-05-20 23:26:48 +03:00
Anna Shaleva
fc3acdc80c core: add native contracts' hashes getters 2020-05-20 23:16:04 +03:00
Anna Shaleva
6b2f4efdd4 core: fix bug in nep5TokenNative.addTokens method 
Sometimes we want to add negative amount (e.g. when burning tokens).
 2020-05-20 23:16:04 +03:00
Anna Shaleva
c7a040ff81 core: fix bug with native contracts methods initialisation 
`NewNEO()` and `NewGAS()` methods are trying to initialise
both `onPersist` and `incBalance` methods of NEO and GAS AFTER
nep5TokenNative is set to the VALUE of created nep5 token.
In this situation an attemmpt to call the corresponding native contracts
methods (e.g. transfer native GAS) leads to contract invocation failure,
as far as `nep5TokenNative.incBalance` method is nil.

Fixed this by initializing both `onPersist` and `incBalance` methods
before getting the value of nep5 contract.
 2020-05-20 23:16:04 +03:00
Anna Shaleva
4ebac5a069 core: fix bug in nep5 notifications handling 
Native contracts deployment creates `Transfer` notifications and adds
them into interop context. However, these notifications were not stored
for two reasons:
1. typo in `Transfer` (so these notifications were not recognised during
processing of the invocation tx in (*Blockchain).storeBlock(...) method)
2. these notifications have `from` adress setted to null, so conversion
to []byte fails. Same thing could happen with `to`.

Related C# issue: https://github.com/neo-project/neo/issues/1646

For now, made both `transfer` and `Transfer` valid.
 2020-05-20 23:16:04 +03:00
Evgenii Stratonikov
c0147c76ac emit: remove tailCall argument from AppCall 
There is no TAILCALL opcode in NEO3.
 2020-05-07 14:59:01 +03:00
Evgenii Stratonikov
73c82584a3 vm,compiler: replace APPCALL with System.Contract.Call 
Contract calls are performed via syscall System.Contract.Call
in NEO3. This implements this in compiler and removes APPCALL from the
VM.
 2020-05-07 14:52:03 +03:00
Evgenii Stratonikov
ec900c7ff7 core: implement System.Contract.Call interop 2020-05-07 14:39:06 +03:00
Evgenii Stratonikov
301c1b7601 native: save native contract state on deploy 2020-05-07 14:03:14 +03:00
Evgenii Stratonikov
f9f3192b48 vm: move RET and SYSCALL opcodes 2020-05-07 13:23:05 +03:00
Evgenii Stratonikov
5ab4b9284b core: remove unused function from tests 2020-05-06 16:00:16 +03:00
Evgenii Stratonikov
977c431bf1 vm: implement ASSERT/ABORT opcodes 
Rename THROWIFNOT to ASSERT, add ABORT opcode.
ABORT cannot be caught, but the implementation should be postponed until
exception handling is implemented.
 2020-05-06 15:54:19 +03:00
Anna Shaleva
5ece9922c1 vm: hide GetContextScriptHash() method 
After adding ScriptHashGetter interface to vm, there's no need in
GetContextScriptHash() to be exported.
 2020-05-04 11:53:31 +03:00
Anna Shaleva
73167999cc core: add cosigners field to transaction 
closes #864
 2020-05-04 11:53:31 +03:00
Anna Shaleva
d1ec01c45e util: implement Serializable interface over Uint160 2020-05-04 11:49:14 +03:00
Anna Shaleva
08f5708edb core: remove Script attribute type 2020-05-04 11:49:14 +03:00
Evgenii Stratonikov
519b31a704 vm: remove crypto-related opcodes 
All cryptography has moved to interops in NEO3.
There is no SHA256 interop RN, but it is to appear later.
Closes #777.
 2020-04-29 19:16:38 +03:00
Roman Khimov
c1aa96d614
Merge pull request #911 from nspcc-dev/feature/convert 
vm: implement CONVERT opcode
 2020-04-28 18:08:36 +03:00
Evgenii Stratonikov
be38798785 vm: move Bool method to the StackItem interface 2020-04-28 15:13:41 +03:00
Anna Shaleva
fa467ce628 core: add txes duplication check on block.DecodeBinary 2020-04-27 18:00:01 +03:00
Anna Shaleva
29d321b5e1 *: drop miner transaction 
1. Completely remove miner transaction

2. Change validation rule for block: block without transactions is
valid.
 2020-04-27 17:57:37 +03:00
Anna Shaleva
55fd9f8d24 core: change block.ConsensusData to neo3 format 
1. Dropped `Base.ConsensusData` block field

2. Added `Block.ConsensusData` field with `Nonce` and `PrimaryIndex`

3. Removed "Neo.Header.GetConsensusData" and
"AntShares.Header.GetConsensusData" interops
 2020-04-27 17:57:37 +03:00
Anna Shaleva
0de5cb1bde core, consensus: nanoseconds-precision timestamp 
Keep timestamp of consensus messages in nanoseconds-precision state
 2020-04-27 17:57:32 +03:00
Anna Shaleva
aa554f0a9a core: update block timestamp format 
Changed block.Timestamp from uint32 to uint64
 2020-04-27 17:56:44 +03:00
Roman Khimov
e6f5cffff6 transaction: drop Enrollment and State types 
They're completely replaced now by the NEO native contract voting system.
 2020-04-27 16:06:58 +03:00
Roman Khimov
b83e84ca08 core: switch to the new voting system (via native NEO contract) 
It has all the methods required now, so you can register, vote and get
voting results. Fixes #865.
 2020-04-27 16:06:50 +03:00
Roman Khimov
2fa3bdf6a9 core/native: move Votes from account to native NEO state 2020-04-27 12:30:39 +03:00
Roman Khimov
bc4a6a6bab core/native: move NextValidators storage to NEO contract 2020-04-27 12:30:39 +03:00
Roman Khimov
66c80d429e native: update voting information when changing Neo balance 2020-04-27 12:30:39 +03:00
Roman Khimov
36c6c6690b native: distribute GAS even for empty transfers 
As it's one of the use cases.
 2020-04-27 12:30:39 +03:00
Roman Khimov
064636768b core/native: move ValidatorsCount processing into native NEO contract 2020-04-27 12:30:39 +03:00
Roman Khimov
3476a18fa9 core/native: store validators in NEO native contract state 
This technically breaks voting with UTXO-based NEO (processTXWithValidators*),
but we're moving towards the new system.
 2020-04-27 12:30:39 +03:00
Roman Khimov
4e8ee697ee native: store NEO and GAS state in the storage 
As it should be done (although current serialization format is not quite
right).
 2020-04-27 12:30:39 +03:00
Roman Khimov
8c02c6b22c native: put proper Null StackItem for transfer event 
nil is not a good StackItem, we have proper VM-level Null for this.
 2020-04-27 12:30:39 +03:00
Roman Khimov
1dcace1594 native: don't distribute gas at block 0 
It fails at the moment and it doesn't make sense at conceptual level.
 2020-04-27 12:30:39 +03:00
Roman Khimov
30836ca69b core/native: untangle native contracts initialization 
The notion of NativeContractState shouldn't ever existed, native contract is a
contract and its state is saved as regular contract state which is critical
because we'll have MPT calculations over this state soon.

Initial minting should be done in Neo.Native.Deploy because it generates
notification that should have proper transaction context.

RegisterNative() shouldn't exist as a public method, native contracts are only
registered at block 0 and they can do it internally, no outside user should be
able to mess with it.

Move some structures from `native` package to `interop` also to avoid circular
references as interop.Context has to have a list of native contracts (exposing
them via Blockchainer is again too dangerous, it's too powerful tool).
 2020-04-27 12:30:39 +03:00
Anna Shaleva
869c7d6afa core: init native interops in the genesis block 
closes #836
 2020-04-27 12:30:39 +03:00
Evgenii Stratonikov
7caa81a81c rpc/test: generate block for submitblock on-the-fly 
There is no need to provide hex when chain is dumped.
 2020-04-24 08:54:22 +03:00
Evgenii Stratonikov
0f17402599 testchain: implement Sign function 
Sign any data by all consensus nodes.
 2020-04-23 17:09:09 +03:00
Evgenii Stratonikov
008e6eb233 vm: implement new PUSH opcodes 2020-04-23 10:52:28 +03:00
Evgenii Stratonikov
cc20ba651d core: use opcodes instead of raw bytes in tests 
It will be helpful during future opcode reordering in NEO3.
 2020-04-23 10:37:01 +03:00
Evgenii Stratonikov
351fdd469f core: calcultate NEO owner on-the-fly 2020-04-23 10:36:44 +03:00
Evgenii Stratonikov
1e50016a91 core,test: move helper functions for testchain to the internal package 
Our test chain is used in core,rpc and (in future) interop packages.
It is better to have all related declarations in one place to avoid
code duplication.
 2020-04-22 18:17:11 +03:00
Evgenii Stratonikov
c508a36827 core,network: shutdown services in tests properly 2020-04-22 17:53:59 +03:00
Anna Shaleva
2b5c14160c core: add sender field to transaction 
closes #860
 2020-04-20 17:21:28 +03:00
Evgenii Stratonikov
cde4ccf01c vm: remove CHECKSIG/VERIFY/CHECKMULTISIG opcodes 2020-04-20 11:55:24 +03:00
Evgenii Stratonikov
4740d937aa vm: dont use SetCheckedHash outside of vm package 2020-04-20 11:55:24 +03:00
Evgenii Stratonikov
941410a840 core: change verification scripts to new format 
Verification scripts now invoke Neo.Crypto.* interops instead of
CHECKSIG/VERIFY opcodes.
 2020-04-20 11:55:24 +03:00
Roman Khimov
ab8296bc57
Merge pull request #857 from nspcc-dev/neo3/verifiable 
core,crypto: implement Verifiable interface
 2020-04-17 13:06:55 +03:00
Evgenii Stratonikov
ceff8736f2 vm: use ID-based syscalls 
In NEO3 SYSCALL opcode has 4-byte ID parameter.
This commit removes support for string-based syscalls and
changes SYSCALL's parameter to be fixed 4-byte value.
 2020-04-17 11:46:31 +03:00
Evgenii Stratonikov
bfbbef952a vm: move InteropNameToID to emit package 2020-04-17 11:46:31 +03:00
Evgenii Stratonikov
82b230f19f core: rename *block.Base.GetHashableData to GetSignedPart() 
This allow to use `Block` as a Verifiable item.
When tx is provided, it is set as an interop's script container.
Otherwise, block is set.
 2020-04-17 11:12:40 +03:00
Evgenii Stratonikov
a92872931c interop/crypto: allow ECDsaVerify to verify ScriptContainer 
When verifying transaction or block, verification script can be
a simple PUSHNULL + SYSCALL, which means that script-encontaining
entity should be verified.
 2020-04-17 11:12:40 +03:00
Evgenii Stratonikov
8f08065a8e interop/crypto: allow ECDsaVerify to accept interop items 
When invokes with interop item on stack, it should check
for the signature of Verifiable item it contains.
 2020-04-17 11:12:40 +03:00
Roman Khimov
888be75445
Merge pull request #863 from nspcc-dev/drop-publish-tx 
Drop publish tx
 2020-04-16 21:23:37 +03:00
Evgenii Stratonikov
76700f31cf core: implement skeletons for NEO/GAS native contracts 2020-04-16 15:55:35 +03:00
Evgenii Stratonikov
25354c44f9 core: implement NativeContract support 2020-04-16 15:55:34 +03:00
Evgenii Stratonikov
9586af32f2 core: move runtime.CheckWitness interop to a separate package 2020-04-16 15:54:58 +03:00
Evgenii Stratonikov
8a2130f5b6 core: extend Blockchainer with GetStandByValidators() 
A list of standby validators can be needed in native contracts.
 2020-04-16 15:54:58 +03:00
Evgenii Stratonikov
b446753c57 core,vm: move get/putContextScriptHash to vm package 2020-04-16 15:54:52 +03:00
Roman Khimov
40991d3be3 transaction: drop AgencyType and VotingType 
I have no idea what are they about.
 2020-04-16 14:44:46 +03:00
Roman Khimov
4912e4f425 *: drop Publish transaction type 
It shouldn't be used even in NEO 2 as it was substituted by contract
deployment interop functions.
 2020-04-16 14:40:20 +03:00
Anna Shaleva
5fa11987d2 core: add validUntilBlock field to transaction 
1. closes #841

2. Commented out test cases where binary transaction are used.
These test cases marked with `TODO NEO3.0: Update binary` and need to be
updated.

3. Updated other tests.

4. Added cache to calculateValidUntilBlock() RPC-client method.
 2020-04-15 13:46:43 +03:00
Roman Khimov
1e3c36433f
Merge pull request #844 from nspcc-dev/neo3/nonce 
core: add nonce field to transaction
 2020-04-14 16:24:02 +03:00
Anna Shaleva
65503aa9b4 core: add nonce field to transaction 
1. Closes #840: added Nonce field to transaction.Transaction and
removed Nonce field from transaction.MinerTx

2. Added following methods to different tx types:
  - NewMinerTx()
  - NewMinerTxWithNonce(...)
  - NewEnrollmentTx(...)
  - NewIssueTx()
  - NewPublishTx(...)
  - NewRegisterTx(...)
  - NewStateTx(...)
in order to avoid code duplication when new transaction is created.

3. Commented out test cases where binary transaction/block are used.
These test cases marked with `TODO NEO3.0: Update binary` and need to be
updated.

4. Updated other tests

5. Added constant Nonce to GoveringTockenTx, UtilityTokenTx and genesis
block to avoid data variability. Also marked with TODO.
 2020-04-14 16:19:41 +03:00
Evgenii Stratonikov
a99e0d7ba6 core: fix typo in TestECDSAVerify 2020-04-14 16:15:50 +03:00
Roman Khimov
ace5614a3d
Merge pull request #783 from nspcc-dev/neo3/movecrypto 
Implement Neo.Crypto.* interops
 2020-04-14 15:54:30 +03:00
Roman Khimov
c40d7aa7f4 core: bump DB version to make it differ from 2.0 branch 2020-04-14 12:54:57 +03:00
Evgenii Stratonikov
c23af595c9 core: implement Neo.Crypto.ECDsaCheckMultisig interop 2020-04-13 13:47:39 +03:00
Evgenii Stratonikov
5a30af2c75 core: implement Neo.Crypto.ECDsaVerify interop 2020-04-13 13:46:31 +03:00
Evgenii Stratonikov
1611ede58c crypto/keys: implement NewPublicKeyFromBytes() 
It is convenient to have a single function instead of
allocating new `PublicKey` and using `DecodeBytes()` on it.
 2020-04-13 13:13:15 +03:00
Anna Shaleva
d1f92a585b core: add NewTransactionFromBytes method to Transaction 
Added `NewTransactionFromBytes(b []byte)` method to
transaction.Transaction in order to avoid code duplication.
 2020-04-13 12:04:43 +03:00
Evgenii Stratonikov
30cc2c5783 core: move enumerator/iterator interops to a separate packages 2020-04-11 10:56:36 +03:00
Evgenii Stratonikov
7ffc6c0936 core: move interopContext to a separate package 2020-04-11 10:56:36 +03:00
Evgenii Stratonikov
90a08986d6 core: fix typo in function doc-comment 2020-04-11 10:56:36 +03:00
Evgenii Stratonikov
89d1f7ce75 core: remove unneeded comment 
Serialization interops are already implemented.
 2020-04-11 10:56:36 +03:00
Evgenii Stratonikov
a7c19d445b core: move Blockchainer interface to a separate package 2020-04-11 10:56:36 +03:00
Evgenii Stratonikov
efddcf3bfe core: remove interop methods from interopContext 
If interops are defined as a separate functions
they can be implemented in a separate packages
which can help us to structure core.
 2020-04-11 10:56:35 +03:00
Anna Shaleva
54cdfe4a23 storage: add support of BadgerDB 
closes #820
 2020-04-09 13:55:59 +03:00
Evgenii Stratonikov
ee0ba9b1b4 core: make SpawnVM a method of context 
spawnVMWithInterops is rather long too type and
it doesn't use Blockchain in any way.
 2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
a71cd0961e core/dao: remove unnecessary slice type 
It is used only once, so a simple `sort.Slice`
invocation will suffice.
 2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
030b7754ad core: move DAO to a separate package 2020-04-08 08:38:44 +03:00
Roman Khimov
9997661998
Merge pull request #821 from nspcc-dev/fix-cache-propagation-to-invocations 
core: wrap cached dao properly, don't miss cached data
 2020-04-06 11:52:41 +03:00
Evgenii Stratonikov
1fcc019bf3 rpc: update test chain 
Also provide info for getblockheader RPC while
generating test chain.
 2020-04-06 09:31:09 +03:00
Roman Khimov
5f09381cf4 core: wrap cached dao properly, don't miss cached data 
Fixes #817 where invoked contract missed updated account information because
it got it one layer below cachedDao used to process the block.
 2020-04-03 10:15:11 +03:00
Roman Khimov
2d0ad30fcf vm: rework Map with internal slice representation 
Which makes iterating over map stable which is important for serialization and
and even fixes occasional test failures. We use the same ordering here as
NEO 3.0 uses, but it should also be fine for NEO 2.0 because it has no
defined order.
 2020-04-01 19:33:53 +03:00
Roman Khimov
9b5dab57e8
Merge pull request #810 from nspcc-dev/refactor/config 
config: move config.go out of config
 2020-04-01 10:29:57 +03:00
Anna Shaleva
5a984fdf88 config: move config.go out of config/ 
closes #423
 2020-03-31 17:55:59 +03:00
Roman Khimov
e41853d0a4
Merge pull request #789 from nspcc-dev/fix/tx_specific_data_marshalling 
rpc: fix marshalling of type-specific tx data
 2020-03-30 16:10:25 +03:00
Anna Shaleva
9c09ad9c89 rpc: fix marshalling of type-specific tx data 
closes #585
 2020-03-30 15:48:50 +03:00
Roman Khimov
237de73dec
Merge pull request #807 from nspcc-dev/optimize-memcached-persist 
storage: optimize (*MemCachedStore).Persist for memory-backed ps
 2020-03-29 13:41:21 +03:00
Roman Khimov
4758de71ec storage: optimize (*MemCachedStore).Persist for memory-backed ps 
Most of the time it's persisted into the MemoryStore or MemCachedStore, when
that's the case there is no real need to go through the Batch mechanism as it
incurs multiple copies of the data.

Importing 1.5M mainnet blocks with verification turned off, before:
real    12m39,484s
user    20m48,300s
sys     2m25,022s

After:
real    11m15,053s
user    18m2,755s
sys     2m4,162s

So it's around 10% improvement which looks good enough.
 2020-03-28 17:21:50 +03:00
Roman Khimov
c738975b7b
Merge pull request #803 from nspcc-dev/feature/appcall 
emit: implement AppCallWithOperationAndArgs
 2020-03-27 13:30:48 +03:00
Evgenii Stratonikov
db2dccf7cb emit: implement AppCallWithOperationAndArgs 
It is nice to have a typical task of calling contract method
with specific arguments incapsulated inside some function.
 2020-03-27 11:05:36 +03:00
Evgenii Stratonikov
0036b3e52b random: make use or random package in tests 
Also implement Bytes/Fill routines for generating byte slices.
 2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
9abda40171 testserdes: implement helpers for encode/decode routines 
Frequently one needs to check if struct serializes/deserializes
properly. This commit implements helpers for such cases including:
1. JSON
2. io.Serializable interface
 2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
fcc4877f43 go.mod: update boltdb to v1.3.4 
Also import lib as declared in it's go.mod:
via `go.etcd.io/bbolt`.
 2020-03-25 17:07:54 +03:00
Evgenii Stratonikov
c992d6c518 core: reuse buffer in (*cacheddao).Persist() 
When serializing multiple accounts, cost of a buffer grow
can become significant. This commit tries to amortize it by
reusing the same buffer in a single `Persist()` call.
 2020-03-25 13:13:54 +03:00
Evgenii Stratonikov
6ed2bd63b9 core: marshal AppExecResult.Stack as an array 
Closes #782.
 2020-03-23 16:27:47 +03:00
Roman Khimov
8400f0add2 mempool: gofmt -s 2020-03-17 16:47:51 +03:00
Roman Khimov
ec76a0bf15 mempool: disallow more than one issue tx at once 
Technically they could conflict for available asset amount, but as they're
very rare (and even can be considered obsolete) we can simplify this check.
 2020-03-17 11:40:01 +03:00
Roman Khimov
d5d0479671 core: verify results of issue transaction 
It shouldn't try to issue more tokens than there is available.
 2020-03-17 11:40:01 +03:00
Roman Khimov
10601cb375 core: add issuer hashes into the verification list for Issue TX 
As it should be done.
 2020-03-17 11:40:00 +03:00
Roman Khimov
dca637d2d0 core: add owner hash to verified list for Register TX 
As it should be done.
 2020-03-17 11:37:02 +03:00
Roman Khimov
77a799f7d0 core: add missing state tx verifications 2020-03-16 20:14:59 +03:00
Roman Khimov
6ede65610d core: add appropriate hashes to check for State TX 
These checks are important for proper transaction verification.
 2020-03-16 19:52:28 +03:00
Evgenii Stratonikov
b193e78def core: remove duplication from IsDoubleClaim/IsDoubleSpend 2020-03-16 14:11:19 +03:00
Evgenii Stratonikov
cdf025bf89 transaction: implement AddVerificationHash() method 2020-03-16 14:11:19 +03:00
Roman Khimov
e9429374aa
Merge pull request #756 from nspcc-dev/feature/unclaimed 
core/state: do not unmarshal Unclaimed balances in account
 2020-03-16 13:21:23 +03:00
Evgenii Stratonikov
e503d1001d core/state: do not unmarshal Unclaimed balances in account 2020-03-16 12:19:32 +03:00
Roman Khimov
e6e8761d35 core: fix contract's state migration and don't swallow errors 
Fixes difference in state changes at mainnet's block 2442790 because contract
migration in b4eb2dc35226e6520ee4e09a56197dff91547b50a7f57edc82930fc18c75dffc
doesn't actually transfer the storage state, it only deletes the old one.

And add an error check just in case.
 2020-03-16 11:52:09 +03:00
Roman Khimov
d9a83373ed
Merge pull request #748 from nspcc-dev/feature/splitnep5 
core: store NEP5 balances separately from account
 2020-03-12 18:31:25 +03:00
Evgenii Stratonikov
ac475940a0 core: cache NEP5Transfers in cached DAO 2020-03-12 17:16:11 +03:00
Evgenii Stratonikov
6fa2a998f4 core: cache NEP5Balances in cached DAO 2020-03-12 17:06:20 +03:00
Evgenii Stratonikov
32401a567e core: store NEP5Transfers in batches 
This is an append-only log which is read only during some RPCs.
It is rather slow to get it from base every time we need to append to
it. This commit stores all NEP5Transfers in batches, so that
only a last batch needs to be unmarshaled during block processing.
 2020-03-12 14:51:12 +03:00
Evgenii Stratonikov
3c6d9653b0 core/state: add Size() method to NEP5TransferLog 2020-03-12 12:58:30 +03:00
Evgenii Stratonikov
df2598c8dc core: store NEP5 balances separately 
There is no need to take and unmarshal an account structure only
to get it's NEP5 balances.
 2020-03-12 12:58:28 +03:00
Roman Khimov
bbd802681e cli: make gas parameter to deployment add gas to the base price 
That's how it was intended to behave originally. One thing questionable here
is contract price (policy thing, basically) being moved to smartcontract
package, but it's probably fine for NEO 2.0 (as it won't change) and we'll
make something better for NEO 3.0.
 2020-03-11 20:34:36 +03:00
Roman Khimov
5f1868af28
Merge pull request #743 from nspcc-dev/optimize-some-db-accesses 
Optimize some db accesses
 2020-03-11 17:33:36 +03:00
Roman Khimov
abd7855890
Merge pull request #742 from nspcc-dev/add-dynamic-appcall 
vm: add support for dynamic invocations in APPCALL
 2020-03-11 17:27:05 +03:00
Roman Khimov
8c902a7223 core: cache UnspentCoins in cachedDao 
1.5M block import time (VerifyBlocks disabled) on AMD Ryzen 5 1600/16GB/HDD,
before:
real    159m16.551s
user    69m58.279s
sys     7m34.334s

after:
real    139m41.836s
user    67m12.477s
sys     6m19.420s

12% which is even a bit more than could be expected from inputs analysis (that
has around 10% cache hits for a block-wide cache), worth doing.
 2020-03-11 12:40:02 +03:00
Roman Khimov
23464401bc core/state: merge spent and unspent coins state, use it to store more things 
This change reduces pressure on DB by doing the following things:
 * not storing additional KV pair for SpentCoin
 * storing Output right in the UnspentCoin, thus eliminating the need to get a
   full transaction from DB

At the same time it makes UnspentCoin more fat and hot, but it should probably
worth it.

Also drop `GetUnspentCoinStateOrNew` as it shouldn't ever existed, UTXOs
can't come out of nowhere.

1.5M block import time (VerifyBlocks disabled) on AMD Ryzen 5 1600/16GB/HDD,
before:
real    302m9.895s
user    96m17.200s
sys     13m37.084s

after:
real    159m16.551s
user    69m58.279s
sys     7m34.334s

So it's almost two-fold which is a great improvement.
 2020-03-11 12:40:02 +03:00
Roman Khimov
e1f194ea7b core: treat state.Coin as a bitfield 
As it was intended to.
 2020-03-11 12:22:57 +03:00
Roman Khimov
377fb382aa core: move (un)SpentCoin structs into the state package 
As they're all about the state.
 2020-03-11 12:22:52 +03:00
Roman Khimov
8318adac56 vm: add support for dynamic invocations in APPCALL 
Fixes #740.
 2020-03-10 17:17:36 +03:00
Evgenii Stratonikov
ebf867b11d state: add a test for NEP5Transfer size 
It's size is used in NEP5TransferLog so we need to be
sure it reflects reality.
 2020-03-10 13:02:14 +03:00
Roman Khimov
ee57b96566
Merge pull request #733 from nspcc-dev/fix-getreferences-interop-regression 
Fix GetReferences interop regression
 2020-03-10 10:27:36 +03:00
Roman Khimov
0e2bda4f21 core: drop txHash from SpentCoinState 
It's a key for it, makes no sense storing it as data.
 2020-03-09 16:58:21 +03:00
Roman Khimov
eb404ceae3 core: fix max contract description limit check 
It differs from other parameters in C# code. Fixes #735.
 2020-03-09 14:18:51 +03:00
Roman Khimov
4587121c7f core: fix error propagation in contract-related interops 
Obvious bug that hides failed contract deployments.
 2020-03-09 14:18:00 +03:00
Roman Khimov
4b83e9a5cd
Merge pull request #732 from nspcc-dev/feature/getvalidators 
rpc: implement getvalidators
 2020-03-07 21:43:38 +03:00
Anna Shaleva
456a2d55fd rpc: implement getvalidators 
closes #714
 2020-03-07 18:05:40 +03:00
Roman Khimov
baeaa3dbe6 core: optimize tx verification, only get references and results once 
Getting references requires DB access and that is expensive.
 2020-03-06 20:15:01 +03:00
Roman Khimov
fe4916f691 core: simplify GetTransactionResults() a bit 
Make less movements.
 2020-03-06 19:26:39 +03:00
Roman Khimov
ced5ddbb9e core: fix wrong references ordering in interop function 
Broken by 9f7018503a. Almost the same problem as
in 01082a8988 (though it is deterministic now,
just not the way the contract expects).
 2020-03-06 19:20:55 +03:00
Roman Khimov
0e8ff558d1
Merge pull request #727 from nspcc-dev/fix-wrong-endian-in-interop-hashes 
core: fix wrong endian used in interop functions
 2020-03-06 19:07:28 +03:00
Roman Khimov
f8eee778f4
Merge pull request #724 from nspcc-dev/feature/submitblock 
rpc: implement submitblock
 2020-03-06 12:08:45 +03:00
Anna Shaleva
a746d8e6e6 rpc: implement submitblock RPC 
closes #344
 2020-03-06 12:03:08 +03:00
Evgenii Stratonikov
40188c5400 core: get rid of unnecessary copies 
There is no need to keep Balances and Unclaimed slices
sorted, we need only to remove a single element.
 2020-03-05 20:42:20 +03:00
Anna Shaleva
d3063c26e1 core: add custom error to blockhain.go 
Add InvalidBlockIndex error to AddBlock func
 2020-03-05 20:30:19 +03:00
Roman Khimov
a9abd3d841 core: fix wrong endian used in interop functions 
C# uses ToArray() or UintXXX(bytes) here which interprets hashes as they
should be interpreted (BE, although they always convert to LE when converting
to String just for the fun of it). It leads to state difference for us at
block 2025204 where even though we have the same value for the key, the key
itself differs, ours:

dd2b538e2a0c1db1ae5061c15be14f916bd1e678e512ffcda6d9499d8e7fe97ee71fd6b8004583d9afe09cc4dadbd5deb63d01e061009b7cffdaa674beae0f930ebe6085af900093e5fe56b34a5c220ccdcf6efc336fc5000000000000000000000000000000000010

theirs:

dd2b538e2a0c1db1ae5061c15be14f916bd1e67861e0013db6ded5dbdac49ce0afd9834500b8d61fe77ee97f8e9d49d9a6cdff12e5009b7cffdaa674beae0f930ebe6085af900093e5fe56b34a5c220ccdcf6efc336fc5000000000000000000000000000000000010

In this key there is a tx hash encoded
(e512ffcda6d9499d8e7fe97ee71fd6b84583d9afe09cc4dadbd5deb63d01e061 in LE used
by all the tools like neoscan).

I love Neo.
 2020-03-05 19:44:09 +03:00
Evgenii Stratonikov
95a8fa234f rpc: implement getnep5transfers RPC 2020-03-05 18:22:40 +03:00
Evgenii Stratonikov
6d270c4550 core,rpc: add NEP5 contract to testdata 
Also transfer tokens between accounts.
 2020-03-05 18:22:40 +03:00
Evgenii Stratonikov
f92fd3c948 core: track NEP5 transfers 2020-03-05 18:22:40 +03:00
Evgenii Stratonikov
e8c4179a9c core: track NEP5 balances 2020-03-05 18:22:19 +03:00
Evgenii Stratonikov
3a510b9dad core: allow transfer amount to be bytes 
VM can produce both big.Int and []byte because they
are converted to each other on demand.
 2020-03-05 15:18:38 +03:00
Roman Khimov
d03b2ef4a1
Merge pull request #706 from nspcc-dev/feature/transfer 
cli: implement transfer from multisig accounts
 2020-03-05 12:28:01 +03:00
Roman Khimov
5c781be08d
Merge pull request #718 from nspcc-dev/fix-invocation-tx-checks-and-fee 
Fix Invocation TX checks and fee
 2020-03-05 11:51:51 +03:00
Roman Khimov
9ac8001c7d core: protect (*Blockchain).keyCache with a lock 
Avoid failures like this:
fatal error: concurrent map writes

goroutine 103 [running]:
runtime.throw(0xca6fe3, 0x15)
        /usr/lib64/go/1.12/src/runtime/panic.go:617 +0x72 fp=0xc000687988 sp=0xc000687958 pc=0x42d8d2
runtime.mapassign(0xb9d4a0, 0xc0000b1f80, 0xc000687a6c, 0x13bd940)
        /usr/lib64/go/1.12/src/runtime/map.go:590 +0x5e3 fp=0xc000687a10 sp=0xc000687988 pc=0x40e2a3
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHashAgainstScript(0xc0000f8000, 0x1300d1f5a3d348be, 0x78607089e4fe9fab, 0xc0a21e4f71, 0xc000d16070, 0x1e2b8881f8178e92, 0xd589cfa965287bb8, 0x24a4e4f292ebc797, 0xb90d3120d878e619, 0xc0001d01e0, ...)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:1866 +0x371 fp=0xc000687ad8 sp=0xc000687a10 pc=0x984ed1
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeaderWitnesses(0xc0000f8000, 0xc000d16000, 0xc00042d950, 0x1, 0xc00008a6c0)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:1914 +0x209 fp=0xc000687c90 sp=0xc000687ad8 pc=0x985999
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeader(0xc0000f8000, 0xc000d16000, 0xc00042d950, 0xf0ee0049d611101f, 0xf8967398a8f9351b)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:1254 +0x235 fp=0xc000687d70 sp=0xc000687c90 pc=0x97e4e5
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).addHeaders(0xc0000f8000, 0x1, 0xc000d12000, 0x7d0, 0x7d0, 0x0, 0x1)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:369 +0x266 fp=0xc000687e68 sp=0xc000687d70 pc=0x973f46
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddHeaders(0xc0000f8000, 0xc000d12000, 0x7d0, 0x7d0, 0x0, 0xc0003640c0)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:338 +0x51 fp=0xc000687eb0 sp=0xc000687e68 pc=0x973cb1
github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleHeadersCmd(0xc00022c140, 0xdd0ca0, 0xc000374000, 0xc00000c020)
        /home/rik/dev/neo-go/pkg/network/server.go:431 +0x74 fp=0xc000687fc0 sp=0xc000687eb0 pc=0x9ce5e4
runtime.goexit()
        /usr/lib64/go/1.12/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000687fc8 sp=0xc000687fc0 pc=0x45ca51
created by github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage
        /home/rik/dev/neo-go/pkg/network/server.go:705 +0xa4a
...
 2020-03-04 20:26:18 +03:00
Evgenii Stratonikov
7eaeb18f18 transaction: marshal ContractTX even if Data is nil 
It contains no information so it doesn't matter anyway.
 2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
634e9483d3 transaction: unmarshal Witness properly 
Both verification and invocation scripts need to
be unmarshaled from hex.
Also fix failing RPC tests: block contains non-pointer
`transaction.Witness` field and (*Witness).MarshalJSON method
is not called.
 2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
33f99104e8 transaction: unmarshal Output properly 
Address is marshaled in base58 and needs to be
decoded accordingly.
 2020-03-04 20:01:26 +03:00
Roman Khimov
81d89cd502 core: fix SystemFee calculation for Invocation TXes 
They have it specified right in the transaction. Unfortunately, this little
change rendered invalid our RPC test chain, but I think it became even better
after it, especially given that chain generation is a nice test by itself, so
it should be running as a regular test.
 2020-03-04 19:23:23 +03:00
Roman Khimov
ae6edf0601 core: add a check for fractional gas in invocation TXes 
It's not allowed in C# code and that's reasonable as it's a sysfee essentially
that can only be integer.
 2020-03-04 19:23:23 +03:00
Roman Khimov
823798514a util: rename Int64Value to IntegralValue, add FractionalValue 
Makes it's interface more clear and allows to easily access fractional part.
 2020-03-04 19:23:23 +03:00
Roman Khimov
7cc847d655 transaction: add some missing InvocationTX decoding checks 
Script must have something inside and GAS can't be negative.
 2020-03-04 19:23:23 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Roman Khimov
59e6dac351
Merge pull request #708 from nspcc-dev/fix/trigger 
core: convert trigger.Type to byte when pushing on stack
 2020-03-03 16:17:01 +03:00
Evgenii Stratonikov
fe3bbbd78a core: convert trigger.Type to byte when pushing on stack 2020-03-03 16:13:19 +03:00
Roman Khimov
c3e73c5b7d core: don't reverify stale headers in addHeader 
During networked synchronization we expect there to be a lot of duplicate
headers received and it makes no sense for us reverifying them.
 2020-03-03 15:34:03 +03:00
Evgenii Stratonikov
05a3625b7d wallet: implement (*Account).SignTx 
It is used in both CLI and RPC.
 2020-03-02 18:03:56 +03:00
Evgenii Stratonikov
6add4f3e50 transaction: disallow negative outputs 
Otherwise it is possible to make outputs which will sum
to the expected value, but steal GAS from some other account.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
9e9d59b49a core: set NetworkFee to 0 for Claim and Miner transactions 
Claim tx have no GAS inputs and a positive output which
can lead to negative network fee.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
de5215a564 core: use GetAccountStateOrNew() in (*Blockchain).GetValidators() 
Target of the transaction output may not yet exist in database.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
1b9968df67 core: optimize CalculateClaimable() 
Because accumulated system fee is stored for every block,
it is easy to calculate sum with just to reads.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
258d8be1dd core: store all accumulated SystemFee with every block 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
f69c8a763b core: store system fee together with block 
Recalculating system fee can be rather costly if done
frequently.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
972e0d8ad1 core: add one more Contract tx to the test chain 
When testing CLI it is useful to have some spent coins
on an account with a known key.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
45b8669b42 core: verify Claim transactions 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
44792ed5f7 core: make test chain format compatible with mainnet 
Write uint32 length before every block.
 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
0e2a1f40ba core: add ContractTX to a testchain generator 
Simple transfer from multisig account to the account
of one of the validators.
 2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
72d72296c3 core: implement (*Transaction).GetSignedPart() 
Marshalling it and taking all but last byte violates incapsulation
and is just wrong in case transaction already contains any witnesses.
 2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
95d9f36c98 core: implement UnclaimedBalance tracking 
To make it easy to get unclaimed coins for the
specified account they must be tracked together.
 2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
7095ec6c51 core: implement (*Blockchain).CalculateClaimable 
Calculating amount of GAS that can be claimed is required
for getclaimable RPC.
 2020-03-02 18:00:00 +03:00
Roman Khimov
252a9f2f31
Merge pull request #690 from nspcc-dev/feature/getapplicationlog 
rpc: implement getapplicationlog RPC
 2020-03-02 17:41:32 +03:00
Anna Shaleva
ff4384d7ff rpc: implement getapplicationlog RPC 
Closes #500
 2020-03-02 17:25:27 +03:00
Anna Shaleva
7d46404e2d smartcontract: turn trigger types into Type 
1) Turn trigger types from byte constants into Type
2) Add auto-generated stringer for future purposes
 2020-03-02 17:25:27 +03:00
Evgenii Stratonikov
a3dacd3b74 tests: replace t.Fatal with require where possible 
This makes tests less verbose and unifies the style
they are written in.
 2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
66f96e3f32 core: shutdown Blockchain gracefully in tests 2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
3a5224344e core: verify headers in AddHeaders() 
Headers can be malformed so public methods should verify them
before adding.
 2020-03-02 17:22:26 +03:00
Evgenii Stratonikov
357bb4ce41 core: get rid of global variables in tests 
It can lead to unnecessary race conditions and is just
a bad practice.
 2020-03-02 17:04:08 +03:00
Evgenii Stratonikov
8ca94e23c8 core: replace makeBlocks() with addBlocks() in tests 
This simplifies tests a bit.
 2020-03-02 17:04:08 +03:00
Evgenii Stratonikov
cbf26bac83 mempool: do not allocate new slice for verified transactions 
Because transactions a iterated in an increasing order,
we can filter slice in-place.
 2020-03-02 10:38:27 +03:00
Roman Khimov
3e2b490025 mempool: rework inputs verification, check Claim txes 
Use more efficient check for Input and also check Claims to avoid double
claiming.
 2020-02-27 13:43:35 +03:00
Roman Khimov
f329de73e8 mempool: fix Remove bug (dropping wrong tx!) 
Uncovered by TestMemPoolAddRemoveWithInputsAndClaims (to be added later)
 2020-02-27 13:43:35 +03:00
Roman Khimov
7d59fa0066
Merge pull request #696 from nspcc-dev/tx-verification-fixes 
Claim and enrollment TX verification fixes
 2020-02-27 12:45:36 +03:00
Roman Khimov
00d199ed85 core: fix NetworkFee calculation 
Add doesn't change the variable state. Thanks to GolangCI for catching this.
 2020-02-27 12:32:56 +03:00
Roman Khimov
7bea6e043e core: add double claim verification check 2020-02-27 12:32:56 +03:00
Roman Khimov
29586f2aa7 core: add duplicate Claim verification check 
Refactor HaveInputsDuplicate() out of the core and Blockchain, it doesn't
depend on the Blockchain state. Make it more efficient.
 2020-02-27 12:32:56 +03:00
Roman Khimov
9b1ad40273 core: drop GetScriptHashesForVerifyingClaim, fix Claim's ScriptHash getter 
Claim transactions should _add_ claims scripthashes to the standard list, not
replace them. And this code is actually very reusable.
 2020-02-27 12:32:56 +03:00
Roman Khimov
9f7018503a core/transaction: use new transaction.InOut for References 
We don't need a map here, use simpler structures.
 2020-02-27 12:00:03 +03:00
Roman Khimov
80de208a68 transaction: unbind GroupInputsByPrevHash from Transaction 
Which allows to use it for ClaimTX. Make it also a bit more efficient as maps
are expensive.
 2020-02-27 12:00:03 +03:00
Roman Khimov
b1b184f77f
Merge pull request #692 from nspcc-dev/persist-timer-fix 
core: reschedule the next persist in a second from previous one
 2020-02-26 10:34:40 +03:00
Roman Khimov
c258adb532 core: move spent coin management out of the inner storeBlock loop 
prevHash == input.PrevHash, so make less DB accesses and more real work. Fix
some bugs along the way:
 * spentCoins structure may already be present in the DB when persisting TX,
   there is nothing wrong with that and we shouldn't overwrite it
 * it's only used for NEO and only to check for claim validity. Thus, when
   processing claim tx the corresponding spentCoins should always be present
   in the DB
 2020-02-25 17:21:09 +03:00
Roman Khimov
36c6b6af14 core: put unspents get/put one loop above in storeBlock 
Everywhere in this code prevHash == input.PrevHash, thus we can easily move
some common code out of the loop saving on DB accesses and
serialization/deserialization.
 2020-02-24 18:53:10 +03:00
Roman Khimov
5c9681508b transaction: strip off a layer of redirection from ClaimTX.Claims 
We don't need a pointer here and this change makes this field compatible with
Transaction.Inputs which is useful in many scenarios.
 2020-02-24 18:22:27 +03:00
Roman Khimov
ed9e9e8590 storage: add locking into (*MemoryStore).Seek 
It was missing there.
 2020-02-24 17:51:50 +03:00
Roman Khimov
b49808b766 core: reschedule the next persist in a second from previous one 
It makes little sense queueing up several persistence goroutines (or actually
even running them concurrently).
 2020-02-24 17:17:25 +03:00
Roman Khimov
d5e6028c13 core: add validator public key into the list of hashes to verify 
That should be done for Enrollment transactions.
 2020-02-21 20:04:16 +03:00
Roman Khimov
a903147b60 keys: rename Signature to GetScriptHash, make it return Uint160 
Signature itself wasn't used at all and its name is very misleading, Uint160
script hash is way more useful.
 2020-02-19 12:19:03 +03:00
Roman Khimov
eb11e5fb11 core: implement basic policying support, fix #370 
Implement mempool and consensus block creation policies, almost the same as
SimplePolicy plugin for C# node provides with two caveats:
 * HighPriorityTxType is not configured and hardcoded to ClaimType
 * BlockedAccounts are not supported

Other than that it allows us to run successfuly as testnet CN, previously our
proposals were rejected because we were proposing blocks with oversized
transactions (that are rejected by PoolTx() now).

Mainnet and testnet configuration files are updated accordingly, but privnet
is left as is with no limits.

Configuration is currently attached to the Blockchain and so is the code that
does policying, it may be moved somewhere in the future, but it works for
now.
 2020-02-19 12:19:02 +03:00
Roman Khimov
22f5667530 mempool: return fee along with tx when requesting tx 
Users of GetVerifiedTransactions() don't want to recalculate tx fee and it's
nice to have it returned from TryGetValue() also sometimes.
 2020-02-19 12:19:02 +03:00
Roman Khimov
06daeb44f3 core: make IsLowPriority work with pre-calculated fee 
Don't recalculate it again and again.
 2020-02-19 12:19:02 +03:00
Roman Khimov
a4294f4b5f core: export UtilityTokenID and GoverningTokenID 
Both are very useful outside of the core, this change also makes respective
transactions initialize with the package as they don't depend on any kind of
input and it makes no sense recreating them again and again on every use.
 2020-02-19 12:13:27 +03:00
Roman Khimov
b41a5239c6 core: always sort the result of GetValidators 
It wasn't sorted when all validators were elected. There is also no need to do
`Unique()` on the result because validators are distinguished by the key, so
no two registered validators can have the same key.
 2020-02-18 20:36:22 +03:00
Evgenii Stratonikov
a2f012e589 core: deploy new smart-contract in test blocks 
Also put smart-contract sources in testdata and
add invocation TX with APPCALL and storage.Put.
 2020-02-18 11:59:28 +03:00
Anna Shaleva
2b9e63c511 core: add func to generate RPC test data 
newRPCTestChain function generates test chain "testblocks.acc" for RPC
tests
 2020-02-17 23:54:25 +03:00
Roman Khimov
dfb38e1da1
Merge pull request #649 from nspcc-dev/feature/dump_storage 
cli: add possibility to dump storage changes on restore
 2020-02-16 23:50:31 +03:00
Roman Khimov
b5453ad24f
Merge pull request #672 from nspcc-dev/fix-sigsegv-on-reference-check 
core: deal with bad transaction Inputs
 2020-02-14 13:23:42 +03:00
Roman Khimov
0fceaf12d2 core: deal with bad transaction Inputs 
They may not really have a reference. Fixes #671.
 2020-02-14 09:42:10 +03:00
Roman Khimov
992ccc0b26
Merge pull request #664 from nspcc-dev/sort-validator-keys 
core: sort GetValidators() result
 2020-02-13 21:46:26 +03:00
Evgeniy Kulikov
8d410a6659
Add JSON tags for RPC response types 
- Attribute should have 2 fields (usage, data)
- VOut should have 4 (5) fields (asset, value, address, n)
- Script should have 2 fields (invocation, verification)
 2020-02-13 19:06:54 +03:00
Roman Khimov
d4cd134aef core: sort GetValidators() result 
As C# node does it. Technically it's only needed for consensus and could be
implemented in the appropriate package, but for better compatibility with C#
node we're better returning it sorted right here.
 2020-02-13 17:07:27 +03:00
Roman Khimov
26af9200b0 core: refactor voting processing, fix pending tx subtractions 
Deduplicate code and add missing token check for subtractions that are being
done in GetValidators() for pending transactions.
 2020-02-13 13:07:34 +03:00
Roman Khimov
c8a248596e core: introduce ValidatorsCount, make a proper count 
We were completely lacking ValidatorsCount that is supposed to track the
number of votes with particular count of consensus nodes which in theory can
change the number of active consensus nodes (if it ever to exceed the number
of standby validators), so we were not producing the right count and based on
that not giving the right set of validators.

Fixes #512.
 2020-02-13 13:07:34 +03:00
Roman Khimov
357b675090 core: don't choose duplicating standby validators 
We may already have chosen some of standby validators, so don't repeat.
 2020-02-13 13:07:34 +03:00
Roman Khimov
f9c6a0d77c core: make voting choose the best validators 
They need to be sorted in order for voting system to work.
 2020-02-13 13:07:34 +03:00
Roman Khimov
2ff7ceb8cf core: properly update validator votes for Account descriptor 
ValidatorsCount is not implemented yet, but we need to update validators
properly, adding account's vote to their votes count.
 2020-02-13 13:07:34 +03:00
Roman Khimov
8310b76d28 core: properly save votes when processing Account descriptor 
They were ignored basically.
 2020-02-13 13:07:34 +03:00
Roman Khimov
807309a0de core/state: only drop unregistered non-voted validators 
Simple as that: UnregisteredAndHasNoVotes != !RegisteredAndHasVotes

Registered validators should stay in the DB, we might be in the process of
updating votes for them and that starts with subtraction.
 2020-02-13 13:07:34 +03:00
Roman Khimov
38d0efc96c core: fix Registered state descriptor processing 
It has simple boolean value encoded, no `strconv` needed (it actually chokes
on such input).
 2020-02-12 21:27:40 +03:00
Roman Khimov
803fb39bb0 transaction: fix wrong state descriptor serdes 
Wrong field order.
 2020-02-12 21:23:28 +03:00
Evgenii Stratonikov
0a894db7f8 storage: add Exists flag to KeyValue in batch 
Set Exists flag if an item with the specified key was already
present in storage before persisting.
 2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
b1d9e1132d cli: dump storage changes into JSON files 2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
fb9af98179 storage: implement GetBatch() to view storage changes 
GetBatch returns changes to be persisted.
 2020-02-12 12:16:31 +03:00
Roman Khimov
38de2af073
Merge pull request #655 from nspcc-dev/fix-mempool-deadlock-on-reverification 
core: get transactions from dao when mempool should not be used
 2020-02-11 13:34:23 +03:00
Roman Khimov
292051d8df core: get transactions from dao when mempool should not be used 
All of these places deal strictly with the chain and shouldn't ever be
bothered with mempool. It also fixes a deadlock on reverification of
non-standard tx:

1 @ 0x42f62f 0x43fbe9 0x43fbbf 0x43f95d 0x967059 0x966f66 0x972c7c 0x974e13 0x97a5d9 0x97bdf0 0x976147 0x966cc0 0x970f70 0x96c8cb 0x9ba858 0x45ca51
	0x43f95c	sync.runtime_SemacquireMutex+0x3c							/usr/local/go/src/runtime/sema.go:71
	0x967058	sync.(*RWMutex).RLock+0x128								/usr/local/go/src/sync/rwmutex.go:50
	0x966f65	github.com/CityOfZion/neo-go/pkg/core/mempool.(*Pool).TryGetValue+0x35			/go/src/github.com/CityOfZion/neo-go/pkg/core/mempool/mem_pool.go:229
	0x972c7b	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).GetTransaction+0x4b			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:782
	0x974e12	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).References+0x132			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:944
	0x97a5d8	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).GetScriptHashesForVerifying+0x58	/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1410
	0x97bdef	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).verifyTxWitnesses+0x4f		/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1545
	0x976146	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).isTxStillRelevant+0x216		/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1067
	0x966cbf	github.com/CityOfZion/neo-go/pkg/core/mempool.(*Pool).RemoveStale+0xff			/go/src/github.com/CityOfZion/neo-go/pkg/core/mempool/mem_pool.go:208
	0x970f6f	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).storeBlock+0x2ecf			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:614
	0x96c8ca	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).AddBlock+0xea			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:308
	0x9ba857	github.com/CityOfZion/neo-go/pkg/network.(*blockQueue).run+0x157			/go/src/github.com/CityOfZion/neo-go/pkg/network/blockqueue.go:48
 2020-02-10 18:53:22 +03:00
Anna Shaleva
9b03f968b1 core: fixe FreeGasLimit constraint 
Lack of FreeGasLimit in privnet leads to gas limit exceeding in case of transactions with small amount of GAS to be used for invoke operation (< real cost of the transaction). Solution: Fixed constraint in case when FreeGasLimit == 0. So now we are able to perform transactions in privnet with FreeGasLimit = 0 for free.
 2020-02-10 18:07:19 +03:00
Roman Khimov
9b9adb28c1
Merge pull request #642 from nspcc-dev/feature/emit 
vm, compiler: move Emit* functions to a separate package

Closes #449, #534.
 2020-02-08 15:56:48 +03:00
Roman Khimov
7ccf7974b6 core: substitute bad notifications with error messages 
Tesnet sync failed with:
Feb 07 00:04:19 nodoka neo-go[1747]: 2020-02-07T00:04:19.838+0300        WARN        blockQueue: failed adding block into the blockchain        {"error": "failed to store notifications: not supported", "blockHeight": 713984, "nextIndex": 713985}

because some (not so) smart contract emitted a notification with an
InteropItem inside.
 2020-02-07 15:34:59 +03:00
Evgenii Stratonikov
8243a8b3a7 emit: use io.BinWriter instead of bytes.Buffer 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
4d8a3a359b vm: move Emit* functions to a separate package 
Also strip 'Emit' prefix because 'emit' is now
in the package name.
 2020-02-06 18:45:37 +03:00
Roman Khimov
ab14a4619d
Merge pull request #647 from nspcc-dev/fix-mempool-and-chain-locking 
Fix mempool and chain locking

This allows us easily make 1000 Tx/s in 4-nodes privnet, fixes potential
double spends and improves mempool testing coverage.
 2020-02-06 18:43:29 +03:00
Evgenii Stratonikov
9204b95d5b core: fix GAS price definitions 2020-02-06 18:19:08 +03:00
Roman Khimov
18695e660b mempool: drop RemoveOverCapacity(), handle it right in the Add() 
Simplifies things a lot and removes useless code.
 2020-02-06 17:50:11 +03:00
Roman Khimov
684cbf5bac mempool: make it almost 100% test-covered 2020-02-06 17:50:11 +03:00
Roman Khimov
6672f4b26f mempool: iterate over slice in GetVerifiedTransactions() 
It's more efficient and keeps transactions sorted by priority.
 2020-02-06 16:59:20 +03:00
Roman Khimov
e97396e56c mempool: test addition of conflicting tx to the pool 2020-02-06 15:57:20 +03:00
Roman Khimov
1133bbe584 mempool: remove unverified transactions pool 
Our mempool only contains valid verified transactions all the time, it never
has any unverified ones. Unverified pool made some sense for quick unverifying
after the new block acceptance (and gradual background reverification), but
reverification needs some non-trivial locking between blockchain and mempool
and internal mempool state locking (reverifying tx and moving it between
unverified and verified pools must be atomic). But our current reverification
is fast enough (and has all the appropriate locks), so bothering with
unverified pool makes little sense.
 2020-02-06 15:45:16 +03:00
Roman Khimov
b675903f52 mempool/core: redesign mempool dances on block acceptance 
We not only need to remove transactions stored in the block, but also
invalidate some potential double spends caused by these transactions. Usually
new block contains a substantial number of transactions from the pool, so it's
easier to make one pass over it only keeping valid items rather than remove
them one by one and make an additional pass to recheck inputs/witnesses.
 2020-02-06 15:45:16 +03:00
Roman Khimov
b567ce86ac mempool: implement insertion to sorted slice 
Which is way faster than sort.Sort'ing things all the time.
 2020-02-06 15:44:18 +03:00
Roman Khimov
35183b6dba mempool: reverse the order of sorted slice 
Chopping off the last element of the slice if way easier than doing it with
the first one.
 2020-02-06 15:44:16 +03:00
Roman Khimov
794027a90b mempool: use one slice for both priorities 
It doesn't harm as we have transactions naturally ordered by fee anyway and it
makes managing them a little easier. This also makes slices store item itself
instead of pointers to it which reduces the pressure on the memory subsystem.
 2020-02-06 15:43:45 +03:00
Roman Khimov
325bea3fa9 mempool: cache Feer invocation results in the item 
They shouldn't depend on the chain state and for the same transaction they
should always produce the same result. Thus, it makes no sense recalculating
them over and over again.
 2020-02-06 15:41:56 +03:00
Roman Khimov
e01bfeeb4d mempool: remove lock indirection from the Pool 
After the f0bb886be3 with all methods of Pool
being pointer-based it makes no sense having this lock as a pointer.
 2020-02-06 15:41:56 +03:00
Roman Khimov
a928ad9cfa mempool: make item an internal thing of mempool package 
Nobody outside should care about these details, mempool operates on
transactions and that's it.
 2020-02-06 15:41:56 +03:00
Roman Khimov
f0e3a31bc8 mempool: fix appending to sorted pools 
Appending and not changing the real Items is utterly wrong.
 2020-02-06 15:41:56 +03:00
Roman Khimov
f9963cca37 core: short-circuit verifyInputs when there are no inputs 2020-02-06 15:41:56 +03:00
Roman Khimov
70b3839fd0 core/mempool: fix AddBlock and tx pooling concurrency issues 
Eliminate races between tx checks and adding them to the mempool, ensure the
chain doesn't change while we're working with the new tx. Ensure only one
block addition attempt could be in progress.
 2020-02-06 15:41:52 +03:00
Roman Khimov
f0bb886be3 mempool: make all methods pointer methods 
Makes no sense copying the Pool around.
 2020-02-04 17:36:11 +03:00
Roman Khimov
388fed06e5 transaction: forbid serializing invalid transactions 
Transaction that has no data is invalid and can't be serialized, so throw an
error if someone tries to.
 2020-02-03 16:33:03 +03:00
Evgeniy Kulikov
eb59460032
Merge pull request #614 from nspcc-dev/configurable-mempool-size 
config: add MemPoolSize configuration parameter
 2020-01-23 13:25:27 +03:00
Roman Khimov
d6784e8f45 config: add MemPoolSize configuration parameter 
Allow tuning/experimenting with mempool.
 2020-01-23 13:10:44 +03:00
Evgenii Stratonikov
303b82cc44 core: set free GAS limit via config 2020-01-22 14:32:21 +03:00
Evgenii Stratonikov
4718375db1 rpc: return amount of consumed GAS in invoke* 2020-01-22 14:31:57 +03:00
Evgenii Stratonikov
4eae55143f core: restrict GAS available for invocation transactions 
There are 10 GAS available for free plus any amount of GAS
attached to a transaction.
 2020-01-22 14:31:54 +03:00
Roman Khimov
32213b1454
Merge pull request #601 from nspcc-dev/refactoring/core 
core: refactor out Block, BlockBase and Header, closes #597.
 2020-01-20 16:19:20 +03:00
Roman Khimov
46b82b4fb5 keys: don't return error from PrivateKey.Sign 
As it can't ever happen.
 2020-01-17 17:00:30 +03:00
Evgenii Stratonikov
28183b81d6 mempool: simplify names of exported types 
With the move to a separate package, naming can be simplified:
MemPool -> Pool, PoolItem -> Item, PoolItems -> Items.
 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
fed6fba9b6 core: refactor out MemPool 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
79bceb3e40 block: use require in tests 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
489b88afbb block: rename BlockBase to Base 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
63c56cca5c core: refactor out Block, BlockBase and Header structs 
See #597.
 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
9dc5571327 core,rpc: close Blockchain in tests 
If blockchain is not closed, logging in defer can occur
after test has finished, which will lead to a panic with
"Log in goroutine after Test* has completed".
 2020-01-10 11:47:56 +03:00
Evgenii Stratonikov
637c99eda7 core: simplify newInteropContext 2020-01-10 11:14:29 +03:00
Evgenii Stratonikov
aecdf470e7 cli,pkg: use zap.Logger 2020-01-10 11:14:27 +03:00
Evgenii Stratonikov
9c79684516 storage: panic on error in boltdb.Seek 
Error in Seek means something is terribly wrong (e.g. db was not opened) and
error drop is not the right thing to do, because caller
will continue working with the wrong view.
 2020-01-09 15:17:35 +03:00
Roman Khimov
c306d2f07c
Merge pull request #581 from nspcc-dev/feat/storage_find 
core: implement Neo.Storage.Find interop, closes #422.
 2019-12-26 15:46:46 +03:00
Evgenii Stratonikov
48cf4f4a84 core: implement Neo.Storage.Find interop 2019-12-26 15:21:41 +03:00
Roman Khimov
f8887b63ba
Merge pull request #578 from nspcc-dev/fix/sign 
consensus: verify payloads correctly, closes #555.
 2019-12-26 13:07:40 +03:00
Evgenii Stratonikov
714c466c2c core: add ScriptFromWitness function 
Extracting verification script from witness became a common task.
This commit adds such a possibility.
 2019-12-26 10:49:56 +03:00
Roman Khimov
9145855d2c
Merge pull request #579 from nspcc-dev/refactor-crypto 
This moves some functionality into micro-packages, improves testing,
unexports some code and fixes bugs along the way.
 2019-12-25 18:12:50 +03:00
Roman Khimov
b246653f62 address: rename functions as per #579 comments 
Make them more clear to understand.
 2019-12-25 17:34:18 +03:00
Roman Khimov
e685e9bf9a address: move into its own package 
Doesn't really belong to the crypto.
 2019-12-25 15:22:02 +03:00
Roman Khimov
ee28fb08f6 crypto: move merkle tree into the hash package 
It's all about hashes, so it makes sense putting it there.
 2019-12-25 11:28:59 +03:00
Vsevolod Brekelov
7f1844a39e transaction: add unmarshaling for TXType 2019-12-25 01:44:37 +03:00
Evgenii Stratonikov
5bc32b523a vm: implement Neo.Iterator.* interops 2019-12-24 10:21:47 +03:00
Evgenii Stratonikov
3ff7fd5262 vm: implement Neo.Enumerator.* interops 2019-12-24 10:21:46 +03:00
Roman Khimov
bf84e1f2fb core: cache top block 
Turns out, our dApps use it a lot and we were going to the DB to get it which
is a useless waste of time. Technically we could also remove blockHeight here,
but not doing it at the moment as it's more involved.

It eliminates this time waste from the pprof graph, but doesn't change 1.4M ->
1.5M 100K mainnet block import test case in any noticeable way.
 2019-12-23 19:18:12 +03:00
Roman Khimov
deb76c5199 core: add negative non-interop test for all interops 
Try to feed some garbage into these functions.
 2019-12-23 16:14:43 +03:00
Roman Khimov
76d69ee5ee core: restore proper interops ordering 2019-12-23 16:14:43 +03:00
Roman Khimov
c287a9e93c core: implement Neo.Witness.GetVerificationScript interop 2019-12-23 16:14:43 +03:00
Roman Khimov
9d638d0fee core: implement Neo.InvocationTransaction.GetScript 2019-12-23 16:14:43 +03:00
Roman Khimov
a7457d08a1 vm/core: add ID support for SYSCALL, redo interop registration 
This solves two problems:
 * adds support for shortened SYSCALL form that uses IDs (similar to #434, but
   for NEO 2.0, supporting both forms), which is important for compatibility
   with C# node and mainnet chain that uses it from some height
 * reworks interop plugging to use callbacks rather than appending to the map,
   these map mangling functions are clearly visible in the VM profiling
   statistics and we want spawning a VM to be fast, so it makes sense
   optimizing it. This change moves most of the work to the init() phase
   making VM setup cheaper.

Caveats:
 * InteropNameToID accepts `[]byte` because that's the thing we have in
   SYSCALL processing and that's the most often usecase for it, it leads to
   some conversions in other places but that's acceptable because those are
   either tests or init()
 * three getInterop functions are: `getDefaultVMInterop`, `getSystemInterop`
   and `getNeoInterop`

Our 100K (1.4M->1.5M) block import time improves by ~4% with this change.
 2019-12-19 13:35:42 +03:00
Evgenii Stratonikov
884779e501 *: goimports 2019-12-17 14:51:28 +03:00
Roman Khimov
7a5e995030 core: add MemPool Verify() test 2019-12-14 00:05:45 +03:00
Roman Khimov
5d68f88196 core: fix and speed up mempool Verify() 
First of all, it was wrong, it was not checking for inputs really, it compared
tx hashes for some reason, second, when it did compare inputs it compared only
the PrevIndex part of them which is also wrong.

Also, there is absolutely no reason to go through GetVerifiedTransactions()
here, we don't need this copy of pointers and it can also be outdated by the
time we're to finish our check.

Before:
BenchmarkTXPerformanceTest-4
    5000            485506 ns/op           65886 B/op        409 allocs/op
ok      github.com/CityOfZion/neo-go/integration        3.212s

After:
enchmarkTXPerformanceTest-4
    5000            371104 ns/op           44367 B/op        408 allocs/op
ok      github.com/CityOfZion/neo-go/integration        2.712s
 2019-12-13 23:23:33 +03:00
Roman Khimov
e631e75718 core: optimize GetVerifiedTransactions() 
This simple change improves our BenchmarkTXPerformanceTest by 14%, just
because we don't waste time on reallocations during append().

Before:
   10000            439754 ns/op          218859 B/op        428 allocs/op
ok      github.com/CityOfZion/neo-go/integration        5.423s

After:
   10000            369833 ns/op           87209 B/op        412 allocs/op
ok      github.com/CityOfZion/neo-go/integration        4.612s
 2019-12-13 19:57:17 +03:00
Roman Khimov
a9401e2ec7 core: do not write new dao elements into DB 
It's useless work being done before it's actually needed. These (updated with
new values) are going to be written with some kind of Put anyway, so writing
them here is just a waste of time.
 2019-12-13 19:57:17 +03:00
Roman Khimov
6896b40dee state: use open-coded array (de)serialization for accounts 
We're spending a lot of time here, 100K blocks import starting at 1.4M, before
this patch:
real    4m17,748s
user    6m23,316s
sys     0m37,866s

After:
real    3m54,968s
user    5m56,547s
sys     0m39,398s

9% is quite a substantial improvement to justify this change.
 2019-12-13 19:57:17 +03:00
Roman Khimov
0d0a27d271 core: add cachedDao to cache accounts and contracts 
Importing 100K blocks starting at 1.4M, before this patch:
real    6m0,356s
user    8m52,293s
sys     0m47,372s

After this patch:
real    4m17,748s
user    6m23,316s
sys     0m37,866s

Almost 30% better.
 2019-12-13 19:57:17 +03:00
Roman Khimov
212cf44e26 core: get data from dao in interops 
It's more logically correct as Blockchain can be several transactions behind
in terms of its state.
 2019-12-13 17:17:14 +03:00
Roman Khimov
0abd55c2c2 vm: add cached ScriptHash() to Context 
Avoid recalculating it over and over again in interop.
 2019-12-13 17:05:03 +03:00
Roman Khimov
09223236f9 core: add Persist() to dao() 
Hide its internals a little, which is gonna be useful for future composition.
 2019-12-12 21:17:13 +03:00
Roman Khimov
eb1749d190 core: remove persisting from GetValidators() 
It's a getter function and even though it's quite fancy with its transactions
processing (for consensus operation) it shouldn't ever change the state of the
Blockchain. If we're to change anything here these changes may conflict with
the actual block processing later or may lead to broken state (if transactions
won't be approved for some reason).
 2019-12-12 21:14:33 +03:00
Roman Khimov
31dbc35279 core: gofmt dao_test.go 2019-12-12 21:05:36 +03:00
Roman Khimov
c5ed3b788b core: simplify dao creation with newDao() 2019-12-12 21:04:55 +03:00
Roman Khimov
8b3080b972 io: rename Read/WriteBytes to Read/WriteB 
go vet is not happy about them:
  pkg/io/binaryReader.go:92:21: method ReadByte() byte should have signature ReadByte() (byte, error)
  pkg/io/binaryWriter.go:75:21: method WriteByte(u8 byte) should have signature WriteByte(byte) error
 2019-12-12 20:19:50 +03:00
Roman Khimov
54d888ba70 io: add type-specific read/write methods 
This seriously improves the serialization/deserialization performance for
several reasons:
 * no time spent in `binary` reflection
 * no memory allocations being made on every read/write
 * uses fast ReadBytes everywhere it's appropriate

It also makes Fixed8 Serializable just for convenience.
 2019-12-12 20:19:50 +03:00
Roman Khimov
89d7f6d26e core/tx: microoptimize block/tx hashing 
Don't hash the data twice.
 2019-12-12 18:01:30 +03:00
Vsevolod Brekelov
c93a8d2bc4 core: extracted same logic to separate methods 2019-12-11 13:14:45 +03:00
Vsevolod Brekelov
c1f39d5c7b internal: moved testutil method to internal package 2019-12-11 13:14:43 +03:00
Vsevolod Brekelov
2d42b14a1d core: renames entities-> state and removed State prefix 2019-12-11 13:14:18 +03:00
Vsevolod Brekelov
8809fe437d core: unit tests for dao 2019-12-11 13:13:51 +03:00
Vsevolod Brekelov
c0e59ebd4e core: unit tests for entities 2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
c7ac4b6dff core: fix encoding and decoding for notification event 2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
ec17654986 core: refactoring blockchain state and storage 
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
 2019-12-11 13:05:31 +03:00
Roman Khimov
d0f9a28196 vm/core: improve block import speed with PublicKey caching 
This change (closely related to the neo-project/neo#1321 proposal) speeds up
1.4M mainnet blocks import by 30%. Basically, we're eliminating key decoding
for block's multisignature that has the same keys most of the time.

Things I don't like about this patch:
 * yet another parameter for verifyHashAgainstScript()
 * vm keys are not copied in/out

But it's rather simple and solves the problem for this particular case, so I
think it's worth it.
 2019-12-10 19:13:29 +03:00
Roman Khimov
f1856bfa8b core/tx: remove publickey indirection from assets and txes 
It makes very little sense having pointers here, these structures MUST have
some kind of key and this key is not gonna be wandering somewhere on its
own. Fixes a part of #519.
 2019-12-09 18:33:04 +03:00
Roman Khimov
5b6c5af704 *: implement EncodeBinary with pointer receivers where appropriate 
Everywhere except ParamType (which is just a byte), reduce copying things
around for no real reason.
 2019-12-09 18:25:15 +03:00
Roman Khimov
7e371588a7 core/tx: remove one layer of indirection for scripts and inouts 
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
 2019-12-09 17:14:10 +03:00
Evgenii Stratonikov
fccb008594 io: implement ReadBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
838050f8b5 io: rename ReadBytes() to ReadVarBytes() 2019-12-09 15:00:15 +03:00
Roman Khimov
e4d821f32d
Merge pull request #546 from nspcc-dev/write-optimizations 
Write optimizations
 2019-12-06 19:40:38 +03:00
Roman Khimov
9992a98007 core: optimize balance utxo removal 
One of my samples had 8 (out of 30) seconds spent here, but values of 100ms
are more typical. After this change it becomes invisible.
 2019-12-06 18:46:24 +03:00
Roman Khimov
844491d365 *: use more efficient WriteBytes where appropriate 
Before this patch on block import we could easily be spending more than 6
seconds out of 30 in Uint256 encoding for UnspentBalance, now it's completely
off the radar.
 2019-12-06 18:22:21 +03:00
Evgenii Stratonikov
7179e4ba9f util: add LE suffix to Uint256 methods 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
57efad912c util: add LE suffix to Uint160 methods 2019-12-06 12:16:55 +03:00
Roman Khimov
8beb135829
Merge pull request #542 from nspcc-dev/fix-storage-interop-reads 
core: fix wrong data being read in interops (part of #501)
 2019-12-05 09:08:38 +03:00
Roman Khimov
c034aae378 core: fix wrong data being read in interops (part of #501) 
When 74590551 introduced this code we had no proper caching layer, so there
were these strange fallbacks in the code. fc0031e5 should'd removed them, but
failed to do so, so do it now and fix processing of transactions that touch
storage for the same key (address) in the same block.
 2019-12-04 19:51:57 +03:00
Vsevolod Brekelov
ff15a0acfd core: add trigger types as constants. Closes #509 2019-12-04 12:27:04 +03:00
Roman Khimov
31add423a8 core: replace open-coded PUSHBYTES64 with the one from opcode 2019-12-03 18:23:46 +03:00
Roman Khimov
138e125646 *: remove duplicate functions producing verification script 
Drop wif.GetVerificationScript(), drop
smartcontract.CreateSignatureRedeemScript(), add GetVerificationScript()
directly to the PublicKey and use it everywhere.
 2019-12-03 18:23:46 +03:00
Roman Khimov
8d4dd2d2e1 vm: move opcodes into their own package 
This allows easier reuse of opcodes and in some cases allows to eliminate
dependencies on the whole vm package, like in compiler that only needs opcodes
and doesn't care about VM for any other purpose.

And yes, they're opcodes because an instruction is a whole thing with
operands, that's what context.Next() returns.
 2019-12-03 18:22:14 +03:00
Roman Khimov
f48228ef7d
Merge pull request #467 from nspcc-dev/errcheck_297 
This patchset closes #297 and #457.
 2019-12-03 15:06:11 +03:00
Vsevolod Brekelov
03ff2976ed io: refactoring for using WriteVarBytes instead of WriteLE 
goal is to be consistent with C# implementation.
For writing []byte WriteBytes used and for byte - WriteVarByte.
 2019-12-03 13:49:33 +03:00
Roman Khimov
1c89c192ac core: optimize some accesses to unsortedTxn in mempool 
These don't need a full lock, they only read things from maps.
 2019-12-02 22:39:43 +03:00
Roman Khimov
0cd3493fa5 core: fix potential locking problem in mempool 
I think it should fix this issue mentioned in the #526:

INFO[1456] blockchain persist completed                  blockHeight=63480 headerHeight=1810000 persistedBlocks=1 persistedKeys=4 took=740.7113ms
fatal error: concurrent map read and map write

goroutine 322 [running]:
runtime.throw(0xc8a6dc, 0x21)
        /usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc003473958 sp=0xc003473928 pc=0x42e282
runtime.mapaccess2(0xb706a0, 0xc0001893b0, 0xc0034739c8, 0xc0028704e0, 0x3)
        /usr/local/go/src/runtime/map.go:470 +0x278 fp=0xc0034739a0 sp=0xc003473958 pc=0x40dc08
github.com/CityOfZion/neo-go/pkg/core.MemPool.ContainsKey(0xc0001d0d20, 0xc0001893b0, 0xc0001893e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /neo-go/pkg/core/mem_pool.go:92 +0xcb fp=0xc003473a30 sp=0xc0034739a0 pc=0x9326db
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).HasTransaction(0xc0001ca8c0, 0x49f0b45d430e441b, 0x553db79b7072821c, 0x28969518de11976, 0xba5100efddbe79d4, 0xc003473cd0)
        /neo-go/pkg/core/blockchain.go:803 +0xa1 fp=0xc003473b68 sp=0xc003473a30 pc=0x914b11
github.com/CityOfZion/neo-go/pkg/core.Blockchainer.HasTransaction-fm(0x49f0b45d430e441b, 0x553db79b7072821c, 0x28969518de11976, 0xba5100efddbe79d4, 0xc005a5d388)
        /neo-go/pkg/core/blockchainer.go:28 +0x46 fp=0xc003473ba8 sp=0xc003473b68 pc=0x997326
github.com/CityOfZion/neo-go/pkg/network.(*Server).handleInvCmd(0xc00018f680, 0xd9d980, 0xc00025e190, 0xc005a5d380, 0x0, 0x0)
        /neo-go/pkg/network/server.go:401 +0x3bb fp=0xc003473d38 sp=0xc003473ba8 pc=0x9924cb
github.com/CityOfZion/neo-go/pkg/network.(*Server).handleMessage(0xc00018f680, 0xd9d980, 0xc00025e190, 0xc007a0d050, 0x0, 0x0)
        /neo-go/pkg/network/server.go:582 +0x1ae fp=0xc003473da0 sp=0xc003473d38 pc=0x993bbe
github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).handleConn(0xc000228420, 0xd9b880, 0xc0001b6f00)
        /neo-go/pkg/network/tcp_transport.go:93 +0x202 fp=0xc003473fc8 sp=0xc003473da0 pc=0x996672
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc003473fd0 sp=0xc003473fc8 pc=0x45b3e1
created by github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).Dial
        /neo-go/pkg/network/tcp_transport.go:36 +0xb4

The problem is that we're modifying `unsortedTxn` under a reader lock.
 2019-12-02 22:36:59 +03:00
Roman Khimov
bc5beb438f *: gofmt 
Somehow these crept into the repository.
 2019-12-02 10:10:50 +03:00
Roman Khimov
2c0ed62680 core: fix blockchain init with (2000*N + 1) header hashes in DB 
Stored header count includes block number zero, so it's (height + 1).
 2019-11-29 16:26:44 +03:00
Roman Khimov
2d41450ac9
Merge pull request #507 from nspcc-dev/dbft 
network: plug in dBFT library
 2019-11-27 15:22:40 +03:00
Roman Khimov
c5e2b3ce73
Merge pull request #523 from nspcc-dev/fix-interim-block-changes-store 
Fix interim block changes store
 2019-11-27 14:51:29 +03:00
Vsevolod
1a9e98caf3
Merge pull request #522 from nspcc-dev/fix-invalid-tx-type-handling 
Fix invalid tx type handling
 2019-11-27 13:40:52 +03:00
Roman Khimov
467c9c146d core: restore intermediate block-level store 
Commit c80ee952a1 removed temporary store used
to contain changes of the block being processed. It's wrong in that the block
changes should be applied to the database in a single transaction so that
there wouldn't be any intermediate state observed from the outside (which is
possible now). Also, this made changes commiting persist them to the
underlying store effectively making our persist loop a no-op (and not
producing `persist completed` log lines that we love so much).
 2019-11-27 13:28:13 +03:00
Roman Khimov
bb765ccfa7 core: gofmt 2019-11-27 13:27:47 +03:00
Roman Khimov
65bf0d01d5 transaction: don't hide decoding error by logging 
Unknown transaction types are wrong and should be treated as errors.
 2019-11-27 13:12:04 +03:00
Evgenii Stratonikov
fdd5276d3e network: plug in dBFT library 2019-11-27 10:57:22 +03:00
Evgenii Stratonikov
af61571ba0 core: do not return error in block.getHashableData 2019-11-26 15:27:37 +03:00
Roman Khimov
eb84ae49da
Merge pull request #508 from nspcc-dev/feature/getvalidators_420 
core: Implement getvalidators interop
 2019-11-22 18:15:08 +03:00
Vsevolod Brekelov
c03de88b41 core: unit tests blockchain state 2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
d28f2dce05 core: unit test validator state 2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
419b031e96 core: add weighted function for validators 2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
672f314be0 core: nit refactoring and add godoc 2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
8ae88593dd core: blockchain fmt 2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
3c4b0e955d core: create blockchain state that represents state of the bc 
move some util methods to blockchain state
 2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
c80ee952a1 core/crypto: add validators and interop for validators 
add processing of validators while block persist;
add validator structure with decoding/encoding;
add validator get from store;
add EnrollmentTX and StateTX processing;
add pubkey decode bytes, unique and contains functions;
 2019-11-21 15:23:35 +03:00
Roman Khimov
34e2122e58 core: only check tx against mempool if it's not in the block 
Fixes failure to process transaction from the block when it was relayed
initially:

WARN[0788] blockQueue: failed adding block into the blockchain  blockHeight=7270 error="transaction 35088916403e5cf2152e16c3bc6e0fba20c955fba38543b9fa5c50a3d3a4ace5 failed to verify: invalid transaction due to conflicts with the memory pool" nextIndex=7271
WARN[0790] blockQueue: failed adding block into the blockchain  blockHeight=7270 error="transaction 35088916403e5cf2152e16c3bc6e0fba20c955fba38543b9fa5c50a3d3a4ace5 failed to verify: invalid transaction due to conflicts with the memory pool" nextIndex=7271
WARN[0790] blockQueue: failed adding block into the blockchain  blockHeight=7270 error="transaction 35088916403e5cf2152e16c3bc6e0fba20c955fba38543b9fa5c50a3d3a4ace5 failed to verify: invalid transaction due to conflicts with the memory pool" nextIndex=7271
 2019-11-20 16:12:40 +03:00
Roman Khimov
27a57e1a2d transaction: allow system fee specification in NewInvocationTX 
It's not possible to create any deployment TX without it.
 2019-11-20 16:12:40 +03:00
Roman Khimov
29882b076c rpc: remove duplicating definition of UTXO 
Port sorting methods to core.
 2019-11-19 17:35:04 +03:00
Roman Khimov
d2bdae99e4
Merge pull request #492 from nspcc-dev/feature/interop_neo_tests1 
core: add unit tests for interops_neo
 2019-11-18 19:39:05 +03:00
Vsevolod Brekelov
d57b6e1091 core: add unit tests for interops_neo 2019-11-18 19:31:27 +03:00
Roman Khimov
5841d3931e rpc: implement getunspents method, fix #473 2019-11-18 11:30:09 +03:00
Roman Khimov
e4868cd429 core: implement EncodeBinary on value 
Fixes panic:
panic: core.NotificationEvent does not have EncodeBinary(*BinWriter)

goroutine 97 [running]:
github.com/CityOfZion/neo-go/pkg/io.(*BinWriter).WriteArray(0xc004f7dda0, 0xabf820, 0xc004f7ddc0)
        /home/rik/dev/neo-go/pkg/io/binaryWriter.go:45 +0x4b7
github.com/CityOfZion/neo-go/pkg/core.(*AppExecResult).EncodeBinary(0xc000635498, 0xc004f7dda0)
        /home/rik/dev/neo-go/pkg/core/notification_event.go:73 +0x9c
github.com/CityOfZion/neo-go/pkg/core.putAppExecResultIntoStore(0xcf0020, 0xc001aafec0, 0xc000635498, 0xc004f81940, 0x40)
        /home/rik/dev/neo-go/pkg/core/notification_event.go:33 +0xab
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).storeBlock(0xc000118000, 0xc006f89ea0, 0xc006f89ea0, 0x0)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:534 +0x2539
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).AddBlock(0xc000118000, 0xc006f89ea0, 0xc004f368b0, 0x1)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:256 +0x91
github.com/CityOfZion/neo-go/pkg/network.(*blockQueue).run(0xc0048f4b80)
        /home/rik/dev/neo-go/pkg/network/blockqueue.go:39 +0x14c
created by github.com/CityOfZion/neo-go/pkg/network.(*Server).Start
        /home/rik/dev/neo-go/pkg/network/server.go:133 +0x272
 2019-11-15 22:05:09 +03:00
Roman Khimov
1a5731e005 core: rework balance accounting 
Store all unspents instead of balance values. Bump store version as this is an
incompatible change.
 2019-11-15 22:05:09 +03:00
Roman Khimov
a16c2c3825 core: add NEP5 transfer tracking stub 
See #498.
 2019-11-15 18:04:59 +03:00
Roman Khimov
01082a8988 core: fix nondeterministic txGetReferences() behavior 
Ranging over map is nondeterministic and contracts may be unprepared for
that. Fixes #454.
 2019-11-15 18:04:59 +03:00
Roman Khimov
5e8122bfac core: add contract hash into the error output for checkStorageContext() 
Makes debugging things easier.
 2019-11-15 18:04:59 +03:00
Roman Khimov
5ce269c035 core: change notify implementation to save notifications 
Fixes #453 and makes it possible to refer to these notifications later.
 2019-11-15 18:04:59 +03:00
Evgenii Stratonikov
1a08ad19aa sc: implement io.Serializable for ParamType 2019-11-14 14:20:38 +03:00
Evgenii Stratonikov
2be18f91df util: implement io.Serializable for Uint256 2019-11-14 14:20:38 +03:00
Evgenii Stratonikov
b16e56a47b io: refactor BinReader.ReadArray() 
Make it accept arbitrary slice pointer.
 2019-11-14 14:19:58 +03:00
Evgenii Stratonikov
ad9091d13d io: implement generic array (de-)serialization 
It is done through reflection and panics
in every unexpected situation.
 2019-11-13 17:27:23 +03:00
Roman Khimov
394346fc26 core: add proper claim transactions processing 2019-11-11 15:35:57 +03:00
Roman Khimov
b05754deac core: add Close() to blockchainer, implement it to properly close chain 
Before it the deferred function in Run() was actually never able to properly
close the Store, so we weren't synching the latest state to the disk.
 2019-11-08 12:19:54 +03:00
Vsevolod
f686069f37
Merge pull request #481 from nspcc-dev/core-fix-init-corner-cases 
Fix init corner cases
 2019-11-06 18:36:40 +03:00
Roman Khimov
c567307748 core: fix potential issue with header list init 
If we're to receive some 500 headers (less than `headerBatchCount`) and quit
before receiving more of them we end up with clean `bc.headerList` that will
be inited going backwards to the `targetHash`, but code path doesn't add add
the `targetHash` itself which it should do in this particular case, otherwise
we end with no genesis block hash in the list.
 2019-11-06 18:13:45 +03:00
Roman Khimov
b5199625cd core: init genesis block header if there are no other headers in DB 
Fixes crash when restarted after the DB initialization and no blocks written
into the DB.
 2019-11-06 17:58:19 +03:00
Roman Khimov
c16c2bf102 core: store current header reference on clean init 
Missing it the following line could fail on subsequent restarts:

        currHeaderHeight, currHeaderHash, err := storage.CurrentHeaderHeight(bc.store)

if the node was stopped before any headers had been received.
 2019-11-06 17:56:06 +03:00
Vsevolod Brekelov
d799c98cfe io: move common function and add unit tests for it 2019-11-06 17:12:33 +03:00
Roman Khimov
9f7625d699 core: don't do useless DB reads if nothing was persisted 2019-11-06 16:10:37 +03:00
Roman Khimov
90d4299f0b core: gofmt blockchain.go 2019-11-06 16:09:45 +03:00
Evgenii Stratonikov
ed758458d5 vm: implement serialization interops in core package 2019-11-05 17:10:52 +03:00
Roman Khimov
2f6e678a19
Merge pull request #463 from nspcc-dev/smartcontract-fixes 
Smartcontract RPC fixes
 2019-10-29 20:54:46 +03:00
Vsevolod Brekelov
d374175170 monitoring: add prometheus monitoring 
add init metrics service which uses prometheus;
add configuration for metrics service;
add monitoring metrics for blockchain,rpc,server;
 2019-10-29 20:51:17 +03:00
Roman Khimov
ebc1ba4f38 rpc/core: implement invokescript method, fix #348 
Extend Blockchainer with one more method to spawn a VM for test runs and use
it to run scripts. Gas consumption is not counted or limited in any way at the
moment (see #424).
 2019-10-29 18:31:39 +03:00
Roman Khimov
a568740f56 core: deduplicate a part of VM setup into spawnVMWithInterops() 2019-10-29 17:10:50 +03:00
Roman Khimov
99bbad331d core: add issue tx processing 
Count available asset quantity, match C# code.
 2019-10-25 17:20:12 +03:00
Roman Khimov
45cac07643 core: remove transactions from MemPool when they're stored in block 
Extend MemPool with Remove() and use it. Fixes #446.
 2019-10-24 13:12:02 +03:00