Commit graph

1355 commits

Author SHA1 Message Date
Evgenii Stratonikov
924d920423 smartcontract: support JSON unmarshaling of Signature param 2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
7eaeb18f18 transaction: marshal ContractTX even if Data is nil
It contains no information so it doesn't matter anyway.
2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
634e9483d3 transaction: unmarshal Witness properly
Both verification and invocation scripts need to
be unmarshaled from hex.
Also fix failing RPC tests: block contains non-pointer
`transaction.Witness` field and (*Witness).MarshalJSON method
is not called.
2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
33f99104e8 transaction: unmarshal Output properly
Address is marshaled in base58 and needs to be
decoded accordingly.
2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
b4f8d66bd3 smartcontract: allow to marshal nil parameters 2020-03-04 19:55:04 +03:00
Evgenii Stratonikov
7cde58f731 smartcontract: adjust parameter value to type properly 2020-03-04 19:49:50 +03:00
Roman Khimov
81d89cd502 core: fix SystemFee calculation for Invocation TXes
They have it specified right in the transaction. Unfortunately, this little
change rendered invalid our RPC test chain, but I think it became even better
after it, especially given that chain generation is a nice test by itself, so
it should be running as a regular test.
2020-03-04 19:23:23 +03:00
Roman Khimov
ae6edf0601 core: add a check for fractional gas in invocation TXes
It's not allowed in C# code and that's reasonable as it's a sysfee essentially
that can only be integer.
2020-03-04 19:23:23 +03:00
Roman Khimov
823798514a util: rename Int64Value to IntegralValue, add FractionalValue
Makes it's interface more clear and allows to easily access fractional part.
2020-03-04 19:23:23 +03:00
Roman Khimov
7cc847d655 transaction: add some missing InvocationTX decoding checks
Script must have something inside and GAS can't be negative.
2020-03-04 19:23:23 +03:00
Roman Khimov
a9d8c9e0d3 rpc/client: implement GetRawTransaction, fix #586 2020-03-03 20:43:57 +03:00
Roman Khimov
f01766131b rpc/client: add a set of GetBlock methods
They differ in input and output types (and data), thus four methods are
added.
2020-03-03 20:26:56 +03:00
Roman Khimov
685e34d83c rpc: update client documentation
Add missing methods, move example into something that can be compiled (and fix
it along the way).
2020-03-03 18:59:21 +03:00
Roman Khimov
f747b39bc6 docs: update RPC server documentation, add missing methods
And drop doc.go from server package as it duplicates docs/rpc.md and has no
value of its own (TODO list is managed with GitHub issues, really). Also, RPC
server is not really expected to be used by non-neo-go packages (contrary to
the client).
2020-03-03 18:18:56 +03:00
Roman Khimov
f5a1b928ce pkg: fix gofmt issues 2020-03-03 17:22:15 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Roman Khimov
59e6dac351
Merge pull request #708 from nspcc-dev/fix/trigger
core: convert trigger.Type to byte when pushing on stack
2020-03-03 16:17:01 +03:00
Evgenii Stratonikov
fe3bbbd78a core: convert trigger.Type to byte when pushing on stack 2020-03-03 16:13:19 +03:00
Roman Khimov
032cb513a6
Merge pull request #707 from nspcc-dev/skip-outdated-headers-on-add
core: don't reverify stale headers in addHeader
2020-03-03 15:43:40 +03:00
Roman Khimov
4c8d327353 rpc: drop duplicating Invoke* structures
And use smartcontract.Parameter instead of vm.StackItem where
appropriate. Closes #689.
2020-03-03 15:38:03 +03:00
Roman Khimov
56e37ad6ba vm: drop duplicating stackItem structure, build JSON from Parameters
smartcontract.Parameter has everything needed now.
2020-03-03 15:38:03 +03:00
Roman Khimov
9b4fd99fbc vm: break circular references when recursing into ToContractParameters
Reference types can have circular pointers to each other, thus we need to
control recursion.
2020-03-03 15:38:03 +03:00
Roman Khimov
c3e73c5b7d core: don't reverify stale headers in addHeader
During networked synchronization we expect there to be a lot of duplicate
headers received and it makes no sense for us reverifying them.
2020-03-03 15:34:03 +03:00
Roman Khimov
3282c6ed41
Merge pull request #704 from nspcc-dev/feature/getrawmempool
rpc: implement getrawmempool RPC
2020-03-02 19:44:19 +03:00
Anna Shaleva
e1fe12a07f rpc: implement getrawmempool RPC
closes #175
2020-03-02 19:35:51 +03:00
Evgenii Stratonikov
1264b629f1 smartcontract: parse Struct item type same way as Array 2020-03-02 18:05:26 +03:00
Evgenii Stratonikov
05a3625b7d wallet: implement (*Account).SignTx
It is used in both CLI and RPC.
2020-03-02 18:03:56 +03:00
Evgenii Stratonikov
6541bd4d42 vm: use Boolean in (*BoolItem).String() 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
6add4f3e50 transaction: disallow negative outputs
Otherwise it is possible to make outputs which will sum
to the expected value, but steal GAS from some other account.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
9e9d59b49a core: set NetworkFee to 0 for Claim and Miner transactions
Claim tx have no GAS inputs and a positive output which
can lead to negative network fee.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
21ef2638c0 consensus: log error if GetValidators() failed 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
de5215a564 core: use GetAccountStateOrNew() in (*Blockchain).GetValidators()
Target of the transaction output may not yet exist in database.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
1b9968df67 core: optimize CalculateClaimable()
Because accumulated system fee is stored for every block,
it is easy to calculate sum with just to reads.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
258d8be1dd core: store all accumulated SystemFee with every block 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
f69c8a763b core: store system fee together with block
Recalculating system fee can be rather costly if done
frequently.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
972e0d8ad1 core: add one more Contract tx to the test chain
When testing CLI it is useful to have some spent coins
on an account with a known key.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
37c2bc4733 rpc: implement (*Client).GetClaimable()
This is needed to form correct Claim transaction.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
45b8669b42 core: verify Claim transactions 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
9d5841df38 rpc: make (*Client).SendRawTransaction public 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
44792ed5f7 core: make test chain format compatible with mainnet
Write uint32 length before every block.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
51c4868641 rpc: implement getclaimable RPC 2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
0e2a1f40ba core: add ContractTX to a testchain generator
Simple transfer from multisig account to the account
of one of the validators.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
72d72296c3 core: implement (*Transaction).GetSignedPart()
Marshalling it and taking all but last byte violates incapsulation
and is just wrong in case transaction already contains any witnesses.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
95d9f36c98 core: implement UnclaimedBalance tracking
To make it easy to get unclaimed coins for the
specified account they must be tracked together.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
7095ec6c51 core: implement (*Blockchain).CalculateClaimable
Calculating amount of GAS that can be claimed is required
for getclaimable RPC.
2020-03-02 18:00:00 +03:00
Roman Khimov
252a9f2f31
Merge pull request #690 from nspcc-dev/feature/getapplicationlog
rpc: implement getapplicationlog RPC
2020-03-02 17:41:32 +03:00
Anna Shaleva
ff4384d7ff rpc: implement getapplicationlog RPC
Closes #500
2020-03-02 17:25:27 +03:00
Anna Shaleva
7d46404e2d smartcontract: turn trigger types into Type
1) Turn trigger types from byte constants into Type
2) Add auto-generated stringer for future purposes
2020-03-02 17:25:27 +03:00
Anna Shaleva
b3621d4a86 smartcontract: remove unused structures
Removed following unused structures:
	-smartcontract.ContextItem
	-smartcontract.Signature
	-smartcontract.ParameterContext
2020-03-02 17:25:27 +03:00
Anna Shaleva
535f391550 smartcontract: add marshaller for Parameter
1) Add marshaller and tests for smartcontract.Parameter
2) Add unmarshaller and tests for missing types of smartcontract.Parameter:
	- MapType
	- BoolType
2020-03-02 17:25:27 +03:00
Anna Shaleva
648e0bb242 rpc/smartcontract: merge contract parameter types
Merged two types:
	- smartcontract.ParamType
	- rpc.StackParamType
into single one:
	- smartcontract.ParamType
as they duplicated the functionality.

NOTE: type smartcontract.MapType was added (as in C# implementation).

From now, list of supported smartcontract parameter types:
	UnknownType
	SignatureType
	BoolType
	IntegerType
	Hash160Type
	Hash256Type
	ByteArrayType
	PublicKeyType
	StringType
	ArrayType
	MapType
	InteropInterfaceType
	VoidType
2020-03-02 17:25:21 +03:00
Evgenii Stratonikov
a3dacd3b74 tests: replace t.Fatal with require where possible
This makes tests less verbose and unifies the style
they are written in.
2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
66f96e3f32 core: shutdown Blockchain gracefully in tests 2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
3a5224344e core: verify headers in AddHeaders()
Headers can be malformed so public methods should verify them
before adding.
2020-03-02 17:22:26 +03:00
Evgenii Stratonikov
357bb4ce41 core: get rid of global variables in tests
It can lead to unnecessary race conditions and is just
a bad practice.
2020-03-02 17:04:08 +03:00
Evgenii Stratonikov
8ca94e23c8 core: replace makeBlocks() with addBlocks() in tests
This simplifies tests a bit.
2020-03-02 17:04:08 +03:00
Evgenii Stratonikov
cbf26bac83 mempool: do not allocate new slice for verified transactions
Because transactions a iterated in an increasing order,
we can filter slice in-place.
2020-03-02 10:38:27 +03:00
Evgenii Stratonikov
0ef7a76e84 network: batch transactions on broadcast 2020-03-02 10:37:27 +03:00
Roman Khimov
ff551d2015
Merge pull request #700 from nspcc-dev/fix/forloop
compiler: do not short-circuit in complex conditions
2020-02-28 20:14:26 +03:00
Roman Khimov
36197056b1
Merge pull request #691 from nspcc-dev/network-conn-overflow-and-shutdown
Network conn overflow and shutdown
2020-02-28 18:16:29 +03:00
Evgenii Stratonikov
b461a6ab63 compiler: do not short-circuit in complex conditions
Current implementation of short-circuting is just plain wrong
as it uses `last` or `before-last` labels which meaning depend
on context. It doesn't even handle simple assignements like
`a := x == 1 && y == 2`.

This commit makes all jumps in such conditions local
and adds tests.

Closes #699, #700.
2020-02-28 17:44:46 +03:00
Roman Khimov
e213e69a33 network: rework shutdown sequence of Server
Close transport and disconnect peers right in the Shutdown(), so that no new
connections would be accepted and so that all the peers would be disconnected
correctly (avoiding the same deadlock as in e2116e4c3f).
2020-02-28 16:22:04 +03:00
Roman Khimov
77624a8847 network: add Close() to discoverer, shut it down on exit 2020-02-28 16:22:04 +03:00
Evgenii Stratonikov
624a14f730 rpc: rename (*Client).SendToAddress to TransferAsset
SendToAddress name is similar to the `sendtoaddress` RPC call,
this can cause confusion.

Closes #686.
2020-02-28 14:16:39 +03:00
Roman Khimov
3e2b490025 mempool: rework inputs verification, check Claim txes
Use more efficient check for Input and also check Claims to avoid double
claiming.
2020-02-27 13:43:35 +03:00
Roman Khimov
f329de73e8 mempool: fix Remove bug (dropping wrong tx!)
Uncovered by TestMemPoolAddRemoveWithInputsAndClaims (to be added later)
2020-02-27 13:43:35 +03:00
Roman Khimov
7d59fa0066
Merge pull request #696 from nspcc-dev/tx-verification-fixes
Claim and enrollment TX verification fixes
2020-02-27 12:45:36 +03:00
Roman Khimov
00d199ed85 core: fix NetworkFee calculation
Add doesn't change the variable state. Thanks to GolangCI for catching this.
2020-02-27 12:32:56 +03:00
Roman Khimov
7bea6e043e core: add double claim verification check 2020-02-27 12:32:56 +03:00
Roman Khimov
29586f2aa7 core: add duplicate Claim verification check
Refactor HaveInputsDuplicate() out of the core and Blockchain, it doesn't
depend on the Blockchain state. Make it more efficient.
2020-02-27 12:32:56 +03:00
Roman Khimov
9b1ad40273 core: drop GetScriptHashesForVerifyingClaim, fix Claim's ScriptHash getter
Claim transactions should _add_ claims scripthashes to the standard list, not
replace them. And this code is actually very reusable.
2020-02-27 12:32:56 +03:00
Roman Khimov
9f7018503a core/transaction: use new transaction.InOut for References
We don't need a map here, use simpler structures.
2020-02-27 12:00:03 +03:00
Roman Khimov
80de208a68 transaction: unbind GroupInputsByPrevHash from Transaction
Which allows to use it for ClaimTX. Make it also a bit more efficient as maps
are expensive.
2020-02-27 12:00:03 +03:00
Roman Khimov
71a18b1727
Merge pull request #688 from nspcc-dev/fix/jmplabels
compiler: use uint16 for labels
2020-02-27 00:28:46 +03:00
Roman Khimov
b1b184f77f
Merge pull request #692 from nspcc-dev/persist-timer-fix
core: reschedule the next persist in a second from previous one
2020-02-26 10:34:40 +03:00
Roman Khimov
2f2a4afe21
Merge pull request #693 from nspcc-dev/memorystore-seek-lock
storage: add locking into (*MemoryStore).Seek
2020-02-26 10:15:40 +03:00
Roman Khimov
c258adb532 core: move spent coin management out of the inner storeBlock loop
prevHash == input.PrevHash, so make less DB accesses and more real work. Fix
some bugs along the way:
 * spentCoins structure may already be present in the DB when persisting TX,
   there is nothing wrong with that and we shouldn't overwrite it
 * it's only used for NEO and only to check for claim validity. Thus, when
   processing claim tx the corresponding spentCoins should always be present
   in the DB
2020-02-25 17:21:09 +03:00
Roman Khimov
36c6b6af14 core: put unspents get/put one loop above in storeBlock
Everywhere in this code prevHash == input.PrevHash, thus we can easily move
some common code out of the loop saving on DB accesses and
serialization/deserialization.
2020-02-24 18:53:10 +03:00
Roman Khimov
5c9681508b transaction: strip off a layer of redirection from ClaimTX.Claims
We don't need a pointer here and this change makes this field compatible with
Transaction.Inputs which is useful in many scenarios.
2020-02-24 18:22:27 +03:00
Roman Khimov
ed9e9e8590 storage: add locking into (*MemoryStore).Seek
It was missing there.
2020-02-24 17:51:50 +03:00
Roman Khimov
b49808b766 core: reschedule the next persist in a second from previous one
It makes little sense queueing up several persistence goroutines (or actually
even running them concurrently).
2020-02-24 17:17:25 +03:00
Roman Khimov
e2116e4c3f network: don't deadlock on connection overflow
(*Peer).Disconnect send an unregister signal to this goroutine, so invoking it
from here is not a good idea, run it asynchronously.
2020-02-24 15:56:49 +03:00
Roman Khimov
d5e6028c13 core: add validator public key into the list of hashes to verify
That should be done for Enrollment transactions.
2020-02-21 20:04:16 +03:00
Anna Shaleva
76a0a6e7e8 rpc: implement getblocksysfee RPC
Closes #341
2020-02-21 18:45:52 +03:00
Evgenii Stratonikov
177b725dc1 compiler: make writeJumps return error for bad jumps
The script is invalid anyway so it is better to notify user.
2020-02-21 17:45:45 +03:00
Evgenii Stratonikov
5e229d84d4 compiler: use uint16 for label numbers
As noted in #687 this will make compiler a bit more predictable.
2020-02-21 17:45:45 +03:00
Roman Khimov
8da20055d6 rpc: drop duplicating structures used in tests
We have proper results now, so use those. The only left is Invoke, but that
depends on another issue at the moment.
2020-02-21 15:23:11 +03:00
Roman Khimov
6a3be6081c smartcontract: support "ByteArray" string for ByteArrayType
We actually do emit it ourselves in String()
2020-02-21 15:23:11 +03:00
Roman Khimov
3fa9de764b rpc/client: only return the Result from calls, handle Error internally
Adjust structures accordingly and throw away most of them, they're useless.
2020-02-21 15:23:11 +03:00
Roman Khimov
28a26d2cae rpc/client: look into data first, then HTTP error code
In case of error our own server responds with an HTTP error and proper
JSON-RPC error in the body, so look there first as it has more specific data.
2020-02-21 15:12:04 +03:00
Roman Khimov
877b987ecf rpc: omitempty some fields in TransactionOutputRaw
These might be undefined for mempool transactions, thus they should be defined
as omitempty.
2020-02-21 15:12:04 +03:00
Roman Khimov
bba8ac15ff rpc: introduce proper type for sendrawtransaction, drop useless types
sendrawtransaction just returns a bool, sendtoaddress returns a proper
transaction and that should be the same as the one we have in
TransactionOutputRaw.
2020-02-21 15:12:04 +03:00
Roman Khimov
465476201c rpc/response: drop duplicating AccountState structure 2020-02-21 15:12:04 +03:00
Evgenii Stratonikov
b50704fd3b rpc: move server-related code to a separate package 2020-02-21 15:12:04 +03:00
Evgenii Stratonikov
d24c6d1d9e rpc: move client-related code to a separate package
This includes Client struct with RPC methods and
BalanceGetter implementation with NeoSCAN.
2020-02-21 15:12:04 +03:00
Evgenii Stratonikov
b6bc4e580a rpc: move functions for Tx building in request package/ 2020-02-21 15:12:04 +03:00
Evgenii Stratonikov
98a1e2170a rpc: move ContractDetails to request/ package 2020-02-21 15:12:04 +03:00
Evgenii Stratonikov
63751a6e6b rpc: merge neoscan-related definitions into one file
It will make refactoring easier.
2020-02-21 15:12:04 +03:00
Roman Khimov
f330f2f40b rpc: separate out request and response structures
Mostly as is, no real effort done yet to optimize them, so there are still a
lot of duplicates there, but at least we sort them out into different smaller
packages.
2020-02-21 15:12:04 +03:00
Roman Khimov
69e1ad512f rpc: move result package into response
`response` is to be populated soon.
2020-02-21 15:12:04 +03:00
Roman Khimov
1801e545a0 rpc: remove wrappers package, move things to result
These are all RPC call results, `wrappers` package doesn't make much sense to
me.
2020-02-21 15:12:04 +03:00
Roman Khimov
b8f7ab8e6a rpc: move Invoke to result package
It's just a data.
2020-02-21 15:12:04 +03:00
Roman Khimov
c189062f40 rpc: move ValidateAddress to the result package
Because it is a result of the RPC call.
2020-02-21 15:12:04 +03:00
Roman Khimov
bfa2d54e16 rpc: move validateAddress() function from wrappers into server
It's a server implementation detail, it has nothing to do with the data format
itself. It also makes no sense exporing it.
2020-02-21 15:12:04 +03:00
Roman Khimov
72a62f1292 result: add trivial test for peers management functions 2020-02-21 15:12:04 +03:00
Roman Khimov
723b33e108 network: implement unconnected/bad peers getters
Which allows node to respond to `getpeers` RPC request correctly.
2020-02-21 15:12:03 +03:00
Roman Khimov
d92e193e63 rpc/network: refactor getpeers logic
Deduplicate and simplify code.
2020-02-21 15:12:03 +03:00
Roman Khimov
336a94456f rpc: make blockHeightFromParam a pointer method
Makes no sense copying Server here.
2020-02-21 15:12:03 +03:00
Roman Khimov
f345db58ab
Merge pull request #678 from nspcc-dev/feature/breakfor
compiler: support break statement in for loops and switch statements

Closes #677.
Implements 4-th point from #628.
2020-02-21 12:22:16 +03:00
Evgenii Stratonikov
da89f18999 compiler: support break and continue in range loops 2020-02-21 12:03:37 +03:00
Evgenii Stratonikov
ccb53414f2 compiler: support break in switch statements 2020-02-21 12:03:37 +03:00
Evgenii Stratonikov
03dc6f7cbb compiler: support continue statement in for loops 2020-02-21 12:03:36 +03:00
Evgenii Stratonikov
fa2edc46e0 compiler: support break statement in for loops 2020-02-21 11:38:23 +03:00
Evgenii Stratonikov
25ffb56982 wallet: support creating multisig accounts
(*Account).ConvertMultisig() will convert an existing account
into a multisig one.
2020-02-21 11:19:35 +03:00
Evgenii Stratonikov
5a727cabf8 wallet: add signature check contracts to new accounts 2020-02-21 10:45:57 +03:00
Evgenii Stratonikov
eacea8bff5 smartcontract: implement json.Unmarshaler for ParamType 2020-02-20 15:52:08 +03:00
Evgenii Stratonikov
ea30122a09 wallet: rewrite file on save
When we are opening a file, it is expected that
it will be rewritten, not appended to a already existing wallet.
2020-02-20 15:52:08 +03:00
Evgenii Stratonikov
ad6fa2aea9 wallet: set WIF and public key on account decrypt
They are set during account creation and open+decrypt is
expected to put account in the same state.
2020-02-20 15:52:08 +03:00
Evgenii Stratonikov
d837eb3761 wallet: allow to create accounts from encrypted WIFs 2020-02-20 15:52:08 +03:00
Roman Khimov
a903147b60 keys: rename Signature to GetScriptHash, make it return Uint160
Signature itself wasn't used at all and its name is very misleading, Uint160
script hash is way more useful.
2020-02-19 12:19:03 +03:00
Roman Khimov
eb11e5fb11 core: implement basic policying support, fix #370
Implement mempool and consensus block creation policies, almost the same as
SimplePolicy plugin for C# node provides with two caveats:
 * HighPriorityTxType is not configured and hardcoded to ClaimType
 * BlockedAccounts are not supported

Other than that it allows us to run successfuly as testnet CN, previously our
proposals were rejected because we were proposing blocks with oversized
transactions (that are rejected by PoolTx() now).

Mainnet and testnet configuration files are updated accordingly, but privnet
is left as is with no limits.

Configuration is currently attached to the Blockchain and so is the code that
does policying, it may be moved somewhere in the future, but it works for
now.
2020-02-19 12:19:02 +03:00
Roman Khimov
22f5667530 mempool: return fee along with tx when requesting tx
Users of GetVerifiedTransactions() don't want to recalculate tx fee and it's
nice to have it returned from TryGetValue() also sometimes.
2020-02-19 12:19:02 +03:00
Roman Khimov
06daeb44f3 core: make IsLowPriority work with pre-calculated fee
Don't recalculate it again and again.
2020-02-19 12:19:02 +03:00
Roman Khimov
37c48b00b4 consensus/network: reinit dbft after block addition
Don't stall on some height if everyone else have moved up already. Fix #673.
2020-02-19 12:19:02 +03:00
Roman Khimov
c5d54e9992 network: introduce (*Server).IsInSync, start consensus in synced state
We define synchronized state as a combination of minimum number of peers and
chain height being not behind of more than 2/3 of these peers.
2020-02-19 12:13:27 +03:00
Roman Khimov
a2616cfafe consensus: correctly generate miner TX outputs based on net fee 2020-02-19 12:13:27 +03:00
Roman Khimov
17d1b1a2ae wallet: implement GetChangeAddress()
To select the default address to send tokens to.
2020-02-19 12:13:27 +03:00
Roman Khimov
a4294f4b5f core: export UtilityTokenID and GoverningTokenID
Both are very useful outside of the core, this change also makes respective
transactions initialize with the package as they don't depend on any kind of
input and it makes no sense recreating them again and again on every use.
2020-02-19 12:13:27 +03:00
Roman Khimov
b41a5239c6 core: always sort the result of GetValidators
It wasn't sorted when all validators were elected. There is also no need to do
`Unique()` on the result because validators are distinguished by the key, so
no two registered validators can have the same key.
2020-02-18 20:36:22 +03:00
Evgenii Stratonikov
ee8514d88b consensus: provide message type and view inside RecoveryMessage
PrepareRequest inside RecoveryMessage should contain both
message type and view.
2020-02-18 15:25:29 +03:00
Evgenii Stratonikov
9b82bbaa8a rpc: implement getstorage RPC 2020-02-18 11:59:31 +03:00
Evgenii Stratonikov
a2f012e589 core: deploy new smart-contract in test blocks
Also put smart-contract sources in testdata and
add invocation TX with APPCALL and storage.Put.
2020-02-18 11:59:28 +03:00
Anna Shaleva
2b9e63c511 core: add func to generate RPC test data
newRPCTestChain function generates test chain "testblocks.acc" for RPC
tests
2020-02-17 23:54:25 +03:00
Anna Shaleva
c99b42f738 rpc: add tests for getcontractstate RPC 2020-02-17 23:54:25 +03:00
Anna Shaleva
3c63ef3dc3 rpc: implement getcontractstate RPC 2020-02-17 23:54:25 +03:00
Anna Shaleva
b8715ddf11 rpc: refactor code of NewAccountState function
Simplified reversing of scriptHash
2020-02-17 23:52:20 +03:00
Anna Shaleva
f310145612 rpc: fix getrawtransaction error returncode
Changed returncode of getrowtransaction method in case when transaction
with specified hash does not exists. Now it returns error with code -100
instead of -32602 (as in c# node)
2020-02-17 23:52:20 +03:00
Anna Shaleva
5dca765bd1 rpc: add custom error with code -100
Fixed getassetstate method: it should return error with code -100 when
asset with specified id is not found (as in c# node).
2020-02-17 23:52:20 +03:00
Evgenii Stratonikov
7ee10ecea5 rpc: implement gettxout RPC 2020-02-17 11:58:18 +03:00
Roman Khimov
dfb38e1da1
Merge pull request #649 from nspcc-dev/feature/dump_storage
cli: add possibility to dump storage changes on restore
2020-02-16 23:50:31 +03:00
Roman Khimov
d16121c10a
Merge pull request #654 from nspcc-dev/feat/subslice
compiler: support sub-slicing
2020-02-16 23:49:53 +03:00
Roman Khimov
4a4cd6416c
Merge pull request #658 from nspcc-dev/feature/range
compiler: support for-range loops
2020-02-16 23:49:30 +03:00
Roman Khimov
b5453ad24f
Merge pull request #672 from nspcc-dev/fix-sigsegv-on-reference-check
core: deal with bad transaction Inputs
2020-02-14 13:23:42 +03:00
Roman Khimov
0fceaf12d2 core: deal with bad transaction Inputs
They may not really have a reference. Fixes #671.
2020-02-14 09:42:10 +03:00
Roman Khimov
992ccc0b26
Merge pull request #664 from nspcc-dev/sort-validator-keys
core: sort GetValidators() result
2020-02-13 21:46:26 +03:00
Evgeniy Kulikov
8d410a6659
Add JSON tags for RPC response types
- Attribute should have 2 fields (usage, data)
- VOut should have 4 (5) fields (asset, value, address, n)
- Script should have 2 fields (invocation, verification)
2020-02-13 19:06:54 +03:00
Roman Khimov
d8a8dc94b4
Merge pull request #657 from nspcc-dev/fix/struct_fields
compiler: support initializing struct fields from a variable
2020-02-13 17:54:01 +03:00
Roman Khimov
d4cd134aef core: sort GetValidators() result
As C# node does it. Technically it's only needed for consensus and could be
implemented in the appropriate package, but for better compatibility with C#
node we're better returning it sorted right here.
2020-02-13 17:07:27 +03:00
Roman Khimov
d1a2296939 network: change the disconnect procedure
We can still lock the (*Server).run with dead peers:

Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: goroutine 40 [select, 871 minutes]:
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).putPacketIntoQueue(0xc030ab5320, 0xc02f251f20, 0xc00af0dcc0, 0x18, 0x40, 0x100000000000000, 0xffffffffffffffff)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:82 +0xf4
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).EnqueueHPPacket(0xc030ab5320, 0xc00af0dcc0, 0x18, 0x40, 0x1367240, 0xc03090ef98)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:124 +0x52
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).iteratePeersWithSendMsg(0xc0000ca000, 0xc00af35800, 0xcb2a58, 0x0)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:720 +0x12a
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).broadcastHPMessage(...)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:731
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).run(0xc0000ca000)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:203 +0xee4
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).Start(0xc0000ca000, 0xc000072ba0)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:173 +0x2ec
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: created by github.com/CityOfZion/neo-go/cli/server.startServer
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/cli/server/server.go:331 +0x476
...
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: goroutine 2199 [chan send, 870 minutes]:
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).Disconnect.func1()
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:366 +0x85
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: sync.(*Once).Do(0xc030ab403c, 0xc02f262788)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/usr/local/go/src/sync/once.go:44 +0xb3
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).Disconnect(0xc030ab4000, 0xd92440, 0xc000065a00)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:365 +0x6d
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).SendPing.func1()
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:394 +0x42
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: created by time.goFunc
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/usr/local/go/src/time/sleep.go:169 +0x44
...
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: goroutine 3448 [chan send, 854 minutes]:
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).handleConn(0xc01ed203f0)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:143 +0x6c
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: created by github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).Accept
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_transport.go:62 +0x44c
...

The problem is that the select in putPacketIntoQueue() only works the way it
was intended to after the `close(p.done)`, but that happens only after
successful unregistration request send. Thus, do disconnects the other way
around, first unblock queueing and exit goroutines, then destroy the
connection (if it wasn't previously destroyed) and only after that signal to
the Server.
2020-02-13 16:24:46 +03:00
Roman Khimov
ef870d0c9d
Merge pull request #661 from nspcc-dev/make-voting-system-work
Make voting system work
2020-02-13 15:16:29 +03:00
Roman Khimov
26af9200b0 core: refactor voting processing, fix pending tx subtractions
Deduplicate code and add missing token check for subtractions that are being
done in GetValidators() for pending transactions.
2020-02-13 13:07:34 +03:00
Roman Khimov
c8a248596e core: introduce ValidatorsCount, make a proper count
We were completely lacking ValidatorsCount that is supposed to track the
number of votes with particular count of consensus nodes which in theory can
change the number of active consensus nodes (if it ever to exceed the number
of standby validators), so we were not producing the right count and based on
that not giving the right set of validators.

Fixes #512.
2020-02-13 13:07:34 +03:00
Roman Khimov
357b675090 core: don't choose duplicating standby validators
We may already have chosen some of standby validators, so don't repeat.
2020-02-13 13:07:34 +03:00
Roman Khimov
f9c6a0d77c core: make voting choose the best validators
They need to be sorted in order for voting system to work.
2020-02-13 13:07:34 +03:00
Roman Khimov
2ff7ceb8cf core: properly update validator votes for Account descriptor
ValidatorsCount is not implemented yet, but we need to update validators
properly, adding account's vote to their votes count.
2020-02-13 13:07:34 +03:00
Roman Khimov
8310b76d28 core: properly save votes when processing Account descriptor
They were ignored basically.
2020-02-13 13:07:34 +03:00
Roman Khimov
807309a0de core/state: only drop unregistered non-voted validators
Simple as that: UnregisteredAndHasNoVotes != !RegisteredAndHasVotes

Registered validators should stay in the DB, we might be in the process of
updating votes for them and that starts with subtraction.
2020-02-13 13:07:34 +03:00
Evgeniy Kulikov
64e325d9af
Update tests for getblock action
- replace `GetBlockResponse.Result` with `wrappers.Block`
- update `getblock` checker and add more assets
2020-02-12 21:58:43 +03:00
Roman Khimov
32a064aa31 keys: add Cmp method to PublicKey
It can be used by code that doesn't operate with PublicKeys, but still needs
to be able to compare keys for some purposes.
2020-02-12 21:27:40 +03:00
Roman Khimov
38d0efc96c core: fix Registered state descriptor processing
It has simple boolean value encoded, no `strconv` needed (it actually chokes
on such input).
2020-02-12 21:27:40 +03:00
Roman Khimov
141553da4c keys: fix PublicKeys decoding
It changes the slice, thus it has to work via a pointer.
2020-02-12 21:23:28 +03:00
Roman Khimov
803fb39bb0 transaction: fix wrong state descriptor serdes
Wrong field order.
2020-02-12 21:23:28 +03:00
Evgeniy Kulikov
41ebf12eb3
Update getblock RPC action response 2020-02-12 19:43:51 +03:00
Evgenii Stratonikov
0a894db7f8 storage: add Exists flag to KeyValue in batch
Set Exists flag if an item with the specified key was already
present in storage before persisting.
2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
b1d9e1132d cli: dump storage changes into JSON files 2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
fb9af98179 storage: implement GetBatch() to view storage changes
GetBatch returns changes to be persisted.
2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
8ea3ef0b62 compiler: process empty []byte{} literals as ByteArrays
It is wrong to count an empty byte slice as an Array.
2020-02-12 12:16:15 +03:00
Evgenii Stratonikov
32bce30777 compiler: support sub-slicing 2020-02-12 12:16:15 +03:00
Evgenii Stratonikov
310f7449d7 compiler: support for-range loops 2020-02-12 12:16:04 +03:00
Evgenii Stratonikov
ae03560589 compiler: support initializing struct fields from a variable
While initializing a struct, it is a top item on ALTSTACK.
This means that if we need to load a local variable,
DUPFROMALTSTACK won't longer push an array of locals on stack
but rather a currently initializing struct.

Closes #656.
2020-02-12 12:15:54 +03:00
Evgenii Stratonikov
3e84f2bdf8 compiler: rewrite jump targets properly
Old implementation could view 0x62 byte in
a script as a JMP instruction irregardless of whether it is
a real opcode or a part of a parameter of another instruction.
In this commit instructions are decoded together with parameters
during jump label rewriting.
2020-02-12 10:58:53 +03:00
Roman Khimov
d3badc3f97 consensus: log more like neo-cli, update dbft 2020-02-11 18:02:13 +03:00
Roman Khimov
38de2af073
Merge pull request #655 from nspcc-dev/fix-mempool-deadlock-on-reverification
core: get transactions from dao when mempool should not be used
2020-02-11 13:34:23 +03:00
Roman Khimov
292051d8df core: get transactions from dao when mempool should not be used
All of these places deal strictly with the chain and shouldn't ever be
bothered with mempool. It also fixes a deadlock on reverification of
non-standard tx:

1 @ 0x42f62f 0x43fbe9 0x43fbbf 0x43f95d 0x967059 0x966f66 0x972c7c 0x974e13 0x97a5d9 0x97bdf0 0x976147 0x966cc0 0x970f70 0x96c8cb 0x9ba858 0x45ca51
	0x43f95c	sync.runtime_SemacquireMutex+0x3c							/usr/local/go/src/runtime/sema.go:71
	0x967058	sync.(*RWMutex).RLock+0x128								/usr/local/go/src/sync/rwmutex.go:50
	0x966f65	github.com/CityOfZion/neo-go/pkg/core/mempool.(*Pool).TryGetValue+0x35			/go/src/github.com/CityOfZion/neo-go/pkg/core/mempool/mem_pool.go:229
	0x972c7b	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).GetTransaction+0x4b			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:782
	0x974e12	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).References+0x132			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:944
	0x97a5d8	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).GetScriptHashesForVerifying+0x58	/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1410
	0x97bdef	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).verifyTxWitnesses+0x4f		/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1545
	0x976146	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).isTxStillRelevant+0x216		/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1067
	0x966cbf	github.com/CityOfZion/neo-go/pkg/core/mempool.(*Pool).RemoveStale+0xff			/go/src/github.com/CityOfZion/neo-go/pkg/core/mempool/mem_pool.go:208
	0x970f6f	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).storeBlock+0x2ecf			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:614
	0x96c8ca	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).AddBlock+0xea			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:308
	0x9ba857	github.com/CityOfZion/neo-go/pkg/network.(*blockQueue).run+0x157			/go/src/github.com/CityOfZion/neo-go/pkg/network/blockqueue.go:48
2020-02-10 18:53:22 +03:00
Roman Khimov
0461827fa3
Merge pull request #653 from nspcc-dev/fix-net-locks-and-leaks
network: fix networking stalls caused by stale peers
2020-02-10 18:51:56 +03:00
Roman Khimov
f7749cea67
Merge pull request #652 from nspcc-dev/Fix_FreeGasLimit
core: fix FreeGasLimit constraint
2020-02-10 18:49:09 +03:00
Roman Khimov
7ee8f9c5d8 network: fix networking stalls caused by stale peers
We can leak sending goroutines and stall broadcasts because of already gone
peers that happened to be cached by some s.Peers() user (more than 800 of
these can be seen in nodoka log along with (*Server).run blocking on
CMDGetAddr send):

Feb 10 16:35:15 nodoka neo-go[1563]: goroutine 41 [chan send, 3320 minutes]:
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).putPacketIntoQueue(...)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:81
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).EnqueueHPPacket(0xc0083d57a0, 0xc017206100, 0x18, 0x40, 0x136a240, 0xc018ef9720)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:119 +0x98
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).iteratePeersWithSendMsg(0xc0000ca000, 0xc0001848a0, 0xcb4550, 0x0)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:720 +0x12a
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).broadcastHPMessage(...)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:731
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).run(0xc0000ca000)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:203 +0xee4
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).Start(0xc0000ca000, 0xc000072c60)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:173 +0x2ec
Feb 10 16:35:15 nodoka neo-go[1563]: created by github.com/CityOfZion/neo-go/cli/server.startServer
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/cli/server/server.go:331 +0x476
2020-02-10 18:47:52 +03:00
Anna Shaleva
9b03f968b1 core: fixe FreeGasLimit constraint
Lack of FreeGasLimit in privnet leads to gas limit exceeding in case of transactions with small amount of GAS to be used for invoke operation (< real cost of the transaction). Solution: Fixed constraint in case when FreeGasLimit == 0. So now we are able to perform transactions in privnet with FreeGasLimit = 0 for free.
2020-02-10 18:07:19 +03:00
Evgenii Stratonikov
895a8d9ebc compiler: reverse args in AppCall
Invoked contract is expecting first argument to be on top of the stack.
Change test to use non-commutative operation to catch this behaviour.
2020-02-10 10:53:58 +03:00
Evgenii Stratonikov
52d8d58593 compiler,interop: make AppCall accept varargs 2020-02-10 10:51:29 +03:00
Evgenii Stratonikov
1fc64d515f compiler: abstract out emitReverse
Extract logic of reversing top n items of the stack
in a separate function.
2020-02-10 10:43:31 +03:00
Roman Khimov
268cad5c06
Merge pull request #548 from nspcc-dev/feature/multisig
vm: implement parallel algorithm for CHECKMULTISIG
2020-02-08 16:10:34 +03:00
Roman Khimov
9b9adb28c1
Merge pull request #642 from nspcc-dev/feature/emit
vm, compiler: move Emit* functions to a separate package

Closes #449, #534.
2020-02-08 15:56:48 +03:00
Roman Khimov
c896e2b731 network: fix minor gofmt issues (spotted by goreportcard) 2020-02-08 15:53:08 +03:00
Roman Khimov
7ccf7974b6 core: substitute bad notifications with error messages
Tesnet sync failed with:
Feb 07 00:04:19 nodoka neo-go[1747]: 2020-02-07T00:04:19.838+0300        WARN        blockQueue: failed adding block into the blockchain        {"error": "failed to store notifications: not supported", "blockHeight": 713984, "nextIndex": 713985}

because some (not so) smart contract emitted a notification with an
InteropItem inside.
2020-02-07 15:34:59 +03:00
Roman Khimov
b805b1a71b vm: make SerializeItem/DeserializeItem public APIs
They're useful as wrappers around EncodeBinaryStackItem/DecodeBinaryStackItem.
2020-02-07 15:34:59 +03:00
Roman Khimov
7e7ce7f19f vm: rephrase serialization errors
Seeing some
    blockQueue: failed adding block into the blockchain        {"error": "failed to store notifications: not supported", "blockHeight": 713984, "nextIndex": 713985}
in logs is not very helpful.
2020-02-07 11:53:53 +03:00
Evgenii Stratonikov
a8dc704197 emit: add tests for Int 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
6fd3f0fa48 emit: add tests for Bytes 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
5e992d8cdd emit: add tests for Syscall, Jmp 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
dbc41b3044 compiler: replace emit* instructions with those from emit/ package 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
8243a8b3a7 emit: use io.BinWriter instead of bytes.Buffer 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
698c647f07 emit: refactor tests
Add structure and call Bytes() method on buffer once.
2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
1400ecfdde emit: fix Int to serialize integers in correct format
Related #605, #623.
2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
c821e1c4c8 vm: move IntToBytes and BytesToInt to emit package 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
4d8a3a359b vm: move Emit* functions to a separate package
Also strip 'Emit' prefix because 'emit' is now
in the package name.
2020-02-06 18:45:37 +03:00
Roman Khimov
ab14a4619d
Merge pull request #647 from nspcc-dev/fix-mempool-and-chain-locking
Fix mempool and chain locking

This allows us easily make 1000 Tx/s in 4-nodes privnet, fixes potential
double spends and improves mempool testing coverage.
2020-02-06 18:43:29 +03:00
Evgenii Stratonikov
9204b95d5b core: fix GAS price definitions 2020-02-06 18:19:08 +03:00
Roman Khimov
7445655437 consensus: switch test to using the new PoolTx API
Fixes GolangCI:
  Error return value of
  (*github.com/CityOfZion/neo-go/pkg/core/mempool.Pool).Add is not checked
  (from errcheck)

and allows us to almost completely forget about mempool here.
2020-02-06 17:50:11 +03:00
Roman Khimov
18695e660b mempool: drop RemoveOverCapacity(), handle it right in the Add()
Simplifies things a lot and removes useless code.
2020-02-06 17:50:11 +03:00
Roman Khimov
684cbf5bac mempool: make it almost 100% test-covered 2020-02-06 17:50:11 +03:00
Roman Khimov
6672f4b26f mempool: iterate over slice in GetVerifiedTransactions()
It's more efficient and keeps transactions sorted by priority.
2020-02-06 16:59:20 +03:00
Roman Khimov
e97396e56c mempool: test addition of conflicting tx to the pool 2020-02-06 15:57:20 +03:00
Roman Khimov
1133bbe584 mempool: remove unverified transactions pool
Our mempool only contains valid verified transactions all the time, it never
has any unverified ones. Unverified pool made some sense for quick unverifying
after the new block acceptance (and gradual background reverification), but
reverification needs some non-trivial locking between blockchain and mempool
and internal mempool state locking (reverifying tx and moving it between
unverified and verified pools must be atomic). But our current reverification
is fast enough (and has all the appropriate locks), so bothering with
unverified pool makes little sense.
2020-02-06 15:45:16 +03:00
Roman Khimov
b675903f52 mempool/core: redesign mempool dances on block acceptance
We not only need to remove transactions stored in the block, but also
invalidate some potential double spends caused by these transactions. Usually
new block contains a substantial number of transactions from the pool, so it's
easier to make one pass over it only keeping valid items rather than remove
them one by one and make an additional pass to recheck inputs/witnesses.
2020-02-06 15:45:16 +03:00
Roman Khimov
b567ce86ac mempool: implement insertion to sorted slice
Which is way faster than sort.Sort'ing things all the time.
2020-02-06 15:44:18 +03:00
Roman Khimov
35183b6dba mempool: reverse the order of sorted slice
Chopping off the last element of the slice if way easier than doing it with
the first one.
2020-02-06 15:44:16 +03:00
Roman Khimov
794027a90b mempool: use one slice for both priorities
It doesn't harm as we have transactions naturally ordered by fee anyway and it
makes managing them a little easier. This also makes slices store item itself
instead of pointers to it which reduces the pressure on the memory subsystem.
2020-02-06 15:43:45 +03:00
Roman Khimov
325bea3fa9 mempool: cache Feer invocation results in the item
They shouldn't depend on the chain state and for the same transaction they
should always produce the same result. Thus, it makes no sense recalculating
them over and over again.
2020-02-06 15:41:56 +03:00
Roman Khimov
e01bfeeb4d mempool: remove lock indirection from the Pool
After the f0bb886be3 with all methods of Pool
being pointer-based it makes no sense having this lock as a pointer.
2020-02-06 15:41:56 +03:00
Roman Khimov
a928ad9cfa mempool: make item an internal thing of mempool package
Nobody outside should care about these details, mempool operates on
transactions and that's it.
2020-02-06 15:41:56 +03:00
Roman Khimov
f0e3a31bc8 mempool: fix appending to sorted pools
Appending and not changing the real Items is utterly wrong.
2020-02-06 15:41:56 +03:00
Roman Khimov
b9b77ac1be network: fix block relaying, don't spit out useless errors
We can only add one block of the given height and we have two competing
goroutines to do that --- consensus and block queue. Whomever adds the block
first shouldn't trigger an error in another one.

Fix block relaying for blocks added via the block queue also, previously one
consensus-generated blocks were broadcasted.
2020-02-06 15:41:56 +03:00
Roman Khimov
f9963cca37 core: short-circuit verifyInputs when there are no inputs 2020-02-06 15:41:56 +03:00
Roman Khimov
70b3839fd0 core/mempool: fix AddBlock and tx pooling concurrency issues
Eliminate races between tx checks and adding them to the mempool, ensure the
chain doesn't change while we're working with the new tx. Ensure only one
block addition attempt could be in progress.
2020-02-06 15:41:52 +03:00
Evgenii Stratonikov
24bb66e606 util: fix a bug with not copying slice of len=1 in ArrayReverse
ArrayReverse copies it's argument only if it's len is > 1.
It needs to be consistent in all cases.
2020-02-06 12:15:35 +03:00
Evgenii Stratonikov
d07d6f3371 util: refactor tests for ArrayReverse 2020-02-06 12:15:35 +03:00
Evgenii Stratonikov
2a86149c82 vm: leave single CHECKMULTISIG implementation
Remove sequential implementation and benchmarks.
They will be still present in commit history.
2020-02-05 17:17:22 +03:00
Roman Khimov
02a5e036fc vm: deduplicate checkMultisigPar
Make its logic more clear.
2020-02-05 17:12:23 +03:00
Evgenii Stratonikov
d93d060010 vm: parallelize CHECKMULTISIG 2020-02-05 17:12:18 +03:00
Roman Khimov
f0bb886be3 mempool: make all methods pointer methods
Makes no sense copying the Pool around.
2020-02-04 17:36:11 +03:00
Roman Khimov
70c22ebc7b
Merge pull request #641 from nspcc-dev/consensus-update
Consensus update
2020-02-04 15:39:53 +03:00
Roman Khimov
9ceefe17e2 consensus: check current chain height before processing timer
The chain may already be more current than our dBFT state (like when the node
has commited something at view 0, but all the other nodes changed view and
accepted something at view 1), so in this case we should reinit dBFT on new
height.
2020-02-04 13:16:09 +03:00
Evgenii Stratonikov
e65b5c9914 consensus: fill preparation hash in recovery message 2020-02-04 13:16:05 +03:00
Evgenii Stratonikov
e441cb1414 consensus: add preparation hash to logger 2020-02-04 13:16:00 +03:00
Evgenii Stratonikov
42dfb52e98 consensus: add more logging in consensus service 2020-02-04 13:15:54 +03:00
Roman Khimov
553b2391a3 consensus: handle encoding errors in Hash() 2020-02-03 16:33:03 +03:00
Roman Khimov
388fed06e5 transaction: forbid serializing invalid transactions
Transaction that has no data is invalid and can't be serialized, so throw an
error if someone tries to.
2020-02-03 16:33:03 +03:00
Roman Khimov
9374c5e7c2
Merge pull request #639 from nspcc-dev/networking-improvements
Networking improvements
2020-02-03 16:26:04 +03:00
Evgenii Stratonikov
c3094123a1 compiler: do not store constants as variables
Because the constants are loaded directly via `emitLoadConst`, there is no need to store
them in an array of locals. It can have a big overhead, because it
is done at the beginning of every function.
2020-02-03 13:29:28 +03:00
Roman Khimov
e5d0125a3f network: do async broadcast when processing P2P TX
It can lead to some goroutine explosion, but supposedly it's better than
stalling other processing and eventually all of these goroutines should finish
their sends. Note that this doesn't change the behavior for RPC-relayed
transactions that are still waiting for the broadcast to finish ensuring
proper transaction distribution before returning the result to the client.
2020-01-30 14:03:52 +03:00
Roman Khimov
0fcbc697ca network: only tell consensus service about new valid transactions
Transactions can be bad and can duplicate each other, consensus service
shouldn't be bothered with that.
2020-01-30 14:03:52 +03:00
Roman Khimov
fdbaac7a30 network: prevent broadcast queue starving, share time with p2p
Blocked broadcast queue of one peer may affect broadcasting capabilities of
the server, so prevent total blocking of it by p2p queue.
2020-01-30 14:03:52 +03:00
Roman Khimov
b2c4587dad network: fix PeerAddr() for not-yet-handshaked case
If we have already got Version message, we don't need the rest of handshake to
complete before being able to properly answer the PeerAddr() requests. Fixes
some duplicate connections between machines.
2020-01-30 14:03:52 +03:00
Roman Khimov
49bd7aada5 network: log peerCount on connection registration
Symmetrical to disconnects, make it easier to look through the logs.
2020-01-30 14:03:52 +03:00
Roman Khimov
32695b4f40 network: don't unregister connected peers
If we drop connection because we're already connected don't unregister this
connected address because it's connected anyway!
2020-01-30 14:03:52 +03:00
Roman Khimov
eb4ec61b8b network: register connected addr in handleVersionCmd()
Prevent useless attempts to connect to this peer if the peer has already made
a connection to us.
2020-01-30 14:03:52 +03:00
Roman Khimov
9eafec0d1d network: introduce peer-to-peer message queue
This one is designed to give more priority to direct nodes communication, that
is that their messaging would have more priority than generic broadcasts. It
should improve consensus process under TX pressure and allow to handle
pings in time (preventing disconnects).
2020-01-30 14:03:52 +03:00
Roman Khimov
72e4eb7172 network: fix wrong NewPing() parameters
They have the opposite order, height first and nonce second. It was done wrong
in 4e6ed902 and never fixed since. Fixes sending wrong peer state leading to
useless getheaders messages (and disconnects when the other side is lagging
behind).
2020-01-30 14:03:52 +03:00
Roman Khimov
da09fda575 network: add debug on message receival 2020-01-30 14:03:52 +03:00
Roman Khimov
1c28dd2567 network: add message type to disconnect error message
If it was caused by message processing, but only after the handshake to
preserve errIdenticalID and other handshaking errors.
2020-01-30 14:03:52 +03:00
Roman Khimov
b79abd1e27 network: detect duplicate connections before handshake completes
We can have more than one connection attempt in progress and not yet completed
the handshake, so if there is a Version already received we should look it.
2020-01-30 14:03:52 +03:00
Roman Khimov
b1b660c779 rpc: fix getaccountstate/getunspents for unknown addresses
Returning error string as a result (not an error) is utterly wrong, but C#
implementation just returns a zero balance for unknown addresses, so we should
follow that.
2020-01-29 19:01:00 +03:00
Roman Khimov
0ccc59628c
Merge pull request #631 from nspcc-dev/fix/return3
compiler: fix a bug with assignment to underscore
2020-01-29 18:32:43 +03:00
Evgenii Stratonikov
390bf58f97 consensus: use our implementations of Recovery* messages
While decoding payload, local implementations of Recovery*
messages were used, but when creating RecoveryMessage inside dBFT
library default NewRecoveryMessage was invoked. This lead to parsing
errors.
2020-01-29 17:56:26 +03:00
Evgenii Stratonikov
8b922c057c compiler: fix a bug with assignment to underscore
When using underscore it does not appear in the list
of local variables, so it can't be assigned.
In this commit the value is dropped.
2020-01-29 17:07:55 +03:00
Evgenii Stratonikov
7053b3b2c0 compiler: optimize append argument processing
Append should leave it's result on top of the stack.
Thus we need to transform top of the stack:
(top) a . b --> (top) a . b . b
It can be done with just OVER + SWAP.
2020-01-29 12:28:38 +03:00
Evgenii Stratonikov
b6629fb6bd compiler: refactor argument handling for builtins
It is more convenient to have all unusual logic in one place.
2020-01-29 12:28:38 +03:00
Evgenii Stratonikov
d2326a8b96 compiler: support panic in source
In situations where VM's FAULT state needs to be reached,
panic function can be used. It compiles to THROW instruction.
2020-01-29 12:28:38 +03:00
Evgenii Stratonikov
e0f47decc7 compiler: replace ROLL(2) with equivalent ROT 2020-01-29 09:56:16 +03:00
Evgenii Stratonikov
28571bd3dc compiler: implement switch statement support 2020-01-29 09:56:16 +03:00
Roman Khimov
0ce0ecb95e
Merge pull request #624 from nspcc-dev/feature/bigint
compiler: emit integers correctly
2020-01-28 18:04:32 +03:00
Roman Khimov
ab03aee2cf
Merge pull request #627 from nspcc-dev/fix-getdata-and-pings
Fix getdata and pings
2020-01-28 17:43:13 +03:00
Roman Khimov
06c3fbe455 network: rework ping sends, fix overpinging
Our node was too pingy because of wrong timer setups (that divided timeout
Duration by time.Second), it also was wrong in its time calculations (using
UTC time to calculate intervals). At the same time missing block is a
server-wide problem, so it's better solved with server-wide protocol loop.
2020-01-28 17:39:52 +03:00
Roman Khimov
39800aecb3 network: fix getdata handling
It was broken by 0ba6b2a7, the peer only responded with one message instead of
full requested set.
2020-01-28 17:39:52 +03:00
Evgenii Stratonikov
d190b3a2e0 compiler: emit integers correctly
A while ago VM serialization format for Integer items was changed
but compiler continued to emit Integers in old format.
This commit changes compiler behaviour to be compatible with VM.
2020-01-28 16:39:19 +03:00
Evgenii Stratonikov
f15ceff592 vm: make IntToBytes, BytesToInt public
VM serialization format should be able to be reused.
2020-01-28 16:39:19 +03:00
Evgenii Stratonikov
77f9a2ee26 compiler: convert AppCall parameter from string properly 2020-01-27 15:34:03 +03:00
Evgenii Stratonikov
d65d6ab08d compiler: allow to convert string constants to []byte
Also load constant directly into stack, not by name.
2020-01-27 15:29:52 +03:00
Evgenii Stratonikov
097d35b9d5 compiler: fix a bug with FromAddress handling
Conversion of string to address with FromAddress is performed
at compile time so there is no need to push parameters on stack.
2020-01-27 13:14:40 +03:00
Evgenii Stratonikov
330db36168 compiler: implement engine.AppCall interop 2020-01-27 13:14:36 +03:00
Evgenii Stratonikov
4fd766fe09 compiler: allow usage of string literals in index expressions 2020-01-23 17:28:35 +03:00
Evgenii Stratonikov
058958729d compiler: support map literals 2020-01-23 17:06:15 +03:00
Evgenii Stratonikov
def73db8e9 compiler: support variables in slice literals 2020-01-23 15:51:16 +03:00
Evgeniy Kulikov
eb59460032
Merge pull request #614 from nspcc-dev/configurable-mempool-size
config: add MemPoolSize configuration parameter
2020-01-23 13:25:27 +03:00
Roman Khimov
67fe99b0ba
Merge pull request #618 from nspcc-dev/feature/for
compiler: support for loops with no init/post condition
2020-01-23 13:19:55 +03:00
Roman Khimov
04db87c770
Merge pull request #617 from nspcc-dev/feature/checksig
compiler: implement ECDSA signature verification
2020-01-23 13:18:21 +03:00
Roman Khimov
9aa5d0cf1e
Merge pull request #615 from nspcc-dev/network-peer-updates
Network peer updates
2020-01-23 13:15:43 +03:00
Roman Khimov
d6784e8f45 config: add MemPoolSize configuration parameter
Allow tuning/experimenting with mempool.
2020-01-23 13:10:44 +03:00
Evgenii Stratonikov
328267ca6f compiler: support for loops with no init/post condition
Make it possible to use `for` loop with a single condition.
2020-01-23 11:48:43 +03:00
Evgenii Stratonikov
bd37359393 compiler: implement ECDSA signature verification
Add VerifySignature interop for signature verification.
It is converted to VERIFY opcode.
2020-01-23 10:56:15 +03:00
Evgenii Stratonikov
303b82cc44 core: set free GAS limit via config 2020-01-22 14:32:21 +03:00
Evgenii Stratonikov
54e0b5a4a7 rpc: set MaxGasInvoke in config
Implement ability to restrict maximum amount of GAS
available for free RPC calls.
2020-01-22 14:31:57 +03:00
Evgenii Stratonikov
4718375db1 rpc: return amount of consumed GAS in invoke* 2020-01-22 14:31:57 +03:00
Evgenii Stratonikov
2f865480d3 util: add YAML marshaler to Fixed8 2020-01-22 14:31:57 +03:00
Evgenii Stratonikov
7a6d6f43ce vm: get rid of recursive (*VM).execute calls
Recursive execute() calls can affect gas calculation.
This commit makes execute() be called only for real opcodes
and moves duplicate logic for CALL/JMP into a separate function.
2020-01-22 14:31:57 +03:00
Evgenii Stratonikov
4eae55143f core: restrict GAS available for invocation transactions
There are 10 GAS available for free plus any amount of GAS
attached to a transaction.
2020-01-22 14:31:54 +03:00
Evgenii Stratonikov
d72d978a19 vm: abstract out GetInterop() function 2020-01-22 14:04:40 +03:00
Evgenii Stratonikov
c22d09adad vm: set GAS limit
Make it possible to set maximum amount of GAS which can be spent
during execution.
2020-01-22 14:04:40 +03:00
Evgenii Stratonikov
0662a7e3c2 vm: calculate GAS spent during execution
If getPrice callback is provided, accumulate spent GAS.
2020-01-22 14:04:40 +03:00
Roman Khimov
99dfdc19e7 network: drop now useless addrReq queue from the server
Just broadcast a high-priority message to everyone.
2020-01-22 11:28:59 +03:00
Roman Khimov
ea3b76ded1 network: make NewServer return an error, fix #612
It can return nil in two cases, so we're better return an error and handle
it.
2020-01-22 11:17:51 +03:00
Roman Khimov
f2ffffddb7 network: rework broadcasting functions, tune priorities
This gives more priority to anything related to consensus.
2020-01-22 11:01:13 +03:00
Roman Khimov
34b863d645 network: introduce Server's MkMsg()
That wraps NewMessage() for a configured network.
2020-01-21 17:31:51 +03:00
Roman Khimov
1f672e0da7 network: move SendVersion() to the Peer
Only leave server-specific `getVersionMsg()` in the Server, all the other
logic is peer-related.
2020-01-21 17:26:08 +03:00
Roman Khimov
9befd8de99
Merge pull request #610 from nspcc-dev/ping-improvements
Ping improvements
2020-01-21 13:34:15 +03:00
Roman Khimov
760b39e324 consensus: close chain in tests
Avoid test failures like this one:

=== RUN   TestService_OnPayload
==================
WARNING: DATA RACE
Read at 0x00c00015a843 by goroutine 112:
  testing.(*common).logDepth()
      /usr/local/go/src/testing/testing.go:665 +0xa1
  testing.(*common).Logf()
      /usr/local/go/src/testing/testing.go:658 +0x8f
  testing.(*T).Logf()
      <autogenerated>:1 +0x75
  go.uber.org/zap/zaptest.testingWriter.Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x11f
  go.uber.org/zap/zaptest.(*testingWriter).Write()
      <autogenerated>:1 +0xa9
  go.uber.org/zap/zapcore.(*ioCore).Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c3
  go.uber.org/zap/zapcore.(*CheckedEntry).Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e7
  go.uber.org/zap.(*Logger).Info()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:187 +0x95
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).persist()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:720 +0x6bb
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run.func2()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:228 +0x53

Previous write at 0x00c00015a843 by goroutine 98:
  testing.tRunner.func1()
      /usr/local/go/src/testing/testing.go:900 +0x353
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:913 +0x1bb

Goroutine 112 (running) created at:
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:227 +0x264

Goroutine 98 (finished) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:960 +0x651
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1202 +0xa6
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:909 +0x199
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1200 +0x521
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1117 +0x2ff
  main.main()
      _testmain.go:162 +0x337
==================
--- FAIL: TestService_OnPayload (4.11s)
2020-01-20 19:40:52 +03:00
Roman Khimov
f56383e9c8 network: use p.LastBlockIndex() in requestBlocks()
Always compare to the best known block index, comparing to the StartHeight is
just plain wrong now.
2020-01-20 19:37:17 +03:00
Roman Khimov
2c4ace022e network/config: redesign ping timeout handling a bit
1) Make timeout a timeout, don't do magic ping counts.
2) Drop additional timer from the main peer's protocol loop, create it
   dynamically and make it disconnect the peer.
3) Don't expose the ping counter to the outside, handle more logic inside the
   Peer.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
62092c703d network: use local timestamp to decide when to ping
We don't and we won't have synchronized clocks in the network so the only
timestamp that we can compare our local time with is the one made
ourselves. What this ping mechanism is used for is to recover from missing the
block broadcast, thus it's appropriate for it to trigger after X seconds of
the local time since the last block received.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
a8252ecc05 network: remove wrong ping condition
In reality it will never be true exactly in the case where we want this ping
mechanism to work --- when the node failed to get a block from the net. It
won't get the header either and thus its block height will be equal to header
height. The only moment when this condition is met is when the node does
initial synchronization and this synchronization works just fine without any
pings.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
247cfa4165 network: either request blocks or ping a peer, but not both
It makes to sense to do both actions, pings are made for a different purpose.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
0ba6b2a754 network: introduce peer sending queues
Two queues for high-priority and ordinary messages. Fixes #590. These queues
are deliberately made small to avoid buffer bloat problem, there is gonna be
another queueing layer above them to compensate for that. The queues are
designed to be synchronous in enqueueing, async capabilities are to be added
layer above later.
2020-01-20 17:23:26 +03:00
Roman Khimov
7f0882767c network: remove useless Done() method from the peer
It's internal state of the peer that no one should care about.
2020-01-20 17:23:26 +03:00
Roman Khimov
f39d5d5a10 network: fix unregistration on peer Disconnect
It should always signal to the server, not duplicating this send and not
missing it like it happened in the Server.run().
2020-01-20 17:23:26 +03:00
Roman Khimov
907a236285 network: move per-peer goroutines into the TCPPeer
As they're directly tied to it.
2020-01-20 17:23:26 +03:00
Roman Khimov
32213b1454
Merge pull request #601 from nspcc-dev/refactoring/core
core: refactor out Block, BlockBase and Header, closes #597.
2020-01-20 16:19:20 +03:00
Roman Khimov
f4aa088d04
Merge pull request #603 from nspcc-dev/feature/wallet
consensus: use wallets instead of WIF, closes #588.
2020-01-20 16:16:10 +03:00
Roman Khimov
bb80ba9b9e
Merge pull request #456 from nspcc-dev/pingpong_430
add ping pong processing
2020-01-20 16:10:29 +03:00
Evgenii Stratonikov
f0083b94c5 vm: use new big.Int (de-)serialization routines
Also fix a test with CAT.
2020-01-17 17:37:50 +03:00