Commit graph

111 commits

Author SHA1 Message Date
Anna Shaleva
995abcf0c3 consensus: prevent invalid message panic
After 5c2f69bd2c there's an `invalid
message` panic when trying to run 4-nodes private network:

```
=> Try to restore blocks before running node
2020-06-30T08:59:39.696Z	INFO	no storage version found! creating genesis block
2020-06-30T08:59:39.696Z	INFO	service is running	{"service": "Prometheus", "endpoint": ":20001"}
2020-06-30T08:59:39.696Z	INFO	service hasn't started since it's disabled	{"service": "Pprof"}

    _   ____________        __________
   / | / / ____/ __ \      / ____/ __ \
  /  |/ / __/ / / / /_____/ / __/ / / /
 / /|  / /___/ /_/ /_____/ /_/ / /_/ /
/_/ |_/_____/\____/      \____/\____/
2020-06-30T08:59:39.696Z	INFO	starting rpc-server	{"endpoint": ":30333"}
2020-06-30T08:59:39.696Z	INFO	node started	{"blockHeight": 0, "headerHeight": 0}

/NEO-GO:/

2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.1:20333", "peerCount": 1}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.2:46692", "peerCount": 2}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.1:54140", "peerCount": 3}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.4:42890", "peerCount": 4}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.4:20336", "peerCount": 5}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.2:20334", "peerCount": 6}
2020-06-30T08:59:39.697Z	INFO	started protocol	{"addr": "172.200.0.4:42890", "userAgent": "/NEO-GO:/", "startHeight": 0, "id": 3797159912}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.4:20336", "reason": "already connected", "peerCount": 5}
2020-06-30T08:59:39.697Z	INFO	started protocol	{"addr": "172.200.0.2:20334", "userAgent": "/NEO-GO:/", "startHeight": 0, "id": 1116452361}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.2:46692", "reason": "already connected", "peerCount": 4}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.1:54140", "reason": "identical node id", "peerCount": 3}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.1:20333", "reason": "identical node id", "peerCount": 2}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:44948", "peerCount": 3}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:20335", "peerCount": 4}
2020-06-30T08:59:40.673Z	WARN	peer disconnected	{"addr": "172.200.0.3:20335", "reason": "already connected", "peerCount": 3}
2020-06-30T08:59:40.673Z	WARN	peer disconnected	{"addr": "172.200.0.3:44948", "reason": "unexpected empty payload", "peerCount": 2}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:44950", "peerCount": 3}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:20335", "peerCount": 4}
2020-06-30T08:59:40.677Z	INFO	node reached synchronized state, starting consensus
2020-06-30T08:59:40.677Z	INFO	started protocol	{"addr": "172.200.0.3:44950", "userAgent": "/NEO-GO:/", "startHeight": 0, "id": 2658713129}
2020-06-30T08:59:40.689Z	WARN	peer disconnected	{"addr": "172.200.0.3:20335", "reason": "already connected", "peerCount": 3}
2020-06-30T08:59:40.697Z	INFO	blockchain persist completed	{"persistedBlocks": 0, "persistedKeys": 23, "headerHeight": 0, "blockHeight": 0, "took": "243.634µs"}
2020-06-30T08:59:41.057Z	INFO	initializing dbft	{"height": 1, "view": 0, "index": 0, "role": "Backup"}
2020-06-30T08:59:41.057Z	DPANIC	invalid message
2020-06-30T08:59:41.058Z	DPANIC	invalid message
```

Reason: payload wasn't decoded as payload.message isn't null anymore.
Fixed.
2020-06-30 12:06:50 +03:00
Roman Khimov
372dd71708
Merge pull request #1108 from nspcc-dev/neo3/compiler/nef
compiler: support *.nef and *.manifest.json generation
2020-06-29 20:55:47 +03:00
Evgenii Stratonikov
5c2f69bd2c consensus: take payload hash based on data
Fix 857ced82.
2020-06-29 15:52:06 +03:00
Roman Khimov
8ad355b444
Merge pull request #1124 from nspcc-dev/forward-port-from-2.x
Forward port from 2.x
2020-06-29 11:52:45 +03:00
Evgenii Stratonikov
f67b8ce607 consensus: check if payload is present in cache before validation
Don't check signature correctness twice.
2020-06-29 10:37:26 +03:00
Anna Shaleva
6b8957243a *: move wallet config from wallet to config package
In order to avoid dependency cycle at the next commits:

	imports github.com/nspcc-dev/neo-go/pkg/config
	imports github.com/nspcc-dev/neo-go/pkg/wallet
	imports github.com/nspcc-dev/neo-go/pkg/vm
	imports github.com/nspcc-dev/neo-go/pkg/smartcontract/nef
	imports github.com/nspcc-dev/neo-go/pkg/config
2020-06-29 09:15:29 +03:00
Roman Khimov
a46c71f2de consensus: prevent synchronization stalls
When CN is not up to date with the network is synchonizes blocks first and
only then starts consensus process. But while synchronizing it receives
consensus payloads and tries to process them even though messages reader
routine is not started yet. This leads to lots of goroutines waiting to send
their messages:

Jun 25 23:55:53 nodoka neo-go[32733]: goroutine 1639919 [chan send, 4 minutes]:
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/consensus.(*service).OnPayload(0xc0000ecb40, 0xc005bd7680)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:329 +0x31b
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleConsensusCmd(...)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:687
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ba160, 0x1053260, 0xc00507d170, 0xc005bdd560, 0x0, 0x0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:806 +0xd58
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc00507d170)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:160 +0x294
Jun 25 23:55:53 nodoka neo-go[32733]: created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:38 +0x1ad
Jun 25 23:55:53 nodoka neo-go[32733]: goroutine 1639181 [chan send, 10 minutes]:
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/consensus.(*service).OnPayload(0xc0000ecb40, 0xc013bb6600)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:329 +0x31b
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleConsensusCmd(...)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:687
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ba160, 0x1053260, 0xc01361ee10, 0xc01342c780, 0x0, 0x0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:806 +0xd58
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc01361ee10)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:160 +0x294
Jun 25 23:55:53 nodoka neo-go[32733]: created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:38 +0x1ad
Jun 25 23:55:53 nodoka neo-go[32733]: goroutine 39454 [chan send, 32 minutes]:
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/consensus.(*service).OnPayload(0xc0000ecb40, 0xc014fea680)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:329 +0x31b
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleConsensusCmd(...)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:687
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ba160, 0x1053260, 0xc0140b2ea0, 0xc014fe0ed0, 0x0, 0x0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:806 +0xd58
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc0140b2ea0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:160 +0x294
Jun 25 23:55:53 nodoka neo-go[32733]: created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:38 +0x1ad

Luckily it doesn't break synchronization completely as eventually connection
timers fire, the node breaks all connections, create new ones and these new
ones request blocks successfully until another consensus payload stalls them
too. In the end the node reaches synchronization, message processing loop
starts and releases all of these waiting goroutines, but it's better for us to
avoid this happening at all.

This also makes double-starting a no-op which is a nice property.
2020-06-27 12:29:41 +03:00
Evgenii Stratonikov
857ced8230 consensus: add network magic to hash data
Consensus payload's hash now depends on the network node is operating
in.
2020-06-26 18:41:04 +03:00
Evgenii Stratonikov
8407031605 consensus: build MerkleRoot from hashes on block creation
Use transaction hashes for merkle root calculation as transactions
itself can not yet be available.
2020-06-25 17:21:26 +03:00
Anna Shaleva
8de0332107 *: use MaxTransactionsPerBlock from Policy native contract
part of #904

1. We now have MaxTransactionsPerBlock set in native Policy contract,
   so this value should be used in (dbft).GetVerified method instead
   of passing it as an argument.
2. Removed (dbft).WithTxPerBlock.
2. DBFT API has changed, so update it's version.
3. Removed MaxTransactionsPerBlock from node configuration, as we
   have it set in native Policy contract.
2020-06-24 07:58:05 +03:00
Roman Khimov
b483c38593 block/transaction: add network magic into the hash
We make it explicit in the appropriate Block/Transaction structures, not via a
singleton as C# node does. I think this approach has a bit more potential and
allows better packages reuse for different purposes.
2020-06-18 12:39:50 +03:00
Roman Khimov
b6eef9d275 consensus: don't duplicate RebuildMerkleRoot in newBlockFromContext 2020-06-18 12:11:26 +03:00
Roman Khimov
63f4f34659 mempool: drop TxWithFee type
All the fees are in transaction, this makes no sense.
2020-06-05 19:21:40 +03:00
Roman Khimov
b7d2b659b4 network: get blocks directly from the chain for rebroadcasting
Simplify network<->consensus relations, also broadcast blocks received by
other means like RPC.
2020-05-25 00:27:39 +03:00
Roman Khimov
462022bbdd consensus: remove OnNewBlock(), use Blockchain subscription
Get new blocks directly from the Blockchain. It may lead to some duplications
(as we'll also receive our own blocks), but at the same time it's more
correct, because technically we can also get blocks via other means besides
network server like RPC (submitblock call). And it simplifies network server
at the same time.
2020-05-25 00:27:39 +03:00
Anna Shaleva
45b8bdb51c core: fix bug in consensus
We should set MerkleRoot in approving block irrespective to
transactions presented in the block.
2020-05-20 23:16:04 +03:00
Anna Shaleva
6db4ca874d consensus: replace NewBlock() with NewBlockFromContext(ctx *Context)
We have to set ConsensusData.PrimaryIndex field of Block, so this value
can be retrieved from consensus context.
2020-04-27 17:58:25 +03:00
Anna Shaleva
29d321b5e1 *: drop miner transaction
1. Completely remove miner transaction

2. Change validation rule for block: block without transactions is
valid.
2020-04-27 17:57:37 +03:00
Roman Khimov
b83e84ca08 core: switch to the new voting system (via native NEO contract)
It has all the methods required now, so you can register, vote and get
voting results. Fixes #865.
2020-04-27 16:06:50 +03:00
Evgenii Stratonikov
008e6eb233 vm: implement new PUSH opcodes 2020-04-23 10:52:28 +03:00
Evgenii Stratonikov
3657f2e21d consensus: set Nonce and ValidUntilBlock on proposal
Related #841.
2020-04-20 11:55:24 +03:00
Anna Shaleva
9dd5ab5e2b consensus: added partial message decoding
closes #849
2020-04-16 14:35:58 +03:00
Roman Khimov
1e3c36433f
Merge pull request #844 from nspcc-dev/neo3/nonce
core: add nonce field to transaction
2020-04-14 16:24:02 +03:00
Anna Shaleva
65503aa9b4 core: add nonce field to transaction
1. Closes #840: added Nonce field to transaction.Transaction and
removed Nonce field from transaction.MinerTx

2. Added following methods to different tx types:
  - NewMinerTx()
  - NewMinerTxWithNonce(...)
  - NewEnrollmentTx(...)
  - NewIssueTx()
  - NewPublishTx(...)
  - NewRegisterTx(...)
  - NewStateTx(...)
in order to avoid code duplication when new transaction is created.

3. Commented out test cases where binary transaction/block are used.
These test cases marked with `TODO NEO3.0: Update binary` and need to be
updated.

4. Updated other tests

5. Added constant Nonce to GoveringTockenTx, UtilityTokenTx and genesis
block to avoid data variability. Also marked with TODO.
2020-04-14 16:19:41 +03:00
Evgenii Stratonikov
a7c19d445b core: move Blockchainer interface to a separate package 2020-04-11 10:56:36 +03:00
Anna Shaleva
5a984fdf88 config: move config.go out of config/
closes #423
2020-03-31 17:55:59 +03:00
Roman Khimov
f5a1b928ce pkg: fix gofmt issues 2020-03-03 17:22:15 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Evgenii Stratonikov
21ef2638c0 consensus: log error if GetValidators() failed 2020-03-02 18:01:49 +03:00
Roman Khimov
a903147b60 keys: rename Signature to GetScriptHash, make it return Uint160
Signature itself wasn't used at all and its name is very misleading, Uint160
script hash is way more useful.
2020-02-19 12:19:03 +03:00
Roman Khimov
eb11e5fb11 core: implement basic policying support, fix #370
Implement mempool and consensus block creation policies, almost the same as
SimplePolicy plugin for C# node provides with two caveats:
 * HighPriorityTxType is not configured and hardcoded to ClaimType
 * BlockedAccounts are not supported

Other than that it allows us to run successfuly as testnet CN, previously our
proposals were rejected because we were proposing blocks with oversized
transactions (that are rejected by PoolTx() now).

Mainnet and testnet configuration files are updated accordingly, but privnet
is left as is with no limits.

Configuration is currently attached to the Blockchain and so is the code that
does policying, it may be moved somewhere in the future, but it works for
now.
2020-02-19 12:19:02 +03:00
Roman Khimov
22f5667530 mempool: return fee along with tx when requesting tx
Users of GetVerifiedTransactions() don't want to recalculate tx fee and it's
nice to have it returned from TryGetValue() also sometimes.
2020-02-19 12:19:02 +03:00
Roman Khimov
37c48b00b4 consensus/network: reinit dbft after block addition
Don't stall on some height if everyone else have moved up already. Fix #673.
2020-02-19 12:19:02 +03:00
Roman Khimov
a2616cfafe consensus: correctly generate miner TX outputs based on net fee 2020-02-19 12:13:27 +03:00
Roman Khimov
d3badc3f97 consensus: log more like neo-cli, update dbft 2020-02-11 18:02:13 +03:00
Roman Khimov
b9b77ac1be network: fix block relaying, don't spit out useless errors
We can only add one block of the given height and we have two competing
goroutines to do that --- consensus and block queue. Whomever adds the block
first shouldn't trigger an error in another one.

Fix block relaying for blocks added via the block queue also, previously one
consensus-generated blocks were broadcasted.
2020-02-06 15:41:56 +03:00
Roman Khimov
9ceefe17e2 consensus: check current chain height before processing timer
The chain may already be more current than our dBFT state (like when the node
has commited something at view 0, but all the other nodes changed view and
accepted something at view 1), so in this case we should reinit dBFT on new
height.
2020-02-04 13:16:09 +03:00
Evgenii Stratonikov
e65b5c9914 consensus: fill preparation hash in recovery message 2020-02-04 13:16:05 +03:00
Evgenii Stratonikov
e441cb1414 consensus: add preparation hash to logger 2020-02-04 13:16:00 +03:00
Evgenii Stratonikov
42dfb52e98 consensus: add more logging in consensus service 2020-02-04 13:15:54 +03:00
Evgenii Stratonikov
390bf58f97 consensus: use our implementations of Recovery* messages
While decoding payload, local implementations of Recovery*
messages were used, but when creating RecoveryMessage inside dBFT
library default NewRecoveryMessage was invoked. This lead to parsing
errors.
2020-01-29 17:56:26 +03:00
Roman Khimov
32213b1454
Merge pull request #601 from nspcc-dev/refactoring/core
core: refactor out Block, BlockBase and Header, closes #597.
2020-01-20 16:19:20 +03:00
Evgenii Stratonikov
55d98ab19a consensus: update keys from wallet after every block 2020-01-17 17:30:45 +03:00
Evgenii Stratonikov
aaefcdecae consensus: use wallet in service 2020-01-17 17:25:51 +03:00
Evgenii Stratonikov
63c56cca5c core: refactor out Block, BlockBase and Header structs
See #597.
2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
3d704a3a3a consensus: try to use previous proposal after ChangeView
When system and network pressure is high it can be beneficial
to use transactions which and were already proposed.
The assumption is that they will be in other node's memory pool
with more probability.
2020-01-14 16:05:27 +03:00
Roman Khimov
565c1940e3
Merge pull request #587 from nspcc-dev/feature/zap
Closes #284.
2020-01-10 14:50:09 +03:00
Evgenii Stratonikov
dc65684708 consensus: replace SugaredLogger with Logger
SugaredLogger was used to make output look more similar to logrus.
This is no longer needed.
2020-01-10 11:14:29 +03:00
Evgenii Stratonikov
aecdf470e7 cli,pkg: use zap.Logger 2020-01-10 11:14:27 +03:00
Roman Khimov
b5b05a969c keys: make NEP2Decrypt return a PrivateKey rather than WIF
There is no point in encoding the output of this function in a WIF format,
most of the users actually want the real key and those who need a WIF can
easily get if from the key (and it's simpler than getting the key from the
WIF).

It also fixes a severe bug in NEP2Decrypt, base58 decoding errors were not
processed correctly.
2020-01-09 18:05:14 +03:00
Evgenii Stratonikov
d803dffca8 consensus: return nil interface from getTx 2019-12-27 13:54:11 +03:00
Evgenii Stratonikov
64d24d8ddd consensus: verify payloads correctly 2019-12-26 10:49:56 +03:00
Roman Khimov
7e371588a7 core/tx: remove one layer of indirection for scripts and inouts
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
2019-12-09 17:14:10 +03:00
Evgenii Stratonikov
138c94eda3 consensus: sign and verify consensus messages 2019-12-06 11:35:06 +03:00
Roman Khimov
8d4dd2d2e1 vm: move opcodes into their own package
This allows easier reuse of opcodes and in some cases allows to eliminate
dependencies on the whole vm package, like in compiler that only needs opcodes
and doesn't care about VM for any other purpose.

And yes, they're opcodes because an instruction is a whole thing with
operands, that's what context.Next() returns.
2019-12-03 18:22:14 +03:00
Roman Khimov
65332f5e7f
Merge pull request #529 from nspcc-dev/peer-communication-fixes
A set of fixes to make neo-go privnet more usable.
2019-11-29 16:29:28 +03:00
Evgenii Stratonikov
60f1648778 consensus: fix a bug with index out of range 2019-11-29 16:05:00 +03:00
Evgenii Stratonikov
189a708988 consensus: fix a bug with nil tx channel 2019-11-29 15:40:11 +03:00
Roman Khimov
293615ea5f network/consensus: add new block relaying
Tell everyone about our new shiny blocks.
2019-11-29 12:27:15 +03:00
Evgenii Stratonikov
fdd5276d3e network: plug in dBFT library 2019-11-27 10:57:22 +03:00
Evgenii Stratonikov
085ca7b770 network: implement Consensus payloads 2019-11-13 17:27:25 +03:00