TrueCloudLab/neo-go
8
0
Fork 0
mirror of https://github.com/nspcc-dev/neo-go.git synced 2024-11-23 23:30:36 +00:00
Code Issues Projects Releases Packages Wiki Activity
4648 commits 40 branches 136 tags 43 MiB
Commit graph

106 commits

Author SHA1 Message Date
Anna Shaleva
50296975e2 crypto: allow to provide non-default scrypt parameters 2021-06-11 15:22:28 +03:00
Roman Khimov
c4e084b0d8 *: fix whitespace errors 
leading/trailing newlines
 2021-05-12 22:51:41 +03:00
Roman Khimov
4caff35e73 consensus: fix error checking in decoders 
Spotted by errcheck.
 2021-05-12 20:14:52 +03:00
Roman Khimov
84b240cc9a services/consensus: add some logging on start 2021-04-02 13:13:26 +03:00
Roman Khimov
a954821b98 consensus: make Shutdown a no-op if not started 
It makes client code a bit simpler, we can just issue a Shutdown() request
without thinking much about it (but only once of course).
 2021-04-02 12:50:46 +03:00
Roman Khimov
f91ff78918 payload: drop Network from Extensible 
It's only used to sign/verify it and is not a part of the structure. It's
still neded in consensus.Payload though because that's the way dbft library
is.
 2021-03-26 13:45:18 +03:00
Roman Khimov
95c279325a block: drop Network from the Header 
It's not network-tied any more, network is only needed to
sign/verify. Unfortunately we still have to keep network in consensus data
structures because of dbft library interface.
 2021-03-26 13:45:18 +03:00
Anna Shaleva
4809cdf0b0 consensus: fix panic during verifyBlock 
Issue: panic during mixed 4-nodes consensus setup:
```
2021-03-18T12:01:50.715Z	INFO	skip change view	{"nc": 0, "nf": 3}
2021-03-18T12:01:52.786Z	INFO	received ChangeView	{"validator": 0, "reason": "Timeout", "new view": 1}
2021-03-18T12:01:53.602Z	INFO	received ChangeView	{"validator": 2, "reason": "Timeout", "new view": 1}
2021-03-18T12:01:56.736Z	INFO	received ChangeView	{"validator": 1, "reason": "Timeout", "new view": 1}
2021-03-18T12:01:56.736Z	INFO	changing dbft view	{"height": 3, "view": 1, "index": 3, "role": "Backup"}
2021-03-18T12:02:01.758Z	INFO	received PrepareRequest	{"validator": 2, "tx": 0}
panic: interface conversion: block.Block is nil, not *consensus.neoBlock

goroutine 315 [running]:
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).verifyBlock(0xc000419540, 0x0, 0x0, 0x4)
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:427 +0x1306
github.com/nspcc-dev/dbft.(*DBFT).createAndCheckBlock(0xc0001f8840, 0x13f0002)
	github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:373 +0x27e
github.com/nspcc-dev/dbft.(*DBFT).onPrepareRequest(0xc0001f8840, 0x13f4378, 0xc0003b8500)
	github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:329 +0xdf1
github.com/nspcc-dev/dbft.(*DBFT).OnReceive(0xc0001f8840, 0x13f4378, 0xc0003b8500)
	github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:247 +0xe25
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).eventLoop(0xc000419540)
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:297 +0x79d
created by github.com/nspcc-dev/neo-go/pkg/consensus.(*service).Start
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:249 +0xa5
```

So (*service).verifyBlock is unable to work with nil block.
 2021-03-18 18:20:33 +03:00
Anna Shaleva
20b70ee9fe config: add MaxBlockSystemFee setting 2021-03-16 12:09:11 +03:00
Anna Shaleva
8f06bf21d7 config: add MaxBlockSize setting 2021-03-16 12:08:47 +03:00
Anna Shaleva
38103dcc7a consensus: check transactions count during PrepareRequest verification 2021-03-15 16:58:37 +03:00
Anna Shaleva
23a3514cc0 consensus: store ProtocolConfiguration in consensus config 2021-03-15 16:58:27 +03:00
Evgeniy Stratonikov
2f490a3403 block: remove ConsensusData field 2021-03-10 13:38:44 +03:00
Evgeniy Stratonikov
bf20db09e0 stateroot: move state-root related logic to core/stateroot 2021-03-09 13:48:29 +03:00
Evgeniy Stratonikov
ffd85dd51d native/policy: remove MaxBlockSize and MaxBlockSystemFee 2021-03-04 16:59:19 +03:00
Roman Khimov
43bfc909eb consensus: flush previous proposal on new block 
Reusing proposals from previous blocks doesn't make sense. And reduce some
code duplication along the way.
 2021-02-04 18:54:01 +03:00
Roman Khimov
8c0a7225e5 consensus: only use previous proposal if it has something in it 
It might just be uninitialized it doesn't really make sense using zero-length
previous proposal anyway.
 2021-02-04 18:48:09 +03:00
Roman Khimov
f081e63674 consensus: fix extensible message category name 
Fix this in 2+2 setup:
2021-02-04T13:50:23.923Z        WARN    peer disconnected       {"addr": "172.30.0.5:20334", "reason": "handling CMDExtensible message: invalid category", "peerCount": 3}
2021-02-04T13:50:26.968Z        WARN    peer disconnected       {"addr": "172.30.0.2:42586", "reason": "handling CMDExtensible message: invalid category", "peerCount": 2}
 2021-02-04 16:55:46 +03:00
Anna Shaleva
bfbd096fed core: introduce mempool notifications 2021-02-02 22:01:32 +03:00
Evgeniy Stratonikov
9592f3e052 network: implement pool for Extensible payloads 2021-01-28 17:09:06 +03:00
Evgeniy Stratonikov
5d83c28bc9 network: replace ConsensusType with ExtensibleType 2021-01-22 10:38:33 +03:00
Evgeniy Stratonikov
b918ec3abc consensus: refactor payloads structure 
1. `Version` and `PrevHash` are now in `PrepareRequest`.
2. Serialization is done via `Extensible` payload.
3. Update dbft version.
 2021-01-22 10:38:32 +03:00
Evgenii Stratonikov
c5f9f6a3fd consensus: validate timestamp in `verifyBlock() 
Not doing this can possibly lead to the same node being validator
again and again.
 2020-12-16 17:41:44 +03:00
Anna Shaleva
0b5cf78468 network: add notary request payload 2020-12-10 18:17:31 +03:00
Evgenii Stratonikov
1869d6d460 core: allow to use state root in header 2020-11-20 17:16:32 +03:00
Evgenii Stratonikov
c30d891aa9 consensus: update NextConsensus only when committee is recalculated 2020-11-09 17:35:32 +03:00
Evgenii Stratonikov
fc77f8b5b2 consensus: exit if wrong password is provided in configuration 2020-09-21 17:56:25 +03:00
Roman Khimov
15b621f0f3 consensus: wait goroutine to finish on Shutdown 
Fixes:
panic: assignment to entry in nil map

goroutine 227 [running]:
github.com/nspcc-dev/neo-go/pkg/core/storage.(*MemoryStore).put(...)
        /home/rik/dev/neo-go/pkg/core/storage/memory_store.go:53
github.com/nspcc-dev/neo-go/pkg/core/storage.(*MemoryStore).PutBatch(0xc00035f580, 0x110a680, 0xc000336750, 0x0, 0x0)
        /home/rik/dev/neo-go/pkg/core/storage/memory_store.go:93 +0x286
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).addHeaders(0xc0000a2340, 0xc0001be301, 0xc00036d428, 0x1, 0x1, 0x0, 0x0)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:516 +0xd5a
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddBlock(0xc0000a2340, 0xc0000bc2c0, 0x0, 0x0)
        /home/rik/dev/neo-go/pkg/core/blockchain.go:407 +0x9ca
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).processBlock(0xc000152160, 0x1122320, 0xc0000bc2c0)
        /home/rik/dev/neo-go/pkg/consensus/consensus.go:440 +0xbf
github.com/nspcc-dev/dbft.(*DBFT).checkCommit(0xc0000d3400)
        /home/rik/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20200911152629-be965ee4d449/check.go:71 +0x918
github.com/nspcc-dev/dbft.(*DBFT).checkPrepare(0xc0000d3400)
        /home/rik/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20200911152629-be965ee4d449/check.go:36 +0x465
github.com/nspcc-dev/dbft.(*DBFT).sendPrepareRequest(0xc0000d3400)
        /home/rik/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20200911152629-be965ee4d449/send.go:42 +0x2f8
github.com/nspcc-dev/dbft.(*DBFT).start(0xc0000d3400)
        /home/rik/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20200911152629-be965ee4d449/dbft.go:269 +0x26f
github.com/nspcc-dev/dbft.(*DBFT).Start(0xc0000d3400)
        /home/rik/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20200911152629-be965ee4d449/dbft.go:82 +0x59
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).Start(0xc000152160)
        /home/rik/dev/neo-go/pkg/consensus/consensus.go:191 +0x56
github.com/nspcc-dev/neo-go/pkg/network.(*Server).tryStartConsensus(0xc000235040)
        /home/rik/dev/neo-go/pkg/network/server.go:311 +0xda
github.com/nspcc-dev/neo-go/pkg/network.(*Server).Start(0xc000235040, 0xc0000faba0)
        /home/rik/dev/neo-go/pkg/network/server.go:173 +0x202
created by github.com/nspcc-dev/neo-go/cli.newTestChain
        /home/rik/dev/neo-go/cli/executor_test.go:77 +0x47d
FAIL    github.com/nspcc-dev/neo-go/cli 14.479s
 2020-09-19 21:49:05 +03:00
Roman Khimov
1608fbff87
Merge pull request #1378 from nspcc-dev/tests/cli 
Implement tests for CLI
 2020-09-19 17:02:34 +03:00
Evgenii Stratonikov
282b55494b consensus: allow to shutdown service 2020-09-18 12:07:01 +03:00
Roman Khimov
af6a6f5f30 consensus: fix potential system deadlock 
There is a notification pushed into the channel when block is being added,
that notification is read by special goroutine that then forwards it to all
subscribers to that particular event. Consensus goroutine is one of that
subscribers, so for the system to properly function it has to read these
events, but at the same time it can generate new blocks inside, so in some
cases it can generate two blocks without ever reading from the subscription
channel and this will lead to a deadlock.

To avoid that we need to check subscription channel for events on every loop.
 2020-09-18 10:21:47 +03:00
Roman Khimov
d52e79668b hash: introduce memory-optimized merkle root hash calculation routine 
NewMerkleTree is a memory hog, we can do better than that:

BenchmarkMerkle/NewMerkleTree-8                       13          88434670 ns/op        20828207 B/op     300035 allocs/op
BenchmarkMerkle/CalcMerkleRoot-8                      15          69264150 ns/op               0 B/op          0 allocs/op
 2020-09-15 18:38:15 +03:00
Roman Khimov
a6a1df4e0d consensus: update dbft, use new timer 2020-09-11 18:55:07 +03:00
Roman Khimov
9c5ef8d234 dbft: rev up, pick performance improvements 2020-09-09 20:46:31 +03:00
Roman Khimov
562e7e371f consensus: switch to uint8 for validator index 
Follow neo-project/neo#1837. Tests will need to be updated with real messages.
 2020-08-23 16:44:56 +03:00
Evgenii Stratonikov
7e34072519 core: implement (*Blockchain).VerifyWitness 
`ScriptFromWitness` is no longer useful, because we support
contract verification.
 2020-08-22 12:45:20 +03:00
Roman Khimov
d9b8704b48 consensus: check for chain's height in verifyBlock 
We may already be behind and this check could be irrelevant.
 2020-08-20 18:50:36 +03:00
Roman Khimov
55b2cbb74d core: refactor and improve verification and pooling 
Now we have VerifyTx() and PoolTx() APIs that either verify transaction in
isolation or verify it against the mempool (either the primary one or the one
given) and then add it there. There is no possibility to check against the
mempool, but not add a transaction to it, but I doubt we really need it.

It allows to remove some duplication between old PoolTx and verifyTx where
they both tried to check transaction against mempool (verifying first and then
adding it). It also saves us utility token balance check because it's done by
the mempool anyway and we no longer need to do that explicitly in verifyTx.

It makes AddBlock() and verifyBlock() transaction's checks more correct,
because previously they could miss that even though sender S has enough
balance to pay for A, B or C, he can't pay for all of them.

Caveats:
 * consensus is running concurrently to other processes, so things could
   change while verifyBlock() is iterating over transactions, this will be
   mitigated in subsequent commits

Improves TPS value for single node by at least 11%.

Fixes #667, fixes #668.
 2020-08-20 18:50:18 +03:00
Roman Khimov
0bf2fa915e consensus: don't decrypt the key again and again, cache it 
It's cached in dbft for a view anyway, so there is no big difference here
from security POV. Lets us squeeze yet another 4% TPS improvement.

Make the system fail if unable to decrypt the key along the way, which is a
part of #1312.
 2020-08-20 18:50:18 +03:00
Roman Khimov
36ce23789a consensus: fix comment typo 2020-08-18 17:41:22 +03:00
Roman Khimov
80302c5c07 consensus: use dbft.M() for new block witness 
We're collecting dbft.M() number of signatures in getBlockWitness(), so we
should use the same value for M here.
 2020-08-10 18:49:48 +03:00
Anna Shaleva
db65ed04d9 consensus: apply policy during verifyBlock 
To follow C# implementation we should also check proposed block on
policy matching.
 2020-08-04 10:59:09 +03:00
Roman Khimov
a43e374ac6 consensus: replace magic 1000000 with something more meaningful 2020-07-11 19:54:50 +03:00
Roman Khimov
7eef895061 consensus: use GetNextBlockValidators where appropriate 
GetValidators without parameter is called upon DBFT initialization and it
should receive validators for the next block (that will create it),
parameterized GetValidators is used for NextConsensus calculation where we
need a list for the current state of the chain.
 2020-07-11 19:54:50 +03:00
Roman Khimov
2278cd5700 consensus: drop NextConsensus from the prepareRequest 
Follow neo-project/neo#744 to make our requests compatible with C# node.
 2020-07-11 19:54:50 +03:00
Anna Shaleva
995abcf0c3 consensus: prevent invalid message panic 
After 5c2f69bd2c there's an `invalid
message` panic when trying to run 4-nodes private network:

```
=> Try to restore blocks before running node
2020-06-30T08:59:39.696Z	INFO	no storage version found! creating genesis block
2020-06-30T08:59:39.696Z	INFO	service is running	{"service": "Prometheus", "endpoint": ":20001"}
2020-06-30T08:59:39.696Z	INFO	service hasn't started since it's disabled	{"service": "Pprof"}

    _   ____________        __________
   / | / / ____/ __ \      / ____/ __ \
  /  |/ / __/ / / / /_____/ / __/ / / /
 / /|  / /___/ /_/ /_____/ /_/ / /_/ /
/_/ |_/_____/\____/      \____/\____/
2020-06-30T08:59:39.696Z	INFO	starting rpc-server	{"endpoint": ":30333"}
2020-06-30T08:59:39.696Z	INFO	node started	{"blockHeight": 0, "headerHeight": 0}

/NEO-GO:/

2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.1:20333", "peerCount": 1}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.2:46692", "peerCount": 2}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.1:54140", "peerCount": 3}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.4:42890", "peerCount": 4}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.4:20336", "peerCount": 5}
2020-06-30T08:59:39.697Z	INFO	new peer connected	{"addr": "172.200.0.2:20334", "peerCount": 6}
2020-06-30T08:59:39.697Z	INFO	started protocol	{"addr": "172.200.0.4:42890", "userAgent": "/NEO-GO:/", "startHeight": 0, "id": 3797159912}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.4:20336", "reason": "already connected", "peerCount": 5}
2020-06-30T08:59:39.697Z	INFO	started protocol	{"addr": "172.200.0.2:20334", "userAgent": "/NEO-GO:/", "startHeight": 0, "id": 1116452361}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.2:46692", "reason": "already connected", "peerCount": 4}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.1:54140", "reason": "identical node id", "peerCount": 3}
2020-06-30T08:59:39.697Z	WARN	peer disconnected	{"addr": "172.200.0.1:20333", "reason": "identical node id", "peerCount": 2}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:44948", "peerCount": 3}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:20335", "peerCount": 4}
2020-06-30T08:59:40.673Z	WARN	peer disconnected	{"addr": "172.200.0.3:20335", "reason": "already connected", "peerCount": 3}
2020-06-30T08:59:40.673Z	WARN	peer disconnected	{"addr": "172.200.0.3:44948", "reason": "unexpected empty payload", "peerCount": 2}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:44950", "peerCount": 3}
2020-06-30T08:59:40.673Z	INFO	new peer connected	{"addr": "172.200.0.3:20335", "peerCount": 4}
2020-06-30T08:59:40.677Z	INFO	node reached synchronized state, starting consensus
2020-06-30T08:59:40.677Z	INFO	started protocol	{"addr": "172.200.0.3:44950", "userAgent": "/NEO-GO:/", "startHeight": 0, "id": 2658713129}
2020-06-30T08:59:40.689Z	WARN	peer disconnected	{"addr": "172.200.0.3:20335", "reason": "already connected", "peerCount": 3}
2020-06-30T08:59:40.697Z	INFO	blockchain persist completed	{"persistedBlocks": 0, "persistedKeys": 23, "headerHeight": 0, "blockHeight": 0, "took": "243.634µs"}
2020-06-30T08:59:41.057Z	INFO	initializing dbft	{"height": 1, "view": 0, "index": 0, "role": "Backup"}
2020-06-30T08:59:41.057Z	DPANIC	invalid message
2020-06-30T08:59:41.058Z	DPANIC	invalid message
```

Reason: payload wasn't decoded as payload.message isn't null anymore.
Fixed.
 2020-06-30 12:06:50 +03:00
Roman Khimov
372dd71708
Merge pull request #1108 from nspcc-dev/neo3/compiler/nef 
compiler: support *.nef and *.manifest.json generation
 2020-06-29 20:55:47 +03:00
Evgenii Stratonikov
5c2f69bd2c consensus: take payload hash based on data 
Fix 857ced82.
 2020-06-29 15:52:06 +03:00
Roman Khimov
8ad355b444
Merge pull request #1124 from nspcc-dev/forward-port-from-2.x 
Forward port from 2.x
 2020-06-29 11:52:45 +03:00
Evgenii Stratonikov
f67b8ce607 consensus: check if payload is present in cache before validation 
Don't check signature correctness twice.
 2020-06-29 10:37:26 +03:00