Commit graph

2204 commits

Author SHA1 Message Date
Roman Khimov
25b742d24e cache: prevent TestRelayCache_Add failures
random.Bytes() might be not random enough.
2020-06-26 12:42:12 +03:00
Roman Khimov
cefad683e3 consensus: prevent synchronization stalls
When CN is not up to date with the network is synchonizes blocks first and
only then starts consensus process. But while synchronizing it receives
consensus payloads and tries to process them even though messages reader
routine is not started yet. This leads to lots of goroutines waiting to send
their messages:

Jun 25 23:55:53 nodoka neo-go[32733]: goroutine 1639919 [chan send, 4 minutes]:
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/consensus.(*service).OnPayload(0xc0000ecb40, 0xc005bd7680)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:329 +0x31b
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleConsensusCmd(...)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:687
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ba160, 0x1053260, 0xc00507d170, 0xc005bdd560, 0x0, 0x0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:806 +0xd58
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc00507d170)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:160 +0x294
Jun 25 23:55:53 nodoka neo-go[32733]: created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:38 +0x1ad
Jun 25 23:55:53 nodoka neo-go[32733]: goroutine 1639181 [chan send, 10 minutes]:
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/consensus.(*service).OnPayload(0xc0000ecb40, 0xc013bb6600)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:329 +0x31b
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleConsensusCmd(...)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:687
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ba160, 0x1053260, 0xc01361ee10, 0xc01342c780, 0x0, 0x0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:806 +0xd58
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc01361ee10)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:160 +0x294
Jun 25 23:55:53 nodoka neo-go[32733]: created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:38 +0x1ad
Jun 25 23:55:53 nodoka neo-go[32733]: goroutine 39454 [chan send, 32 minutes]:
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/consensus.(*service).OnPayload(0xc0000ecb40, 0xc014fea680)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:329 +0x31b
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleConsensusCmd(...)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:687
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage(0xc0000ba160, 0x1053260, 0xc0140b2ea0, 0xc014fe0ed0, 0x0, 0x0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:806 +0xd58
Jun 25 23:55:53 nodoka neo-go[32733]: github.com/nspcc-dev/neo-go/pkg/network.(*TCPPeer).handleConn(0xc0140b2ea0)
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_peer.go:160 +0x294
Jun 25 23:55:53 nodoka neo-go[32733]: created by github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Dial
Jun 25 23:55:53 nodoka neo-go[32733]: #011/go/src/github.com/nspcc-dev/neo-go/pkg/network/tcp_transport.go:38 +0x1ad

Luckily it doesn't break synchronization completely as eventually connection
timers fire, the node breaks all connections, create new ones and these new
ones request blocks successfully until another consensus payload stalls them
too. In the end the node reaches synchronization, message processing loop
starts and releases all of these waiting goroutines, but it's better for us to
avoid this happening at all.

This also makes double-starting a no-op which is a nice property.
2020-06-26 12:32:12 +03:00
Roman Khimov
c6894bfb3f vm/cli: fix load commands help 2020-06-26 00:13:58 +03:00
Roman Khimov
bc3f17a890 CHANGELOG: release 0.76.0 2020-06-25 22:05:27 +03:00
Roman Khimov
7e5533e77e
Merge pull request #1114 from nspcc-dev/fix-test-failures-2.x
Fix test failures 2.x
2020-06-25 22:01:16 +03:00
Roman Khimov
8f1ddc0651 rpc/server: fix error reporting in Start
This error message makes no sense when shutting down the server:
2020-06-25T19:29:53.251+0300    ERROR   failed to start RPC server      {"error": "http: Server closed"}

And ListenAndServer is documented to always return non-nil error one of which
is http.ErrServerClosed. This should also fix the following test failure:

==================
WARNING: DATA RACE
Read at 0x00c000254243 by goroutine 49:
  testing.(*common).logDepth()
      /usr/local/go/src/testing/testing.go:665 +0xa1
  testing.(*common).Logf()
      /usr/local/go/src/testing/testing.go:658 +0x8f
  testing.(*T).Logf()
      <autogenerated>:1 +0x75
  go.uber.org/zap/zaptest.testingWriter.Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x11f
  go.uber.org/zap/zaptest.(*testingWriter).Write()
      <autogenerated>:1 +0xa9
  go.uber.org/zap/zapcore.(*ioCore).Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c3
  go.uber.org/zap/zapcore.(*CheckedEntry).Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e7
  go.uber.org/zap.(*Logger).Error()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:203 +0x95
  github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).Start()
      /go/src/github.com/nspcc-dev/neo-go/pkg/rpc/server/server.go:179 +0x5c5

Previous write at 0x00c000254243 by goroutine 44:
  testing.tRunner.func1()
      /usr/local/go/src/testing/testing.go:900 +0x353
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:913 +0x1bb

Goroutine 49 (running) created at:
  github.com/nspcc-dev/neo-go/pkg/rpc/server.initClearServerWithInMemoryChain()
      /go/src/github.com/nspcc-dev/neo-go/pkg/rpc/server/server_helper_test.go:69 +0x305
  github.com/nspcc-dev/neo-go/pkg/rpc/server.initServerWithInMemoryChain()
      /go/src/github.com/nspcc-dev/neo-go/pkg/rpc/server/server_helper_test.go:78 +0x3c
  github.com/nspcc-dev/neo-go/pkg/rpc/server.testRPCProtocol()
      /go/src/github.com/nspcc-dev/neo-go/pkg/rpc/server/server_test.go:805 +0x53
  github.com/nspcc-dev/neo-go/pkg/rpc/server.TestRPC.func1()
      /go/src/github.com/nspcc-dev/neo-go/pkg/rpc/server/server_test.go:793 +0x44
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:909 +0x199

Goroutine 44 (finished) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:960 +0x651
  github.com/nspcc-dev/neo-go/pkg/rpc/server.TestRPC()
      /go/src/github.com/nspcc-dev/neo-go/pkg/rpc/server/server_test.go:792 +0x5d
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:909 +0x199
==================
2020-06-25 19:41:29 +03:00
Roman Khimov
c489f975d4 core: fix TestSubscriptions occasional failures
panic: Log in goroutine after TestSubscriptions has completed

goroutine 1079 [running]:
testing.(*common).logDepth(0xc00057a100, 0xc00039e210, 0xa4, 0x3)
	/usr/local/go/src/testing/testing.go:634 +0x51a
testing.(*common).log(...)
	/usr/local/go/src/testing/testing.go:614
testing.(*common).Logf(0xc00057a100, 0xe32eaa, 0x2, 0xc0009560e0, 0x1, 0x1)
	/usr/local/go/src/testing/testing.go:649 +0x91
go.uber.org/zap/zaptest.testingWriter.Write(0xf64120, 0xc00057a100, 0x0, 0xc0003fe400, 0xa5, 0x400, 0xc000958e40, 0xc0009560d0, 0xc000958e60)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x120
go.uber.org/zap/zapcore.(*ioCore).Write(0xc0005cd050, 0x0, 0xbfb54ffc0626aba2, 0x916de700, 0x1485500, 0x0, 0x0, 0xe43fb0, 0x1c, 0x0, ...)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c5
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000102d10, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e8
go.uber.org/zap.(*Logger).Info(0xc00035eba0, 0xe43fb0, 0x1c, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:187 +0x96
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).persist(0xc00000cb40, 0xc00017c2c0, 0xbe8a00)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:839 +0x6c9
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run.func2(0xc00000cb40, 0xc0005c6c30)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:302 +0x54
created by github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:301 +0x25d
FAIL	github.com/nspcc-dev/neo-go/pkg/core	2.463s
2020-06-25 19:41:17 +03:00
Roman Khimov
a7e8f07073
Merge pull request #1109 from nspcc-dev/neox-doc
docs: add neox documentation
2020-06-25 17:36:02 +03:00
Roman Khimov
d12258e79c
Merge pull request #1106 from nspcc-dev/fix/rpc
Add missing neox rpc client wrappers
2020-06-25 17:11:02 +03:00
Evgenii Stratonikov
eb37f92881 rpc/client: support verifyproof RPC 2020-06-25 17:07:23 +03:00
Roman Khimov
e97c153a26 docs: add neox documentation 2020-06-25 17:05:10 +03:00
Evgenii Stratonikov
654877fb1b rpc/client: support getproof RPC 2020-06-25 17:04:23 +03:00
Roman Khimov
b517aa43db
Merge pull request #1105 from nspcc-dev/fix-key-recovery-doc
interop: fix key recovery functions documentation
2020-06-25 16:34:28 +03:00
Evgenii Stratonikov
d8dddabc86 rpc/client: support getstateheight RPC 2020-06-25 16:19:47 +03:00
Evgenii Stratonikov
b4b0ae0f51 rpc/client: support getstateroot RPC 2020-06-25 16:19:47 +03:00
Evgenii Stratonikov
45a95b5242 rpc/server: remove debug fmt.Println() from tests 2020-06-25 16:19:47 +03:00
Roman Khimov
d8556b80bc interop: fix key recovery functions documentation
Follow 7d786fac79 update.
2020-06-25 14:30:53 +03:00
Roman Khimov
3ccf19fc1e
Merge pull request #1097 from nspcc-dev/fix/mpt
core,dao: use MPT only if it is enabled in config
2020-06-24 16:18:59 +03:00
Evgenii Stratonikov
2b21102c14 core,dao: use MPT only if it is enabled in config 2020-06-24 16:13:27 +03:00
Roman Khimov
fbadb317f5
Merge pull request #1096 from nspcc-dev/neox-2.x
Neox 2.x
2020-06-24 16:08:19 +03:00
Roman Khimov
25fdc62203 core: add state height to prometheus metrics 2020-06-24 14:47:08 +03:00
Roman Khimov
ea17793aee core: fix unconditional MPT collapsing
It should depend on EnableStateRoot.
2020-06-24 14:41:57 +03:00
Roman Khimov
6d5e70163d Revert "Makefile: add neox suffix to neox-2.x branch"
This reverts commit 806b28aab7, we're merging to
master-2.x now.
2020-06-24 14:37:16 +03:00
Roman Khimov
03977083c0
Merge pull request #1095 from nspcc-dev/update-dbft-neox-2.x
consensus: update dbft library (neox-2.x)
2020-06-24 14:32:06 +03:00
Roman Khimov
cce4380eee
Merge pull request #1090 from nspcc-dev/feature/protocol
Enable state root exchange base on config
2020-06-24 13:47:47 +03:00
Roman Khimov
86110445a6 consensus: update dbft library
Pick up the following changes:
 * recovery message sending fix
 * tx re-requests on sendRecoveryRequest
 * proposed block checks fix
 * timeout calculation fix for InitializeConsensus
 * removal of TxPerBlock which is irrelevant for neo-go (policying is done by
   the node, not dbft library)
2020-06-24 13:39:12 +03:00
Evgenii Stratonikov
9ee9cb8e39 core: provide key recovery interops only if neox is enabled 2020-06-24 12:37:27 +03:00
Evgenii Stratonikov
b5fb63d091 *: specify first state root index in config 2020-06-24 12:37:27 +03:00
Evgenii Stratonikov
d128b55dbf *: add config flag for enabling state root feature 2020-06-24 12:37:27 +03:00
Evgenii Stratonikov
c06b3b669d io: make maxArraySize public 2020-06-24 12:14:33 +03:00
Evgenii Stratonikov
bc81b56708 consensus: fix decodeData doc-comment 2020-06-24 10:47:44 +03:00
Roman Khimov
2cddeadd9a
Merge pull request #1089 from nspcc-dev/fix/docker-startup
docker: fix shell condition check in entrypoint script
2020-06-23 16:42:39 +03:00
alexvanin
423495ecc3 docker: fix shell condition check in entrypoint script 2020-06-23 12:08:49 +03:00
Evgenii Stratonikov
2863343f03 consensus: use a method function for NewPayload() 2020-06-22 17:06:28 +03:00
Roman Khimov
3a6186af54
Merge pull request #1083 from nspcc-dev/fix/stateroot
Request verified state roots
2020-06-22 16:42:41 +03:00
Roman Khimov
fec8916cb9
Merge pull request #1087 from nspcc-dev/fix/readvarbytes
Allow to restrict slice size in `io.ReadVarBytes()`
2020-06-22 15:42:07 +03:00
Evgenii Stratonikov
ad0cf146e8 consensus: restrict invocation script size 2020-06-22 15:36:12 +03:00
Evgenii Stratonikov
511d18d409 io: allow to restrict slice size in ReadVarBytes 2020-06-22 15:35:42 +03:00
Roman Khimov
9767817ee4
Merge pull request #1086 from nspcc-dev/fix/recovery
consensus: fix commitCompact payload
2020-06-22 14:52:15 +03:00
Evgenii Stratonikov
aebed3826b consensus: fix commitCompact payload
Add state root signature to `commitCompact` in `recoveryMessage`.
2020-06-22 14:30:21 +03:00
Evgenii Stratonikov
caf53740d3 network: do not process state roots below verified height
Implement fast path to skip state root retrieval and decoding.
2020-06-22 12:33:35 +03:00
Evgenii Stratonikov
e2e1bd09ae network: request state roots if needed 2020-06-22 12:33:35 +03:00
Evgenii Stratonikov
f8051da0bd core: extend Blockchainer interface with StateHeight()
Allow to query current verified state root height.
2020-06-22 10:53:13 +03:00
Evgenii Stratonikov
f665843887 core: update verified state root height 2020-06-22 10:53:12 +03:00
Roman Khimov
9615e83c00
Merge pull request #952 from nspcc-dev/fix/structfield
compiler: set default values for complex struct fields (2.x)
2020-06-11 17:50:18 +03:00
Evgenii Stratonikov
cc5b5bff2e compiler: emit NOTEQUAL only for numbers
Dispatch based on types similarly to EQUAL.
2020-06-11 17:41:52 +03:00
Evgenii Stratonikov
9d6b0ee4a8 compiler: disallow comparing slices with nil
NEO VM does not distinguish between empty and nil slices. Supporting
this is not easy and requires changing lots of other opcodes.
Pointers are not supported anyway and slices can be checked for
emptiness by inspecting their `len`.
2020-06-11 17:40:02 +03:00
Evgenii Stratonikov
bf6aa02dcf compiler: set default values for complex struct fields
Set default value also for complex (struct, map) types.
Note: because VM does not distinguish empty array and nil array,
comparison 'a == nil' can't be handled properly without substantial
effort. This will be fixed in NEO3.
2020-06-11 17:40:02 +03:00
Roman Khimov
d5ba2e49a2
Merge pull request #1045 from nspcc-dev/fix/stateroot
consensus: verify state root in PrepareRequest
2020-06-11 11:25:02 +03:00
Evgenii Stratonikov
8614867439 consensus: verify state root in PrepareRequest
Fixes #1042.
2020-06-11 10:18:12 +03:00