Commit graph

364 commits

Author SHA1 Message Date
Roman Khimov
bd98940a54
Merge pull request #982 from nspcc-dev/neo3/protocol/getblockdata
protocol: implement getblockdata p2p command
2020-05-22 19:28:42 +03:00
Anna Shaleva
8142caaf8b protocol: implement getblockdata p2p command
closes #891
2020-05-22 19:16:49 +03:00
Anna Shaleva
2f6a3e9af5 protocol: refactor getblocks payload
closes #890
2020-05-22 17:33:59 +03:00
Anna Shaleva
64a2fb63e1 protocol: move magic exchange to version payload
closes #889
2020-05-21 14:23:41 +03:00
Anna Shaleva
3bcc56bdcf protocol: switch to binary MessageCommand
closes #888
2020-05-21 13:57:49 +03:00
Anna Shaleva
1afd630169 core: add senders' fees monitor to mempool
In order to simplify maintainance of overall current senders` fees
in the mempool we are to keep them in a separate map.
2020-05-20 23:34:25 +03:00
Anna Shaleva
7acf5b2841 core: add SystemFee and NetworkFee to transaction
closes #831
2020-05-20 23:26:48 +03:00
Anna Shaleva
29d321b5e1 *: drop miner transaction
1. Completely remove miner transaction

2. Change validation rule for block: block without transactions is
valid.
2020-04-27 17:57:37 +03:00
Anna Shaleva
aa554f0a9a core: update block timestamp format
Changed block.Timestamp from uint32 to uint64
2020-04-27 17:56:44 +03:00
Roman Khimov
b83e84ca08 core: switch to the new voting system (via native NEO contract)
It has all the methods required now, so you can register, vote and get
voting results. Fixes #865.
2020-04-27 16:06:50 +03:00
Evgenii Stratonikov
c508a36827 core,network: shutdown services in tests properly 2020-04-22 17:53:59 +03:00
Roman Khimov
9a67bfac0b network: drop checksums from messages
Follow neo-project/neo#710 changes.
2020-04-19 23:40:31 +03:00
Evgenii Stratonikov
8a2130f5b6 core: extend Blockchainer with GetStandByValidators()
A list of standby validators can be needed in native contracts.
2020-04-16 15:54:58 +03:00
Evgenii Stratonikov
a7c19d445b core: move Blockchainer interface to a separate package 2020-04-11 10:56:36 +03:00
Roman Khimov
5f09381cf4 core: wrap cached dao properly, don't miss cached data
Fixes #817 where invoked contract missed updated account information because
it got it one layer below cachedDao used to process the block.
2020-04-03 10:15:11 +03:00
Roman Khimov
9b5dab57e8
Merge pull request #810 from nspcc-dev/refactor/config
config: move config.go out of config
2020-04-01 10:29:57 +03:00
Anna Shaleva
5a984fdf88 config: move config.go out of config/
closes #423
2020-03-31 17:55:59 +03:00
Evgenii Stratonikov
9abda40171 testserdes: implement helpers for encode/decode routines
Frequently one needs to check if struct serializes/deserializes
properly. This commit implements helpers for such cases including:
1. JSON
2. io.Serializable interface
2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
df2598c8dc core: store NEP5 balances separately
There is no need to take and unmarshal an account structure only
to get it's NEP5 balances.
2020-03-12 12:58:28 +03:00
Roman Khimov
5f1868af28
Merge pull request #743 from nspcc-dev/optimize-some-db-accesses
Optimize some db accesses
2020-03-11 17:33:36 +03:00
Roman Khimov
377fb382aa core: move (un)SpentCoin structs into the state package
As they're all about the state.
2020-03-11 12:22:52 +03:00
Evgenii Stratonikov
b7dee156e2 network: fix a deadlock in DefaultDiscovery
Why a deadlock can occur:
1. (*DefaultDiscovery).run() has a for loop over requestCh channel.
2. (*DefaultDiscovery).RequestRemote() send to this channel while
    holding a mutex.
3. (*DefaultDiscovery).RegisterBadAddr() tries to take mutex for write.
4. Second select-case can't take mutex for read because of (3).
2020-03-10 15:40:23 +03:00
Anna Shaleva
456a2d55fd rpc: implement getvalidators
closes #714
2020-03-07 18:05:40 +03:00
Roman Khimov
fbdc60b731
Merge pull request #723 from nspcc-dev/feature/nep5
core,rpc: implement NEP5-related logic
2020-03-05 18:33:58 +03:00
Evgenii Stratonikov
95a8fa234f rpc: implement getnep5transfers RPC 2020-03-05 18:22:40 +03:00
Roman Khimov
31948ee4a7 network: add a nil check in (*TCPTransport).Close, prevent panic
You have to try to trigger that, but in the event of server shutdown before it
actually started listening Close will easily panic like this:

2020-03-05T11:48:08.660+0300    INFO    node started    {"blockHeight": 6458, "headerHeight": 15605}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x9d5f0a]

goroutine 1 [running]:
github.com/nspcc-dev/neo-go/pkg/network.(*TCPTransport).Close(0xc000fd7050)
        /home/rik/dev/neo-go/pkg/network/tcp_transport.go:78 +0x2a
github.com/nspcc-dev/neo-go/pkg/network.(*Server).Shutdown(0xc000182280)
        /home/rik/dev/neo-go/pkg/network/server.go:190 +0x189
github.com/nspcc-dev/neo-go/cli/server.startServer(0xc000200160, 0x0, 0x0)
        /home/rik/dev/neo-go/cli/server/server.go:367 +0x79e
github.com/urfave/cli.HandleAction(0xb84860, 0xccf240, 0xc000200160, 0xc0001c4500, 0x0)
        /home/rik/dev/neo-go/vendor/github.com/urfave/cli/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0xc9a160, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0xca407f, 0x10, 0x0, ...)
        /home/rik/dev/neo-go/vendor/github.com/urfave/cli/command.go:210 +0x996
github.com/urfave/cli.(*App).Run(0xc0001d64e0, 0xc0000ca000, 0x3, 0x3, 0x0, 0x0)
        /home/rik/dev/neo-go/vendor/github.com/urfave/cli/app.go:255 +0x6af
main.main()
        /home/rik/dev/neo-go/cli/main.go:25 +0x505
2020-03-05 11:53:26 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Evgenii Stratonikov
7095ec6c51 core: implement (*Blockchain).CalculateClaimable
Calculating amount of GAS that can be claimed is required
for getclaimable RPC.
2020-03-02 18:00:00 +03:00
Roman Khimov
252a9f2f31
Merge pull request #690 from nspcc-dev/feature/getapplicationlog
rpc: implement getapplicationlog RPC
2020-03-02 17:41:32 +03:00
Anna Shaleva
ff4384d7ff rpc: implement getapplicationlog RPC
Closes #500
2020-03-02 17:25:27 +03:00
Evgenii Stratonikov
a3dacd3b74 tests: replace t.Fatal with require where possible
This makes tests less verbose and unifies the style
they are written in.
2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
0ef7a76e84 network: batch transactions on broadcast 2020-03-02 10:37:27 +03:00
Roman Khimov
36197056b1
Merge pull request #691 from nspcc-dev/network-conn-overflow-and-shutdown
Network conn overflow and shutdown
2020-02-28 18:16:29 +03:00
Roman Khimov
e213e69a33 network: rework shutdown sequence of Server
Close transport and disconnect peers right in the Shutdown(), so that no new
connections would be accepted and so that all the peers would be disconnected
correctly (avoiding the same deadlock as in e2116e4c3f).
2020-02-28 16:22:04 +03:00
Roman Khimov
77624a8847 network: add Close() to discoverer, shut it down on exit 2020-02-28 16:22:04 +03:00
Roman Khimov
9f7018503a core/transaction: use new transaction.InOut for References
We don't need a map here, use simpler structures.
2020-02-27 12:00:03 +03:00
Roman Khimov
e2116e4c3f network: don't deadlock on connection overflow
(*Peer).Disconnect send an unregister signal to this goroutine, so invoking it
from here is not a good idea, run it asynchronously.
2020-02-24 15:56:49 +03:00
Roman Khimov
723b33e108 network: implement unconnected/bad peers getters
Which allows node to respond to `getpeers` RPC request correctly.
2020-02-21 15:12:03 +03:00
Roman Khimov
d92e193e63 rpc/network: refactor getpeers logic
Deduplicate and simplify code.
2020-02-21 15:12:03 +03:00
Roman Khimov
eb11e5fb11 core: implement basic policying support, fix #370
Implement mempool and consensus block creation policies, almost the same as
SimplePolicy plugin for C# node provides with two caveats:
 * HighPriorityTxType is not configured and hardcoded to ClaimType
 * BlockedAccounts are not supported

Other than that it allows us to run successfuly as testnet CN, previously our
proposals were rejected because we were proposing blocks with oversized
transactions (that are rejected by PoolTx() now).

Mainnet and testnet configuration files are updated accordingly, but privnet
is left as is with no limits.

Configuration is currently attached to the Blockchain and so is the code that
does policying, it may be moved somewhere in the future, but it works for
now.
2020-02-19 12:19:02 +03:00
Roman Khimov
06daeb44f3 core: make IsLowPriority work with pre-calculated fee
Don't recalculate it again and again.
2020-02-19 12:19:02 +03:00
Roman Khimov
37c48b00b4 consensus/network: reinit dbft after block addition
Don't stall on some height if everyone else have moved up already. Fix #673.
2020-02-19 12:19:02 +03:00
Roman Khimov
c5d54e9992 network: introduce (*Server).IsInSync, start consensus in synced state
We define synchronized state as a combination of minimum number of peers and
chain height being not behind of more than 2/3 of these peers.
2020-02-19 12:13:27 +03:00
Roman Khimov
d1a2296939 network: change the disconnect procedure
We can still lock the (*Server).run with dead peers:

Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: goroutine 40 [select, 871 minutes]:
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).putPacketIntoQueue(0xc030ab5320, 0xc02f251f20, 0xc00af0dcc0, 0x18, 0x40, 0x100000000000000, 0xffffffffffffffff)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:82 +0xf4
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).EnqueueHPPacket(0xc030ab5320, 0xc00af0dcc0, 0x18, 0x40, 0x1367240, 0xc03090ef98)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:124 +0x52
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).iteratePeersWithSendMsg(0xc0000ca000, 0xc00af35800, 0xcb2a58, 0x0)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:720 +0x12a
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).broadcastHPMessage(...)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:731
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).run(0xc0000ca000)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:203 +0xee4
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*Server).Start(0xc0000ca000, 0xc000072ba0)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:173 +0x2ec
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: created by github.com/CityOfZion/neo-go/cli/server.startServer
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/cli/server/server.go:331 +0x476
...
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: goroutine 2199 [chan send, 870 minutes]:
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).Disconnect.func1()
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:366 +0x85
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: sync.(*Once).Do(0xc030ab403c, 0xc02f262788)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/usr/local/go/src/sync/once.go:44 +0xb3
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).Disconnect(0xc030ab4000, 0xd92440, 0xc000065a00)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:365 +0x6d
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).SendPing.func1()
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:394 +0x42
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: created by time.goFunc
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/usr/local/go/src/time/sleep.go:169 +0x44
...
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: goroutine 3448 [chan send, 854 minutes]:
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).handleConn(0xc01ed203f0)
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:143 +0x6c
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: created by github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).Accept
Feb 13 16:14:50 neo-go-node-2 neo-go[9448]: #011/go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_transport.go:62 +0x44c
...

The problem is that the select in putPacketIntoQueue() only works the way it
was intended to after the `close(p.done)`, but that happens only after
successful unregistration request send. Thus, do disconnects the other way
around, first unblock queueing and exit goroutines, then destroy the
connection (if it wasn't previously destroyed) and only after that signal to
the Server.
2020-02-13 16:24:46 +03:00
Roman Khimov
7ee8f9c5d8 network: fix networking stalls caused by stale peers
We can leak sending goroutines and stall broadcasts because of already gone
peers that happened to be cached by some s.Peers() user (more than 800 of
these can be seen in nodoka log along with (*Server).run blocking on
CMDGetAddr send):

Feb 10 16:35:15 nodoka neo-go[1563]: goroutine 41 [chan send, 3320 minutes]:
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).putPacketIntoQueue(...)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:81
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).EnqueueHPPacket(0xc0083d57a0, 0xc017206100, 0x18, 0x40, 0x136a240, 0xc018ef9720)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:119 +0x98
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).iteratePeersWithSendMsg(0xc0000ca000, 0xc0001848a0, 0xcb4550, 0x0)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:720 +0x12a
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).broadcastHPMessage(...)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:731
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).run(0xc0000ca000)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:203 +0xee4
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).Start(0xc0000ca000, 0xc000072c60)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:173 +0x2ec
Feb 10 16:35:15 nodoka neo-go[1563]: created by github.com/CityOfZion/neo-go/cli/server.startServer
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/cli/server/server.go:331 +0x476
2020-02-10 18:47:52 +03:00
Roman Khimov
c896e2b731 network: fix minor gofmt issues (spotted by goreportcard) 2020-02-08 15:53:08 +03:00
Roman Khimov
b9b77ac1be network: fix block relaying, don't spit out useless errors
We can only add one block of the given height and we have two competing
goroutines to do that --- consensus and block queue. Whomever adds the block
first shouldn't trigger an error in another one.

Fix block relaying for blocks added via the block queue also, previously one
consensus-generated blocks were broadcasted.
2020-02-06 15:41:56 +03:00
Roman Khimov
70b3839fd0 core/mempool: fix AddBlock and tx pooling concurrency issues
Eliminate races between tx checks and adding them to the mempool, ensure the
chain doesn't change while we're working with the new tx. Ensure only one
block addition attempt could be in progress.
2020-02-06 15:41:52 +03:00
Roman Khimov
f0bb886be3 mempool: make all methods pointer methods
Makes no sense copying the Pool around.
2020-02-04 17:36:11 +03:00
Roman Khimov
e5d0125a3f network: do async broadcast when processing P2P TX
It can lead to some goroutine explosion, but supposedly it's better than
stalling other processing and eventually all of these goroutines should finish
their sends. Note that this doesn't change the behavior for RPC-relayed
transactions that are still waiting for the broadcast to finish ensuring
proper transaction distribution before returning the result to the client.
2020-01-30 14:03:52 +03:00
Roman Khimov
0fcbc697ca network: only tell consensus service about new valid transactions
Transactions can be bad and can duplicate each other, consensus service
shouldn't be bothered with that.
2020-01-30 14:03:52 +03:00
Roman Khimov
fdbaac7a30 network: prevent broadcast queue starving, share time with p2p
Blocked broadcast queue of one peer may affect broadcasting capabilities of
the server, so prevent total blocking of it by p2p queue.
2020-01-30 14:03:52 +03:00
Roman Khimov
b2c4587dad network: fix PeerAddr() for not-yet-handshaked case
If we have already got Version message, we don't need the rest of handshake to
complete before being able to properly answer the PeerAddr() requests. Fixes
some duplicate connections between machines.
2020-01-30 14:03:52 +03:00
Roman Khimov
49bd7aada5 network: log peerCount on connection registration
Symmetrical to disconnects, make it easier to look through the logs.
2020-01-30 14:03:52 +03:00
Roman Khimov
32695b4f40 network: don't unregister connected peers
If we drop connection because we're already connected don't unregister this
connected address because it's connected anyway!
2020-01-30 14:03:52 +03:00
Roman Khimov
eb4ec61b8b network: register connected addr in handleVersionCmd()
Prevent useless attempts to connect to this peer if the peer has already made
a connection to us.
2020-01-30 14:03:52 +03:00
Roman Khimov
9eafec0d1d network: introduce peer-to-peer message queue
This one is designed to give more priority to direct nodes communication, that
is that their messaging would have more priority than generic broadcasts. It
should improve consensus process under TX pressure and allow to handle
pings in time (preventing disconnects).
2020-01-30 14:03:52 +03:00
Roman Khimov
72e4eb7172 network: fix wrong NewPing() parameters
They have the opposite order, height first and nonce second. It was done wrong
in 4e6ed902 and never fixed since. Fixes sending wrong peer state leading to
useless getheaders messages (and disconnects when the other side is lagging
behind).
2020-01-30 14:03:52 +03:00
Roman Khimov
da09fda575 network: add debug on message receival 2020-01-30 14:03:52 +03:00
Roman Khimov
1c28dd2567 network: add message type to disconnect error message
If it was caused by message processing, but only after the handshake to
preserve errIdenticalID and other handshaking errors.
2020-01-30 14:03:52 +03:00
Roman Khimov
b79abd1e27 network: detect duplicate connections before handshake completes
We can have more than one connection attempt in progress and not yet completed
the handshake, so if there is a Version already received we should look it.
2020-01-30 14:03:52 +03:00
Roman Khimov
06c3fbe455 network: rework ping sends, fix overpinging
Our node was too pingy because of wrong timer setups (that divided timeout
Duration by time.Second), it also was wrong in its time calculations (using
UTC time to calculate intervals). At the same time missing block is a
server-wide problem, so it's better solved with server-wide protocol loop.
2020-01-28 17:39:52 +03:00
Roman Khimov
39800aecb3 network: fix getdata handling
It was broken by 0ba6b2a7, the peer only responded with one message instead of
full requested set.
2020-01-28 17:39:52 +03:00
Roman Khimov
9aa5d0cf1e
Merge pull request #615 from nspcc-dev/network-peer-updates
Network peer updates
2020-01-23 13:15:43 +03:00
Roman Khimov
99dfdc19e7 network: drop now useless addrReq queue from the server
Just broadcast a high-priority message to everyone.
2020-01-22 11:28:59 +03:00
Roman Khimov
ea3b76ded1 network: make NewServer return an error, fix #612
It can return nil in two cases, so we're better return an error and handle
it.
2020-01-22 11:17:51 +03:00
Roman Khimov
f2ffffddb7 network: rework broadcasting functions, tune priorities
This gives more priority to anything related to consensus.
2020-01-22 11:01:13 +03:00
Roman Khimov
34b863d645 network: introduce Server's MkMsg()
That wraps NewMessage() for a configured network.
2020-01-21 17:31:51 +03:00
Roman Khimov
1f672e0da7 network: move SendVersion() to the Peer
Only leave server-specific `getVersionMsg()` in the Server, all the other
logic is peer-related.
2020-01-21 17:26:08 +03:00
Roman Khimov
f56383e9c8 network: use p.LastBlockIndex() in requestBlocks()
Always compare to the best known block index, comparing to the StartHeight is
just plain wrong now.
2020-01-20 19:37:17 +03:00
Roman Khimov
2c4ace022e network/config: redesign ping timeout handling a bit
1) Make timeout a timeout, don't do magic ping counts.
2) Drop additional timer from the main peer's protocol loop, create it
   dynamically and make it disconnect the peer.
3) Don't expose the ping counter to the outside, handle more logic inside the
   Peer.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
62092c703d network: use local timestamp to decide when to ping
We don't and we won't have synchronized clocks in the network so the only
timestamp that we can compare our local time with is the one made
ourselves. What this ping mechanism is used for is to recover from missing the
block broadcast, thus it's appropriate for it to trigger after X seconds of
the local time since the last block received.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
a8252ecc05 network: remove wrong ping condition
In reality it will never be true exactly in the case where we want this ping
mechanism to work --- when the node failed to get a block from the net. It
won't get the header either and thus its block height will be equal to header
height. The only moment when this condition is met is when the node does
initial synchronization and this synchronization works just fine without any
pings.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
247cfa4165 network: either request blocks or ping a peer, but not both
It makes to sense to do both actions, pings are made for a different purpose.

Relates to #430.
2020-01-20 19:37:17 +03:00
Roman Khimov
0ba6b2a754 network: introduce peer sending queues
Two queues for high-priority and ordinary messages. Fixes #590. These queues
are deliberately made small to avoid buffer bloat problem, there is gonna be
another queueing layer above them to compensate for that. The queues are
designed to be synchronous in enqueueing, async capabilities are to be added
layer above later.
2020-01-20 17:23:26 +03:00
Roman Khimov
7f0882767c network: remove useless Done() method from the peer
It's internal state of the peer that no one should care about.
2020-01-20 17:23:26 +03:00
Roman Khimov
f39d5d5a10 network: fix unregistration on peer Disconnect
It should always signal to the server, not duplicating this send and not
missing it like it happened in the Server.run().
2020-01-20 17:23:26 +03:00
Roman Khimov
907a236285 network: move per-peer goroutines into the TCPPeer
As they're directly tied to it.
2020-01-20 17:23:26 +03:00
Roman Khimov
32213b1454
Merge pull request #601 from nspcc-dev/refactoring/core
core: refactor out Block, BlockBase and Header, closes #597.
2020-01-20 16:19:20 +03:00
Roman Khimov
bb80ba9b9e
Merge pull request #456 from nspcc-dev/pingpong_430
add ping pong processing
2020-01-20 16:10:29 +03:00
Vsevolod Brekelov
4e6ed9021c network: add ping pong processing
add pingInterval same as used in ref C# implementation with the same logic
add pingTimeout which is used to check whether pong received. If not -- drop the peer.
add pingLimit which is hardcoded to 4 in TCPPeer. It's limit for unsuccessful ping/pong calls (where pong wasn't received in pingTimeout interval)
2020-01-17 13:24:14 +03:00
Evgenii Stratonikov
28183b81d6 mempool: simplify names of exported types
With the move to a separate package, naming can be simplified:
MemPool -> Pool, PoolItem -> Item, PoolItems -> Items.
2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
fed6fba9b6 core: refactor out MemPool 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
489b88afbb block: rename BlockBase to Base 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
63c56cca5c core: refactor out Block, BlockBase and Header structs
See #597.
2020-01-16 10:16:24 +03:00
Roman Khimov
7d4d57351e network: fix requestTx() behaviour for consensus service
It wasn't actually requesting transactions but rather sending an inventory
message telling everyone that we have them which is completely wrong and
easily leads to ChangeView that could be avoided.
2020-01-15 14:31:56 +03:00
Roman Khimov
0420d48e56 network: micro-optimize relayInventory
Do less allocations, we're sending the same message with the same payload.
2020-01-15 13:16:09 +03:00
Roman Khimov
7ba5267494
Merge pull request #596 from nspcc-dev/feature/single_node
network: allow single-node privnet setup. Closes #595.
2020-01-14 13:17:04 +03:00
Evgenii Stratonikov
70b23076f8 network: allow single-node privnet setup 2020-01-13 18:01:20 +03:00
Evgenii Stratonikov
5f5d0097e2 network: use TimePerBlock from config 2020-01-13 17:58:12 +03:00
Roman Khimov
2192b6602a network: fix logic and logging for sendVersion() errors
It doesn't make sense trying to read from a broken connection and the
disconnect is to be logger from the server on unregistration.
2020-01-13 17:33:29 +03:00
Evgenii Stratonikov
f3921ea3bd network: fix NPE in logging 2020-01-13 15:06:57 +03:00
Evgenii Stratonikov
9b8b77c9ea network: return error if header message is too big
Big messages can still be processed but only first
2000 headers will be used.
2020-01-10 11:14:29 +03:00
Evgenii Stratonikov
aecdf470e7 cli,pkg: use zap.Logger 2020-01-10 11:14:27 +03:00
Roman Khimov
234d94d27e network: implement getblocks command
Fixes #577, tested with C# nodes connecting to neo-go privnet.
2019-12-25 19:46:57 +03:00
Evgenii Stratonikov
884779e501 *: goimports 2019-12-17 14:51:28 +03:00
Evgenii Stratonikov
014be31a0a network: fix link in pprof 2019-12-17 14:49:28 +03:00
Vsevolod Brekelov
0085831ec5 network: add Pprof metrics
Since we have some perf issues from time to time it is good to have pprof debugger. Disabled by default.
2019-12-17 14:13:07 +03:00
Roman Khimov
8b3080b972 io: rename Read/WriteBytes to Read/WriteB
go vet is not happy about them:
  pkg/io/binaryReader.go:92:21: method ReadByte() byte should have signature ReadByte() (byte, error)
  pkg/io/binaryWriter.go:75:21: method WriteByte(u8 byte) should have signature WriteByte(byte) error
2019-12-12 20:19:50 +03:00
Roman Khimov
54d888ba70 io: add type-specific read/write methods
This seriously improves the serialization/deserialization performance for
several reasons:
 * no time spent in `binary` reflection
 * no memory allocations being made on every read/write
 * uses fast ReadBytes everywhere it's appropriate

It also makes Fixed8 Serializable just for convenience.
2019-12-12 20:19:50 +03:00
Vsevolod Brekelov
2d42b14a1d core: renames entities-> state and removed State prefix 2019-12-11 13:14:18 +03:00
Vsevolod Brekelov
ec17654986 core: refactoring blockchain state and storage
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
2019-12-11 13:05:31 +03:00
Roman Khimov
7e371588a7 core/tx: remove one layer of indirection for scripts and inouts
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
2019-12-09 17:14:10 +03:00
Evgenii Stratonikov
fccb008594 io: implement ReadBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
838050f8b5 io: rename ReadBytes() to ReadVarBytes() 2019-12-09 15:00:15 +03:00
Roman Khimov
e4d821f32d
Merge pull request #546 from nspcc-dev/write-optimizations
Write optimizations
2019-12-06 19:40:38 +03:00
Roman Khimov
844491d365 *: use more efficient WriteBytes where appropriate
Before this patch on block import we could easily be spending more than 6
seconds out of 30 in Uint256 encoding for UnspentBalance, now it's completely
off the radar.
2019-12-06 18:22:21 +03:00
Evgenii Stratonikov
7179e4ba9f util: add LE suffix to Uint256 methods 2019-12-06 12:16:55 +03:00
Roman Khimov
f48228ef7d
Merge pull request #467 from nspcc-dev/errcheck_297
This patchset closes #297 and #457.
2019-12-03 15:06:11 +03:00
Vsevolod Brekelov
03ff2976ed io: refactoring for using WriteVarBytes instead of WriteLE
goal is to be consistent with C# implementation.
For writing []byte WriteBytes used and for byte - WriteVarByte.
2019-12-03 13:49:33 +03:00
Roman Khimov
abc0ec33bd
Merge pull request #532 from nspcc-dev/fix/peer_count
Consensus fixes.
2019-12-02 18:08:36 +03:00
Evgenii Stratonikov
7c900edd2d network: count only handshaked peers as connected 2019-12-02 15:09:09 +03:00
Roman Khimov
1da76a8eb4 network: rework inventory handling, check for item presence
Don't ask peers about the items we already have.
2019-12-02 11:02:52 +03:00
Roman Khimov
b4d9935bda network: deduplicate a part of RelayTxn()
We already have relayInventory() for this.
2019-11-29 16:26:44 +03:00
Roman Khimov
293615ea5f network/consensus: add new block relaying
Tell everyone about our new shiny blocks.
2019-11-29 12:27:15 +03:00
Roman Khimov
4d286dcfeb network: check height before requesting headers
Only request headers from the other peer if his height is bigger than
ours. Otherwise we routinely ask 0-height newcomers for some random headers
that they know nothing about.
2019-11-29 11:48:17 +03:00
Roman Khimov
a730529b0c network: process incoming p2p transactions
It's the same relaying as for RPC, but we don't need to return any result for
it.
2019-11-29 11:09:54 +03:00
Roman Khimov
9f9cf4ae3f network: add getheaders message processing
This one is essential for the consensus nodes as otherwise they won't give out
the blocks they generate making their generation almost useless. It also makes
our networking part more complete.
2019-11-29 11:08:22 +03:00
Roman Khimov
734338ad70 network: move Version sending to the tcp transport
We have a race between reader and writer goroutines for the same connection
that leads to handshake failures when reader is faster to read the incoming
version (and try to reply to it) than writer is to write our own Version:

WARN[0000] peer disconnected                             addr="172.200.0.4:20334" peerCount=5 reason="invalid handshake: tried to send VersionAck, but didn't send Version yet

Fix it by moving Version sending before the reader loop starts.
2019-11-29 11:05:42 +03:00
Evgenii Stratonikov
b5fb43bac9 network: don't reconnect to peers with identical ID 2019-11-27 11:58:17 +03:00
Evgenii Stratonikov
fdd5276d3e network: plug in dBFT library 2019-11-27 10:57:22 +03:00
Vsevolod Brekelov
c80ee952a1 core/crypto: add validators and interop for validators
add processing of validators while block persist;
add validator structure with decoding/encoding;
add validator get from store;
add EnrollmentTX and StateTX processing;
add pubkey decode bytes, unique and contains functions;
2019-11-21 15:23:35 +03:00
Roman Khimov
5076704019
Merge pull request #502 from nspcc-dev/docker-compose-environment-for-neo-go
Create docker-compose environment
2019-11-18 17:21:11 +03:00
Evgeniy Kulikov
2f8b11dabf
reformat code at pkg/network/metrics/metrics.go 2019-11-18 17:17:21 +03:00
Evgeniy Kulikov
0a56d3ddbc
network: generate randomized server id
math/rand might generate same id's on one environment, so.. use crypto/rand for generation id's
2019-11-18 17:17:21 +03:00
Evgenii Stratonikov
e3098ed0f8 network: write messages atomically
Right now message can be written in several Write's so
concurrent calls of writeMsg() can in theory interleave.
This commit fixes it.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-18 09:31:00 +03:00
Evgenii Stratonikov
2be18f91df util: implement io.Serializable for Uint256 2019-11-14 14:20:38 +03:00
Evgenii Stratonikov
b16e56a47b io: refactor BinReader.ReadArray()
Make it accept arbitrary slice pointer.
2019-11-14 14:19:58 +03:00
Evgenii Stratonikov
085ca7b770 network: implement Consensus payloads 2019-11-13 17:27:25 +03:00
Evgenii Stratonikov
ad9091d13d io: implement generic array (de-)serialization
It is done through reflection and panics
in every unexpected situation.
2019-11-13 17:27:23 +03:00
Roman Khimov
b05754deac core: add Close() to blockchainer, implement it to properly close chain
Before it the deferred function in Run() was actually never able to properly
close the Store, so we weren't synching the latest state to the disk.
2019-11-08 12:19:54 +03:00
Vsevolod
11b372b45d
Merge pull request #480 from nspcc-dev/handshake-wait-for-version
network: wait for both Version messages before ACKing
2019-11-06 18:33:20 +03:00
Roman Khimov
d7f747fa9a network: wait for both Version messages before ACKing
Otherwise the node might crash in `startProtocol` because of missing Version
field in the peer. And it also keeps the sequence correct, Version MUST be
sent first and ACKs can only follow it.
2019-11-06 18:05:50 +03:00
Vsevolod Brekelov
11ce73af28 server: add log-path and address configuration
- LogPath can be configured through config
- node,rpc and monitoring address can be configured thought command line
or config
2019-11-06 15:58:54 +03:00
Roman Khimov
7cf9a40468 network: fix MaxPeers, introduce AttemptConnPeers
Our node didn't respect the MaxPeers setting, fix it with a drop of random
connection when this limit is reached (to give a chance for newcomers to
communicate), but also introduce AttemptConnPeers setting to tune the number
of attempted connections.

This also raises the default MaxPeers for testnet/mainnet to 100, because
neo-go nodes love making friends.
2019-11-06 15:29:58 +03:00
Roman Khimov
31954bb20c network: disallow double connections to the same peer
Makes no sense and C# node does it too.
2019-11-06 15:29:58 +03:00
Roman Khimov
d5a7ad2c47 network: fix data race in server peers map access 2019-11-06 15:29:58 +03:00
Roman Khimov
ec76ed23a5 network: rework peer handshaking, fix #458
This allows to start handshaking from both client and server (mainnet/testnet
nodes were seen to not care about string ordering for it), but still maintains
some sane checks in the process. It also makes functions thread-safe because
we have two goroutines servicing read and write side of the Peer connection,
so they can clash on access to the struct fields.

Add a test for it also.
2019-11-06 15:29:58 +03:00
Roman Khimov
e859e03240 network: split Peer's NetAddr into RemoteAddr and PeerAddr
As they are different things used for different purposes.
2019-11-06 15:26:24 +03:00
Roman Khimov
e9f8b25776 network: add default MinPeers
If there is nothing specified in the configuration, set it to some reasonable
value.
2019-11-01 13:39:47 +03:00
Roman Khimov
b12a68994d network: make minpeers configurable
Without it node in privnet tries to reconnect to four existing nodes
indefinitely for no good reason.
2019-10-31 15:10:00 +03:00
Roman Khimov
2f6e678a19
Merge pull request #463 from nspcc-dev/smartcontract-fixes
Smartcontract RPC fixes
2019-10-29 20:54:46 +03:00
Vsevolod Brekelov
d374175170 monitoring: add prometheus monitoring
add init metrics service which uses prometheus;
add configuration for metrics service;
add monitoring metrics for blockchain,rpc,server;
2019-10-29 20:51:17 +03:00
Roman Khimov
ebc1ba4f38 rpc/core: implement invokescript method, fix #348
Extend Blockchainer with one more method to spawn a VM for test runs and use
it to run scripts. Gas consumption is not counted or limited in any way at the
moment (see #424).
2019-10-29 18:31:39 +03:00
Roman Khimov
006337b1f8 network: rework discovery with rwmutex, add test
Keeping run() as the owner of all maps would mean adding at least three more
channels to keep address getters with thread-safety. But then there also is a
race between requestToWork() and run() which is way harder to solve with
channels because there are lots of possibilities for deadlocks. So rework all
of this with good old mutexes.

While at it, fix `requestCh` handling in the inner select of run, it will waste
one loop to handle it, so we should add one to the `requested`.

Fixes #445.
2019-10-28 13:37:27 +03:00
Roman Khimov
77a50d6dc6 network: remove useless checks in discovery
These are useless.
2019-10-27 16:11:32 +03:00
Roman Khimov
9ff55ac02c network: move inventory semantic check into common code
As per @volekerb suggestion.
2019-10-24 13:12:02 +03:00
Roman Khimov
257ba1c2ad network: implement getdata message handling in server
To properly implement relaying we need to answer these requests from other
nodes.
2019-10-24 13:12:01 +03:00
Vsevolod Brekelov
8ee421db14 fix spelling and godoc comments 2019-10-22 17:56:03 +03:00
Roman Khimov
3fc2bf5452 *: fix some misspellings
Goreport:
   neo-go/pkg/core/contract_state_test.go
        Line 21: warning: "Contracto" is a misspelling of "Contraction" (misspell)
        Line 64: warning: "Contracto" is a misspelling of "Contraction" (misspell)

   neo-go/pkg/core/interop_neo.go
        Line 420: warning: "succeedes" is a misspelling of "succeeds" (misspell)

   neo-go/pkg/network/discovery.go
        Line 118: warning: "succeded" is a misspelling of "succeeded" (misspell)
        Line 128: warning: "successfuly" is a misspelling of "successfully" (misspell)
2019-10-17 12:30:24 +03:00