Commit graph

6594 commits

Author SHA1 Message Date
Anna Shaleva
673a495527 rpc: add "since" filter to WS block events 2022-10-24 06:09:36 +03:00
Anna Shaleva
10a0716217 rpc: implement transaction awaiting functionality
Close #2704.
2022-10-24 06:09:36 +03:00
Anna Shaleva
6d38e75149 rpc: support multiple WSClient notification receivers 2022-10-24 06:09:36 +03:00
Anna Shaleva
4ce6bc6a66 rpc: adjust comment to Notification value cast
And explicitly specify field names on Notification creation.
2022-10-24 06:09:36 +03:00
Roman Khimov
0c3b03617e core: optimize HasBlock check for recent blocks
When block is being spread through the network we can get a lot of invs with
the same hash. Some more stale nodes may also announce previous or some
earlier block. We can avoid full DB lookup for them and minimize inv handling
time (timeouts in inv handler had happened in #2744).

It doesn't affect tests, just makes node a little less likely to spend some
considerable amount of time in the inv handler.
2022-10-21 21:21:58 +03:00
Roman Khimov
e26055190e network: check inv against currently processed transactions
Sometimes we already have it, but it's not yet processed, so we can save on
getdata request. It only affects very high-speed networks like 4-1 scenario
and it doesn't affect it a lot, but still we can do it.
2022-10-21 21:16:18 +03:00
Roman Khimov
cfb5058018 network: batch getdata replies
This is not exactly the protocol-level batching as was tried in #1770 and
proposed by neo-project/neo#2365, but it's a TCP-level change in that we now
Write() a set of messages and given that Go sets up TCP sockets with
TCP_NODELAY by default this is a substantial change, we have less packets
generated with the same amount of data. It doesn't change anything on properly
connected networks, but the ones with delays benefit from it a lot.

This also improves queueing because we no longer generate 32 messages to
deliver on transaction's GetData, it's just one stream of bytes with 32
messages inside.

Do the same with GetBlocksByIndex, we can have a lot of messages there too.

But don't forget about potential peer DoS attacks, if a peer is to request a
lot of big blocks we need to flush them before we process the whole set.
2022-10-21 17:16:32 +03:00
Roman Khimov
bf4636f70a
Merge pull request #2755 from nspcc-dev/improve-networking
Improve P2P transaction handling
2022-10-21 16:22:02 +07:00
Roman Khimov
e1b5ac9b81 network: separate tx handling from msg handling
This allows to naturally scale transaction processing if we have some peer
that is sending a lot of them while others are mostly silent. It also can help
somewhat in the event we have 50 peers that all send transactions. 4+1
scenario benefits a lot from it, while 7+2 slows down a little. Delayed
scenarios don't care.

Surprisingly, this also makes disconnects (#2744) much more rare, 4-node
scenario almost never sees it now. Most probably this is the case where peers
affect each other a lot, single-threaded transaction receiver can be slow
enough to trigger some timeout in getdata handler of its peer (because it
tries to push a number of replies).
2022-10-21 12:11:24 +03:00
Roman Khimov
09015e06d5
Merge pull request #2754 from nspcc-dev/move-neofs-mainnet-fork-date
config: reschedule Aspidochelone fork block for NeoFS mainnet
2022-10-21 16:02:23 +07:00
Roman Khimov
e003b67418 network: reuse inventory hash list for request hashes
Microoptimization, we can do this because we only use them in handleInvCmd().
2022-10-21 11:28:40 +03:00
Roman Khimov
831067c29e config: reschedule Aspidochelone fork block for NeoFS mainnet
We can't make it before 2550000, so move to 2600000. @anatoly-bogatyrev,
please ensure you're using this new height for mainnet NeoFS configurations
next week.
2022-10-21 10:52:11 +03:00
Roman Khimov
295add052b
Merge pull request #2752 from nspcc-dev/initialize-dbft-with-previous-timestamp
dbft: rev up, fix context timestamp initialization
2022-10-20 16:53:08 +07:00
Roman Khimov
493b979b95 dbft: rev up, fix context timestamp initialization
See nspcc-dev/dbft#64, fixes #2753.
2022-10-20 12:38:14 +03:00
Roman Khimov
0f625f04f0
Merge pull request #2748 from nspcc-dev/stop-tx-flow
network/consensus: use new dbft StopTxFlow callback
2022-10-18 16:29:37 +07:00
Roman Khimov
73ce898e27 network/consensus: use new dbft StopTxFlow callback
It makes sense in general (further narrowing down the time window when
transactions are processed by consensus thread) and it improves block times a
little too, especially in the 7+2 scenario.

Related to #2744.
2022-10-18 11:06:20 +03:00
Roman Khimov
903beeb5df
Merge pull request #2750 from nspcc-dev/network-prometheus-metrics
network: add prometheus histogram with cmd processing time
2022-10-18 13:07:50 +07:00
Roman Khimov
2791127ee4 network: add prometheus histogram with cmd processing time
It can be useful to detect some performance issues.
2022-10-17 22:51:16 +03:00
Roman Khimov
73079745ab
Merge pull request #2746 from nspcc-dev/optimize-tx-callbacks
network: only call tx callback if we're waiting for transactions
2022-10-17 16:39:41 +07:00
Roman Khimov
45f4e8c649
Merge pull request #2747 from aprasolova/codeowners
Add CODEOWNERS file
2022-10-17 13:37:25 +07:00
anastasia prasolova
f843b983ef Add CODEOWNERS file
Signed-off-by: anastasia prasolova <anastasia@nspcc.ru>
2022-10-16 21:14:58 +03:00
Roman Khimov
dce9f80585
Merge pull request #2743 from nspcc-dev/log-fan-out
Logarithmic gossip fan out
2022-10-14 23:18:34 +07:00
Roman Khimov
4dd3fd4ac0 network: only call tx callback if we're waiting for transactions
Until the consensus process starts for a new block and until it really needs
some transactions we can spare some cycles by not delivering transactions to
it. In tests this doesn't affect TPS, but makes block delays a bit more
stable. Related to #2744, I think it also may cause timeouts during
transaction processing (waiting on the consensus process channel while it does
something dBFT-related).
2022-10-14 18:45:48 +03:00
Roman Khimov
65f0fadddb network: register peer only if it's not a duplicate 2022-10-14 15:53:32 +03:00
Roman Khimov
851cbc7dab network: implement adaptive peer requests
When the network is big enough, MinPeers may be suboptimal for good network
connectivity, but if we know the network size we can do some estimation on the
number of sufficient peers.
2022-10-14 15:53:32 +03:00
Roman Khimov
c17b2afab5 network: add BroadcastFactor to control gossip, fix #2678 2022-10-14 15:53:32 +03:00
Roman Khimov
3ed140abbf config: raise mainnet/testnet MinPeers to 10
That's the configuration C# node has by default and 5 is really a very small
number of peers.
2022-10-14 15:53:32 +03:00
Roman Khimov
215e8704f1 network: simplify discoverer, make it almost a lib
We already have two basic lists: connected and unconnected nodes, we don't
need an additional channel and we don't need a goroutine to handle it.
2022-10-14 15:53:32 +03:00
Roman Khimov
c1ef326183 network: re-add addresses to the pool on UnregisterConnectedAddr
That's what we do anyway, but this way we can be a bit more efficient.
2022-10-14 14:12:33 +03:00
Roman Khimov
631f166709 network: broadcast to log-dependent number of nodes
Fixes #608.
2022-10-14 14:12:33 +03:00
Roman Khimov
c3001bc5bd
Merge pull request #2740 from nspcc-dev/cli-improvement
cli: improve VM CLI a bit more
2022-10-13 20:21:47 +07:00
Anna Shaleva
8b91428db8 cli: unify documentation of VM CLI commands
1. UsageText shows the command usage rule. Fixed/added where needed.
2. Description shows the command description, huh. It is shown right after
   UsageText, so there's no need to repeat the command usage rule. If
   Description contains Example, then it should be printed on a new line.
2022-10-13 16:07:34 +03:00
Anna Shaleva
62197929f4 cli: unify the style of Usage message for VM CLI commands
Usage message is shown on common --help command, thus it should be meaningful
and short. If user needs more detailed command description, then he can use
command-specific help.

As a result, current VM help looks pretty simple:
```
NEO-GO-VM > help
NAME:
   VM CLI - Official VM CLI for Neo-Go

USAGE:
    [global options] command [command options] [arguments...]

VERSION:
   0.99.5-pre-15-g5463ec41

COMMANDS:
   exit          Exit the VM prompt
   ip            Show current instruction
   break         Place a breakpoint
   jump          Jump to the specified instruction (absolute IP value)
   estack        Show evaluation stack contents
   istack        Show invocation stack contents
   sslot         Show static slot contents
   lslot         Show local slot contents
   aslot         Show arguments slot contents
   loadnef       Load a NEF-consistent script into the VM optionally attaching to it provided signers with scopes
   loadbase64    Load a base64-encoded script string into the VM optionally attaching to it provided signers with scopes
   loadhex       Load a hex-encoded script string into the VM optionally attaching to it provided signers with scopes
   loadgo        Compile and load a Go file with the manifest into the VM optionally attaching to it provided signers with scopes
   loadtx        Load transaction into the VM from chain or from parameter context file
   loaddeployed  Load deployed contract into the VM from chain optionally attaching to it provided signers with scopes
   reset         Unload compiled script from the VM and reset context to proper (possibly, historic) state
   parse         Parse provided argument and convert it into other possible formats
   run           Execute the current loaded script
   cont          Continue execution of the current loaded script
   step          Step (n) instruction in the program
   stepinto      Stepinto instruction to take in the debugger
   stepout       Stepout instruction to take in the debugger
   stepover      Stepover instruction to take in the debugger
   ops           Dump opcodes of the current loaded program
   events        Dump events emitted by the current loaded program
   env           Dump state of the chain that is used for VM CLI invocations (use -v for verbose node configuration)
   storage       Dump storage of the contract with the specified hash, address or ID as is at the current stage of script invocation
   changes       Dump storage changes as is at the current stage of loaded script invocation
   help, h       Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

```
2022-10-13 16:07:34 +03:00
Anna Shaleva
884428ab93 cli: add 'jump' command
Close #2606.
2022-10-13 16:07:34 +03:00
Anna Shaleva
811e09675f cli: simplify VM preparation for VM CLI
Load script from provided transaction if so.
2022-10-13 16:07:34 +03:00
Anna Shaleva
7db9258104 vm: make LoadFileWithFlags actually load with flags provided 2022-10-13 16:07:34 +03:00
Anna Shaleva
d09a0c18a7 cli: unify signers parsing
Share signers parsing code between 'contract invokefunction' and
'vm load*' commands, quite a useful thing when it comes to witness
checks.
2022-10-13 16:07:34 +03:00
Anna Shaleva
4dbaf2a123 smartcontract: add comment to GetCompleteTransaction 2022-10-13 16:07:34 +03:00
Anna Shaleva
af658bc3e5 cli: support Null as an argument for invocation-related commands 2022-10-13 16:07:31 +03:00
Roman Khimov
cff3cd906b
Merge pull request #2742 from nspcc-dev/network-code-cleanup
Some network code cleanup
2022-10-13 12:39:24 +07:00
Anna Shaleva
7eb87afab8 cli: unify parameters parsing
Share parameters parsing code between 'contract invokefunction' and
'vm run' commands. It allows VM CLI to parse more complicated parameter
types including arrays and file-backed bytestrings.
2022-10-13 08:20:27 +03:00
Roman Khimov
dc62046019 network: add network size estimation metric 2022-10-12 22:29:55 +03:00
Roman Khimov
bcf77c3c42 network: filter out not-yet-ready nodes when broadcasting
They can fail right in the getPeers or they can fail later when packet send
is attempted. Of course they can complete handshake in-between these events,
but most likely they won't and we'll waste more resources on this attempt. So
rule out bad peers immediately.
2022-10-12 16:51:01 +03:00
Roman Khimov
137f2cb192 network: deduplicate TCPPeer code a bit
context.Background() is never canceled and has no deadline, so we can avoid
duplicating some code.
2022-10-12 15:43:31 +03:00
Roman Khimov
104da8caff network: broadcast messages, enqueue packets
Drop EnqueueP2PPacket, replace EnqueueHPPacket with EnqueueHPMessage. We use
Enqueue* when we have a specific per-peer message, it makes zero sense
duplicating serialization code for it (unlike Broadcast*).
2022-10-12 15:39:20 +03:00
Roman Khimov
d5f2ad86a1 network: drop unused EnqueueMessage interface from Peer 2022-10-12 15:27:08 +03:00
Roman Khimov
b345581c72 network: pings are broadcasted, don't send them to everyone
Follow the general rules of broadcasts, even though it's somewhat different
from Inv, we just want to get some reply from our neighbors to see if we're
behind. We don't strictly need all neighbors for it.
2022-10-12 15:25:03 +03:00
Anna Shaleva
3fba4e4f17 cli: support 'loaddeployed' VM CLI command 2022-10-12 14:57:13 +03:00
Anna Shaleva
9977606e40 cli: support 'loadtx' VM CLI command 2022-10-12 14:57:10 +03:00
Roman Khimov
ec4983e88e
Merge pull request #2741 from nspcc-dev/separate-broadcast-queue-handling
Rework broadcast logic
2022-10-12 16:33:27 +07:00