TrueCloudLab/neoneo-go
7
0
Fork 2
Code Issues Pull requests Projects Releases Packages Wiki Activity
6475 commits 34 branches 134 tags 35 MiB
Commit graph

6475 commits

This branch
This branch
All branches
Author SHA1 Message Date
Anna Shaleva
0a5905390c rpc: refactor WSClient subscriptions API 
Make it more specific, close #2756.
 2022-10-26 15:32:54 +03:00
Anna Shaleva
2a53db42af neorpc: adjust and extend event filters documentation 2022-10-26 15:32:54 +03:00
Anna Shaleva
0d39602a50 docs: adjust docmentation for execution results notifications 2022-10-26 15:32:54 +03:00
Roman Khimov
ea48982b35
Merge pull request #2765 from nspcc-dev/fix-query-test 
cli: fix race in TestQueryTx
 2022-10-26 18:50:56 +07:00
Anna Shaleva
499adccbcb cli: fix race in TestQueryTx 
Problem - failing TestQueryTx test:
```
=== RUN   TestQueryTx/verbose/FAULT
=== CONT  TestQueryTx
    logger.go:130: 2022-10-26T10:47:51.414Z	DEBUG	processing rpc request	{"method": "getversion", "params": "[]"}
    logger.go:130: 2022-10-26T10:47:51.415Z	DEBUG	processing rpc request	{"method": "getnativecontracts", "params": "[]"}
    logger.go:130: 2022-10-26T10:47:51.612Z	DEBUG	processing rpc request	{"method": "getversion", "params": "[]"}
    logger.go:130: 2022-10-26T10:47:51.615Z	DEBUG	processing rpc request	{"method": "invokefunction", "params": "[11fdb7bc30a306a60dac874711a2b37b7da402c4 randomMethod  ]"}
    logger.go:130: 2022-10-26T10:47:51.617Z	DEBUG	processing rpc request	{"method": "getblockcount", "params": "[]"}
    logger.go:130: 2022-10-26T10:47:51.617Z	INFO	persisted to disk	{"blocks": 2, "keys": 154, "headerHeight": 2, "blockHeight": 2, "took": "260.704µs"}
    logger.go:130: 2022-10-26T10:47:51.618Z	DEBUG	processing rpc request	{"method": "calculatenetworkfee", "params": "[AIpp7dreAw8AAAAAAAAAAAAAAAAABQAAAAHYzFqQTVJvyabKtVugLJpv54nJVgAAK8IfDAxyYW5kb21NZXRob2QMFMQCpH17s6IRR4esDaYGozC8t/0RQWJ9W1IBACoRDCECs2Ir9AF73+MXxYrtX0x1PyBrfbiWBG+n13S7xL9/jcIRQZ7Q3Do=]"}
    logger.go:130: 2022-10-26T10:47:51.712Z	DEBUG	processing rpc request	{"method": "sendrawtransaction", "params": "[AIpp7dreAw8AAAAAABQbEgAAAAAABQAAAAHYzFqQTVJvyabKtVugLJpv54nJVgAAK8IfDAxyYW5kb21NZXRob2QMFMQCpH17s6IRR4esDaYGozC8t/0RQWJ9W1IBQgxAO2nxSMSRNFqBD5lOA37E9Px+nYDGMy6IqZromHXFtVTYD1c1hdUK4vTccoOr2AksdGwDsdQ8qIGJhXdEDxv8NSoRDCECs2Ir9AF73+MXxYrtX0x1PyBrfbiWBG+n13S7xL9/jcIRQZ7Q3Do=]"}
    logger.go:130: 2022-10-26T10:47:51.713Z	DEBUG	done processing headers	{"headerIndex": 3, "blockHeight": 2, "took": "228.756µs"}
    logger.go:130: 2022-10-26T10:47:51.813Z	DEBUG	processing rpc request	{"method": "getversion", "params": "[]"}
    logger.go:130: 2022-10-26T10:47:51.815Z	DEBUG	processing rpc request	{"method": "getnativecontracts", "params": "[]"}
    logger.go:130: 2022-10-26T10:47:51.923Z	DEBUG	processing rpc request	{"method": "getrawtransaction", "params": "[22bbb08d3f4b9e601a05c1bb1f7ac456d44bfad013287599bf3f843a75b30076 1]"}
=== CONT  TestQueryTx/verbose/FAULT
    executor.go:238:
        	Error Trace:	executor.go:238
        	            				executor.go:234
        	            				query_test.go:119
        	            				query_test.go:97
        	Error:      	Expect "OnChain:		false" to match "OnChain:\s+true"
        	Test:       	TestQueryTx/verbose/FAULT
```

chain.GetTransaction is looking into mempool also, so we need to check for the
AER instead of transaction.
 2022-10-26 14:24:56 +03:00
Roman Khimov
6af71755c1
Merge pull request #2762 from nspcc-dev/optimize-emit-bool 
emit: optimize Bool GAS cost
 2022-10-26 15:06:05 +07:00
Roman Khimov
4e58bd7411 compiler: use shorter and cheaper sequence to convert to Boolean 2022-10-25 18:20:55 +03:00
Roman Khimov
e9b752cbbb
Merge pull request #2763 from nspcc-dev/wallet-config-doc 
docs: clarify wallet-config format
 2022-10-25 20:38:13 +07:00
Roman Khimov
265221f65a docs: clarify wallet-config format 2022-10-25 13:30:39 +03:00
Roman Khimov
5d43367082 emit: optimize Bool GAS cost 
NOT is 1 byte shorter and 2048 times cheaper than CONVERT. Inspired by
neo-project/neo-vm#493.
 2022-10-25 13:08:33 +03:00
Roman Khimov
e19d867d4e
Merge pull request #2761 from nspcc-dev/fancy-getaddr 
Fancy getaddr
 2022-10-25 16:51:38 +07:00
Roman Khimov
4313b7f0c7
Merge pull request #2760 from nspcc-dev/net42 
network: it is 42
 2022-10-24 21:23:01 +07:00
Roman Khimov
28f54d352a network: do getaddr requests periodically, fix #2745 
Every 1000 blocks seems to be OK for big networks (that only had done some
initial requests previously and then effectively never requested addresses
again because there was a sufficient number of addresses), won't hurt smaller
ones as well (that effectively keep doing this on every connect/disconnect,
peer changes are very rare there, but when they happen we want to have some
quick reaction to these changes).
 2022-10-24 15:10:51 +03:00
Roman Khimov
9efc110058 network: it is 42 
32 is a very good number, but we all know 42 is a better one. And it can even
be proven by tests with higher peaking TPS values.

You may wonder why is it so good? Because we're using packet-switching
networks mostly and a packet is a packet almost irrespectively of how bit it
is. Yet a packet has some maximum possible size (hi, MTU) and this size most
of the time is 1500 (or a little less than that, hi VPN). Subtract IP header
(20 for IPv4 or 40 for IPv6 not counting options), TCP header (another 20) and
Neo message/payload headers (~8 for this case) and we have just a little more
than 1400 bytes for our dear hashes. Which means that in a single packet most
of the time we can have 42-44 of them, maybe 45. Choosing between these
numbers is not hard then.
 2022-10-24 14:44:19 +03:00
Roman Khimov
9d6b18adec network: drop minPoolCount magic constant 
We have AttemptConnPeers that is closely related, the more we have there the
bigger the network supposedly is, so it's much better than magic minPoolCount.
 2022-10-24 14:36:10 +03:00
Roman Khimov
af24051bf5 network: sleep a bit before retrying reconnects 
If Dial() is to exit quickly we can end up in a retry loop eating CPU.
 2022-10-24 14:34:48 +03:00
Roman Khimov
a17d9f80a4
Merge pull request #2759 from nspcc-dev/avoid-db-lookup-in-hasblock 
core: optimize HasBlock check for recent blocks
 2022-10-24 14:33:38 +07:00
Roman Khimov
f42b8e78fc
Merge pull request #2758 from nspcc-dev/check-inflight-tx-invs 
network: check inv against currently processed transactions
 2022-10-24 14:16:33 +07:00
Roman Khimov
52df03ecbc
Merge pull request #2757 from nspcc-dev/batched-getdata 
network: batch getdata replies
 2022-10-24 14:15:55 +07:00
Roman Khimov
0551ddff4e
Merge pull request #2749 from nspcc-dev/rpc-waiter 
rpc: implement transaction awaiting
 2022-10-24 13:08:06 +07:00
Anna Shaleva
3cccc89dac rpc: add Wait wrapper to Notary actor 2022-10-24 06:09:36 +03:00
Anna Shaleva
1a6f1c805c rpc: fix race in TestWSClientEvents 
```
2022-10-21T08:59:45.2219797Z === RUN   TestWSClientEvents/default_ntf_channel
2022-10-21T08:59:45.2219901Z ==================
2022-10-21T08:59:45.2220017Z WARNING: DATA RACE
2022-10-21T08:59:45.2220177Z Write at 0x00c000c82778 by goroutine 371:
2022-10-21T08:59:45.2220580Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.TestWSClientEvents.func2()
2022-10-21T08:59:45.2221112Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient_test.go:171 +0x1c4
2022-10-21T08:59:45.2221244Z   testing.tRunner()
2022-10-21T08:59:45.2221617Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1439 +0x213
2022-10-21T08:59:45.2221759Z   testing.(*T).Run.func1()
2022-10-21T08:59:45.2222124Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x47
2022-10-21T08:59:45.2222138Z
2022-10-21T08:59:45.2222308Z Previous read at 0x00c000c82778 by goroutine 37:
2022-10-21T08:59:45.2222694Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.(*Client).StateRootInHeader()
2022-10-21T08:59:45.2223151Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/rpc.go:1104 +0xb0
2022-10-21T08:59:45.2223482Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.(*WSClient).wsReader()
2022-10-21T08:59:45.2224077Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient.go:210 +0x651
2022-10-21T08:59:45.2224416Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.NewWS.func2()
2022-10-21T08:59:45.2224892Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient.go:149 +0x39
2022-10-21T08:59:45.2224901Z
2022-10-21T08:59:45.2225049Z Goroutine 371 (running) created at:
2022-10-21T08:59:45.2225182Z   testing.(*T).Run()
2022-10-21T08:59:45.2225548Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x724
2022-10-21T08:59:45.2225911Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.TestWSClientEvents()
2022-10-21T08:59:45.2226408Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient_test.go:167 +0x404
2022-10-21T08:59:45.2226539Z   testing.tRunner()
2022-10-21T08:59:45.2226900Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1439 +0x213
2022-10-21T08:59:45.2227042Z   testing.(*T).Run.func1()
2022-10-21T08:59:45.2227398Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x47
2022-10-21T08:59:45.2227406Z
2022-10-21T08:59:45.2227552Z Goroutine 37 (finished) created at:
2022-10-21T08:59:45.2227851Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.NewWS()
2022-10-21T08:59:45.2228327Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient.go:149 +0x6fb
2022-10-21T08:59:45.2228843Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.TestWSClientEvents.func2()
2022-10-21T08:59:45.2229434Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient_test.go:168 +0x131
2022-10-21T08:59:45.2229569Z   testing.tRunner()
2022-10-21T08:59:45.2229930Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1439 +0x213
2022-10-21T08:59:45.2230069Z   testing.(*T).Run.func1()
2022-10-21T08:59:45.2230424Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x47
2022-10-21T08:59:45.2230526Z ==================
2022-10-21T08:59:45.2230703Z     wsclient_test.go:186:
2022-10-21T08:59:45.2230988Z         	Error Trace:	wsclient_test.go:186
2022-10-21T08:59:45.2231209Z         	Error:      	Should be true
2022-10-21T08:59:45.2231536Z         	Test:       	TestWSClientEvents/default_ntf_channel
2022-10-21T08:59:45.2231812Z     testing.go:1312: race detected during execution of test
```
 2022-10-24 06:09:36 +03:00
Anna Shaleva
d2a9e9120d rpc: extend Waiter interface to wait for several txs with context 2022-10-24 06:09:36 +03:00
Anna Shaleva
6b216050f3 rpc: add compat tests for RPC* interfaces and Client implementations 2022-10-24 06:09:36 +03:00
Anna Shaleva
00d44235c1 rpc: add tests for RPC waiters 2022-10-24 06:09:36 +03:00
Anna Shaleva
388112dcaa rpc: mark old WSClient's SubscribeFor* methods as deprecated 2022-10-24 06:09:36 +03:00
Anna Shaleva
5b81cb065f rpc: refactor waiter-related actor code 2022-10-24 06:09:36 +03:00
Anna Shaleva
8e84bb51d5 rpc: add "till" filter to WS block events 2022-10-24 06:09:36 +03:00
Anna Shaleva
345d48d051 rpc: improve WS-based transaction awaiting 
Subscribe only for required aers.
 2022-10-24 06:09:36 +03:00
Anna Shaleva
d7c1f3eac7 rpc: add "container" filter to WS execution notifications 2022-10-24 06:09:36 +03:00
Anna Shaleva
71069b0ed0 rpc: improve WS-enabled transaction awaiting 
Fetch blocks since VUB+1, if block received and we haven't returned the
result yet, then transaction wasn't accepted at all.
 2022-10-24 06:09:36 +03:00
Anna Shaleva
673a495527 rpc: add "since" filter to WS block events 2022-10-24 06:09:36 +03:00
Anna Shaleva
10a0716217 rpc: implement transaction awaiting functionality 
Close #2704.
 2022-10-24 06:09:36 +03:00
Anna Shaleva
6d38e75149 rpc: support multiple WSClient notification receivers 2022-10-24 06:09:36 +03:00
Anna Shaleva
4ce6bc6a66 rpc: adjust comment to Notification value cast 
And explicitly specify field names on Notification creation.
 2022-10-24 06:09:36 +03:00
Roman Khimov
0c3b03617e core: optimize HasBlock check for recent blocks 
When block is being spread through the network we can get a lot of invs with
the same hash. Some more stale nodes may also announce previous or some
earlier block. We can avoid full DB lookup for them and minimize inv handling
time (timeouts in inv handler had happened in #2744).

It doesn't affect tests, just makes node a little less likely to spend some
considerable amount of time in the inv handler.
 2022-10-21 21:21:58 +03:00
Roman Khimov
e26055190e network: check inv against currently processed transactions 
Sometimes we already have it, but it's not yet processed, so we can save on
getdata request. It only affects very high-speed networks like 4-1 scenario
and it doesn't affect it a lot, but still we can do it.
 2022-10-21 21:16:18 +03:00
Roman Khimov
cfb5058018 network: batch getdata replies 
This is not exactly the protocol-level batching as was tried in #1770 and
proposed by neo-project/neo#2365, but it's a TCP-level change in that we now
Write() a set of messages and given that Go sets up TCP sockets with
TCP_NODELAY by default this is a substantial change, we have less packets
generated with the same amount of data. It doesn't change anything on properly
connected networks, but the ones with delays benefit from it a lot.

This also improves queueing because we no longer generate 32 messages to
deliver on transaction's GetData, it's just one stream of bytes with 32
messages inside.

Do the same with GetBlocksByIndex, we can have a lot of messages there too.

But don't forget about potential peer DoS attacks, if a peer is to request a
lot of big blocks we need to flush them before we process the whole set.
 2022-10-21 17:16:32 +03:00
Roman Khimov
bf4636f70a
Merge pull request #2755 from nspcc-dev/improve-networking 
Improve P2P transaction handling
 2022-10-21 16:22:02 +07:00
Roman Khimov
e1b5ac9b81 network: separate tx handling from msg handling 
This allows to naturally scale transaction processing if we have some peer
that is sending a lot of them while others are mostly silent. It also can help
somewhat in the event we have 50 peers that all send transactions. 4+1
scenario benefits a lot from it, while 7+2 slows down a little. Delayed
scenarios don't care.

Surprisingly, this also makes disconnects (#2744) much more rare, 4-node
scenario almost never sees it now. Most probably this is the case where peers
affect each other a lot, single-threaded transaction receiver can be slow
enough to trigger some timeout in getdata handler of its peer (because it
tries to push a number of replies).
 2022-10-21 12:11:24 +03:00
Roman Khimov
09015e06d5
Merge pull request #2754 from nspcc-dev/move-neofs-mainnet-fork-date 
config: reschedule Aspidochelone fork block for NeoFS mainnet
 2022-10-21 16:02:23 +07:00
Roman Khimov
e003b67418 network: reuse inventory hash list for request hashes 
Microoptimization, we can do this because we only use them in handleInvCmd().
 2022-10-21 11:28:40 +03:00
Roman Khimov
831067c29e config: reschedule Aspidochelone fork block for NeoFS mainnet 
We can't make it before 2550000, so move to 2600000. @anatoly-bogatyrev,
please ensure you're using this new height for mainnet NeoFS configurations
next week.
 2022-10-21 10:52:11 +03:00
Roman Khimov
295add052b
Merge pull request #2752 from nspcc-dev/initialize-dbft-with-previous-timestamp 
dbft: rev up, fix context timestamp initialization
 2022-10-20 16:53:08 +07:00
Roman Khimov
493b979b95 dbft: rev up, fix context timestamp initialization 
See nspcc-dev/dbft#64, fixes #2753.
 2022-10-20 12:38:14 +03:00
Roman Khimov
0f625f04f0
Merge pull request #2748 from nspcc-dev/stop-tx-flow 
network/consensus: use new dbft StopTxFlow callback
 2022-10-18 16:29:37 +07:00
Roman Khimov
73ce898e27 network/consensus: use new dbft StopTxFlow callback 
It makes sense in general (further narrowing down the time window when
transactions are processed by consensus thread) and it improves block times a
little too, especially in the 7+2 scenario.

Related to #2744.
 2022-10-18 11:06:20 +03:00
Roman Khimov
903beeb5df
Merge pull request #2750 from nspcc-dev/network-prometheus-metrics 
network: add prometheus histogram with cmd processing time
 2022-10-18 13:07:50 +07:00
Roman Khimov
2791127ee4 network: add prometheus histogram with cmd processing time 
It can be useful to detect some performance issues.
 2022-10-17 22:51:16 +03:00
Roman Khimov
73079745ab
Merge pull request #2746 from nspcc-dev/optimize-tx-callbacks 
network: only call tx callback if we're waiting for transactions
 2022-10-17 16:39:41 +07:00