Commit graph

609 commits

Author SHA1 Message Date
Ekaterina Pavlova
31548dc28f network: fix logging data race in the package
Nested goroutines could cause logging race.

Close #3316

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-03-20 13:12:14 +03:00
Anna Shaleva
1560df6913 *: add nolint comment to deprecated math/rand usages
They are used in tests only.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-03-05 13:54:10 +03:00
Ekaterina Pavlova
5cf0c75744 network: fix server shutdown by waiting for goroutines to finish
s.Shutdown() does not wait for all goroutines of the node server to
finish normally just because the server exits without dependent
goroutines awaiting. Which causes logs to attempt to write after the
test has ended. The consequence of this bug fix is that corresponding
tests are fixed.

Close #2973
Close #2974

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-02-27 15:10:51 +03:00
Ekaterina Pavlova
5bb7c6b715 services: update logs flush after services shutdown
Added sync logs for every service separately to provide the ability to
have a custom logger for each service. This commit makes the code follow
the zap usages rules: `Sync calls the underlying Core's Sync method,
flushing any buffered log entries. Applications should take care to
call Sync before exiting.`

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-02-27 15:10:51 +03:00
Ekaterina Pavlova
9b540770cd network: fix typo
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-02-27 15:10:51 +03:00
Ekaterina Pavlova
775c56e87e network: ensure server is started and shut down only once
Use started atomic.Bool field to ensure that the node server shutdown
procedure is executed only once. Prevent the following panic caused by
server double-shutdown in testing code:
```
--- FAIL: TestServerRegisterPeer (0
.06s)
 panic: closed twice
 goroutine 60 [running]:
 testing.tRunner.func1.2({0x104c40b20, 0x104d0ec90})
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1545 +0x1c8
 testing.tRunner.func1()
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1548 +0x360
 panic({0x104c40b20?, 0x104d0ec90?})
 	/opt/homebrew/opt/go/libexec/src/runtime/panic.go:914 +0x218
 github.com/nspcc-dev/neo-go/pkg/network.(*fakeTransp).Close
 (0x14000159e08?)
 	/Users/ekaterinapavlova/Workplace/neo-go/pkg/network
 	/discovery_test.go:83 +0x54
 github.com/nspcc-dev/neo-go/pkg/network.(*Server).Shutdown
 (0x14000343400)
 	/Users/ekaterinapavlova/Workplace/neo-go/pkg/network/server.go:299
 	 +0x104
 github.com/nspcc-dev/neo-go/pkg/network.startWithCleanup.func1()
 	/Users/ekaterinapavlova/Workplace/neo-go/pkg/network/server_test
 	.go:408 +0x20
 testing.(*common).Cleanup.func1()
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1169 +0x110
 testing.(*common).runCleanup(0x1400032c340, 0x14000159d80?)
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1347 +0xd8
 testing.tRunner.func2()
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1589 +0x2c
 testing.tRunner(0x1400032c340, 0x104d0c5d0)
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1601 +0x114
 created by testing.(*T).Run in goroutine 1
 	/opt/homebrew/opt/go/libexec/src/testing/testing.go:1648 +0x33c
```

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-02-27 15:10:51 +03:00
Ekaterina Pavlova
f8dc5ec44f network: change server Start() behavior
Previously user should Start server in a separate goroutine. Now
separate goroutine is created inside the Start(). For normal server
operation, the caller should wait for Start to finish. Also, fixed
TestTryInitStateSync test which was exiting earlier than logs are
called.

Close #3112

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-02-27 15:10:51 +03:00
Dmitrii Stepanov
6147bf452a tcp_peer: Fix possible goroutine leak
Scenario:
1. Two messages were read from the connection `p.conn`
2. The first message has started to be processed
3. The second message was queued to be added to the channel `p.incoming`
4. Processing of the first message failed with an error
5. TCP peer is closed, but processing of the second message continues

Signed-off-by: Dmitrii Stepanov <dima-stepan@yandex.ru>
2024-02-25 22:38:18 +03:00
Roman Khimov
f9a82ae45f network: add debug for transactions as well
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-01-12 12:18:06 +03:00
Roman Khimov
db2e214f7e network: add debug for notary payloads
Can be handy if things go wrong.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-01-12 12:18:06 +03:00
Roman Khimov
dffd87c6c8
Merge pull request #3171 from nspcc-dev/fix-linter
Fix linter problems
2023-10-20 21:14:41 +03:00
Roman Khimov
c3955f87d1
Merge pull request #3165 from nspcc-dev/fix-win-timeout
Improve timeout of RPC tests on Windows
2023-10-20 19:07:24 +03:00
Anna Shaleva
6e0af5ac90 network: fix linter warning
Fix the following linter warning:
```
dot-imports: should not use dot imports (revive)
```

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-20 17:23:35 +03:00
Anna Shaleva
c93658b1c1 Partially revert "*: use localhost address instead of 127.0.0.1, fix #2575"
This partially reverts commit c26a962b55 for testing
chains configurations.

Ref. #2975, although this commit doesn't close it. This commit is an attempt to
enforce IPv4 for our test clients to avoid problem described in the issue.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-20 17:17:37 +03:00
Anna Shaleva
1815bc8a32 *: update Stringers wrt fresh Stringer version
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-19 11:33:07 +03:00
Anna Shaleva
3aa858a69f *: drop go 1.18 support in doc and build targets
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-12 17:59:42 +03:00
Anna Shaleva
dc3d1300dd *: replace go.uber.org/atomic with sync/atomic
Use sync/atomic everywhere and exclude go.uber.org/atomic from go.mod.
Close #2626.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-12 17:59:34 +03:00
Roman Khimov
6865848ca7 config: drop deprecated P2P configuration
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
a59afa8ea3 config: drop obsolete SecondsPerBlock protocol configuration
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Anna Shaleva
f3c1283ac6 *: move NVB and Conflicts attributes out of extensions
They're a part of the regular protocol now.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-04 16:39:44 +03:00
Anna Shaleva
966111f4a8 network: forbid Notary contract to be a sender of main transaction
This prevents the possible attack on notary request sender when
malicious partie is allowed to send notary request with main transaction
being someone else's fallback.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-07-20 10:56:48 +03:00
Anna Shaleva
649b9ac7b0 network: add neogo_version metric, deprecate serv_node_version
Close #2999.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-11 13:48:38 +03:00
Roman Khimov
433275265f *: use require.ErrorIs instead of require.True(t, error.Is())
This is just a much better way to do the same thing. Inspired by
nspcc-dev/neofs-sdk-go#407.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-05-04 17:03:47 +03:00
Roman Khimov
0a160ee93b *: use CompareAndSwap instead of CAS for atomics
go.uber.org/atomic deprecated CAS methods in version 1.10 (that introduced
CompareAndSwap), so we need to fix it.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-04-28 12:07:19 +03:00
Anna Shaleva
3a71aafc43 core: distinguish notarypool/mempool metrics
Move them to the core/network packages, close #2950. The name of
mempool's unsorted transactions metrics has been changed along the
way to match the core's metrics naming convention.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 18:40:19 +03:00
Anna Shaleva
55ab38ed81 network: do not use error channel to start network srv
It's obsolete thing, we have looger and it perfectly suits our needs.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 13:21:34 +03:00
Roman Khimov
c053f1a4af
Merge pull request #2957 from nspcc-dev/rm-go-17
*: drop go 1.17 support
2023-04-04 15:25:14 +03:00
Anna Shaleva
6b21ad9922 *: replace interface{} with any keyword
Everywhere including examples, external interop APIs, bindings generators
code and in other valuable places. A couple of `interface{}` usages are
intentionally left in the CHANGELOG.md, documentation and tests.
2023-04-04 13:22:42 +03:00
Anna Shaleva
83545b8451 network: fix Address test failing on Windows/macOS with go 1.20
See the trace:
```
2023-03-31T07:46:21.1886260Z === RUN   TestEncodeDecodeAddress
2023-03-31T07:46:21.1886420Z     address_test.go:30:
2023-03-31T07:46:21.1887000Z         	Error Trace:	/Users/runner/work/neo-go/neo-go/pkg/network/payload/address_test.go:30
2023-03-31T07:46:21.1887180Z         	Error:      	Not equal:
2023-03-31T07:46:21.1887580Z         	            	expected: net.IP{0x7f, 0x0, 0x0, 0x1}
2023-03-31T07:46:21.1888290Z         	            	actual  : net.IP{0x7f, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
2023-03-31T07:46:21.1888450Z
2023-03-31T07:46:21.1888640Z         	            	Diff:
2023-03-31T07:46:21.1888960Z         	            	--- Expected
2023-03-31T07:46:21.1889180Z         	            	+++ Actual
2023-03-31T07:46:21.1889510Z         	            	@@ -1,3 +1,3 @@
2023-03-31T07:46:21.1889870Z         	            	-(net.IP) (len=4) {
2023-03-31T07:46:21.1890530Z         	            	- 00000000  7f 00 00 01                                       |....|
2023-03-31T07:46:21.1891140Z         	            	+(net.IP) (len=16) {
2023-03-31T07:46:21.1891780Z         	            	+ 00000000  7f 00 00 01 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2023-03-31T07:46:21.1891970Z         	            	 }
2023-03-31T07:46:21.1892510Z         	Test:       	TestEncodeDecodeAddress
2023-03-31T07:46:21.1892770Z --- FAIL: TestEncodeDecodeAddress (0.00s)
```

I'm still not sure what's the original root of this problem, there's nothing
about resolving scheme changes in https://go.dev/doc/go1.20, but it didn't
happen earlier and it works as expected in Ubuntu.
2023-04-03 09:43:14 +03:00
ZhangTao1596
d36df15878 ci: fix lint issues (fix #2948) 2023-03-29 11:19:23 +08:00
Anna Shaleva
0cbef58b3c consensus: enqueue newly created blocks
Do not add them directly to chain, it will be done by the block queue
manager. Close https://github.com/nspcc-dev/neo-go/issues/2923. However,
this commit is not valid without
https://github.com/roman-khimov/dbft/pull/4.
It's the neo-go's duty to initialize consensus after subsequent block
addition; the dBFT itself must wait for the neo-go to complete the block
addition and notify the dBFT, so that it can initialize at 0-th view to
collect the next block.
2023-03-15 17:37:47 +03:00
Anna Shaleva
04d0b45ceb network: move blockqueue to a separate package 2023-03-15 17:37:47 +03:00
Anna Shaleva
91a77c25a2 network: refactor blockqueuer interface
Remove unused argument.
2023-03-15 17:37:47 +03:00
Roman Khimov
4f708c037d network: drain send queues on peer disconnection
Fix potential memory leak with a lot of connected clients that keep requesting
things from node and then disconnect.
2023-02-21 16:19:06 +03:00
Anna Shaleva
da757fa387 network: fix grammar typo in the error message 2023-02-20 11:08:07 +03:00
Anna Shaleva
28927228f0 *: adjust subscription-related doc
Add a warning about received events modification where applicable.
2023-01-17 17:11:19 +03:00
Anna Shaleva
9b364aa7ee network: do not allow to request invalid block count
The problem is in peer disconnection due to invalid GetBlockByIndex
payload (the logs are from some patched neo-go version):
```
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        INFO        new peer connected        {"addr": "10.78.69.115:50846", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        WARN        peer disconnected        {"addr": "10.78.69.115:50846", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        INFO        started protocol        {"addr": "10.78.69.115:50846", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.491Z        INFO        new peer connected        {"addr": "10.78.69.115:50856", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        WARN        peer disconnected        {"addr": "10.78.69.115:50856", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        INFO        started protocol        {"addr": "10.78.69.115:50856", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        INFO        new peer connected        {"addr": "10.78.69.115:50858", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.493Z        INFO        started protocol        {"addr": "10.78.69.115:50858", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.493Z        WARN        peer disconnected        {"addr": "10.78.69.115:50858", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        INFO        new peer connected        {"addr": "10.78.69.115:50874", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        INFO        started protocol        {"addr": "10.78.69.115:50874", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        WARN        peer disconnected        {"addr": "10.78.69.115:50874", "error": "invalid block count", "peerCount": 2}
```

GetBlockByIndex payload can't be decoded, and the only possible cause
is zero (or <-1, but it's probably not the case) block count requested.

Error is improved as far.
2022-12-28 13:04:56 +03:00
Anna Shaleva
c0a453a53b network: adjust requestBlocs logic
If the lastQueued block index is the same as the one we'd like to
request in payload, then we need to increment the payload's count.
2022-12-28 12:50:30 +03:00
Roman Khimov
e79dec15f9 *: use zap.Stringer instead of zap.String where it can be used
It's a bit more efficient in case we're not logging the message (mostly for
debug), makes the code somewhat simpler as well.
2022-12-13 12:44:54 +03:00
Roman Khimov
7589733017 config: add a special Blockchain type to configure Blockchain
And include some node-specific configurations there with backwards
compatibility. Note that in the future we'll remove Ledger's
fields from the ProtocolConfiguration and it'll be possible to access them in
Blockchain directly (not via .Ledger).

The other option tried was using two configuration types separately, but that
incurs more changes to the codebase, single structure that behaves almost like
the old one is better for backwards compatibility.

Fixes #2676.
2022-12-07 17:35:53 +03:00
Anna Shaleva
82221b0ca7 *: fix Neo and NeoGo misuses 2022-12-07 17:29:09 +03:00
Anna Shaleva
54c2aa8582 config: move P2P options to a separate config section
And convert time-related settings to a Duration format along the way.
2022-12-07 13:06:05 +03:00
Anna Shaleva
9cf6cc61f4 network: allow multiple bind addresses for server
And replace Transporter.Address() with Transporter.HostPort() along the way.
2022-12-07 13:06:03 +03:00
Roman Khimov
c2adbf768b config: add TimePerBlock to replace SecondsPerBlock
It's more generic and convenient than MillisecondsPerBlock. This setting is
made in backwards-compatible fashion, but it'll override SecondsPerBlock if
both are used. Configurations are specifically not changed here, it's
important to check compatibility.

Fixes #2675.
2022-12-02 19:52:14 +03:00
Roman Khimov
0ad6e295ea core: make GetHeaderHash accept uint32
It should've always been this way because block indexes are uint32.
2022-11-25 14:30:51 +03:00
Roman Khimov
b8c09f509f network: add random slight delay to connection attempts
Small (especially dockerized/virtualized) networks often start all nodes at
ones and then we see a lot of connection flapping in the log. This happens
because nodes try to connect to each other simultaneously, establish two
connections, then each one finds a duplicate and drops it, but this can be
different duplicate connections on other sides, so they retry and it all
happens for some time. Eventually everything settles, but we have a lot of
garbage in the log and a lot of useless attempts.

This random waiting timeout doesn't change the logic much, adds a minimal
delay, but increases chances for both nodes to establish a proper single
connection on both sides to only then see another one and drop it on both
sides as well. It leads to almost no flapping in small networks, doesn't
affect much bigger ones. The delay is close to unnoticeable especially if
there is something in the DB for node to process during startup.
2022-11-17 18:42:43 +03:00
Roman Khimov
075a54192c network: don't try too many connections
Consider mainnet, it has an AttemptConnPeers of 20, so may already have 3
peers and request 20 more, then have 4th connected and attemtp 20 more again,
this leads to a huge number of connections easily.
2022-11-17 18:03:04 +03:00
Roman Khimov
6bce973ac2 network: drop duplicationg check from handleAddrCmd()
It was relevant with the queue-based discoverer, now it's not, discoverer
handles this internally.
2022-11-17 17:42:36 +03:00
Roman Khimov
1c7487b8e4 network: add a timer to check for peers
Consider initial connection phase for public networks:
 * simultaneous connections to seeds
 * very quick handshakes
 * got five handshaked peers and some getaddr requests sent
 * but addr replies won't trigger new connections
 * so we can stay with just five connections until any of them breaks or a
   (long) address checking timer fires

This new timers solves the problem, it's adaptive at the same time. If we have
enough peers we won't be waking up often.
2022-11-17 17:32:05 +03:00
Roman Khimov
23f118a1a9 network: rework discoverer/server interaction
* treat connected/handshaked peers separately in the discoverer, save
   "original" address for connected ones, it can be a name instead of IP and
   it's important to keep it to avoid reconnections
 * store name->IP mapping for seeds if and when they're connected to avoid
   reconnections
 * block seed if it's detected to be our own node (which is often the case for
   small private networks)
 * add an event for handshaked peers in the server, connected but
   non-handshaked ones are not really helpful for MinPeers or GetAddr logic

Fixes #2796.
2022-11-17 17:07:19 +03:00