Commit graph

5632 commits

Author SHA1 Message Date
Ekaterina Pavlova
f5ea79d649 services: refactor NeoFS client naming
There is just NeoFS client/conn.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-16 11:12:46 +03:00
Ekaterina Pavlova
e741053f0c services: drop check of Unimplemented gRPC status in neofs helper
NeoFS oracle can miss problem NeoFS server on dial. So `Unimplemented`
gRPC status should not be ignored.

Close #3757

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-16 11:12:43 +03:00
Ekaterina Pavlova
9e3f75e977 *: update neofs-sdk-go
Update to the version without pool panics
https://github.com/nspcc-dev/neofs-sdk-go/pull/643.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-13 17:07:52 +03:00
Anna Shaleva
9834b83cf0
Merge pull request #3750 from nspcc-dev/removeuntraceableheaders
RemoveUntraceableHeaders
2024-12-13 16:30:31 +03:00
Roman Khimov
90b6a42331 network: make GetHeaders best-effort
Be a nice neighbor, try to reply with whatever headers we have, don't fail
completely because of a single missing header.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
9599fba24f core: fix removing old transfer data with RemoveUntraceableHeaders
Transfer data is timestamp-based, previously it always had and used headers,
no we can go via a small cache (we don't want it to grow or be stored forever).
Otherwise it's unable to do the job:

    2024-12-13T12:55:15.056+0300    ERROR   failed to find block header for transfer GC     {"time": "19.066µs", "error": "key not found"}

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
7d89a53043 core: drop redundant tgtBlock normalization
It's there since 423c7883b8, but looks like it
never changed anything, the same thing is done six lines above and tgtBlock is
not changed since.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
c53b0645bb core: extend NewBlockchain coverage a bit
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
c7f5f173ae core: introduce RemoveUntraceableHeaders
With blocks available from NeoFS we can drop them from the local DB.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Ekaterina Pavlova
65bdc82da8 *: move constant and NeoFS related code into separate package
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-13 11:43:19 +03:00
Ekaterina Pavlova
c79ffa967f services: add default values for NeoFSBlockFetcher configuration
The minimum sufficient configuration is Addresses and ContainerID,
example:
```
  NeoFSBlockFetcher:
    Enabled: true
    Addresses:
      - st1.storage.fs.neo.org:8080
      - st2.storage.fs.neo.org:8080
      - st3.storage.fs.neo.org:8080
      - st4.storage.fs.neo.org:8080
    ContainerID: "87JRc7vyWcjW8uS32LMoLTAj4ckCzFZWfKbacjU3sAob"
```

Close #3718

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-13 11:39:22 +03:00
Anna Shaleva
6d1eea307b
Merge pull request #3696 from nspcc-dev/basic-chain-tests
basicchain: use UnitTestNet default config for generation
2024-12-12 10:51:42 +03:00
Ekaterina Pavlova
e993c1bdac basicchain: use UnitTestNet default config
It's important to have the same chain configuration for all tests.
Otherwise, a mismatched hardfork configuration is used to dump/restore
the basic chain.

Close #3681

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-11 21:42:00 +03:00
Anna Shaleva
4fa5fdc39c
Merge pull request #3715 from nspcc-dev/sub-test
rpcsrv: fix `TestFilteredSubscriptions`
2024-12-11 20:50:52 +03:00
Ekaterina Pavlova
0bd378770d rpcsrv: fix TestFilteredSubscriptions
Close #3693

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-11 14:18:23 +03:00
Anna Shaleva
827acfca23 services: fix timer draining
Ref. #3736.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-12-09 12:11:01 +03:00
Ekaterina Pavlova
3010324c4f services: fix defer function in retry of NeoFSBlockFetcher
Drain the channel only if it has pending events. This avoids trying to
read from an already-empty channel.

Close #3734

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-08 00:04:14 +03:00
Roman Khimov
703bf6c458 network: take into account good known peers when thinking of GetAddr
They will be returned to pool when disconnected anyway. On a smaller network
this can make a difference because there are not a lot of addresses in the
pool usually.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-06 15:31:33 +03:00
Anna Shaleva
2bc41dbe30
Merge pull request #3724 from nspcc-dev/hf-latest-stable
Latest stable hardforks
2024-12-06 13:18:03 +03:00
Anna Shaleva
a68856c27c
Merge pull request #3728 from nspcc-dev/blockfetcher-close
Fix blockfetcher closing
2024-12-06 13:07:19 +03:00
Roman Khimov
a164db92cc config: replace LatestHardfork() with HFLatestStable
Function doesn't make much sense here. The change is rather trivial and this
is not expected to be used by external code, so no deprecation.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-06 12:24:00 +03:00
Roman Khimov
981ae10091 config: declare and use the latest stable hardfork
Differentiate released and stable ones from test-only not-yet-ready and such.
Enabled only stable ones by default to avoid surprises in private networks
when some beta hardfork is made available with some node release.

Fixes #3719.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-06 12:23:46 +03:00
Roman Khimov
109319d220 network: avoid accidental double-close of blockFetcherFin channel
It's not supposed to happen, but if blockfetcher goes wild and calls this
twice it can ruin the system, so better safe than sorry.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-06 11:39:44 +03:00
Roman Khimov
c812150d83 network: avoid looping forever
blockFetcherFin is closed by the block fetcher, so it will forever return
something in this loop making it an infinite loop, creating useless load
and affecting normal node operation.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-06 11:35:54 +03:00
Anna Shaleva
49267f3412
Merge pull request #3726 from nspcc-dev/fix-unexpected-empty-payload
network: fix "unexpected empty payload: CMDVersion"
2024-12-05 23:34:05 +03:00
Anna Shaleva
1aed0faeec
Merge pull request #3727 from nspcc-dev/network-optimaln
network: optimize optimalN for small networks
2024-12-05 23:26:41 +03:00
Roman Khimov
d69f8ebbab network: optimize optimalN for small networks
Consider 10-node network. FanOut is 6 for it. optimalN is 12. But it's a
10-node network, you can't have more than 9 peers there. So adjust the formula
for netSize-1.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-05 22:31:19 +03:00
Roman Khimov
121307d349 network: fix "unexpected empty payload: CMDVersion"
We can have _a lot_ of these in the log, but the only reason they happen is
because we're trying to interpret length before checking for reader error,
CMDVersion is just 0.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-05 18:34:55 +03:00
Anna Shaleva
2244d2ad75 blockfetcher: decrease default pool deal timeout
Dealing is fast, we don't need 10 minutes to check that connection can
(or can't) be established. Improve documentation along the way.

Close #3721.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-12-04 16:34:09 +03:00
Anna Shaleva
6a7d2bc250
Merge pull request #3712 from nspcc-dev/move-metrics
*: move metric `neogo_version` out of pkg/network
2024-12-02 10:53:54 +03:00
Roman Khimov
7b82c759ce
Merge pull request #3711 from nspcc-dev/fetch-dbft
go.mod: upgrade dBFT version
2024-11-29 18:39:56 +03:00
Anna Shaleva
9778fd88fb *: migrate onto simplified dbft.PublicKey interface
Adopt https://github.com/nspcc-dev/dbft/pull/137.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-11-29 18:33:19 +03:00
Anna Shaleva
a5cf1635d8 go.mod: fetch dbft v0.3.1
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-11-29 18:33:03 +03:00
Ekaterina Pavlova
ebb71497a5 *: move metric neogo_version out of pkg/network
Close #3682

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 16:58:11 +03:00
Anna Shaleva
dc2d22110f
Merge pull request #3709 from nspcc-dev/compile
neotest: extend cached compiled contract identifier in CompileFile
2024-11-29 14:46:19 +03:00
Ekaterina Pavlova
3f95476ce3 neotest: extend cached compiled contract identifier in CompileFile
Close #3542

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 14:43:37 +03:00
Ekaterina Pavlova
718ff9aa0a blockfetcher: add check on IndexFileSize of index files
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:37:50 +03:00
Ekaterina Pavlova
14c980a685 network: refactor return value from NeoFSBlockFetcher service
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:37:50 +03:00
Ekaterina Pavlova
119ca27994 blockfetcher: fix block enqueue logic
Previously, the `blockQueuer` routine, which enqueues blocks into
`bQueue`, could be blocked on enqueing newer blocks if older blocks
downloading is delayed by NeoFS.

The `blocksCh` channel, acting as a queue ordered by download speed,
conflicted with the BQueue requirement for strict sequential enqueuing
(expecting an exact range of blocks), resulting in a deadlock that
stalled the process.

Before with default config settings:
```
2024-11-27T17:12:19.348+0300	INFO	persisted to disk	{"blocks":
 0, "keys": 116, "headerHeight": 0, "blockHeight": 0, "took": "15
 .509083ms"}
2024-11-27T17:19:39.574+0300	INFO	persisted to disk	{"blocks":
 16, "keys": 11107, "headerHeight": 216768, "blockHeight": 216768,
 "took": "62.762041ms"}
```
Average block persistence speed: 492.40 block/s
Average blocks number for each persist log: 584.28

After:

```
2024-11-27T17:29:03.362+0300	INFO	persisted to disk	{"blocks":
 0, "keys": 116, "headerHeight": 0, "blockHeight": 0, "took": "19
 .485084ms"}
2024-11-27T17:34:58.527+0300	INFO	persisted to disk	{"blocks":
 16, "keys": 11109, "headerHeight": 216770, "blockHeight": 216769,
 "took": "52.43925ms"}
```
Average block persistence speed: 610.33 block/s
Average blocks number for each persist log: 752.61

Close #3699

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:37:50 +03:00
Ekaterina Pavlova
c1ce6904c4 network: discard bFetcherQueue earlier
Discard `bFetcherQueue` to avoid persisting
new blocks during termination before service Shutdown.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:37:50 +03:00
Ekaterina Pavlova
df05cd2858 blockfetcher: change shutdown logic
Move the `isActive` check earlier to enable faster shutdown and prevent
the service from hanging.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:37:50 +03:00
Ekaterina Pavlova
8216f538c3 blockfetcher: check container and network Magic
Close #3644

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:37:11 +03:00
Ekaterina Pavlova
92ff8409a9 blockfetcher: add retry for GET and SEARCH operations
Close #3564

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-29 13:34:40 +03:00
Ekaterina Pavlova
9fa07d8d6d blockfetcher: fix invalid wallet test
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-28 16:14:25 +03:00
Ekaterina Pavlova
c321eed8ee blockfetcher: use pool for GET and SEARCH operations
Use NeoFS storage nodes pool during blocks fetching to spread the load.

Close #3568

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-28 16:14:25 +03:00
Ekaterina Pavlova
04516e7d26 neofs: add pool support for NeoFS operations
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-28 16:13:36 +03:00
Anna Shaleva
e89f9fe2a4
Merge pull request #3704 from nspcc-dev/interop-md-fixes
Native contract manifest build fixes
2024-11-26 17:03:31 +03:00
Anna Shaleva
f98169a762
Merge pull request #3689 from nspcc-dev/feat/filter-events-by-parameters
ws: allow filtering notification by parameters
2024-11-26 10:12:40 +03:00
Pavel Karpy
97bd73b38c rpcsrv/tests: simplify tests with parsed test contract hash
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2024-11-26 00:26:06 +03:00
Pavel Karpy
0ba16fe580 ws: allow filtering notification by parameters
`Any` type with nil/null value is treated as a parameter filter that allows
any notification value. Not more than 16 filter parameters are allowed.
Closes #3624.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2024-11-26 00:26:04 +03:00