Commit graph

5666 commits

Author SHA1 Message Date
Roman Khimov
8d728b4ec1
Merge pull request #3789 from nspcc-dev/headers
Headers fetching via NeoFS BlockFetcher service
2025-02-06 17:05:20 +03:00
Ekaterina Pavlova
1ad62ef5f5 blockfetcher: add headerSizeMap to GetRange headers accordingly
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2025-02-06 16:13:01 +03:00
Ekaterina Pavlova
91b60ea7d3 network: integrate state sync module with blockfetcher
Close #3574

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2025-02-06 16:12:55 +03:00
Roman Khimov
3fa02d062b
Merge pull request #3804 from nspcc-dev/toscparameter
rpcbinding: expand struct generator with ToSCParameter
2025-02-04 17:52:41 +03:00
Ekaterina Pavlova
de50d0be49 bqueue: make queue generic
The queue can operate with any item that implements proper interface.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2025-02-04 16:48:03 +03:00
Roman Khimov
e70b1063a6 rpcbinding: expand struct generator with ToSCParameter
Follow-up to 00e22b9751, while
stackitem.Convertible is good for vm/emit we still need
smartcontract.Convertible for invokers.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-02-04 12:10:26 +03:00
Ekaterina Pavlova
267d7dca78 zkpbinding: update interop
This place was not updated with update_deps.sh.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2025-02-03 22:00:37 +03:00
Ekaterina Pavlova
83a02fc0c0 zkpbinding: remove range usage from smartcontract
Reverting a part of 1b83dc2, because ranging over integers is not
supported by smart contract compiler, ref. #3525.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2025-02-03 21:56:54 +03:00
Ekaterina Pavlova
d02e2b629d *: neo-vm submodule update dependency
Close #3204

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2025-02-03 12:09:02 +03:00
Roman Khimov
8ba2bbf87c rpcbindings: fix missing return item
... not enough return values
        have (error)
        want (stackitem.Item, error)

An omission of 00e22b9751.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-26 23:47:36 +03:00
Anna Shaleva
7ba8048530
Merge pull request #3569 from ixje/applog-invocations
*: add invocations to applicationlog
2025-01-24 18:42:53 +03:00
ixje
4b2ee9a424
state, rpc: add invocations to applicationlog
Add setting to include smart contract invocations data to the applicationlog.
Original issue at https://github.com/neo-project/neo/issues/3386

Signed-off-by: ixje <erik@coz.io>
2025-01-24 15:44:10 +01:00
Roman Khimov
00e22b9751 rpcbindings: produce ToStackItem companion method
We need it for structs to be usable as function parameters, otherwise
they're not implementing stackitam.Convertible and they fail at transaction
script creation phase:

    2025/01/22 20:31:26 bootstrap error: could not invoke method (addNode): test invocation failed: unsupported operation: *netmap.NetmapNode2 type

Related to https://github.com/nspcc-dev/neofs-node/pull/3088.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-24 17:07:20 +03:00
Roman Khimov
4da753f822 stackitem: split TryMake and Make
Sometimes we may need something that doesn't panic.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-24 17:07:20 +03:00
Anna Shaleva
257fa7c6c9 native: add NEP-27 to Notary's supported standards
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2025-01-21 19:41:45 +03:00
Anna Shaleva
b2322a804a native: add NEP-27 to Neo's supported standards
Port https://github.com/neo-project/neo/pull/3643.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2025-01-21 19:41:43 +03:00
Anna Shaleva
fcd83e9fd7 smartcontract: support NEP-26 and NEP-27
A replacement of NEP11Payable and NEP17Payable. The essence is the same,
but these standards have official name since
https://github.com/neo-project/proposals/pull/169.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2025-01-21 18:14:55 +03:00
Roman Khimov
9513780c45 core: adjust in-memory processed set dynamically
Instead of tick-tocking with sync/async and having an unpredictable data
set we can just try to check for the real amount of keys that be processed
by the underlying DB. Can't be perfect, but still this adds some hard
limit to the amount of in-memory data. It's also adaptive, slower machines
will keep less and faster machines will keep more.

This gives almost perfect 4s cycles for mainnet BoltDB with no tail cutting,
it makes zero sense to process more blocks since we're clearly DB-bound:

2025-01-15T11:35:00.567+0300    INFO    persisted to disk       {"blocks": 1469, "keys": 40579, "headerHeight": 5438141, "blockHeight": 5438140, "velocity": 9912, "took": "4.378939648s"}
2025-01-15T11:35:04.699+0300    INFO    persisted to disk       {"blocks": 1060, "keys": 39976, "headerHeight": 5439201, "blockHeight": 5439200, "velocity": 9888, "took": "4.131985438s"}
2025-01-15T11:35:08.752+0300    INFO    persisted to disk       {"blocks": 1508, "keys": 39658, "headerHeight": 5440709, "blockHeight": 5440708, "velocity": 9877, "took": "4.052347569s"}
2025-01-15T11:35:12.807+0300    INFO    persisted to disk       {"blocks": 1645, "keys": 39565, "headerHeight": 5442354, "blockHeight": 5442353, "velocity": 9864, "took": "4.05547743s"}
2025-01-15T11:35:17.011+0300    INFO    persisted to disk       {"blocks": 1472, "keys": 39519, "headerHeight": 5443826, "blockHeight": 5443825, "velocity": 9817, "took": "4.203258142s"}
2025-01-15T11:35:21.089+0300    INFO    persisted to disk       {"blocks": 1345, "keys": 39529, "headerHeight": 5445171, "blockHeight": 5445170, "velocity": 9804, "took": "4.078297579s"}
2025-01-15T11:35:25.090+0300    INFO    persisted to disk       {"blocks": 1054, "keys": 39326, "headerHeight": 5446225, "blockHeight": 5446224, "velocity": 9806, "took": "4.000524899s"}
2025-01-15T11:35:30.372+0300    INFO    persisted to disk       {"blocks": 1239, "keys": 39349, "headerHeight": 5447464, "blockHeight": 5447463, "velocity": 9744, "took": "4.281444939s"}

2× can be considered, but this calculation isn't perfect for low number of
keys, so somewhat bigger tolerance is preferable for now. Overall it's not
degrading performance, my mainnet/bolt run was even 8% better with this.

Fixes #3249, we don't need any option this way.

Fixes #3783 as well, it no longer OOMs in that scenario. It however can OOM in
case of big GarbageCollectionPeriod (like 400K), but this can't be solved easily.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-15 22:08:08 +03:00
Roman Khimov
c14492e8c8 core: raise severity of persistence failure message
It is very critical in fact.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-14 12:46:01 +03:00
Roman Khimov
70aaeb06ad core: fix old transfer data deletion in RUB-only configuration
RemoveUntraceableBlocks without RemoveUntraceableHeaders is still a valid
configuration and removeOldTransfers() only checks for gcBlockTimes now
for timestamps, so they should always be added. This fixes

    2025-01-13T23:28:57.340+0300    ERROR   failed to get block timestamp transfer GC       {"time": "1.162µs", "index": 20000}

for RemoveUntraceableBlocks-only configurations.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-14 12:43:15 +03:00
Roman Khimov
ba0ca6a4ab core: perform synchronized persist if GC took some time
The intention here is to reduce the amount of in-flight changes and prevent
OOM. It doesn't matter what we're doing, persisting or collecting garbage,
what matters is that we're behind the schedule of regular persist cycle.

Refs. #3783.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-13 23:39:48 +03:00
Roman Khimov
1f83f472c7 core: swap transfer and MPT GC
It doesn't change anything logically, but transfer GC needs to have current
block timestamp for its logic and if we're delaying it with MPT GC it can
more often fail to obtain it:

    2025-01-13T16:15:18.311+0300    ERROR   failed to get block timestamp transfer GC       {"time": "1.022µs", "index": 20000}

It's not critical, this garbage can still be collected on the next run, but
we better avoid this anyway.

Refs. #3783.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2025-01-13 23:37:21 +03:00
Roman Khimov
82b400700f capability: add ArchivalNode, fix #3776
This will be used to differentiate full history nodes from state only. Currently
it's just a definition, we can not use it safely on current networks because
nodes will refuse this Version, so actual code using the attribute will be
added in newer versions.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-28 10:51:40 +03:00
Roman Khimov
4720727bf5 capability: allow to decode unknown capabilities
It's more symmetric than the C# counterpart, but the essence is the same: we
can accept capabilities we can't interpret. Refs. https://github.com/neo-project/neo/pull/3639

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-28 10:51:40 +03:00
Roman Khimov
47b679341b capability: specify reserved type range
Follow https://github.com/neo-project/neo/pull/3639.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-28 10:51:40 +03:00
Roman Khimov
e0338b7ed0 capability: improve FullNode documentation
That's the way it's defined and used in C#, see https://github.com/neo-project/neo/pull/3639 also.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-27 12:14:48 +03:00
Roman Khimov
38b9b13098 blockfetcher: make UT work with AIO started
Connection succeeds when AIO is running, some more obscure port is less likely
to cause this problem.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-25 10:36:38 +03:00
Roman Khimov
4d45be8434 native: add candidate registration via onNEP17Payment
Solves two problems:
 * inability to estimate GAS needed for registerCandidate in a regular way
   because of its very high fee (more than what normal RPC servers allow)
 * inability to have MaxBlockSystemFee lower than the registration price
   which is very high on its own (more than practically possible to execute)

See https://github.com/neo-project/neo/issues/3552.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-25 10:36:38 +03:00
Roman Khimov
ab5128fb48 config: enable Echidna for tests
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-24 18:01:12 +03:00
Anna Shaleva
dc747ce3b7
Merge pull request #3761 from nspcc-dev/designate
native: add Echidna changes
2024-12-24 10:36:36 +03:00
Ekaterina Pavlova
80e18222bc native: add callflag.AllowNotify to some methods to HFEchidna
Close #3702

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-23 15:54:56 +01:00
Ekaterina Pavlova
b096e68428 native: add Base64URL support to the StdLib starting from HFEchidna
Close #3550

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-23 15:54:56 +01:00
Ekaterina Pavlova
dc68e39811 native: add Designation event extension starting from HFEchidna
Close #3549

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-23 15:54:54 +01:00
Chubru
0b21d4caf6
neotest: Add 'ReadNEF' method
Signed-off-by: Chubru <chubru.alexander@gmail.com>
2024-12-20 16:03:03 +03:00
Ekaterina Pavlova
f5ea79d649 services: refactor NeoFS client naming
There is just NeoFS client/conn.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-16 11:12:46 +03:00
Ekaterina Pavlova
e741053f0c services: drop check of Unimplemented gRPC status in neofs helper
NeoFS oracle can miss problem NeoFS server on dial. So `Unimplemented`
gRPC status should not be ignored.

Close #3757

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-16 11:12:43 +03:00
Ekaterina Pavlova
9e3f75e977 *: update neofs-sdk-go
Update to the version without pool panics
https://github.com/nspcc-dev/neofs-sdk-go/pull/643.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-13 17:07:52 +03:00
Anna Shaleva
9834b83cf0
Merge pull request #3750 from nspcc-dev/removeuntraceableheaders
RemoveUntraceableHeaders
2024-12-13 16:30:31 +03:00
Roman Khimov
90b6a42331 network: make GetHeaders best-effort
Be a nice neighbor, try to reply with whatever headers we have, don't fail
completely because of a single missing header.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
9599fba24f core: fix removing old transfer data with RemoveUntraceableHeaders
Transfer data is timestamp-based, previously it always had and used headers,
no we can go via a small cache (we don't want it to grow or be stored forever).
Otherwise it's unable to do the job:

    2024-12-13T12:55:15.056+0300    ERROR   failed to find block header for transfer GC     {"time": "19.066µs", "error": "key not found"}

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
7d89a53043 core: drop redundant tgtBlock normalization
It's there since 423c7883b8, but looks like it
never changed anything, the same thing is done six lines above and tgtBlock is
not changed since.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
c53b0645bb core: extend NewBlockchain coverage a bit
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Roman Khimov
c7f5f173ae core: introduce RemoveUntraceableHeaders
With blocks available from NeoFS we can drop them from the local DB.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-12-13 16:09:00 +03:00
Ekaterina Pavlova
65bdc82da8 *: move constant and NeoFS related code into separate package
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-13 11:43:19 +03:00
Ekaterina Pavlova
c79ffa967f services: add default values for NeoFSBlockFetcher configuration
The minimum sufficient configuration is Addresses and ContainerID,
example:
```
  NeoFSBlockFetcher:
    Enabled: true
    Addresses:
      - st1.storage.fs.neo.org:8080
      - st2.storage.fs.neo.org:8080
      - st3.storage.fs.neo.org:8080
      - st4.storage.fs.neo.org:8080
    ContainerID: "87JRc7vyWcjW8uS32LMoLTAj4ckCzFZWfKbacjU3sAob"
```

Close #3718

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-13 11:39:22 +03:00
Anna Shaleva
6d1eea307b
Merge pull request #3696 from nspcc-dev/basic-chain-tests
basicchain: use UnitTestNet default config for generation
2024-12-12 10:51:42 +03:00
Ekaterina Pavlova
e993c1bdac basicchain: use UnitTestNet default config
It's important to have the same chain configuration for all tests.
Otherwise, a mismatched hardfork configuration is used to dump/restore
the basic chain.

Close #3681

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-11 21:42:00 +03:00
Anna Shaleva
4fa5fdc39c
Merge pull request #3715 from nspcc-dev/sub-test
rpcsrv: fix `TestFilteredSubscriptions`
2024-12-11 20:50:52 +03:00
Ekaterina Pavlova
0bd378770d rpcsrv: fix TestFilteredSubscriptions
Close #3693

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-12-11 14:18:23 +03:00
Anna Shaleva
827acfca23 services: fix timer draining
Ref. #3736.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-12-09 12:11:01 +03:00