Commit graph

5068 commits

Author SHA1 Message Date
Anna Shaleva
a0d991a500 rpcbinding: support map[any]any conversion for extended types
Unfortunately, without pre-set user extended types configuration for events
and without --guess-eventtypes flag set we are allowed to rely only on manifest
information about types. Manifest can't give us a lot of information, but we
still need to be able to generate RPC binding. Arrays and structs are correctly
handled by the current code, but maps always rely on the fact that map's value
type is set. It's not true in the described case, so make the maps type convertor
handle this situation in a similar way how arrays are handled.

Without this commit the following panic occurs on attempt to generate RPC binding:
```
    --- FAIL: TestAssistedRPCBindings/testdata/notifications (0.01s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7f7c0e]

goroutine 190 [running]:
testing.tRunner.func1.2({0x109cb40, 0x1d58760})
	/usr/local/go/src/testing/testing.go:1396 +0x24e
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1399 +0x39f
panic({0x109cb40, 0x1d58760})
	/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.extendedTypeToGo({0x22, {0x0, 0x0}, {0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:515 +0x36e
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.scTypeToGo({0xc000206d92?, 0xc000206d80?}, 0x22, 0xc0005d70e0)
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:643 +0x138
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.scTemplateToRPC({{0xc00049bb07, 0x7}, 0xc0004c89c0, {0x33, 0x22, 0x11, 0x0, 0xff, 0xee, 0xdd, ...}, ...}, ...)
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:686 +0xbc4
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.Generate({{0xc00049bb07, 0x7}, 0xc0004c89c0, {0x33, 0x22, 0x11, 0x0, 0xff, 0xee, 0xdd, ...}, ...})
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:421 +0x387
github.com/nspcc-dev/neo-go/cli/smartcontract.contractGenerateSomething(0xc00043e2c0, 0x137cd00)
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/generate.go:99 +0x855
github.com/nspcc-dev/neo-go/cli/smartcontract.contractGenerateRPCWrapper(0xc00043e2c0?)
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/generate.go:60 +0x25
github.com/urfave/cli.HandleAction({0x1048380?, 0x137c660?}, 0x13?)
	/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/app.go:524 +0x50
github.com/urfave/cli.Command.Run({{0x123539d, 0x13}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x12577ad, 0x2a}, {0x127ad35, ...}, ...}, ...)
	/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/command.go:173 +0x65b
github.com/urfave/cli.(*App).RunAsSubcommand(0xc0001f4000, 0xc00043e000)
	/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/app.go:405 +0x91b
github.com/urfave/cli.Command.startApp({{0x12281e1, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x1254d8a, 0x28}, {0x0, ...}, ...}, ...)
	/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/command.go:372 +0x6e7
github.com/urfave/cli.Command.Run({{0x12281e1, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x1254d8a, 0x28}, {0x0, ...}, ...}, ...)
	/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/command.go:102 +0x825
github.com/urfave/cli.(*App).Run(0xc00024e000, {0xc0004f6420, 0xb, 0xb})
	/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/app.go:277 +0x8a7
github.com/nspcc-dev/neo-go/cli/smartcontract.TestAssistedRPCBindings.func1.1(0x9f8829?)
	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/generate_test.go:395 +0x5fc
testing.tRunner(0xc0006824e0, 0xc0004a3680)
	/usr/local/go/src/testing/testing.go:1446 +0x10b
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1493 +0x35f
```

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:53:43 +03:00
Anna Shaleva
37af2031bb rpcbinding: properly add imports for simple types of event parameters
There are two ways of doing this: first one is to emit all notifications
parameter data into rpcbindings configuration on compile time (event if
the parameter has a simple type), and the second one is to fetch parameter
type from the manifest on rpcbinding file generation if needed (we always
have manifest at this stage, thus it's not a problem to retrieve necessary
information). The latter case is chosen to reduce the bindings configuration
file size.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:53:43 +03:00
Anna Shaleva
19cc6c6369 compiler: store ready-to-use notification names in bindings config
Notification and its parameters may have any UTF8-compatible name
which is inappropriate for bindings configuration and for the resulting
RPC bindings file. This commit stores the prettified version of
notification's name and parameters that are ready to be used in the
resulting RPC binding without any changes.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:53:43 +03:00
Anna Shaleva
41938ffa78 smartcontract: drop standard events before RPC binging generation
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:53:43 +03:00
Anna Shaleva
2c36802e23 binding: fix comment to the extended type field
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:53:43 +03:00
Anna Shaleva
e2580187a1 cli: fetch extended evet types from contract config
The user should specify it via parameter's `extendedtype` field and
via upper-level `namedtypes` field of the contract configuration YAML.

Also, as we have proper event structure source, make the `--guess-eventtype`
compilation option and make event types guess optional.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:53:41 +03:00
Anna Shaleva
194639bb15 compiler: fix typo in the method description
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:52:41 +03:00
Anna Shaleva
044ae477ca smartconract: generate RPC binding wrappers for events
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-31 15:52:39 +03:00
Anna Shaleva
ae52b2c2fa rpcbinding: fix binding indentation
New rule for writing blocks of code to our template: new line before
the block starts and new line after the block ends. This rule is the
same as the one we use during manual typing.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-30 17:37:06 +03:00
Anna Shaleva
db812f7fa5 mempool: add test for concurrent conflicting tx addition
Ref. https://github.com/neo-project/neo/pull/2818#discussion_r1206658847.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-26 20:20:47 +03:00
Anna Shaleva
0f0f7b364f rpcbinding: use typed return err value in etTypeConverter
Otherwise the resulting code can't be compiled.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-25 16:41:23 +03:00
Anna Shaleva
8beb9f23c3 smartcontract: add comments to binding config fields
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-25 16:41:23 +03:00
Anna Shaleva
36af361c2b smartcontract: add FromStackItem wrapper for RPC bindings
Make it exported and reusable from the user's code.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-24 11:27:49 +03:00
Roman Khimov
aca12b58c0
Merge pull request #3020 from nspcc-dev/loadnef-enh-2
cli: properly load specified method for `run` VM CLI command
2023-05-19 16:39:41 +03:00
Anna Shaleva
b6b80f3abf cli: properly handle run VM CLI command
Properly load the provided method using NEF and hash specified. It allows
to have NEF properly set in the VM context and handle CALLT instruction
correctly.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-19 13:26:56 +03:00
Anna Shaleva
fc6029d006 vm: allow to emit uint and uint64 as a part of array
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-17 19:29:42 +03:00
Anna Shaleva
15138b2004 vm: allow to emit convertible
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-17 12:47:31 +03:00
Anna Shaleva
8e085d3ca3 vm: allow to make stackitem from *Uint160 and *Uint256
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-17 11:16:32 +03:00
Anna Shaleva
649b9ac7b0 network: add neogo_version metric, deprecate serv_node_version
Close #2999.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-05-11 13:48:38 +03:00
Roman Khimov
433275265f *: use require.ErrorIs instead of require.True(t, error.Is())
This is just a much better way to do the same thing. Inspired by
nspcc-dev/neofs-sdk-go#407.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-05-04 17:03:47 +03:00
Roman Khimov
fcaa24f928 rpcclient: provide some exported error for disconnected WSClient
Regular Client doesn't care much about connections, because HTTP client's Do
method can reuse old ones or create additional ones on the fly. So one request
can fail and the next one easily succeed. WSClient is different, it works via
a single connection and if it breaks, it breaks forever for this
client. Callers will get some error on every request afterwards and it'd be
nice for this error to be the same so that API users could detect
disconnection this way too.

Related to nspcc-dev/neofs-node#2325.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-05-03 16:08:54 +03:00
Roman Khimov
8bd9a7d420
Merge pull request #2995 from nspcc-dev/neofs-sdk-update
NeoFS SDK update
2023-05-02 14:25:31 +03:00
Anna Shaleva
9c0a45c65e core: add nolint comment for deprecated error format style
Should be a part of 67d4d891ef.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-28 12:13:21 +03:00
Roman Khimov
0a160ee93b *: use CompareAndSwap instead of CAS for atomics
go.uber.org/atomic deprecated CAS methods in version 1.10 (that introduced
CompareAndSwap), so we need to fix it.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-04-28 12:07:19 +03:00
Roman Khimov
2567c4c672 oracle: update NeoFS SDK to 1.0.0-rc.8
We were using _very_ old version, so there is a number of changes, including:
 * fix for unlimited reads
 * UTF-8 check for HTTP requests

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-04-27 21:29:59 +03:00
Roman Khimov
dcea3f6107
Merge pull request #2988 from nspcc-dev/client/close-blockers
rpcclient: close WSClient subscriber on overflow
2023-04-26 22:23:49 +03:00
Anna Shaleva
67d4d891ef core: prevent direct access to Notary contract if not active
Otherwise it will cause panic, which isn't expected behaviour.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-26 14:11:12 +03:00
Anna Shaleva
edb2d46d5b core: initialize natives cache wrt NativeActivations
If the contract was deployed then cache must be initialized after
in-memory data reset. If the contract isn't active yet, then no
cache will be initialized on deploy (i.e. on call to Initialize()
method by native Management).

Close #2984.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-26 13:57:45 +03:00
Anna Shaleva
33c971b0e4 core: add InitializeCache method to Contract interface
Make the contracts cache initialization unified. The order of cache
iniitialization is not important and Nottary contract is added to the
bc.contracts.Contracts wrt P2PSigExtensions setting, thus no functional
changes, just refactoring for future applications.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-26 12:57:48 +03:00
Roman Khimov
29b3df10b6
Merge pull request #2992 from nspcc-dev/init-metrics-on-start
*: adjust Prometheus metrics initialisation on node start
2023-04-25 16:27:38 +03:00
Anna Shaleva
e2782aef05 *: adjust Prometheus metrics initialisation on node start
Initialize Prometheus metrics on node start where appropriate and review
the usage of the following metrics:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ find | grep prometheus.go
./pkg/network/prometheus.go
./pkg/core/stateroot/prometheus.go
./pkg/core/prometheus.go
./pkg/services/rpcsrv/prometheus.go
./pkg/services/metrics/prometheus.go
```

Close #2970.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-25 16:25:18 +03:00
Anna Shaleva
3f2e0e5441 Revert "native: make management compatible with C# node 3.5.0"
This reverts commit 236e633ee4.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-25 12:22:11 +03:00
Anna Shaleva
dab13a4e2d rpcclient: close WSClient subscriber on overflow
Close #2894.

It should be noted that the subscriber's channel is being removed from the
list of receivers and closed, but it is still *in the list of subscribers*
and no unsubscription is performed by WSClient. Which means that RPC server
keeps sending notifications to WSClient and WSClient keeps dropping them
(because there's no receiver for this subscription and it's OK, WSClient
can handle this and this behaviour is documented). However, it's still the
caller's duty to call Unsubscribe() method for this subscription.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-25 11:42:17 +03:00
Anna Shaleva
4a49bf5de4 rpcclient: introduce WSOptions for WSClient
Make a separate structure for WSClient configuration.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-25 11:42:15 +03:00
Leonard Lyubich
d90626d556 interop/util: Drop unused import
Remove no longer used import left after corresponding code removal in
4671fbb3be.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
2023-04-22 16:36:49 +04:00
ZhangTao1596
fb7fce0775 native: optimize vote reward data (fix #2844)
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2023-04-20 17:41:14 +08:00
Roman Khimov
6eaa76520f rpcclient: close subscriber channels on wsReader exit
The reader is about to exit and it will close legacy c.Notifications, but it
will leave subscription channels at the same time. This is wrong since these
channels will no longer receive any new events, game over.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-04-19 16:25:43 +03:00
Roman Khimov
45b353781f rpcclient: correctly handle request channel closure
wsReader() closes c.done first and then goes over the list of
c.respChannels. Technically this means that any of the two can be taken in
this select.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-04-18 19:36:27 +03:00
Roman Khimov
a4cc6da766
Merge pull request #2966 from nspcc-dev/bugfix/2896-rpc-shutdown-deadlock
rpc: Fix deadlock produced during server shutdown
2023-04-17 10:36:30 +03:00
Leonard Lyubich
e126bcc462 services/rpcsrv: Wait for subscription process to complete when stopped
Previously RPC server shutdown procedure listened to the execution
channel and stopped at the first element that arrived in the queue. This
could lead to the following problems:
 * stopper could steal the execution result from subscriber
 * stopper didn't wait for other subscription actions to complete

Add dedicated channel to `Server` for subscription routine. Close the
channel on `handleSubEvents` return and wait for signal in `Shutdown`.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
2023-04-17 10:42:50 +04:00
Leonard Lyubich
a113940f0b services/rpcsrv: Fix potential shutdown deadlock of RPC server
Previously RPC server could never be shut down completely due to
some start precondition failure (in particular, inability to serve HTTP
on any configured endpoint). The problem was caused by next facts:
 * start method ran subscription routine after HTTP init succeeded only
 * stop method blocked waiting for the subscription routine to return

Run `handleSubEvents` routine on fresh `Start` unconditionally. With
this change, `Shutdown` method won't produce deadlock since
`handleSubEvents` closes wait channel.

Refs #2896.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
2023-04-17 10:42:50 +04:00
Leonard Lyubich
649877d8f3 services/rpcsrv: Test Server shutdown with failed precondition
There is an existing problem with RPC server shutdown freeze after start
failure due to some init actions (at least HTTP listen) described in
#2896.

Add dedicated unit test which checks that `Shutdown` returns within 5s
after `Start` method encounters internal problems.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
2023-04-17 10:42:49 +04:00
Roman Khimov
e0abe2b858
Merge pull request #2969 from nspcc-dev/mp-metrics
Split notarypool and mempool metrics. Also, do not use metrics for temporary pools
created during block/transaction verification.

Close #2950.
2023-04-14 15:43:37 +03:00
Anna Shaleva
3a71aafc43 core: distinguish notarypool/mempool metrics
Move them to the core/network packages, close #2950. The name of
mempool's unsorted transactions metrics has been changed along the
way to match the core's metrics naming convention.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 18:40:19 +03:00
Anna Shaleva
7bcc62d99c *: fix Prometheus metrics comment formatting
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 18:36:08 +03:00
Anna Shaleva
49cea72e41 core: improve documentation to SetOracle/SetNotary
I've carefully checked the way how new service can be added to the
Blockchain instance or to be removed from it. Current implemention
of SetNotary and SetOracle methods doesn't contain dangerous code,
and native contracts have atomic values everywhere where service
is stored.

Current implementation of Notary, Oracle and StateRoot services'
reload/disabling/enabling on SIGUSR1 is safe and doesn't require
any adjustment.

This commit closes #2944, it's not a bug in the code, it's just
stale documentation.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 18:29:58 +03:00
Roman Khimov
a875409055
Merge pull request #2968 from nspcc-dev/bad-cfg-panic
config: do not allow negative validators/committee count
2023-04-13 14:31:07 +03:00
Anna Shaleva
a74454aaca config: do not allow negative validators count
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 13:43:14 +03:00
Anna Shaleva
0c049f620f config: do not allow zero numbers for validators/committee history
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 13:42:41 +03:00
Anna Shaleva
8149d33fef config: use uint32 for validators/committee members count
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 13:42:40 +03:00
Anna Shaleva
55ab38ed81 network: do not use error channel to start network srv
It's obsolete thing, we have looger and it perfectly suits our needs.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 13:21:34 +03:00
Anna Shaleva
d5bea0ad4c core: add Backwards option for storage iterators
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-12 16:45:38 +03:00
Anna Shaleva
400620a9fb core: avoid squashing of data from different state reset stages
`cache` persisting operation is concurrent with the storage modifications
made by further state reset stages. We can't allow situation when data
from the next stage are leaking into the previous stage. State reset stages
must be atomic in turms of DB persisting, thus, use another `upperCache`
MemCached store to keep them apart.

Here are the results of mainnet's BoltDB reset from 1.1M to 6K:

This patch:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000
2023-04-11T16:15:25.783+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2023-04-11T16:15:25.783+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2023-04-11T16:15:25.783+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2023-04-11T16:15:25.783+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2023-04-11T16:15:25.787+0300	INFO	restoring blockchain	{"version": "0.2.8"}
2023-04-11T16:15:25.906+0300	INFO	initializing state reset	{"target height": 600000}
2023-04-11T16:15:25.906+0300	DEBUG	trying to reset blocks, transactions and AERs
2023-04-11T16:15:57.031+0300	INFO	intermediate batch of removed blocks, transactions and AERs is collected	{"batch": 1, "took": "31.125057214s"}
2023-04-11T16:16:12.644+0300	DEBUG	intermediate batch of removed blocks, transactions and AERs is persisted	{"batch": 1, "took": "15.613156971s", "keys": 321895}
2023-04-11T16:16:13.895+0300	INFO	intermediate batch of removed blocks, transactions and AERs is collected	{"batch": 2, "took": "16.663444208s"}
2023-04-11T16:16:19.784+0300	INFO	last batch of removed blocks, transactions and AERs is collected	{"batch": 3, "took": "5.760308543s"}
2023-04-11T16:16:19.784+0300	INFO	blocks, transactions ans AERs are reset	{"took": "53.878632911s"}
2023-04-11T16:16:22.870+0300	DEBUG	intermediate batch of removed blocks, transactions and AERs is persisted	{"batch": 2, "took": "8.974838893s", "keys": 334823}
2023-04-11T16:16:22.870+0300	DEBUG	trying to reset contract storage items
2023-04-11T16:16:27.272+0300	DEBUG	last batch of removed blocks, transactions and AERs is persisted	{"batch": 3, "took": "7.487357441s", "keys": 208913}
2023-04-11T16:17:23.678+0300	INFO	intermediate batch of contract storage items and IDs is collected	{"batch": 1, "took": "1m0.80711106s"}
2023-04-11T16:18:00.769+0300	INFO	intermediate batch of contract storage items and IDs is collected	{"batch": 2, "took": "36.967660061s"}
2023-04-11T16:18:20.478+0300	DEBUG	intermediate batch of contract storage items is persisted	{"batch": 1, "took": "56.796257788s", "keys": 200000}
2023-04-11T16:18:54.115+0300	INFO	intermediate batch of contract storage items and IDs is collected	{"batch": 3, "took": "33.637412437s"}
2023-04-11T16:19:18.844+0300	DEBUG	intermediate batch of contract storage items is persisted	{"batch": 2, "took": "1m18.0737668s", "keys": 200000}
2023-04-11T16:19:27.650+0300	INFO	last batch of contract storage items is collected	{"batch": 4, "took": "8.806264019s"}
2023-04-11T16:19:27.650+0300	INFO	contract storage items are reset	{"took": "3m4.780232077s", "keys": 656944}
2023-04-11T16:20:15.660+0300	DEBUG	intermediate batch of contract storage items is persisted	{"batch": 3, "took": "1m21.544386403s", "keys": 200000}
2023-04-11T16:20:15.660+0300	DEBUG	trying to reset headers information
2023-04-11T16:20:16.385+0300	INFO	headers information is reset	{"took": "725.174932ms"}
2023-04-11T16:20:19.586+0300	DEBUG	last batch of contract storage items and IDs is persisted	{"batch": 4, "took": "51.936278608s", "keys": 56945}
2023-04-11T16:20:19.587+0300	DEBUG	trying to reset state root information and NEP transfers
2023-04-11T16:20:35.845+0300	INFO	state root information and NEP transfers are reset	{"took": "16.25852114s"}
2023-04-11T16:21:10.000+0300	DEBUG	headers information is persisted	{"took": "53.613638429s", "keys": 528438}
2023-04-11T16:21:10.003+0300	DEBUG	trying to remove stale storage items
2023-04-11T16:21:18.108+0300	INFO	stale storage items are reset	{"took": "8.105140658s", "keys": 1350176}
2023-04-11T16:21:18.108+0300	DEBUG	trying to remove state reset point
2023-04-11T16:21:18.108+0300	INFO	state reset point is removed	{"took": "8.554µs"}
2023-04-11T16:21:20.151+0300	DEBUG	state root information and NEP transfers are persisted	{"took": "44.305707049s", "keys": 602578}
2023-04-11T16:21:20.212+0300	INFO	state reset point information is persisted	{"took": "2.103764633s", "keys": 2}
2023-04-11T16:21:20.213+0300	INFO	reset finished successfully	{"took": "5m54.306861367s"}
```

The previous commit:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000
2023-04-11T16:24:04.256+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2023-04-11T16:24:04.256+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2023-04-11T16:24:04.256+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2023-04-11T16:24:04.256+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2023-04-11T16:24:04.261+0300	INFO	restoring blockchain	{"version": "0.2.8"}
2023-04-11T16:24:04.368+0300	INFO	initializing state reset	{"target height": 600000}
2023-04-11T16:24:04.368+0300	DEBUG	trying to reset blocks, transactions and AERs
2023-04-11T16:24:30.363+0300	INFO	intermediate batch of removed blocks, transactions and AERs is collected	{"batch": 1, "took": "25.995261037s"}
2023-04-11T16:24:44.947+0300	DEBUG	intermediate batch of removed blocks, transactions and AERs is persisted	{"batch": 1, "took": "14.584447338s", "keys": 321897}
2023-04-11T16:24:45.791+0300	INFO	intermediate batch of removed blocks, transactions and AERs is collected	{"batch": 2, "took": "15.428492824s"}
2023-04-11T16:24:51.252+0300	INFO	last batch of removed blocks, transactions and AERs is collected	{"batch": 3, "took": "5.460662766s"}
2023-04-11T16:24:51.252+0300	INFO	blocks, transactions ans AERs are reset	{"took": "46.884558096s"}
2023-04-11T16:24:55.399+0300	DEBUG	intermediate batch of removed blocks, transactions and AERs is persisted	{"batch": 2, "took": "9.607820004s", "keys": 334821}
2023-04-11T16:24:55.399+0300	DEBUG	trying to reset contract storage items
2023-04-11T16:24:59.981+0300	DEBUG	last batch of removed blocks, transactions and AERs is persisted	{"batch": 3, "took": "8.728713255s", "keys": 208913}
2023-04-11T16:25:50.827+0300	INFO	intermediate batch of contract storage items and IDs is collected	{"batch": 1, "took": "55.426411416s"}
2023-04-11T16:26:28.734+0300	INFO	intermediate batch of contract storage items and IDs is collected	{"batch": 2, "took": "37.902647706s"}
2023-04-11T16:26:53.960+0300	DEBUG	intermediate batch of contract storage items is persisted	{"batch": 1, "took": "1m3.129453265s", "keys": 200001}
2023-04-11T16:27:27.645+0300	INFO	intermediate batch of contract storage items and IDs is collected	{"batch": 3, "took": "33.685283662s"}
2023-04-11T16:27:52.173+0300	DEBUG	intermediate batch of contract storage items is persisted	{"batch": 2, "took": "1m23.438465575s", "keys": 199999}
2023-04-11T16:28:00.995+0300	INFO	last batch of contract storage items is collected	{"batch": 4, "took": "8.821990443s"}
2023-04-11T16:28:00.995+0300	INFO	contract storage items are reset	{"took": "3m5.595950958s", "keys": 656944}
2023-04-11T16:28:49.164+0300	DEBUG	intermediate batch of contract storage items is persisted	{"batch": 3, "took": "1m21.518344712s", "keys": 200000}
2023-04-11T16:28:49.164+0300	DEBUG	trying to reset headers information
2023-04-11T16:28:49.936+0300	INFO	headers information is reset	{"took": "772.36435ms"}
2023-04-11T16:28:53.122+0300	DEBUG	last batch of contract storage items and IDs is persisted	{"batch": 4, "took": "52.126928092s", "keys": 56945}
2023-04-11T16:28:53.122+0300	DEBUG	trying to reset state root information and NEP transfers
2023-04-11T16:29:09.332+0300	INFO	state root information and NEP transfers are reset	{"took": "16.20921699s"}
2023-04-11T16:29:46.264+0300	DEBUG	headers information is persisted	{"took": "56.326715249s", "keys": 528438}
2023-04-11T16:29:46.267+0300	DEBUG	trying to remove stale storage items
2023-04-11T16:29:53.986+0300	INFO	stale storage items are reset	{"took": "7.718950145s", "keys": 1350176}
2023-04-11T16:29:53.986+0300	DEBUG	trying to remove state reset point
2023-04-11T16:29:53.986+0300	INFO	state reset point is removed	{"took": "6.013µs"}
2023-04-11T16:29:55.899+0300	DEBUG	state root information and NEP transfers are persisted	{"took": "46.567302762s", "keys": 602578}
2023-04-11T16:29:55.929+0300	INFO	state reset point information is persisted	{"took": "1.942392208s", "keys": 2}
2023-04-11T16:29:55.929+0300	INFO	reset finished successfully	{"took": "5m51.561573137s"}
```

Master:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000
2023-04-11T16:34:12.410+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2023-04-11T16:34:12.410+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2023-04-11T16:34:12.410+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2023-04-11T16:34:12.410+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2023-04-11T16:34:12.413+0300	INFO	restoring blockchain	{"version": "0.2.8"}
2023-04-11T16:34:12.495+0300	INFO	initialize state reset	{"target height": 600000}
2023-04-11T16:34:12.513+0300	INFO	trying to reset blocks, transactions and AERs
2023-04-11T16:35:03.582+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 1, "took": "51.087226195s", "keys": 321895}
2023-04-11T16:35:31.302+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 2, "took": "27.719871393s", "keys": 334823}
2023-04-11T16:35:41.309+0300	INFO	last batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 3, "took": "10.007017388s", "keys": 208913}
2023-04-11T16:35:41.309+0300	INFO	blocks, transactions ans AERs are reset	{"took": "1m28.814245057s", "overall persisted keys": 865631}
2023-04-11T16:35:41.309+0300	INFO	trying to reset contract storage items
2023-04-11T16:37:38.315+0300	INFO	intermediate batch of contract storage items and IDs is persisted	{"batch": 1, "took": "1m57.00650253s", "keys": 200000}
2023-04-11T16:39:29.704+0300	INFO	intermediate batch of contract storage items and IDs is persisted	{"batch": 2, "took": "1m51.385224725s", "keys": 200000}
2023-04-11T16:41:14.991+0300	INFO	intermediate batch of contract storage items and IDs is persisted	{"batch": 3, "took": "1m45.287483794s", "keys": 200000}
2023-04-11T16:41:31.667+0300	INFO	last batch of contract storage items and IDs is persisted	{"batch": 4, "took": "16.675347478s", "keys": 56945}
2023-04-11T16:41:31.667+0300	INFO	contract storage items and IDs are reset	{"took": "5m50.357775401s", "keys": 656944}
2023-04-11T16:41:31.667+0300	INFO	trying to reset headers information
2023-04-11T16:42:16.779+0300	INFO	headers information is reset	{"took": "45.111354262s", "keys": 528438}
2023-04-11T16:42:16.784+0300	INFO	trying to reset state root information and NEP transfers
2023-04-11T16:42:35.778+0300	INFO	state root information and NEP transfers are reset	{"took": "18.99373117s", "keys": 602578}
2023-04-11T16:42:35.781+0300	INFO	trying to remove stale storage items
2023-04-11T16:42:43.884+0300	INFO	stale storage items are reset	{"took": "8.103929306s", "keys": 1350176}
2023-04-11T16:42:43.885+0300	INFO	trying to remove state reset point
2023-04-11T16:42:43.926+0300	INFO	stale reset point is removed	{"took": "41.858883ms", "keys": 2}
2023-04-11T16:42:43.932+0300	INFO	reset finished successfully	{"took": "8m31.437493325s"}
```

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-11 18:49:09 +03:00
Anna Shaleva
cb0f786b28 core: move batch persist to a separate routine
Resetting mainnet from 2512046 blocks (full logs are attached
to https://github.com/nspcc-dev/neo-go/pull/2813#issuecomment-1324115555).

--------
LevelDB |
------------------------
  to  |  old   |   new  |
------|--------|--------|
  1   | 5m11s  | 4m50s  |
------|--------|--------|
  1M  | 10m40s | 9m40s  |
------|--------|--------|
 2.5M | 17m38s | 17m36s |
------------------------

--------
BoltDB  |
------------------------
  to  |  old   |   new  |
------|--------|--------|
  1   |  8m3s  | 5m51s  |
------|--------|--------|
  1M  | 20m30s | 13m2s  |
------|--------|--------|
 2.5M | 31m26s | 18m47s |
------------------------

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-11 16:58:11 +03:00
Anna Shaleva
e3747b1d57 core: change log level of reset stages notifications
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-11 16:58:03 +03:00
Anna Shaleva
41caeed5c0 core: fix state reset log message
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-11 16:57:56 +03:00
Roman Khimov
5322a3535b
Merge pull request #2940 from nspcc-dev/groth16
core: add BLS12_381 interops
2023-04-10 10:20:17 +03:00
Erik van den Brink
f479b2bc57 neorpc: change peer port to int type (fixes nspcc-dev#2910) 2023-04-06 18:33:09 +02:00
Roman Khimov
e2cf5b868a
Merge pull request #2941 from nspcc-dev/drop-deprecated-0.102.0
Drop some deprecated things in 0.102.0
2023-04-06 10:42:26 +03:00
Anna Shaleva
01ac2d9f31 rpc: emit Null in case of Any parameter with zero-len value
Otherwise it leads to the following error in the TestActor_CallWithNilParam:
```
=== RUN   TestActor_CallWithNilParam
    logger.go:130: 2023-04-03T15:58:27.672+0300	INFO	initial gas supply is not set or wrong, setting default value	{"InitialGASSupply": "52000000"}
    logger.go:130: 2023-04-03T15:58:27.672+0300	INFO	P2PNotaryRequestPayloadPool size is not set or wrong, setting default value	{"P2PNotaryRequestPayloadPoolSize": 1000}
    logger.go:130: 2023-04-03T15:58:27.672+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
    logger.go:130: 2023-04-03T15:58:27.672+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
    logger.go:130: 2023-04-03T15:58:27.672+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
    logger.go:130: 2023-04-03T15:58:27.672+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
    logger.go:130: 2023-04-03T15:58:27.675+0300	INFO	no storage version found! creating genesis block
    logger.go:130: 2023-04-03T15:58:27.675+0300	INFO	ExtensiblePoolSize is not set or wrong, using default value	{"ExtensiblePoolSize": 20}
    logger.go:130: 2023-04-03T15:58:27.675+0300	INFO	SessionPoolSize is not set or wrong, setting default value	{"SessionPoolSize": 20}
    logger.go:130: 2023-04-03T15:58:27.675+0300	INFO	MaxWebSocketClients is not set or wrong, setting default value	{"MaxWebSocketClients": 64}
    logger.go:130: 2023-04-03T15:58:27.675+0300	INFO	starting rpc-server	{"endpoint": "localhost:0"}
    logger.go:130: 2023-04-03T15:58:27.677+0300	DEBUG	done processing headers	{"headerIndex": 1, "blockHeight": 0, "took": "436.313µs"}
    logger.go:130: 2023-04-03T15:58:27.679+0300	DEBUG	done processing headers	{"headerIndex": 2, "blockHeight": 1, "took": "272.891µs"}
    logger.go:130: 2023-04-03T15:58:27.680+0300	DEBUG	done processing headers	{"headerIndex": 3, "blockHeight": 2, "took": "276.949µs"}
    logger.go:130: 2023-04-03T15:58:27.681+0300	DEBUG	done processing headers	{"headerIndex": 4, "blockHeight": 3, "took": "286.028µs"}
    logger.go:130: 2023-04-03T15:58:27.681+0300	DEBUG	done processing headers	{"headerIndex": 5, "blockHeight": 4, "took": "268.673µs"}
    logger.go:130: 2023-04-03T15:58:27.681+0300	INFO	bad notification	{"contract": "565cff9508ebc75aadd7fe59f38dac610ab6093c", "event": "Transfer", "error": "parameter 0 type mismatch: Hash160 vs ByteString"}
    logger.go:130: 2023-04-03T15:58:27.682+0300	DEBUG	done processing headers	{"headerIndex": 6, "blockHeight": 5, "took": "380.988µs"}
    logger.go:130: 2023-04-03T15:58:27.683+0300	DEBUG	done processing headers	{"headerIndex": 7, "blockHeight": 6, "took": "273.543µs"}
    logger.go:130: 2023-04-03T15:58:27.683+0300	DEBUG	done processing headers	{"headerIndex": 8, "blockHeight": 7, "took": "275.163µs"}
    logger.go:130: 2023-04-03T15:58:27.684+0300	DEBUG	done processing headers	{"headerIndex": 9, "blockHeight": 8, "took": "259.578µs"}
    logger.go:130: 2023-04-03T15:58:27.685+0300	DEBUG	done processing headers	{"headerIndex": 10, "blockHeight": 9, "took": "266.882µs"}
    logger.go:130: 2023-04-03T15:58:27.686+0300	DEBUG	done processing headers	{"headerIndex": 11, "blockHeight": 10, "took": "295.3µs"}
    logger.go:130: 2023-04-03T15:58:27.687+0300	DEBUG	done processing headers	{"headerIndex": 12, "blockHeight": 11, "took": "295.568µs"}
    logger.go:130: 2023-04-03T15:58:27.688+0300	DEBUG	done processing headers	{"headerIndex": 13, "blockHeight": 12, "took": "258.197µs"}
    logger.go:130: 2023-04-03T15:58:27.689+0300	DEBUG	done processing headers	{"headerIndex": 14, "blockHeight": 13, "took": "261.602µs"}
    logger.go:130: 2023-04-03T15:58:27.689+0300	DEBUG	done processing headers	{"headerIndex": 15, "blockHeight": 14, "took": "268.922µs"}
    logger.go:130: 2023-04-03T15:58:27.690+0300	DEBUG	done processing headers	{"headerIndex": 16, "blockHeight": 15, "took": "276.176µs"}
    logger.go:130: 2023-04-03T15:58:27.691+0300	DEBUG	done processing headers	{"headerIndex": 17, "blockHeight": 16, "took": "256.068µs"}
    logger.go:130: 2023-04-03T15:58:27.692+0300	DEBUG	done processing headers	{"headerIndex": 18, "blockHeight": 17, "took": "262.303µs"}
    logger.go:130: 2023-04-03T15:58:27.692+0300	DEBUG	done processing headers	{"headerIndex": 19, "blockHeight": 18, "took": "265.087µs"}
    logger.go:130: 2023-04-03T15:58:27.693+0300	DEBUG	done processing headers	{"headerIndex": 20, "blockHeight": 19, "took": "260.758µs"}
    logger.go:130: 2023-04-03T15:58:27.694+0300	DEBUG	done processing headers	{"headerIndex": 21, "blockHeight": 20, "took": "263.482µs"}
    logger.go:130: 2023-04-03T15:58:27.694+0300	DEBUG	done processing headers	{"headerIndex": 22, "blockHeight": 21, "took": "327.812µs"}
    logger.go:130: 2023-04-03T15:58:27.696+0300	DEBUG	done processing headers	{"headerIndex": 23, "blockHeight": 22, "took": "284.104µs"}
    logger.go:130: 2023-04-03T15:58:27.697+0300	WARN	contract invocation failed	{"tx": "82279bfe9bada282ca0f8cb8e0bb124b921af36f00c69a518320322c6f4fef60", "block": 23, "error": "at instruction 0 (ABORT): ABORT"}
    logger.go:130: 2023-04-03T15:58:27.697+0300	DEBUG	processing rpc request	{"method": "getversion", "params": "[]"}
    logger.go:130: 2023-04-03T15:58:27.698+0300	DEBUG	processing rpc request	{"method": "invokefunction", "params": "[565cff9508ebc75aadd7fe59f38dac610ab6093c putValue  ]"}
    client_test.go:2562:
        	Error Trace:	/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/services/rpcsrv/client_test.go:2562
        	Error:      	Should be true
        	Test:       	TestActor_CallWithNilParam
        	Messages:   	at instruction 6 (PACK): OPACK: invalid length
    logger.go:130: 2023-04-03T15:58:27.699+0300	INFO	shutting down RPC server	{"endpoint": "127.0.0.1:46005"}
    logger.go:130: 2023-04-03T15:58:27.700+0300	INFO	persisted to disk	{"blocks": 23, "keys": 1236, "headerHeight": 23, "blockHeight": 23, "took": "908.825µs"}
--- FAIL: TestActor_CallWithNilParam (0.03s)

FAIL
```

See also the ref. df534f6b0c/src/Neo/SmartContract/ContractParameter.cs (L141)
and the way how parameters are handled by ref. RPC server:
4b3a76e1b7/src/RpcServer/RpcServer.SmartContract.cs (L202)
and FromJSON implementation:
df534f6b0c/src/Neo/SmartContract/ContractParameter.cs (L70)
2023-04-05 16:28:30 +03:00
Anna Shaleva
69102a6aa3 interop: add groth16 interop API 2023-04-05 15:37:50 +03:00
Anna Shaleva
115ec4d8dd core: add BLS12_381 interops 2023-04-05 15:37:50 +03:00
Roman Khimov
c053f1a4af
Merge pull request #2957 from nspcc-dev/rm-go-17
*: drop go 1.17 support
2023-04-04 15:25:14 +03:00
Anna Shaleva
6b21ad9922 *: replace interface{} with any keyword
Everywhere including examples, external interop APIs, bindings generators
code and in other valuable places. A couple of `interface{}` usages are
intentionally left in the CHANGELOG.md, documentation and tests.
2023-04-04 13:22:42 +03:00
Anna Shaleva
83545b8451 network: fix Address test failing on Windows/macOS with go 1.20
See the trace:
```
2023-03-31T07:46:21.1886260Z === RUN   TestEncodeDecodeAddress
2023-03-31T07:46:21.1886420Z     address_test.go:30:
2023-03-31T07:46:21.1887000Z         	Error Trace:	/Users/runner/work/neo-go/neo-go/pkg/network/payload/address_test.go:30
2023-03-31T07:46:21.1887180Z         	Error:      	Not equal:
2023-03-31T07:46:21.1887580Z         	            	expected: net.IP{0x7f, 0x0, 0x0, 0x1}
2023-03-31T07:46:21.1888290Z         	            	actual  : net.IP{0x7f, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
2023-03-31T07:46:21.1888450Z
2023-03-31T07:46:21.1888640Z         	            	Diff:
2023-03-31T07:46:21.1888960Z         	            	--- Expected
2023-03-31T07:46:21.1889180Z         	            	+++ Actual
2023-03-31T07:46:21.1889510Z         	            	@@ -1,3 +1,3 @@
2023-03-31T07:46:21.1889870Z         	            	-(net.IP) (len=4) {
2023-03-31T07:46:21.1890530Z         	            	- 00000000  7f 00 00 01                                       |....|
2023-03-31T07:46:21.1891140Z         	            	+(net.IP) (len=16) {
2023-03-31T07:46:21.1891780Z         	            	+ 00000000  7f 00 00 01 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2023-03-31T07:46:21.1891970Z         	            	 }
2023-03-31T07:46:21.1892510Z         	Test:       	TestEncodeDecodeAddress
2023-03-31T07:46:21.1892770Z --- FAIL: TestEncodeDecodeAddress (0.00s)
```

I'm still not sure what's the original root of this problem, there's nothing
about resolving scheme changes in https://go.dev/doc/go1.20, but it didn't
happen earlier and it works as expected in Ubuntu.
2023-04-03 09:43:14 +03:00
Anna Shaleva
856edcca39 *: bump min supported go version from 1.17 to 1.18 2023-03-31 09:59:24 +03:00
Roman Khimov
1633ae32df
Merge pull request #2954 from ZhangTao1596/fix-format
Fix lint jobs issues
2023-03-31 09:49:39 +03:00
ZhangTao1596
d36df15878 ci: fix lint issues (fix #2948) 2023-03-29 11:19:23 +08:00
ZhangTao1596
f21e618be5 rpcclient: add GetProof and VerifyProof (fix #2942) 2023-03-23 17:03:22 +08:00
Roman Khimov
6e27883c10 rpcsrv: remove deprecated RPC counters 2023-03-18 10:52:24 +03:00
Roman Khimov
4671fbb3be interop: drop deprecated util.FromAddress
It still was used in a number of places, surprisingly.
2023-03-18 10:44:12 +03:00
Roman Khimov
0d5613cfcb
Merge pull request #2938 from nspcc-dev/failing-actor-test
actor: don't close already closed channel, fix #2932
2023-03-17 11:46:31 +03:00
Roman Khimov
da7eafd4c7
Merge pull request #2937 from nspcc-dev/copy-wsclient-filters
Copy wsclient filters
2023-03-17 10:49:13 +03:00
Roman Khimov
e197f3faef rpcclient: copy subscription params, fix #2890
Clients can change things and we better be safe here.
2023-03-17 10:33:30 +03:00
Roman Khimov
e84ea0207d actor: don't close already closed channel, fix #2932
Waiter should close its channels, but WSClient can also do that and it can do
that in a drain loop as well.
2023-03-17 09:57:41 +03:00
Roman Khimov
1a4da8c462 neorpc: add Copy to filters for easy deep copying 2023-03-16 23:43:00 +03:00
Roman Khimov
74623e64bc rpcclient: improve wsclient doc, fix #2895 2023-03-16 21:27:35 +03:00
Roman Khimov
44e84a5943 stateroot: fix spelling and enhance FindStates doc, fix #2925 2023-03-15 22:55:19 +03:00
Anna Shaleva
c7566c2a01 consensus: perform batched chain's block sync
If there are several blocks from chain, then initialize consensus only for the
latest one.
2023-03-15 17:45:56 +03:00
Anna Shaleva
0b352349eb consensus: adjust TestService_NextConsensus after dBFT upgrade 2023-03-15 17:37:47 +03:00
Anna Shaleva
0cbef58b3c consensus: enqueue newly created blocks
Do not add them directly to chain, it will be done by the block queue
manager. Close https://github.com/nspcc-dev/neo-go/issues/2923. However,
this commit is not valid without
https://github.com/roman-khimov/dbft/pull/4.
It's the neo-go's duty to initialize consensus after subsequent block
addition; the dBFT itself must wait for the neo-go to complete the block
addition and notify the dBFT, so that it can initialize at 0-th view to
collect the next block.
2023-03-15 17:37:47 +03:00
Anna Shaleva
04d0b45ceb network: move blockqueue to a separate package 2023-03-15 17:37:47 +03:00
Anna Shaleva
91a77c25a2 network: refactor blockqueuer interface
Remove unused argument.
2023-03-15 17:37:47 +03:00
Anna Shaleva
e57e74692f consensus: drain block notification channel after each message
Make blockchain lock less possible.
2023-03-15 17:37:47 +03:00
Anna Shaleva
ea46943815 services: use buffered channels for block subscription
Add a tiny buffer where possible to avoid Blockchain's blocking
on new block addition.
2023-03-15 17:37:47 +03:00
Anna Shaleva
5f6c01336c *: add nolint comments to multiple errors wrapping
To be enabled after go 1.20 support is added.
2023-03-15 16:38:01 +03:00
Roman Khimov
4f708c037d network: drain send queues on peer disconnection
Fix potential memory leak with a lot of connected clients that keep requesting
things from node and then disconnect.
2023-02-21 16:19:06 +03:00
Anna Shaleva
da757fa387 network: fix grammar typo in the error message 2023-02-20 11:08:07 +03:00
Roman Khimov
475d9de2d5
Merge pull request #2916 from nspcc-dev/not-so-local-client
Not so local RPC client
2023-02-20 10:38:30 +03:00
Roman Khimov
d54418bad0
Merge pull request #2920 from nspcc-dev/fix-seek-doc
core: adjust the documentation of SeekRange's Prefix field
2023-02-18 23:18:39 +03:00
Anna Shaleva
09186127da core: adjust the documentation of SeekRange's Prefix field
It is used in different context (seeking through the storage or
through the contract storage), and sometimes empty prefix is not
supported.
2023-02-18 21:06:30 +03:00
Roman Khimov
b1e7f40226 rpcbinding: fix wrappers for Any type, fix #2898 2023-02-18 00:06:45 +03:00
Roman Khimov
bd4a9f3f22 rpcsrv: drop ws from subscriber
Subscribers can be local as well and no one uses this field anyway.
2023-02-16 23:49:58 +03:00
Roman Khimov
6f7fed344b rpcclient: add Internal client for direct RPC server calls
It has a special `requestF` and a special initialization function, but other
than that it's an absolutely regular WSClient. Can be used to call, can be
used to subscribe. Fixes #2909.
2023-02-16 23:49:58 +03:00
Roman Khimov
a55a01d456 rpcsrv: provide RegisterLocal for local clients
This allows to call RPC server more effectively in the same process (bypassing
TCP/HTTP). Refs. #2909.
2023-02-16 23:49:58 +03:00
Roman Khimov
e496084bee params: add FromAny function
Creating RPC server parameters from any data can be useful. Refs. #2909.
2023-02-15 16:46:59 +03:00
Erik van den Brink
3bdb3a87b8 rpcclient: expose endpoint (fixes #2912) 2023-02-15 08:29:44 +01:00
Anna Shaleva
25ed5fcd60 vm: allow custom limit duing Map\Array\Struct deserialization 2023-02-08 10:58:55 +03:00
Anna Shaleva
15fa65d30f vm: export stackitem's deserContext 2023-02-08 10:58:54 +03:00
Anna Shaleva
51a28fac8d rpcclient: adjust new subscriptions API doc
Add a warning about consequences of unproper notifications reading.
Ref. https://github.com/morphbits/neofs-cdn-edge-manager/pull/88#discussion_r1088007680.
2023-01-30 14:36:11 +03:00
Leonard Lyubich
d09158161e services/rpcsrv: Strengthen Server error channel's type
According to docs, `Server` uses provided error channel only to write
encountered error to it. In this case, there is no need to accept rw
channel to create `Server` instance. Strengthening the type to
write-only will allow the caller to ensure control of reading errors
from the provided channel.

The change is backward compatible since any `chan` is `chan<-`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2023-01-23 10:33:46 +03:00
Leonard Lyubich
cbdc6ba469 rpcclient/actor: Document RPCActor.CalculateNetworkFee method
`Actor.MakeUnsignedUncheckedRun` method imposes restriction to
`CalculateNetworkFee` method's implementations: `Hash` or `Size` methods
must not be called on the pointer to the given transaction.

Add docs to adjust described requirement.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2023-01-17 21:28:20 +03:00
Anna Shaleva
28927228f0 *: adjust subscription-related doc
Add a warning about received events modification where applicable.
2023-01-17 17:11:19 +03:00
Roman Khimov
40ef567527
Merge pull request #2872 from nspcc-dev/fix-missing-exception-in-applog
state: always marshal applog exception into JSON, fix #2869
2023-01-12 12:49:00 +07:00
Roman Khimov
babc44bfb3
Merge pull request #2871 from nspcc-dev/fix-shadowed-infinite-loop
Fix shadowed infinite loop
2023-01-12 12:48:39 +07:00
Roman Khimov
9a6fa84f70
Merge pull request #2867 from nspcc-dev/store-magic-in-db
core: store magic in the DB version, fix #2847
2023-01-11 20:42:00 +07:00
Roman Khimov
4e23695441
Merge pull request #2866 from nspcc-dev/findstates-ret-nothing
Fix findstates response difference
2023-01-11 20:41:10 +07:00
Roman Khimov
817f64f9fe state: always marshal applog exception into JSON, fix #2869
Fortunately, this is backwards-compatible in any direction. See
98b5e2b353 also.
2023-01-11 16:36:41 +03:00
Roman Khimov
8305389852 compiler: don't panic, return error for strange range loops
See #2870.
2023-01-11 16:21:40 +03:00
Roman Khimov
92acc71c80 compiler: create new locals for range loops when needed, fix #2855 2023-01-11 15:49:10 +03:00
Roman Khimov
1c4c783c3a core: store magic in the DB version, fix #2847 2023-01-11 12:05:05 +03:00
Roman Khimov
e780037cb4 rpcsrv: use valid stateroot hash in failing tests
Just in case.
2023-01-11 11:23:47 +03:00
Roman Khimov
9718602ce3 rpcsrv: return empty set from findstates when there is no data
Fix #2863.
2023-01-11 11:21:58 +03:00
Roman Khimov
584675ec23 state: optimize NEP17Transfer struct
We have both from and to here, so technically we can either drop the neg/neg
trick from the processTokenTransfer() or drop one field from the structure
(the other side is a part of the key). Drop the field since this can make the
DB a bit more compact. Change Amount to be a pointer along the way since
that's the "native" thing for big.Int, we've used non-pointer field
specifically to avoid Neg/Neg problems, but it looks like this is not
necessary.

This structure is only used by the RPC server and I doubt anyone uses it via
the *Blockchain.
2023-01-10 22:51:45 +03:00
Roman Khimov
dfd4f6978f bigint: don't reallocate big.Int in ToBytes(), fix #2864
In some cases n.Add() can reuse the []Word buffer and n.Sub() reallocate it
away. If that happens, we're out of luck with 0.99.0+ versions (since
3945e81857). I'm not sure why it does that, bit
width doesn't change in most of the cases and even if it does, we still have
enough of it in cap() to hold the old Abs() value (when we have a negative
value we in fact decreate its Abs() first and increase it back
afterwards). Still, that's what we have.

So when we have processTokenTransfer() doing Neg/Neg in-place its value is not
affected, but the original []Word bits that are reused by amount value are
(they're shared initially, Amount: *amount).

name                   old time/op    new time/op    delta
ToPreallocatedBytes-8    65.8ns ± 2%    45.6ns ± 2%  -30.73%  (p=0.008 n=5+5)

name                   old alloc/op   new alloc/op   delta
ToPreallocatedBytes-8     0.00B          0.00B          ~     (all equal)

name                   old allocs/op  new allocs/op  delta
ToPreallocatedBytes-8      0.00           0.00          ~     (all equal)
2023-01-10 22:51:45 +03:00
Roman Khimov
4e6ce9c3e6
Merge pull request #2846 from nspcc-dev/fix-getblockbyindex-req
network: do not allow to request invalid block count
2022-12-29 04:04:01 +07:00
Anna Shaleva
9b364aa7ee network: do not allow to request invalid block count
The problem is in peer disconnection due to invalid GetBlockByIndex
payload (the logs are from some patched neo-go version):
```
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        INFO        new peer connected        {"addr": "10.78.69.115:50846", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        WARN        peer disconnected        {"addr": "10.78.69.115:50846", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        INFO        started protocol        {"addr": "10.78.69.115:50846", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.491Z        INFO        new peer connected        {"addr": "10.78.69.115:50856", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        WARN        peer disconnected        {"addr": "10.78.69.115:50856", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        INFO        started protocol        {"addr": "10.78.69.115:50856", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        INFO        new peer connected        {"addr": "10.78.69.115:50858", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.493Z        INFO        started protocol        {"addr": "10.78.69.115:50858", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.493Z        WARN        peer disconnected        {"addr": "10.78.69.115:50858", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        INFO        new peer connected        {"addr": "10.78.69.115:50874", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        INFO        started protocol        {"addr": "10.78.69.115:50874", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        WARN        peer disconnected        {"addr": "10.78.69.115:50874", "error": "invalid block count", "peerCount": 2}
```

GetBlockByIndex payload can't be decoded, and the only possible cause
is zero (or <-1, but it's probably not the case) block count requested.

Error is improved as far.
2022-12-28 13:04:56 +03:00
Anna Shaleva
c0a453a53b network: adjust requestBlocs logic
If the lastQueued block index is the same as the one we'd like to
request in payload, then we need to increment the payload's count.
2022-12-28 12:50:30 +03:00
Evgeniy Stratonikov
db977ce38d vm: add fuzz test for ParseMultiSigContract
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2022-12-28 11:50:15 +03:00
Evgeniy Stratonikov
affe1ecb4f encoding: add bigint fuzz test
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2022-12-28 11:50:13 +03:00
Roman Khimov
3339824fe9 native: simplify code somewhat, get Metadata() only once 2022-12-16 23:48:04 +03:00
Roman Khimov
5d478b5514 native: add old management deploy/update call flags to Aspidochelone
d5a9af5860 is incompatible with the NeoFS
mainnet sidechain, so we add the old logic to the pre-Aspidochelone
behaviour. Changing flags at newMethodAndPrice() is a bit less convenient
unfortunately because this will affect interop validity checks, so let's have
this kludge here.
2022-12-16 23:45:47 +03:00
Roman Khimov
3e714a348e
Merge pull request #2842 from nspcc-dev/stringer
*: use zap.Stringer instead of zap.String where it can be used
2022-12-13 22:12:18 +07:00
Roman Khimov
e79dec15f9 *: use zap.Stringer instead of zap.String where it can be used
It's a bit more efficient in case we're not logging the message (mostly for
debug), makes the code somewhat simpler as well.
2022-12-13 12:44:54 +03:00
Roman Khimov
c1afaf8ea8 rpcsrv: use more robust NEP-11 divisibility check
It's more heavyweight at the same time, but should be OK for the
purpose. Inspired by https://github.com/neo-project/neo-modules/issues/754#issuecomment-1345176986
2022-12-10 11:45:43 +03:00
Roman Khimov
3ce1fc41a4 config: fix the default P2P ping settings
54c2aa8582 broke them, they're in seconds and we
have a 90s timeout.
2022-12-07 21:29:40 +03:00
Roman Khimov
0d65071abc config: use string type for Port
6b4dd5703e made it to be a uint16 which was
somewhat important for RPC, but now it's irrelevant and the fact that it was a
string in some cases may lead to errors like these:

  failed to unmarshal config YAML: yaml: unmarshal errors:
    line 48: cannot unmarshal !!str `20011` into uint16
    line 52: cannot unmarshal !!str `40001` into uint16

So for maximum backwards compatibility we better have string here and
eventually it'll be deleted anyway.
2022-12-07 21:21:05 +03:00
Roman Khimov
4e7cee4e12 config: replace VerifyBlocks with SkipBlockVerification
It directly affects node security and the default here MUST BE the safe choice
which is to do the verification. Otherwise it's just dangerous, absent any
VerifyBlocks configuration we'll get an insecure node. This option is not
supposed to be frequently used and it doesn't affect the ability to process
blocks, so breaking compatibility (in a safe manner) should be OK here.
2022-12-07 17:35:56 +03:00
Roman Khimov
7589733017 config: add a special Blockchain type to configure Blockchain
And include some node-specific configurations there with backwards
compatibility. Note that in the future we'll remove Ledger's
fields from the ProtocolConfiguration and it'll be possible to access them in
Blockchain directly (not via .Ledger).

The other option tried was using two configuration types separately, but that
incurs more changes to the codebase, single structure that behaves almost like
the old one is better for backwards compatibility.

Fixes #2676.
2022-12-07 17:35:53 +03:00
Roman Khimov
199a6de737 Merge pull request #2838 from nspcc-dev/neo-misuse
*: fix Neo and NeoGo misuses
2022-12-07 21:34:27 +07:00
Anna Shaleva
2d6db6f5c4 vm: adjust stale comment 2022-12-07 17:30:02 +03:00
Anna Shaleva
82221b0ca7 *: fix Neo and NeoGo misuses 2022-12-07 17:29:09 +03:00
Roman Khimov
b1f1405f42
Merge pull request #2836 from nspcc-dev/fix-subs
rpcclient: fix filtered naive subscriptions receiver
2022-12-07 21:17:07 +07:00
Anna Shaleva
2c0844221a rpcclient: fix filtered naive subscriptions receiver
We should return the filter itself instead of pointer.
2022-12-07 16:56:16 +03:00
Roman Khimov
c4d519f549
Merge pull request #2837 from nspcc-dev/350-compat
native: make management compatible with C# node 3.5.0
2022-12-07 20:19:04 +07:00
Roman Khimov
236e633ee4 native: make management compatible with C# node 3.5.0
It doesn't store id->hash mappings for native contracts. We need blockchain's
GetContractScriptHash to serve both anyway, so it was changed a bit. The only
other direct user of native.GetContractScriptHash is the VM CLI, but I doubt
anyone will use it for native contracts (they have ~zero VM code anyway).
2022-12-07 15:13:17 +03:00
Roman Khimov
883c6c5286 config: add Consensus subsection for dBFT config, fix #2677
Old UnlockWallet is still supported and works just fine.
2022-12-07 13:20:21 +03:00
Roman Khimov
730849a1cd config: move common part of notary/state into InternalService 2022-12-07 13:18:50 +03:00
Roman Khimov
90a85259a8
Merge pull request #2827 from nspcc-dev/multiaddr
network: support multiple bind addresses
2022-12-07 17:15:56 +07:00
Anna Shaleva
54c2aa8582 config: move P2P options to a separate config section
And convert time-related settings to a Duration format along the way.
2022-12-07 13:06:05 +03:00
Anna Shaleva
9cf6cc61f4 network: allow multiple bind addresses for server
And replace Transporter.Address() with Transporter.HostPort() along the way.
2022-12-07 13:06:03 +03:00
Anna Shaleva
9087854b1e services: allow multiple listen addresses for simple services
And deprecate the old way of Address/Port configuration for these services.
2022-12-07 11:23:01 +03:00
Roman Khimov
cad0fab704
Merge pull request #2828 from nspcc-dev/rpcwrapper-structures
Handle structures in the RPC wrapper generator
2022-12-06 21:40:16 +07:00
Anna Shaleva
6b4dd5703e config: unify BasicService config within existing services
There are no changes visible from the user side (at least for those
users who doesn't put Prometheus's or pprof's port in quotes), just
internal refactoring. From now and on, BasicService configuration is
used by RPC server config, TLS for RPC server, pprof and Prometheus.
2022-12-06 16:35:09 +03:00
Roman Khimov
e5aa5ca294 rpcbinding: improve error reporting in generated code 2022-12-06 13:15:10 +03:00
Roman Khimov
ec5ebc8c18 rpcbinding: improve indentation for internal unwrappers 2022-12-06 13:15:10 +03:00
Roman Khimov
c058ab5604 rpcbinding: handle more complex non-structured types 2022-12-06 13:15:10 +03:00
Roman Khimov
ce67e6795e rpcbinding: properly support maps
I'm not sure that map with a `*big.Int` key is a good one, but it can work
this way.
2022-12-06 13:15:10 +03:00
Roman Khimov
8e0be6e7a5 rpcbinding: improve indentation for internal array code 2022-12-06 13:15:10 +03:00
Roman Khimov
0214628127 rpcbinding: generate bindings using new extended type data 2022-12-06 13:15:10 +03:00