Commit graph

5188 commits

Author SHA1 Message Date
Anna Shaleva
0cbef58b3c consensus: enqueue newly created blocks
Do not add them directly to chain, it will be done by the block queue
manager. Close https://github.com/nspcc-dev/neo-go/issues/2923. However,
this commit is not valid without
https://github.com/roman-khimov/dbft/pull/4.
It's the neo-go's duty to initialize consensus after subsequent block
addition; the dBFT itself must wait for the neo-go to complete the block
addition and notify the dBFT, so that it can initialize at 0-th view to
collect the next block.
2023-03-15 17:37:47 +03:00
Anna Shaleva
04d0b45ceb network: move blockqueue to a separate package 2023-03-15 17:37:47 +03:00
Anna Shaleva
91a77c25a2 network: refactor blockqueuer interface
Remove unused argument.
2023-03-15 17:37:47 +03:00
Anna Shaleva
e57e74692f consensus: drain block notification channel after each message
Make blockchain lock less possible.
2023-03-15 17:37:47 +03:00
Anna Shaleva
ea46943815 services: use buffered channels for block subscription
Add a tiny buffer where possible to avoid Blockchain's blocking
on new block addition.
2023-03-15 17:37:47 +03:00
Anna Shaleva
5f6c01336c *: add nolint comments to multiple errors wrapping
To be enabled after go 1.20 support is added.
2023-03-15 16:38:01 +03:00
Roman Khimov
4f708c037d network: drain send queues on peer disconnection
Fix potential memory leak with a lot of connected clients that keep requesting
things from node and then disconnect.
2023-02-21 16:19:06 +03:00
Anna Shaleva
da757fa387 network: fix grammar typo in the error message 2023-02-20 11:08:07 +03:00
Roman Khimov
475d9de2d5
Merge pull request #2916 from nspcc-dev/not-so-local-client
Not so local RPC client
2023-02-20 10:38:30 +03:00
Roman Khimov
d54418bad0
Merge pull request #2920 from nspcc-dev/fix-seek-doc
core: adjust the documentation of SeekRange's Prefix field
2023-02-18 23:18:39 +03:00
Anna Shaleva
09186127da core: adjust the documentation of SeekRange's Prefix field
It is used in different context (seeking through the storage or
through the contract storage), and sometimes empty prefix is not
supported.
2023-02-18 21:06:30 +03:00
Roman Khimov
b1e7f40226 rpcbinding: fix wrappers for Any type, fix #2898 2023-02-18 00:06:45 +03:00
Roman Khimov
bd4a9f3f22 rpcsrv: drop ws from subscriber
Subscribers can be local as well and no one uses this field anyway.
2023-02-16 23:49:58 +03:00
Roman Khimov
6f7fed344b rpcclient: add Internal client for direct RPC server calls
It has a special `requestF` and a special initialization function, but other
than that it's an absolutely regular WSClient. Can be used to call, can be
used to subscribe. Fixes #2909.
2023-02-16 23:49:58 +03:00
Roman Khimov
a55a01d456 rpcsrv: provide RegisterLocal for local clients
This allows to call RPC server more effectively in the same process (bypassing
TCP/HTTP). Refs. #2909.
2023-02-16 23:49:58 +03:00
Roman Khimov
e496084bee params: add FromAny function
Creating RPC server parameters from any data can be useful. Refs. #2909.
2023-02-15 16:46:59 +03:00
Erik van den Brink
3bdb3a87b8 rpcclient: expose endpoint (fixes #2912) 2023-02-15 08:29:44 +01:00
Anna Shaleva
25ed5fcd60 vm: allow custom limit duing Map\Array\Struct deserialization 2023-02-08 10:58:55 +03:00
Anna Shaleva
15fa65d30f vm: export stackitem's deserContext 2023-02-08 10:58:54 +03:00
Anna Shaleva
51a28fac8d rpcclient: adjust new subscriptions API doc
Add a warning about consequences of unproper notifications reading.
Ref. https://github.com/morphbits/neofs-cdn-edge-manager/pull/88#discussion_r1088007680.
2023-01-30 14:36:11 +03:00
Leonard Lyubich
d09158161e services/rpcsrv: Strengthen Server error channel's type
According to docs, `Server` uses provided error channel only to write
encountered error to it. In this case, there is no need to accept rw
channel to create `Server` instance. Strengthening the type to
write-only will allow the caller to ensure control of reading errors
from the provided channel.

The change is backward compatible since any `chan` is `chan<-`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2023-01-23 10:33:46 +03:00
Leonard Lyubich
cbdc6ba469 rpcclient/actor: Document RPCActor.CalculateNetworkFee method
`Actor.MakeUnsignedUncheckedRun` method imposes restriction to
`CalculateNetworkFee` method's implementations: `Hash` or `Size` methods
must not be called on the pointer to the given transaction.

Add docs to adjust described requirement.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2023-01-17 21:28:20 +03:00
Anna Shaleva
28927228f0 *: adjust subscription-related doc
Add a warning about received events modification where applicable.
2023-01-17 17:11:19 +03:00
Roman Khimov
40ef567527
Merge pull request #2872 from nspcc-dev/fix-missing-exception-in-applog
state: always marshal applog exception into JSON, fix #2869
2023-01-12 12:49:00 +07:00
Roman Khimov
babc44bfb3
Merge pull request #2871 from nspcc-dev/fix-shadowed-infinite-loop
Fix shadowed infinite loop
2023-01-12 12:48:39 +07:00
Roman Khimov
9a6fa84f70
Merge pull request #2867 from nspcc-dev/store-magic-in-db
core: store magic in the DB version, fix #2847
2023-01-11 20:42:00 +07:00
Roman Khimov
4e23695441
Merge pull request #2866 from nspcc-dev/findstates-ret-nothing
Fix findstates response difference
2023-01-11 20:41:10 +07:00
Roman Khimov
817f64f9fe state: always marshal applog exception into JSON, fix #2869
Fortunately, this is backwards-compatible in any direction. See
98b5e2b353 also.
2023-01-11 16:36:41 +03:00
Roman Khimov
8305389852 compiler: don't panic, return error for strange range loops
See #2870.
2023-01-11 16:21:40 +03:00
Roman Khimov
92acc71c80 compiler: create new locals for range loops when needed, fix #2855 2023-01-11 15:49:10 +03:00
Roman Khimov
1c4c783c3a core: store magic in the DB version, fix #2847 2023-01-11 12:05:05 +03:00
Roman Khimov
e780037cb4 rpcsrv: use valid stateroot hash in failing tests
Just in case.
2023-01-11 11:23:47 +03:00
Roman Khimov
9718602ce3 rpcsrv: return empty set from findstates when there is no data
Fix #2863.
2023-01-11 11:21:58 +03:00
Roman Khimov
584675ec23 state: optimize NEP17Transfer struct
We have both from and to here, so technically we can either drop the neg/neg
trick from the processTokenTransfer() or drop one field from the structure
(the other side is a part of the key). Drop the field since this can make the
DB a bit more compact. Change Amount to be a pointer along the way since
that's the "native" thing for big.Int, we've used non-pointer field
specifically to avoid Neg/Neg problems, but it looks like this is not
necessary.

This structure is only used by the RPC server and I doubt anyone uses it via
the *Blockchain.
2023-01-10 22:51:45 +03:00
Roman Khimov
dfd4f6978f bigint: don't reallocate big.Int in ToBytes(), fix #2864
In some cases n.Add() can reuse the []Word buffer and n.Sub() reallocate it
away. If that happens, we're out of luck with 0.99.0+ versions (since
3945e81857). I'm not sure why it does that, bit
width doesn't change in most of the cases and even if it does, we still have
enough of it in cap() to hold the old Abs() value (when we have a negative
value we in fact decreate its Abs() first and increase it back
afterwards). Still, that's what we have.

So when we have processTokenTransfer() doing Neg/Neg in-place its value is not
affected, but the original []Word bits that are reused by amount value are
(they're shared initially, Amount: *amount).

name                   old time/op    new time/op    delta
ToPreallocatedBytes-8    65.8ns ± 2%    45.6ns ± 2%  -30.73%  (p=0.008 n=5+5)

name                   old alloc/op   new alloc/op   delta
ToPreallocatedBytes-8     0.00B          0.00B          ~     (all equal)

name                   old allocs/op  new allocs/op  delta
ToPreallocatedBytes-8      0.00           0.00          ~     (all equal)
2023-01-10 22:51:45 +03:00
Roman Khimov
4e6ce9c3e6
Merge pull request #2846 from nspcc-dev/fix-getblockbyindex-req
network: do not allow to request invalid block count
2022-12-29 04:04:01 +07:00
Anna Shaleva
9b364aa7ee network: do not allow to request invalid block count
The problem is in peer disconnection due to invalid GetBlockByIndex
payload (the logs are from some patched neo-go version):
```
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        INFO        new peer connected        {"addr": "10.78.69.115:50846", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        WARN        peer disconnected        {"addr": "10.78.69.115:50846", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.490Z        INFO        started protocol        {"addr": "10.78.69.115:50846", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.491Z        INFO        new peer connected        {"addr": "10.78.69.115:50856", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        WARN        peer disconnected        {"addr": "10.78.69.115:50856", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        INFO        started protocol        {"addr": "10.78.69.115:50856", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.492Z        INFO        new peer connected        {"addr": "10.78.69.115:50858", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.493Z        INFO        started protocol        {"addr": "10.78.69.115:50858", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.493Z        WARN        peer disconnected        {"addr": "10.78.69.115:50858", "error": "invalid block count", "peerCount": 2}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        INFO        new peer connected        {"addr": "10.78.69.115:50874", "peerCount": 3}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        INFO        started protocol        {"addr": "10.78.69.115:50874", "userAgent": "/NEO-GO:1.0.0/", "startHeight": 0, "id": 1339571820}
дек 15 16:02:39 glagoli neo-go[928530]: 2022-12-15T16:02:39.494Z        WARN        peer disconnected        {"addr": "10.78.69.115:50874", "error": "invalid block count", "peerCount": 2}
```

GetBlockByIndex payload can't be decoded, and the only possible cause
is zero (or <-1, but it's probably not the case) block count requested.

Error is improved as far.
2022-12-28 13:04:56 +03:00
Anna Shaleva
c0a453a53b network: adjust requestBlocs logic
If the lastQueued block index is the same as the one we'd like to
request in payload, then we need to increment the payload's count.
2022-12-28 12:50:30 +03:00
Evgeniy Stratonikov
db977ce38d vm: add fuzz test for ParseMultiSigContract
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2022-12-28 11:50:15 +03:00
Evgeniy Stratonikov
affe1ecb4f encoding: add bigint fuzz test
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2022-12-28 11:50:13 +03:00
Roman Khimov
3339824fe9 native: simplify code somewhat, get Metadata() only once 2022-12-16 23:48:04 +03:00
Roman Khimov
5d478b5514 native: add old management deploy/update call flags to Aspidochelone
d5a9af5860 is incompatible with the NeoFS
mainnet sidechain, so we add the old logic to the pre-Aspidochelone
behaviour. Changing flags at newMethodAndPrice() is a bit less convenient
unfortunately because this will affect interop validity checks, so let's have
this kludge here.
2022-12-16 23:45:47 +03:00
Roman Khimov
3e714a348e
Merge pull request #2842 from nspcc-dev/stringer
*: use zap.Stringer instead of zap.String where it can be used
2022-12-13 22:12:18 +07:00
Roman Khimov
e79dec15f9 *: use zap.Stringer instead of zap.String where it can be used
It's a bit more efficient in case we're not logging the message (mostly for
debug), makes the code somewhat simpler as well.
2022-12-13 12:44:54 +03:00
Roman Khimov
c1afaf8ea8 rpcsrv: use more robust NEP-11 divisibility check
It's more heavyweight at the same time, but should be OK for the
purpose. Inspired by https://github.com/neo-project/neo-modules/issues/754#issuecomment-1345176986
2022-12-10 11:45:43 +03:00
Roman Khimov
3ce1fc41a4 config: fix the default P2P ping settings
54c2aa8582 broke them, they're in seconds and we
have a 90s timeout.
2022-12-07 21:29:40 +03:00
Roman Khimov
0d65071abc config: use string type for Port
6b4dd5703e made it to be a uint16 which was
somewhat important for RPC, but now it's irrelevant and the fact that it was a
string in some cases may lead to errors like these:

  failed to unmarshal config YAML: yaml: unmarshal errors:
    line 48: cannot unmarshal !!str `20011` into uint16
    line 52: cannot unmarshal !!str `40001` into uint16

So for maximum backwards compatibility we better have string here and
eventually it'll be deleted anyway.
2022-12-07 21:21:05 +03:00
Roman Khimov
4e7cee4e12 config: replace VerifyBlocks with SkipBlockVerification
It directly affects node security and the default here MUST BE the safe choice
which is to do the verification. Otherwise it's just dangerous, absent any
VerifyBlocks configuration we'll get an insecure node. This option is not
supposed to be frequently used and it doesn't affect the ability to process
blocks, so breaking compatibility (in a safe manner) should be OK here.
2022-12-07 17:35:56 +03:00
Roman Khimov
7589733017 config: add a special Blockchain type to configure Blockchain
And include some node-specific configurations there with backwards
compatibility. Note that in the future we'll remove Ledger's
fields from the ProtocolConfiguration and it'll be possible to access them in
Blockchain directly (not via .Ledger).

The other option tried was using two configuration types separately, but that
incurs more changes to the codebase, single structure that behaves almost like
the old one is better for backwards compatibility.

Fixes #2676.
2022-12-07 17:35:53 +03:00
Roman Khimov
199a6de737 Merge pull request #2838 from nspcc-dev/neo-misuse
*: fix Neo and NeoGo misuses
2022-12-07 21:34:27 +07:00
Anna Shaleva
2d6db6f5c4 vm: adjust stale comment 2022-12-07 17:30:02 +03:00
Anna Shaleva
82221b0ca7 *: fix Neo and NeoGo misuses 2022-12-07 17:29:09 +03:00
Roman Khimov
b1f1405f42
Merge pull request #2836 from nspcc-dev/fix-subs
rpcclient: fix filtered naive subscriptions receiver
2022-12-07 21:17:07 +07:00
Anna Shaleva
2c0844221a rpcclient: fix filtered naive subscriptions receiver
We should return the filter itself instead of pointer.
2022-12-07 16:56:16 +03:00
Roman Khimov
c4d519f549
Merge pull request #2837 from nspcc-dev/350-compat
native: make management compatible with C# node 3.5.0
2022-12-07 20:19:04 +07:00
Roman Khimov
236e633ee4 native: make management compatible with C# node 3.5.0
It doesn't store id->hash mappings for native contracts. We need blockchain's
GetContractScriptHash to serve both anyway, so it was changed a bit. The only
other direct user of native.GetContractScriptHash is the VM CLI, but I doubt
anyone will use it for native contracts (they have ~zero VM code anyway).
2022-12-07 15:13:17 +03:00
Roman Khimov
883c6c5286 config: add Consensus subsection for dBFT config, fix #2677
Old UnlockWallet is still supported and works just fine.
2022-12-07 13:20:21 +03:00
Roman Khimov
730849a1cd config: move common part of notary/state into InternalService 2022-12-07 13:18:50 +03:00
Roman Khimov
90a85259a8
Merge pull request #2827 from nspcc-dev/multiaddr
network: support multiple bind addresses
2022-12-07 17:15:56 +07:00
Anna Shaleva
54c2aa8582 config: move P2P options to a separate config section
And convert time-related settings to a Duration format along the way.
2022-12-07 13:06:05 +03:00
Anna Shaleva
9cf6cc61f4 network: allow multiple bind addresses for server
And replace Transporter.Address() with Transporter.HostPort() along the way.
2022-12-07 13:06:03 +03:00
Anna Shaleva
9087854b1e services: allow multiple listen addresses for simple services
And deprecate the old way of Address/Port configuration for these services.
2022-12-07 11:23:01 +03:00
Roman Khimov
cad0fab704
Merge pull request #2828 from nspcc-dev/rpcwrapper-structures
Handle structures in the RPC wrapper generator
2022-12-06 21:40:16 +07:00
Anna Shaleva
6b4dd5703e config: unify BasicService config within existing services
There are no changes visible from the user side (at least for those
users who doesn't put Prometheus's or pprof's port in quotes), just
internal refactoring. From now and on, BasicService configuration is
used by RPC server config, TLS for RPC server, pprof and Prometheus.
2022-12-06 16:35:09 +03:00
Roman Khimov
e5aa5ca294 rpcbinding: improve error reporting in generated code 2022-12-06 13:15:10 +03:00
Roman Khimov
ec5ebc8c18 rpcbinding: improve indentation for internal unwrappers 2022-12-06 13:15:10 +03:00
Roman Khimov
c058ab5604 rpcbinding: handle more complex non-structured types 2022-12-06 13:15:10 +03:00
Roman Khimov
ce67e6795e rpcbinding: properly support maps
I'm not sure that map with a `*big.Int` key is a good one, but it can work
this way.
2022-12-06 13:15:10 +03:00
Roman Khimov
8e0be6e7a5 rpcbinding: improve indentation for internal array code 2022-12-06 13:15:10 +03:00
Roman Khimov
0214628127 rpcbinding: generate bindings using new extended type data 2022-12-06 13:15:10 +03:00
Roman Khimov
2bcf3a4ad5 compiler: unwrap pointers for debug types
We have almost no difference between Type and *Type.
2022-12-06 13:15:10 +03:00
Roman Khimov
d7fafea328 compiler: handle ledger/management enums better for debug
Fix scAndVMInteropTypeFromExpr(), these types are not structures at all.
2022-12-06 13:15:10 +03:00
Roman Khimov
0f61a13006 compiler: Contract type is in the management package 2022-12-06 13:15:10 +03:00
Roman Khimov
b9d20b32e9 compiler: push additional type data into the bindings file
Structures/arrays and maps.
2022-12-06 13:15:10 +03:00
Roman Khimov
cceb044a02 config: add LogLevel option to control logging level
And update zap, because the old version doesn't have zapcore.ParseLevel().
2022-12-05 14:58:16 +03:00
Roman Khimov
b7518423d4
Merge pull request #2829 from nspcc-dev/ms-per-block
TimePerBlock config
2022-12-05 14:27:55 +07:00
Roman Khimov
c2adbf768b config: add TimePerBlock to replace SecondsPerBlock
It's more generic and convenient than MillisecondsPerBlock. This setting is
made in backwards-compatible fashion, but it'll override SecondsPerBlock if
both are used. Configurations are specifically not changed here, it's
important to check compatibility.

Fixes #2675.
2022-12-02 19:52:14 +03:00
Roman Khimov
d909cab4a4 rpcclient/management: add new methods 2022-12-02 10:57:05 +03:00
Roman Khimov
4ce968e4d0 *: update interops, add tests for new management methods 2022-12-02 10:57:05 +03:00
Roman Khimov
cb5c18dc73 interop: add getContractByID and getContractHashes to management 2022-12-02 10:54:45 +03:00
Roman Khimov
4a626f505e core: drop old STContractID data
We have it in the ContractManagement now.
2022-12-02 10:54:45 +03:00
Roman Khimov
970862765d native: implement management.getContractById
Follow neo-project/neo#2807. Notice that this data is not cached, our previous
implementation wasn't too and it shouldn't be a problem (not on the hot path).
2022-12-02 10:21:08 +03:00
Roman Khimov
6c68da7a52 rpcsrv: add block 20 hash comparison test
Makes updating it somewhat easier.
2022-12-02 10:21:08 +03:00
Roman Khimov
f6a9969fa8
Merge pull request #2770 from nspcc-dev/push-bool
vm: add PUSHT and PUSHF opcodes
2022-12-02 14:20:11 +07:00
Roman Khimov
389bdfd1b6
Merge pull request #2719 from nspcc-dev/eval
runtime: implement System.Runtime.LoadScript, fix #2701
2022-12-02 14:08:40 +07:00
Roman Khimov
93cec0bba5
Merge pull request #2786 from nspcc-dev/simplify-getversion
Simplify result.Version
2022-12-02 14:07:55 +07:00
Roman Khimov
3ef66239a8
Merge pull request #2814 from nspcc-dev/startup-time
Optimize startup time and memory usage
2022-12-02 14:06:23 +07:00
Anna Shaleva
2b140631f4 rpc: adjust Boolean handling in ExpandArrayIntoScript
Use new PUSHT and PUSHF opcodes for Boolean scparameter.
2022-12-02 10:02:33 +03:00
Anna Shaleva
1250e82c2a vm: add PUSHT and PUSHF opcodes
Port https://github.com/neo-project/neo-vm/pull/497.
2022-12-02 10:02:33 +03:00
Roman Khimov
6912695b9d
Merge pull request #2826 from nspcc-dev/check-for-max-block-sysfee
core: filter out txes with system fee > MaxBlockSystemFee
2022-11-29 14:57:10 +07:00
Roman Khimov
6847e1760c core: filter out txes with system fee > MaxBlockSystemFee
They can stay in the memory pool forever because consensus process will never
accept these transactions (and maybe even block consensus process at all).
2022-11-29 10:31:00 +03:00
Anna Shaleva
6d3cc0f7e7 core: extend error text for storage context in dynamic script 2022-11-28 17:17:36 +03:00
Anna Shaleva
0ceae612c6 vm: improve failed SYSCALL logging
For better user experience.
2022-11-28 17:16:27 +03:00
Roman Khimov
07e32e221d *: update interops, add System.Runtime.LoadScript wrapper check 2022-11-25 15:16:27 +03:00
Roman Khimov
de2de986a7 interop: add runtime.LoadScript 2022-11-25 15:11:49 +03:00
Roman Khimov
4e26f4b26e runtime: implement System.Runtime.LoadScript, fix #2701 2022-11-25 15:11:49 +03:00
Roman Khimov
1c38b45074 core: don't always store all hashes in memory
We're paging these hashes, so we need a previous full page and a current one
plus some cache for various requests. Storing 1M of hashes is 32M of memory
and it grows quickly. It also seriously affects node startup time, most of
what it's doing is reading these hashes, the longer the chain the more time it
needs to do that.

Notice that this doesn't change the underlying DB scheme in any way.
2022-11-25 14:30:51 +03:00
Roman Khimov
0ad6e295ea core: make GetHeaderHash accept uint32
It should've always been this way because block indexes are uint32.
2022-11-25 14:30:51 +03:00
Roman Khimov
16db4746cc core: drop another useless genesis-specific header handling branch
If we only have genesis block (or <2000 headers) then we might as well use
generic logic below with zero targetHash because genesis block has zero
PrevHash (and its hash will naturally be the last on the chain going
backwards).
2022-11-25 13:08:03 +03:00
Roman Khimov
83e2ca054f core: simplify header hash list restoration logic
We don't care about header, we only need hashes here.
2022-11-25 13:08:03 +03:00
Roman Khimov
ff284d5d5c core: remove genesis-specific header hash logic from init
The code below will do the same thing (via CreateGenesisBlock, but it doesn't
matter) generically, so this branch is useless.
2022-11-25 13:08:03 +03:00
Roman Khimov
027e94fbde compiler: don't push overrides for unexported methods
They're internal, no one cares about them externally.
2022-11-23 17:05:37 +03:00
Roman Khimov
327dfb8dbf rpcsrv: add wsclient error message test for #2818
Now it can be done easily.
2022-11-23 12:24:20 +03:00
Roman Khimov
2591c39500 rpcsrv: make websocket client limit configurable 2022-11-23 12:19:49 +03:00
Roman Khimov
fb09670fd1 rpcclient: extract more detailed server-side on WS connection problem
If available. Fixes #2818.
2022-11-23 12:19:35 +03:00
Roman Khimov
ab0b23625b
Merge pull request #2817 from nspcc-dev/always-return-hash-vub-from-sender
Some actor/waiter interaction fixes
2022-11-23 14:37:38 +07:00
Roman Khimov
fd04b2befd actor: don't abort waiter on "already exists" error
It can happen in many cases of distributed tx generation/submission, we can
just wait normally in this case and there will be some proper result.
2022-11-23 10:07:37 +03:00
Roman Khimov
66ddeccdad
Merge pull request #2813 from nspcc-dev/fix-state-reset
core: fix broken state reset
2022-11-23 13:43:42 +07:00
Anna Shaleva
f3ef2890f0 core: check headers at the proper state on state reset
And fix the comment along the way.
2022-11-23 09:16:33 +03:00
Roman Khimov
cd6bb68246 actor: check for tx after the subscription in wsWaiter, fix #2805
Don't wait for VUB block, solve this race immediately.
2022-11-22 17:28:55 +03:00
Roman Khimov
c95d140113 rpcclient: always return tx hash from sendrawtransaction
Let upper-layer APIs like actor.Send() return it as well. Server can return
"already exists" which is an error and yet at the same time a very special
one, in many cases it means we can proceed with waiting for the TX to settle.
2022-11-22 15:18:37 +03:00
Anna Shaleva
b27a9bcf95 core: adjust info message for proper-stated chains
Make it prettier for those cases when `db reset` command was called
after interrupted reset.
2022-11-22 11:53:39 +03:00
Anna Shaleva
b82374823e core: increase persist batch size for reset storage changes 2022-11-22 11:53:39 +03:00
Anna Shaleva
bdc42cd595 core: reset blocks, txs and AERs in several stages
Sometimes it can be hard to persist all changes at ones, the process
can take almost all RAM and a lot of time. Here's the example of reset
for mainnet from 2.4M to 1:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:16:48.236+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2022-11-20T17:16:48.236+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2022-11-20T17:16:48.237+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2022-11-20T17:16:48.237+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:16:48.240+0300	INFO	restoring blockchain	{"version": "0.2.6"}
2022-11-20T17:16:48.297+0300	INFO	initialize state reset	{"target height": 1}
2022-11-20T17:16:48.300+0300	INFO	trying to reset blocks, transactions and AERs
2022-11-20T17:19:29.313+0300	INFO	blocks, transactions ans AERs are reset	{"took": "2m41.015126493s", "keys": 3958420}
...
```
To avoid OOM killer, split blocks reset into multiple stages. It increases
operation time due to intermediate DB persists, but makes things cleaner, the
result for almost the same DB height with the new approach:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:39:42.023+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2022-11-20T17:39:42.023+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2022-11-20T17:39:42.023+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2022-11-20T17:39:42.023+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:39:42.026+0300	INFO	restoring blockchain	{"version": "0.2.6"}
2022-11-20T17:39:42.071+0300	INFO	initialize state reset	{"target height": 1}
2022-11-20T17:39:42.073+0300	INFO	trying to reset blocks, transactions and AERs
2022-11-20T17:40:11.735+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 1, "took": "29.66363737s", "keys": 210973}
2022-11-20T17:40:33.574+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 2, "took": "21.839208683s", "keys": 241203}
2022-11-20T17:41:29.325+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 3, "took": "55.750698386s", "keys": 250593}
2022-11-20T17:42:12.532+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 4, "took": "43.205892757s", "keys": 321896}
2022-11-20T17:43:07.978+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 5, "took": "55.445398156s", "keys": 334822}
2022-11-20T17:43:35.603+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 6, "took": "27.625292032s", "keys": 317131}
2022-11-20T17:43:51.747+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 7, "took": "16.144359017s", "keys": 355832}
2022-11-20T17:44:05.176+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 8, "took": "13.428733899s", "keys": 357690}
2022-11-20T17:44:32.895+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 9, "took": "27.718548783s", "keys": 393356}
2022-11-20T17:44:51.814+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 10, "took": "18.917954658s", "keys": 366492}
2022-11-20T17:45:07.208+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 11, "took": "15.392642196s", "keys": 326030}
2022-11-20T17:45:18.776+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 12, "took": "11.568255716s", "keys": 299884}
2022-11-20T17:45:25.862+0300	INFO	last batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 13, "took": "7.086079594s", "keys": 190399}
2022-11-20T17:45:25.862+0300	INFO	blocks, transactions ans AERs are reset	{"took": "5m43.791214084s", "overall persisted keys": 3966301}
...
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
d67f0df516 core: reset block headers together with header height info
We need to keep the headers information consistent with header batches
and headers. This comit fixes the bug with failing blockchain
initialization on recovering from state reset interrupted after the
second stage (blocks/txs/AERs removal):
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T16:28:29.437+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T16:28:29.440+0300	INFO	restoring blockchain	{"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: could not get header 1898cd356a4a2688ed1c6c7ba1fd6ba7d516959d8add3f8dd26232474d4539bd: key not found
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
283da8f599 core: use DAO-provided block height during during state reset
Don't use cache because it's not yet initialized. Also, perform
safety checks only if state reset wasn't yet started. These fixes
alloww to solve the following problem while recovering from
interrupted state reset:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T15:51:31.431+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T15:51:31.434+0300	INFO	restoring blockchain	{"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: current block height is 0, can't reset state to height 83000
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
7d55bf2cc1 core: log persisted storage item batches count during state reset 2022-11-22 11:53:39 +03:00
Anna Shaleva
f52451e582 core: fix state reset with broken contract
Sync up with #2802, bad contract -> no contract ID at all.
2022-11-22 11:53:39 +03:00
Anna Shaleva
ecda07736e core: stop storage items reset after any seek error 2022-11-22 11:53:39 +03:00
Anna Shaleva
bfe7aeae7b core: stop storage items reset after the first persist error
It's a bug, we mustn't continue if something bad had happend on persist,
otherwise this error will be overwritten by subsequent successfull persist.
2022-11-22 11:53:39 +03:00
Anna Shaleva
235518eb6c core: reset batch counter to zero after each persist in resetStateInternal
It's a bug, otherwise we'll persist each storage item after 10K-th one,
that's the reason of abnormous long storage items resetting stage.
2022-11-22 11:53:39 +03:00
Anna Shaleva
9f23fafc03 core: improve logging of resetStateInternal
Inform when starting subsequent stage, inform about keys persisted.
2022-11-22 11:53:36 +03:00
Roman Khimov
0039615ae3
Merge pull request #2816 from nspcc-dev/fix-pointer-serialization
Fix pointer serialization
2022-11-20 22:59:52 +07:00
Roman Khimov
9ba18b5dfa stackitem: serialize/deserialize pointers, fix #2815
They of course can't be serialized, but in protected mode we still need to
handle them somehow.
2022-11-20 16:02:50 +03:00
Roman Khimov
48140320db
Merge pull request #2812 from nspcc-dev/improve-vm-context-handling
Improve vm istack/estack handling
2022-11-20 19:42:35 +07:00
Roman Khimov
ca9fde745b
Merge pull request #2809 from nspcc-dev/fix-subs
rpcsrv: do not block blockchain events receiver by subscription requests
2022-11-18 16:16:41 +07:00
Roman Khimov
8e7f65be17 vm: use proper estack for exception handler
v.estack might be some inner invoked contract and its stack must not be used
for exception handler set up by higher-order contract.
2022-11-18 11:36:38 +03:00
Roman Khimov
cb64957af5 vm: don't use Stack for istack
We don't use all of the Stack functionality for it, so drop useless methods
and avoid some interface conversions. It increases single-node TPS by about
0.9%, so nothing really important there, but not a bad change either. Maybe it
can be reworked again with generics though.
2022-11-18 11:35:29 +03:00
Anna Shaleva
4df9a5e379 rpcsrv: refactor subscribe routine
Move shutdown check after subsCounterLock is taken in the end of
`(s *Server) subscribe` in order to avoid extra locks holding.
2022-11-18 10:54:10 +03:00
Anna Shaleva
b7f19a54d5 services: fix chain locked by WS subscriptions handlers
Blockchain's subscriptions, unsubscriptions and notifications are
handled by a single notificationDispatcher routine. Thus, on attempt
to send the subsequent event to Blockchain's subscribers, dispatcher
can't handle subscriptions\unsubscriptions. Make subscription and
unsubscription to be a non-blocking operation for blockchain on the
server side, otherwise it may cause the dispatcher locks.

To achieve this, use a separate lock for those code that make calls
to blockchain's subscription API and for subscription counters on
the server side.
2022-11-18 09:30:12 +03:00
Roman Khimov
2bcb7bd06f compiler: don't use (*VM).Istack when it's not needed 2022-11-17 20:46:06 +03:00
Roman Khimov
b8c09f509f network: add random slight delay to connection attempts
Small (especially dockerized/virtualized) networks often start all nodes at
ones and then we see a lot of connection flapping in the log. This happens
because nodes try to connect to each other simultaneously, establish two
connections, then each one finds a duplicate and drops it, but this can be
different duplicate connections on other sides, so they retry and it all
happens for some time. Eventually everything settles, but we have a lot of
garbage in the log and a lot of useless attempts.

This random waiting timeout doesn't change the logic much, adds a minimal
delay, but increases chances for both nodes to establish a proper single
connection on both sides to only then see another one and drop it on both
sides as well. It leads to almost no flapping in small networks, doesn't
affect much bigger ones. The delay is close to unnoticeable especially if
there is something in the DB for node to process during startup.
2022-11-17 18:42:43 +03:00
Roman Khimov
075a54192c network: don't try too many connections
Consider mainnet, it has an AttemptConnPeers of 20, so may already have 3
peers and request 20 more, then have 4th connected and attemtp 20 more again,
this leads to a huge number of connections easily.
2022-11-17 18:03:04 +03:00
Roman Khimov
6bce973ac2 network: drop duplicationg check from handleAddrCmd()
It was relevant with the queue-based discoverer, now it's not, discoverer
handles this internally.
2022-11-17 17:42:36 +03:00
Roman Khimov
1c7487b8e4 network: add a timer to check for peers
Consider initial connection phase for public networks:
 * simultaneous connections to seeds
 * very quick handshakes
 * got five handshaked peers and some getaddr requests sent
 * but addr replies won't trigger new connections
 * so we can stay with just five connections until any of them breaks or a
   (long) address checking timer fires

This new timers solves the problem, it's adaptive at the same time. If we have
enough peers we won't be waking up often.
2022-11-17 17:32:05 +03:00
Anna Shaleva
e73c3c7ec4 services: adjust WS waiter test
Make it more stable.
2022-11-17 17:15:01 +03:00
Roman Khimov
23f118a1a9 network: rework discoverer/server interaction
* treat connected/handshaked peers separately in the discoverer, save
   "original" address for connected ones, it can be a name instead of IP and
   it's important to keep it to avoid reconnections
 * store name->IP mapping for seeds if and when they're connected to avoid
   reconnections
 * block seed if it's detected to be our own node (which is often the case for
   small private networks)
 * add an event for handshaked peers in the server, connected but
   non-handshaked ones are not really helpful for MinPeers or GetAddr logic

Fixes #2796.
2022-11-17 17:07:19 +03:00
Roman Khimov
6ba4afc977 network: consider handshaked peers only when comparing with MinPeers
We don't know a lot about non-handshaked ones, so it's safer to try more
connection.
2022-11-17 16:40:29 +03:00
Roman Khimov
ab0ff63ce1
Merge pull request #2804 from nspcc-dev/check-aer-sub
rpc: fix subscribers locking logic and properly drain poll-based waiter receiver
2022-11-17 04:24:35 +07:00
Anna Shaleva
1399496dfb rpcclient: refactor event-based waiting loop
Avoid receiver channels locks.
2022-11-16 23:57:00 +03:00
Anna Shaleva
95e23c8e46 actor: fix event-based tx awaiting
If VUB-th block is received, we still can't guaranty that transaction
wasn't accepted to chain. Back this situation by rolling back to a
poll-based waiter.
2022-11-16 23:44:31 +03:00
Anna Shaleva
6dbae7edc4 rpcclient: fix WS-client unsubscription process
Do not block subscribers until the unsubscription request to RPC server
is completed. Otherwise, another notification may be received from the
RPC server which will block the unsubscription process.

At the same time, fix event-based waiter. We must not block the receiver
channel during unsubscription because there's a chance that subsequent
event will be sent by the server. We need to read this event in order not
to block the WSClient's readloop.
2022-11-16 23:44:30 +03:00
Anna Shaleva
ddaba9e74d rpcsrv: fix "subscribe" parameters handling
If it's a subscription for AERs, we need to check the filter's state only
if it has been provided, otherwise filter is always valid.
2022-11-16 14:05:13 +03:00
Anna Shaleva
d043139b66 rpcsrv: adjust "subscribe" response error
Make it more detailed for better debugging experience.
2022-11-16 13:35:19 +03:00
Anna Shaleva
3f122fd591 rpcclient: adjust WS waiter error formatting
Follow the other errors formatting style.
2022-11-16 12:22:18 +03:00
Roman Khimov
822722bd2e native: ignore decoding errors during cache init
Bad contract -> no contract. Unfortunately we've got a broken
6f1837723768f27a6f6a14452977e3e0e264f2cc contract on the mainnet which can't
be decoded (even though it had been saved successfully), so this is a
temporary fix for #2801 to be able to start mainnet node after shutdown.
2022-11-16 12:00:28 +03:00
Roman Khimov
aef01bf663 vm: fix istack marshaling, fix #2799 2022-11-16 00:40:12 +03:00
Roman Khimov
90582faacd vm: save current stack slice when loading new context
v.estack is used throughout the code to work with estack, while ctx.sc.estack
is (theoretically) just a reference to it that is saved on script load and
restored to v.estack on context unload. The problem is that v.estack can grow
as we use it and can be reallocated away from its original slice (saved in the
ctx.sc.estack), so either ctx.sc.estack should be a pointer or we need to
ensure that it's correct when loading a new script. The second approach is a
bit safer for now and it fixes #2798.
2022-11-15 23:48:02 +03:00
Anna Shaleva
6f3a0a6b4c network: adjust warning for deposit expiration
Provide additional info for better user experience.
2022-11-15 14:16:34 +03:00
Roman Khimov
c67ee54566
Merge pull request #2792 from nspcc-dev/rpcwrapper-arrays
RPC wrapper for simple arrays
2022-11-15 13:08:25 +07:00
Roman Khimov
82c6ce218b rpcbinding: use binding condig to generate code for simple arrays
Part of #2767.
2022-11-14 13:01:13 +03:00
Roman Khimov
b5c79f4be3 unwrap: add a complete set of simple array unwrappers
Arrays of basic types should be covered completely.
2022-11-14 13:01:13 +03:00
Roman Khimov
c405092953 network: pre-filter transactions going into dbft
Drop some load from dbft loop during consensus process.
2022-11-11 15:32:51 +03:00
Roman Khimov
f78231fd9c
Merge pull request #2773 from nspcc-dev/state-reset
core: implement state reset
2022-11-10 22:26:43 +07:00
Anna Shaleva
e7effef052 core: reuse appendTokenTransferInfo from appendTokenTransfer 2022-11-10 18:08:18 +03:00
Anna Shaleva
bd6bb9e9e2 core: allow to reset blockchain state 2022-11-10 18:08:17 +03:00
Anna Shaleva
ec9317d5b4 core: fix typo 2022-11-10 17:58:06 +03:00
Roman Khimov
cb806d4233 result: drop Version (un)marshaler, we only need it for Protocol
Simplify code a bit.
2022-11-10 17:51:02 +03:00
Roman Khimov
8324a247d3 result: drop pre-0.99.0 compatibility code
0.99.0 is too old already.
2022-11-10 16:49:38 +03:00
Anna Shaleva
1dac45bbbb core: add ability to check whether blockchain is running 2022-11-10 16:47:04 +03:00
Roman Khimov
7f8a79ffaa result: drop deprecated Magic and StateRootInHeader from Version
It's more than a year now we have them deprecated.
2022-11-10 16:32:49 +03:00
Roman Khimov
4c9473872e
Merge pull request #2783 from nspcc-dev/rpcbindings-iterators
Initial iterator support for RPC bindings
2022-11-10 12:51:40 +07:00
Roman Khimov
145ebad90e binding: drop the only error condition from TemplateFromManifest
Simplify the interface, we do IsValid() check anyway in the CLI and it covers
this condition as well.
2022-11-09 18:13:45 +03:00
Roman Khimov
be02eea7b1 binding: precompile template, remove useless error condition 2022-11-09 18:13:45 +03:00
Roman Khimov
a7f86dcb7f rpcbinding: generate Expanded methods for iterators
Refs. #2768.
2022-11-09 18:13:45 +03:00
Roman Khimov
d569fe01e6 rpcbinding: initial support for iterators, see #2768
Already better than stackitem.Item.
2022-11-09 18:13:45 +03:00
Roman Khimov
d0b1015b2c rpcsrv: provide Prometheus histograms for calls
They're a bit more useful and they're naturally grouped under rpc
prefix. Simple counters will be removed eventually to avoid duplication.
2022-11-09 13:26:45 +03:00
Roman Khimov
3247aa40a7 rpcsrv: allow any Origin in WS connections if EnableCORSWorkaround
Break origin checks even more. Alternative to #2772.
2022-11-09 09:28:09 +03:00
Roman Khimov
e3c503cee9
Merge pull request #2781 from nspcc-dev/btcd-update
Update 256k1 dependency
2022-11-08 23:50:19 +07:00
Roman Khimov
1e07cacc79 rpcsrv: fix test
client_test.go:1935:
                Error Trace:    /home/rik/dev/neo-go/pkg/services/rpcsrv/client_test.go:1935
                Error:          Should NOT be empty, but was 00000000-0000-0000-0000-000000000000
                Test:           TestClient_Iterator_SessionConfigVariations/sessions_disabled

It's obviously empty, since we have sessions disabled, but it was not
considered to be empty in testify 1.7.0, now it is, see 840cb80149
2022-11-08 18:31:25 +03:00
Roman Khimov
8746d9877e keys: move from btcd/btcec to decred/secp256k1 package
There is a security issue found in github.com/btcsuite/btcd that we don't care
about (we're only using 256k1 implementation), but GitHub complains about
it. We could update to github.com/btcsuite/btcd/btcec/v2, but it's now just a
thin wrapper over github.com/decred/dcrd/dcrec/secp256k1/v4, so we better use
it directly.
2022-11-08 17:59:59 +03:00
Roman Khimov
69d8905ad9 rpcbinding: exclude onNEPXXPayment methods from wrappers
They make no sense there.
2022-11-08 17:01:36 +03:00
Roman Khimov
130608ac67 rpcbinding: support writer-only wrappers
"verify" contract doesn't have any safe methods.
2022-11-08 17:01:36 +03:00
Roman Khimov
df29008a50 rpcbinding: add GAS testcase, fix methodless wrappers
* strip NEP-XX methods before going into generator to avoid unused imports
 * nepXX.Invoker types already include Call
 * always import util, it's used for Hash
2022-11-08 17:01:36 +03:00
Roman Khimov
aeb61fb61d rpcbinding: generate ASSERT for bool-returning methods
It's a common pattern.
2022-11-08 17:01:36 +03:00
Roman Khimov
2a4a5ab479 rpcbinding: support simple wrappers for writer methods
Fixes #2769.
2022-11-08 17:01:36 +03:00
Roman Khimov
bb47d971dc rpcclient/nep11: make DivisibleWriter part reusable
Same rationale as for BaseWriter or nep17.TokenWriter, otherwise it's hard to
build on top of plain Divisible structure.
2022-11-07 10:44:39 +03:00
Roman Khimov
b590d4ca04
Merge pull request #2766 from nspcc-dev/rpc-wrapper-autogen
Rpc wrapper autogeneration
2022-10-28 11:54:47 +07:00
Roman Khimov
02ce59cfd5 binding: avoid name conflicts with Go keywords
And clashing one name on another after rename.
2022-10-27 22:57:49 +03:00
Roman Khimov
617c31093f smartcontract: initial rpcbinding implementation, fix #2705
It can do some unwrapping and reuse nepXX packages. It only uses manifest data
at the moment, see #2767, #2768, #2769.
2022-10-27 22:57:49 +03:00
Roman Khimov
f0abc035af unwrap: add PublicKey to unwrap public keys
We have this type in NEP-14 directly.
2022-10-27 18:32:00 +03:00
Roman Khimov
e0eff94094 standard: correct Comply* comments 2022-10-27 18:32:00 +03:00
Roman Khimov
64b603b056 standard: export standard definitions
Make Comply* functions useful and expose standard definitions for some reuse.
2022-10-27 18:32:00 +03:00
Anna Shaleva
4fc11c2924 rpc: revert deprecated subscriptions API changes
Revert deprecated subscriptions-related method signature changed in
673a495527, 8e84bb51d5
and d7c1f3eac7.
2022-10-26 15:32:54 +03:00
Anna Shaleva
5811687836 rpc: fix bug in Actor's waiter
Execution events are followed by block events, not vise versa, thus,
we can wait until VUB block to be accepted to be sure that
transaction wasn't accepted to chain.
2022-10-26 15:32:54 +03:00
Anna Shaleva
0a5905390c rpc: refactor WSClient subscriptions API
Make it more specific, close #2756.
2022-10-26 15:32:54 +03:00
Anna Shaleva
2a53db42af neorpc: adjust and extend event filters documentation 2022-10-26 15:32:54 +03:00
Roman Khimov
396f56f792 binding: drop unused lowerFirst 2022-10-26 12:47:39 +03:00
Roman Khimov
4191b18728 binding: make TemplateFromManifest more reusable
Other template generators can make use of it.
2022-10-26 12:43:48 +03:00
Roman Khimov
d6d4f07280 binding: always import some packages and do it outside
If the contract has no methods, it's probably a broken one.
2022-10-26 12:40:59 +03:00
Roman Khimov
6af71755c1
Merge pull request #2762 from nspcc-dev/optimize-emit-bool
emit: optimize Bool GAS cost
2022-10-26 15:06:05 +07:00
Roman Khimov
4e58bd7411 compiler: use shorter and cheaper sequence to convert to Boolean 2022-10-25 18:20:55 +03:00
Roman Khimov
5d43367082 emit: optimize Bool GAS cost
NOT is 1 byte shorter and 2048 times cheaper than CONVERT. Inspired by
neo-project/neo-vm#493.
2022-10-25 13:08:33 +03:00
Roman Khimov
e19d867d4e
Merge pull request #2761 from nspcc-dev/fancy-getaddr
Fancy getaddr
2022-10-25 16:51:38 +07:00
Roman Khimov
28f54d352a network: do getaddr requests periodically, fix #2745
Every 1000 blocks seems to be OK for big networks (that only had done some
initial requests previously and then effectively never requested addresses
again because there was a sufficient number of addresses), won't hurt smaller
ones as well (that effectively keep doing this on every connect/disconnect,
peer changes are very rare there, but when they happen we want to have some
quick reaction to these changes).
2022-10-24 15:10:51 +03:00
Roman Khimov
9efc110058 network: it is 42
32 is a very good number, but we all know 42 is a better one. And it can even
be proven by tests with higher peaking TPS values.

You may wonder why is it so good? Because we're using packet-switching
networks mostly and a packet is a packet almost irrespectively of how bit it
is. Yet a packet has some maximum possible size (hi, MTU) and this size most
of the time is 1500 (or a little less than that, hi VPN). Subtract IP header
(20 for IPv4 or 40 for IPv6 not counting options), TCP header (another 20) and
Neo message/payload headers (~8 for this case) and we have just a little more
than 1400 bytes for our dear hashes. Which means that in a single packet most
of the time we can have 42-44 of them, maybe 45. Choosing between these
numbers is not hard then.
2022-10-24 14:44:19 +03:00
Roman Khimov
9d6b18adec network: drop minPoolCount magic constant
We have AttemptConnPeers that is closely related, the more we have there the
bigger the network supposedly is, so it's much better than magic minPoolCount.
2022-10-24 14:36:10 +03:00
Roman Khimov
af24051bf5 network: sleep a bit before retrying reconnects
If Dial() is to exit quickly we can end up in a retry loop eating CPU.
2022-10-24 14:34:48 +03:00
Roman Khimov
a17d9f80a4
Merge pull request #2759 from nspcc-dev/avoid-db-lookup-in-hasblock
core: optimize HasBlock check for recent blocks
2022-10-24 14:33:38 +07:00
Roman Khimov
f42b8e78fc
Merge pull request #2758 from nspcc-dev/check-inflight-tx-invs
network: check inv against currently processed transactions
2022-10-24 14:16:33 +07:00
Roman Khimov
52df03ecbc
Merge pull request #2757 from nspcc-dev/batched-getdata
network: batch getdata replies
2022-10-24 14:15:55 +07:00
Roman Khimov
0551ddff4e
Merge pull request #2749 from nspcc-dev/rpc-waiter
rpc: implement transaction awaiting
2022-10-24 13:08:06 +07:00
Anna Shaleva
3cccc89dac rpc: add Wait wrapper to Notary actor 2022-10-24 06:09:36 +03:00
Anna Shaleva
1a6f1c805c rpc: fix race in TestWSClientEvents
```
2022-10-21T08:59:45.2219797Z === RUN   TestWSClientEvents/default_ntf_channel
2022-10-21T08:59:45.2219901Z ==================
2022-10-21T08:59:45.2220017Z WARNING: DATA RACE
2022-10-21T08:59:45.2220177Z Write at 0x00c000c82778 by goroutine 371:
2022-10-21T08:59:45.2220580Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.TestWSClientEvents.func2()
2022-10-21T08:59:45.2221112Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient_test.go:171 +0x1c4
2022-10-21T08:59:45.2221244Z   testing.tRunner()
2022-10-21T08:59:45.2221617Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1439 +0x213
2022-10-21T08:59:45.2221759Z   testing.(*T).Run.func1()
2022-10-21T08:59:45.2222124Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x47
2022-10-21T08:59:45.2222138Z
2022-10-21T08:59:45.2222308Z Previous read at 0x00c000c82778 by goroutine 37:
2022-10-21T08:59:45.2222694Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.(*Client).StateRootInHeader()
2022-10-21T08:59:45.2223151Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/rpc.go:1104 +0xb0
2022-10-21T08:59:45.2223482Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.(*WSClient).wsReader()
2022-10-21T08:59:45.2224077Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient.go:210 +0x651
2022-10-21T08:59:45.2224416Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.NewWS.func2()
2022-10-21T08:59:45.2224892Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient.go:149 +0x39
2022-10-21T08:59:45.2224901Z
2022-10-21T08:59:45.2225049Z Goroutine 371 (running) created at:
2022-10-21T08:59:45.2225182Z   testing.(*T).Run()
2022-10-21T08:59:45.2225548Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x724
2022-10-21T08:59:45.2225911Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.TestWSClientEvents()
2022-10-21T08:59:45.2226408Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient_test.go:167 +0x404
2022-10-21T08:59:45.2226539Z   testing.tRunner()
2022-10-21T08:59:45.2226900Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1439 +0x213
2022-10-21T08:59:45.2227042Z   testing.(*T).Run.func1()
2022-10-21T08:59:45.2227398Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x47
2022-10-21T08:59:45.2227406Z
2022-10-21T08:59:45.2227552Z Goroutine 37 (finished) created at:
2022-10-21T08:59:45.2227851Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.NewWS()
2022-10-21T08:59:45.2228327Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient.go:149 +0x6fb
2022-10-21T08:59:45.2228843Z   github.com/nspcc-dev/neo-go/pkg/rpcclient.TestWSClientEvents.func2()
2022-10-21T08:59:45.2229434Z       /home/runner/work/neo-go/neo-go/pkg/rpcclient/wsclient_test.go:168 +0x131
2022-10-21T08:59:45.2229569Z   testing.tRunner()
2022-10-21T08:59:45.2229930Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1439 +0x213
2022-10-21T08:59:45.2230069Z   testing.(*T).Run.func1()
2022-10-21T08:59:45.2230424Z       /opt/hostedtoolcache/go/1.18.7/x64/src/testing/testing.go:1486 +0x47
2022-10-21T08:59:45.2230526Z ==================
2022-10-21T08:59:45.2230703Z     wsclient_test.go:186:
2022-10-21T08:59:45.2230988Z         	Error Trace:	wsclient_test.go:186
2022-10-21T08:59:45.2231209Z         	Error:      	Should be true
2022-10-21T08:59:45.2231536Z         	Test:       	TestWSClientEvents/default_ntf_channel
2022-10-21T08:59:45.2231812Z     testing.go:1312: race detected during execution of test
```
2022-10-24 06:09:36 +03:00
Anna Shaleva
d2a9e9120d rpc: extend Waiter interface to wait for several txs with context 2022-10-24 06:09:36 +03:00
Anna Shaleva
6b216050f3 rpc: add compat tests for RPC* interfaces and Client implementations 2022-10-24 06:09:36 +03:00
Anna Shaleva
00d44235c1 rpc: add tests for RPC waiters 2022-10-24 06:09:36 +03:00
Anna Shaleva
388112dcaa rpc: mark old WSClient's SubscribeFor* methods as deprecated 2022-10-24 06:09:36 +03:00
Anna Shaleva
5b81cb065f rpc: refactor waiter-related actor code 2022-10-24 06:09:36 +03:00
Anna Shaleva
8e84bb51d5 rpc: add "till" filter to WS block events 2022-10-24 06:09:36 +03:00
Anna Shaleva
345d48d051 rpc: improve WS-based transaction awaiting
Subscribe only for required aers.
2022-10-24 06:09:36 +03:00
Anna Shaleva
d7c1f3eac7 rpc: add "container" filter to WS execution notifications 2022-10-24 06:09:36 +03:00
Anna Shaleva
71069b0ed0 rpc: improve WS-enabled transaction awaiting
Fetch blocks since VUB+1, if block received and we haven't returned the
result yet, then transaction wasn't accepted at all.
2022-10-24 06:09:36 +03:00
Anna Shaleva
673a495527 rpc: add "since" filter to WS block events 2022-10-24 06:09:36 +03:00
Anna Shaleva
10a0716217 rpc: implement transaction awaiting functionality
Close #2704.
2022-10-24 06:09:36 +03:00
Anna Shaleva
6d38e75149 rpc: support multiple WSClient notification receivers 2022-10-24 06:09:36 +03:00
Anna Shaleva
4ce6bc6a66 rpc: adjust comment to Notification value cast
And explicitly specify field names on Notification creation.
2022-10-24 06:09:36 +03:00
Roman Khimov
0c3b03617e core: optimize HasBlock check for recent blocks
When block is being spread through the network we can get a lot of invs with
the same hash. Some more stale nodes may also announce previous or some
earlier block. We can avoid full DB lookup for them and minimize inv handling
time (timeouts in inv handler had happened in #2744).

It doesn't affect tests, just makes node a little less likely to spend some
considerable amount of time in the inv handler.
2022-10-21 21:21:58 +03:00
Roman Khimov
e26055190e network: check inv against currently processed transactions
Sometimes we already have it, but it's not yet processed, so we can save on
getdata request. It only affects very high-speed networks like 4-1 scenario
and it doesn't affect it a lot, but still we can do it.
2022-10-21 21:16:18 +03:00
Roman Khimov
cfb5058018 network: batch getdata replies
This is not exactly the protocol-level batching as was tried in #1770 and
proposed by neo-project/neo#2365, but it's a TCP-level change in that we now
Write() a set of messages and given that Go sets up TCP sockets with
TCP_NODELAY by default this is a substantial change, we have less packets
generated with the same amount of data. It doesn't change anything on properly
connected networks, but the ones with delays benefit from it a lot.

This also improves queueing because we no longer generate 32 messages to
deliver on transaction's GetData, it's just one stream of bytes with 32
messages inside.

Do the same with GetBlocksByIndex, we can have a lot of messages there too.

But don't forget about potential peer DoS attacks, if a peer is to request a
lot of big blocks we need to flush them before we process the whole set.
2022-10-21 17:16:32 +03:00
Roman Khimov
bf4636f70a
Merge pull request #2755 from nspcc-dev/improve-networking
Improve P2P transaction handling
2022-10-21 16:22:02 +07:00
Roman Khimov
e1b5ac9b81 network: separate tx handling from msg handling
This allows to naturally scale transaction processing if we have some peer
that is sending a lot of them while others are mostly silent. It also can help
somewhat in the event we have 50 peers that all send transactions. 4+1
scenario benefits a lot from it, while 7+2 slows down a little. Delayed
scenarios don't care.

Surprisingly, this also makes disconnects (#2744) much more rare, 4-node
scenario almost never sees it now. Most probably this is the case where peers
affect each other a lot, single-threaded transaction receiver can be slow
enough to trigger some timeout in getdata handler of its peer (because it
tries to push a number of replies).
2022-10-21 12:11:24 +03:00
Roman Khimov
e003b67418 network: reuse inventory hash list for request hashes
Microoptimization, we can do this because we only use them in handleInvCmd().
2022-10-21 11:28:40 +03:00
Roman Khimov
493b979b95 dbft: rev up, fix context timestamp initialization
See nspcc-dev/dbft#64, fixes #2753.
2022-10-20 12:38:14 +03:00
Roman Khimov
0f625f04f0
Merge pull request #2748 from nspcc-dev/stop-tx-flow
network/consensus: use new dbft StopTxFlow callback
2022-10-18 16:29:37 +07:00
Roman Khimov
73ce898e27 network/consensus: use new dbft StopTxFlow callback
It makes sense in general (further narrowing down the time window when
transactions are processed by consensus thread) and it improves block times a
little too, especially in the 7+2 scenario.

Related to #2744.
2022-10-18 11:06:20 +03:00
Roman Khimov
2791127ee4 network: add prometheus histogram with cmd processing time
It can be useful to detect some performance issues.
2022-10-17 22:51:16 +03:00
Roman Khimov
73079745ab
Merge pull request #2746 from nspcc-dev/optimize-tx-callbacks
network: only call tx callback if we're waiting for transactions
2022-10-17 16:39:41 +07:00
Roman Khimov
dce9f80585
Merge pull request #2743 from nspcc-dev/log-fan-out
Logarithmic gossip fan out
2022-10-14 23:18:34 +07:00
Roman Khimov
4dd3fd4ac0 network: only call tx callback if we're waiting for transactions
Until the consensus process starts for a new block and until it really needs
some transactions we can spare some cycles by not delivering transactions to
it. In tests this doesn't affect TPS, but makes block delays a bit more
stable. Related to #2744, I think it also may cause timeouts during
transaction processing (waiting on the consensus process channel while it does
something dBFT-related).
2022-10-14 18:45:48 +03:00
Roman Khimov
65f0fadddb network: register peer only if it's not a duplicate 2022-10-14 15:53:32 +03:00
Roman Khimov
851cbc7dab network: implement adaptive peer requests
When the network is big enough, MinPeers may be suboptimal for good network
connectivity, but if we know the network size we can do some estimation on the
number of sufficient peers.
2022-10-14 15:53:32 +03:00
Roman Khimov
c17b2afab5 network: add BroadcastFactor to control gossip, fix #2678 2022-10-14 15:53:32 +03:00
Roman Khimov
215e8704f1 network: simplify discoverer, make it almost a lib
We already have two basic lists: connected and unconnected nodes, we don't
need an additional channel and we don't need a goroutine to handle it.
2022-10-14 15:53:32 +03:00
Roman Khimov
c1ef326183 network: re-add addresses to the pool on UnregisterConnectedAddr
That's what we do anyway, but this way we can be a bit more efficient.
2022-10-14 14:12:33 +03:00
Roman Khimov
631f166709 network: broadcast to log-dependent number of nodes
Fixes #608.
2022-10-14 14:12:33 +03:00
Roman Khimov
c3001bc5bd
Merge pull request #2740 from nspcc-dev/cli-improvement
cli: improve VM CLI a bit more
2022-10-13 20:21:47 +07:00
Anna Shaleva
7db9258104 vm: make LoadFileWithFlags actually load with flags provided 2022-10-13 16:07:34 +03:00
Anna Shaleva
4dbaf2a123 smartcontract: add comment to GetCompleteTransaction 2022-10-13 16:07:34 +03:00
Anna Shaleva
af658bc3e5 cli: support Null as an argument for invocation-related commands 2022-10-13 16:07:31 +03:00
Anna Shaleva
7eb87afab8 cli: unify parameters parsing
Share parameters parsing code between 'contract invokefunction' and
'vm run' commands. It allows VM CLI to parse more complicated parameter
types including arrays and file-backed bytestrings.
2022-10-13 08:20:27 +03:00
Roman Khimov
dc62046019 network: add network size estimation metric 2022-10-12 22:29:55 +03:00
Roman Khimov
bcf77c3c42 network: filter out not-yet-ready nodes when broadcasting
They can fail right in the getPeers or they can fail later when packet send
is attempted. Of course they can complete handshake in-between these events,
but most likely they won't and we'll waste more resources on this attempt. So
rule out bad peers immediately.
2022-10-12 16:51:01 +03:00
Roman Khimov
137f2cb192 network: deduplicate TCPPeer code a bit
context.Background() is never canceled and has no deadline, so we can avoid
duplicating some code.
2022-10-12 15:43:31 +03:00
Roman Khimov
104da8caff network: broadcast messages, enqueue packets
Drop EnqueueP2PPacket, replace EnqueueHPPacket with EnqueueHPMessage. We use
Enqueue* when we have a specific per-peer message, it makes zero sense
duplicating serialization code for it (unlike Broadcast*).
2022-10-12 15:39:20 +03:00
Roman Khimov
d5f2ad86a1 network: drop unused EnqueueMessage interface from Peer 2022-10-12 15:27:08 +03:00
Roman Khimov
b345581c72 network: pings are broadcasted, don't send them to everyone
Follow the general rules of broadcasts, even though it's somewhat different
from Inv, we just want to get some reply from our neighbors to see if we're
behind. We don't strictly need all neighbors for it.
2022-10-12 15:25:03 +03:00
Roman Khimov
e1d5f18ff4 network: fix outdated Peer interface comments 2022-10-12 10:16:07 +03:00
Roman Khimov
8b26d9475b network: speculatively set GetAddrSent status
Otherwise we routinely get "unexpected addr received" error.
2022-10-11 18:42:40 +03:00
Roman Khimov
e80c60a3b9 network: rework broadcast logic
We have a number of queues for different purposes:
 * regular broadcast queue
 * direct p2p queue
 * high-priority queue

And two basic egress scenarios:
 * direct p2p messages (replies to requests in Server's handle* methods)
 * broadcasted messages

Low priority broadcasted messages:
 * transaction inventories
 * block inventories
 * notary inventories
 * non-consensus extensibles

High-priority broadcasted messages:
 * consensus extensibles
 * getdata transaction requests from consensus process
 * getaddr requests

P2P messages are a bit more complicated, most of the time they use p2p queue,
but extensible message requests/replies use HP queue.

Server's handle* code is run from Peer's handleIncoming, every peer has this
thread that handles incoming messages. When working with the peer it's
important to reply to requests and blocking this thread until we send (queue)
a reply is fine, if the peer is slow we just won't get anything new from
it. The queue used is irrelevant wrt this issue.

Broadcasted messages are radically different, we want them to be delivered to
many peers, but we don't care about specific ones. If it's delivered to 2/3 of
the peers we're fine, if it's delivered to more of them --- it's not an
issue. But doing this fairly is not an easy thing, current code tries performing
unblocked sends and if this doesn't yield enough results it then blocks (but
has a timeout, we can't wait indefinitely). But it does so in sequential
manner, once the peer is chosen the code will wait for it (and only it) until
timeout happens.

What can be done instead is an attempt to push the message to all of the peers
simultaneously (or close to that). If they all deliver --- OK, if some block
and wait then we can wait until _any_ of them pushes the message through (or
global timeout happens, we still can't wait forever). If we have enough
deliveries then we can cancel pending ones and it's again not an error if
these canceled threads still do their job.

This makes the system more dynamic and adds some substantial processing
overhead, but it's a networking code, any of this overhead is much lower than
the actual packet delivery time. It also allows to spread the load more
fairly, if there is any spare queue it'll get the packet and release the
broadcaster. On the next broadcast iteration another peer is more likely to be
chosen just because it didn't get a message previously (and had some time to
deliver already queued messages).

It works perfectly in tests, with optimal networking conditions we have much
better block times and TPS increases by 5-25%% depending on the scenario.

I'd go as far as to say that it fixes the original problem of #2678, because
in this particular scenario we have empty queues in ~100% of the cases and
this new logic will likely lead to 100% fan out in this case (cancelation just
won't happen fast enough). But when the load grows and there is some waiting
in the queue it will optimize out the slowest links.
2022-10-11 18:42:40 +03:00
Roman Khimov
0294e2eb18
Merge pull request #2738 from nspcc-dev/dont-block-forever-2
network: don't wait indefinitely for packet to be sent
2022-10-11 19:40:10 +07:00
Anna Shaleva
641abd4d1c smartcontract: fix underlying PublicKey parameter value
Value of PublicKey parameter always stores public key bytes, not the
deserialized representation. All other code (CLI parameters parsing with
its NewParameterFromString, Parameter unmarshaller, etc.) is based on
the idea that value of PublicKey is []byte.
2022-10-11 13:50:32 +03:00
Roman Khimov
dabdad20ad network: don't wait indefinitely for packet to be sent
Peers can be slow, very slow, slow enough to affect node's regular
operation. We can't wait for them indefinitely, there has to be a timeout for
send operations.

This patch uses TimePerBlock as a reference for its timeout. It's relatively
big and it doesn't affect tests much, 4+1 scenarios tend to perform a little
worse with while 7+2 scenarios work a little better. The difference is in some
percents, but all of these tests easily have 10-15% variations from run to
run.

It's an important step in making our gossip better because we can't have any
behavior where neighbors directly block the node forever, refs. #2678 and
2022-10-10 22:15:21 +03:00
Anna Shaleva
63fddb3f1a core: close BoltDB on failed root bucket creation 2022-10-10 10:12:34 +03:00
Anna Shaleva
735db08f84 services: adjust RPC server's getHistoricParams
Update documentation and add index upper bound check to get rid of
CodeQL warning.
2022-10-07 16:06:12 +03:00
Anna Shaleva
95cbddf19e cli: use custom logger to filter out runtime.Log messages
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go vm -p

    _   ____________        __________      _    ____  ___
   / | / / ____/ __ \      / ____/ __ \    | |  / /  |/  /
  /  |/ / __/ / / / /_____/ / __/ / / /____| | / / /|_/ /
 / /|  / /___/ /_/ /_____/ /_/ / /_/ /_____/ |/ / /  / /
/_/ |_/_____/\____/      \____/\____/      |___/_/  /_/

NEO-GO-VM > loadgo ./1-print/1-print.go
READY: loaded 21 instructions
NEO-GO-VM 0 > run
2022-10-07T15:28:20.461+0300	INFO	runtime log	{"tx": "", "script": "db03ceb3f672ee8cd0d714989b4d103ff7eed2f3", "msg": "Hello, world!"}
[]
```
2022-10-07 15:57:33 +03:00
Anna Shaleva
0db4e8d62c core: allow to perform storage search within given amount of DAO layers 2022-10-07 15:56:34 +03:00
Anna Shaleva
79e13f73d8 core, rpc: move getFakeNextBlock to Blockchain
It's needed for VM CLI as far and may be improved later.
2022-10-07 15:56:34 +03:00
Anna Shaleva
f45d8fc08d vm: remove default syscall handler
It's not needed anymore. Close #1075.
2022-10-07 15:56:34 +03:00
Anna Shaleva
0b717b0c22 vm: move vm CLI to cli/vm package 2022-10-07 15:56:34 +03:00
Anna Shaleva
4a46001746 smartcontract: fix error message for CreateMultiSigRedeemScript 2022-10-07 15:56:34 +03:00
Anna Shaleva
a91cf2a007 core: set default SecondsPerBlock value on blockchain creation
As mentioned in the node configuration docs.
2022-10-07 15:56:34 +03:00
Anna Shaleva
70e59d83c9 docs: fix supported database types 2022-10-07 15:56:34 +03:00
Anna Shaleva
2f5137e9b7 core: allow RO mode for Bolt and Level 2022-10-07 15:56:29 +03:00
Anna Shaleva
cbdd45cc96 core: return error on root BoltDB bucket creation if so 2022-10-06 14:01:56 +03:00
Anna Shaleva
03a1cf9f59 core: simplify newLevelDBForTesting function 2022-10-06 14:01:56 +03:00
Roman Khimov
4616600636
Merge pull request #2728 from nspcc-dev/fix-vub-comment
core: add example to VUB comment
2022-10-06 16:33:24 +07:00
Anna Shaleva
78cd2b4566 core: add example to VUB comment 2022-10-06 10:32:18 +03:00
Roman Khimov
1c99c21d9a
Merge pull request #2725 from nspcc-dev/move-cli-tests
Move CLI tests
2022-10-06 13:43:39 +07:00
Roman Khimov
1ac60ada19 cli: move tests to subpackages
Refs. #2379, but not completely solves it, one package seriously outweights
others:

?       github.com/nspcc-dev/neo-go/cli [no test files]
ok      github.com/nspcc-dev/neo-go/cli/app     0.036s  coverage: 100.0% of statements
ok      github.com/nspcc-dev/neo-go/cli/cmdargs 0.011s  coverage: 60.8% of statements
ok      github.com/nspcc-dev/neo-go/cli/flags   0.009s  coverage: 97.7% of statements
?       github.com/nspcc-dev/neo-go/cli/input   [no test files]
ok      github.com/nspcc-dev/neo-go/cli/options 0.033s  coverage: 50.0% of statements
?       github.com/nspcc-dev/neo-go/cli/paramcontext    [no test files]
ok      github.com/nspcc-dev/neo-go/cli/query   2.155s  coverage: 45.3% of statements
ok      github.com/nspcc-dev/neo-go/cli/server  1.373s  coverage: 67.8% of statements
ok      github.com/nspcc-dev/neo-go/cli/smartcontract   8.819s  coverage: 94.3% of statements
ok      github.com/nspcc-dev/neo-go/cli/util    0.006s  coverage: 10.9% of statements
?       github.com/nspcc-dev/neo-go/cli/vm      [no test files]
ok      github.com/nspcc-dev/neo-go/cli/wallet  72.103s coverage: 88.2% of statements

Still a nice thing to have.
2022-10-06 09:21:26 +03:00
Roman Khimov
1c376ffa62
Merge pull request #2724 from nspcc-dev/rpc-options
rpcsrv: handle preflight OPTIONS with CORS kludge, fix #2721
2022-10-05 17:02:55 +07:00
Roman Khimov
b48d02f4a6 rpcsrv: handle preflight OPTIONS with CORS kludge, fix #2721 2022-10-05 11:09:45 +03:00
Roman Khimov
8893163803 smartcontract: define parameter lengths as constants and use them 2022-10-05 10:46:21 +03:00
Roman Khimov
317dd42513 *: use uint*Size and SignatureLen constants where appropriate 2022-10-05 10:45:52 +03:00
Roman Khimov
79887f9d78 runtime: check notifications against ABI
Related to #2703, just a logged thing for now.
2022-10-04 17:52:38 +03:00
Roman Khimov
7d0840d5d5
Merge pull request #2720 from nspcc-dev/notifications-check
compiler: enforce runtime.Notify parameters cast to proper type
2022-10-01 03:02:29 +07:00
Anna Shaleva
554e48e7b7 compiler: enforce runtime.Notify parameters cast
If notification parameters type can be defined in a compile time then enforce
parameter cast to the desired type got from manifest.
2022-09-30 14:42:43 +03:00
Anna Shaleva
80f71a4e6e compiler: do not enforce variadic event args check on ellipsis usage
In case of ellipsis usage compiler defines argument type as ArrayT
(which is correct, because it's a natural representation of the last
argument, it represents the array of interface{}).
Here goes the problem:
```
=== RUN   TestEventWarnings/variadic_event_args_via_ellipsis
    compiler_test.go:251:
        	Error Trace:	compiler_test.go:251
        	Error:      	Received unexpected error:
        	            	event 'Event' should have 'Integer' as type of 1 parameter, got: Array
        	Test:       	TestEventWarnings/variadic_event_args_via_ellipsis
```

Parsing the last argument in this case is a separate complicated problem
due to the fact that we need to grab types of elements of []interface{} inside the
fully qualified ast node which may looks like:
```
runtime.Notify("Event", (append([]interface{}{1, 2}, (([]interface{}{someVar, 4}))...))...)
```

Temporary solution is to exclude such notifications from analysis until we're
able to properly resolve element types of []interface{}.
2022-09-30 08:42:48 +03:00
Anna Shaleva
08427f23b6 compiler: do not check Any event parameter for compliance
It's possible that declared manifest event has parameter of AnyT for
those cases when parameter type differs from method to method. If so,
then we don't need to enforce type check after compilation.
2022-09-30 08:40:55 +03:00
Roman Khimov
b7be4edf7f
Merge pull request #2718 from nspcc-dev/ok-conversion
compiler: prohibit to compile type assertion with two return values
2022-09-29 02:54:03 +07:00
Anna Shaleva
b98848bf49 compiler: prohibit to compile type assertion with two return values
Close #2692.
2022-09-28 11:27:13 +03:00
Anna Shaleva
1828e79412 compiler: add test for foreign function inlining 2022-09-27 15:36:06 +03:00
Anna Shaleva
24c107e3a2 interop: refactor address.FromHash160 code
Make it more simple.
2022-09-27 15:32:57 +03:00
Anna Shaleva
5d578fdd95 compiler: consider inlined types info on "append" handling
We need to search for "append" argument type info not only inside local
package type info map, but also inside the inlined type info map.
Close #2696.
2022-09-27 15:32:51 +03:00
Anna Shaleva
ea08a81726 interop: add CallWithVersion helper 2022-09-21 17:38:42 +03:00
Anna Shaleva
df802b6fc6 vm: adjust emit.AppCall comment
There's no APPCALL anymore.
2022-09-21 17:25:45 +03:00
Anna Shaleva
8d5f97a699 interop: adjust Iterator documentation
The only way to get Iterator is as a result of storage.Find.
2022-09-20 17:09:48 +03:00
Roman Khimov
5a7ab2054d
Merge pull request #2698 from nspcc-dev/address-helpers
interop: add a couple of `interop.Hash160` encoding helpers
2022-09-20 17:07:00 +07:00
Anna Shaleva
293dbf3d1b compiler: adjust test's checkInstrCount
Remove unnecessary code.
2022-09-20 09:45:07 +03:00
Anna Shaleva
7e13140b04 interop: add Hash160 encoder\decoder helper
Close #2690.
2022-09-20 09:37:04 +03:00
Roman Khimov
1b753cd4bc native: add some tests for stdlib's atoi
See neo-project/neo#2804 and neo-project/neo#2813. We're already compatible.
2022-09-19 16:18:53 +03:00
Roman Khimov
18ed26194f smartcontract: add Len to Builder
Which is useful in some cases.
2022-09-14 10:25:10 +03:00
Roman Khimov
5979138306 stateroot: fix panic on shutdown
Stateroot service is always active, but it might have no wallet.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc57d41]

goroutine 1 [running]:
github.com/nspcc-dev/neo-go/pkg/wallet.(*Wallet).Close(...)
        github.com/nspcc-dev/neo-go/pkg/wallet/wallet.go:175
github.com/nspcc-dev/neo-go/pkg/services/stateroot.(*service).Shutdown(0xc000105880?)
        github.com/nspcc-dev/neo-go/pkg/services/stateroot/validators.go:77 +0x81
github.com/nspcc-dev/neo-go/pkg/network.(*Server).Shutdown(0xc000105880)
        github.com/nspcc-dev/neo-go/pkg/network/server.go:271 +0x205
github.com/nspcc-dev/neo-go/cli/server.startServer(0xc0002702c0)
        github.com/nspcc-dev/neo-go/cli/server/server.go:641 +0x2675
github.com/urfave/cli.HandleAction({0xe456e0?, 0x1155f20?}, 0x4?)
        github.com/urfave/cli@v1.22.5/app.go:524 +0x50
github.com/urfave/cli.Command.Run({{0xfca38b, 0x4}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0xfd6a46, 0x10}, {0xffebe3, ...}, ...}, ...)
        github.com/urfave/cli@v1.22.5/command.go:173 +0x65b
github.com/urfave/cli.(*App).Run(0xc000272000, {0xc00003e180, 0x3, 0x3})
        github.com/urfave/cli@v1.22.5/app.go:277 +0x8a7
main.main()
        ./main.go:21 +0x33
2022-09-13 13:18:13 +03:00
Roman Khimov
31792e3132
Merge pull request #2686 from nspcc-dev/drop-at-block-rpcs
rpcclient: simplify historic API
2022-09-09 15:18:27 +03:00
Roman Khimov
d40eb79975 rpcclient: simplify historic API
util.Uint256 is util.Uint256 and it's same RPC behind the scenes, so we can
make it a bit easier to digest. See #2545 also.
2022-09-09 13:41:45 +03:00
Roman Khimov
f78a065230 vm/cli: generate Go 1.17 go.mods 2022-09-08 21:18:51 +03:00
Roman Khimov
541d4b49e1 context: define a constant for transaction context type 2022-09-08 14:33:04 +03:00
Roman Khimov
6be9367f03 rpcclient/notary: add OnNEP17PaymentData and an example
Update documentation as well to mention it and not mention outdated APIs. We
can't link them yet, this will be done after the release.
2022-09-08 14:33:04 +03:00
Roman Khimov
4fb4f5a1ac smartcontract: make *util.Uint160 and *util.Uint256 usable for parameters
Use Any type for NULL.
2022-09-08 14:33:04 +03:00
Roman Khimov
1e54b422cd emit: make *util.Uint160 and *util.Uint256 emittable
They can be nil or can be regular uint types we're used to.
2022-09-08 14:33:04 +03:00
Roman Khimov
3c02177d3c rpcclient: improve comments for some methods 2022-09-08 14:33:04 +03:00
Roman Khimov
3e8e6857e5 rpcclient; deprecate more methods
They make little sense now.
2022-09-08 14:33:04 +03:00
Roman Khimov
c4ddf80742 rpcclient: correct Init requirement in documentation 2022-09-08 14:33:04 +03:00
Roman Khimov
ee55e95c28 rpcclient: add examples for nep11/nep17/neo
GAS doesn't need any, so just mention nep17 package there.
2022-09-08 14:33:04 +03:00
Roman Khimov
a1c9871d95 nns: it's NEP-11, so make NEP-11 methods available too 2022-09-08 14:33:04 +03:00
Roman Khimov
186e5c19b6 rpcclient: update documentation, mention subpackages
Drop TODOs (we have relevant GitHub issues), drop verbosity comment (we have
*Verbose APIs for that).
2022-09-08 14:33:04 +03:00
Roman Khimov
cb1a1f8532 actor: extend documentation, add example 2022-09-08 14:33:04 +03:00
Roman Khimov
e1fe76137e rpcclient: use separate reader/writer structs in nep11 and nep17
Which greatly simplifies reuse of these packages (and they're expected to be
reused since real tokens implement standards and also add something of their
own) and allows to avoid effects like

  doc_test.go:68:28: ambiguous selector neoContract.BalanceOf

when neo.Contract is used. Avoids duplication in NEP-11 implementation as
well.
2022-09-08 14:33:03 +03:00
Roman Khimov
00a9376311 invoker: update documentation, add example 2022-09-08 14:33:03 +03:00
Roman Khimov
ea92f3d716 smartcontract: add some notes on API limitations 2022-09-08 13:27:07 +03:00
Roman Khimov
69176168c3 smartcontract: modernize Builder example
And make it a bit more useful.
2022-09-07 22:40:25 +03:00
Roman Khimov
aca8ce0d28 unwrap: provide ErrNoSessionID, add some explanations 2022-09-07 22:40:25 +03:00
Roman Khimov
97193cf337 golangci: add predeclared linter
These can be confusing.
2022-09-02 18:36:26 +03:00
Roman Khimov
63f212f4b3 golangci: enable/fix misspell 2022-09-02 18:36:26 +03:00
Roman Khimov
4f3ffe7290 golangci: enable errorlint and fix everything it found 2022-09-02 18:36:23 +03:00
Roman Khimov
c703ac6805 golangci: enable contextcheck linter, fix WSClient
pkg/rpcclient/wsclient.go:93:30  contextcheck  Function `Dial` should pass the context parameter
2022-09-02 18:35:54 +03:00
Roman Khimov
3c009271f8 golangci: enable bodyclose checker and fix related code
It has found an issue in the oracle code, so I think it's worth doing.
2022-09-02 18:35:54 +03:00
Roman Khimov
3da8b98fc3
Merge pull request #2672 from nspcc-dev/private-key-cleanup
Private key cleanup
2022-09-02 16:20:39 +03:00
Roman Khimov
eb67145f81 keys: check length first, then do things in WIFDecode
Otherwise we can easily panic there on bad input.
2022-09-02 14:44:32 +03:00
Roman Khimov
3c722a9498 keys: clean temporary data during key imports
Don't leak anything this way.
2022-09-02 14:44:32 +03:00
Roman Khimov
74bf4a8e3f slice: add Clean microfunction
To be used for various cleaning purposes, one line is better than three lines.
2022-09-02 14:44:32 +03:00
Roman Khimov
58dc8d0c9b *: always close the wallet after use
Fix #2631.
2022-09-02 14:44:32 +03:00
Roman Khimov
ee5f8b6c21 consensus: update dbft, drop marshaling from private key
dbft doesn not need this and we must not leak the key in any way.
2022-09-02 14:44:18 +03:00
Roman Khimov
cad0d7f00d wallet: add some warnings to Decrypt and PrivateKey docs 2022-09-02 14:44:18 +03:00
Roman Khimov
e164625a7f wallet: provide (*Account).SignHashable API
Make PrivateKey() less used and less useful.
2022-09-02 14:44:01 +03:00
Roman Khimov
e569edc841 wallet: add ScriptHash() to Account
It allows to simplify a lot of code and avoid getting a PrivateKey in some
cases.
2022-09-02 14:43:34 +03:00
Roman Khimov
fd8da6fdb9 *: do not get private key from Account to check if it CanSign()
We have this API now to performs checks.
2022-09-02 14:43:34 +03:00
Roman Khimov
8d33206bb8 *: don't get private key from account if just public one is needed
Add PublicKey() API to the Account and use it as appropriate, avoid creating
additional references to the private key.
2022-09-02 14:43:28 +03:00
Roman Khimov
f79672c4c3
Merge pull request #2624 from nspcc-dev/optimize-unused-globals
compiler: do not emit initialisation code for unused global vars
2022-09-02 14:28:44 +03:00
Roman Khimov
62be6f959c keys/wallet: add Destroy/Close/Close
PrivateKey can be destroyed and Account/Wallet can be closed (destroying keys
in the process).
2022-09-01 17:07:00 +03:00
Roman Khimov
a30e73a0d7 wallet: drop publicKey from Account
It's not very useful and it's only available when we have a private key
anyway.
2022-09-01 17:07:00 +03:00
Roman Khimov
53edbd569f wallet: don't allow to ConvertMultisig a locked account
This stretched the definition of Locked somewhat, but still makes sense to
me, locked accounts better not be touched.
2022-09-01 17:07:00 +03:00
Roman Khimov
0090577446 wallet: don't permanently store wif in the Account
It's useless and it's just another copy of the key. If really needed, it can
be derived from the key.
2022-09-01 17:07:00 +03:00
Roman Khimov
411ebdf51e cli: add complete support for offline signing, fix #2662
See documentation update for an example. Some code is made generic as well,
GetCompleteTransaction can now be used directly on ParameterContext.
2022-09-01 15:30:34 +03:00
Roman Khimov
773bcc3a59 context: make error messages a bit less cryptic
Refs. #2664.
2022-09-01 15:30:33 +03:00
Roman Khimov
58707c2b1e context: handle the case when we have more sigs than needed
We can technically have more signatures in the file than we need and it's OK,
this case should be handled.
2022-09-01 15:30:33 +03:00
Anna Shaleva
7f613e63aa compiler: add test for #2661 2022-09-01 13:39:22 +03:00
Anna Shaleva
800321db06 compiler: rename named unused global vars to "_"
So that (*codegen).Visit is able to omit code generation for these
unused global vars. The most tricky part is to detect unused global
variables, it is done in several steps:
1. Collect the set of named used/unused global vars.
2. Collect the set of globally declared expressions that contain
function calls.
3. Pick up global vars from the set made at step 2.
4. Traverse used functions and puck up those global vars that are used
from these functions.
5. Rename all globals that are presented in the set made at step 1
but are not presented in the set made on step 3 or step 4.
2022-09-01 13:39:19 +03:00
Anna Shaleva
1e6b70d570 compiler: adjust TestInline template
Move all auxiliary function declaration after Main, so that INITSLOT
instructions counter works properly. `vmAndCompileInterop` loads program
and moves nextIP to the Main function offset if there's no _init
function. If _init is there, then nextIP will be moved to the start of
_init. In TestInline we don't handle instructions properly (CALL/JMP
don't change nextIP), we just perform instruction traversal from the
start point via Next(), thus INITSLOT counter value depends on the
starting instruction, which depends on _init presence.
2022-09-01 09:19:20 +03:00
Anna Shaleva
91b36657d6 compiler: do not emit code for unnamed unused variables
If variable is unnamed and does not contain function call then it's
treated as unused and code generation may be omitted for it
initialization/declaration.
2022-09-01 09:18:56 +03:00
Anna Shaleva
1dcbdb011a compiler: emit code for unnamed global var decls more careful
In case if global var is unnamed (and, as a consequence, unused) and
contains a function call inside its value specification, we need to emit
code for this var to be able to call the function as it can have
side-effects. See the example:
```
package foo

import "github.com/nspcc-dev/neo-go/pkg/interop/runtime"

var A = f()

func Main() int {
   return 3
}

func f() int {
   runtime.Notify("Valuable notification", 1)
   return 2
}
```
2022-09-01 09:18:07 +03:00
Roman Khimov
314cd3341b
Merge pull request #2667 from nspcc-dev/rpc-nep-token-info
Drop NEPXXTokenInfo from the RPC client
2022-08-30 14:41:30 +03:00
Roman Khimov
673c8954cc
Merge pull request #2665 from nspcc-dev/notary-rpcclient
Notary RPC client
2022-08-30 12:43:13 +03:00
Roman Khimov
07f3023e84 rpcclient: add notary.Actor for seamless notary experience 2022-08-30 11:47:56 +03:00
Roman Khimov
ed6ed61712 neptoken: add Info to replace old NEPXXTokenInfo methods
I'm still not sure it's good to have this exposed from neptoken at all, but
let's try it this way.
2022-08-29 22:52:27 +03:00
Roman Khimov
ac5c609063 core: add a bit more data into NVB errors
It's not always obvious what they mean and the NVB value is.
2022-08-29 15:08:37 +03:00
Roman Khimov
a95984febf actor: allow providing default attributes/hooks to be used
Which expands Actor use cases greatly.
2022-08-29 15:08:37 +03:00
Roman Khimov
a3f32bf306 neptoken: move BalanceOf implementation to Base from nep11/nep17
It's the same, even though standards define parameter name in a bit different
way.
2022-08-26 21:52:19 +03:00
Roman Khimov
0d9158bd79 rpcclient: add enough of NNS into nns to deprecate NNS methods 2022-08-26 19:45:37 +03:00
Roman Khimov
fe50879bb7 wallet: add (*Account).CanSign API 2022-08-26 18:21:58 +03:00