Commit graph

2149 commits

Author SHA1 Message Date
Anna Shaleva
1250e82c2a vm: add PUSHT and PUSHF opcodes
Port https://github.com/neo-project/neo-vm/pull/497.
2022-12-02 10:02:33 +03:00
Roman Khimov
6912695b9d
Merge pull request #2826 from nspcc-dev/check-for-max-block-sysfee
core: filter out txes with system fee > MaxBlockSystemFee
2022-11-29 14:57:10 +07:00
Roman Khimov
6847e1760c core: filter out txes with system fee > MaxBlockSystemFee
They can stay in the memory pool forever because consensus process will never
accept these transactions (and maybe even block consensus process at all).
2022-11-29 10:31:00 +03:00
Anna Shaleva
6d3cc0f7e7 core: extend error text for storage context in dynamic script 2022-11-28 17:17:36 +03:00
Roman Khimov
4e26f4b26e runtime: implement System.Runtime.LoadScript, fix #2701 2022-11-25 15:11:49 +03:00
Roman Khimov
1c38b45074 core: don't always store all hashes in memory
We're paging these hashes, so we need a previous full page and a current one
plus some cache for various requests. Storing 1M of hashes is 32M of memory
and it grows quickly. It also seriously affects node startup time, most of
what it's doing is reading these hashes, the longer the chain the more time it
needs to do that.

Notice that this doesn't change the underlying DB scheme in any way.
2022-11-25 14:30:51 +03:00
Roman Khimov
0ad6e295ea core: make GetHeaderHash accept uint32
It should've always been this way because block indexes are uint32.
2022-11-25 14:30:51 +03:00
Roman Khimov
16db4746cc core: drop another useless genesis-specific header handling branch
If we only have genesis block (or <2000 headers) then we might as well use
generic logic below with zero targetHash because genesis block has zero
PrevHash (and its hash will naturally be the last on the chain going
backwards).
2022-11-25 13:08:03 +03:00
Roman Khimov
83e2ca054f core: simplify header hash list restoration logic
We don't care about header, we only need hashes here.
2022-11-25 13:08:03 +03:00
Roman Khimov
ff284d5d5c core: remove genesis-specific header hash logic from init
The code below will do the same thing (via CreateGenesisBlock, but it doesn't
matter) generically, so this branch is useless.
2022-11-25 13:08:03 +03:00
Roman Khimov
66ddeccdad
Merge pull request #2813 from nspcc-dev/fix-state-reset
core: fix broken state reset
2022-11-23 13:43:42 +07:00
Anna Shaleva
f3ef2890f0 core: check headers at the proper state on state reset
And fix the comment along the way.
2022-11-23 09:16:33 +03:00
Anna Shaleva
b27a9bcf95 core: adjust info message for proper-stated chains
Make it prettier for those cases when `db reset` command was called
after interrupted reset.
2022-11-22 11:53:39 +03:00
Anna Shaleva
b82374823e core: increase persist batch size for reset storage changes 2022-11-22 11:53:39 +03:00
Anna Shaleva
bdc42cd595 core: reset blocks, txs and AERs in several stages
Sometimes it can be hard to persist all changes at ones, the process
can take almost all RAM and a lot of time. Here's the example of reset
for mainnet from 2.4M to 1:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:16:48.236+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2022-11-20T17:16:48.236+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2022-11-20T17:16:48.237+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2022-11-20T17:16:48.237+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:16:48.240+0300	INFO	restoring blockchain	{"version": "0.2.6"}
2022-11-20T17:16:48.297+0300	INFO	initialize state reset	{"target height": 1}
2022-11-20T17:16:48.300+0300	INFO	trying to reset blocks, transactions and AERs
2022-11-20T17:19:29.313+0300	INFO	blocks, transactions ans AERs are reset	{"took": "2m41.015126493s", "keys": 3958420}
...
```
To avoid OOM killer, split blocks reset into multiple stages. It increases
operation time due to intermediate DB persists, but makes things cleaner, the
result for almost the same DB height with the new approach:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:39:42.023+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2022-11-20T17:39:42.023+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2022-11-20T17:39:42.023+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2022-11-20T17:39:42.023+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:39:42.026+0300	INFO	restoring blockchain	{"version": "0.2.6"}
2022-11-20T17:39:42.071+0300	INFO	initialize state reset	{"target height": 1}
2022-11-20T17:39:42.073+0300	INFO	trying to reset blocks, transactions and AERs
2022-11-20T17:40:11.735+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 1, "took": "29.66363737s", "keys": 210973}
2022-11-20T17:40:33.574+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 2, "took": "21.839208683s", "keys": 241203}
2022-11-20T17:41:29.325+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 3, "took": "55.750698386s", "keys": 250593}
2022-11-20T17:42:12.532+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 4, "took": "43.205892757s", "keys": 321896}
2022-11-20T17:43:07.978+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 5, "took": "55.445398156s", "keys": 334822}
2022-11-20T17:43:35.603+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 6, "took": "27.625292032s", "keys": 317131}
2022-11-20T17:43:51.747+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 7, "took": "16.144359017s", "keys": 355832}
2022-11-20T17:44:05.176+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 8, "took": "13.428733899s", "keys": 357690}
2022-11-20T17:44:32.895+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 9, "took": "27.718548783s", "keys": 393356}
2022-11-20T17:44:51.814+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 10, "took": "18.917954658s", "keys": 366492}
2022-11-20T17:45:07.208+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 11, "took": "15.392642196s", "keys": 326030}
2022-11-20T17:45:18.776+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 12, "took": "11.568255716s", "keys": 299884}
2022-11-20T17:45:25.862+0300	INFO	last batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 13, "took": "7.086079594s", "keys": 190399}
2022-11-20T17:45:25.862+0300	INFO	blocks, transactions ans AERs are reset	{"took": "5m43.791214084s", "overall persisted keys": 3966301}
...
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
d67f0df516 core: reset block headers together with header height info
We need to keep the headers information consistent with header batches
and headers. This comit fixes the bug with failing blockchain
initialization on recovering from state reset interrupted after the
second stage (blocks/txs/AERs removal):
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T16:28:29.437+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T16:28:29.440+0300	INFO	restoring blockchain	{"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: could not get header 1898cd356a4a2688ed1c6c7ba1fd6ba7d516959d8add3f8dd26232474d4539bd: key not found
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
283da8f599 core: use DAO-provided block height during during state reset
Don't use cache because it's not yet initialized. Also, perform
safety checks only if state reset wasn't yet started. These fixes
alloww to solve the following problem while recovering from
interrupted state reset:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T15:51:31.431+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T15:51:31.434+0300	INFO	restoring blockchain	{"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: current block height is 0, can't reset state to height 83000
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
7d55bf2cc1 core: log persisted storage item batches count during state reset 2022-11-22 11:53:39 +03:00
Anna Shaleva
f52451e582 core: fix state reset with broken contract
Sync up with #2802, bad contract -> no contract ID at all.
2022-11-22 11:53:39 +03:00
Anna Shaleva
ecda07736e core: stop storage items reset after any seek error 2022-11-22 11:53:39 +03:00
Anna Shaleva
bfe7aeae7b core: stop storage items reset after the first persist error
It's a bug, we mustn't continue if something bad had happend on persist,
otherwise this error will be overwritten by subsequent successfull persist.
2022-11-22 11:53:39 +03:00
Anna Shaleva
235518eb6c core: reset batch counter to zero after each persist in resetStateInternal
It's a bug, otherwise we'll persist each storage item after 10K-th one,
that's the reason of abnormous long storage items resetting stage.
2022-11-22 11:53:39 +03:00
Anna Shaleva
9f23fafc03 core: improve logging of resetStateInternal
Inform when starting subsequent stage, inform about keys persisted.
2022-11-22 11:53:36 +03:00
Roman Khimov
cb64957af5 vm: don't use Stack for istack
We don't use all of the Stack functionality for it, so drop useless methods
and avoid some interface conversions. It increases single-node TPS by about
0.9%, so nothing really important there, but not a bad change either. Maybe it
can be reworked again with generics though.
2022-11-18 11:35:29 +03:00
Roman Khimov
ab0ff63ce1
Merge pull request #2804 from nspcc-dev/check-aer-sub
rpc: fix subscribers locking logic and properly drain poll-based waiter receiver
2022-11-17 04:24:35 +07:00
Anna Shaleva
6dbae7edc4 rpcclient: fix WS-client unsubscription process
Do not block subscribers until the unsubscription request to RPC server
is completed. Otherwise, another notification may be received from the
RPC server which will block the unsubscription process.

At the same time, fix event-based waiter. We must not block the receiver
channel during unsubscription because there's a chance that subsequent
event will be sent by the server. We need to read this event in order not
to block the WSClient's readloop.
2022-11-16 23:44:30 +03:00
Roman Khimov
822722bd2e native: ignore decoding errors during cache init
Bad contract -> no contract. Unfortunately we've got a broken
6f1837723768f27a6f6a14452977e3e0e264f2cc contract on the mainnet which can't
be decoded (even though it had been saved successfully), so this is a
temporary fix for #2801 to be able to start mainnet node after shutdown.
2022-11-16 12:00:28 +03:00
Roman Khimov
f78231fd9c
Merge pull request #2773 from nspcc-dev/state-reset
core: implement state reset
2022-11-10 22:26:43 +07:00
Anna Shaleva
e7effef052 core: reuse appendTokenTransferInfo from appendTokenTransfer 2022-11-10 18:08:18 +03:00
Anna Shaleva
bd6bb9e9e2 core: allow to reset blockchain state 2022-11-10 18:08:17 +03:00
Anna Shaleva
ec9317d5b4 core: fix typo 2022-11-10 17:58:06 +03:00
Anna Shaleva
1dac45bbbb core: add ability to check whether blockchain is running 2022-11-10 16:47:04 +03:00
Roman Khimov
8746d9877e keys: move from btcd/btcec to decred/secp256k1 package
There is a security issue found in github.com/btcsuite/btcd that we don't care
about (we're only using 256k1 implementation), but GitHub complains about
it. We could update to github.com/btcsuite/btcd/btcec/v2, but it's now just a
thin wrapper over github.com/decred/dcrd/dcrec/secp256k1/v4, so we better use
it directly.
2022-11-08 17:59:59 +03:00
Roman Khimov
0c3b03617e core: optimize HasBlock check for recent blocks
When block is being spread through the network we can get a lot of invs with
the same hash. Some more stale nodes may also announce previous or some
earlier block. We can avoid full DB lookup for them and minimize inv handling
time (timeouts in inv handler had happened in #2744).

It doesn't affect tests, just makes node a little less likely to spend some
considerable amount of time in the inv handler.
2022-10-21 21:21:58 +03:00
Anna Shaleva
63fddb3f1a core: close BoltDB on failed root bucket creation 2022-10-10 10:12:34 +03:00
Anna Shaleva
95cbddf19e cli: use custom logger to filter out runtime.Log messages
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go vm -p

    _   ____________        __________      _    ____  ___
   / | / / ____/ __ \      / ____/ __ \    | |  / /  |/  /
  /  |/ / __/ / / / /_____/ / __/ / / /____| | / / /|_/ /
 / /|  / /___/ /_/ /_____/ /_/ / /_/ /_____/ |/ / /  / /
/_/ |_/_____/\____/      \____/\____/      |___/_/  /_/

NEO-GO-VM > loadgo ./1-print/1-print.go
READY: loaded 21 instructions
NEO-GO-VM 0 > run
2022-10-07T15:28:20.461+0300	INFO	runtime log	{"tx": "", "script": "db03ceb3f672ee8cd0d714989b4d103ff7eed2f3", "msg": "Hello, world!"}
[]
```
2022-10-07 15:57:33 +03:00
Anna Shaleva
0db4e8d62c core: allow to perform storage search within given amount of DAO layers 2022-10-07 15:56:34 +03:00
Anna Shaleva
79e13f73d8 core, rpc: move getFakeNextBlock to Blockchain
It's needed for VM CLI as far and may be improved later.
2022-10-07 15:56:34 +03:00
Anna Shaleva
a91cf2a007 core: set default SecondsPerBlock value on blockchain creation
As mentioned in the node configuration docs.
2022-10-07 15:56:34 +03:00
Anna Shaleva
70e59d83c9 docs: fix supported database types 2022-10-07 15:56:34 +03:00
Anna Shaleva
2f5137e9b7 core: allow RO mode for Bolt and Level 2022-10-07 15:56:29 +03:00
Anna Shaleva
cbdd45cc96 core: return error on root BoltDB bucket creation if so 2022-10-06 14:01:56 +03:00
Anna Shaleva
03a1cf9f59 core: simplify newLevelDBForTesting function 2022-10-06 14:01:56 +03:00
Anna Shaleva
78cd2b4566 core: add example to VUB comment 2022-10-06 10:32:18 +03:00
Roman Khimov
317dd42513 *: use uint*Size and SignatureLen constants where appropriate 2022-10-05 10:45:52 +03:00
Roman Khimov
79887f9d78 runtime: check notifications against ABI
Related to #2703, just a logged thing for now.
2022-10-04 17:52:38 +03:00
Roman Khimov
1b753cd4bc native: add some tests for stdlib's atoi
See neo-project/neo#2804 and neo-project/neo#2813. We're already compatible.
2022-09-19 16:18:53 +03:00
Roman Khimov
97193cf337 golangci: add predeclared linter
These can be confusing.
2022-09-02 18:36:26 +03:00
Roman Khimov
63f212f4b3 golangci: enable/fix misspell 2022-09-02 18:36:26 +03:00
Roman Khimov
4f3ffe7290 golangci: enable errorlint and fix everything it found 2022-09-02 18:36:23 +03:00
Roman Khimov
e569edc841 wallet: add ScriptHash() to Account
It allows to simplify a lot of code and avoid getting a PrivateKey in some
cases.
2022-09-02 14:43:34 +03:00
Roman Khimov
8d33206bb8 *: don't get private key from account if just public one is needed
Add PublicKey() API to the Account and use it as appropriate, avoid creating
additional references to the private key.
2022-09-02 14:43:28 +03:00
Roman Khimov
ac5c609063 core: add a bit more data into NVB errors
It's not always obvious what they mean and the NVB value is.
2022-08-29 15:08:37 +03:00
Roman Khimov
9916832e2e core: set Tx for interop.Context when verifying witnesses
Notary contract uses it in the verification context and it's not harmful to
have it always be there when it's there.
2022-08-22 13:38:43 +03:00
Roman Khimov
eeeb0f6f0e core: accept two-side channels for sub/unsub, read on unsub
Blockchain's notificationDispatcher sends events to channels and these
channels must be read from. Unfortunately, regular service shutdown procedure
does unsubscription first (outside of the read loop) and only then drains the
channel. While it waits for unsubscription request to be accepted
notificationDispatcher can try pushing more data into the same channel which
will lead to a deadlock. Reading in the same method solves this, any number of
events can be pushed until unsub channel accepts the data.
2022-08-19 22:08:40 +03:00
Roman Khimov
f7c5ab4f43 state: check for array length in (*Contract).FromStackItem
Panicing here is not appropriate.
2022-08-16 17:19:47 +03:00
Roman Khimov
a1a5db8fcd state: add more convenient method to get native contract hashes 2022-08-15 10:54:29 +03:00
Anna Shaleva
94e84f0364 core: fix native Management's hasMethod signature
Affects states, see
28ab45a6ec/src/Neo/SmartContract/Native/ContractManagement.cs (L155).
2022-08-11 16:10:01 +03:00
Roman Khimov
bc3bffea53 native: fix oracle.finish reentrancy bug
See neo-project/neo#2795.
2022-08-10 19:02:36 +03:00
Roman Khimov
8d170a1eb8 native: add Oracle.finish reentrancy test 2022-08-10 19:02:36 +03:00
Anna Shaleva
916f2293b8 *: apply go 1.19 formatter heuristics
And make manual corrections where needed. See the "Common mistakes
and pitfalls" section of https://tip.golang.org/doc/comment.
2022-08-09 15:37:52 +03:00
Anna Shaleva
bb751535d3 *: bump minimum supported go version
Close #2497.
2022-08-08 13:59:32 +03:00
Roman Khimov
b8a4a6dc5b
Merge pull request #2613 from nspcc-dev/exec-changes-3.4.0
Exec changes for 3.4.0
2022-08-05 15:04:35 +03:00
Roman Khimov
e8d2277fe5 contract/vm: only push NULL after call in dynamic contexts
And determine the need for Null dynamically. For some reason the only dynamic
context is Contract.Call. CALLT is not dynamic and neither is a call from
native contract, go figure...
2022-08-05 14:35:00 +03:00
Roman Khimov
99e2681d3a interop/vm: use more robust CalledByEntry check
Directly check contexts.
2022-08-05 14:35:00 +03:00
Roman Khimov
13f5fdbe8a vm: extract shared parts of the Context
Local calls reuse them, cross-contract calls create new ones. This allows to
avoid some allocations and use a little less memory.
2022-08-05 14:26:25 +03:00
Roman Khimov
ab7743d78d
Merge pull request #2619 from nspcc-dev/script-oob-and-hasmethod-3.4.0
Script OOB checks and HasMethod for 3.4.0
2022-08-04 12:41:43 +03:00
Roman Khimov
cfd2a35172
Merge pull request #2612 from nspcc-dev/fancy-service-restart
Fancy service restart
2022-08-02 14:11:44 +03:00
Roman Khimov
9341bb6628 cli: restart notary service on USR1 2022-07-28 19:05:56 +03:00
Roman Khimov
2adcf406d3 cli: reload Oracle service on USR1
Which allows to enable/disable the service, change nodes, keys and other
settings. Unfortunately, atomic.Value doesn't allow Store(nil), so we have to
store a pointer there that can point to nil interface.
2022-07-28 19:05:56 +03:00
Roman Khimov
61cfbbd33f Revert "vm/core: revert out-of-bounds script checks"
This reverts commit bd5644aa02 restoring changes
from #2538.
2022-07-28 17:01:23 +03:00
Roman Khimov
55f910777e Revert "native/interop: revert management.hasMethod()"
This reverts commit 6c7a401f77, introducing
changes from #2598 again.
2022-07-28 17:00:34 +03:00
Roman Khimov
4e98ca3358
Merge pull request #2616 from nspcc-dev/rollback-to-3.3.1
Rollback to 3.3.1
2022-07-28 16:00:49 +03:00
Roman Khimov
6c7a401f77 native/interop: revert management.hasMethod()
This reverts commits
 * f50bcf617a
 * 4f184498bc
 * ab3330564a

because they're 3.4.0-compatible while we need 3.3.1.
2022-07-27 23:26:24 +03:00
Roman Khimov
bd5644aa02 vm/core: revert out-of-bounds script checks
This reverts commits 1005c1f7db and
a5b5f88fe2 which are 3.4.0-compatible changes
while we need a 3.3.1-compatible release.
2022-07-27 23:25:23 +03:00
Roman Khimov
8e9bd3e28f native: trigger committee update on candidate registration
It can change the committee even if noone voted. Fixes state diff at block
390726 of T5 testnet where there are no transactions, but committee changes
because there were some registrations in previous 21 blocks.
2022-07-27 23:21:21 +03:00
Roman Khimov
e5c59f8ddd interop/runtime: disable notifications in dynamic scripts
That are only entry scripts today. See neo-project/neo#2796.
2022-07-27 14:49:53 +03:00
Roman Khimov
c8ff489287 native: use CreateOracleResponseScript directly
It wasn't possible way back when this test was written
(CreateOracleResponseScript was a method), now we can simplify things.
2022-07-26 12:19:49 +03:00
Roman Khimov
f749aaff3c *: reuse smartcontract package to create standard entry scripts 2022-07-26 12:19:49 +03:00
Roman Khimov
88542630ac blockchainer: drop the package completely
It's not an ideal solution, but at least it solves the problem for
now. Caveats:
 * consensus only needs one method, so it's mirrored to Blockchain
 * rpcsrv uses core.* definition of the StateRoot (so technically it might as
   well not have an internal Ledger), but it uses core already unfortunately
2022-07-25 11:58:13 +03:00
Roman Khimov
5ee7ea34b1 blockchainer: drop Blockchainer completely
It's only used by the RPC server now, so it can be internalized.
2022-07-25 11:58:13 +03:00
Roman Khimov
284335a4d2 blockchainer: strip unused methods 2022-07-25 11:58:13 +03:00
Roman Khimov
fcbda00f8a blockchainer/services: drop this package
It doesn't add any value.
2022-07-25 11:58:13 +03:00
Roman Khimov
3a6626f21f blockchainer: drop unused services dependency 2022-07-25 11:58:13 +03:00
Roman Khimov
4acd1688a1 subscriptions: move NotificationEvent to state
1. It's not good for pkg/core to import anything from pkg/neorpc.
2. The type is closely tied to the state package, even though it's not stored
   in the DB
2022-07-25 11:58:13 +03:00
Roman Khimov
1e0750e3cd rpc: merge response and request under pkg/neorpc
Move result there also.
2022-07-25 11:57:53 +03:00
Roman Khimov
43a59adbd0 rpc/server: move to services/rpcsrv
`server` is not a good package name and it's an internal service, so it can be
just about anywhere.
2022-07-21 22:14:12 +03:00
Roman Khimov
03b559bd44 block: JSONize tx-less block as [] instead of null
Improve C# compatibility.
2022-07-21 13:15:31 +03:00
Roman Khimov
ed53fd3221
Merge pull request #2600 from nspcc-dev/handle-mptdata
network: allow to handle GetMPTData with KeepOnlyLatestState on
2022-07-15 13:13:31 +03:00
Anna Shaleva
1ae601787d network: allow to handle GetMPTData with KeepOnlyLatestState on
And adjust documentation along the way.
2022-07-14 14:33:20 +03:00
Roman Khimov
ab3330564a native: add hasMethod, fix #2588
Yet another state change.
2022-07-13 17:29:46 +03:00
Roman Khimov
125c2805d3 storage: reduce lock time in (*MemoryStore).Seek
It makes a copy of the resulting set, so the lock can be released
earlier. This helps a lot with iterators that keep Seek() unfinished for a
long time,
2022-07-11 16:15:14 +03:00
Roman Khimov
96c4e61063 storage: move Operation into package of its own
Don't use storage.* types in rpc/response/result.
2022-07-08 23:30:30 +03:00
Roman Khimov
8cd7b93208 limits: new package with storage limits
Packages like core/state or core/mpt shouldn't import whole core/storage just
to get some constant value, it's not a good dependency.
2022-07-08 23:30:30 +03:00
Roman Khimov
9987afea4c storage: move DB configuration into a package on its own
Lightweight thing to import anywhere, pkg/config should not be dependent on
Level/Bolt/anything else.
2022-07-08 23:30:30 +03:00
Roman Khimov
fab8dfb9f8 vm: move State type into a package of its own
It's used a lot in other places that need it, but don't need whole VM at the
same time.
2022-07-08 18:34:52 +03:00
Anna Shaleva
cbd20eb959 rpc: implement iterator sessions 2022-07-08 17:05:18 +03:00
Roman Khimov
251c9bd89b block: push PrevStateRoot data into stack item, fix #2551
And add compiler/interop support for this.
2022-07-07 15:10:29 +03:00
Roman Khimov
926b082d39
Merge pull request #2538 from nspcc-dev/restrict-out-of-bounds
core: check methods offsets while contract deploying
2022-07-04 12:54:32 +03:00
Anna Shaleva
7c48edaccb core, rpc: fix height-related issue of historic call functionality
Specifying a certain stateroot R as `invoke*historic` RPC-call
parameter, we're willing to perform historic call based on the storage
state of root R. Thus, next block should be of the height h(R)+1. This
allows to use historic functionality for the current blockchain height.
2022-06-20 19:05:22 +03:00
Roman Khimov
5108d1c2c7
Merge pull request #2539 from nspcc-dev/rollback-methods-sorting
core: sort native methods in ASCII-compatible way
2022-06-14 22:09:23 +03:00
Roman Khimov
7b88aedb27 transaction: add fuzz-test, refs. #2374 2022-06-09 13:43:22 +03:00
Roman Khimov
1b7c8f262b core: add some comment for TestDesignate_DesignateAsRole
Closes #1472 at last.
2022-06-08 23:11:14 +03:00
Roman Khimov
ef50518b83 core: move notary tests into the notary package 2022-06-08 23:07:28 +03:00
Roman Khimov
5d573895ad core: move oracle tests into oracle service 2022-06-08 23:04:47 +03:00
Roman Khimov
017cd558cb core: move statesync tests into the statesync package 2022-06-08 22:57:48 +03:00
Roman Khimov
ff75e67610 core: move stateroot tests into services/stateroot
They test both module and service which is a bit wrong, but separating these
tests will lead to some duplication, so it's OK for now to have them in the
higher-order package (service imports module).
2022-06-08 22:53:09 +03:00
Roman Khimov
167bb72424 core: move Policy contract tests to native 2022-06-08 22:44:25 +03:00
Roman Khimov
a1eccc16e6 core: move benchmarks into bench_test.go
They share some functions and they're benchmarks, so let's have them here.
2022-06-08 22:39:57 +03:00
Roman Khimov
2e2d886a2f core: move Management contract test into native 2022-06-08 22:34:09 +03:00
Roman Khimov
8057c096c6 core: move native invocation tests into native 2022-06-08 22:26:24 +03:00
Roman Khimov
2086bca303 core: move storage-related interop code into the storage package 2022-06-08 22:26:24 +03:00
Roman Khimov
f0d7a1da2a core: move contract-related tests to the contract package 2022-06-08 19:01:34 +03:00
Roman Khimov
e7e80fda64 core: move TestNativeGetMethod to the native package 2022-06-08 18:51:27 +03:00
Roman Khimov
2127cc4146 core: move Runtime tests to runtime package 2022-06-08 18:46:49 +03:00
Roman Khimov
0055b18a8a core: export CreateGenesisBlock
Nothing bad with it being public.
2022-06-08 18:20:34 +03:00
Roman Khimov
d70caf1da1 core: move GetScriptContainer to runtime
It also brings ToStackItem to Block and Transaction, previously this was
avoided to separate block and transaction packages from VM. But turns out
`transaction` depends on `stackitem` already, so this makes little sense (but
can be shuffled in another way if needed).

Context.Container is still a hash.Hashable because we have a number of
occasions (header or MPT root verification) where there is no ToStackItem
implementation possible. Maybe they can go with `nil` Container, but I don't
want to have this risk for now.
2022-06-08 18:12:41 +03:00
Roman Khimov
cdb55740ea core: move ContractCreate*Account tests into the contract pkg
No functional changes.
2022-06-08 16:56:06 +03:00
Roman Khimov
ff1545b7eb core: move TestContractCall into the contract package
It belongs there and now it can be moved. No functional changes.
2022-06-08 16:53:01 +03:00
Roman Khimov
bb021d0778 native: unbind PutContractState from Management
It doesn't need Management's state, ID can't really change.
2022-06-08 16:41:28 +03:00
Roman Khimov
209b977e9a core: move contract-related interop code into appropriate package
And move one of the tests with it.
2022-06-08 16:02:07 +03:00
Roman Khimov
10f94e6119 core: move chain dump test into its own package 2022-06-08 15:28:08 +03:00
Roman Khimov
c2b3ee3d8e core: move basic chain creation into a package of its own
This allows to reuse it across different packages.

testchain can't be used because of circular dependencies.

Init() is not changed except for filepath.Join() use instead of direct string
appends which is a better approach anyway. rootpath is required because
current directory will change from package to package.
2022-06-08 15:25:27 +03:00
Roman Khimov
92c94f265c interop/vm: make VM reusable and use on VM for all in-block execs
Avoid allocating again and again. Increases TPS by about 3%.
2022-06-07 10:05:47 +03:00
Roman Khimov
bdc6624c9d interop: unify VM price getter setting 2022-06-06 22:00:16 +03:00
Roman Khimov
638b04b29a interop: wrap contract.LoadToken in context.LoadToken
Creating a closure in runtime is a relatively costly thing, but it can easily
be avoided.
2022-06-06 21:53:03 +03:00
Anna Shaleva
a5b5f88fe2 core: doesn't allow to deploy contract with invalid method offset
This commit partially reverts 16bf7c1426.
2022-06-06 12:18:16 +03:00
Anna Shaleva
bc85f00cc7 Partially revert "core: fix native NEO ABI"
This partially reverts commit ca127f1615.
2022-06-06 12:08:52 +03:00
Roman Khimov
799394192b state: create buffer/io writer once per TokenTransferLog
name                       old time/op    new time/op    delta
TokenTransferLog_Append-8   93.0µs ±170%   46.8µs ±152%     ~     (p=0.053 n=10+9)

name                       old alloc/op   new alloc/op   delta
TokenTransferLog_Append-8    53.8kB ± 4%    38.6kB ±39%  -28.26%  (p=0.004 n=8+10)

name                       old allocs/op  new allocs/op  delta
TokenTransferLog_Append-8       384 ± 0%       128 ± 0%  -66.67%  (p=0.000 n=10+10)
2022-06-04 00:11:11 +03:00
Roman Khimov
1e5825c4af core: don't allocate another int for notification handler
It'll be serialized anyway.
2022-06-03 23:39:46 +03:00
Anna Shaleva
d3b15a60a2 core: add test for ABI checks on deploy
Make sure it compatible with the reference implementation, see
736c346b9d/src/neo/SmartContract/Helper.cs (L83).
2022-06-03 11:37:50 +03:00
Anna Shaleva
16bf7c1426 core: adjust contract script check on deploy
Reference implementation doesn't panic if the method offset is out of
the contract script bounds, see:
736c346b9d/src/neo/SmartContract/Helper.cs (L82)
and
a65487fa56/src/Neo.VM/Script.cs (L146).

This commit fixes T5 statediff at block #125000. Neo-go node FAULTed the
deploying transaction:
```
         {
            "version" : 0,
            "sysfee" : "1000106065",
            "validuntilblock" : 130758,
            "script" : "DdMDeyJuYW1lIjoiTmVwMTdUb2tlbiIsImdyb3VwcyI6W10sImZlYXR1cmVzIjp7fSwic3VwcG9ydGVkc3RhbmRhcmRzIjpbIk5FUC0xNyJdLCJhYmkiOnsibWV0aG9kcyI6W3sibmFtZSI6InN5bWJvbCIsInBhcmFtZXRlcnMiOltdLCJyZXR1cm50eXBlIjoiU3RyaW5nIiwib2Zmc2V0IjoyMiwic2FmZSI6dHJ1ZX0seyJuYW1lIjoiZGVjaW1hbHMiLCJwYXJhbWV0ZXJzIjpbXSwicmV0dXJudHlwZSI6IkludGVnZXIiLCJvZmZzZXQiOjIyLCJzYWZlIjp0cnVlfSx7Im5hbWUiOiJ0b3RhbFN1cHBseSIsInBhcmFtZXRlcnMiOltdLCJyZXR1cm50eXBlIjoiSW50ZWdlciIsIm9mZnNldCI6MjIsInNhZmUiOnRydWV9LHsibmFtZSI6ImJhbGFuY2VPZiIsInBhcmFtZXRlcnMiOlt7Im5hbWUiOiJvd25lciIsInR5cGUiOiJIYXNoMTYwIn1dLCJyZXR1cm50eXBlIjoiSW50ZWdlciIsIm9mZnNldCI6MjIsInNhZmUiOnRydWV9LHsibmFtZSI6InRyYW5zZmVyIiwicGFyYW1ldGVycyI6W3sibmFtZSI6ImZyb20iLCJ0eXBlIjoiSGFzaDE2MCJ9LHsibmFtZSI6InRvIiwidHlwZSI6Ikhhc2gxNjAifSx7Im5hbWUiOiJhbW91bnQiLCJ0eXBlIjoiSW50ZWdlciJ9LHsibmFtZSI6ImRhdGEiLCJ0eXBlIjoiQW55In1dLCJyZXR1cm50eXBlIjoiQm9vbGVhbiIsIm9mZnNldCI6MjIsInNhZmUiOmZhbHNlfV0sImV2ZW50cyI6W3sibmFtZSI6IlRyYW5zZmVyIiwicGFyYW1ldGVycyI6W3sibmFtZSI6ImZyb20iLCJ0eXBlIjoiSGFzaDE2MCJ9LHsibmFtZSI6InRvIiwidHlwZSI6Ikhhc2gxNjAifSx7Im5hbWUiOiJhbW91bnQiLCJ0eXBlIjoiSW50ZWdlciJ9XX1dfSwicGVybWlzc2lvbnMiOlt7ImNvbnRyYWN0IjoiKiIsIm1ldGhvZHMiOiIqIn1dLCJ0cnVzdHMiOltdLCJleHRyYSI6eyJlbWFpbCI6ImRldmVsb3BlckBuZW8ub3JnIiwiYXV0aG9yIjoibGF6eW5vZGUiLCJkZXNjcmlwdGlvbiI6IkEgU2ltcGxlIE5lcC0xNyBDb250cmFjdCJ9fQyyTkVGM25lb21sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABOaHR0cHM6Ly9naXRodWIuY29tL2xhenlub2RlL25lb21sL2Jsb2IvZGV2L2V4YW1wbGVzL2EuZnVuY3Rpb24uc2ltcGxlbmVwMTcueG1sAAAAABYMBU5lb01MQBhAAgDh9QVAATkFQBFAkA456BLAHwwGZGVwbG95DBT9o/pDRupTKiWPxJfdrdtkN8n9/0FifVtS",
            "hash" : "0x40302bcf2021f63a1c24f6009e154c3200f73ad2fe1462d7d599145823dbfa7e",
            "witnesses" : [
               {
                  "verification" : "DCECExn08eznGBdguHbcwI+R2//EtVdDx4qf6CeizHqOJgBBVuezJw==",
                  "invocation" : "DEBtoq+T9NrammQjuYnifco7KHCTk2v+woEJqJCUMr9IscS7PaZaN3FNzSt11yUglIi3T0CJ17KwArBOBvJ8kwq2"
               }
            ],
            "attributes" : [],
            "signers" : [
               {
                  "scopes" : "None",
                  "account" : "0x13a192c56738900f9918d7f1ec07d9d8c278b804"
               }
            ],
            "size" : 1360,
            "nonce" : 1829882407,
            "sender" : "NLLvsqs7AyBNmQT6NThUxYWDFwV5b1evaK",
            "netfee" : "234352"
         }
```
Transaction script contains malformed contract manifest (all methods
offsets are set to be 22, while the contract script lenght is 22):
```
{
   "name" : "Nep17Token",
   "groups" : [],
   "extra" : {
      "description" : "A Simple Nep-17 Contract",
      "email" : "developer@neo.org",
      "author" : "lazynode"
   },
   "permissions" : [
      {
         "contract" : "*",
         "methods" : "*"
      }
   ],
   "features" : {},
   "supportedstandards" : [
      "NEP-17"
   ],
   "abi" : {
      "events" : [
         {
            "parameters" : [
               {
                  "name" : "from",
                  "type" : "Hash160"
               },
               {
                  "type" : "Hash160",
                  "name" : "to"
               },
               {
                  "name" : "amount",
                  "type" : "Integer"
               }
            ],
            "name" : "Transfer"
         }
      ],
      "methods" : [
         {
            "safe" : true,
            "offset" : 22,
            "name" : "symbol",
            "returntype" : "String",
            "parameters" : []
         },
         {
            "returntype" : "Integer",
            "parameters" : [],
            "safe" : true,
            "offset" : 22,
            "name" : "decimals"
         },
         {
            "parameters" : [],
            "returntype" : "Integer",
            "name" : "totalSupply",
            "safe" : true,
            "offset" : 22
         },
         {
            "parameters" : [
               {
                  "name" : "owner",
                  "type" : "Hash160"
               }
            ],
            "returntype" : "Integer",
            "name" : "balanceOf",
            "offset" : 22,
            "safe" : true
         },
         {
            "name" : "transfer",
            "offset" : 22,
            "safe" : false,
            "parameters" : [
               {
                  "type" : "Hash160",
                  "name" : "from"
               },
               {
                  "name" : "to",
                  "type" : "Hash160"
               },
               {
                  "name" : "amount",
                  "type" : "Integer"
               },
               {
                  "name" : "data",
                  "type" : "Any"
               }
            ],
            "returntype" : "Boolean"
         }
      ]
   },
   "trusts" : []
}
```
2022-06-03 11:37:50 +03:00
Anna Shaleva
1472c271e6 core: fix native NEO's getCandidateVote signature
This commit fixes T5 statediff at block #0. Management's storage item
differs between neo-go and C# nodes, the reason in native NEO contract
state.

The parameter should have `pubKey` name, unlike the other `pubkey`
arguments in this contract.
2022-06-03 11:37:50 +03:00
Anna Shaleva
ca127f1615 core: fix native NEO ABI
This commit fixes T5 statediff at block #0. The reason in Management's
storage state. The diff occurs because of inconsistent NEO methods
order. See
https://github.com/neo-project/neo/issues/2766#issue-1257870089.

The current solution is to preserve C#'s order of methods to be
compatible with current testnet.

See also
https://docs.microsoft.com/ru-ru/dotnet/api/system.stringcomparer?view=net-6.0
and
https://stackoverflow.com/questions/28638714/easiest-method-to-orderby-a-string-using-stringcomparison-ordinal
for more details.
2022-06-03 11:37:41 +03:00
Roman Khimov
edb6ca8926
Merge pull request #2531 from nspcc-dev/perf
Minor performance improvements
2022-06-03 10:40:56 +03:00
Roman Khimov
400cbde32c state: optimize allocations in (*NEP17Transfer).EncodeBinary 2022-06-02 15:38:39 +03:00
Roman Khimov
9a06995460 bigint: don't allocate in ToPreallocatedBytes
Turns out, it's almost always allocating because we're mostly dealing with
small integers while the buffer size is calculated in 8-byte chunks here, so
preallocated buffer is always insufficient.

name                   old time/op    new time/op    delta
ToPreallocatedBytes-8    28.5ns ± 7%    19.7ns ± 5%   -30.72%  (p=0.000 n=10+10)

name                   old alloc/op   new alloc/op   delta
ToPreallocatedBytes-8     16.0B ± 0%      0.0B       -100.00%  (p=0.000 n=10+10)

name                   old allocs/op  new allocs/op  delta
ToPreallocatedBytes-8      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)

Fix StorageItem reuse at the same time. We don't copy when getting values from
the storage, but we don when we're putting them, so buffer reuse could corrupt
old values.
2022-06-02 15:38:39 +03:00
Roman Khimov
c3d989ebda stackitem: reusable serialization context
We serialize items a lot and this allows to avoid a number of allocations.
2022-06-02 15:38:39 +03:00
Anna Shaleva
3bec6657f5 core: adjust notary-related attributes encoding 2022-06-01 15:09:06 +03:00
Anna Shaleva
1b80215415 core: adjust types of NVB, Conflicts and NotaryAssisted attributes 2022-06-01 15:09:06 +03:00
Anna Shaleva
d540f16e37 core: adjust noderoles package usages
Don't use interop roles package for internal tests.
2022-06-01 15:03:29 +03:00
Anna Shaleva
56d5839a72 core: log stateroot N20 hash inside TestCreateBasicChain 2022-06-01 15:03:29 +03:00
AnnaShaleva
50157b31cd core: add maximum constraint to setNotaryServiceFeePerKey 2022-06-01 15:03:29 +03:00
AnnaShaleva
d7afb7bb2f core: adjust value of P2PNotary node role 2022-06-01 15:03:29 +03:00
AnnaShaleva
9008cc500c core: adjust Notary contract ID 2022-06-01 15:03:29 +03:00
Roman Khimov
3509523baf contract: tune isolation logic
Wrap when we have outer exception handler and the method is not
safe. Otherwise we wrap almost always, since non-readonly methods are very
common.
2022-05-31 11:44:12 +03:00
Roman Khimov
e1607e23c2
Merge pull request #2525 from nspcc-dev/immutable-items
vm: implement immutable stackitems
2022-05-31 10:36:56 +03:00
Anna Shaleva
42a051e55a core: DeepCopy notifiction event args inside System.Runtime.Notify 2022-05-31 08:07:53 +03:00
Anna Shaleva
7296f0c913 vm: support immutable compound types 2022-05-31 08:07:50 +03:00
Roman Khimov
5e013311e6 native: emit vote/candidate events
Follow neo-project/neo#2754.
2022-05-28 18:44:20 +03:00