Commit graph

4808 commits

Author SHA1 Message Date
Roman Khimov
cb5c18dc73 interop: add getContractByID and getContractHashes to management 2022-12-02 10:54:45 +03:00
Roman Khimov
4a626f505e core: drop old STContractID data
We have it in the ContractManagement now.
2022-12-02 10:54:45 +03:00
Roman Khimov
970862765d native: implement management.getContractById
Follow neo-project/neo#2807. Notice that this data is not cached, our previous
implementation wasn't too and it shouldn't be a problem (not on the hot path).
2022-12-02 10:21:08 +03:00
Roman Khimov
6c68da7a52 rpcsrv: add block 20 hash comparison test
Makes updating it somewhat easier.
2022-12-02 10:21:08 +03:00
Roman Khimov
f6a9969fa8
Merge pull request #2770 from nspcc-dev/push-bool
vm: add PUSHT and PUSHF opcodes
2022-12-02 14:20:11 +07:00
Roman Khimov
389bdfd1b6
Merge pull request #2719 from nspcc-dev/eval
runtime: implement System.Runtime.LoadScript, fix #2701
2022-12-02 14:08:40 +07:00
Roman Khimov
93cec0bba5
Merge pull request #2786 from nspcc-dev/simplify-getversion
Simplify result.Version
2022-12-02 14:07:55 +07:00
Roman Khimov
3ef66239a8
Merge pull request #2814 from nspcc-dev/startup-time
Optimize startup time and memory usage
2022-12-02 14:06:23 +07:00
Anna Shaleva
2b140631f4 rpc: adjust Boolean handling in ExpandArrayIntoScript
Use new PUSHT and PUSHF opcodes for Boolean scparameter.
2022-12-02 10:02:33 +03:00
Anna Shaleva
1250e82c2a vm: add PUSHT and PUSHF opcodes
Port https://github.com/neo-project/neo-vm/pull/497.
2022-12-02 10:02:33 +03:00
Roman Khimov
6912695b9d
Merge pull request #2826 from nspcc-dev/check-for-max-block-sysfee
core: filter out txes with system fee > MaxBlockSystemFee
2022-11-29 14:57:10 +07:00
Roman Khimov
6847e1760c core: filter out txes with system fee > MaxBlockSystemFee
They can stay in the memory pool forever because consensus process will never
accept these transactions (and maybe even block consensus process at all).
2022-11-29 10:31:00 +03:00
Anna Shaleva
6d3cc0f7e7 core: extend error text for storage context in dynamic script 2022-11-28 17:17:36 +03:00
Anna Shaleva
0ceae612c6 vm: improve failed SYSCALL logging
For better user experience.
2022-11-28 17:16:27 +03:00
Roman Khimov
07e32e221d *: update interops, add System.Runtime.LoadScript wrapper check 2022-11-25 15:16:27 +03:00
Roman Khimov
de2de986a7 interop: add runtime.LoadScript 2022-11-25 15:11:49 +03:00
Roman Khimov
4e26f4b26e runtime: implement System.Runtime.LoadScript, fix #2701 2022-11-25 15:11:49 +03:00
Roman Khimov
1c38b45074 core: don't always store all hashes in memory
We're paging these hashes, so we need a previous full page and a current one
plus some cache for various requests. Storing 1M of hashes is 32M of memory
and it grows quickly. It also seriously affects node startup time, most of
what it's doing is reading these hashes, the longer the chain the more time it
needs to do that.

Notice that this doesn't change the underlying DB scheme in any way.
2022-11-25 14:30:51 +03:00
Roman Khimov
0ad6e295ea core: make GetHeaderHash accept uint32
It should've always been this way because block indexes are uint32.
2022-11-25 14:30:51 +03:00
Roman Khimov
16db4746cc core: drop another useless genesis-specific header handling branch
If we only have genesis block (or <2000 headers) then we might as well use
generic logic below with zero targetHash because genesis block has zero
PrevHash (and its hash will naturally be the last on the chain going
backwards).
2022-11-25 13:08:03 +03:00
Roman Khimov
83e2ca054f core: simplify header hash list restoration logic
We don't care about header, we only need hashes here.
2022-11-25 13:08:03 +03:00
Roman Khimov
ff284d5d5c core: remove genesis-specific header hash logic from init
The code below will do the same thing (via CreateGenesisBlock, but it doesn't
matter) generically, so this branch is useless.
2022-11-25 13:08:03 +03:00
Roman Khimov
327dfb8dbf rpcsrv: add wsclient error message test for #2818
Now it can be done easily.
2022-11-23 12:24:20 +03:00
Roman Khimov
2591c39500 rpcsrv: make websocket client limit configurable 2022-11-23 12:19:49 +03:00
Roman Khimov
fb09670fd1 rpcclient: extract more detailed server-side on WS connection problem
If available. Fixes #2818.
2022-11-23 12:19:35 +03:00
Roman Khimov
ab0b23625b
Merge pull request #2817 from nspcc-dev/always-return-hash-vub-from-sender
Some actor/waiter interaction fixes
2022-11-23 14:37:38 +07:00
Roman Khimov
fd04b2befd actor: don't abort waiter on "already exists" error
It can happen in many cases of distributed tx generation/submission, we can
just wait normally in this case and there will be some proper result.
2022-11-23 10:07:37 +03:00
Roman Khimov
66ddeccdad
Merge pull request #2813 from nspcc-dev/fix-state-reset
core: fix broken state reset
2022-11-23 13:43:42 +07:00
Anna Shaleva
f3ef2890f0 core: check headers at the proper state on state reset
And fix the comment along the way.
2022-11-23 09:16:33 +03:00
Roman Khimov
cd6bb68246 actor: check for tx after the subscription in wsWaiter, fix #2805
Don't wait for VUB block, solve this race immediately.
2022-11-22 17:28:55 +03:00
Roman Khimov
c95d140113 rpcclient: always return tx hash from sendrawtransaction
Let upper-layer APIs like actor.Send() return it as well. Server can return
"already exists" which is an error and yet at the same time a very special
one, in many cases it means we can proceed with waiting for the TX to settle.
2022-11-22 15:18:37 +03:00
Anna Shaleva
b27a9bcf95 core: adjust info message for proper-stated chains
Make it prettier for those cases when `db reset` command was called
after interrupted reset.
2022-11-22 11:53:39 +03:00
Anna Shaleva
b82374823e core: increase persist batch size for reset storage changes 2022-11-22 11:53:39 +03:00
Anna Shaleva
bdc42cd595 core: reset blocks, txs and AERs in several stages
Sometimes it can be hard to persist all changes at ones, the process
can take almost all RAM and a lot of time. Here's the example of reset
for mainnet from 2.4M to 1:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:16:48.236+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2022-11-20T17:16:48.236+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2022-11-20T17:16:48.237+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2022-11-20T17:16:48.237+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:16:48.240+0300	INFO	restoring blockchain	{"version": "0.2.6"}
2022-11-20T17:16:48.297+0300	INFO	initialize state reset	{"target height": 1}
2022-11-20T17:16:48.300+0300	INFO	trying to reset blocks, transactions and AERs
2022-11-20T17:19:29.313+0300	INFO	blocks, transactions ans AERs are reset	{"took": "2m41.015126493s", "keys": 3958420}
...
```
To avoid OOM killer, split blocks reset into multiple stages. It increases
operation time due to intermediate DB persists, but makes things cleaner, the
result for almost the same DB height with the new approach:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:39:42.023+0300	INFO	MaxBlockSize is not set or wrong, setting default value	{"MaxBlockSize": 262144}
2022-11-20T17:39:42.023+0300	INFO	MaxBlockSystemFee is not set or wrong, setting default value	{"MaxBlockSystemFee": 900000000000}
2022-11-20T17:39:42.023+0300	INFO	MaxTransactionsPerBlock is not set or wrong, using default value	{"MaxTransactionsPerBlock": 512}
2022-11-20T17:39:42.023+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:39:42.026+0300	INFO	restoring blockchain	{"version": "0.2.6"}
2022-11-20T17:39:42.071+0300	INFO	initialize state reset	{"target height": 1}
2022-11-20T17:39:42.073+0300	INFO	trying to reset blocks, transactions and AERs
2022-11-20T17:40:11.735+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 1, "took": "29.66363737s", "keys": 210973}
2022-11-20T17:40:33.574+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 2, "took": "21.839208683s", "keys": 241203}
2022-11-20T17:41:29.325+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 3, "took": "55.750698386s", "keys": 250593}
2022-11-20T17:42:12.532+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 4, "took": "43.205892757s", "keys": 321896}
2022-11-20T17:43:07.978+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 5, "took": "55.445398156s", "keys": 334822}
2022-11-20T17:43:35.603+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 6, "took": "27.625292032s", "keys": 317131}
2022-11-20T17:43:51.747+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 7, "took": "16.144359017s", "keys": 355832}
2022-11-20T17:44:05.176+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 8, "took": "13.428733899s", "keys": 357690}
2022-11-20T17:44:32.895+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 9, "took": "27.718548783s", "keys": 393356}
2022-11-20T17:44:51.814+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 10, "took": "18.917954658s", "keys": 366492}
2022-11-20T17:45:07.208+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 11, "took": "15.392642196s", "keys": 326030}
2022-11-20T17:45:18.776+0300	INFO	intermediate batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 12, "took": "11.568255716s", "keys": 299884}
2022-11-20T17:45:25.862+0300	INFO	last batch of removed blocks, transactions and AERs is persisted	{"batches persisted": 13, "took": "7.086079594s", "keys": 190399}
2022-11-20T17:45:25.862+0300	INFO	blocks, transactions ans AERs are reset	{"took": "5m43.791214084s", "overall persisted keys": 3966301}
...
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
d67f0df516 core: reset block headers together with header height info
We need to keep the headers information consistent with header batches
and headers. This comit fixes the bug with failing blockchain
initialization on recovering from state reset interrupted after the
second stage (blocks/txs/AERs removal):
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T16:28:29.437+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T16:28:29.440+0300	INFO	restoring blockchain	{"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: could not get header 1898cd356a4a2688ed1c6c7ba1fd6ba7d516959d8add3f8dd26232474d4539bd: key not found
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
283da8f599 core: use DAO-provided block height during during state reset
Don't use cache because it's not yet initialized. Also, perform
safety checks only if state reset wasn't yet started. These fixes
alloww to solve the following problem while recovering from
interrupted state reset:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T15:51:31.431+0300	INFO	MaxValidUntilBlockIncrement is not set or wrong, using default value	{"MaxValidUntilBlockIncrement": 5760}
2022-11-20T15:51:31.434+0300	INFO	restoring blockchain	{"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: current block height is 0, can't reset state to height 83000
```
2022-11-22 11:53:39 +03:00
Anna Shaleva
7d55bf2cc1 core: log persisted storage item batches count during state reset 2022-11-22 11:53:39 +03:00
Anna Shaleva
f52451e582 core: fix state reset with broken contract
Sync up with #2802, bad contract -> no contract ID at all.
2022-11-22 11:53:39 +03:00
Anna Shaleva
ecda07736e core: stop storage items reset after any seek error 2022-11-22 11:53:39 +03:00
Anna Shaleva
bfe7aeae7b core: stop storage items reset after the first persist error
It's a bug, we mustn't continue if something bad had happend on persist,
otherwise this error will be overwritten by subsequent successfull persist.
2022-11-22 11:53:39 +03:00
Anna Shaleva
235518eb6c core: reset batch counter to zero after each persist in resetStateInternal
It's a bug, otherwise we'll persist each storage item after 10K-th one,
that's the reason of abnormous long storage items resetting stage.
2022-11-22 11:53:39 +03:00
Anna Shaleva
9f23fafc03 core: improve logging of resetStateInternal
Inform when starting subsequent stage, inform about keys persisted.
2022-11-22 11:53:36 +03:00
Roman Khimov
0039615ae3
Merge pull request #2816 from nspcc-dev/fix-pointer-serialization
Fix pointer serialization
2022-11-20 22:59:52 +07:00
Roman Khimov
9ba18b5dfa stackitem: serialize/deserialize pointers, fix #2815
They of course can't be serialized, but in protected mode we still need to
handle them somehow.
2022-11-20 16:02:50 +03:00
Roman Khimov
48140320db
Merge pull request #2812 from nspcc-dev/improve-vm-context-handling
Improve vm istack/estack handling
2022-11-20 19:42:35 +07:00
Roman Khimov
ca9fde745b
Merge pull request #2809 from nspcc-dev/fix-subs
rpcsrv: do not block blockchain events receiver by subscription requests
2022-11-18 16:16:41 +07:00
Roman Khimov
8e7f65be17 vm: use proper estack for exception handler
v.estack might be some inner invoked contract and its stack must not be used
for exception handler set up by higher-order contract.
2022-11-18 11:36:38 +03:00
Roman Khimov
cb64957af5 vm: don't use Stack for istack
We don't use all of the Stack functionality for it, so drop useless methods
and avoid some interface conversions. It increases single-node TPS by about
0.9%, so nothing really important there, but not a bad change either. Maybe it
can be reworked again with generics though.
2022-11-18 11:35:29 +03:00
Anna Shaleva
4df9a5e379 rpcsrv: refactor subscribe routine
Move shutdown check after subsCounterLock is taken in the end of
`(s *Server) subscribe` in order to avoid extra locks holding.
2022-11-18 10:54:10 +03:00
Anna Shaleva
b7f19a54d5 services: fix chain locked by WS subscriptions handlers
Blockchain's subscriptions, unsubscriptions and notifications are
handled by a single notificationDispatcher routine. Thus, on attempt
to send the subsequent event to Blockchain's subscribers, dispatcher
can't handle subscriptions\unsubscriptions. Make subscription and
unsubscription to be a non-blocking operation for blockchain on the
server side, otherwise it may cause the dispatcher locks.

To achieve this, use a separate lock for those code that make calls
to blockchain's subscription API and for subscription counters on
the server side.
2022-11-18 09:30:12 +03:00