Prevent the risk of a division by zero error when accessing the
`o.MainCfg.NeoFS.Nodes[index]` array.
Close#3419
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Do not use the updated contract state from native Management to perform
permissions checks. We need to use the currently executing state
instead got from the currently executing VM context until context is
unloaded.
Close#3471.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Account is blocked when it's in the Policy's storage, not when it's
missing from the Policy storage. Introduced in
bbbc6805a8.
This bug leads to the fact that during native Neo cache initialization
at the last block in the dBFT epoch, all candidates accounts are
"blocked", and thus, stand-by committee and validators are used in the
subsequent new epoch. Close#3424.
This bug may lead to the consequences described in #3273, but it needs
to be confirmed.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
It's possible for transaction to include block hash into Conflicts
attribure. If so, then we must not remove block executable record while
cleaning transation's conflict records.
This commit is a direct consequence of
e6ceee0f230a21c87006a9297636be29c0d8ea47. Ref. #3427.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Conflicts-related code contains more and more these magic numbers, and
there's no good in it even if all the usages are commented. This
approach produces bugs like #3426.
No functional changes, just a refactoring.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Conflict record stub has value of 5 bytes length: 1 byte for
storage.ExecTransaction prefix and 4 bytes for the block index LE. This
scheme was implemented in #3138, and this commit should be a part of
this PR.
Also, transaction.DummyVersion is removed since it's unused anymore.
Close#3426. The reason of `failed to locate application log: EOF` error
during genesis AER request is in the following: genesis executable was
overwritten by conflict record stub produced by transaction
0x289c235dcdab8be7426d05f0fbb5e86c619f81481ea136493fa95deee5dbb7cc (ref.
#3427). As a consequence, an attempt to decode transaction AER was
initited, but conflict record scheme was changed in #3138.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Transaction
0x289c235dcdab8be7426d05f0fbb5e86c619f81481ea136493fa95deee5dbb7cc is
already on mainnet at block 5272006 and we can't do anything with it.
This transaction has genesis block hash in Conflicts attribute. It leads
to the following consequences:
1. Genesis block executable record is overwritten by conflict record
stub. Genesis block can't be retrieved anymore. This bug is described
in #3427.
2. Somehow this transaction has passed verification on NeoGo CN without
any warnings:
```
Apr 24 16:12:30 kangra neo-go[2453907]: 2024-04-24T16:12:30.865+0300 INFO initializing dbft {"height": 5272006, "view": 0, "index": 6, "role": "Backup"}
Apr 24 16:12:31 kangra neo-go[2453907]: 2024-04-24T16:12:31.245+0300 INFO persisted to disk {"blocks": 1, "keys": 37, "headerHeight": 5272005, "blockHeight": 5272005, "took": "14.548903ms"}
Apr 24 16:12:34 kangra neo-go[2453907]: 2024-04-24T16:12:34.977+0300 ERROR can't add SV-signed state root {"error": "stateroot mismatch at block 5272005: 9d5f95784f26c862d6f889f213aad1e3330611880c02330e88db8802c750aa46 vs d25304d518645df725014897d13bbf023919928e79074abcea48f31cf9f32a25"}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.820+0300 INFO received PrepareRequest {"validator": 5, "tx": 1}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.821+0300 INFO sending PrepareResponse {"height": 5272006, "view": 0}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.827+0300 INFO received PrepareResponse {"validator": 4}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.830+0300 INFO received PrepareResponse {"validator": 3}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.875+0300 INFO received PrepareResponse {"validator": 2}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.878+0300 INFO sending Commit {"height": 5272006, "view": 0}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.879+0300 INFO received Commit {"validator": 4}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.881+0300 INFO received PrepareResponse {"validator": 0}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.881+0300 INFO received Commit {"validator": 3}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.906+0300 INFO received Commit {"validator": 0}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.907+0300 INFO received PrepareResponse {"validator": 1}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.915+0300 INFO received Commit {"validator": 1}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.915+0300 INFO approving block {"height": 5272006, "hash": "6b111519537343ce579d04ccad71c43318b12c680d0f374dfcd466aa22643fb6", "tx_count": 1, "merkle": "ccb7dbe5ee5da93f4936a11e48819f616ce8b5fbf0056d42e78babcd5d239c28", "prev": "12ad6cc5d0cd357b9fc9fb0c1a016ba8014d3cdd5a96818598e6a40a1a4a2a21"}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.917+0300 WARN contract invocation failed {"tx": "289c235dcdab8be7426d05f0fbb5e86c619f81481ea136493fa95deee5dbb7cc", "block": 5272006, "error": "at instruction 86 (ASSERT): ASSERT failed"}
Apr 24 16:12:45 kangra neo-go[2453907]: 2024-04-24T16:12:45.950+0300 INFO initializing dbft {"height": 5272007, "view": 0, "index": 6, "role": "Primary"}
Apr 24 16:12:46 kangra neo-go[2453907]: 2024-04-24T16:12:46.256+0300 INFO persisted to disk {"blocks": 1, "keys": 67, "headerHeight": 5272006, "blockHeight": 5272006, "took": "16.576594ms"}
```
And thus, we must treat this transaction as valid for this behaviour
to be reproducable.
This commit contains two fixes:
1. Do not overwrite block executable records by conflict record stubs.
If some transaction conflicts with block, then just skip the conflict
record stub for this attribute since it's impossible to create
transaction with the same hash.
2. Do not fail verification for those transactions that have Conflicts
attribute with block hash inside. This one is controversial, but we
have to adjust this code to treat already accepted transaction as
valid.
Close#3427.
The transaction itself:
```
{
"id" : 1,
"jsonrpc" : "2.0",
"result" : {
"attributes" : [
{
"height" : 0,
"type" : "NotValidBefore"
},
{
"hash" : "0x1f4d1defa46faa5e7b9b8d3f79a06bec777d7c26c4aa5f6f5899a291daa87c15",
"type" : "Conflicts"
}
],
"blockhash" : "0xb63f6422aa66d4fc4d370f0d682cb11833c471adcc049d57ce4373531915116b",
"blocktime" : 1713964365700,
"confirmations" : 108335,
"hash" : "0x289c235dcdab8be7426d05f0fbb5e86c619f81481ea136493fa95deee5dbb7cc",
"netfee" : "237904",
"nonce" : 0,
"script" : "CxAMFIPvkoyXujYCRmgq9qEfMJQ4wNveDBSD75KMl7o2AkZoKvahHzCUOMDb3hTAHwwIdHJhbnNmZXIMFPVj6kC8KD1NDgXEjqMFs/Kgc0DvQWJ9W1I5",
"sender" : "NbcGB1tBEGM5MfhNbDAimvpJKzvVjLQ3jW",
"signers" : [
{
"account" : "0x649ca095e38a790d6c15ff78e0c6175099b428ac",
"scopes" : "None"
},
{
"account" : "0xdedbc03894301fa1f62a68460236ba978c92ef83",
"scopes" : "None"
}
],
"size" : 412,
"sysfee" : "997778",
"validuntilblock" : 5277629,
"version" : 0,
"vmstate" : "FAULT",
"witnesses" : [
{
"invocation" : "DECw8XNuyRg5vPeHxisQXlZ7VYNDxxK4xEm8zwpPyWJSSu+JaRKQxdrlPkXxXj34wc4ZSrZvKICGgPFE0ZHXhLPo",
"verification" : "DCEC+PI2tRSlp0wGwnjRuQdWdI0tBXNS7SlzSBBHFsaKUsdBVuezJw=="
},
{
"invocation" : "DEAxwi97t+rg9RsccOUzdJTJK7idbR7uUqQp0/0/ob9FbuW/tFius3/FOi82PDZtwdhk7s7KiNM/pU7vZLsgIbM0",
"verification" : "DCEDbInkzF5llzmgljE4HSMvtrNgPaz73XO5wgVJXLHNLXRBVuezJw=="
}
]
}
}
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
These warnings must be monitored by developers since it might be a sign
of behaviour difference between Go and C# nodes.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Fix#3130. "Exception" is used for name since it's shorter and that's the name
used in JSON. "VMFault" was also considered as well as "FaultException"
(which mirrors result.Invoke).
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Make the script a bit shorter. ABORTMSG would cost a bit more.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
1. Make prologue be exactly the same as regular CheckMultisig.
2. But instead of "SYSCALL System.Crypto.CheckMultisig" do INITSLOT and K check.
3. This makes all of the code from INITSLOT below be independent of N/M, so
one can parse the script beginning in the same way CheckMultisig is parsed and
then just compare the rest of it with some known-good blob.
4. The script becomes a tiny bit larger now, but properties above are too good.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Value calculated by calculatenetworkfee is enough to pass the real
tx verification. However, network fee may be decreased, so calculations
are not quite accurate. Need to investigate, why.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Replace native CryptoLib's verifyWithECDsa `curve` parameter by
`curveHash` parameter which is a enum over supported pairs of named
curves and hash functions.
Even though this change is a compatible extension of the protocol, it
changes the genesis state due to parameter renaming. But we're going to
resync chain in 3.7 release anyway, so it's not a big deal.
Also, we need to check mainnet and testnet compatibility in case if
anyone has ever called verifyWithECDsa with 24 or 25 `curve` value.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This reverts commit 874ed1ac2e.
We can't allow this gnark update since minimum required Go version for
NeoGo and all examples is 1.20. It will be done as a part of #3089.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Since the AllowedGroups []*keys.PublicKey slice is used in the
initialization, the test should use the same structures.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
In case if source of stringers or any other autogenerated code is
updated we need to regenerate the target files. It's easy to forget
about it.
Ref. 8995f11d39cc6b5f5b0081039bbe616dd7aaf38e.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Ensure that Blockchain constructor is able to distinguish empty
Hardforks map (no hardforks should be enabled) from nil hardforks map
(the default value should be used in this case, i.e. all hardforks
should be active from genesis).
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Ensure that ParameterContext's Item with nil script can be properly
marshalled. Ref. https://github.com/neo-project/neo/pull/3198.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Make them behave as locals. We must initialize them at the start
because the default value could also be used inside the function body.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
We don't have a reliable way to know when transports are started since
their start is being performed in a separate goroutine:
927dbb6dc4/pkg/network/server.go (L297-L299)
And transports start is not connected with main server routine, thus,
just wait for some time for the transports goroutine to be started.
Also wait for the peer to be properly registered.
Close#3399.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Do not wait until wsReader routine gracefully finishes its work before
WS connection close. Instead, firstly close the connection, and after
that wait for proper wsReader exit.
It's a harsh way, but I don't have any other options to try, because
wsReader routine hangs on `ws.ReadMessage()` operation for more than
ReadDeadline (more than 5 seconds) during test cleanup which results in
the test timeout.
Close#3378.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Close#2956. The failure reason is similar to the one described in #3396
for TestNotary: Blockchain's notificationDispatcher is listening to
block events from storeBlock via separate channel. By the moment single
block addition is finished, notification may or may not be properly
handled by notificationDispatcher, especially given the fact that our
runners are slow. As a result, assert.Eventually with 1-second awaiting
period may fail. This issue is solved by adding one more block, because
the second AddBlock finishes only when it sends block addition event to
notificationDispatcher loop, which means that the previous event was
handled.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
1. Allow 1-block slippage for multisignature request block generation to
ensure that PostPersist for fb's NVB-th block is properly finished.
2. Allow 1-block slippage before sent request checks to ensure that
PostPersist for NVB fb's NVB-th block was finished and all
stale mains were properly marked by Notary service.
Overall, this commits ports the approach of #3390 to the whole test.
Close#3366.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Made connections in parallel to check the limit and attempt to make one
more connection beyond the limit. Only "default" test case is failing,
because default number of connections is 64 (quite large for slow
runners). The failure reason is:
* given ~1 second for [dealing + request] per connection (time is taken
from the test log)
* by the moment 65-th connection is reached, some connections from the
start of the test have died due to inactivity (this process literally
takes 1 minute)
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
1. Replace isFinished atomic variable with a channel, no functional changes
here, just use more common way as all our services do.
2. Do not check erors from SetReadDeadline and ReadMessage on exit. It
seems to be not quite right because connection is not closed by this
moment, and thus, these error checks are racy.
3. Add read timeout for the message reader. It is needed because some
tests may leave message unread in the end which results in hanging
test cleanup.
4. Add drain loop to message reader in order not to block WS reader on
sending message.
Ref. https://github.com/nspcc-dev/neo-go/pull/3392#issuecomment-2031590403.
It's clear that TestBadSubUnsub is hanging on test cleanup, in
particular, on attempt to wait for wsRead routine to exit. The only
place where wsRead routine may hang is sending to msgCh in case if
receiver is not going to read from this channel:
```
2024-04-02T08:14:51.4957621Z goroutine 14329 [chan receive]:
2024-04-02T08:14:51.4958010Z github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.initCleanServerAndWSClient.func1()
2024-04-02T08:14:51.4958344Z D:/a/neo-go/neo-go/pkg/services/rpcsrv/subscription_test.go:80 +0x71
2024-04-02T08:14:51.4958457Z testing.(*common).Cleanup.func1()
2024-04-02T08:14:51.4958757Z C:/hostedtoolcache/windows/go/1.22.1/x64/src/testing/testing.go:1175 +0x17a
2024-04-02T08:14:51.4958903Z testing.(*common).runCleanup(0xc002cf5860, 0x0)
2024-04-02T08:14:51.4959193Z C:/hostedtoolcache/windows/go/1.22.1/x64/src/testing/testing.go:1353 +0x262
2024-04-02T08:14:51.4959291Z testing.tRunner.func2()
2024-04-02T08:14:51.4959566Z C:/hostedtoolcache/windows/go/1.22.1/x64/src/testing/testing.go:1683 +0x51
2024-04-02T08:14:51.4959695Z testing.tRunner(0xc002cf5860, 0x141687410)
2024-04-02T08:14:51.4959976Z C:/hostedtoolcache/windows/go/1.22.1/x64/src/testing/testing.go:1695 +0x25e
2024-04-02T08:14:51.4960115Z created by testing.(*T).Run in goroutine 1
2024-04-02T08:14:51.4960385Z C:/hostedtoolcache/windows/go/1.22.1/x64/src/testing/testing.go:1742 +0x826
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Previously findstorage returns null for the results key if a contract
state is not found. C# returns an empty list.
Close#3370
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Always return "Target transaction accepted" error if the target transation was
either accepted by the moment of command run or during the command
handling. Exit with code 1 in both cases. Adjust TestAwaitUtilCancelTx
correspondingly, allow both cases in test. Simplify the test along the
way, remove useless code.
Close#3365.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Close#3366. Roll back the test structure to the one that it has before
d2a7162217. Add one extra block to ensure
that main transaction won't be finilized. Add NVB increment slippage to
avoid race between the first fallback acceptance and block acceptance.
Add NVB increment slippage to the rest of fallbacks for test structure
unification.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Add error channel to prevent data race in the test.
Increase waiting interval for subscriptions awaiting up to 2 seconds.
Failing is caused by slow subscriptions.
Close#2958
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
To prevent `Expired transaction (-510) - transaction has expired:
ValidUntilBlock = 8, current height = 8` VUB have been increased.
Close#3343
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Occasionally the block is being accepted right after main transaction
submission. Added two branches into this TestAwaitUtilCancelTx. One
branch handles the case of original transaction acceptance, the
other branch handles the conflicting transaction acceptance.
Close#3365
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Default http.Client and http.Transport can be really slow. Also on
windows Timeout: time.Second was not enough. Probably network related
issue. As mentioned in https://github
.com/nspcc-dev/neo-go/issues/2975#issuecomment-1750523651 forcely use
only tcp4 and FallbackDelay: -1. This made TestRPC little bit faster so
gh windows runner can manage it without timeout or POST request fails.
Close#2975Close#3314
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
FaultException can be non-empty even in Halt state when there were
problems with stack marshaling to JSON.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
This is not the way intended in https://github.com/neo-project/proposals/pull/156.
-511 covers _both_ cases because users hardly can distinguish one from another,
it's just that our mempool implementation has error codes for both..
Signed-off-by: Roman Khimov <roman@nspcc.ru>
mempool.ErrInsufficientFunds is used when sender doesn't have enough
balance to pay the submitted transaction fees (-511 code according to
https://github.com/neo-project/proposals/pull/156). mempool.ErrConflict is
used when sender is not able to pay the overall transactions fee sum in
the pool (generic -500 error according to the proposal).
This bugfix is kind of breaking change for those users who relied on the
old -511 code previously returning "insufficient funds" error.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If we're withdrawing funds to contract that has onNEP17Payment method,
then it may call Notary's withdraw one more time, but the account's
state is not yet updated by this moment.
The problem is similar to https://github.com/neo-project/neo/pull/2734.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Shouldn't affect things much, yet at the same time we no longer need any
connection after we're done with the request.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Add waiting for startSending to ensure that the client is ready before
the server starts sending messages.
Close#3005Close#3312
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
We have more and more timeouts happening for test runs, so 10m is not enough
for busy GH machines sometime. I think we can just give them a bit more time.
Refs. #2379 though.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
The theory is:
* we add a block
* it triggers notary
* which can process the fallback before we're to setFinalizeWithError
* it'll fail and require an additional block to enter, but it's never added
* FAIL
The solution is:
* always add an additional block, but treat the first fb specially
Signed-off-by: Roman Khimov <roman@nspcc.ru>
And refactor some code a bit, don't use bytes.Clone where type-specific
helpers may be used instead.
Close#2907.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
It's a bug since Prefix is shared between all iterator items and
appending is not enough. If prefix has enough capacity, then new slice
won't be created and the previous item's prefix will be changed.
This commit fixes the following test failure caused by moving from
bytes.Clone to slice.Copy:
```
--- FAIL: TestComlileAndInvokeFunction/test_Storage.Find (0.02s)
--- FAIL: TestComlileAndInvokeFunction/test_Storage.Find/keys_only (0.01s)
contract_test.go:866:
Error Trace: /home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/contract_test.go:866
Error: Not equal:
expected: []stackitem.Item{(*stackitem.ByteArray)(0xc000a1cdf8), (*stackitem.ByteArray)(0xc000a1ce10)}
actual : []stackitem.Item{(*stackitem.ByteArray)(0xc000a1cdb0), (*stackitem.ByteArray)(0xc000a1cdc8)}
Diff:
--- Expected
+++ Actual
@@ -2,3 +2,3 @@
(*stackitem.ByteArray)((len=8) {
- 00000000 66 69 6e 64 6b 65 79 31 |findkey1|
+ 00000000 66 69 6e 64 6b 65 79 32 |findkey2|
}),
Test: TestComlileAndInvokeFunction/test_Storage.Find/keys_only
--- FAIL: TestComlileAndInvokeFunction/test_Storage.Find/both (0.01s)
contract_test.go:881:
Error Trace: /home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/contract_test.go:881
Error: Not equal:
expected: []stackitem.Item{(*stackitem.ByteArray)(0xc000515920), (*stackitem.ByteArray)(0xc000515938)}
actual : []stackitem.Item{(*stackitem.ByteArray)(0xc000515848), (*stackitem.ByteArray)(0xc000515860)}
Diff:
--- Expected
+++ Actual
@@ -2,3 +2,3 @@
(*stackitem.ByteArray)((len=8) {
- 00000000 66 69 6e 64 6b 65 79 31 |findkey1|
+ 00000000 66 69 6e 64 6b 65 79 32 |findkey2|
}),
Test: TestComlileAndInvokeFunction/test_Storage.Find/both
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Revert 5f6c01336c, remove all multierror
related nolint comments and use multierror wrapping instead.
Close#2906.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
`wallet1_solo.json` can't be used in this test because it contains
committee member that receives reward for accepted blocks. The race in
this test happens due to the fact that it uses running blockchain and
checks expected GAS balance via call to blockchain's
GetUtilityTokenBalance API. This call is racy with the CLI command
executed prior to the call which causes the following test failure:
```
2023-04-06T08:44:08.6038406Z === NAME TestNEP17Balance/all_accounts
2023-04-06T08:44:08.6038738Z executor.go:240:
2023-04-06T08:44:08.6039454Z Error Trace: /opt/github-runner/_work/neo-go/neo-go/cli/nep_test/executor.go:240
2023-04-06T08:44:08.6040628Z /opt/github-runner/_work/neo-go/neo-go/cli/nep_test/executor.go:236
2023-04-06T08:44:08.6041787Z /opt/github-runner/_work/neo-go/neo-go/cli/nep_test/nep17_test.go:73
2023-04-06T08:44:08.6042389Z Error: Expect " Amount : 1" to match "^\s*Amount\s*:\s*1.5$"
2023-04-06T08:44:08.6042865Z Test: TestNEP17Balance/all_accounts
```
The fix is do not use committee accounts for balance checks. Let's use
side accounts and transfer funds to some of them before the test start.
Close#2960.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
It contains several simple signature accounts that are not related to
network committee or validators. Needed for tests, existing
`testwallet.json` can't de reused since some tests need testing wallet
with a single account.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
We have smartcontract.ParameterPair structure that can be properly
marshalled and passed to RPC server as an element of smartcontract.Map
structure. However, RPC server can't unmarshal map values properly
without this change.
This change is compatible with C# node.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
s.Shutdown() does not wait for all goroutines of the node server to
finish normally just because the server exits without dependent
goroutines awaiting. Which causes logs to attempt to write after the
test has ended. The consequence of this bug fix is that corresponding
tests are fixed.
Close#2973Close#2974
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Added sync logs for every service separately to provide the ability to
have a custom logger for each service. This commit makes the code follow
the zap usages rules: `Sync calls the underlying Core's Sync method,
flushing any buffered log entries. Applications should take care to
call Sync before exiting.`
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Start of some services is bound to blockchain subscriptions, and thus,
can't be run before the blockchain notifications dispatcher.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Previously user should Start server in a separate goroutine. Now
separate goroutine is created inside the Start(). For normal server
operation, the caller should wait for Start to finish. Also, fixed
TestTryInitStateSync test which was exiting earlier than logs are
called.
Close#3112
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
This one doesn't really work now, it's Legacy. Replace with something N3-ish.
Thanks @lock9 for reporting.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Scenario:
1. Two messages were read from the connection `p.conn`
2. The first message has started to be processed
3. The second message was queued to be added to the channel `p.incoming`
4. Processing of the first message failed with an error
5. TCP peer is closed, but processing of the second message continues
Signed-off-by: Dmitrii Stepanov <dima-stepan@yandex.ru>
Add exit function at the end of the test to prevent
errors in t.Cleanup of the t.TempDir for levelDB instance.
Close#3154
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
I'm tired of warnings for our workflow runs.
The only unupdated is Codecov, we can't switch to token-based uploads at the
moment (and it's required for new version).
Signed-off-by: Roman Khimov <roman@nspcc.ru>
According to https://go.dev/doc/modules/gomod-ref go version should be
placed before `require` section.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Needs to updated dependencies inside the test for go1.18 and higher to
match with interop module.
Close#3152
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Fix VM test for windows. We don't have notary enabled in unit testchains
so it was removed.
Close#3309
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
We have a full list of public keys in the import-multisig command, so if
we have a key in the wallet that corresponds to one of these keys
(simple sig), just reuse it for the account automatically
Closes#3266
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Autogenerated RPC wrapper used underscores to differentiate between
methods/event overloads. Now it adds increasing suffices instead.
Close#3296
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
ProtocolConfiguration section must include stand by committee and the
number of validators or committee/validators history.
Close#3247
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Let the user know whether it's main or fallback transaction that failed
to be relayed to the network.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Try to subscribe for headers firstly, and then if RPC server doesn't
have this ability, fallback to block subscriptions to manage transaction
awaiting.
Close#3260.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Adjust names of all used structures, no need to duplicate `Waiter` everywhere,
we already in the `waiter` package. Also, adjust comments to Actor so that links
to Waiter are properly described in docs.
Ref. #3265.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
New --await flag is an option to synchronize on transaction execution
for CLI commands.
Closes#3244
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
It can be used to work with the results of
CreateCallAndPrefetchIteratorScript() execution. The first item must be
an array and the optional second item must be an iterator, containing
remaining elements.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
There is CreateCallAndUnwrapIteratorScript() which can traverse
iterator for nodes with sessions disabled. For other nodes this may
still be beneficial: if there is a small number of items, we might read
(or prefetch) all of them in one request. However, this script continues
to work even for large collections, returning both accumulated array and
remaining iterator.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
If main transaction is known, then conflicting transaction shouldn't be
valid longer than the main one.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
There are use-cases when not only Actor, but also Invoker and even
simple RPC client must wait (e.g. sendtx or dumptx CLI commands). Actor
requires optional signers in constructor, and it's not always
appropriate to create Actor only to be able to use Waiter, sometimes
it's needed to use only Waiter without Actor.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Change first argument of NewWaiter to be able to directly accept RPC
Client and export for external usage.
Refs #3244.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
BlockFilter has PrinaryIndex of int type while block.Block structure
itself has PrimaryIndex of byte. It's needed to prevent changing
filters field type and all associated subscriptions logic on
server side.
Refs #3241.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
The text must appear as the first line of a properly formatted Go //
comment, and that comment must appear before but not be attached to the
package clause and before any /* */ comment.
Closes#3254
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
New event is to notify the user about header's content by the moment
when block is stored (which happens after block's processing). This is
needed for proper Waiter work.
Closes#2751.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
dBFT doesn't use validators got from this call to GetValidators callback,
because NeoGo doesn't properly set WithGetConsensusAddress, and thus
this call can be safely skipped. Instead, NeoGo fills NextConsensus field
by itself in NewBlockFromContext callback.
This commit technically doesn't perform any functional changes and doesn't
affect the problem described in #3253 in any way. This commit is just a
removal of the code that was never used by NeoGo library.
This commit is a direct consequence of https://github.com/nspcc-dev/dbft/issues/84.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
We have cache update mechanism (Neo's cache votesChanged flag), it must
be used for current epoch and new epoch cached values update. And the
cached current/new epoch values themselves must always contain valid
information for the current/new epoch. These cached values must only be
changed once per epoch, never set them to nil.
This commit prevents CN node panic described in #3253 when dBFT tries
to retrieve new epoch validators with some votes modifications made
before at the same dBFT epoch.
Close#3253.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This was recently added in neotest, but working with the real RPC is
still not enjoyable. This commit extends `wallet.Account` with
invocation script builder to aid network fee calculations and signing.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
Add new filter NotaryRequestFilter, support for filtering
NotaryRequestEvents by mempoolevent.Type
(added or removed).
Closes#2425.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
StateRootInHeader depends on it, so we better immediately return error
if client is not initialised.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No functional changes.
We have 2 exactly the same cycles,and in the 1st cycle we set
cosignersOffset to the right value. So we don't need to check for arg ==
cmdargs.CosignersSeparator one more time.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Move actor handling into CLI package.
GetRPCWithActor returns RPC with actor instances.
Close#2682.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Move GetAccFromContext, GetUnlockedAccount, ReadWalletConfig handling
into options package to reuse this code from all CLI handlers.
getDecryptedAccount is replaced by GetUnlockedAccount.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
I had some reason to suspect something is wrong here (but it's all OK). Even
though this is kinda tested in TestDeployGetUpdateDestroyContract, it uses
internal APIs and with neotest we can ensure it work OK for a complete tx/block
environment. So, won't hurt having these tests as well.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
No functional changes, just moved a part of emit.Array to a separate
exported method. It may be useful for contract-based witness invocation
scripts construction and not only. Ref.
https://github.com/nspcc-dev/neo-go/pull/3233#discussion_r1407786333.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Turns out that Zcash swiched to BLS12-381 since https://github.com/zcash/zcash/issues/2502,
thanks to @EdgeDLT for pointing that out. I've checked that our
TestCubicCircuit_EndToEnd_Prod test passes with response file downloaded
from the attestations page of Zcash ceremony, thus I propose to put theirs
attestations link before the link to PPoT, because PPoT attestations contain
outdated links and not all responses can be downloaded.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Users of NeoGo interop package may have a demand to use these limits
for custom purposes, it would be nice to have them as constants.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
CLI side method for canceling not yet accepted transaction. It's
alternative to unsupported `canceltransaction` RPC method.
Close#3151.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
Perform initialisation of StateRoot service with designated
StateValidator's node list in the service constructor. There's no need
to wait until the next role update event, and it may lead to inaccuraces
in service work on SIGHUP/server restart.
A part of #3228.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Initialise services dependant on roles designation based on N+1
block so that Genesis roles extension work properly. There's not
much sence to fetch node roles information for the latest persisted
block because Designation contract itself makes designated nodes
responsible since the next subsequent block.
A part of #3228.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
To be able running the node from any working directory by simply
pointing the relative-path as prefix for relative parameters set in
config.
Closes#3179.
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
It affects both Debug data and bindings, otherwise two unnamed structures can
get any of unnamed/unnamedx names depending on particular invocation.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Manifest will be a part of the state.Contract which will be checked on its
way to the storage. Tiny optimisation which allows not to serialize manifest
twice. Ref. https://github.com/nspcc-dev/neo-go/pull/3218#discussion_r1402374232.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
His account is disabled (which technically leads to an invalid CODEOWNERS
file) and he's no longer active, unfortunately.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Unfortunately, when import cycle happens somewhere deep in the import chain we
dont't get an error from packages.Load(). But it leaves some imports
uninitialized, so at least we can check for them.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Otherwise it's undertermined which of two unnamed structures will get "Unnamed"
and "UnnamedX" which can break the test from time to time.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Type data is added while walking through function ins/outs even if we're to
throw this function away. But we don't need it, these types are not used, so
we can deal with the main part of #3071 by optimizing this out.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
TestAssistedRPCBindings didn't respect rewriteExpectedOutputs setting, with
this behavior compiler could produce any result and the test wouldn't notice.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
This method returns persisted block height and doesn't take into account
persisting block height. Some of the callers of this method relay on
the wrong assumption that BlockHeight() returns persisting block index.
Fix improper usages of this method and adjust tests. Ref.
61a066583e/src/Neo/SmartContract/ApplicationEngine.cs (L634).
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
We already have tests for Permission deserialisation, so port the first
part of https://github.com/neo-project/neo/pull/2948.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
We have this log on the network server side, but it would also be
useful in case of failed blockchain initialization.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Refactored native NeoToken cache scheme introduced in #3110 sometimes requires
validators list recalculation during native cache initialization process (when
initializing with the existing storage from the block that is preceded each N-th block).
To recalculate validators from candidates, native NeoToken needs an access to
cached native Policy blocked accounts. By the moment of native Neo initialization,
the cache of native Policy is not yet initialized, thus we need a direct DAO access
for Policy to handle blocked account check.
Close#3181.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Disable the following linter warning:
```
superfluous-else: if block ends with call to panic function, so drop this else and outdent its block (revive)
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Fix the following linter warning:
```
indent-error-flow: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary) (revive)
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This partially reverts commit c26a962b55 for testing
chains configurations.
Ref. #2975, although this commit doesn't close it. This commit is an attempt to
enforce IPv4 for our test clients to avoid problem described in the issue.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This section contains genesis-related settings including genesis-related or natives-related
extensions. Currently it includes the set of node roles that may be designated
duing the native Designation contract initialisation.
Close#3156.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
And rename roles.go to role.go to match the role_string.go and the
existing naming pattern for enums.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
* Rename c.done channel to c.readerDone
* Introduce c.writerDone channel that is closed if the wsWriter's loop
is done
* Make the error returned by makeWsRequests verbose
Signed-off-by: Ayrtat <amper.meter1775@gmail.com>
We often use binary.PutUint*, but almost all these cases have preallocated
buffer of the size that matches exactly the desired one and use a single or
a couple of calls to PutUint*. Thus, I don't think that replacing
binary.PutUint* by AppendUint* will make things better for all these usages.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Don't need to keep the copy of the official Golang image in our
dockerfile, let's use it as a base builder image.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Currently it's failing, but no token is required for public repos:
```
[2023-10-10T10:35:11.917Z] ['error'] There was an error running the uploader: Error uploading to https://codecov.io: Error: There was an error fetching the storage URL during POST: 404 - {'detail': ErrorDetail(string='Unable to locate build via Github Actions API. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If it's the end of epoch, then it contains the updated validators list recalculated
during the last block's PostPersist. If it's middle of the epoch, then it contains
previously calculated value (value for the previous completed epoch) that is equal
to the current nextValidators cache value.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Do not recalculate new committee/validators value in the start of every
subsequent epoch. Use values that was calculated in the PostPersist method
of the previously processed block in the end of the previous epoch.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No funcional changes, just refactoring. It doesn't need the whole cache,
only the set of committee keys with votes.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Recalculate them once per epoch. Consensus is aware of it and must
call CalculateNextValidators exactly when needed.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
We have two similar blockchain APIs: GetNextBlockValidators and GetValidators.
It's hard to distinguish them, thus renaming it to match the meaning, so what
we have now is:
GetNextBlockValidators literally just returns the top of the committee that
was elected in the start of batch of CommitteeSize blocks batch. It doesn't
change its valie every block.
ComputeNextBlockValidators literally computes the list of validators based on
the most fresh committee members information got from the NeoToken's storage
and based on the latest register/unregister/vote events. The list returned by
this method may be updated every block.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Blockchain passes his own pure unwrapped DAO to
(*Blockchain).ComputeNextBlockValidators which means that native
RW NEO cache structure stored inside this DAO can be modified by
anyone who uses exported ComputeNextBlockValidators Blockchain API,
and technically it's valid, and we should allow this, because it's
the only purpose of `validators` caching. However, at the same time
some RPC server is allowed to request a subsequent wrapped DAO for
some test invocation. It means that descendant wrapped DAO
eventually will request RW NEO cache and try to `Copy()`
the underlying's DAO cache which is in direct use of
ComputeNextBlockValidators. Here's the race:
ComputeNextBlockValidators called by Consensus service tries to
update cached `validators` value, and descendant wrapped DAO
created by the RPC server tries to copy DAO's native cache and
read the cached `validators` value.
So the problem is that native cache not designated to handle
concurrent access between parent DAO layer and derived (wrapped)
DAO layer. I've carefully reviewed all the usages of native cache,
and turns out that the described situation is the only place where
parent DAO is used directly to modify its cache concurrently with
some descendant DAO that is trying to access the cache. All other
usages of native cache (not only NEO, but also all other native
contrcts) strictly rely on the hierarchical DAO structure and don't
try to perform these concurrent operations between DAO layers.
There's also persist operation, but it keeps cache RW lock taken,
so it doesn't have this problem as far. Thus, in this commit we rework
NEO's `validators` cache value so that it always contain the relevant
list for upper Blockchain's DAO and is updated every PostPersist (if
needed).
Note: we must be very careful extending our native cache in the
future, every usage of native cache must be checked against the
described problem.
Close#2989.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Network server constructor reads config.Version variable, and
testcli.DeployContract writes dummy config.Version which causes
race in tests. Avoid this race by moving config.Version initialisation
to a separate package and perform it inside test packages init().
Close#3011, close#3017.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
To dump the DB, the service must be stopped.
If this is not the case `dump` command just hangs without any output,
which _may_ be unexpected from the ops POV.
Introduce a 1 second timeous, which is more than enough, given
that bbolt retries doing flock() every 50ms.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
Currently we take int64 value from the Go parser and push it to the
stack. Using uint64 is not a common practice (usually we just use `int`),
but can be a problem while doing bit arithmetic and serializing numbers.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
And ignore on dependency checks, we update all the examples at once
anyway, so may safely skip the check for zkp folder.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
An upgrade from gnark v0.8.X to v0.9.0 changes serialization format of verifying/proving keys
and proofs. In neo-go zkpbinding package we have to support both at least for now, because
gnark@v0.9.0 requires minimum go 1.19.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
The example shows that the proover knows the solution of the cubic
equation: y = x^3 + x + 5. The example is constructed for BLS12-381
curve points using Groth-16 prooving algorithm. The example includes
everything that developer needs to start using ZKP on the NEO platform
with Go SDK:
1. The described cubic circuit implementation.
2. The off-chain proof generation with the help of gnark-crypto library.
3. Go verification contract generation and deployment with the help of
NeoGo libraries.
4. The on-chain proof verification for various sets of input data.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Valid transactions can't use more than MaxVerificationGAS for script execution
and this applies to the whole set of signers, so use this value by default
unless local instance configuration suggests something lower for generic
invocations.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
We can ignore core.ErrInvalidSignature (which means that the script has
executed, but returned false), but we shouldn't ignore other errors which
likely mean that the script is incorrect (or hits some resource limits).
Use neorpc.ErrInvalidSignature as a return to separate this case from
contract-based verification.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
It requires explicit signers, and we have a problem with the old code:
2023-09-12T18:42:00.063Z WARN failed to perform oracle request {"url": "neofs:FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH/5DCg4wUgWuWN3zsF4P4HdAzY2iKvXcrZ8QBLYGd1D1g2", "error": "failed to create client: incorrect signer: expected ECDSA_DETERMINISTIC_SHA256 scheme"}
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Testnet doesn't need any (confirmed to have the same state for non-NEO
contracts) and can go with Basilisk from height 0.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
It can be non-zero even if VoteTo is NULL. Fixes state diff with 3.6.0:
block 41660: value mismatch for key +////xTrvgat3qG/w8hQoD/I4MgUz6rygA==: QQQhAS8hA7yiAAAhAA== vs QQQhAS8hA7yiAAAhB+POSWfBCAE=
Related to #2844.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
If the parameter in the opcode dump is a 20-byte value, the converted values,
such as the address and the swapped endianness, have been added.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
GetRawNotaryTransaction returns a fallback or main transaction that was
previously added to the memory pool by P2PNotaryRequest. This function
invokes the RPC server's `getrawnotarytransaction` method.
GetRawNotaryPool returns hashes from all the verified transactions,
including both main and fallback transactions. This function invokes
the RPC server's `getrawnotarypool` method.
Also, these functions were added to doc.go.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
`getrawnotarytransaction` takes a transaction hash and attempts to find
the corresponding transaction in the notary requests mempool. It searches
through all the verified main and fallback transactions.
`getrawnotarypool` returns hashes of all the verified transactions,
including both main and fallback transactions.
Additionally add struct result.RawNotaryPool.
Close https://github.com/nspcc-dev/neo-go/issues/2951
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
IterateVerifiedTransactions iterates through verified transactions in
memory pool and invokes function cont. Where cont callback returns
whether we should continue with the traversal process.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
Do not retrieve the whole set of storage items when trying to find
the ones from the specified start. Use DAO's Seek interface
implemented over MPT TrieStore to retrieve only the necessary items.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Either non-pointer or pointer, both cases are disallowed to be generic.
Need to be reverted and properly handled within the scope of #2376.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
So that hardfork name was explicitly present in the test name. We'll
have a set of similar tests later.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No functional changes, just a refactoring.
Change error text to be able to use this error from external packages.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
Behaviour change.
`terminatesession` returns ErrUnknownSession in case of impossibility of finding session,
previously there was no-error response with `false` result.
`traverseIterator`returns ErrUnknownSession in case of impossibility of finding session,
previously there was no-error response with default result; `traverseIterator`returns ErrUnknownIterator,
there were no such errors before.
Accordingly to proposal:
https://github.com/neo-project/proposals/pull/156
Also adding description of `traverseIterator` in docs/rpc.md.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
This change makes code incompatible with C# node,
because currently no error is returned on invalid proof.
According to proposal:
https://github.com/neo-project/proposals/pull/156
Also adding `verifyProof` descpiption in docs/rpc.md.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
No functional changes, just a refactoring.
Use more specific and meaningful names to be able to use these errors from external packages.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
While our server no longer uses these codes (-100, -400) they still can come
from C# servers and while we consider them deprecated we better at least have
some definition of them until C# implements our proposal:
https://github.com/neo-project/proposals/pull/156
Also adding description of deprecated RPC error codes in ROADMAP.md.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
RPC binding config may be malformed or the source .go contract may contain
structures like this:
```
type Str struct {
Field int
field int
}
```
We need to recognise these cases and return error. otherwise the resulting
binding can't be compiled.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
It helps us to keep the index clean and ordered. When some new RPC binding
test should be added, we don't need to search for a suitable place for it.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Perform private -> public transformation at the last step of RPC binding
generation so that it works not only with NeoGo contracts, but with any
other contracts.
Close#3083.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
And simplify atomic service value stored by RPC server. Oracle service can
either be an untyped nil or be the proper non-nil *oracle.Oracle.
Otherwise `submitoracleresponse` RPC handler doesn't work properly.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
52-bit precision is not enough for our 256-bit VM, but this value
matches the reference implementation, see the
https://github.com/neo-project/neo/issues/2879.
MaxIntegerPrec will be increased (or even removed) as soon as the
ref. issue is resolved.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Ensure that Scopes can be properly parsed not only from the string
representation, but also from a single byte. transaction.Signer
is not affected (checked against the C# implementation), only
RPC-related signer scopes are allowed to be unmarshalled from byte.
Close#3059.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This prevents the possible attack on notary request sender when
malicious partie is allowed to send notary request with main transaction
being someone else's fallback.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If the contract was deployed then cache must be initialized after
in-memory data reset. If the contract isn't active yet, then no
cache will be initialized on deploy (i.e. on call to Initialize()
method by native Management).
Close#2984.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Make the contracts cache initialization unified. The order of cache
iniitialization is not important and Nottary contract is added to the
bc.contracts.Contracts wrt P2PSigExtensions setting, thus no functional
changes, just refactoring for future applications.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Initialize Prometheus metrics on node start where appropriate and review
the usage of the following metrics:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ find | grep prometheus.go
./pkg/network/prometheus.go
./pkg/core/stateroot/prometheus.go
./pkg/core/prometheus.go
./pkg/services/rpcsrv/prometheus.go
./pkg/services/metrics/prometheus.go
```
Close#2970.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
The reader is about to exit and it will close legacy c.Notifications, but it
will leave subscription channels at the same time. This is wrong since these
channels will no longer receive any new events, game over.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
wsReader() closes c.done first and then goes over the list of
c.respChannels. Technically this means that any of the two can be taken in
this select.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
If `StartWhenSynchronized` is unset in config, `node` command runs RPC
service instantly. Previously there was a ground for deadlock. Command
started RPC server synchronously. According to server implementation, it
sends all internal failures to the parameterized error channel. Deadlock
occured because main routine didn't scan the channel.
Run `rpcsrv.Server.Start` in a separate go-routine in `startServer`.
This prevents potential deadlock caused by writing into unread channel.
Fixes#2896.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Previously RPC server shutdown procedure listened to the execution
channel and stopped at the first element that arrived in the queue. This
could lead to the following problems:
* stopper could steal the execution result from subscriber
* stopper didn't wait for other subscription actions to complete
Add dedicated channel to `Server` for subscription routine. Close the
channel on `handleSubEvents` return and wait for signal in `Shutdown`.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Previously RPC server could never be shut down completely due to
some start precondition failure (in particular, inability to serve HTTP
on any configured endpoint). The problem was caused by next facts:
* start method ran subscription routine after HTTP init succeeded only
* stop method blocked waiting for the subscription routine to return
Run `handleSubEvents` routine on fresh `Start` unconditionally. With
this change, `Shutdown` method won't produce deadlock since
`handleSubEvents` closes wait channel.
Refs #2896.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
There is an existing problem with RPC server shutdown freeze after start
failure due to some init actions (at least HTTP listen) described in
#2896.
Add dedicated unit test which checks that `Shutdown` returns within 5s
after `Start` method encounters internal problems.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
I've carefully checked the way how new service can be added to the
Blockchain instance or to be removed from it. Current implemention
of SetNotary and SetOracle methods doesn't contain dangerous code,
and native contracts have atomic values everywhere where service
is stored.
Current implementation of Notary, Oracle and StateRoot services'
reload/disabling/enabling on SIGUSR1 is safe and doesn't require
any adjustment.
This commit closes#2944, it's not a bug in the code, it's just
stale documentation.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
52-bit precision is not enough for our 256-bit VM, but this value
matches the reference implementation, see the
https://github.com/neo-project/neo/issues/2879.
MaxIntegerPrec will be increased (or even removed) as soon as the
ref. issue is resolved.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
During new transaction verification if there's an on-chain conflicting
transaction, we should check the signers of this conflicting transaction.
If the signers intersect with signers of the incoming transaction, then
the conflict is treated as valid and verification for new incoming
transaction should fail. Otherwise, the conflict is treated as the
malicious attack attempt and will not be taken into account;
verification for the new incoming transaction should continue.
This commint implements the scheme described at
https://github.com/neo-project/neo/pull/2818#issuecomment-1632972055,
thanks to @shargon for digging.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Witnesses are not yet created by the moment we return this error,
thus, it was always 0 as an actual number of witnesses in
ErrInvalidWitnessNum.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
`(*Blockchain).HasTransaction` is one of the oldest methods in our
codebase, and currently it's completely unused. I also doubt that
this method works as expected because it returns `true` if transaction
in the mempool.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This prevents the possible attack on notary request sender when
malicious partie is allowed to send notary request with main transaction
being someone else's fallback.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Ensure that Scopes can be properly parsed not only from the string
representation, but also from a single byte. transaction.Signer
is not affected (checked against the C# implementation), only
RPC-related signer scopes are allowed to be unmarshalled from byte.
Close#3059.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This check is good and was present here since #1729, but it was
accidently removed from the reference implementation (see the
discussion in https://github.com/neo-project/neo/issues/2848). The
removal of this check from the C# node leaded to the T5 testnet state
diff since 1670095 heigh which causes inability to process new blocks
since 2272533 height (see #3049). This check was added back to the
C# node in https://github.com/neo-project/neo/pull/2849, but it is
planned to be the part of the upcoming 3.6.0 C# node release.
We need to keep our testnet healthy, thus, strict contract script
check will be temporary removed from the node code and is planned
to be added back to be a part of the next 3.6.0-compatible release.
Close#3049.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This check is good and was present here since #1729, but it was
accidently removed from the reference implementation (see the
discussion in https://github.com/neo-project/neo/issues/2848). The
removal of this check from the C# node leaded to the T5 testnet state
diff since 1670095 heigh which causes inability to process new blocks
since 2272533 height (see #3049). This check was added back to the
C# node in https://github.com/neo-project/neo/pull/2849, but it is
planned to be the part of the upcoming 3.6.0 C# node release.
We need to keep our testnet healthy, thus, strict contract script
check will be temporary removed from the node code and is planned
to be added back to be a part of the next 3.6.0-compatible release.
Close#3049.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
All Notary contract witnesses in incomplete transaction (both main
and fallback) may either have invocation scripts pushing dummy signature
on stack or be empty, both ways are OK. Notary actor keeps main tx's
Notary witness empty and keeps fallback tx's Notary witness filled
with dummy signature.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Our wrapping optimization relied on the caller context having a TRY block,
but each context (including internal calls!) has an exception handling stack
of its own, which means that for an invocation stack of
entry
A.someMethodFromEntry() # this one has a TRY
A.internalMethodViaCALL() # this one doesn't
B.someMethod()
we get `HasTryBlock() == false` for `A.internalMethodViaCALL()` context, which
leads to missing wrapper and missing rollbacks if B is to THROW. What this
patch does instead is it checks for any context within contract boundaries.
Fixes#3045.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Our wrapping optimization relied on the caller context having a TRY block,
but each context (including internal calls!) has an exception handling stack
of its own, which means that for an invocation stack of
entry
A.someMethodFromEntry() # this one has a TRY
A.internalMethodViaCALL() # this one doesn't
B.someMethod()
we get `HasTryBlock() == false` for `A.internalMethodViaCALL()` context, which
leads to missing wrapper and missing rollbacks if B is to THROW. What this
patch does instead is it checks for any context within contract boundaries.
Fixes#3045.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
That's the way how C# node handles equality checks for stackitem.Interop types
for these points. Ref. https://github.com/nspcc-dev/neo-go/issues/3002#issuecomment-1591220501.
Along the way, add GT case for CryptoLib's bls12381Equal method. It should be there since #2940.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
In this case emitted event parameters should match from invocation to
invocation. It's an error otherwise (and if the type is not Any).
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Currently we have problems with unnamed structs used as a parameter for
notifications, because there's no one-to-one correspondence between
notification parameter and the list of extended types used by the whole
contract. Otherwise eventtypes guessing works OK.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
config_extended.yml contains an example of user-defined configuration
file with extended event types. User-defined event types are allowed
to be named and complicated, i.e. properly support extended types
notation.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Unfortunately, without pre-set user extended types configuration for events
and without --guess-eventtypes flag set we are allowed to rely only on manifest
information about types. Manifest can't give us a lot of information, but we
still need to be able to generate RPC binding. Arrays and structs are correctly
handled by the current code, but maps always rely on the fact that map's value
type is set. It's not true in the described case, so make the maps type convertor
handle this situation in a similar way how arrays are handled.
Without this commit the following panic occurs on attempt to generate RPC binding:
```
--- FAIL: TestAssistedRPCBindings/testdata/notifications (0.01s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7f7c0e]
goroutine 190 [running]:
testing.tRunner.func1.2({0x109cb40, 0x1d58760})
/usr/local/go/src/testing/testing.go:1396 +0x24e
testing.tRunner.func1()
/usr/local/go/src/testing/testing.go:1399 +0x39f
panic({0x109cb40, 0x1d58760})
/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.extendedTypeToGo({0x22, {0x0, 0x0}, {0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:515 +0x36e
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.scTypeToGo({0xc000206d92?, 0xc000206d80?}, 0x22, 0xc0005d70e0)
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:643 +0x138
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.scTemplateToRPC({{0xc00049bb07, 0x7}, 0xc0004c89c0, {0x33, 0x22, 0x11, 0x0, 0xff, 0xee, 0xdd, ...}, ...}, ...)
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:686 +0xbc4
github.com/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding.Generate({{0xc00049bb07, 0x7}, 0xc0004c89c0, {0x33, 0x22, 0x11, 0x0, 0xff, 0xee, 0xdd, ...}, ...})
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/pkg/smartcontract/rpcbinding/binding.go:421 +0x387
github.com/nspcc-dev/neo-go/cli/smartcontract.contractGenerateSomething(0xc00043e2c0, 0x137cd00)
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/generate.go:99 +0x855
github.com/nspcc-dev/neo-go/cli/smartcontract.contractGenerateRPCWrapper(0xc00043e2c0?)
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/generate.go:60 +0x25
github.com/urfave/cli.HandleAction({0x1048380?, 0x137c660?}, 0x13?)
/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/app.go:524 +0x50
github.com/urfave/cli.Command.Run({{0x123539d, 0x13}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x12577ad, 0x2a}, {0x127ad35, ...}, ...}, ...)
/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/command.go:173 +0x65b
github.com/urfave/cli.(*App).RunAsSubcommand(0xc0001f4000, 0xc00043e000)
/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/app.go:405 +0x91b
github.com/urfave/cli.Command.startApp({{0x12281e1, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x1254d8a, 0x28}, {0x0, ...}, ...}, ...)
/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/command.go:372 +0x6e7
github.com/urfave/cli.Command.Run({{0x12281e1, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x1254d8a, 0x28}, {0x0, ...}, ...}, ...)
/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/command.go:102 +0x825
github.com/urfave/cli.(*App).Run(0xc00024e000, {0xc0004f6420, 0xb, 0xb})
/home/anna/go/pkg/mod/github.com/urfave/cli@v1.22.5/app.go:277 +0x8a7
github.com/nspcc-dev/neo-go/cli/smartcontract.TestAssistedRPCBindings.func1.1(0x9f8829?)
/home/anna/Documents/GitProjects/nspcc-dev/neo-go/cli/smartcontract/generate_test.go:395 +0x5fc
testing.tRunner(0xc0006824e0, 0xc0004a3680)
/usr/local/go/src/testing/testing.go:1446 +0x10b
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1493 +0x35f
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
There are two ways of doing this: first one is to emit all notifications
parameter data into rpcbindings configuration on compile time (event if
the parameter has a simple type), and the second one is to fetch parameter
type from the manifest on rpcbinding file generation if needed (we always
have manifest at this stage, thus it's not a problem to retrieve necessary
information). The latter case is chosen to reduce the bindings configuration
file size.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Notification and its parameters may have any UTF8-compatible name
which is inappropriate for bindings configuration and for the resulting
RPC bindings file. This commit stores the prettified version of
notification's name and parameters that are ready to be used in the
resulting RPC binding without any changes.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
The user should specify it via parameter's `extendedtype` field and
via upper-level `namedtypes` field of the contract configuration YAML.
Also, as we have proper event structure source, make the `--guess-eventtype`
compilation option and make event types guess optional.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
New rule for writing blocks of code to our template: new line before
the block starts and new line after the block ends. This rule is the
same as the one we use during manual typing.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
cli's testdata folder stores contracts examples and expected compilation
output, thus doesn't need to be ignored.
*.out files are the bindings outputs only, do not ignore them as far.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Properly load the provided method using NEF and hash specified. It allows
to have NEF properly set in the VM context and handle CALLT instruction
correctly.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
`update` and `destroy` methods of the contract should check for owner
witness without checking the trigger (and that's the way how Verify
method works for this contract).
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
It's enough to specify the input file only to get the standard output:
```
$ neo-go contract compile -i ./1-print/1-print.go
$ neo-go contract compile -i ./1-print/
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Regular Client doesn't care much about connections, because HTTP client's Do
method can reuse old ones or create additional ones on the fly. So one request
can fail and the next one easily succeed. WSClient is different, it works via
a single connection and if it breaks, it breaks forever for this
client. Callers will get some error on every request afterwards and it'd be
nice for this error to be the same so that API users could detect
disconnection this way too.
Related to nspcc-dev/neofs-node#2325.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
It should be this way because we can't provide some features golangci-lint
require us to do due to the need to support not only the latest Go version,
but also two versions below.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
go.uber.org/atomic deprecated CAS methods in version 1.10 (that introduced
CompareAndSwap), so we need to fix it.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
We were using _very_ old version, so there is a number of changes, including:
* fix for unlimited reads
* UTF-8 check for HTTP requests
Signed-off-by: Roman Khimov <roman@nspcc.ru>
If the contract was deployed then cache must be initialized after
in-memory data reset. If the contract isn't active yet, then no
cache will be initialized on deploy (i.e. on call to Initialize()
method by native Management).
Close#2984.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Make the contracts cache initialization unified. The order of cache
iniitialization is not important and Nottary contract is added to the
bc.contracts.Contracts wrt P2PSigExtensions setting, thus no functional
changes, just refactoring for future applications.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Initialize Prometheus metrics on node start where appropriate and review
the usage of the following metrics:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ find | grep prometheus.go
./pkg/network/prometheus.go
./pkg/core/stateroot/prometheus.go
./pkg/core/prometheus.go
./pkg/services/rpcsrv/prometheus.go
./pkg/services/metrics/prometheus.go
```
Close#2970.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Close#2894.
It should be noted that the subscriber's channel is being removed from the
list of receivers and closed, but it is still *in the list of subscribers*
and no unsubscription is performed by WSClient. Which means that RPC server
keeps sending notifications to WSClient and WSClient keeps dropping them
(because there's no receiver for this subscription and it's OK, WSClient
can handle this and this behaviour is documented). However, it's still the
caller's duty to call Unsubscribe() method for this subscription.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
All Notary contract witnesses in incomplete transaction (both main
and fallback) may either have invocation scripts pushing dummy signature
on stack or be empty, both ways are OK. Notary actor keeps main tx's
Notary witness empty and keeps fallback tx's Notary witness filled
with dummy signature.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
The reader is about to exit and it will close legacy c.Notifications, but it
will leave subscription channels at the same time. This is wrong since these
channels will no longer receive any new events, game over.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
wsReader() closes c.done first and then goes over the list of
c.respChannels. Technically this means that any of the two can be taken in
this select.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
If `StartWhenSynchronized` is unset in config, `node` command runs RPC
service instantly. Previously there was a ground for deadlock. Command
started RPC server synchronously. According to server implementation, it
sends all internal failures to the parameterized error channel. Deadlock
occured because main routine didn't scan the channel.
Run `rpcsrv.Server.Start` in a separate go-routine in `startServer`.
This prevents potential deadlock caused by writing into unread channel.
Fixes#2896.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Previously RPC server shutdown procedure listened to the execution
channel and stopped at the first element that arrived in the queue. This
could lead to the following problems:
* stopper could steal the execution result from subscriber
* stopper didn't wait for other subscription actions to complete
Add dedicated channel to `Server` for subscription routine. Close the
channel on `handleSubEvents` return and wait for signal in `Shutdown`.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Previously RPC server could never be shut down completely due to
some start precondition failure (in particular, inability to serve HTTP
on any configured endpoint). The problem was caused by next facts:
* start method ran subscription routine after HTTP init succeeded only
* stop method blocked waiting for the subscription routine to return
Run `handleSubEvents` routine on fresh `Start` unconditionally. With
this change, `Shutdown` method won't produce deadlock since
`handleSubEvents` closes wait channel.
Refs #2896.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
There is an existing problem with RPC server shutdown freeze after start
failure due to some init actions (at least HTTP listen) described in
#2896.
Add dedicated unit test which checks that `Shutdown` returns within 5s
after `Start` method encounters internal problems.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Move them to the core/network packages, close#2950. The name of
mempool's unsorted transactions metrics has been changed along the
way to match the core's metrics naming convention.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
I've carefully checked the way how new service can be added to the
Blockchain instance or to be removed from it. Current implemention
of SetNotary and SetOracle methods doesn't contain dangerous code,
and native contracts have atomic values everywhere where service
is stored.
Current implementation of Notary, Oracle and StateRoot services'
reload/disabling/enabling on SIGUSR1 is safe and doesn't require
any adjustment.
This commit closes#2944, it's not a bug in the code, it's just
stale documentation.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
`cache` persisting operation is concurrent with the storage modifications
made by further state reset stages. We can't allow situation when data
from the next stage are leaking into the previous stage. State reset stages
must be atomic in turms of DB persisting, thus, use another `upperCache`
MemCached store to keep them apart.
Here are the results of mainnet's BoltDB reset from 1.1M to 6K:
This patch:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000
2023-04-11T16:15:25.783+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144}
2023-04-11T16:15:25.783+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000}
2023-04-11T16:15:25.783+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512}
2023-04-11T16:15:25.783+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2023-04-11T16:15:25.787+0300 INFO restoring blockchain {"version": "0.2.8"}
2023-04-11T16:15:25.906+0300 INFO initializing state reset {"target height": 600000}
2023-04-11T16:15:25.906+0300 DEBUG trying to reset blocks, transactions and AERs
2023-04-11T16:15:57.031+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 1, "took": "31.125057214s"}
2023-04-11T16:16:12.644+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 1, "took": "15.613156971s", "keys": 321895}
2023-04-11T16:16:13.895+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 2, "took": "16.663444208s"}
2023-04-11T16:16:19.784+0300 INFO last batch of removed blocks, transactions and AERs is collected {"batch": 3, "took": "5.760308543s"}
2023-04-11T16:16:19.784+0300 INFO blocks, transactions ans AERs are reset {"took": "53.878632911s"}
2023-04-11T16:16:22.870+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 2, "took": "8.974838893s", "keys": 334823}
2023-04-11T16:16:22.870+0300 DEBUG trying to reset contract storage items
2023-04-11T16:16:27.272+0300 DEBUG last batch of removed blocks, transactions and AERs is persisted {"batch": 3, "took": "7.487357441s", "keys": 208913}
2023-04-11T16:17:23.678+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 1, "took": "1m0.80711106s"}
2023-04-11T16:18:00.769+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 2, "took": "36.967660061s"}
2023-04-11T16:18:20.478+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 1, "took": "56.796257788s", "keys": 200000}
2023-04-11T16:18:54.115+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 3, "took": "33.637412437s"}
2023-04-11T16:19:18.844+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 2, "took": "1m18.0737668s", "keys": 200000}
2023-04-11T16:19:27.650+0300 INFO last batch of contract storage items is collected {"batch": 4, "took": "8.806264019s"}
2023-04-11T16:19:27.650+0300 INFO contract storage items are reset {"took": "3m4.780232077s", "keys": 656944}
2023-04-11T16:20:15.660+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 3, "took": "1m21.544386403s", "keys": 200000}
2023-04-11T16:20:15.660+0300 DEBUG trying to reset headers information
2023-04-11T16:20:16.385+0300 INFO headers information is reset {"took": "725.174932ms"}
2023-04-11T16:20:19.586+0300 DEBUG last batch of contract storage items and IDs is persisted {"batch": 4, "took": "51.936278608s", "keys": 56945}
2023-04-11T16:20:19.587+0300 DEBUG trying to reset state root information and NEP transfers
2023-04-11T16:20:35.845+0300 INFO state root information and NEP transfers are reset {"took": "16.25852114s"}
2023-04-11T16:21:10.000+0300 DEBUG headers information is persisted {"took": "53.613638429s", "keys": 528438}
2023-04-11T16:21:10.003+0300 DEBUG trying to remove stale storage items
2023-04-11T16:21:18.108+0300 INFO stale storage items are reset {"took": "8.105140658s", "keys": 1350176}
2023-04-11T16:21:18.108+0300 DEBUG trying to remove state reset point
2023-04-11T16:21:18.108+0300 INFO state reset point is removed {"took": "8.554µs"}
2023-04-11T16:21:20.151+0300 DEBUG state root information and NEP transfers are persisted {"took": "44.305707049s", "keys": 602578}
2023-04-11T16:21:20.212+0300 INFO state reset point information is persisted {"took": "2.103764633s", "keys": 2}
2023-04-11T16:21:20.213+0300 INFO reset finished successfully {"took": "5m54.306861367s"}
```
The previous commit:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000
2023-04-11T16:24:04.256+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144}
2023-04-11T16:24:04.256+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000}
2023-04-11T16:24:04.256+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512}
2023-04-11T16:24:04.256+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2023-04-11T16:24:04.261+0300 INFO restoring blockchain {"version": "0.2.8"}
2023-04-11T16:24:04.368+0300 INFO initializing state reset {"target height": 600000}
2023-04-11T16:24:04.368+0300 DEBUG trying to reset blocks, transactions and AERs
2023-04-11T16:24:30.363+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 1, "took": "25.995261037s"}
2023-04-11T16:24:44.947+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 1, "took": "14.584447338s", "keys": 321897}
2023-04-11T16:24:45.791+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 2, "took": "15.428492824s"}
2023-04-11T16:24:51.252+0300 INFO last batch of removed blocks, transactions and AERs is collected {"batch": 3, "took": "5.460662766s"}
2023-04-11T16:24:51.252+0300 INFO blocks, transactions ans AERs are reset {"took": "46.884558096s"}
2023-04-11T16:24:55.399+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 2, "took": "9.607820004s", "keys": 334821}
2023-04-11T16:24:55.399+0300 DEBUG trying to reset contract storage items
2023-04-11T16:24:59.981+0300 DEBUG last batch of removed blocks, transactions and AERs is persisted {"batch": 3, "took": "8.728713255s", "keys": 208913}
2023-04-11T16:25:50.827+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 1, "took": "55.426411416s"}
2023-04-11T16:26:28.734+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 2, "took": "37.902647706s"}
2023-04-11T16:26:53.960+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 1, "took": "1m3.129453265s", "keys": 200001}
2023-04-11T16:27:27.645+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 3, "took": "33.685283662s"}
2023-04-11T16:27:52.173+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 2, "took": "1m23.438465575s", "keys": 199999}
2023-04-11T16:28:00.995+0300 INFO last batch of contract storage items is collected {"batch": 4, "took": "8.821990443s"}
2023-04-11T16:28:00.995+0300 INFO contract storage items are reset {"took": "3m5.595950958s", "keys": 656944}
2023-04-11T16:28:49.164+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 3, "took": "1m21.518344712s", "keys": 200000}
2023-04-11T16:28:49.164+0300 DEBUG trying to reset headers information
2023-04-11T16:28:49.936+0300 INFO headers information is reset {"took": "772.36435ms"}
2023-04-11T16:28:53.122+0300 DEBUG last batch of contract storage items and IDs is persisted {"batch": 4, "took": "52.126928092s", "keys": 56945}
2023-04-11T16:28:53.122+0300 DEBUG trying to reset state root information and NEP transfers
2023-04-11T16:29:09.332+0300 INFO state root information and NEP transfers are reset {"took": "16.20921699s"}
2023-04-11T16:29:46.264+0300 DEBUG headers information is persisted {"took": "56.326715249s", "keys": 528438}
2023-04-11T16:29:46.267+0300 DEBUG trying to remove stale storage items
2023-04-11T16:29:53.986+0300 INFO stale storage items are reset {"took": "7.718950145s", "keys": 1350176}
2023-04-11T16:29:53.986+0300 DEBUG trying to remove state reset point
2023-04-11T16:29:53.986+0300 INFO state reset point is removed {"took": "6.013µs"}
2023-04-11T16:29:55.899+0300 DEBUG state root information and NEP transfers are persisted {"took": "46.567302762s", "keys": 602578}
2023-04-11T16:29:55.929+0300 INFO state reset point information is persisted {"took": "1.942392208s", "keys": 2}
2023-04-11T16:29:55.929+0300 INFO reset finished successfully {"took": "5m51.561573137s"}
```
Master:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000
2023-04-11T16:34:12.410+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144}
2023-04-11T16:34:12.410+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000}
2023-04-11T16:34:12.410+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512}
2023-04-11T16:34:12.410+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2023-04-11T16:34:12.413+0300 INFO restoring blockchain {"version": "0.2.8"}
2023-04-11T16:34:12.495+0300 INFO initialize state reset {"target height": 600000}
2023-04-11T16:34:12.513+0300 INFO trying to reset blocks, transactions and AERs
2023-04-11T16:35:03.582+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 1, "took": "51.087226195s", "keys": 321895}
2023-04-11T16:35:31.302+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 2, "took": "27.719871393s", "keys": 334823}
2023-04-11T16:35:41.309+0300 INFO last batch of removed blocks, transactions and AERs is persisted {"batches persisted": 3, "took": "10.007017388s", "keys": 208913}
2023-04-11T16:35:41.309+0300 INFO blocks, transactions ans AERs are reset {"took": "1m28.814245057s", "overall persisted keys": 865631}
2023-04-11T16:35:41.309+0300 INFO trying to reset contract storage items
2023-04-11T16:37:38.315+0300 INFO intermediate batch of contract storage items and IDs is persisted {"batch": 1, "took": "1m57.00650253s", "keys": 200000}
2023-04-11T16:39:29.704+0300 INFO intermediate batch of contract storage items and IDs is persisted {"batch": 2, "took": "1m51.385224725s", "keys": 200000}
2023-04-11T16:41:14.991+0300 INFO intermediate batch of contract storage items and IDs is persisted {"batch": 3, "took": "1m45.287483794s", "keys": 200000}
2023-04-11T16:41:31.667+0300 INFO last batch of contract storage items and IDs is persisted {"batch": 4, "took": "16.675347478s", "keys": 56945}
2023-04-11T16:41:31.667+0300 INFO contract storage items and IDs are reset {"took": "5m50.357775401s", "keys": 656944}
2023-04-11T16:41:31.667+0300 INFO trying to reset headers information
2023-04-11T16:42:16.779+0300 INFO headers information is reset {"took": "45.111354262s", "keys": 528438}
2023-04-11T16:42:16.784+0300 INFO trying to reset state root information and NEP transfers
2023-04-11T16:42:35.778+0300 INFO state root information and NEP transfers are reset {"took": "18.99373117s", "keys": 602578}
2023-04-11T16:42:35.781+0300 INFO trying to remove stale storage items
2023-04-11T16:42:43.884+0300 INFO stale storage items are reset {"took": "8.103929306s", "keys": 1350176}
2023-04-11T16:42:43.885+0300 INFO trying to remove state reset point
2023-04-11T16:42:43.926+0300 INFO stale reset point is removed {"took": "41.858883ms", "keys": 2}
2023-04-11T16:42:43.932+0300 INFO reset finished successfully {"took": "8m31.437493325s"}
```
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Everywhere including examples, external interop APIs, bindings generators
code and in other valuable places. A couple of `interface{}` usages are
intentionally left in the CHANGELOG.md, documentation and tests.
Do not add them directly to chain, it will be done by the block queue
manager. Close https://github.com/nspcc-dev/neo-go/issues/2923. However,
this commit is not valid without
https://github.com/roman-khimov/dbft/pull/4.
It's the neo-go's duty to initialize consensus after subsequent block
addition; the dBFT itself must wait for the neo-go to complete the block
addition and notify the dBFT, so that it can initialize at 0-th view to
collect the next block.
It has a special `requestF` and a special initialization function, but other
than that it's an absolutely regular WSClient. Can be used to call, can be
used to subscribe. Fixes#2909.
If NEP2 account label is not provided as a flag for import-related
CLI commands then ask it in the interactive mode.
This change breaks the compatibility with the old behaviour (add
the unnamed account if the name wasn't specified).
Fetch account's name from the CLI argument and do not ask for the
user's input if provided, close#2882. Fetch password from config
if provided, close#2883.
According to docs, `Server` uses provided error channel only to write
encountered error to it. In this case, there is no need to accept rw
channel to create `Server` instance. Strengthening the type to
write-only will allow the caller to ensure control of reading errors
from the provided channel.
The change is backward compatible since any `chan` is `chan<-`.
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
`Actor.MakeUnsignedUncheckedRun` method imposes restriction to
`CalculateNetworkFee` method's implementations: `Hash` or `Size` methods
must not be called on the pointer to the given transaction.
Add docs to adjust described requirement.
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
It won't work with 1 for a single node and 3 for 4 CN scenario doesn't allow
the network to start with just three nodes (although technically 3 out of 4 is
sufficient to operate). See #2874.
We have both from and to here, so technically we can either drop the neg/neg
trick from the processTokenTransfer() or drop one field from the structure
(the other side is a part of the key). Drop the field since this can make the
DB a bit more compact. Change Amount to be a pointer along the way since
that's the "native" thing for big.Int, we've used non-pointer field
specifically to avoid Neg/Neg problems, but it looks like this is not
necessary.
This structure is only used by the RPC server and I doubt anyone uses it via
the *Blockchain.
In some cases n.Add() can reuse the []Word buffer and n.Sub() reallocate it
away. If that happens, we're out of luck with 0.99.0+ versions (since
3945e81857). I'm not sure why it does that, bit
width doesn't change in most of the cases and even if it does, we still have
enough of it in cap() to hold the old Abs() value (when we have a negative
value we in fact decreate its Abs() first and increase it back
afterwards). Still, that's what we have.
So when we have processTokenTransfer() doing Neg/Neg in-place its value is not
affected, but the original []Word bits that are reused by amount value are
(they're shared initially, Amount: *amount).
name old time/op new time/op delta
ToPreallocatedBytes-8 65.8ns ± 2% 45.6ns ± 2% -30.73% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
ToPreallocatedBytes-8 0.00B 0.00B ~ (all equal)
name old allocs/op new allocs/op delta
ToPreallocatedBytes-8 0.00 0.00 ~ (all equal)
d5a9af5860 is incompatible with the NeoFS
mainnet sidechain, so we add the old logic to the pre-Aspidochelone
behaviour. Changing flags at newMethodAndPrice() is a bit less convenient
unfortunately because this will affect interop validity checks, so let's have
this kludge here.
6b4dd5703e made it to be a uint16 which was
somewhat important for RPC, but now it's irrelevant and the fact that it was a
string in some cases may lead to errors like these:
failed to unmarshal config YAML: yaml: unmarshal errors:
line 48: cannot unmarshal !!str `20011` into uint16
line 52: cannot unmarshal !!str `40001` into uint16
So for maximum backwards compatibility we better have string here and
eventually it'll be deleted anyway.
It directly affects node security and the default here MUST BE the safe choice
which is to do the verification. Otherwise it's just dangerous, absent any
VerifyBlocks configuration we'll get an insecure node. This option is not
supposed to be frequently used and it doesn't affect the ability to process
blocks, so breaking compatibility (in a safe manner) should be OK here.
And include some node-specific configurations there with backwards
compatibility. Note that in the future we'll remove Ledger's
fields from the ProtocolConfiguration and it'll be possible to access them in
Blockchain directly (not via .Ledger).
The other option tried was using two configuration types separately, but that
incurs more changes to the codebase, single structure that behaves almost like
the old one is better for backwards compatibility.
Fixes#2676.
It doesn't store id->hash mappings for native contracts. We need blockchain's
GetContractScriptHash to serve both anyway, so it was changed a bit. The only
other direct user of native.GetContractScriptHash is the VM CLI, but I doubt
anyone will use it for native contracts (they have ~zero VM code anyway).
There are no changes visible from the user side (at least for those
users who doesn't put Prometheus's or pprof's port in quotes), just
internal refactoring. From now and on, BasicService configuration is
used by RPC server config, TLS for RPC server, pprof and Prometheus.
I do see some reduced testing time for cli components with this setting even
though it doesn't affect the overall time for me. Maybe it'll be beneficial in
some cases. Refs. #2379.
It's more generic and convenient than MillisecondsPerBlock. This setting is
made in backwards-compatible fashion, but it'll override SecondsPerBlock if
both are used. Configurations are specifically not changed here, it's
important to check compatibility.
Fixes#2675.
Follow neo-project/neo#2807. Notice that this data is not cached, our previous
implementation wasn't too and it shouldn't be a problem (not on the hot path).
They can stay in the memory pool forever because consensus process will never
accept these transactions (and maybe even block consensus process at all).
We're paging these hashes, so we need a previous full page and a current one
plus some cache for various requests. Storing 1M of hashes is 32M of memory
and it grows quickly. It also seriously affects node startup time, most of
what it's doing is reading these hashes, the longer the chain the more time it
needs to do that.
Notice that this doesn't change the underlying DB scheme in any way.
If we only have genesis block (or <2000 headers) then we might as well use
generic logic below with zero targetHash because genesis block has zero
PrevHash (and its hash will naturally be the last on the chain going
backwards).
Let upper-layer APIs like actor.Send() return it as well. Server can return
"already exists" which is an error and yet at the same time a very special
one, in many cases it means we can proceed with waiting for the TX to settle.
Sometimes it can be hard to persist all changes at ones, the process
can take almost all RAM and a lot of time. Here's the example of reset
for mainnet from 2.4M to 1:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:16:48.236+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144}
2022-11-20T17:16:48.236+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000}
2022-11-20T17:16:48.237+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512}
2022-11-20T17:16:48.237+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:16:48.240+0300 INFO restoring blockchain {"version": "0.2.6"}
2022-11-20T17:16:48.297+0300 INFO initialize state reset {"target height": 1}
2022-11-20T17:16:48.300+0300 INFO trying to reset blocks, transactions and AERs
2022-11-20T17:19:29.313+0300 INFO blocks, transactions ans AERs are reset {"took": "2m41.015126493s", "keys": 3958420}
...
```
To avoid OOM killer, split blocks reset into multiple stages. It increases
operation time due to intermediate DB persists, but makes things cleaner, the
result for almost the same DB height with the new approach:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --height 1
2022-11-20T17:39:42.023+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144}
2022-11-20T17:39:42.023+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000}
2022-11-20T17:39:42.023+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512}
2022-11-20T17:39:42.023+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2022-11-20T17:39:42.026+0300 INFO restoring blockchain {"version": "0.2.6"}
2022-11-20T17:39:42.071+0300 INFO initialize state reset {"target height": 1}
2022-11-20T17:39:42.073+0300 INFO trying to reset blocks, transactions and AERs
2022-11-20T17:40:11.735+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 1, "took": "29.66363737s", "keys": 210973}
2022-11-20T17:40:33.574+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 2, "took": "21.839208683s", "keys": 241203}
2022-11-20T17:41:29.325+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 3, "took": "55.750698386s", "keys": 250593}
2022-11-20T17:42:12.532+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 4, "took": "43.205892757s", "keys": 321896}
2022-11-20T17:43:07.978+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 5, "took": "55.445398156s", "keys": 334822}
2022-11-20T17:43:35.603+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 6, "took": "27.625292032s", "keys": 317131}
2022-11-20T17:43:51.747+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 7, "took": "16.144359017s", "keys": 355832}
2022-11-20T17:44:05.176+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 8, "took": "13.428733899s", "keys": 357690}
2022-11-20T17:44:32.895+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 9, "took": "27.718548783s", "keys": 393356}
2022-11-20T17:44:51.814+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 10, "took": "18.917954658s", "keys": 366492}
2022-11-20T17:45:07.208+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 11, "took": "15.392642196s", "keys": 326030}
2022-11-20T17:45:18.776+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 12, "took": "11.568255716s", "keys": 299884}
2022-11-20T17:45:25.862+0300 INFO last batch of removed blocks, transactions and AERs is persisted {"batches persisted": 13, "took": "7.086079594s", "keys": 190399}
2022-11-20T17:45:25.862+0300 INFO blocks, transactions ans AERs are reset {"took": "5m43.791214084s", "overall persisted keys": 3966301}
...
```
We need to keep the headers information consistent with header batches
and headers. This comit fixes the bug with failing blockchain
initialization on recovering from state reset interrupted after the
second stage (blocks/txs/AERs removal):
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T16:28:29.437+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2022-11-20T16:28:29.440+0300 INFO restoring blockchain {"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: could not get header 1898cd356a4a2688ed1c6c7ba1fd6ba7d516959d8add3f8dd26232474d4539bd: key not found
```
Don't use cache because it's not yet initialized. Also, perform
safety checks only if state reset wasn't yet started. These fixes
alloww to solve the following problem while recovering from
interrupted state reset:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -t --height 83000
2022-11-20T15:51:31.431+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2022-11-20T15:51:31.434+0300 INFO restoring blockchain {"version": "0.2.6"}
failed to create Blockchain instance: could not initialize blockchain: current block height is 0, can't reset state to height 83000
```
It's useful to reproduce different execution problems including executions
stopped because of GAS limit. Satoshi representation is deliberately used,
because that's what is usually found in logs.
We don't use all of the Stack functionality for it, so drop useless methods
and avoid some interface conversions. It increases single-node TPS by about
0.9%, so nothing really important there, but not a bad change either. Maybe it
can be reworked again with generics though.
Blockchain's subscriptions, unsubscriptions and notifications are
handled by a single notificationDispatcher routine. Thus, on attempt
to send the subsequent event to Blockchain's subscribers, dispatcher
can't handle subscriptions\unsubscriptions. Make subscription and
unsubscription to be a non-blocking operation for blockchain on the
server side, otherwise it may cause the dispatcher locks.
To achieve this, use a separate lock for those code that make calls
to blockchain's subscription API and for subscription counters on
the server side.
Small (especially dockerized/virtualized) networks often start all nodes at
ones and then we see a lot of connection flapping in the log. This happens
because nodes try to connect to each other simultaneously, establish two
connections, then each one finds a duplicate and drops it, but this can be
different duplicate connections on other sides, so they retry and it all
happens for some time. Eventually everything settles, but we have a lot of
garbage in the log and a lot of useless attempts.
This random waiting timeout doesn't change the logic much, adds a minimal
delay, but increases chances for both nodes to establish a proper single
connection on both sides to only then see another one and drop it on both
sides as well. It leads to almost no flapping in small networks, doesn't
affect much bigger ones. The delay is close to unnoticeable especially if
there is something in the DB for node to process during startup.
Consider mainnet, it has an AttemptConnPeers of 20, so may already have 3
peers and request 20 more, then have 4th connected and attemtp 20 more again,
this leads to a huge number of connections easily.
Consider initial connection phase for public networks:
* simultaneous connections to seeds
* very quick handshakes
* got five handshaked peers and some getaddr requests sent
* but addr replies won't trigger new connections
* so we can stay with just five connections until any of them breaks or a
(long) address checking timer fires
This new timers solves the problem, it's adaptive at the same time. If we have
enough peers we won't be waking up often.
* treat connected/handshaked peers separately in the discoverer, save
"original" address for connected ones, it can be a name instead of IP and
it's important to keep it to avoid reconnections
* store name->IP mapping for seeds if and when they're connected to avoid
reconnections
* block seed if it's detected to be our own node (which is often the case for
small private networks)
* add an event for handshaked peers in the server, connected but
non-handshaked ones are not really helpful for MinPeers or GetAddr logic
Fixes#2796.
Problem: failing part of TestLoad:
```
=== RUN TestLoad/loadgo,_check_signers
cli_test.go:160:
Error Trace: /home/circleci/go/src/github.com/nspcc-dev/neo-go/cli/vm/cli_test.go:160
/home/circleci/go/src/github.com/nspcc-dev/neo-go/cli/vm/cli_test.go:147
/home/circleci/go/src/github.com/nspcc-dev/neo-go/cli/vm/cli_test.go:444
Error: command took too long time
Test: TestLoad/loadgo,_check_signers
```
Solution: split the test into multiple parts to reduce test execution time.
If VUB-th block is received, we still can't guaranty that transaction
wasn't accepted to chain. Back this situation by rolling back to a
poll-based waiter.
Do not block subscribers until the unsubscription request to RPC server
is completed. Otherwise, another notification may be received from the
RPC server which will block the unsubscription process.
At the same time, fix event-based waiter. We must not block the receiver
channel during unsubscription because there's a chance that subsequent
event will be sent by the server. We need to read this event in order not
to block the WSClient's readloop.
Bad contract -> no contract. Unfortunately we've got a broken
6f1837723768f27a6f6a14452977e3e0e264f2cc contract on the mainnet which can't
be decoded (even though it had been saved successfully), so this is a
temporary fix for #2801 to be able to start mainnet node after shutdown.
v.estack is used throughout the code to work with estack, while ctx.sc.estack
is (theoretically) just a reference to it that is saved on script load and
restored to v.estack on context unload. The problem is that v.estack can grow
as we use it and can be reallocated away from its original slice (saved in the
ctx.sc.estack), so either ctx.sc.estack should be a pointer or we need to
ensure that it's correct when loading a new script. The second approach is a
bit safer for now and it fixes#2798.
And fix failing test along the way:
```
2022-11-11T12:37:47.0413934Z === RUN TestResetDB
2022-11-11T12:37:47.0414557Z 2022-11-11T12:36:54.510Z INFO initial gas supply is not set or wrong, setting default value {"InitialGASSupply": "52000000"}
2022-11-11T12:37:47.0415288Z 2022-11-11T12:36:54.510Z INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144}
2022-11-11T12:37:47.0416020Z 2022-11-11T12:36:54.510Z INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000}
2022-11-11T12:37:47.0416786Z 2022-11-11T12:36:54.510Z INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512}
2022-11-11T12:37:47.0417725Z 2022-11-11T12:36:54.510Z INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760}
2022-11-11T12:37:47.0418415Z 2022-11-11T12:36:54.510Z INFO Hardforks are not set, using default value
2022-11-11T12:37:47.0419272Z 2022-11-11T12:36:54.523Z INFO no storage version found! creating genesis block
2022-11-11T12:37:47.0419997Z 2022-11-11T12:36:54.529Z INFO chain is already at the proper state {"height": 0}
2022-11-11T12:37:47.0420974Z testing.go:1097: TempDir RemoveAll cleanup: remove C:\Users\RUNNER~1\AppData\Local\Temp\TestResetDB671187463\001\chains\privnet\000001.log: The process cannot access the file because it is being used by another process.
2022-11-11T12:37:47.0421606Z --- FAIL: TestResetDB (1.99s)
```
client_test.go:1935:
Error Trace: /home/rik/dev/neo-go/pkg/services/rpcsrv/client_test.go:1935
Error: Should NOT be empty, but was 00000000-0000-0000-0000-000000000000
Test: TestClient_Iterator_SessionConfigVariations/sessions_disabled
It's obviously empty, since we have sessions disabled, but it was not
considered to be empty in testify 1.7.0, now it is, see 840cb80149
There is a security issue found in github.com/btcsuite/btcd that we don't care
about (we're only using 256k1 implementation), but GitHub complains about
it. We could update to github.com/btcsuite/btcd/btcec/v2, but it's now just a
thin wrapper over github.com/decred/dcrd/dcrec/secp256k1/v4, so we better use
it directly.
* strip NEP-XX methods before going into generator to avoid unused imports
* nepXX.Invoker types already include Call
* always import util, it's used for Hash
Execution events are followed by block events, not vise versa, thus,
we can wait until VUB block to be accepted to be sure that
transaction wasn't accepted to chain.
Every 1000 blocks seems to be OK for big networks (that only had done some
initial requests previously and then effectively never requested addresses
again because there was a sufficient number of addresses), won't hurt smaller
ones as well (that effectively keep doing this on every connect/disconnect,
peer changes are very rare there, but when they happen we want to have some
quick reaction to these changes).
32 is a very good number, but we all know 42 is a better one. And it can even
be proven by tests with higher peaking TPS values.
You may wonder why is it so good? Because we're using packet-switching
networks mostly and a packet is a packet almost irrespectively of how bit it
is. Yet a packet has some maximum possible size (hi, MTU) and this size most
of the time is 1500 (or a little less than that, hi VPN). Subtract IP header
(20 for IPv4 or 40 for IPv6 not counting options), TCP header (another 20) and
Neo message/payload headers (~8 for this case) and we have just a little more
than 1400 bytes for our dear hashes. Which means that in a single packet most
of the time we can have 42-44 of them, maybe 45. Choosing between these
numbers is not hard then.
We have AttemptConnPeers that is closely related, the more we have there the
bigger the network supposedly is, so it's much better than magic minPoolCount.
When block is being spread through the network we can get a lot of invs with
the same hash. Some more stale nodes may also announce previous or some
earlier block. We can avoid full DB lookup for them and minimize inv handling
time (timeouts in inv handler had happened in #2744).
It doesn't affect tests, just makes node a little less likely to spend some
considerable amount of time in the inv handler.
Sometimes we already have it, but it's not yet processed, so we can save on
getdata request. It only affects very high-speed networks like 4-1 scenario
and it doesn't affect it a lot, but still we can do it.
This is not exactly the protocol-level batching as was tried in #1770 and
proposed by neo-project/neo#2365, but it's a TCP-level change in that we now
Write() a set of messages and given that Go sets up TCP sockets with
TCP_NODELAY by default this is a substantial change, we have less packets
generated with the same amount of data. It doesn't change anything on properly
connected networks, but the ones with delays benefit from it a lot.
This also improves queueing because we no longer generate 32 messages to
deliver on transaction's GetData, it's just one stream of bytes with 32
messages inside.
Do the same with GetBlocksByIndex, we can have a lot of messages there too.
But don't forget about potential peer DoS attacks, if a peer is to request a
lot of big blocks we need to flush them before we process the whole set.
This allows to naturally scale transaction processing if we have some peer
that is sending a lot of them while others are mostly silent. It also can help
somewhat in the event we have 50 peers that all send transactions. 4+1
scenario benefits a lot from it, while 7+2 slows down a little. Delayed
scenarios don't care.
Surprisingly, this also makes disconnects (#2744) much more rare, 4-node
scenario almost never sees it now. Most probably this is the case where peers
affect each other a lot, single-threaded transaction receiver can be slow
enough to trigger some timeout in getdata handler of its peer (because it
tries to push a number of replies).
We can't make it before 2550000, so move to 2600000. @anatoly-bogatyrev,
please ensure you're using this new height for mainnet NeoFS configurations
next week.
It makes sense in general (further narrowing down the time window when
transactions are processed by consensus thread) and it improves block times a
little too, especially in the 7+2 scenario.
Related to #2744.
Until the consensus process starts for a new block and until it really needs
some transactions we can spare some cycles by not delivering transactions to
it. In tests this doesn't affect TPS, but makes block delays a bit more
stable. Related to #2744, I think it also may cause timeouts during
transaction processing (waiting on the consensus process channel while it does
something dBFT-related).
When the network is big enough, MinPeers may be suboptimal for good network
connectivity, but if we know the network size we can do some estimation on the
number of sufficient peers.
1. UsageText shows the command usage rule. Fixed/added where needed.
2. Description shows the command description, huh. It is shown right after
UsageText, so there's no need to repeat the command usage rule. If
Description contains Example, then it should be printed on a new line.
Usage message is shown on common --help command, thus it should be meaningful
and short. If user needs more detailed command description, then he can use
command-specific help.
As a result, current VM help looks pretty simple:
```
NEO-GO-VM > help
NAME:
VM CLI - Official VM CLI for Neo-Go
USAGE:
[global options] command [command options] [arguments...]
VERSION:
0.99.5-pre-15-g5463ec41
COMMANDS:
exit Exit the VM prompt
ip Show current instruction
break Place a breakpoint
jump Jump to the specified instruction (absolute IP value)
estack Show evaluation stack contents
istack Show invocation stack contents
sslot Show static slot contents
lslot Show local slot contents
aslot Show arguments slot contents
loadnef Load a NEF-consistent script into the VM optionally attaching to it provided signers with scopes
loadbase64 Load a base64-encoded script string into the VM optionally attaching to it provided signers with scopes
loadhex Load a hex-encoded script string into the VM optionally attaching to it provided signers with scopes
loadgo Compile and load a Go file with the manifest into the VM optionally attaching to it provided signers with scopes
loadtx Load transaction into the VM from chain or from parameter context file
loaddeployed Load deployed contract into the VM from chain optionally attaching to it provided signers with scopes
reset Unload compiled script from the VM and reset context to proper (possibly, historic) state
parse Parse provided argument and convert it into other possible formats
run Execute the current loaded script
cont Continue execution of the current loaded script
step Step (n) instruction in the program
stepinto Stepinto instruction to take in the debugger
stepout Stepout instruction to take in the debugger
stepover Stepover instruction to take in the debugger
ops Dump opcodes of the current loaded program
events Dump events emitted by the current loaded program
env Dump state of the chain that is used for VM CLI invocations (use -v for verbose node configuration)
storage Dump storage of the contract with the specified hash, address or ID as is at the current stage of script invocation
changes Dump storage changes as is at the current stage of loaded script invocation
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help
--version, -v print the version
```
Share parameters parsing code between 'contract invokefunction' and
'vm run' commands. It allows VM CLI to parse more complicated parameter
types including arrays and file-backed bytestrings.
They can fail right in the getPeers or they can fail later when packet send
is attempted. Of course they can complete handshake in-between these events,
but most likely they won't and we'll waste more resources on this attempt. So
rule out bad peers immediately.
Drop EnqueueP2PPacket, replace EnqueueHPPacket with EnqueueHPMessage. We use
Enqueue* when we have a specific per-peer message, it makes zero sense
duplicating serialization code for it (unlike Broadcast*).
Follow the general rules of broadcasts, even though it's somewhat different
from Inv, we just want to get some reply from our neighbors to see if we're
behind. We don't strictly need all neighbors for it.
We have a number of queues for different purposes:
* regular broadcast queue
* direct p2p queue
* high-priority queue
And two basic egress scenarios:
* direct p2p messages (replies to requests in Server's handle* methods)
* broadcasted messages
Low priority broadcasted messages:
* transaction inventories
* block inventories
* notary inventories
* non-consensus extensibles
High-priority broadcasted messages:
* consensus extensibles
* getdata transaction requests from consensus process
* getaddr requests
P2P messages are a bit more complicated, most of the time they use p2p queue,
but extensible message requests/replies use HP queue.
Server's handle* code is run from Peer's handleIncoming, every peer has this
thread that handles incoming messages. When working with the peer it's
important to reply to requests and blocking this thread until we send (queue)
a reply is fine, if the peer is slow we just won't get anything new from
it. The queue used is irrelevant wrt this issue.
Broadcasted messages are radically different, we want them to be delivered to
many peers, but we don't care about specific ones. If it's delivered to 2/3 of
the peers we're fine, if it's delivered to more of them --- it's not an
issue. But doing this fairly is not an easy thing, current code tries performing
unblocked sends and if this doesn't yield enough results it then blocks (but
has a timeout, we can't wait indefinitely). But it does so in sequential
manner, once the peer is chosen the code will wait for it (and only it) until
timeout happens.
What can be done instead is an attempt to push the message to all of the peers
simultaneously (or close to that). If they all deliver --- OK, if some block
and wait then we can wait until _any_ of them pushes the message through (or
global timeout happens, we still can't wait forever). If we have enough
deliveries then we can cancel pending ones and it's again not an error if
these canceled threads still do their job.
This makes the system more dynamic and adds some substantial processing
overhead, but it's a networking code, any of this overhead is much lower than
the actual packet delivery time. It also allows to spread the load more
fairly, if there is any spare queue it'll get the packet and release the
broadcaster. On the next broadcast iteration another peer is more likely to be
chosen just because it didn't get a message previously (and had some time to
deliver already queued messages).
It works perfectly in tests, with optimal networking conditions we have much
better block times and TPS increases by 5-25%% depending on the scenario.
I'd go as far as to say that it fixes the original problem of #2678, because
in this particular scenario we have empty queues in ~100% of the cases and
this new logic will likely lead to 100% fan out in this case (cancelation just
won't happen fast enough). But when the load grows and there is some waiting
in the queue it will optimize out the slowest links.
Value of PublicKey parameter always stores public key bytes, not the
deserialized representation. All other code (CLI parameters parsing with
its NewParameterFromString, Parameter unmarshaller, etc.) is based on
the idea that value of PublicKey is []byte.
Peers can be slow, very slow, slow enough to affect node's regular
operation. We can't wait for them indefinitely, there has to be a timeout for
send operations.
This patch uses TimePerBlock as a reference for its timeout. It's relatively
big and it doesn't affect tests much, 4+1 scenarios tend to perform a little
worse with while 7+2 scenarios work a little better. The difference is in some
percents, but all of these tests easily have 10-15% variations from run to
run.
It's an important step in making our gossip better because we can't have any
behavior where neighbors directly block the node forever, refs. #2678 and
The tests are still there, coverage should counted fine, but it improves things:
? github.com/nspcc-dev/neo-go/cli [no test files]
ok github.com/nspcc-dev/neo-go/cli/app 0.058s coverage: 100.0% of statements
ok github.com/nspcc-dev/neo-go/cli/cmdargs 0.005s coverage: 60.8% of statements
ok github.com/nspcc-dev/neo-go/cli/flags 0.027s coverage: 97.7% of statements
? github.com/nspcc-dev/neo-go/cli/input [no test files]
ok github.com/nspcc-dev/neo-go/cli/nep_test 30.443s coverage: [no statements]
ok github.com/nspcc-dev/neo-go/cli/options 0.054s coverage: 50.0% of statements
? github.com/nspcc-dev/neo-go/cli/paramcontext [no test files]
ok github.com/nspcc-dev/neo-go/cli/query 2.089s coverage: 45.3% of statements
ok github.com/nspcc-dev/neo-go/cli/server 1.510s coverage: 67.8% of statements
ok github.com/nspcc-dev/neo-go/cli/smartcontract 8.433s coverage: 94.3% of statements
ok github.com/nspcc-dev/neo-go/cli/util 0.013s coverage: 10.9% of statements
? github.com/nspcc-dev/neo-go/cli/vm [no test files]
ok github.com/nspcc-dev/neo-go/cli/wallet 47.252s coverage: 63.0% of statements
Refs. #2379, but not completely solves it, one package seriously outweights
others:
? github.com/nspcc-dev/neo-go/cli [no test files]
ok github.com/nspcc-dev/neo-go/cli/app 0.036s coverage: 100.0% of statements
ok github.com/nspcc-dev/neo-go/cli/cmdargs 0.011s coverage: 60.8% of statements
ok github.com/nspcc-dev/neo-go/cli/flags 0.009s coverage: 97.7% of statements
? github.com/nspcc-dev/neo-go/cli/input [no test files]
ok github.com/nspcc-dev/neo-go/cli/options 0.033s coverage: 50.0% of statements
? github.com/nspcc-dev/neo-go/cli/paramcontext [no test files]
ok github.com/nspcc-dev/neo-go/cli/query 2.155s coverage: 45.3% of statements
ok github.com/nspcc-dev/neo-go/cli/server 1.373s coverage: 67.8% of statements
ok github.com/nspcc-dev/neo-go/cli/smartcontract 8.819s coverage: 94.3% of statements
ok github.com/nspcc-dev/neo-go/cli/util 0.006s coverage: 10.9% of statements
? github.com/nspcc-dev/neo-go/cli/vm [no test files]
ok github.com/nspcc-dev/neo-go/cli/wallet 72.103s coverage: 88.2% of statements
Still a nice thing to have.
In case of ellipsis usage compiler defines argument type as ArrayT
(which is correct, because it's a natural representation of the last
argument, it represents the array of interface{}).
Here goes the problem:
```
=== RUN TestEventWarnings/variadic_event_args_via_ellipsis
compiler_test.go:251:
Error Trace: compiler_test.go:251
Error: Received unexpected error:
event 'Event' should have 'Integer' as type of 1 parameter, got: Array
Test: TestEventWarnings/variadic_event_args_via_ellipsis
```
Parsing the last argument in this case is a separate complicated problem
due to the fact that we need to grab types of elements of []interface{} inside the
fully qualified ast node which may looks like:
```
runtime.Notify("Event", (append([]interface{}{1, 2}, (([]interface{}{someVar, 4}))...))...)
```
Temporary solution is to exclude such notifications from analysis until we're
able to properly resolve element types of []interface{}.
It's possible that declared manifest event has parameter of AnyT for
those cases when parameter type differs from method to method. If so,
then we don't need to enforce type check after compilation.
Otherwise the following error occurs while updating dependency:
```
github.com/nspcc-dev/neo-go/examples/nft-nd-nns tested by
github.com/nspcc-dev/neo-go/examples/nft-nd-nns.test imports
github.com/nspcc-dev/neo-go/pkg/compiler imports
gopkg.in/yaml.v3 tested by
gopkg.in/yaml.v3.test imports
gopkg.in/check.v1 imports
github.com/kr/pretty loaded from github.com/kr/pretty@v0.1.0,
but go 1.16 would select v0.3.0
github.com/nspcc-dev/neo-go/examples/nft-nd-nns tested by
github.com/nspcc-dev/neo-go/examples/nft-nd-nns.test imports
github.com/nspcc-dev/neo-go/pkg/compiler imports
gopkg.in/yaml.v3 tested by
gopkg.in/yaml.v3.test imports
gopkg.in/check.v1 imports
github.com/kr/pretty imports
github.com/kr/text loaded from github.com/kr/text@v0.1.0,
but go 1.16 would select v0.2.0
To upgrade to the versions selected by go 1.16:
go mod tidy -go=1.16 && go mod tidy -go=1.17
If reproducibility with go 1.16 is not needed:
go mod tidy -compat=1.17
For other options, see:
https://golang.org/doc/modules/pruning
```
We've declared that we are using semantic versioning. We also want to use `git
describe` to make version strings for us because it's very convenient for
development builds (tagged versions are way simpler). The problem is that the
default `git describe` behavior is not semver compliant. If the most recent
tag is v0.99.2 then it'll generate something like '0.99.2-131-g8dc5b385',
which according to semver is a development version _before_ 0.99.2. While it's
obviously a version _after_ 0.99.2.
That's the one and only reason we have vX.Y.Z-pre tags in our repo. We set
them right after the release according to the release process and that gives
us some '0.99.3-pre-131-g8dc5b385' versions we're all used to. But these tags
are ugly as hell and they clutter up our repo over time.
So there is this idea that we can do patch version increment dynamically.
Making '0.99.2-131-g8dc5b385' be '0.99.3-pre-131-g8dc5b385' without any *-pre
tags. This patch implements this. It's ugly as hell as well, but at least
that's an ugliness somewhere inside our Makefile and not directly visible in
our tags. If we're to do this we can then greatly simplify our release process
(and even allow for CHANGELOG patches to be merged normally).
I know this can be done with awk in somewhat easier way, but no, I'm not into
awk, sorry.
Which greatly simplifies reuse of these packages (and they're expected to be
reused since real tokens implement standards and also add something of their
own) and allows to avoid effects like
doc_test.go:68:28: ambiguous selector neoContract.BalanceOf
when neo.Contract is used. Avoids duplication in NEP-11 implementation as
well.
They have not found anything and maybe we better not have them find anything
in the future. Commented ones can be useful in future, but find too many
problems at the moment.
WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter. Replaced by unused.
WARN [runner] The linter 'deadcode' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter. Replaced by unused.
WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter. Replaced by unused.
So that (*codegen).Visit is able to omit code generation for these
unused global vars. The most tricky part is to detect unused global
variables, it is done in several steps:
1. Collect the set of named used/unused global vars.
2. Collect the set of globally declared expressions that contain
function calls.
3. Pick up global vars from the set made at step 2.
4. Traverse used functions and puck up those global vars that are used
from these functions.
5. Rename all globals that are presented in the set made at step 1
but are not presented in the set made on step 3 or step 4.
Move all auxiliary function declaration after Main, so that INITSLOT
instructions counter works properly. `vmAndCompileInterop` loads program
and moves nextIP to the Main function offset if there's no _init
function. If _init is there, then nextIP will be moved to the start of
_init. In TestInline we don't handle instructions properly (CALL/JMP
don't change nextIP), we just perform instruction traversal from the
start point via Next(), thus INITSLOT counter value depends on the
starting instruction, which depends on _init presence.
If variable is unnamed and does not contain function call then it's
treated as unused and code generation may be omitted for it
initialization/declaration.
In case if global var is unnamed (and, as a consequence, unused) and
contains a function call inside its value specification, we need to emit
code for this var to be able to call the function as it can have
side-effects. See the example:
```
package foo
import "github.com/nspcc-dev/neo-go/pkg/interop/runtime"
var A = f()
func Main() int {
return 3
}
func f() int {
runtime.Notify("Valuable notification", 1)
return 2
}
```
2022-09-01 09:18:07 +03:00
731 changed files with 50924 additions and 15674 deletions
cli.BoolFlag{Name:"privnet, p",Usage:"use private network configuration (if --config-file option is not specified)"},
cli.BoolFlag{Name:"mainnet, m",Usage:"use mainnet network configuration (if --config-file option is not specified)"},
cli.BoolFlag{Name:"testnet, t",Usage:"use testnet network configuration (if --config-file option is not specified)"},
cli.BoolFlag{Name:"unittest",Hidden:true},
}
@ -42,7 +77,42 @@ var RPC = []cli.Flag{
},
}
// Historic is a flag for commands that can perform historic invocations.
varHistoric=cli.StringFlag{
Name:"historic",
Usage:"Use historic state (height, block hash or state root hash)",
}
// Config is a flag for commands that use node configuration.
varConfig=cli.StringFlag{
Name:"config-path",
Usage:"path to directory with per-network configuration files (may be overridden by --config-file option for the configuration file)",
}
// ConfigFile is a flag for commands that use node configuration and provide
// path to the specific config file instead of config path.
varConfigFile=cli.StringFlag{
Name:"config-file",
Usage:"path to the node configuration file (overrides --config-path option)",
}
// RelativePath is a flag for commands that use node configuration and provide
// a prefix to all relative paths in config files.
varRelativePath=cli.StringFlag{
Name:"relative-path",
Usage:"a prefix to all relative paths in the node configuration file",
}
// Debug is a flag for commands that allow node in debug mode usage.
varDebug=cli.BoolFlag{
Name:"debug, d",
Usage:"enable debug logging (LOTS of output, overrides configuration)",
}
varerrNoEndpoint=errors.New("no RPC endpoint specified, use option '--"+RPCEndpointFlag+"' or '-r'")
varerrInvalidHistoric=errors.New("invalid 'historic' parameter, neither a block number, nor a block/state hash")
varerrNoWallet=errors.New("no wallet parameter found, specify it with the '--wallet' or '-w' flag or specify wallet config file with the '--wallet-config' flag")
varerrConflictingWalletFlags=errors.New("--wallet flag conflicts with --wallet-config flag, please, provide one of them to specify wallet location")
// GetNetwork examines Context's flags and returns the appropriate network. It
checkError(t,"json: cannot unmarshal array into Go value of type manifest.Manifest","--manifest",manifestFile,"--hash",util.Uint160{}.StringLE())
checkError(t,"json: cannot unmarshal array into Go value of type manifest.Manifest","--manifest",manifestFile,"--hash",util.Uint160{}.StringLE(), "--out","zzz")
t.Run("not declared in manifest",func(t*testing.T){
check(t,filepath.Join("testdata","rpcbindings","invalid1"),"inconsistent usages of event `Non declared event`: not declared in the contract config")
})
t.Run("invalid number of params",func(t*testing.T){
check(t,filepath.Join("testdata","rpcbindings","invalid2"),"inconsistent usages of event `SomeEvent` against config: number of params mismatch: 2 vs 1")
})
/*
// TODO: this on is a controversial one. If event information is provided in the config file, then conversion code
// will be emitted by the compiler according to the parameter type provided via config. Thus, we can be sure that
// either event parameter has the type specified in the config file or the execution of the contract will fail.
// Thus, this testcase is always failing (no compilation error occures).
// Question: do we want to compare `RealType` of the emitted parameter with the one expected in the manifest?
t.Run("SC parameter type mismatch",func(t*testing.T){
check(t,filepath.Join("testdata","rpcbindings","invalid3"),"inconsistent usages of event `SomeEvent` against config: number of params mismatch: 2 vs 1")
check(t,filepath.Join("testdata","rpcbindings","invalid4"),"inconsistent usages of event `SomeEvent`: extended type of param #0 mismatch")
})
t.Run("named types redeclare",func(t*testing.T){
check(t,filepath.Join("testdata","rpcbindings","invalid5"),"configured declared named type intersects with the contract's one: `invalid5.NamedStruct`")
errNoInput=errors.New("no input file was found, specify an input file with the '--in or -i' flag")
errNoConfFile=errors.New("no config file was found, specify a config file with the '--config' or '-c' flag")
errNoManifestFile=errors.New("no manifest file was found, specify manifest file with '--manifest' or '-m' flag")
errNoMethod=errors.New("no method specified for function invocation command")
errNoWallet=errors.New("no wallet parameter found, specify it with the '--wallet' or '-w' flag or specify wallet config file with the '--wallet-config' flag")
errConflictingWalletFlags=errors.New("--wallet flag conflicts with --wallet-config flag, please, provide one of them to specify wallet location")
errNoScriptHash=errors.New("no smart contract hash was provided, specify one as the first argument")
errNoSmartContractName=errors.New("no name was provided, specify the '--name or -n' flag")
errFileExist=errors.New("A file with given smart-contract name already exists")
walletFlag=cli.StringFlag{
Name:"wallet, w",
Usage:"wallet to use to get the key for transaction signing; conflicts with --wallet-config flag",
}
walletConfigFlag=cli.StringFlag{
Name:"wallet-config",
Usage:"path to wallet config to use to get the key for transaction signing; conflicts with --wallet flag",
}
addressFlag=flags.AddressFlag{
errNoInput=errors.New("no input file was found, specify an input file with the '--in or -i' flag")
errNoConfFile=errors.New("no config file was found, specify a config file with the '--config' or '-c' flag")
errNoManifestFile=errors.New("no manifest file was found, specify manifest file with '--manifest' or '-m' flag")
errNoMethod=errors.New("no method specified for function invocation command")
errNoScriptHash=errors.New("no smart contract hash was provided, specify one as the first argument")
errNoSmartContractName=errors.New("no name was provided, specify the '--name or -n' flag")
errFileExist=errors.New("A file with given smart-contract name already exists")
addressFlag=flags.AddressFlag{
Name:addressFlagName,
Usage:"address to use as transaction signee (and gas source)",
}
gasFlag=flags.Fixed8Flag{
Name:"gas, g",
Usage:"network fee to add to the transaction (prioritizing it)",
}
sysGasFlag=flags.Fixed8Flag{
Name:"sysgas, e",
Usage:"system fee to add to transaction (compensating for execution)",
}
outFlag=cli.StringFlag{
Name:"out",
Usage:"file to put JSON transaction to",
}
forceFlag=cli.BoolFlag{
Name:"force",
Usage:"force-push the transaction in case of bad VM state after test script invocation",
}
)
// ModVersion contains `pkg/interop` module version
@ -98,7 +68,7 @@ func init() {
}
// RuntimeNotify sends runtime notification with "Hello world!" name