Commit graph

1687 commits

Author SHA1 Message Date
Roman Khimov
01d15ff473
Merge pull request #2270 from nspcc-dev/vm-invoked-contracts
Add invoked contract tracing
2021-12-01 11:27:27 +03:00
Anna Shaleva
1d1538c566 services: fix Oracle response transaction creation
Problem: transactions with wrong hashes are accepted to the chain if
consensus nodes are designated as Oracle nodes. The result is wrong
MerkleRoot for the accepted block. Consensus nodes got such blocks
right from the dbft and store them without errors, but if
non-consensus nodes are present in the network, they just can't accept
these "bad" blocks:
```
2021-11-29T12:56:40.533+0300	WARN	blockQueue: failed adding block into the blockchain	{"error": "invalid block: MerkleRoot mismatch (expected a866b57ad637934f7a7700e3635a549387e644970b42681d865a54c3b3a46122, calculated d465aafabaf4539a3f619d373d178eeeeab7acb9847e746e398706c8c1582bf8)", "blockHeight": 17, "nextIndex": 18}
```

This problem happens because of transaction hash caching. We can't set
transaction hash if transaction construction wasn't yet completed.
2021-11-30 10:43:58 +03:00
Roman Khimov
33e37e60e5
Merge pull request #2264 from nspcc-dev/fix-win-tests
*: Windows compatibility fixes
2021-11-29 11:25:35 +03:00
AnnaShaleva
aefb6f9fee *: fix tests failing due to path.Join usage
Solution:
Use `file/filepath` package to construct expected path. This package is OS-aware, see https://github.com/golang/go/issues/30616.
2021-11-29 11:11:09 +03:00
AnnaShaleva
06beb4d534 core: fix TestMemCachedPersist test
Problem:
```
--- FAIL: TestMemCachedPersist (0.07s)
    --- FAIL: TestMemCachedPersist/BoltDBStore (0.07s)
        testing.go:894: TempDir RemoveAll cleanup: remove C:\Users\Anna\AppData\Local\Temp\TestMemCachedPersist_BoltDBStore294966711\001\test_bolt_db: The process cannot access the file because it is being used by another process.
```

Solution:
Release the resources occupied by the DB.
2021-11-26 18:26:27 +03:00
Evgeniy Stratonikov
fac595bbdf core: remove old storage items asynchronously
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:25:11 +03:00
Evgeniy Stratonikov
582d489c90 dao: add P2PStateExchangeExtensions setting to dao.Version
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:27 +03:00
Evgeniy Stratonikov
f1767f361d dao: add KeepOnlyLatestState setting to dao.Version
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:27 +03:00
Evgeniy Stratonikov
f7e2d3d717 dao: add stateroot-related settings to Version
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:27 +03:00
Evgeniy Stratonikov
856e9cf67b statesync: copy state by swapping prefix
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:27 +03:00
Evgeniy Stratonikov
6c5a7d9b29 dao: include settings in Version
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:26 +03:00
Evgeniy Stratonikov
856385b106 dao: use custom storage prefix
We use 2 prefixes for storing items because of state synchronization.
This commit allows to parametrize dao with the default prefix.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-24 16:22:07 +03:00
Roman Khimov
5fbe838fd4 core: add and use synchronous persist to avoid OOM
b9be892bf9 has made Persist asynchronous which
is very effective in allowing the system to continue processing
blocks/transactions while flushing things to disk. It at the same time is very
dangerous in that if the disk is slow and it takes much time to flush KV set
(more than persisting interval), there might be even bigger new KV set in
MemCachedStore by the time it finishes. Even if the system immediately starts
to flush this new data set it (being bigger) can take more time than the
previous one. And while doing so a new data set will appear in memory,
potentially again bigger than this.

So we can easily end up with the system going out of control, consuming more
and more memory and taking more and more time to persist a single set of
data. To avoid this we need to detect such condition and just wait for Persist
to really finish its job and release the resources.
2021-11-22 10:41:40 +03:00
Roman Khimov
83d0b2f465 interop: move invocation counter from VM to Context
It's created for interop and used by interop, VM doesn't care.
2021-11-20 21:57:41 +03:00
Roman Khimov
29cda5112a vm: simplify NEF loading by providing a special method
Offsets are still handled outside of VM to avoid introducing manifest
dependency (that's likely to be circular).
2021-11-20 21:57:41 +03:00
Roman Khimov
c53d978edd vm: drop ParamCount from Context
It's unused and it's useless.
2021-11-20 21:57:41 +03:00
Roman Khimov
765235eca3 vm: omit context parameter from Call()
It's not used and should not be used to switch from contract to contract.
2021-11-20 21:57:41 +03:00
Roman Khimov
7b713762be vm: do not increment invocation counter in Call
Everywhere it matters (and that's callExFromNative() now) it's incremented
already, so when we're doing Call() at the same time (and it's done to invoke
`_initialize` method) we're effectively double-incrementing it.
2021-11-20 21:57:41 +03:00
Roman Khimov
a577d40de9 vm: move Jump method to the Context
Because it's only relevant for execution context itself.
2021-11-20 21:57:35 +03:00
Roman Khimov
8c704a49b5 interop/contract: micro-optimize argument pushes
Calling Estack() again and again is not very productive.
2021-11-19 16:26:01 +03:00
Roman Khimov
d1f4928e29 contract: drop useless code from call processing
It's done already by LoadScriptWithCallingHash()
2021-11-19 16:22:30 +03:00
Roman Khimov
aa06770b3d
Merge pull request #2266 from nspcc-dev/nep-11-tracking
NEP-11 tracking
2021-11-19 13:07:37 +03:00
Roman Khimov
ce9d0b22cf *: use NEP-XX naming consistently in docs/comments
Standards are NEP-11 and NEP-17, not NEP11, not NEP17, not anything
else. Variable/function names of course can use whatever fits, but documents
and comments should be consistent wrt this.
2021-11-19 12:58:46 +03:00
Roman Khimov
b622c1934d core: only save token logs if there is something to save
Makes no sense storing empty ones.
2021-11-18 00:09:10 +03:00
Roman Khimov
f3278ea5b2 core: reuse the transfer log buffer
Make it a bit more efficient.
2021-11-18 00:09:10 +03:00
Roman Khimov
125e4231b0 core: store NEP-11 transfers, add accessor functions 2021-11-18 00:09:10 +03:00
Roman Khimov
7f40a0cfd8 state: prime token transfer info with the demand for new logs
Setting NewNEPXXBatch avoids the need to get them from the database which is
useful for newly tracked accounts.
2021-11-16 22:52:10 +03:00
Roman Khimov
095ed3f64e dao: drop unused AppendNEP17Transfer() 2021-11-16 22:51:04 +03:00
Roman Khimov
c63aeb38bb state: prepare for NEP-11 data tracking, refactor/rename
There is a lot of similarity, so try reusing common code and use more neutral
naming.
2021-11-16 19:18:06 +03:00
Roman Khimov
ecfe3e7247
Merge pull request #2260 from nspcc-dev/fix-notary-test-failure
Fix notary test failure
2021-11-16 16:09:41 +03:00
Roman Khimov
3b08b5b19b native: add NEP-11 tracking to management contract 2021-11-16 13:28:59 +03:00
Roman Khimov
f96d9d168e native/interop: use oracle request signers for oracle response witness
Oracle responses must use the same set of signers as oracle requests even
though the transaction itself is signed by oracle nodes/contract.

We can probably improve interop.Context by removing Tx field completely and
adding more functionality to Container, but it's not very convenient for
VerifyWitness and will require adding more stub-like methods for Block, so Tx
is used for now (and we do have it in every relevant case).
2021-11-16 11:51:15 +03:00
Roman Khimov
65016e8070 core: fix data race in notary test
I don't think it's possible with regular service functioning, but it happens
during testing because of pointer reuse:

WARNING: DATA RACE
Read at 0x00c003a0e3f0 by goroutine 114:
  github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).verifyIncompleteWitnesses()
      /home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:441 +0x1dc
  github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).OnNewRequest()
      /home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:188 +0x205
  github.com/nspcc-dev/neo-go/pkg/core.TestNotary.func11()
      /home/runner/work/neo-go/neo-go/pkg/core/notary_test.go:347 +0x612
  github.com/nspcc-dev/neo-go/pkg/core.TestNotary()
      /home/runner/work/neo-go/neo-go/pkg/core/notary_test.go:443 +0xe33
  testing.tRunner()
      /opt/hostedtoolcache/go/1.16.10/x64/src/testing/testing.go:1193 +0x202

Previous write at 0x00c003a0e3f0 by goroutine 104:
  github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).finalize()
      /home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:338 +0x50a
  github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).PostPersist()
      /home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:314 +0x297
  github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).Run()
      /home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:169 +0x4a7
2021-11-15 17:33:02 +03:00
Roman Khimov
f48dffe0e6 core: fix fallback VUB value in notary test
Serializing/deserializing the payload yields this:

 Error:          Received unexpected error:
                 both main and fallback transactions should have the same ValidUntil value
2021-11-15 17:32:14 +03:00
Roman Khimov
b31a8d750e
Merge pull request #2252 from nspcc-dev/packmap-packstruct
vm: add PACKMAP/PACKSTRUCT, extend UNPACK
2021-11-12 18:25:54 +03:00
Roman Khimov
9cdd360606
Merge pull request #2251 from nspcc-dev/rules-witness-scope
transaction: add new Rules witness scope
2021-11-12 18:25:39 +03:00
Roman Khimov
3e6ce3c221 vm: add PACKMAP/PACKSTRUCT, extend UNPACK
See neo-project/neo-vm#443.
2021-11-12 17:19:59 +03:00
Roman Khimov
9875799893 transaction: add new Rules witness scope
See neo-project/neo#2622. The implementation is somewhat asymmetric (and not
very efficient) for binary/JSON encoding/decoding, but it should be
sufficient.
2021-11-12 15:29:28 +03:00
Roman Khimov
0a7f8afcea
Merge pull request #2229 from nspcc-dev/contract-test
Generic contract testing infrastructure
2021-11-11 11:46:55 +03:00
Evgeniy Stratonikov
ce549e4cb2 core/test: allow to determine system fee automatically
Eventually this will be replaced by `pkg/neotest` invocations but for
now it allows us to remove NNS constants together with the tests.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-11 11:15:15 +03:00
Evgeniy Stratonikov
e3625152c6 core: move NNS test out of core
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-11-11 11:15:15 +03:00
Roman Khimov
6b8e615094
Merge pull request #2234 from nspcc-dev/rpc/params-parsing
rpc: method-specific parameters parsing optimisation
2021-11-10 20:45:44 +03:00
Anna Shaleva
1a32fcb2dc core: specify method not found call error
It's useful for debugging and external users.
2021-11-08 17:19:00 +03:00
Roman Khimov
328f8b7954
Merge pull request #2238 from nspcc-dev/fix-block-queue
network: remove priority queue from block queue
2021-11-01 12:03:45 +03:00
Roman Khimov
8bb1ecb45a network: remove priority queue from block queue
Use circular buffer which is a bit more appropriate. The problem is that
priority queue accepts and stores equal items which wastes memory even in
normal usage scenario, but it's especially dangerous if the node is stuck for
some reason. In this case it'll accept from peers and put into queue the same
blocks again and again leaking memory up to OOM condition.

Notice that queue length calculation might be wrong in case circular buffer
wraps, but it's not very likely to happen (usually blocks not coming from the
queue are added by consensus and it's not very fast in doing so).
2021-11-01 11:49:01 +03:00
Roman Khimov
67eac3a27f
Merge pull request #2196 from nspcc-dev/drop-badger-and-redis
Drop Badger and Redis
2021-10-28 16:08:49 +03:00
AnnaShaleva
3fed7b8eec core: handle empty MPT batch properly
It's OK to have it.
2021-10-27 18:44:39 +03:00
Roman Khimov
1144a03486 storage: drop RedisDB, close #2130 2021-10-27 17:32:25 +03:00
Roman Khimov
fb4b87bb96 storage: drop BadgerDB support, close #2130 2021-10-27 17:31:55 +03:00
AnnaShaleva
2d196b3f35 rpc: refactor calculatenetworkfee handler
Use (Blockchainer).VerifyWitness() to calculate network fee for
contract-based witnesses.
2021-10-25 19:07:25 +03:00