Commit graph

5114 commits

Author SHA1 Message Date
Anna Shaleva
d9644204ee native: make newEpochNextValidators always contain non-empty value
If it's the end of epoch, then it contains the updated validators list recalculated
during the last block's PostPersist. If it's middle of the epoch, then it contains
previously calculated value (value for the previous completed epoch) that is equal
to the current nextValidators cache value.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
f78f915071 native: optimize NEO's committee/validators cache handling
Do not recalculate new committee/validators value in the start of every
subsequent epoch. Use values that was calculated in the PostPersist method
of the previously processed block in the end of the previous epoch.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
27dddb4d50 native: refactor argument of NEO's getCommitteeMembers function
No funcional changes, just refactoring. It doesn't need the whole cache,
only the set of committee keys with votes.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
794658b54c native: rename NEO's cached validators list to newEpochNextValidators
Adjust all comments, make the field name match its meaning.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
312534af7c native: don't recalculate validators every block
Recalculate them once per epoch. Consensus is aware of it and must
call CalculateNextValidators exactly when needed.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
07e1bc7cd7 core: rename (*Blockchain).GetValidators to ComputeNextBlockValidators
We have two similar blockchain APIs: GetNextBlockValidators and GetValidators.
It's hard to distinguish them, thus renaming it to match the meaning, so what
we have now is:

GetNextBlockValidators literally just returns the top of the committee that
was elected in the start of batch of CommitteeSize blocks batch. It doesn't
change its valie every block.

ComputeNextBlockValidators literally computes the list of validators based on
the most fresh committee members information got from the NeoToken's storage
and based on the latest register/unregister/vote events. The list returned by
this method may be updated every block.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
96449d803a native: rework native NEO next block validators cache
Blockchain passes his own pure unwrapped DAO to
(*Blockchain).ComputeNextBlockValidators which means that native
RW NEO cache structure stored inside this DAO can be modified by
anyone who uses exported ComputeNextBlockValidators Blockchain API,
and technically it's valid, and we should allow this, because it's
the only purpose of `validators` caching. However, at the same time
some RPC server is allowed to request a subsequent wrapped DAO for
some test invocation. It means that descendant wrapped DAO
eventually will request RW NEO cache and try to `Copy()`
the underlying's DAO cache which is in direct use of
ComputeNextBlockValidators. Here's the race:
ComputeNextBlockValidators called by Consensus service tries to
update cached `validators` value, and descendant wrapped DAO
created by the  RPC server tries to copy DAO's native cache and
read the cached `validators` value.

So the problem is that native cache not designated to handle
concurrent access between parent DAO layer and derived (wrapped)
DAO layer. I've carefully reviewed all the usages of native cache,
and turns out that the described situation is the only place where
parent DAO is used directly to modify its cache concurrently with
some descendant DAO that is trying to access the cache. All other
usages of native cache (not only NEO, but also all other native
contrcts) strictly rely on the hierarchical DAO structure and don't
try to perform these concurrent operations between DAO layers.
There's also persist operation, but it keeps cache RW lock taken,
so it doesn't have this problem as far. Thus, in this commit we rework
NEO's `validators` cache value so that it always contain the relevant
list for upper Blockchain's DAO and is updated every PostPersist (if
needed).

Note: we must be very careful extending our native cache in the
future, every usage of native cache must be checked against the
described problem.

Close #2989.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Roman Khimov
19c59c3a59
Merge pull request #3149 from nspcc-dev/fix-compiler-version-race
internal: avoid race access to config.Version by tests
2023-10-10 12:23:08 +03:00
Anna Shaleva
eeb439f548 internal: avoid race access to config.Version by tests
Network server constructor reads config.Version variable, and
testcli.DeployContract writes dummy config.Version which causes
race in tests. Avoid this race by moving config.Version initialisation
to a separate package and perform it inside test packages init().

Close #3011, close #3017.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 11:59:53 +03:00
Roman Khimov
cef70091f6
Merge pull request #3148 from fyfyrchik/fix-dump
storage: Use timeout for boltdb database opening
2023-10-09 12:40:07 +03:00
Evgenii Stratonikov
f1c6b9fe8f storage: Use timeout for boltdb database opening
To dump the DB, the service must be stopped.
If this is not the case `dump` command just hangs without any output,
which _may_ be unexpected from the ops POV.
Introduce a 1 second timeous, which is more than enough, given
that bbolt retries doing flock() every 50ms.

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
2023-10-06 13:10:11 +03:00
Roman Khimov
b78bea17c0
Merge pull request #3147 from fyfyrchik/fix-big-uint64
compiler: Fix emitting big uint64 constants
2023-10-05 22:34:07 +03:00
Evgenii Stratonikov
96ee2e2b2d compiler: Fix emitting big uint64 constants
Currently we take int64 value from the Go parser and push it to the
stack. Using uint64 is not a common practice (usually we just use `int`),
but can be a problem while doing bit arithmetic and serializing numbers.

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
2023-10-05 16:03:57 +03:00
Anna Shaleva
d27d2a8561 zkpbinding: fix example link
Should be a part of #3043.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 15:16:11 +03:00
Anna Shaleva
a2d28272ef native: fix error message on unexpected BLS12-381 curve point
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
e2abb5cb7c zkpbinding: update templates of go.sum and go.mod for Verifier contract
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
ca71bd51d3 zkpbinding: format formulae in Verifier template
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
b7e019e7ef native: ensure proper endianness is used for CryptoLib's field element multiplier
Field element multiplier must be provided in the LE form, confirmed by
cross-node invocation: https://github.com/nspcc-dev/neo-go/pull/3043#issuecomment-1733424840.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
8c46517522 zkpbinding: use proper field size for BLS12-381
It's not differ from BN254, but we'd better use the proper package.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
ff260a6a9b zkpbinding: allow to handle serialisation format of gnark >= v0.9.0
An upgrade from gnark v0.8.X to v0.9.0 changes serialization format of verifying/proving keys
and proofs. In neo-go zkpbinding package we have to support both at least for now, because
gnark@v0.9.0 requires minimum go 1.19.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
9e74fc5b47 zkp: add end-to-end Groth-16 proof generation/verification example
The example shows that the proover knows the solution of the cubic
equation: y = x^3 + x + 5. The example is constructed for BLS12-381
curve points using Groth-16 prooving algorithm. The example includes
everything that developer needs to start using ZKP on the NEO platform
with Go SDK:
1. The described cubic circuit implementation.
2. The off-chain proof generation with the help of gnark-crypto library.
3. Go verification contract generation and deployment with the help of
   NeoGo libraries.
4. The on-chain proof verification for various sets of input data.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 12:32:47 +03:00
Anna Shaleva
0a3260c22c examples: add compatibility example for Groth16 veification
Port the C# contract provided in the
https://github.com/neo-project/neo/issues/2647#issuecomment-1129849870 and
add an integration test for it. Part of the #3002.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 11:01:25 +03:00
Roman Khimov
49a44b0b9d rpcsrv: use stricter GAS limit for calculatenetworkfee
Valid transactions can't use more than MaxVerificationGAS for script execution
and this applies to the whole set of signers, so use this value by default
unless local instance configuration suggests something lower for generic
invocations.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-27 19:33:06 +03:00
Roman Khimov
fd2774ea91 rpcsrv: return core.ErrVerificationFailed from calculatenetworkfee
We can ignore core.ErrInvalidSignature (which means that the script has
executed, but returned false), but we shouldn't ignore other errors which
likely mean that the script is incorrect (or hits some resource limits).

Use neorpc.ErrInvalidSignature as a return to separate this case from
contract-based verification.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-27 19:30:23 +03:00
Anna Shaleva
2581146a01 smartcontract: fix wildcard trusts deserialization
If manifest trusts presented as Null stackitem, then they should be
treated as wildcard, not as restricted.

It's not the same problem as described and fixed in
https://github.com/neo-project/neo/pull/2901 and
https://github.com/neo-project/neo/pull/2892, although these
two PRs are related.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-09-25 11:53:47 +03:00
Roman Khimov
a0f9743515 oracle/neofs: avoid panic on failed neofs requests
2023-09-13T06:04:07.114Z        WARN        failed to perform oracle request        {"url": "neofs:BE2c15AbYnKdcsVh77LisCtjifoNEJUekANo1yhQ211X/FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH", "error": "header: status: code = 3072 message = container not found"}
  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xde7c8f]
  goroutine 302 [running]:
  github.com/nspcc-dev/neofs-sdk-go/client.(*PayloadReader).close(0x0?, 0x0?)
          github.com/nspcc-dev/neofs-sdk-go@v1.0.0-rc.11/client/object_get.go:200 +0x2f
  github.com/nspcc-dev/neofs-sdk-go/client.(*PayloadReader).Close(0x0?)
          github.com/nspcc-dev/neofs-sdk-go@v1.0.0-rc.11/client/object_get.go:229 +0x1e
  github.com/nspcc-dev/neo-go/pkg/services/oracle/neofs.clientCloseWrapper.Close({{0x14b45f8?, 0x0?}, 0xc0010a1380?})
          github.com/nspcc-dev/neo-go/pkg/services/oracle/neofs/neofs.go:97 +0x3c
  github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).processRequest(0xc000255800, 0xc001dfa210, {0xb2d?, 0xc001ad8000?})
          github.com/nspcc-dev/neo-go/pkg/services/oracle/request.go:168 +0xee7
  github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).runRequestWorker(0xc000255800)
          github.com/nspcc-dev/neo-go/pkg/services/oracle/request.go:36 +0xe5
  created by github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).start
          github.com/nspcc-dev/neo-go/pkg/services/oracle/oracle.go:216 +0xae

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-13 10:50:42 +03:00
Roman Khimov
0f57ad4a12 oracle/neofs: upgrade to SDK RC11
It requires explicit signers, and we have a problem with the old code:

  2023-09-12T18:42:00.063Z        WARN        failed to perform oracle request        {"url": "neofs:FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH/5DCg4wUgWuWN3zsF4P4HdAzY2iKvXcrZ8QBLYGd1D1g2", "error": "failed to create client: incorrect signer: expected ECDSA_DETERMINISTIC_SHA256 scheme"}

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-13 10:50:42 +03:00
Roman Khimov
9bdf66a5e0 state: always deserialize LastGasPerVote
It can be non-zero even if VoteTo is NULL. Fixes state diff with 3.6.0:

  block 41660: value mismatch for key +////xTrvgat3qG/w8hQoD/I4MgUz6rygA==: QQQhAS8hA7yiAAAhAA== vs QQQhAS8hA7yiAAAhB+POSWfBCAE=

Related to #2844.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 23:44:14 +03:00
Roman Khimov
4c015b30d5 neotest: improve doc based on #3120
We don't want anyone to have the same problem.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 15:20:00 +03:00
Roman Khimov
7cedcb9197 neotest: fix NewSingleSigner description, fix #3118
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 12:36:49 +03:00
Roman Khimov
fff7e91709 dao: simplify NewSimple()
We no longer need P2PSigExtension flag here, conflicts attribute is a part
of the normal protocol.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-04 16:56:59 +03:00
Anna Shaleva
f3c1283ac6 *: move NVB and Conflicts attributes out of extensions
They're a part of the regular protocol now.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-04 16:39:44 +03:00
Tatiana Nesterenko
259cbc3356 vm: add address&swap endianness to opcode dump for hashes
If the parameter in the opcode dump is a 20-byte value, the converted values,
such as the address and the swapped endianness, have been added.

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
2023-09-04 08:41:22 +01:00
Roman Khimov
5180305240
Merge pull request #3117 from nspcc-dev/3070-unwrap-nothing
Add `unwrap.Nothing` function
2023-09-04 08:49:14 +03:00
Tatiana Nesterenko
59f72429b4 dbconfig: fix DBConfiguration description
The list of three supported types (`Type`) in the `DBConfiguration` struct
has been added.

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
2023-09-03 18:02:38 +01:00
Tatiana Nesterenko
acd821948f unwrap: add Nothing function
The `Nothing` function expects zero stack items and a successful invocation
(HALT state).

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
2023-09-03 16:54:50 +01:00
Tatiana Nesterenko
d06f135792 rpcclient: support getrawnotarytransaction and getrawnotarypool RPC methods
GetRawNotaryTransaction returns a fallback or main transaction that was
previously added to the memory pool by P2PNotaryRequest. This function
invokes the RPC server's `getrawnotarytransaction` method.
GetRawNotaryPool returns hashes from all the verified transactions,
including both main and fallback transactions. This function invokes
the RPC server's `getrawnotarypool` method.

Also, these functions were added to doc.go.

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
2023-08-31 18:51:43 +01:00
Tatiana Nesterenko
9e31e42bd9 rpcsrv: add getrawnotarypool, getrawnotarytransaction handlers
`getrawnotarytransaction` takes a transaction hash and attempts to find
the corresponding transaction in the notary requests mempool. It searches
through all the verified main and fallback transactions.
`getrawnotarypool` returns hashes of all the verified transactions,
including both main and fallback transactions.

Additionally add struct result.RawNotaryPool.

Close https://github.com/nspcc-dev/neo-go/issues/2951

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
2023-08-31 18:51:43 +01:00
Tatiana Nesterenko
d285342d54 core: add function IterateVerifiedTransactions
IterateVerifiedTransactions iterates through verified transactions in
memory pool and invokes function cont. Where cont callback returns
whether we should continue with the traversal process.

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
2023-08-31 18:51:43 +01:00
Roman Khimov
5463a91edc
Merge pull request #3111 from nspcc-dev/rpcsrv-drop-unused-iface
rpcsrv: drop unused method from the Ledger interface
2023-08-30 10:55:23 +03:00
Roman Khimov
4a648692a2 rpcsrv: drop unused method from the Ledger interface
Inspired by #3110.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-08-29 22:33:26 +03:00
Roman Khimov
ad24aad9c0
Merge pull request #3108 from nspcc-dev/create-fallback-copy
notary: avoid changing pooled fallback transaction witnesses
2023-08-29 16:16:24 +03:00
Anna Shaleva
c63289a564 notary: avoid changing pooled fallback transaction witnesses
Forbid Notary service to change the fallback's witnesses in any way.
Fix the problem described in review comment:
https://github.com/nspcc-dev/neo-go/pull/3098#discussion_r1308336339.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-29 16:06:40 +03:00
Anna Shaleva
b89078c42a rpcsrv: set MaxNEP11Tokens to default if not specified
Do not use RPC configuration constructor for this, some external services
may skip this part.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-29 13:26:10 +03:00
Anna Shaleva
f5b0489d74 rpcsrv: set MaxFindStorageResultItems to default if not specified
Do not use RPC configuration constructor for this, some external services
may skip this part.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-29 13:26:10 +03:00
Anna Shaleva
dd7c762ff9 rpcsrv: set MaxFindResultItems to default if not specified
Do not use RPC configuration constructor for this, some external services
may skip this part.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-29 13:26:10 +03:00
Anna Shaleva
aeb7ee1021 rpcsrv: improve error formatting
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-29 13:26:05 +03:00
Anna Shaleva
97a57de82d rpcsrv: set MaxIteratorResultItems to default if not specified
Do not use RPC configuration constructor for this, some external services
may skip this part.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-29 13:19:39 +03:00
Anna Shaleva
5d3938ae23 core: adjust hardfork enabling logic
Follow the logic described in https://github.com/neo-project/neo/pull/2886#issuecomment-1674745298
and port the https://github.com/neo-project/neo/pull/2886.

Close #3096.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-25 18:24:15 +03:00
Anna Shaleva
7b64b693bd rpcsrv: refactor findstoragehistoric handler to avoid DoS attack
Do not retrieve the whole set of storage items when trying to find
the ones from the specified start. Use DAO's Seek interface
implemented over MPT TrieStore to retrieve only the necessary items.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-08-24 17:15:59 +03:00