Commit graph

7204 commits

Author SHA1 Message Date
Anna Shaleva
f78f915071 native: optimize NEO's committee/validators cache handling
Do not recalculate new committee/validators value in the start of every
subsequent epoch. Use values that was calculated in the PostPersist method
of the previously processed block in the end of the previous epoch.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
27dddb4d50 native: refactor argument of NEO's getCommitteeMembers function
No funcional changes, just refactoring. It doesn't need the whole cache,
only the set of committee keys with votes.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
794658b54c native: rename NEO's cached validators list to newEpochNextValidators
Adjust all comments, make the field name match its meaning.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
312534af7c native: don't recalculate validators every block
Recalculate them once per epoch. Consensus is aware of it and must
call CalculateNextValidators exactly when needed.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
07e1bc7cd7 core: rename (*Blockchain).GetValidators to ComputeNextBlockValidators
We have two similar blockchain APIs: GetNextBlockValidators and GetValidators.
It's hard to distinguish them, thus renaming it to match the meaning, so what
we have now is:

GetNextBlockValidators literally just returns the top of the committee that
was elected in the start of batch of CommitteeSize blocks batch. It doesn't
change its valie every block.

ComputeNextBlockValidators literally computes the list of validators based on
the most fresh committee members information got from the NeoToken's storage
and based on the latest register/unregister/vote events. The list returned by
this method may be updated every block.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
96449d803a native: rework native NEO next block validators cache
Blockchain passes his own pure unwrapped DAO to
(*Blockchain).ComputeNextBlockValidators which means that native
RW NEO cache structure stored inside this DAO can be modified by
anyone who uses exported ComputeNextBlockValidators Blockchain API,
and technically it's valid, and we should allow this, because it's
the only purpose of `validators` caching. However, at the same time
some RPC server is allowed to request a subsequent wrapped DAO for
some test invocation. It means that descendant wrapped DAO
eventually will request RW NEO cache and try to `Copy()`
the underlying's DAO cache which is in direct use of
ComputeNextBlockValidators. Here's the race:
ComputeNextBlockValidators called by Consensus service tries to
update cached `validators` value, and descendant wrapped DAO
created by the  RPC server tries to copy DAO's native cache and
read the cached `validators` value.

So the problem is that native cache not designated to handle
concurrent access between parent DAO layer and derived (wrapped)
DAO layer. I've carefully reviewed all the usages of native cache,
and turns out that the described situation is the only place where
parent DAO is used directly to modify its cache concurrently with
some descendant DAO that is trying to access the cache. All other
usages of native cache (not only NEO, but also all other native
contrcts) strictly rely on the hierarchical DAO structure and don't
try to perform these concurrent operations between DAO layers.
There's also persist operation, but it keeps cache RW lock taken,
so it doesn't have this problem as far. Thus, in this commit we rework
NEO's `validators` cache value so that it always contain the relevant
list for upper Blockchain's DAO and is updated every PostPersist (if
needed).

Note: we must be very careful extending our native cache in the
future, every usage of native cache must be checked against the
described problem.

Close #2989.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Roman Khimov
19c59c3a59
Merge pull request #3149 from nspcc-dev/fix-compiler-version-race
internal: avoid race access to config.Version by tests
2023-10-10 12:23:08 +03:00
Anna Shaleva
eeb439f548 internal: avoid race access to config.Version by tests
Network server constructor reads config.Version variable, and
testcli.DeployContract writes dummy config.Version which causes
race in tests. Avoid this race by moving config.Version initialisation
to a separate package and perform it inside test packages init().

Close #3011, close #3017.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 11:59:53 +03:00
Roman Khimov
cef70091f6
Merge pull request #3148 from fyfyrchik/fix-dump
storage: Use timeout for boltdb database opening
2023-10-09 12:40:07 +03:00
Evgenii Stratonikov
f1c6b9fe8f storage: Use timeout for boltdb database opening
To dump the DB, the service must be stopped.
If this is not the case `dump` command just hangs without any output,
which _may_ be unexpected from the ops POV.
Introduce a 1 second timeous, which is more than enough, given
that bbolt retries doing flock() every 50ms.

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
2023-10-06 13:10:11 +03:00
Roman Khimov
b78bea17c0
Merge pull request #3147 from fyfyrchik/fix-big-uint64
compiler: Fix emitting big uint64 constants
2023-10-05 22:34:07 +03:00
Roman Khimov
32eb1e2606
Merge pull request #3146 from nspcc-dev/fix-link
zkpbinding: fix example link
2023-10-05 16:16:24 +03:00
Evgenii Stratonikov
96ee2e2b2d compiler: Fix emitting big uint64 constants
Currently we take int64 value from the Go parser and push it to the
stack. Using uint64 is not a common practice (usually we just use `int`),
but can be a problem while doing bit arithmetic and serializing numbers.

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
2023-10-05 16:03:57 +03:00
Anna Shaleva
d27d2a8561 zkpbinding: fix example link
Should be a part of #3043.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 15:16:11 +03:00
Roman Khimov
91c928e8d3
Merge pull request #3043 from nspcc-dev/groth16-verify-compat
smartcontract, examples: add API for Groth16 Verifier contract building and examples
2023-10-05 13:58:21 +03:00
Anna Shaleva
fc6e87af2f *: update interop deps
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:56 +03:00
Anna Shaleva
a2d28272ef native: fix error message on unexpected BLS12-381 curve point
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
b5a75b1485 scripts: add zkp examples to dependencies updator
And ignore on dependency checks, we update all the examples at once
anyway, so may safely skip the check for zkp folder.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
1745b21142 *: upgrade gnark-crypto up to v0.12.0
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
e2abb5cb7c zkpbinding: update templates of go.sum and go.mod for Verifier contract
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
ca71bd51d3 zkpbinding: format formulae in Verifier template
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
b7e019e7ef native: ensure proper endianness is used for CryptoLib's field element multiplier
Field element multiplier must be provided in the LE form, confirmed by
cross-node invocation: https://github.com/nspcc-dev/neo-go/pull/3043#issuecomment-1733424840.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
8c46517522 zkpbinding: use proper field size for BLS12-381
It's not differ from BN254, but we'd better use the proper package.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
ff260a6a9b zkpbinding: allow to handle serialisation format of gnark >= v0.9.0
An upgrade from gnark v0.8.X to v0.9.0 changes serialization format of verifying/proving keys
and proofs. In neo-go zkpbinding package we have to support both at least for now, because
gnark@v0.9.0 requires minimum go 1.19.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
03951c94b0 examples: add production CRS generation instructions for ZKP examples
Add an example and instructions for production CRS generation to ZKP examples
with gnark >= v0.9.0.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:45:58 +03:00
Anna Shaleva
9e74fc5b47 zkp: add end-to-end Groth-16 proof generation/verification example
The example shows that the proover knows the solution of the cubic
equation: y = x^3 + x + 5. The example is constructed for BLS12-381
curve points using Groth-16 prooving algorithm. The example includes
everything that developer needs to start using ZKP on the NEO platform
with Go SDK:
1. The described cubic circuit implementation.
2. The off-chain proof generation with the help of gnark-crypto library.
3. Go verification contract generation and deployment with the help of
   NeoGo libraries.
4. The on-chain proof verification for various sets of input data.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 12:32:47 +03:00
Anna Shaleva
0a3260c22c examples: add compatibility example for Groth16 veification
Port the C# contract provided in the
https://github.com/neo-project/neo/issues/2647#issuecomment-1129849870 and
add an integration test for it. Part of the #3002.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 11:01:25 +03:00
Roman Khimov
4598f3d3c2
Merge pull request #3141 from nspcc-dev/limit-calculatenetworkfee-gas
Limit calculatenetworkfee GAS
2023-09-28 09:13:27 +03:00
Roman Khimov
49a44b0b9d rpcsrv: use stricter GAS limit for calculatenetworkfee
Valid transactions can't use more than MaxVerificationGAS for script execution
and this applies to the whole set of signers, so use this value by default
unless local instance configuration suggests something lower for generic
invocations.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-27 19:33:06 +03:00
Roman Khimov
fd2774ea91 rpcsrv: return core.ErrVerificationFailed from calculatenetworkfee
We can ignore core.ErrInvalidSignature (which means that the script has
executed, but returned false), but we shouldn't ignore other errors which
likely mean that the script is incorrect (or hits some resource limits).

Use neorpc.ErrInvalidSignature as a return to separate this case from
contract-based verification.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-27 19:30:23 +03:00
Roman Khimov
3628b824e1
Merge pull request #3140 from nspcc-dev/fix-wildcard-trusts-deserialization
smartcontract: fix wildcard trusts deserialization
2023-09-25 12:01:20 +03:00
Anna Shaleva
2581146a01 smartcontract: fix wildcard trusts deserialization
If manifest trusts presented as Null stackitem, then they should be
treated as wildcard, not as restricted.

It's not the same problem as described and fixed in
https://github.com/neo-project/neo/pull/2901 and
https://github.com/neo-project/neo/pull/2892, although these
two PRs are related.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-09-25 11:53:47 +03:00
Roman Khimov
f66abd020d
Merge pull request #3136 from nspcc-dev/fix-mainnet-neofs-config
config: fix mainnet neofs config
2023-09-19 18:40:19 +03:00
Roman Khimov
fda0e8364f config: fix mainnet neofs config
For some reason it got out of sync with the reality, Notary is enabled since
April there.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-19 18:30:33 +03:00
Roman Khimov
eb571a1356
Merge pull request #3129 from nspcc-dev/neofs-sdk-up
oracle/neofs: upgrade to SDK RC11
2023-09-13 11:13:53 +03:00
Roman Khimov
a0f9743515 oracle/neofs: avoid panic on failed neofs requests
2023-09-13T06:04:07.114Z        WARN        failed to perform oracle request        {"url": "neofs:BE2c15AbYnKdcsVh77LisCtjifoNEJUekANo1yhQ211X/FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH", "error": "header: status: code = 3072 message = container not found"}
  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xde7c8f]
  goroutine 302 [running]:
  github.com/nspcc-dev/neofs-sdk-go/client.(*PayloadReader).close(0x0?, 0x0?)
          github.com/nspcc-dev/neofs-sdk-go@v1.0.0-rc.11/client/object_get.go:200 +0x2f
  github.com/nspcc-dev/neofs-sdk-go/client.(*PayloadReader).Close(0x0?)
          github.com/nspcc-dev/neofs-sdk-go@v1.0.0-rc.11/client/object_get.go:229 +0x1e
  github.com/nspcc-dev/neo-go/pkg/services/oracle/neofs.clientCloseWrapper.Close({{0x14b45f8?, 0x0?}, 0xc0010a1380?})
          github.com/nspcc-dev/neo-go/pkg/services/oracle/neofs/neofs.go:97 +0x3c
  github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).processRequest(0xc000255800, 0xc001dfa210, {0xb2d?, 0xc001ad8000?})
          github.com/nspcc-dev/neo-go/pkg/services/oracle/request.go:168 +0xee7
  github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).runRequestWorker(0xc000255800)
          github.com/nspcc-dev/neo-go/pkg/services/oracle/request.go:36 +0xe5
  created by github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).start
          github.com/nspcc-dev/neo-go/pkg/services/oracle/oracle.go:216 +0xae

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-13 10:50:42 +03:00
Roman Khimov
0f57ad4a12 oracle/neofs: upgrade to SDK RC11
It requires explicit signers, and we have a problem with the old code:

  2023-09-12T18:42:00.063Z        WARN        failed to perform oracle request        {"url": "neofs:FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH/5DCg4wUgWuWN3zsF4P4HdAzY2iKvXcrZ8QBLYGd1D1g2", "error": "failed to create client: incorrect signer: expected ECDSA_DETERMINISTIC_SHA256 scheme"}

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-13 10:50:42 +03:00
Roman Khimov
2d88ed9fe5
Merge pull request #3125 from nspcc-dev/rel-0.102.0
Release 0.102.0
2023-09-06 23:15:10 +03:00
Roman Khimov
31cc349725 ROADMAP: adjust wrt 0.102.0 release
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-06 22:08:26 +03:00
Roman Khimov
eb9735750c CHANGELOG: release 0.102.0
Co-Authored-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-06 22:08:26 +03:00
Roman Khimov
26c97a58ed
Merge pull request #3124 from nspcc-dev/hardforks
Set Basilisk hardforks for all networks
2023-09-06 16:54:29 +03:00
Roman Khimov
ee12520cbd config: set Basilisk fork height for NeoFS mainnet
Testnet doesn't need any (confirmed to have the same state for non-NEO
contracts) and can go with Basilisk from height 0.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-06 16:04:18 +03:00
Roman Khimov
a703e54563
Merge pull request #3122 from nspcc-dev/fix-neo-state-deserialization
state: always deserialize LastGasPerVote
2023-09-06 08:28:32 +03:00
Roman Khimov
9bdf66a5e0 state: always deserialize LastGasPerVote
It can be non-zero even if VoteTo is NULL. Fixes state diff with 3.6.0:

  block 41660: value mismatch for key +////xTrvgat3qG/w8hQoD/I4MgUz6rygA==: QQQhAS8hA7yiAAAhAA== vs QQQhAS8hA7yiAAAhB+POSWfBCAE=

Related to #2844.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 23:44:14 +03:00
Roman Khimov
98b946dcd3 config: set Basilisk hardfork heights for mainnet and testnet
Right from the https://github.com/neo-project/neo-node/pull/900

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 19:31:20 +03:00
Roman Khimov
c6850b49d9
Merge pull request #3121 from nspcc-dev/neotest-doc
neotest: improve doc based on #3120
2023-09-05 17:34:33 +03:00
Roman Khimov
4c015b30d5 neotest: improve doc based on #3120
We don't want anyone to have the same problem.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 15:20:00 +03:00
Roman Khimov
2f54f176fb
Merge pull request #3119 from nspcc-dev/neotest-doc
neotest: fix NewSingleSigner description, fix #3118
2023-09-05 13:04:48 +03:00
Roman Khimov
7cedcb9197 neotest: fix NewSingleSigner description, fix #3118
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-05 12:36:49 +03:00
Roman Khimov
e9d582f525
Merge pull request #2962 from nspcc-dev/notary-attr-upd
*: move NVB and Conflicts attributes out of extensions
2023-09-04 17:37:28 +03:00