Commit graph

7167 commits

Author SHA1 Message Date
Anna Shaleva
32bfed4741 core: improve conflict records storage scheme
Implement the https://github.com/neo-project/neo/issues/2907#issuecomment-1722506014
and port a part of https://github.com/neo-project/neo/pull/2913.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 17:34:27 +03:00
Roman Khimov
fce98a0af8
Merge pull request #3153 from nspcc-dev/fix-codecov
.github: upgrade coverage uploading action
2023-10-10 14:30:30 +03:00
Roman Khimov
2f747dd8d9
Merge pull request #3150 from nspcc-dev/drop-deprecated
Drop deprecated things
2023-10-10 14:29:47 +03:00
Roman Khimov
8fcf1eecfa
Merge pull request #3110 from nspcc-dev/fix-tests
Fix race caused by design of native Neo validators/committee cache
2023-10-10 14:29:28 +03:00
Anna Shaleva
bb9b8368a8 .github: upgrade coverage uploading action
Currently it's failing, but no token is required for public repos:
```
[2023-10-10T10:35:11.917Z] ['error'] There was an error running the uploader: Error uploading to https://codecov.io: Error: There was an error fetching the storage URL during POST: 404 - {'detail': ErrorDetail(string='Unable to locate build via Github Actions API. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}
```

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:53:20 +03:00
Roman Khimov
460362ab2e config: drop deprecated Protocol configurations
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
9f7b19f886 config: drop implicit consensus config via UnlockWallet
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
6865848ca7 config: drop deprecated P2P configuration
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
de69560c7d config: drop deprecated address/port configurations
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
a59afa8ea3 config: drop obsolete SecondsPerBlock protocol configuration
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
eb995c8600 rpcclient: drop deprecated Notifications an Subscribe*
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Roman Khimov
413caac941 rpcclient: drop deprecated Client APIs
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-10-10 13:26:07 +03:00
Anna Shaleva
d9644204ee native: make newEpochNextValidators always contain non-empty value
If it's the end of epoch, then it contains the updated validators list recalculated
during the last block's PostPersist. If it's middle of the epoch, then it contains
previously calculated value (value for the previous completed epoch) that is equal
to the current nextValidators cache value.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
f78f915071 native: optimize NEO's committee/validators cache handling
Do not recalculate new committee/validators value in the start of every
subsequent epoch. Use values that was calculated in the PostPersist method
of the previously processed block in the end of the previous epoch.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
27dddb4d50 native: refactor argument of NEO's getCommitteeMembers function
No funcional changes, just refactoring. It doesn't need the whole cache,
only the set of committee keys with votes.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
794658b54c native: rename NEO's cached validators list to newEpochNextValidators
Adjust all comments, make the field name match its meaning.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
312534af7c native: don't recalculate validators every block
Recalculate them once per epoch. Consensus is aware of it and must
call CalculateNextValidators exactly when needed.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
07e1bc7cd7 core: rename (*Blockchain).GetValidators to ComputeNextBlockValidators
We have two similar blockchain APIs: GetNextBlockValidators and GetValidators.
It's hard to distinguish them, thus renaming it to match the meaning, so what
we have now is:

GetNextBlockValidators literally just returns the top of the committee that
was elected in the start of batch of CommitteeSize blocks batch. It doesn't
change its valie every block.

ComputeNextBlockValidators literally computes the list of validators based on
the most fresh committee members information got from the NeoToken's storage
and based on the latest register/unregister/vote events. The list returned by
this method may be updated every block.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
96449d803a native: rework native NEO next block validators cache
Blockchain passes his own pure unwrapped DAO to
(*Blockchain).ComputeNextBlockValidators which means that native
RW NEO cache structure stored inside this DAO can be modified by
anyone who uses exported ComputeNextBlockValidators Blockchain API,
and technically it's valid, and we should allow this, because it's
the only purpose of `validators` caching. However, at the same time
some RPC server is allowed to request a subsequent wrapped DAO for
some test invocation. It means that descendant wrapped DAO
eventually will request RW NEO cache and try to `Copy()`
the underlying's DAO cache which is in direct use of
ComputeNextBlockValidators. Here's the race:
ComputeNextBlockValidators called by Consensus service tries to
update cached `validators` value, and descendant wrapped DAO
created by the  RPC server tries to copy DAO's native cache and
read the cached `validators` value.

So the problem is that native cache not designated to handle
concurrent access between parent DAO layer and derived (wrapped)
DAO layer. I've carefully reviewed all the usages of native cache,
and turns out that the described situation is the only place where
parent DAO is used directly to modify its cache concurrently with
some descendant DAO that is trying to access the cache. All other
usages of native cache (not only NEO, but also all other native
contrcts) strictly rely on the hierarchical DAO structure and don't
try to perform these concurrent operations between DAO layers.
There's also persist operation, but it keeps cache RW lock taken,
so it doesn't have this problem as far. Thus, in this commit we rework
NEO's `validators` cache value so that it always contain the relevant
list for upper Blockchain's DAO and is updated every PostPersist (if
needed).

Note: we must be very careful extending our native cache in the
future, every usage of native cache must be checked against the
described problem.

Close #2989.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Roman Khimov
19c59c3a59
Merge pull request #3149 from nspcc-dev/fix-compiler-version-race
internal: avoid race access to config.Version by tests
2023-10-10 12:23:08 +03:00
Anna Shaleva
eeb439f548 internal: avoid race access to config.Version by tests
Network server constructor reads config.Version variable, and
testcli.DeployContract writes dummy config.Version which causes
race in tests. Avoid this race by moving config.Version initialisation
to a separate package and perform it inside test packages init().

Close #3011, close #3017.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 11:59:53 +03:00
Roman Khimov
cef70091f6
Merge pull request #3148 from fyfyrchik/fix-dump
storage: Use timeout for boltdb database opening
2023-10-09 12:40:07 +03:00
Evgenii Stratonikov
f1c6b9fe8f storage: Use timeout for boltdb database opening
To dump the DB, the service must be stopped.
If this is not the case `dump` command just hangs without any output,
which _may_ be unexpected from the ops POV.
Introduce a 1 second timeous, which is more than enough, given
that bbolt retries doing flock() every 50ms.

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
2023-10-06 13:10:11 +03:00
Roman Khimov
b78bea17c0
Merge pull request #3147 from fyfyrchik/fix-big-uint64
compiler: Fix emitting big uint64 constants
2023-10-05 22:34:07 +03:00
Roman Khimov
32eb1e2606
Merge pull request #3146 from nspcc-dev/fix-link
zkpbinding: fix example link
2023-10-05 16:16:24 +03:00
Evgenii Stratonikov
96ee2e2b2d compiler: Fix emitting big uint64 constants
Currently we take int64 value from the Go parser and push it to the
stack. Using uint64 is not a common practice (usually we just use `int`),
but can be a problem while doing bit arithmetic and serializing numbers.

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
2023-10-05 16:03:57 +03:00
Anna Shaleva
d27d2a8561 zkpbinding: fix example link
Should be a part of #3043.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 15:16:11 +03:00
Roman Khimov
91c928e8d3
Merge pull request #3043 from nspcc-dev/groth16-verify-compat
smartcontract, examples: add API for Groth16 Verifier contract building and examples
2023-10-05 13:58:21 +03:00
Anna Shaleva
fc6e87af2f *: update interop deps
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:56 +03:00
Anna Shaleva
a2d28272ef native: fix error message on unexpected BLS12-381 curve point
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
b5a75b1485 scripts: add zkp examples to dependencies updator
And ignore on dependency checks, we update all the examples at once
anyway, so may safely skip the check for zkp folder.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
1745b21142 *: upgrade gnark-crypto up to v0.12.0
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
e2abb5cb7c zkpbinding: update templates of go.sum and go.mod for Verifier contract
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
ca71bd51d3 zkpbinding: format formulae in Verifier template
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
b7e019e7ef native: ensure proper endianness is used for CryptoLib's field element multiplier
Field element multiplier must be provided in the LE form, confirmed by
cross-node invocation: https://github.com/nspcc-dev/neo-go/pull/3043#issuecomment-1733424840.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
8c46517522 zkpbinding: use proper field size for BLS12-381
It's not differ from BN254, but we'd better use the proper package.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
ff260a6a9b zkpbinding: allow to handle serialisation format of gnark >= v0.9.0
An upgrade from gnark v0.8.X to v0.9.0 changes serialization format of verifying/proving keys
and proofs. In neo-go zkpbinding package we have to support both at least for now, because
gnark@v0.9.0 requires minimum go 1.19.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:46:05 +03:00
Anna Shaleva
03951c94b0 examples: add production CRS generation instructions for ZKP examples
Add an example and instructions for production CRS generation to ZKP examples
with gnark >= v0.9.0.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 13:45:58 +03:00
Anna Shaleva
9e74fc5b47 zkp: add end-to-end Groth-16 proof generation/verification example
The example shows that the proover knows the solution of the cubic
equation: y = x^3 + x + 5. The example is constructed for BLS12-381
curve points using Groth-16 prooving algorithm. The example includes
everything that developer needs to start using ZKP on the NEO platform
with Go SDK:
1. The described cubic circuit implementation.
2. The off-chain proof generation with the help of gnark-crypto library.
3. Go verification contract generation and deployment with the help of
   NeoGo libraries.
4. The on-chain proof verification for various sets of input data.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 12:32:47 +03:00
Anna Shaleva
0a3260c22c examples: add compatibility example for Groth16 veification
Port the C# contract provided in the
https://github.com/neo-project/neo/issues/2647#issuecomment-1129849870 and
add an integration test for it. Part of the #3002.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-05 11:01:25 +03:00
Roman Khimov
4598f3d3c2
Merge pull request #3141 from nspcc-dev/limit-calculatenetworkfee-gas
Limit calculatenetworkfee GAS
2023-09-28 09:13:27 +03:00
Roman Khimov
49a44b0b9d rpcsrv: use stricter GAS limit for calculatenetworkfee
Valid transactions can't use more than MaxVerificationGAS for script execution
and this applies to the whole set of signers, so use this value by default
unless local instance configuration suggests something lower for generic
invocations.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-27 19:33:06 +03:00
Roman Khimov
fd2774ea91 rpcsrv: return core.ErrVerificationFailed from calculatenetworkfee
We can ignore core.ErrInvalidSignature (which means that the script has
executed, but returned false), but we shouldn't ignore other errors which
likely mean that the script is incorrect (or hits some resource limits).

Use neorpc.ErrInvalidSignature as a return to separate this case from
contract-based verification.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-27 19:30:23 +03:00
Roman Khimov
3628b824e1
Merge pull request #3140 from nspcc-dev/fix-wildcard-trusts-deserialization
smartcontract: fix wildcard trusts deserialization
2023-09-25 12:01:20 +03:00
Anna Shaleva
2581146a01 smartcontract: fix wildcard trusts deserialization
If manifest trusts presented as Null stackitem, then they should be
treated as wildcard, not as restricted.

It's not the same problem as described and fixed in
https://github.com/neo-project/neo/pull/2901 and
https://github.com/neo-project/neo/pull/2892, although these
two PRs are related.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-09-25 11:53:47 +03:00
Roman Khimov
f66abd020d
Merge pull request #3136 from nspcc-dev/fix-mainnet-neofs-config
config: fix mainnet neofs config
2023-09-19 18:40:19 +03:00
Roman Khimov
fda0e8364f config: fix mainnet neofs config
For some reason it got out of sync with the reality, Notary is enabled since
April there.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-19 18:30:33 +03:00
Roman Khimov
eb571a1356
Merge pull request #3129 from nspcc-dev/neofs-sdk-up
oracle/neofs: upgrade to SDK RC11
2023-09-13 11:13:53 +03:00
Roman Khimov
a0f9743515 oracle/neofs: avoid panic on failed neofs requests
2023-09-13T06:04:07.114Z        WARN        failed to perform oracle request        {"url": "neofs:BE2c15AbYnKdcsVh77LisCtjifoNEJUekANo1yhQ211X/FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH", "error": "header: status: code = 3072 message = container not found"}
  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xde7c8f]
  goroutine 302 [running]:
  github.com/nspcc-dev/neofs-sdk-go/client.(*PayloadReader).close(0x0?, 0x0?)
          github.com/nspcc-dev/neofs-sdk-go@v1.0.0-rc.11/client/object_get.go:200 +0x2f
  github.com/nspcc-dev/neofs-sdk-go/client.(*PayloadReader).Close(0x0?)
          github.com/nspcc-dev/neofs-sdk-go@v1.0.0-rc.11/client/object_get.go:229 +0x1e
  github.com/nspcc-dev/neo-go/pkg/services/oracle/neofs.clientCloseWrapper.Close({{0x14b45f8?, 0x0?}, 0xc0010a1380?})
          github.com/nspcc-dev/neo-go/pkg/services/oracle/neofs/neofs.go:97 +0x3c
  github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).processRequest(0xc000255800, 0xc001dfa210, {0xb2d?, 0xc001ad8000?})
          github.com/nspcc-dev/neo-go/pkg/services/oracle/request.go:168 +0xee7
  github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).runRequestWorker(0xc000255800)
          github.com/nspcc-dev/neo-go/pkg/services/oracle/request.go:36 +0xe5
  created by github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).start
          github.com/nspcc-dev/neo-go/pkg/services/oracle/oracle.go:216 +0xae

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-13 10:50:42 +03:00
Roman Khimov
0f57ad4a12 oracle/neofs: upgrade to SDK RC11
It requires explicit signers, and we have a problem with the old code:

  2023-09-12T18:42:00.063Z        WARN        failed to perform oracle request        {"url": "neofs:FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH/5DCg4wUgWuWN3zsF4P4HdAzY2iKvXcrZ8QBLYGd1D1g2", "error": "failed to create client: incorrect signer: expected ECDSA_DETERMINISTIC_SHA256 scheme"}

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-09-13 10:50:42 +03:00