* Rename c.done channel to c.readerDone
* Introduce c.writerDone channel that is closed if the wsWriter's loop
is done
* Make the error returned by makeWsRequests verbose
Signed-off-by: Ayrtat <amper.meter1775@gmail.com>
We often use binary.PutUint*, but almost all these cases have preallocated
buffer of the size that matches exactly the desired one and use a single or
a couple of calls to PutUint*. Thus, I don't think that replacing
binary.PutUint* by AppendUint* will make things better for all these usages.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If it's the end of epoch, then it contains the updated validators list recalculated
during the last block's PostPersist. If it's middle of the epoch, then it contains
previously calculated value (value for the previous completed epoch) that is equal
to the current nextValidators cache value.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Do not recalculate new committee/validators value in the start of every
subsequent epoch. Use values that was calculated in the PostPersist method
of the previously processed block in the end of the previous epoch.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No funcional changes, just refactoring. It doesn't need the whole cache,
only the set of committee keys with votes.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Recalculate them once per epoch. Consensus is aware of it and must
call CalculateNextValidators exactly when needed.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
We have two similar blockchain APIs: GetNextBlockValidators and GetValidators.
It's hard to distinguish them, thus renaming it to match the meaning, so what
we have now is:
GetNextBlockValidators literally just returns the top of the committee that
was elected in the start of batch of CommitteeSize blocks batch. It doesn't
change its valie every block.
ComputeNextBlockValidators literally computes the list of validators based on
the most fresh committee members information got from the NeoToken's storage
and based on the latest register/unregister/vote events. The list returned by
this method may be updated every block.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Blockchain passes his own pure unwrapped DAO to
(*Blockchain).ComputeNextBlockValidators which means that native
RW NEO cache structure stored inside this DAO can be modified by
anyone who uses exported ComputeNextBlockValidators Blockchain API,
and technically it's valid, and we should allow this, because it's
the only purpose of `validators` caching. However, at the same time
some RPC server is allowed to request a subsequent wrapped DAO for
some test invocation. It means that descendant wrapped DAO
eventually will request RW NEO cache and try to `Copy()`
the underlying's DAO cache which is in direct use of
ComputeNextBlockValidators. Here's the race:
ComputeNextBlockValidators called by Consensus service tries to
update cached `validators` value, and descendant wrapped DAO
created by the RPC server tries to copy DAO's native cache and
read the cached `validators` value.
So the problem is that native cache not designated to handle
concurrent access between parent DAO layer and derived (wrapped)
DAO layer. I've carefully reviewed all the usages of native cache,
and turns out that the described situation is the only place where
parent DAO is used directly to modify its cache concurrently with
some descendant DAO that is trying to access the cache. All other
usages of native cache (not only NEO, but also all other native
contrcts) strictly rely on the hierarchical DAO structure and don't
try to perform these concurrent operations between DAO layers.
There's also persist operation, but it keeps cache RW lock taken,
so it doesn't have this problem as far. Thus, in this commit we rework
NEO's `validators` cache value so that it always contain the relevant
list for upper Blockchain's DAO and is updated every PostPersist (if
needed).
Note: we must be very careful extending our native cache in the
future, every usage of native cache must be checked against the
described problem.
Close#2989.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Network server constructor reads config.Version variable, and
testcli.DeployContract writes dummy config.Version which causes
race in tests. Avoid this race by moving config.Version initialisation
to a separate package and perform it inside test packages init().
Close#3011, close#3017.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
To dump the DB, the service must be stopped.
If this is not the case `dump` command just hangs without any output,
which _may_ be unexpected from the ops POV.
Introduce a 1 second timeous, which is more than enough, given
that bbolt retries doing flock() every 50ms.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
Currently we take int64 value from the Go parser and push it to the
stack. Using uint64 is not a common practice (usually we just use `int`),
but can be a problem while doing bit arithmetic and serializing numbers.
Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
An upgrade from gnark v0.8.X to v0.9.0 changes serialization format of verifying/proving keys
and proofs. In neo-go zkpbinding package we have to support both at least for now, because
gnark@v0.9.0 requires minimum go 1.19.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
The example shows that the proover knows the solution of the cubic
equation: y = x^3 + x + 5. The example is constructed for BLS12-381
curve points using Groth-16 prooving algorithm. The example includes
everything that developer needs to start using ZKP on the NEO platform
with Go SDK:
1. The described cubic circuit implementation.
2. The off-chain proof generation with the help of gnark-crypto library.
3. Go verification contract generation and deployment with the help of
NeoGo libraries.
4. The on-chain proof verification for various sets of input data.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Valid transactions can't use more than MaxVerificationGAS for script execution
and this applies to the whole set of signers, so use this value by default
unless local instance configuration suggests something lower for generic
invocations.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
We can ignore core.ErrInvalidSignature (which means that the script has
executed, but returned false), but we shouldn't ignore other errors which
likely mean that the script is incorrect (or hits some resource limits).
Use neorpc.ErrInvalidSignature as a return to separate this case from
contract-based verification.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
It requires explicit signers, and we have a problem with the old code:
2023-09-12T18:42:00.063Z WARN failed to perform oracle request {"url": "neofs:FMDZvqUCqcduZa8HD6wJNsHWrJ6sqkgBveGuYuL38pvH/5DCg4wUgWuWN3zsF4P4HdAzY2iKvXcrZ8QBLYGd1D1g2", "error": "failed to create client: incorrect signer: expected ECDSA_DETERMINISTIC_SHA256 scheme"}
Signed-off-by: Roman Khimov <roman@nspcc.ru>
It can be non-zero even if VoteTo is NULL. Fixes state diff with 3.6.0:
block 41660: value mismatch for key +////xTrvgat3qG/w8hQoD/I4MgUz6rygA==: QQQhAS8hA7yiAAAhAA== vs QQQhAS8hA7yiAAAhB+POSWfBCAE=
Related to #2844.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
If the parameter in the opcode dump is a 20-byte value, the converted values,
such as the address and the swapped endianness, have been added.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
GetRawNotaryTransaction returns a fallback or main transaction that was
previously added to the memory pool by P2PNotaryRequest. This function
invokes the RPC server's `getrawnotarytransaction` method.
GetRawNotaryPool returns hashes from all the verified transactions,
including both main and fallback transactions. This function invokes
the RPC server's `getrawnotarypool` method.
Also, these functions were added to doc.go.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
`getrawnotarytransaction` takes a transaction hash and attempts to find
the corresponding transaction in the notary requests mempool. It searches
through all the verified main and fallback transactions.
`getrawnotarypool` returns hashes of all the verified transactions,
including both main and fallback transactions.
Additionally add struct result.RawNotaryPool.
Close https://github.com/nspcc-dev/neo-go/issues/2951
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
IterateVerifiedTransactions iterates through verified transactions in
memory pool and invokes function cont. Where cont callback returns
whether we should continue with the traversal process.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
Do not retrieve the whole set of storage items when trying to find
the ones from the specified start. Use DAO's Seek interface
implemented over MPT TrieStore to retrieve only the necessary items.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Either non-pointer or pointer, both cases are disallowed to be generic.
Need to be reverted and properly handled within the scope of #2376.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
So that hardfork name was explicitly present in the test name. We'll
have a set of similar tests later.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No functional changes, just a refactoring.
Change error text to be able to use this error from external packages.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
Behaviour change.
`terminatesession` returns ErrUnknownSession in case of impossibility of finding session,
previously there was no-error response with `false` result.
`traverseIterator`returns ErrUnknownSession in case of impossibility of finding session,
previously there was no-error response with default result; `traverseIterator`returns ErrUnknownIterator,
there were no such errors before.
Accordingly to proposal:
https://github.com/neo-project/proposals/pull/156
Also adding description of `traverseIterator` in docs/rpc.md.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
This change makes code incompatible with C# node,
because currently no error is returned on invalid proof.
According to proposal:
https://github.com/neo-project/proposals/pull/156
Also adding `verifyProof` descpiption in docs/rpc.md.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
No functional changes, just a refactoring.
Use more specific and meaningful names to be able to use these errors from external packages.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
While our server no longer uses these codes (-100, -400) they still can come
from C# servers and while we consider them deprecated we better at least have
some definition of them until C# implements our proposal:
https://github.com/neo-project/proposals/pull/156
Also adding description of deprecated RPC error codes in ROADMAP.md.
Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
RPC binding config may be malformed or the source .go contract may contain
structures like this:
```
type Str struct {
Field int
field int
}
```
We need to recognise these cases and return error. otherwise the resulting
binding can't be compiled.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Perform private -> public transformation at the last step of RPC binding
generation so that it works not only with NeoGo contracts, but with any
other contracts.
Close#3083.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
And simplify atomic service value stored by RPC server. Oracle service can
either be an untyped nil or be the proper non-nil *oracle.Oracle.
Otherwise `submitoracleresponse` RPC handler doesn't work properly.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
52-bit precision is not enough for our 256-bit VM, but this value
matches the reference implementation, see the
https://github.com/neo-project/neo/issues/2879.
MaxIntegerPrec will be increased (or even removed) as soon as the
ref. issue is resolved.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Ensure that Scopes can be properly parsed not only from the string
representation, but also from a single byte. transaction.Signer
is not affected (checked against the C# implementation), only
RPC-related signer scopes are allowed to be unmarshalled from byte.
Close#3059.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This prevents the possible attack on notary request sender when
malicious partie is allowed to send notary request with main transaction
being someone else's fallback.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If the contract was deployed then cache must be initialized after
in-memory data reset. If the contract isn't active yet, then no
cache will be initialized on deploy (i.e. on call to Initialize()
method by native Management).
Close#2984.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Make the contracts cache initialization unified. The order of cache
iniitialization is not important and Nottary contract is added to the
bc.contracts.Contracts wrt P2PSigExtensions setting, thus no functional
changes, just refactoring for future applications.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Initialize Prometheus metrics on node start where appropriate and review
the usage of the following metrics:
```
anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ find | grep prometheus.go
./pkg/network/prometheus.go
./pkg/core/stateroot/prometheus.go
./pkg/core/prometheus.go
./pkg/services/rpcsrv/prometheus.go
./pkg/services/metrics/prometheus.go
```
Close#2970.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
The reader is about to exit and it will close legacy c.Notifications, but it
will leave subscription channels at the same time. This is wrong since these
channels will no longer receive any new events, game over.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
wsReader() closes c.done first and then goes over the list of
c.respChannels. Technically this means that any of the two can be taken in
this select.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Previously RPC server shutdown procedure listened to the execution
channel and stopped at the first element that arrived in the queue. This
could lead to the following problems:
* stopper could steal the execution result from subscriber
* stopper didn't wait for other subscription actions to complete
Add dedicated channel to `Server` for subscription routine. Close the
channel on `handleSubEvents` return and wait for signal in `Shutdown`.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Previously RPC server could never be shut down completely due to
some start precondition failure (in particular, inability to serve HTTP
on any configured endpoint). The problem was caused by next facts:
* start method ran subscription routine after HTTP init succeeded only
* stop method blocked waiting for the subscription routine to return
Run `handleSubEvents` routine on fresh `Start` unconditionally. With
this change, `Shutdown` method won't produce deadlock since
`handleSubEvents` closes wait channel.
Refs #2896.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
There is an existing problem with RPC server shutdown freeze after start
failure due to some init actions (at least HTTP listen) described in
#2896.
Add dedicated unit test which checks that `Shutdown` returns within 5s
after `Start` method encounters internal problems.
Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
I've carefully checked the way how new service can be added to the
Blockchain instance or to be removed from it. Current implemention
of SetNotary and SetOracle methods doesn't contain dangerous code,
and native contracts have atomic values everywhere where service
is stored.
Current implementation of Notary, Oracle and StateRoot services'
reload/disabling/enabling on SIGUSR1 is safe and doesn't require
any adjustment.
This commit closes#2944, it's not a bug in the code, it's just
stale documentation.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
52-bit precision is not enough for our 256-bit VM, but this value
matches the reference implementation, see the
https://github.com/neo-project/neo/issues/2879.
MaxIntegerPrec will be increased (or even removed) as soon as the
ref. issue is resolved.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
During new transaction verification if there's an on-chain conflicting
transaction, we should check the signers of this conflicting transaction.
If the signers intersect with signers of the incoming transaction, then
the conflict is treated as valid and verification for new incoming
transaction should fail. Otherwise, the conflict is treated as the
malicious attack attempt and will not be taken into account;
verification for the new incoming transaction should continue.
This commint implements the scheme described at
https://github.com/neo-project/neo/pull/2818#issuecomment-1632972055,
thanks to @shargon for digging.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Witnesses are not yet created by the moment we return this error,
thus, it was always 0 as an actual number of witnesses in
ErrInvalidWitnessNum.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
`(*Blockchain).HasTransaction` is one of the oldest methods in our
codebase, and currently it's completely unused. I also doubt that
this method works as expected because it returns `true` if transaction
in the mempool.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This prevents the possible attack on notary request sender when
malicious partie is allowed to send notary request with main transaction
being someone else's fallback.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Ensure that Scopes can be properly parsed not only from the string
representation, but also from a single byte. transaction.Signer
is not affected (checked against the C# implementation), only
RPC-related signer scopes are allowed to be unmarshalled from byte.
Close#3059.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This check is good and was present here since #1729, but it was
accidently removed from the reference implementation (see the
discussion in https://github.com/neo-project/neo/issues/2848). The
removal of this check from the C# node leaded to the T5 testnet state
diff since 1670095 heigh which causes inability to process new blocks
since 2272533 height (see #3049). This check was added back to the
C# node in https://github.com/neo-project/neo/pull/2849, but it is
planned to be the part of the upcoming 3.6.0 C# node release.
We need to keep our testnet healthy, thus, strict contract script
check will be temporary removed from the node code and is planned
to be added back to be a part of the next 3.6.0-compatible release.
Close#3049.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
This check is good and was present here since #1729, but it was
accidently removed from the reference implementation (see the
discussion in https://github.com/neo-project/neo/issues/2848). The
removal of this check from the C# node leaded to the T5 testnet state
diff since 1670095 heigh which causes inability to process new blocks
since 2272533 height (see #3049). This check was added back to the
C# node in https://github.com/neo-project/neo/pull/2849, but it is
planned to be the part of the upcoming 3.6.0 C# node release.
We need to keep our testnet healthy, thus, strict contract script
check will be temporary removed from the node code and is planned
to be added back to be a part of the next 3.6.0-compatible release.
Close#3049.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Our wrapping optimization relied on the caller context having a TRY block,
but each context (including internal calls!) has an exception handling stack
of its own, which means that for an invocation stack of
entry
A.someMethodFromEntry() # this one has a TRY
A.internalMethodViaCALL() # this one doesn't
B.someMethod()
we get `HasTryBlock() == false` for `A.internalMethodViaCALL()` context, which
leads to missing wrapper and missing rollbacks if B is to THROW. What this
patch does instead is it checks for any context within contract boundaries.
Fixes#3045.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Our wrapping optimization relied on the caller context having a TRY block,
but each context (including internal calls!) has an exception handling stack
of its own, which means that for an invocation stack of
entry
A.someMethodFromEntry() # this one has a TRY
A.internalMethodViaCALL() # this one doesn't
B.someMethod()
we get `HasTryBlock() == false` for `A.internalMethodViaCALL()` context, which
leads to missing wrapper and missing rollbacks if B is to THROW. What this
patch does instead is it checks for any context within contract boundaries.
Fixes#3045.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
