Commit graph

147 commits

Author SHA1 Message Date
Anna Shaleva
dc3d1300dd *: replace go.uber.org/atomic with sync/atomic
Use sync/atomic everywhere and exclude go.uber.org/atomic from go.mod.
Close .

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-12 17:59:34 +03:00
Anna Shaleva
d9644204ee native: make newEpochNextValidators always contain non-empty value
If it's the end of epoch, then it contains the updated validators list recalculated
during the last block's PostPersist. If it's middle of the epoch, then it contains
previously calculated value (value for the previous completed epoch) that is equal
to the current nextValidators cache value.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
312534af7c native: don't recalculate validators every block
Recalculate them once per epoch. Consensus is aware of it and must
call CalculateNextValidators exactly when needed.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
07e1bc7cd7 core: rename (*Blockchain).GetValidators to ComputeNextBlockValidators
We have two similar blockchain APIs: GetNextBlockValidators and GetValidators.
It's hard to distinguish them, thus renaming it to match the meaning, so what
we have now is:

GetNextBlockValidators literally just returns the top of the committee that
was elected in the start of batch of CommitteeSize blocks batch. It doesn't
change its valie every block.

ComputeNextBlockValidators literally computes the list of validators based on
the most fresh committee members information got from the NeoToken's storage
and based on the latest register/unregister/vote events. The list returned by
this method may be updated every block.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Anna Shaleva
96449d803a native: rework native NEO next block validators cache
Blockchain passes his own pure unwrapped DAO to
(*Blockchain).ComputeNextBlockValidators which means that native
RW NEO cache structure stored inside this DAO can be modified by
anyone who uses exported ComputeNextBlockValidators Blockchain API,
and technically it's valid, and we should allow this, because it's
the only purpose of `validators` caching. However, at the same time
some RPC server is allowed to request a subsequent wrapped DAO for
some test invocation. It means that descendant wrapped DAO
eventually will request RW NEO cache and try to `Copy()`
the underlying's DAO cache which is in direct use of
ComputeNextBlockValidators. Here's the race:
ComputeNextBlockValidators called by Consensus service tries to
update cached `validators` value, and descendant wrapped DAO
created by the  RPC server tries to copy DAO's native cache and
read the cached `validators` value.

So the problem is that native cache not designated to handle
concurrent access between parent DAO layer and derived (wrapped)
DAO layer. I've carefully reviewed all the usages of native cache,
and turns out that the described situation is the only place where
parent DAO is used directly to modify its cache concurrently with
some descendant DAO that is trying to access the cache. All other
usages of native cache (not only NEO, but also all other native
contrcts) strictly rely on the hierarchical DAO structure and don't
try to perform these concurrent operations between DAO layers.
There's also persist operation, but it keeps cache RW lock taken,
so it doesn't have this problem as far. Thus, in this commit we rework
NEO's `validators` cache value so that it always contain the relevant
list for upper Blockchain's DAO and is updated every PostPersist (if
needed).

Note: we must be very careful extending our native cache in the
future, every usage of native cache must be checked against the
described problem.

Close .

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-10 13:18:05 +03:00
Roman Khimov
0a160ee93b *: use CompareAndSwap instead of CAS for atomics
go.uber.org/atomic deprecated CAS methods in version 1.10 (that introduced
CompareAndSwap), so we need to fix it.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-04-28 12:07:19 +03:00
Anna Shaleva
3a71aafc43 core: distinguish notarypool/mempool metrics
Move them to the core/network packages, close . The name of
mempool's unsorted transactions metrics has been changed along the
way to match the core's metrics naming convention.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-13 18:40:19 +03:00
Anna Shaleva
6b21ad9922 *: replace interface{} with any keyword
Everywhere including examples, external interop APIs, bindings generators
code and in other valuable places. A couple of `interface{}` usages are
intentionally left in the CHANGELOG.md, documentation and tests.
2023-04-04 13:22:42 +03:00
Anna Shaleva
c7566c2a01 consensus: perform batched chain's block sync
If there are several blocks from chain, then initialize consensus only for the
latest one.
2023-03-15 17:45:56 +03:00
Anna Shaleva
0cbef58b3c consensus: enqueue newly created blocks
Do not add them directly to chain, it will be done by the block queue
manager. Close https://github.com/nspcc-dev/neo-go/issues/2923. However,
this commit is not valid without
https://github.com/roman-khimov/dbft/pull/4.
It's the neo-go's duty to initialize consensus after subsequent block
addition; the dBFT itself must wait for the neo-go to complete the block
addition and notify the dBFT, so that it can initialize at 0-th view to
collect the next block.
2023-03-15 17:37:47 +03:00
Anna Shaleva
e57e74692f consensus: drain block notification channel after each message
Make blockchain lock less possible.
2023-03-15 17:37:47 +03:00
Anna Shaleva
ea46943815 services: use buffered channels for block subscription
Add a tiny buffer where possible to avoid Blockchain's blocking
on new block addition.
2023-03-15 17:37:47 +03:00
Roman Khimov
7589733017 config: add a special Blockchain type to configure Blockchain
And include some node-specific configurations there with backwards
compatibility. Note that in the future we'll remove Ledger's
fields from the ProtocolConfiguration and it'll be possible to access them in
Blockchain directly (not via .Ledger).

The other option tried was using two configuration types separately, but that
incurs more changes to the codebase, single structure that behaves almost like
the old one is better for backwards compatibility.

Fixes .
2022-12-07 17:35:53 +03:00
Anna Shaleva
82221b0ca7 *: fix Neo and NeoGo misuses 2022-12-07 17:29:09 +03:00
Roman Khimov
883c6c5286 config: add Consensus subsection for dBFT config, fix
Old UnlockWallet is still supported and works just fine.
2022-12-07 13:20:21 +03:00
Roman Khimov
493b979b95 dbft: rev up, fix context timestamp initialization
See nspcc-dev/dbft#64, fixes .
2022-10-20 12:38:14 +03:00
Roman Khimov
73ce898e27 network/consensus: use new dbft StopTxFlow callback
It makes sense in general (further narrowing down the time window when
transactions are processed by consensus thread) and it improves block times a
little too, especially in the 7+2 scenario.

Related to .
2022-10-18 11:06:20 +03:00
Roman Khimov
58dc8d0c9b *: always close the wallet after use
Fix .
2022-09-02 14:44:32 +03:00
Roman Khimov
fd8da6fdb9 *: do not get private key from Account to check if it CanSign()
We have this API now to performs checks.
2022-09-02 14:43:34 +03:00
Roman Khimov
8d33206bb8 *: don't get private key from account if just public one is needed
Add PublicKey() API to the Account and use it as appropriate, avoid creating
additional references to the private key.
2022-09-02 14:43:28 +03:00
Roman Khimov
eeeb0f6f0e core: accept two-side channels for sub/unsub, read on unsub
Blockchain's notificationDispatcher sends events to channels and these
channels must be read from. Unfortunately, regular service shutdown procedure
does unsubscription first (outside of the read loop) and only then drains the
channel. While it waits for unsubscription request to be accepted
notificationDispatcher can try pushing more data into the same channel which
will lead to a deadlock. Reading in the same method solves this, any number of
events can be pushed until unsub channel accepts the data.
2022-08-19 22:08:40 +03:00
Roman Khimov
9b0ea2c21b network/consensus: always process dBFT messages as high priority
Move category definition from consensus to payload, consensus service is the
one of its kind (HP), so network.Server can be adjusted accordingly.
2022-08-02 13:07:18 +03:00
Roman Khimov
5a7fa2d3df cli: restart consensus service on USR2
Fix . Also drop wallet from the ServerConfig since it's not used in any
meaningful way after this change.
2022-08-02 13:05:07 +03:00
Roman Khimov
88542630ac blockchainer: drop the package completely
It's not an ideal solution, but at least it solves the problem for
now. Caveats:
 * consensus only needs one method, so it's mirrored to Blockchain
 * rpcsrv uses core.* definition of the StateRoot (so technically it might as
   well not have an internal Ledger), but it uses core already unfortunately
2022-07-25 11:58:13 +03:00
Roman Khimov
9f05009d1a
Merge pull request from nspcc-dev/service-review
Service review
2022-07-05 12:23:25 +03:00
Roman Khimov
3e2eda6752 *: add some comments to service Start/Shutdown methods 2022-07-04 23:03:50 +03:00
Roman Khimov
593f4e8734
Merge pull request from nspcc-dev/cli/wallet-config
cli: allow to specify wallet via configuration file
2022-07-04 19:24:23 +03:00
Anna Shaleva
5f36a7ca0f *: do not call wallet.Close() explicitly
NewWallet and NewWalletFromFile close underlying io.Closer by itself,
no need to close it manually. Introduced in .
2022-07-04 19:09:48 +03:00
Roman Khimov
527505ea5e consensus: drain messages and transactions on exit as well
There might be some threads blocked on these channels.
2022-07-01 21:55:41 +03:00
Roman Khimov
cd4f46247d consensus: make double-call to Shutdown a no-op 2022-07-01 21:49:40 +03:00
Anna Shaleva
8ab422da66 *: properly unsubscribe from Blockchain events 2022-06-28 19:09:25 +03:00
Anna Shaleva
de203b5c76 consensus: check whether server is started before transaction handling
consensus.OnTransaction is a callback, so it can be called at any time.
We need to check whether service (and dBFT) is started before the
subsequent transaction handling like it is done inside the OnPayload
callback.
2022-06-28 18:42:12 +03:00
Elizaveta Chichindaeva
28908aa3cf [] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-04 19:48:27 +03:00
Roman Khimov
2593bb0535 network: extend Service with Name, use it to distinguish services 2022-04-26 00:31:48 +03:00
Anna Shaleva
d3672eb14a core: use BaseExecFee from InteropContext instead of Blockchain's one
The InteropContext's one contains all relevant fee changes applied in
the prevouse transactions of the current block.
2022-04-08 12:56:46 +03:00
Roman Khimov
e621f746a7 config/core: allow to change the number of validators
Fixes .
2022-01-31 23:14:38 +03:00
Roman Khimov
59b2696849 services/consensus: drop blockchainer.Blockchainer use
Simplify testing, avoid monstrous Blockchainer interface.
2022-01-14 19:57:16 +03:00
Roman Khimov
66aafd868b network: unplug stateroot service from the Server
Notice that it makes the node accept Extensible payloads with any category
which is the same way C# node works. We're trusting Extensible senders,
improper payloads are harmless until they DoS the network, but we have some
protections against that too (and spamming with proper category doesn't differ
a lot).
2022-01-14 19:55:50 +03:00
Roman Khimov
5dd4db2c02 network/services: unify service lifecycle management
Run with Start, Stop with Shutdown, make behavior uniform.
2022-01-14 19:53:45 +03:00
Roman Khimov
4d2ecab16f consensus: fix nonce handling
It was broken somewhere between 2f490a3403 and
85ce207f40 leading to panic on watch only node:

2021-07-21T16:21:39.201+0200    INFO    received Commit {"validator": 3}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xbcc59e]

goroutine 486 [running]:
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).newBlockFromContext(0xc0001629a0, 0xc000308000, 0xc0010fa000, 0x2cb417800)
        github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:664 +0xbe
github.com/nspcc-dev/dbft.(*Context).MakeHeader(...)
        github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/context.go:270
github.com/nspcc-dev/dbft.(*DBFT).onCommit(0xc000308000, 0x138c998, 0xc000115110)
        github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:487 +0x575
github.com/nspcc-dev/dbft.(*DBFT).OnReceive(0xc000308000, 0x138c998, 0xc000115110)
        github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:251 +0xef5
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).eventLoop(0xc0001629a0)
        github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:312 +0x7d6
created by github.com/nspcc-dev/neo-go/pkg/consensus.(*service).Start
        github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:262 +0xdc

In fact, nonce is correctly provided by dbft library (since Legacy), we just
need to use it here.
2021-07-21 19:06:19 +03:00
Evgeniy Stratonikov
85ce207f40 consensus: add Nonce to PrepareRequest
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-15 16:00:02 +03:00
Anna Shaleva
50296975e2 crypto: allow to provide non-default scrypt parameters 2021-06-11 15:22:28 +03:00
Roman Khimov
c4e084b0d8 *: fix whitespace errors
leading/trailing newlines
2021-05-12 22:51:41 +03:00
Roman Khimov
4caff35e73 consensus: fix error checking in decoders
Spotted by errcheck.
2021-05-12 20:14:52 +03:00
Roman Khimov
84b240cc9a services/consensus: add some logging on start 2021-04-02 13:13:26 +03:00
Roman Khimov
a954821b98 consensus: make Shutdown a no-op if not started
It makes client code a bit simpler, we can just issue a Shutdown() request
without thinking much about it (but only once of course).
2021-04-02 12:50:46 +03:00
Roman Khimov
f91ff78918 payload: drop Network from Extensible
It's only used to sign/verify it and is not a part of the structure. It's
still neded in consensus.Payload though because that's the way dbft library
is.
2021-03-26 13:45:18 +03:00
Roman Khimov
95c279325a block: drop Network from the Header
It's not network-tied any more, network is only needed to
sign/verify. Unfortunately we still have to keep network in consensus data
structures because of dbft library interface.
2021-03-26 13:45:18 +03:00
Anna Shaleva
4809cdf0b0 consensus: fix panic during verifyBlock
Issue: panic during mixed 4-nodes consensus setup:
```
2021-03-18T12:01:50.715Z	INFO	skip change view	{"nc": 0, "nf": 3}
2021-03-18T12:01:52.786Z	INFO	received ChangeView	{"validator": 0, "reason": "Timeout", "new view": 1}
2021-03-18T12:01:53.602Z	INFO	received ChangeView	{"validator": 2, "reason": "Timeout", "new view": 1}
2021-03-18T12:01:56.736Z	INFO	received ChangeView	{"validator": 1, "reason": "Timeout", "new view": 1}
2021-03-18T12:01:56.736Z	INFO	changing dbft view	{"height": 3, "view": 1, "index": 3, "role": "Backup"}
2021-03-18T12:02:01.758Z	INFO	received PrepareRequest	{"validator": 2, "tx": 0}
panic: interface conversion: block.Block is nil, not *consensus.neoBlock

goroutine 315 [running]:
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).verifyBlock(0xc000419540, 0x0, 0x0, 0x4)
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:427 +0x1306
github.com/nspcc-dev/dbft.(*DBFT).createAndCheckBlock(0xc0001f8840, 0x13f0002)
	github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:373 +0x27e
github.com/nspcc-dev/dbft.(*DBFT).onPrepareRequest(0xc0001f8840, 0x13f4378, 0xc0003b8500)
	github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:329 +0xdf1
github.com/nspcc-dev/dbft.(*DBFT).OnReceive(0xc0001f8840, 0x13f4378, 0xc0003b8500)
	github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:247 +0xe25
github.com/nspcc-dev/neo-go/pkg/consensus.(*service).eventLoop(0xc000419540)
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:297 +0x79d
created by github.com/nspcc-dev/neo-go/pkg/consensus.(*service).Start
	github.com/nspcc-dev/neo-go/pkg/consensus/consensus.go:249 +0xa5
```

So (*service).verifyBlock is unable to work with nil block.
2021-03-18 18:20:33 +03:00
Anna Shaleva
20b70ee9fe config: add MaxBlockSystemFee setting 2021-03-16 12:09:11 +03:00