400620a9fb
`cache` persisting operation is concurrent with the storage modifications made by further state reset stages. We can't allow situation when data from the next stage are leaking into the previous stage. State reset stages must be atomic in turms of DB persisting, thus, use another `upperCache` MemCached store to keep them apart. Here are the results of mainnet's BoltDB reset from 1.1M to 6K: This patch: ``` anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000 2023-04-11T16:15:25.783+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144} 2023-04-11T16:15:25.783+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000} 2023-04-11T16:15:25.783+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512} 2023-04-11T16:15:25.783+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760} 2023-04-11T16:15:25.787+0300 INFO restoring blockchain {"version": "0.2.8"} 2023-04-11T16:15:25.906+0300 INFO initializing state reset {"target height": 600000} 2023-04-11T16:15:25.906+0300 DEBUG trying to reset blocks, transactions and AERs 2023-04-11T16:15:57.031+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 1, "took": "31.125057214s"} 2023-04-11T16:16:12.644+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 1, "took": "15.613156971s", "keys": 321895} 2023-04-11T16:16:13.895+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 2, "took": "16.663444208s"} 2023-04-11T16:16:19.784+0300 INFO last batch of removed blocks, transactions and AERs is collected {"batch": 3, "took": "5.760308543s"} 2023-04-11T16:16:19.784+0300 INFO blocks, transactions ans AERs are reset {"took": "53.878632911s"} 2023-04-11T16:16:22.870+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 2, "took": "8.974838893s", "keys": 334823} 2023-04-11T16:16:22.870+0300 DEBUG trying to reset contract storage items 2023-04-11T16:16:27.272+0300 DEBUG last batch of removed blocks, transactions and AERs is persisted {"batch": 3, "took": "7.487357441s", "keys": 208913} 2023-04-11T16:17:23.678+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 1, "took": "1m0.80711106s"} 2023-04-11T16:18:00.769+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 2, "took": "36.967660061s"} 2023-04-11T16:18:20.478+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 1, "took": "56.796257788s", "keys": 200000} 2023-04-11T16:18:54.115+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 3, "took": "33.637412437s"} 2023-04-11T16:19:18.844+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 2, "took": "1m18.0737668s", "keys": 200000} 2023-04-11T16:19:27.650+0300 INFO last batch of contract storage items is collected {"batch": 4, "took": "8.806264019s"} 2023-04-11T16:19:27.650+0300 INFO contract storage items are reset {"took": "3m4.780232077s", "keys": 656944} 2023-04-11T16:20:15.660+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 3, "took": "1m21.544386403s", "keys": 200000} 2023-04-11T16:20:15.660+0300 DEBUG trying to reset headers information 2023-04-11T16:20:16.385+0300 INFO headers information is reset {"took": "725.174932ms"} 2023-04-11T16:20:19.586+0300 DEBUG last batch of contract storage items and IDs is persisted {"batch": 4, "took": "51.936278608s", "keys": 56945} 2023-04-11T16:20:19.587+0300 DEBUG trying to reset state root information and NEP transfers 2023-04-11T16:20:35.845+0300 INFO state root information and NEP transfers are reset {"took": "16.25852114s"} 2023-04-11T16:21:10.000+0300 DEBUG headers information is persisted {"took": "53.613638429s", "keys": 528438} 2023-04-11T16:21:10.003+0300 DEBUG trying to remove stale storage items 2023-04-11T16:21:18.108+0300 INFO stale storage items are reset {"took": "8.105140658s", "keys": 1350176} 2023-04-11T16:21:18.108+0300 DEBUG trying to remove state reset point 2023-04-11T16:21:18.108+0300 INFO state reset point is removed {"took": "8.554µs"} 2023-04-11T16:21:20.151+0300 DEBUG state root information and NEP transfers are persisted {"took": "44.305707049s", "keys": 602578} 2023-04-11T16:21:20.212+0300 INFO state reset point information is persisted {"took": "2.103764633s", "keys": 2} 2023-04-11T16:21:20.213+0300 INFO reset finished successfully {"took": "5m54.306861367s"} ``` The previous commit: ``` anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000 2023-04-11T16:24:04.256+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144} 2023-04-11T16:24:04.256+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000} 2023-04-11T16:24:04.256+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512} 2023-04-11T16:24:04.256+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760} 2023-04-11T16:24:04.261+0300 INFO restoring blockchain {"version": "0.2.8"} 2023-04-11T16:24:04.368+0300 INFO initializing state reset {"target height": 600000} 2023-04-11T16:24:04.368+0300 DEBUG trying to reset blocks, transactions and AERs 2023-04-11T16:24:30.363+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 1, "took": "25.995261037s"} 2023-04-11T16:24:44.947+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 1, "took": "14.584447338s", "keys": 321897} 2023-04-11T16:24:45.791+0300 INFO intermediate batch of removed blocks, transactions and AERs is collected {"batch": 2, "took": "15.428492824s"} 2023-04-11T16:24:51.252+0300 INFO last batch of removed blocks, transactions and AERs is collected {"batch": 3, "took": "5.460662766s"} 2023-04-11T16:24:51.252+0300 INFO blocks, transactions ans AERs are reset {"took": "46.884558096s"} 2023-04-11T16:24:55.399+0300 DEBUG intermediate batch of removed blocks, transactions and AERs is persisted {"batch": 2, "took": "9.607820004s", "keys": 334821} 2023-04-11T16:24:55.399+0300 DEBUG trying to reset contract storage items 2023-04-11T16:24:59.981+0300 DEBUG last batch of removed blocks, transactions and AERs is persisted {"batch": 3, "took": "8.728713255s", "keys": 208913} 2023-04-11T16:25:50.827+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 1, "took": "55.426411416s"} 2023-04-11T16:26:28.734+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 2, "took": "37.902647706s"} 2023-04-11T16:26:53.960+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 1, "took": "1m3.129453265s", "keys": 200001} 2023-04-11T16:27:27.645+0300 INFO intermediate batch of contract storage items and IDs is collected {"batch": 3, "took": "33.685283662s"} 2023-04-11T16:27:52.173+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 2, "took": "1m23.438465575s", "keys": 199999} 2023-04-11T16:28:00.995+0300 INFO last batch of contract storage items is collected {"batch": 4, "took": "8.821990443s"} 2023-04-11T16:28:00.995+0300 INFO contract storage items are reset {"took": "3m5.595950958s", "keys": 656944} 2023-04-11T16:28:49.164+0300 DEBUG intermediate batch of contract storage items is persisted {"batch": 3, "took": "1m21.518344712s", "keys": 200000} 2023-04-11T16:28:49.164+0300 DEBUG trying to reset headers information 2023-04-11T16:28:49.936+0300 INFO headers information is reset {"took": "772.36435ms"} 2023-04-11T16:28:53.122+0300 DEBUG last batch of contract storage items and IDs is persisted {"batch": 4, "took": "52.126928092s", "keys": 56945} 2023-04-11T16:28:53.122+0300 DEBUG trying to reset state root information and NEP transfers 2023-04-11T16:29:09.332+0300 INFO state root information and NEP transfers are reset {"took": "16.20921699s"} 2023-04-11T16:29:46.264+0300 DEBUG headers information is persisted {"took": "56.326715249s", "keys": 528438} 2023-04-11T16:29:46.267+0300 DEBUG trying to remove stale storage items 2023-04-11T16:29:53.986+0300 INFO stale storage items are reset {"took": "7.718950145s", "keys": 1350176} 2023-04-11T16:29:53.986+0300 DEBUG trying to remove state reset point 2023-04-11T16:29:53.986+0300 INFO state reset point is removed {"took": "6.013µs"} 2023-04-11T16:29:55.899+0300 DEBUG state root information and NEP transfers are persisted {"took": "46.567302762s", "keys": 602578} 2023-04-11T16:29:55.929+0300 INFO state reset point information is persisted {"took": "1.942392208s", "keys": 2} 2023-04-11T16:29:55.929+0300 INFO reset finished successfully {"took": "5m51.561573137s"} ``` Master: ``` anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go db reset -m --debug --height 600000 2023-04-11T16:34:12.410+0300 INFO MaxBlockSize is not set or wrong, setting default value {"MaxBlockSize": 262144} 2023-04-11T16:34:12.410+0300 INFO MaxBlockSystemFee is not set or wrong, setting default value {"MaxBlockSystemFee": 900000000000} 2023-04-11T16:34:12.410+0300 INFO MaxTransactionsPerBlock is not set or wrong, using default value {"MaxTransactionsPerBlock": 512} 2023-04-11T16:34:12.410+0300 INFO MaxValidUntilBlockIncrement is not set or wrong, using default value {"MaxValidUntilBlockIncrement": 5760} 2023-04-11T16:34:12.413+0300 INFO restoring blockchain {"version": "0.2.8"} 2023-04-11T16:34:12.495+0300 INFO initialize state reset {"target height": 600000} 2023-04-11T16:34:12.513+0300 INFO trying to reset blocks, transactions and AERs 2023-04-11T16:35:03.582+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 1, "took": "51.087226195s", "keys": 321895} 2023-04-11T16:35:31.302+0300 INFO intermediate batch of removed blocks, transactions and AERs is persisted {"batches persisted": 2, "took": "27.719871393s", "keys": 334823} 2023-04-11T16:35:41.309+0300 INFO last batch of removed blocks, transactions and AERs is persisted {"batches persisted": 3, "took": "10.007017388s", "keys": 208913} 2023-04-11T16:35:41.309+0300 INFO blocks, transactions ans AERs are reset {"took": "1m28.814245057s", "overall persisted keys": 865631} 2023-04-11T16:35:41.309+0300 INFO trying to reset contract storage items 2023-04-11T16:37:38.315+0300 INFO intermediate batch of contract storage items and IDs is persisted {"batch": 1, "took": "1m57.00650253s", "keys": 200000} 2023-04-11T16:39:29.704+0300 INFO intermediate batch of contract storage items and IDs is persisted {"batch": 2, "took": "1m51.385224725s", "keys": 200000} 2023-04-11T16:41:14.991+0300 INFO intermediate batch of contract storage items and IDs is persisted {"batch": 3, "took": "1m45.287483794s", "keys": 200000} 2023-04-11T16:41:31.667+0300 INFO last batch of contract storage items and IDs is persisted {"batch": 4, "took": "16.675347478s", "keys": 56945} 2023-04-11T16:41:31.667+0300 INFO contract storage items and IDs are reset {"took": "5m50.357775401s", "keys": 656944} 2023-04-11T16:41:31.667+0300 INFO trying to reset headers information 2023-04-11T16:42:16.779+0300 INFO headers information is reset {"took": "45.111354262s", "keys": 528438} 2023-04-11T16:42:16.784+0300 INFO trying to reset state root information and NEP transfers 2023-04-11T16:42:35.778+0300 INFO state root information and NEP transfers are reset {"took": "18.99373117s", "keys": 602578} 2023-04-11T16:42:35.781+0300 INFO trying to remove stale storage items 2023-04-11T16:42:43.884+0300 INFO stale storage items are reset {"took": "8.103929306s", "keys": 1350176} 2023-04-11T16:42:43.885+0300 INFO trying to remove state reset point 2023-04-11T16:42:43.926+0300 INFO stale reset point is removed {"took": "41.858883ms", "keys": 2} 2023-04-11T16:42:43.932+0300 INFO reset finished successfully {"took": "8m31.437493325s"} ``` Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru> |
||
---|---|---|
.docker | ||
.github | ||
cli | ||
config | ||
docs | ||
examples | ||
internal | ||
pkg | ||
scripts | ||
.dockerignore | ||
.gitignore | ||
.gitmodules | ||
.golangci.yml | ||
CHANGELOG.md | ||
codecov.yml | ||
CONTRIBUTING.md | ||
Dockerfile | ||
Dockerfile.wsc | ||
go.mod | ||
go.sum | ||
LICENSE.md | ||
Makefile | ||
neo-go.service.template | ||
README.md | ||
ROADMAP.md |
Go Node and SDK for the Neo blockchain.
Overview
NeoGo is a complete platform for distributed application development built on top of and compatible with the Neo project. This includes, but not limited to (see documentation for more details):
- Consensus node
- RPC node & client
- CLI tool
- Smart contract compiler
- Neo virtual machine
- Smart contract examples
- Oracle service
- State validation service
The protocol implemented here is Neo N3-compatible, however you can also find an implementation of the Neo Legacy protocol in the master-2.x branch and releases before 0.80.0 (0.7X.Y track).
Getting started
Installation
NeoGo is distributed as a single binary that includes all the functionality
provided (but smart contract compiler requires Go compiler to operate). You
can grab it from releases
page, use a Docker image (see
Docker Hub for various releases of
NeoGo, :latest
points to the latest release) or build yourself.
Building
Building NeoGo requires Go 1.18+ and make
:
make
The resulting binary is bin/neo-go
. Notice that using some random revision
from the master
branch is not recommended (it can have any number of
incompatibilities and bugs depending on the development stage), please use
tagged released versions.
Building on Windows
To build NeoGo on Windows platform we recommend you to install make
from MinGW
package. Then, you can build NeoGo with:
make
The resulting binary is bin/neo-go.exe
.
Running a node
A node needs to connect to some network, either local one (usually referred to
as privnet
) or public (like mainnet
or testnet
). Network configuration
is stored in a file and NeoGo allows you to store multiple files in one
directory (./config
by default) and easily switch between them using network
flags.
To start Neo node on a private network, use:
./bin/neo-go node
Or specify a different network with an appropriate flag like this:
./bin/neo-go node --mainnet
Available network flags:
--mainnet, -m
--privnet, -p
--testnet, -t
To run a consensus/committee node, refer to consensus documentation.
If you're running a node on Windows, please turn off or configure Windows Firewall appropriately (allowing inbound connections to the P2P port).
Docker
By default, the CMD
is set to run a node on privnet
. So, to do this, simply run:
docker run -d --name neo-go -p 20332:20332 -p 20331:20331 nspccdev/neo-go
Which will start a node on privnet
and expose node's ports 20332
(P2P
protocol) and 20331
(JSON-RPC server).
Importing mainnet/testnet dump files
If you want to jump-start your mainnet or testnet node with chain archives provided by NGD, follow these instructions:
$ wget .../chain.acc.zip # chain dump file
$ unzip chain.acc.zip
$ ./bin/neo-go db restore -m -i chain.acc # for testnet use '-t' flag instead of '-m'
The process differs from the C# node in that block importing is a separate mode. After it ends, the node can be started normally.
Running a private network
Refer to consensus node documentation.
Smart contract development
Please refer to NeoGo smart contract development workshop that shows some simple contracts that can be compiled/deployed/run using NeoGo compiler, SDK and a private network. For details on how Go code is translated to Neo VM bytecode and what you can and can not do in a smart contract, please refer to the compiler documentation.
Refer to examples for more Neo smart contract examples written in Go.
Wallets
NeoGo wallet is just a NEP-6 file that is used by CLI commands to sign various things. CLI commands are not a direct part of the node, but rather a part of the NeoGo binary, their implementations use RPC to query data from the blockchain and perform any required actions. It's not required to open a wallet on an RPC node (unless your node provides some service for the network like consensus or oracle nodes do).
Monitoring
NeoGo provides Prometheus and Pprof services that can be enabled in the node in order to provide additional monitoring and debugging data.
Configuring any of the two services is easy, add the following section (Pprof
instead of Prometheus
if you need that) to the respective config/protocol.*.yml
:
Prometheus:
Enabled: true
Addresses:
- ":2112"
where you can switch on/off and define port. Prometheus is enabled and Pprof is disabled by default.
Contributing
Feel free to contribute to this project after reading the contributing guidelines.
Before starting to work on a certain topic, create a new issue first describing the feature/topic you are going to implement.
Contact
- @roman-khimov on GitHub
- @AnnaShaleva on GitHub
- @fyrchik on GitHub
- Reach out to us on the Neo Discord channel
License
- Open-source MIT