e80c60a3b9
We have a number of queues for different purposes: * regular broadcast queue * direct p2p queue * high-priority queue And two basic egress scenarios: * direct p2p messages (replies to requests in Server's handle* methods) * broadcasted messages Low priority broadcasted messages: * transaction inventories * block inventories * notary inventories * non-consensus extensibles High-priority broadcasted messages: * consensus extensibles * getdata transaction requests from consensus process * getaddr requests P2P messages are a bit more complicated, most of the time they use p2p queue, but extensible message requests/replies use HP queue. Server's handle* code is run from Peer's handleIncoming, every peer has this thread that handles incoming messages. When working with the peer it's important to reply to requests and blocking this thread until we send (queue) a reply is fine, if the peer is slow we just won't get anything new from it. The queue used is irrelevant wrt this issue. Broadcasted messages are radically different, we want them to be delivered to many peers, but we don't care about specific ones. If it's delivered to 2/3 of the peers we're fine, if it's delivered to more of them --- it's not an issue. But doing this fairly is not an easy thing, current code tries performing unblocked sends and if this doesn't yield enough results it then blocks (but has a timeout, we can't wait indefinitely). But it does so in sequential manner, once the peer is chosen the code will wait for it (and only it) until timeout happens. What can be done instead is an attempt to push the message to all of the peers simultaneously (or close to that). If they all deliver --- OK, if some block and wait then we can wait until _any_ of them pushes the message through (or global timeout happens, we still can't wait forever). If we have enough deliveries then we can cancel pending ones and it's again not an error if these canceled threads still do their job. This makes the system more dynamic and adds some substantial processing overhead, but it's a networking code, any of this overhead is much lower than the actual packet delivery time. It also allows to spread the load more fairly, if there is any spare queue it'll get the packet and release the broadcaster. On the next broadcast iteration another peer is more likely to be chosen just because it didn't get a message previously (and had some time to deliver already queued messages). It works perfectly in tests, with optimal networking conditions we have much better block times and TPS increases by 5-25%% depending on the scenario. I'd go as far as to say that it fixes the original problem of #2678, because in this particular scenario we have empty queues in ~100% of the cases and this new logic will likely lead to 100% fan out in this case (cancelation just won't happen fast enough). But when the load grows and there is some waiting in the queue it will optimize out the slowest links. |
||
|---|---|---|
| .circleci | ||
| .docker | ||
| .github | ||
| cli | ||
| config | ||
| docs | ||
| examples | ||
| internal | ||
| pkg | ||
| scripts | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .golangci.yml | ||
| CHANGELOG.md | ||
| codecov.yml | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| Dockerfile.wsc | ||
| go.mod | ||
| go.sum | ||
| LICENSE.md | ||
| Makefile | ||
| neo-go.service.template | ||
| README.md | ||
| ROADMAP.md | ||
Go Node and SDK for the Neo blockchain.
Overview
This project aims to be a full port of the original C# Neo project. A complete toolkit for the NEO blockchain, including:
- Consensus node
- RPC node & client
- CLI tool
- Smart contract compiler
- NEO virtual machine
- Smart contract examples
- Oracle service
- State validation service
This branch (master) is Neo N3-compatible. For the current Legacy-compatible version please refer to the master-2.x branch and releases before 0.80.0 (0.7X.Y track).
Getting started
Installation
NeoGo is distributed as a single binary that includes all the functionality
provided (but smart contract compiler requires Go compiler to operate). You
can grab it from releases
page, use a Docker image (see
Docker Hub for various releases of
NeoGo, :latest points to the latest release) or build yourself.
Building
To build NeoGo you need Go 1.17+ and make:
make build
The resulting binary is bin/neo-go.
Building on Windows
To build NeoGo on Windows platform we recommend you to install make from MinGW
package. Then, you can build NeoGo with:
make build
The resulting binary is bin/neo-go.exe.
We also recommend you to switch the Windows Firewall off for further NeoGo node run.
Running a node
A node needs to connect to some network, either local one (usually referred to
as privnet) or public (like mainnet or testnet). Network configuration
is stored in a file and NeoGo allows you to store multiple files in one
directory (./config by default) and easily switch between them using network
flags.
To start Neo node on a private network, use:
./bin/neo-go node
Or specify a different network with an appropriate flag like this:
./bin/neo-go node --mainnet
Available network flags:
--mainnet, -m--privnet, -p--testnet, -t
To run a consensus/committee node, refer to consensus documentation.
Docker
By default, the CMD is set to run a node on privnet. So, to do this, simply run:
docker run -d --name neo-go -p 20332:20332 -p 20331:20331 nspccdev/neo-go
Which will start a node on privnet and expose node's ports 20332 (P2P
protocol) and 20331 (JSON-RPC server).
Importing mainnet/testnet dump files
If you want to jump-start your mainnet or testnet node with chain archives provided by NGD, follow these instructions:
$ wget .../chain.acc.zip # chain dump file
$ unzip chain.acc.zip
$ ./bin/neo-go db restore -m -i chain.acc # for testnet use '-t' flag instead of '-m'
The process differs from the C# node in that block importing is a separate mode. After it ends, the node can be started normally.
Running a private network
Refer to consensus node documentation.
Smart contract development
Please refer to neo-go smart contract development workshop that shows some simple contracts that can be compiled/deployed/run using neo-go compiler, SDK and a private network. For details on how Go code is translated to Neo VM bytecode and what you can and can not do in a smart contract, please refer to the compiler documentation.
Refer to examples for more NEO smart contract examples written in Go.
Wallets
NeoGo differs substantially from C# implementation in its approach to wallets. NeoGo wallet is just a NEP-6 file that is used by CLI commands to sign various things. There is no database behind it, the blockchain is the database and CLI commands use RPC to query data from it. At the same time, it's not required to open a wallet on an RPC node to perform various actions (unless your node provides some service for the network like consensus or oracle nodes do).
Developer notes
Nodes have such features as Prometheus and Pprof in order to have additional information about them for debugging.
How to configure Prometheus or Pprof:
In config/protocol.*.yml there is
Prometheus:
Enabled: true
Port: 2112
where you can switch on/off and define port. Prometheus is enabled and Pprof is disabled by default.
Contributing
Feel free to contribute to this project after reading the contributing guidelines.
Before starting to work on a certain topic, create a new issue first describing the feature/topic you are going to implement.
Contact
- @roman-khimov on GitHub
- @AnnaShaleva on GitHub
- @fyrchik on GitHub
- Reach out to us on the NEO Discord channel
License
- Open-source MIT