Commit graph

72 commits

Author SHA1 Message Date
Roman Khimov
d686fe4e5d network: get blocks directly from the chain for rebroadcasting
Simplify network<->consensus relations, also broadcast blocks received by
other means like RPC.
2020-05-12 17:42:34 +03:00
Roman Khimov
dd8bcfae47 consensus: remove OnNewBlock(), use Blockchain subscription
Get new blocks directly from the Blockchain. It may lead to some duplications
(as we'll also receive our own blocks), but at the same time it's more
correct, because technically we can also get blocks via other means besides
network server like RPC (submitblock call). And it simplifies network server
at the same time.
2020-05-12 17:41:23 +03:00
Anna Shaleva
c813873577 consensus: added partial message decoding
closes #849
2020-04-16 17:16:29 +03:00
Roman Khimov
9b5dab57e8
Merge pull request #810 from nspcc-dev/refactor/config
config: move config.go out of config
2020-04-01 10:29:57 +03:00
Anna Shaleva
5a984fdf88 config: move config.go out of config/
closes #423
2020-03-31 17:55:59 +03:00
Evgenii Stratonikov
0036b3e52b random: make use or random package in tests
Also implement Bytes/Fill routines for generating byte slices.
2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
9abda40171 testserdes: implement helpers for encode/decode routines
Frequently one needs to check if struct serializes/deserializes
properly. This commit implements helpers for such cases including:
1. JSON
2. io.Serializable interface
2020-03-27 10:27:46 +03:00
Roman Khimov
f5a1b928ce pkg: fix gofmt issues 2020-03-03 17:22:15 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Evgenii Stratonikov
21ef2638c0 consensus: log error if GetValidators() failed 2020-03-02 18:01:49 +03:00
Roman Khimov
a903147b60 keys: rename Signature to GetScriptHash, make it return Uint160
Signature itself wasn't used at all and its name is very misleading, Uint160
script hash is way more useful.
2020-02-19 12:19:03 +03:00
Roman Khimov
eb11e5fb11 core: implement basic policying support, fix #370
Implement mempool and consensus block creation policies, almost the same as
SimplePolicy plugin for C# node provides with two caveats:
 * HighPriorityTxType is not configured and hardcoded to ClaimType
 * BlockedAccounts are not supported

Other than that it allows us to run successfuly as testnet CN, previously our
proposals were rejected because we were proposing blocks with oversized
transactions (that are rejected by PoolTx() now).

Mainnet and testnet configuration files are updated accordingly, but privnet
is left as is with no limits.

Configuration is currently attached to the Blockchain and so is the code that
does policying, it may be moved somewhere in the future, but it works for
now.
2020-02-19 12:19:02 +03:00
Roman Khimov
22f5667530 mempool: return fee along with tx when requesting tx
Users of GetVerifiedTransactions() don't want to recalculate tx fee and it's
nice to have it returned from TryGetValue() also sometimes.
2020-02-19 12:19:02 +03:00
Roman Khimov
06daeb44f3 core: make IsLowPriority work with pre-calculated fee
Don't recalculate it again and again.
2020-02-19 12:19:02 +03:00
Roman Khimov
37c48b00b4 consensus/network: reinit dbft after block addition
Don't stall on some height if everyone else have moved up already. Fix #673.
2020-02-19 12:19:02 +03:00
Roman Khimov
a2616cfafe consensus: correctly generate miner TX outputs based on net fee 2020-02-19 12:13:27 +03:00
Evgenii Stratonikov
ee8514d88b consensus: provide message type and view inside RecoveryMessage
PrepareRequest inside RecoveryMessage should contain both
message type and view.
2020-02-18 15:25:29 +03:00
Roman Khimov
d4cd134aef core: sort GetValidators() result
As C# node does it. Technically it's only needed for consensus and could be
implemented in the appropriate package, but for better compatibility with C#
node we're better returning it sorted right here.
2020-02-13 17:07:27 +03:00
Roman Khimov
d3badc3f97 consensus: log more like neo-cli, update dbft 2020-02-11 18:02:13 +03:00
Roman Khimov
7445655437 consensus: switch test to using the new PoolTx API
Fixes GolangCI:
  Error return value of
  (*github.com/CityOfZion/neo-go/pkg/core/mempool.Pool).Add is not checked
  (from errcheck)

and allows us to almost completely forget about mempool here.
2020-02-06 17:50:11 +03:00
Roman Khimov
a928ad9cfa mempool: make item an internal thing of mempool package
Nobody outside should care about these details, mempool operates on
transactions and that's it.
2020-02-06 15:41:56 +03:00
Roman Khimov
b9b77ac1be network: fix block relaying, don't spit out useless errors
We can only add one block of the given height and we have two competing
goroutines to do that --- consensus and block queue. Whomever adds the block
first shouldn't trigger an error in another one.

Fix block relaying for blocks added via the block queue also, previously one
consensus-generated blocks were broadcasted.
2020-02-06 15:41:56 +03:00
Roman Khimov
70b3839fd0 core/mempool: fix AddBlock and tx pooling concurrency issues
Eliminate races between tx checks and adding them to the mempool, ensure the
chain doesn't change while we're working with the new tx. Ensure only one
block addition attempt could be in progress.
2020-02-06 15:41:52 +03:00
Roman Khimov
70c22ebc7b
Merge pull request #641 from nspcc-dev/consensus-update
Consensus update
2020-02-04 15:39:53 +03:00
Roman Khimov
9ceefe17e2 consensus: check current chain height before processing timer
The chain may already be more current than our dBFT state (like when the node
has commited something at view 0, but all the other nodes changed view and
accepted something at view 1), so in this case we should reinit dBFT on new
height.
2020-02-04 13:16:09 +03:00
Evgenii Stratonikov
e65b5c9914 consensus: fill preparation hash in recovery message 2020-02-04 13:16:05 +03:00
Evgenii Stratonikov
e441cb1414 consensus: add preparation hash to logger 2020-02-04 13:16:00 +03:00
Evgenii Stratonikov
42dfb52e98 consensus: add more logging in consensus service 2020-02-04 13:15:54 +03:00
Roman Khimov
553b2391a3 consensus: handle encoding errors in Hash() 2020-02-03 16:33:03 +03:00
Roman Khimov
388fed06e5 transaction: forbid serializing invalid transactions
Transaction that has no data is invalid and can't be serialized, so throw an
error if someone tries to.
2020-02-03 16:33:03 +03:00
Evgenii Stratonikov
390bf58f97 consensus: use our implementations of Recovery* messages
While decoding payload, local implementations of Recovery*
messages were used, but when creating RecoveryMessage inside dBFT
library default NewRecoveryMessage was invoked. This lead to parsing
errors.
2020-01-29 17:56:26 +03:00
Roman Khimov
760b39e324 consensus: close chain in tests
Avoid test failures like this one:

=== RUN   TestService_OnPayload
==================
WARNING: DATA RACE
Read at 0x00c00015a843 by goroutine 112:
  testing.(*common).logDepth()
      /usr/local/go/src/testing/testing.go:665 +0xa1
  testing.(*common).Logf()
      /usr/local/go/src/testing/testing.go:658 +0x8f
  testing.(*T).Logf()
      <autogenerated>:1 +0x75
  go.uber.org/zap/zaptest.testingWriter.Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x11f
  go.uber.org/zap/zaptest.(*testingWriter).Write()
      <autogenerated>:1 +0xa9
  go.uber.org/zap/zapcore.(*ioCore).Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c3
  go.uber.org/zap/zapcore.(*CheckedEntry).Write()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e7
  go.uber.org/zap.(*Logger).Info()
      /go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:187 +0x95
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).persist()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:720 +0x6bb
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run.func2()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:228 +0x53

Previous write at 0x00c00015a843 by goroutine 98:
  testing.tRunner.func1()
      /usr/local/go/src/testing/testing.go:900 +0x353
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:913 +0x1bb

Goroutine 112 (running) created at:
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:227 +0x264

Goroutine 98 (finished) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:960 +0x651
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1202 +0xa6
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:909 +0x199
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1200 +0x521
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1117 +0x2ff
  main.main()
      _testmain.go:162 +0x337
==================
--- FAIL: TestService_OnPayload (4.11s)
2020-01-20 19:40:52 +03:00
Roman Khimov
32213b1454
Merge pull request #601 from nspcc-dev/refactoring/core
core: refactor out Block, BlockBase and Header, closes #597.
2020-01-20 16:19:20 +03:00
Evgenii Stratonikov
55d98ab19a consensus: update keys from wallet after every block 2020-01-17 17:30:45 +03:00
Evgenii Stratonikov
aaefcdecae consensus: use wallet in service 2020-01-17 17:25:51 +03:00
Roman Khimov
46b82b4fb5 keys: don't return error from PrivateKey.Sign
As it can't ever happen.
2020-01-17 17:00:30 +03:00
Evgenii Stratonikov
fed6fba9b6 core: refactor out MemPool 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
489b88afbb block: rename BlockBase to Base 2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
63c56cca5c core: refactor out Block, BlockBase and Header structs
See #597.
2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
3d704a3a3a consensus: try to use previous proposal after ChangeView
When system and network pressure is high it can be beneficial
to use transactions which and were already proposed.
The assumption is that they will be in other node's memory pool
with more probability.
2020-01-14 16:05:27 +03:00
Roman Khimov
565c1940e3
Merge pull request #587 from nspcc-dev/feature/zap
Closes #284.
2020-01-10 14:50:09 +03:00
Evgenii Stratonikov
dc65684708 consensus: replace SugaredLogger with Logger
SugaredLogger was used to make output look more similar to logrus.
This is no longer needed.
2020-01-10 11:14:29 +03:00
Evgenii Stratonikov
aecdf470e7 cli,pkg: use zap.Logger 2020-01-10 11:14:27 +03:00
Roman Khimov
b5b05a969c keys: make NEP2Decrypt return a PrivateKey rather than WIF
There is no point in encoding the output of this function in a WIF format,
most of the users actually want the real key and those who need a WIF can
easily get if from the key (and it's simpler than getting the key from the
WIF).

It also fixes a severe bug in NEP2Decrypt, base58 decoding errors were not
processed correctly.
2020-01-09 18:05:14 +03:00
Evgenii Stratonikov
8d1f564411 consensus: add tests for getTx 2019-12-27 14:12:58 +03:00
Evgenii Stratonikov
d803dffca8 consensus: return nil interface from getTx 2019-12-27 13:54:11 +03:00
Evgenii Stratonikov
64d24d8ddd consensus: verify payloads correctly 2019-12-26 10:49:56 +03:00
Evgenii Stratonikov
0efd9a6062 consensus: fix payload sign test
Do not fill verification script randomly as there is a probability
for it to be executed sucessfully.

time="2019-12-12T17:24:22+03:00" level=info msg="blockchain persist completed" blockHeight=0 headerHeight=0 persistedBlocks=0 persistedKeys=15 took="54.474µs"
time="2019-12-12T17:24:23+03:00" level=info msg="blockchain persist completed" blockHeight=0 headerHeight=0 persistedBlocks=0 persistedKeys=15 took="49.312µs"
2019-12-12T17:24:24.026+0300    DEBUG   can't verify payload from #%d1  {"module": "dbft"}
--- FAIL: TestPayload_Sign (0.00s)
    payload_test.go:302:
                Error Trace:    payload_test.go:302
                Error:          Should be false
                Test:           TestPayload_Sign
FAIL
coverage: 75.8% of statements
FAIL    github.com/CityOfZion/neo-go/pkg/consensus      2.145s
2019-12-13 12:09:51 +03:00
Roman Khimov
8b3080b972 io: rename Read/WriteBytes to Read/WriteB
go vet is not happy about them:
  pkg/io/binaryReader.go:92:21: method ReadByte() byte should have signature ReadByte() (byte, error)
  pkg/io/binaryWriter.go:75:21: method WriteByte(u8 byte) should have signature WriteByte(byte) error
2019-12-12 20:19:50 +03:00
Roman Khimov
54d888ba70 io: add type-specific read/write methods
This seriously improves the serialization/deserialization performance for
several reasons:
 * no time spent in `binary` reflection
 * no memory allocations being made on every read/write
 * uses fast ReadBytes everywhere it's appropriate

It also makes Fixed8 Serializable just for convenience.
2019-12-12 20:19:50 +03:00