Commit graph

78 commits

Author SHA1 Message Date
Roman Khimov
55b2cbb74d core: refactor and improve verification and pooling
Now we have VerifyTx() and PoolTx() APIs that either verify transaction in
isolation or verify it against the mempool (either the primary one or the one
given) and then add it there. There is no possibility to check against the
mempool, but not add a transaction to it, but I doubt we really need it.

It allows to remove some duplication between old PoolTx and verifyTx where
they both tried to check transaction against mempool (verifying first and then
adding it). It also saves us utility token balance check because it's done by
the mempool anyway and we no longer need to do that explicitly in verifyTx.

It makes AddBlock() and verifyBlock() transaction's checks more correct,
because previously they could miss that even though sender S has enough
balance to pay for A, B or C, he can't pay for all of them.

Caveats:
 * consensus is running concurrently to other processes, so things could
   change while verifyBlock() is iterating over transactions, this will be
   mitigated in subsequent commits

Improves TPS value for single node by at least 11%.

Fixes #667, fixes #668.
2020-08-20 18:50:18 +03:00
Roman Khimov
c7032022f8 core: don't search through the whole DAO in isTxStillRelevant
New transactions are added to the chain with blocks. If there is no
transaction X at height N in DAO, it could only be added with block N+1, so
it has to be present there. Therefore we can replace `dao.HasTransaction()`
check with a search through in-block transactions. HasTransaction() is nasty
in that it may add useless load the DB and this code is being run with a big
Blockchain lock held, so we don't want to be delayed here at all.

Improves single-node TPS by ~2%.
2020-08-20 18:50:18 +03:00
Roman Khimov
3a379e4f3e core: don't reverify mempooled transactions in AddBlock
The end effect is almost as if `VerifyTransactions: false` was set in the
config, but without actually compromising the guarantees provided by it.

It almost doubles performance for single-mode benchmarks and makes block
processing smoother (more smaller blocks are being produced).
2020-08-19 15:16:19 +03:00
Roman Khimov
eb8122837e core: fix failing TestGetTransaction
3677cc0e2 ruined it:
    TestGetTransaction: blockchain_test.go:413:
                Error Trace:    blockchain_test.go:413
                Error:          Not equal:
                                expected: 467
                                actual  : 399
                Test:           TestGetTransaction
2020-08-19 13:22:31 +03:00
Roman Khimov
6dffd4967a
Merge pull request #1335 from nspcc-dev/tests/binary
Update binary test data
2020-08-19 11:33:47 +03:00
Evgenii Stratonikov
3677cc0e24 core: update GetTransaction test 2020-08-19 09:41:11 +03:00
Roman Khimov
d0c29f52c9 core: verification script must return exactly one value
C# node is quite picky as it expects there to be exactly one value returned,
but our testchain actually adds 4 signatures for multisig cases instead of 3
which makes it technically incompatible with C# node.
2020-08-17 22:02:15 +03:00
Roman Khimov
8e619cc671
Merge pull request #1318 from nspcc-dev/fix/verifytests
Add tests for `verifyTx` and `verifyHeader`
2020-08-14 17:24:03 +03:00
Evgenii Stratonikov
7854dcfd8f core: replace interop names with named constants 2020-08-14 14:21:54 +03:00
Evgenii Stratonikov
cadebdfc19 core: add tests for (*Blockchain).verifyHashAgainstScript 2020-08-14 09:40:36 +03:00
Evgenii Stratonikov
1eb9a4c6c6 core: allow to use verification contracts
In NEO3 we can't just appcall hash, as verification script has no access
to state. Instead we use `verify` method of an arbitrary contract.
2020-08-14 09:40:34 +03:00
Evgenii Stratonikov
7f2a931fb6 core: add tests for (*Blockchain).verifyHeader 2020-08-14 09:37:32 +03:00
Evgenii Stratonikov
e8cf4d96ce core: add tests for (*Blockchain).verifyTx 2020-08-14 09:37:30 +03:00
Anna Shaleva
90825efa16 core: move transaction's sender to cosigners
Closes #1184

Ported changes from https://github.com/neo-project/neo/pull/1752
2020-08-04 17:33:50 +03:00
Anna Shaleva
70ef733ce7 core, vm: store VMState as byte instead of string
Part of #1183
2020-07-29 10:14:08 +03:00
Anna Shaleva
abe3c94b95 core: use big.Int to store NEP5 balances
closes #1133
2020-07-09 13:26:39 +03:00
Roman Khimov
cab437284d core: fix TestGetClaimable occasional failures 2020-06-27 12:36:43 +03:00
Roman Khimov
efa8ae5be4 core: fix TestSubscriptions occasional failures
panic: Log in goroutine after TestSubscriptions has completed

goroutine 1079 [running]:
testing.(*common).logDepth(0xc00057a100, 0xc00039e210, 0xa4, 0x3)
	/usr/local/go/src/testing/testing.go:634 +0x51a
testing.(*common).log(...)
	/usr/local/go/src/testing/testing.go:614
testing.(*common).Logf(0xc00057a100, 0xe32eaa, 0x2, 0xc0009560e0, 0x1, 0x1)
	/usr/local/go/src/testing/testing.go:649 +0x91
go.uber.org/zap/zaptest.testingWriter.Write(0xf64120, 0xc00057a100, 0x0, 0xc0003fe400, 0xa5, 0x400, 0xc000958e40, 0xc0009560d0, 0xc000958e60)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x120
go.uber.org/zap/zapcore.(*ioCore).Write(0xc0005cd050, 0x0, 0xbfb54ffc0626aba2, 0x916de700, 0x1485500, 0x0, 0x0, 0xe43fb0, 0x1c, 0x0, ...)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c5
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000102d10, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e8
go.uber.org/zap.(*Logger).Info(0xc00035eba0, 0xe43fb0, 0x1c, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:187 +0x96
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).persist(0xc00000cb40, 0xc00017c2c0, 0xbe8a00)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:839 +0x6c9
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run.func2(0xc00000cb40, 0xc0005c6c30)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:302 +0x54
created by github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:301 +0x25d
FAIL	github.com/nspcc-dev/neo-go/pkg/core	2.463s
2020-06-25 19:22:38 +03:00
Evgenii Stratonikov
295d8fc70e core: save application logs for native persist 2020-06-18 15:32:27 +03:00
Roman Khimov
b483c38593 block/transaction: add network magic into the hash
We make it explicit in the appropriate Block/Transaction structures, not via a
singleton as C# node does. I think this approach has a bit more potential and
allows better packages reuse for different purposes.
2020-06-18 12:39:50 +03:00
Evgenii Stratonikov
0472a0b0b1 core: move Neo.Runtime/Enumerator/Iterator.* interops to System.* 2020-06-10 12:13:35 +03:00
Roman Khimov
21efccd300 transaction: remove type field, set Version to 0
Two changes being done here, because they require a lot of updates to
tests. Now we're back into version 0 and we only have one type of
transaction.

It also removes GetType and GetScript interops, both are obsolete in Neo 3.
2020-06-05 19:20:16 +03:00
Roman Khimov
f445f7c602 transaction: drop Contract transaction type 2020-06-05 19:20:16 +03:00
Roman Khimov
3d18f09def core: fix CalculateClaimable for NEP5 NEO
It's not stored as Fixed8, so calculations need to be adjusted for that.
2020-06-05 19:20:16 +03:00
Roman Khimov
39dfebccc4 core: no longer treat sysfee as claimable
As it's not on Neo 3, it just gets burned and that's it. Only network fee is
being redistributed to CNs.
2020-06-05 19:20:16 +03:00
Roman Khimov
50ed4c5967 core: don't return an error from CalculateClaimable
As it never returns one.
2020-06-05 19:20:16 +03:00
Roman Khimov
3ca71d5e8d core: add Blockchain event subscription mechanism
A deep internal part of #895. Blockchainer interface is also extended for
various uses of these methods.
2020-05-25 00:27:39 +03:00
Anna Shaleva
29d321b5e1 *: drop miner transaction
1. Completely remove miner transaction

2. Change validation rule for block: block without transactions is
valid.
2020-04-27 17:57:37 +03:00
Anna Shaleva
2b5c14160c core: add sender field to transaction
closes #860
2020-04-20 17:21:28 +03:00
Anna Shaleva
5fa11987d2 core: add validUntilBlock field to transaction
1. closes #841

2. Commented out test cases where binary transaction are used.
These test cases marked with `TODO NEO3.0: Update binary` and need to be
updated.

3. Updated other tests.

4. Added cache to calculateValidUntilBlock() RPC-client method.
2020-04-15 13:46:43 +03:00
Anna Shaleva
65503aa9b4 core: add nonce field to transaction
1. Closes #840: added Nonce field to transaction.Transaction and
removed Nonce field from transaction.MinerTx

2. Added following methods to different tx types:
  - NewMinerTx()
  - NewMinerTxWithNonce(...)
  - NewEnrollmentTx(...)
  - NewIssueTx()
  - NewPublishTx(...)
  - NewRegisterTx(...)
  - NewStateTx(...)
in order to avoid code duplication when new transaction is created.

3. Commented out test cases where binary transaction/block are used.
These test cases marked with `TODO NEO3.0: Update binary` and need to be
updated.

4. Updated other tests

5. Added constant Nonce to GoveringTockenTx, UtilityTokenTx and genesis
block to avoid data variability. Also marked with TODO.
2020-04-14 16:19:41 +03:00
Evgenii Stratonikov
030b7754ad core: move DAO to a separate package 2020-04-08 08:38:44 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Roman Khimov
c3e73c5b7d core: don't reverify stale headers in addHeader
During networked synchronization we expect there to be a lot of duplicate
headers received and it makes no sense for us reverifying them.
2020-03-03 15:34:03 +03:00
Evgenii Stratonikov
1b9968df67 core: optimize CalculateClaimable()
Because accumulated system fee is stored for every block,
it is easy to calculate sum with just to reads.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
7095ec6c51 core: implement (*Blockchain).CalculateClaimable
Calculating amount of GAS that can be claimed is required
for getclaimable RPC.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
a3dacd3b74 tests: replace t.Fatal with require where possible
This makes tests less verbose and unifies the style
they are written in.
2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
66f96e3f32 core: shutdown Blockchain gracefully in tests 2020-03-02 17:22:27 +03:00
Evgenii Stratonikov
3a5224344e core: verify headers in AddHeaders()
Headers can be malformed so public methods should verify them
before adding.
2020-03-02 17:22:26 +03:00
Evgenii Stratonikov
357bb4ce41 core: get rid of global variables in tests
It can lead to unnecessary race conditions and is just
a bad practice.
2020-03-02 17:04:08 +03:00
Evgenii Stratonikov
8ca94e23c8 core: replace makeBlocks() with addBlocks() in tests
This simplifies tests a bit.
2020-03-02 17:04:08 +03:00
Evgenii Stratonikov
63c56cca5c core: refactor out Block, BlockBase and Header structs
See #597.
2020-01-16 10:16:24 +03:00
Evgenii Stratonikov
714c466c2c core: add ScriptFromWitness function
Extracting verification script from witness became a common task.
This commit adds such a possibility.
2019-12-26 10:49:56 +03:00
Vsevolod Brekelov
ec17654986 core: refactoring blockchain state and storage
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
2019-12-11 13:05:31 +03:00
Evgenii Stratonikov
7179e4ba9f util: add LE suffix to Uint256 methods 2019-12-06 12:16:55 +03:00
Roman Khimov
b05754deac core: add Close() to blockchainer, implement it to properly close chain
Before it the deferred function in Run() was actually never able to properly
close the Store, so we weren't synching the latest state to the disk.
2019-11-08 12:19:54 +03:00
Roman Khimov
07c2105aa5 core: log values from the store in persist()
We're about stored values here, so print those, which avoids blocking in
bc.HeaderHeight() and removes duplication between blockHeight and
persistedHeight. Fixes saving the blockchain on exit (deferred function in
Run() blocked in persist()).

Test modification was required because storeBlocks() doesn't actually save
headers and thus TestGetTransaction started to fail on persist().
2019-10-21 14:18:09 +03:00
Roman Khimov
70407f0c19 core: remove unused context parameter from persist() 2019-10-21 14:18:09 +03:00
Roman Khimov
fc0031e5aa core: move write caching layer into MemCacheStore
Simplify Blockchain and associated functions, deduplicate code, fix Get() and
Seek() implementations.
2019-10-16 17:33:45 +03:00
Roman Khimov
a6610ba082 core: verify blocks, fix #12
This adds the following verifications:
 * merkleroot check
 * index check
 * timestamp check
 * witnesses verification

VerifyWitnesses is also renamed to verifyTxWitnesses here to not confuse it
with verifyBlockWitnesse and to hide it from external access (no users at the
moment).
2019-10-15 18:58:17 +03:00