Commit graph

64 commits

Author SHA1 Message Date
Evgenii Stratonikov
7d91a3a89e pkg: move internal/ package to the root directory
This way we can use it in scripts and cli.
2020-11-24 16:39:56 +03:00
Roman Khimov
9142906abe
Merge pull request #1539 from nspcc-dev/core/fix_mempool_test
core: fix failing mempool test
2020-11-12 15:48:28 +03:00
Anna Shaleva
a84e4c1e89 core: make some checks in TestMempoolAddRemoveConflicts non-critical
It will help to investigate test failures.
2020-11-12 15:14:32 +03:00
Anna Shaleva
54e6bcad12 core: fix failing mempool test
There might be a case when identical nonces are generated for tx6, tx7 or
tx8 (they are not in mempool, so each of them pass mempool-presence
check). In this case test fails due to the lack of hashes into mp.conflicts
map (two of tx6, tx7 or tx8 have identical hashes) with the following
error:

```
=== RUN   TestMempoolAddRemoveConflicts
--- FAIL: TestMempoolAddRemoveConflicts (0.00s)
    mem_pool_test.go:376:
        	Error Trace:	mem_pool_test.go:376
        	Error:      	Not equal:
        	            	expected: 4
        	            	actual  : 3
        	Test:       	TestMempoolAddRemoveConflicts
```

Fixed by maling the nonce non-random.
2020-11-12 15:05:01 +03:00
Evgenii Stratonikov
3e5b84348d network: retransmit stale transactions 2020-11-12 13:51:44 +03:00
Anna Shaleva
ec63d5c456 core: add conflicts attribute
Close #1491
2020-10-29 10:57:31 +03:00
Roman Khimov
83fc38ae3a mempool: don't create new big.Int in tryAddSendersFee() if possible
Do a little less allocations.
2020-09-10 15:35:19 +03:00
Roman Khimov
fc7ea6217d mempool: avoid reassigning utilityBalanceAndFees value
It's not needed, we're either creating a new one and assigning it 6 lines
above or we're changing already existing big.Int via a pointer, so no update
is needed.
2020-09-10 15:20:04 +03:00
Roman Khimov
fe1f1d19be mempool: store only pointer in the verifiedMap
It's only used for presence checks, there is no need for metadata here.
2020-09-09 20:46:31 +03:00
Roman Khimov
9591d64e53 mempool: don't sort items by hash
There is nothing requiring us to do so. It also is bad because it allows for
new transaction to replace some already existing one with the same fee
parameters just because it has "better" hash.

But the other thing is that for transactions with equal fees it's always
better for us to append them to the end of the list, instead of inserting them
in the middle, so this change allows to reduce slice item movements and gain
some 6-7% increase for single-node TPS.
2020-09-09 20:46:31 +03:00
Roman Khimov
5df726db68 mempool: replace timeStamp with blockStamp
Time is not really relevant for us here and we don't use this timestamp in any
way. Yet it occupies 24 bytes and we do two clock_gettime calls to get it.

Replace it with blockStamp which is going to be used in the future for
transaction retransmissions.

It allows to improve single-node TPS by another 3%.
2020-09-09 20:46:31 +03:00
Roman Khimov
0ea8c8ba67 mempool: drop a level of indirection
`item` is so small that it makes no sense bothering memory allocator with
every instance of it.
2020-09-09 20:46:31 +03:00
Evgenii Stratonikov
890c752b3b mempool: add tests for items ordering 2020-08-23 09:39:46 +03:00
Evgenii Stratonikov
2661ebd295 transaction: add HighPriority attribute
HighPriority attributes specifies that transaction was
signed by a committee.
2020-08-23 09:39:46 +03:00
Roman Khimov
0e086d61ac mempool: store feeSum as big.Int
Prevent (very) potential overflow.
2020-08-20 19:06:59 +03:00
Roman Khimov
55b2cbb74d core: refactor and improve verification and pooling
Now we have VerifyTx() and PoolTx() APIs that either verify transaction in
isolation or verify it against the mempool (either the primary one or the one
given) and then add it there. There is no possibility to check against the
mempool, but not add a transaction to it, but I doubt we really need it.

It allows to remove some duplication between old PoolTx and verifyTx where
they both tried to check transaction against mempool (verifying first and then
adding it). It also saves us utility token balance check because it's done by
the mempool anyway and we no longer need to do that explicitly in verifyTx.

It makes AddBlock() and verifyBlock() transaction's checks more correct,
because previously they could miss that even though sender S has enough
balance to pay for A, B or C, he can't pay for all of them.

Caveats:
 * consensus is running concurrently to other processes, so things could
   change while verifyBlock() is iterating over transactions, this will be
   mitigated in subsequent commits

Improves TPS value for single node by at least 11%.

Fixes #667, fixes #668.
2020-08-20 18:50:18 +03:00
Roman Khimov
0d8cc437fe mempool: swap checks in Add, fail fast
Checking for duplicates is easier than checking the balance, so it should be
done first.
2020-08-20 18:50:18 +03:00
Roman Khimov
e998c102ca mempool: rename NewMemPool into New
"mempool" is a package name already.
2020-08-20 18:50:18 +03:00
Roman Khimov
fb70c82157 transaction: there are no valid attributes defined for preview3 2020-08-11 21:10:12 +03:00
Anna Shaleva
90825efa16 core: move transaction's sender to cosigners
Closes #1184

Ported changes from https://github.com/neo-project/neo/pull/1752
2020-08-04 17:33:50 +03:00
Roman Khimov
fedcc6b6fc transaction: drop old attributes
They're not supported in Neo 3. Also change data encoding to base64 following
Neo 3 changes.
2020-07-10 20:40:27 +03:00
Anna Shaleva
789ee2d3c1 core: do not update mempool while verifying tx
Closes #1168
2020-07-09 20:45:59 +03:00
Anna Shaleva
4be1009def core: refactor checkBalanceAndUpdate 2020-07-09 18:27:20 +03:00
Anna Shaleva
abe3c94b95 core: use big.Int to store NEP5 balances
closes #1133
2020-07-09 13:26:39 +03:00
Anna Shaleva
73b630db9b *: switch from fixed8 to int64
Follow C# implementation, we have to marshall JSON Fixed8 fields without
taking into account decimals.
2020-06-29 21:39:27 +03:00
Anna Shaleva
9f11a55bd6 core: add policy check to mempool
After block was stored it's possible to have new FeePerByte constraint,
so we should remove all transactions which do not meet this requirement.
Also caching of FeePerByte was added in order not to re-verify
transactions each time mempool needs to be updated.
2020-06-24 07:58:09 +03:00
Evgenii Stratonikov
5354352d63 core: remove transaction priority
There is no such thing as high/low priority transactions, as there are
no free transactions anymore and they are ordered by fees contained
in transaction itself.

Closes #1063.
2020-06-18 22:44:10 +03:00
Roman Khimov
b483c38593 block/transaction: add network magic into the hash
We make it explicit in the appropriate Block/Transaction structures, not via a
singleton as C# node does. I think this approach has a bit more potential and
allows better packages reuse for different purposes.
2020-06-18 12:39:50 +03:00
Roman Khimov
63f4f34659 mempool: drop TxWithFee type
All the fees are in transaction, this makes no sense.
2020-06-05 19:21:40 +03:00
Roman Khimov
709146f295 transaction: drop Inputs and Outputs, forget UTXO 2020-06-05 19:20:16 +03:00
Roman Khimov
21efccd300 transaction: remove type field, set Version to 0
Two changes being done here, because they require a lot of updates to
tests. Now we're back into version 0 and we only have one type of
transaction.

It also removes GetType and GetScript interops, both are obsolete in Neo 3.
2020-06-05 19:20:16 +03:00
Roman Khimov
169c5ae775 transaction: drop Issue TX support 2020-06-05 19:20:16 +03:00
Roman Khimov
f445f7c602 transaction: drop Contract transaction type 2020-06-05 19:20:16 +03:00
Roman Khimov
dfc7a9bfd1 transaction: drop Claim TX type 2020-06-05 19:20:16 +03:00
Anna Shaleva
1afd630169 core: add senders' fees monitor to mempool
In order to simplify maintainance of overall current senders` fees
in the mempool we are to keep them in a separate map.
2020-05-20 23:34:25 +03:00
Anna Shaleva
7acf5b2841 core: add SystemFee and NetworkFee to transaction
closes #831
2020-05-20 23:26:48 +03:00
Anna Shaleva
29d321b5e1 *: drop miner transaction
1. Completely remove miner transaction

2. Change validation rule for block: block without transactions is
valid.
2020-04-27 17:57:37 +03:00
Anna Shaleva
65503aa9b4 core: add nonce field to transaction
1. Closes #840: added Nonce field to transaction.Transaction and
removed Nonce field from transaction.MinerTx

2. Added following methods to different tx types:
  - NewMinerTx()
  - NewMinerTxWithNonce(...)
  - NewEnrollmentTx(...)
  - NewIssueTx()
  - NewPublishTx(...)
  - NewRegisterTx(...)
  - NewStateTx(...)
in order to avoid code duplication when new transaction is created.

3. Commented out test cases where binary transaction/block are used.
These test cases marked with `TODO NEO3.0: Update binary` and need to be
updated.

4. Updated other tests

5. Added constant Nonce to GoveringTockenTx, UtilityTokenTx and genesis
block to avoid data variability. Also marked with TODO.
2020-04-14 16:19:41 +03:00
Roman Khimov
8400f0add2 mempool: gofmt -s 2020-03-17 16:47:51 +03:00
Roman Khimov
ec76a0bf15 mempool: disallow more than one issue tx at once
Technically they could conflict for available asset amount, but as they're
very rare (and even can be considered obsolete) we can simplify this check.
2020-03-17 11:40:01 +03:00
Roman Khimov
e41d434a49 *: move all packages from CityOfZion to nspcc-dev 2020-03-03 17:21:42 +03:00
Evgenii Stratonikov
cbf26bac83 mempool: do not allocate new slice for verified transactions
Because transactions a iterated in an increasing order,
we can filter slice in-place.
2020-03-02 10:38:27 +03:00
Roman Khimov
3e2b490025 mempool: rework inputs verification, check Claim txes
Use more efficient check for Input and also check Claims to avoid double
claiming.
2020-02-27 13:43:35 +03:00
Roman Khimov
f329de73e8 mempool: fix Remove bug (dropping wrong tx!)
Uncovered by TestMemPoolAddRemoveWithInputsAndClaims (to be added later)
2020-02-27 13:43:35 +03:00
Roman Khimov
22f5667530 mempool: return fee along with tx when requesting tx
Users of GetVerifiedTransactions() don't want to recalculate tx fee and it's
nice to have it returned from TryGetValue() also sometimes.
2020-02-19 12:19:02 +03:00
Roman Khimov
06daeb44f3 core: make IsLowPriority work with pre-calculated fee
Don't recalculate it again and again.
2020-02-19 12:19:02 +03:00
Roman Khimov
18695e660b mempool: drop RemoveOverCapacity(), handle it right in the Add()
Simplifies things a lot and removes useless code.
2020-02-06 17:50:11 +03:00
Roman Khimov
684cbf5bac mempool: make it almost 100% test-covered 2020-02-06 17:50:11 +03:00
Roman Khimov
6672f4b26f mempool: iterate over slice in GetVerifiedTransactions()
It's more efficient and keeps transactions sorted by priority.
2020-02-06 16:59:20 +03:00
Roman Khimov
e97396e56c mempool: test addition of conflicting tx to the pool 2020-02-06 15:57:20 +03:00