Commit graph

549 commits

Author SHA1 Message Date
Anna Shaleva
4a8259caea rpc: adjust getrawtransaction and gettransactionheight RPC call
We should not return transaction metadata from `getrawtransaction` in case
transaction is not in the mempool. Height shouldn't be returned from
`gettransactionheight` in case transaction is in the mempool.
2020-12-31 12:44:07 +03:00
Anna Shaleva
66a64dd4c9 core: fix bug with mempool.verifiedMap
Changes ported from #1621.
2020-12-16 14:14:09 +03:00
Roman Khimov
64d0876fc0
Merge pull request #1503 from nspcc-dev/mpt/refcount
Implement reference counting in MPT
2020-11-18 14:20:42 +03:00
Evgenii Stratonikov
0ffffb93d7 mpt: implement reference counting 2020-11-18 12:16:05 +03:00
Evgenii Stratonikov
d93ddfda10 network: retransmit stale transactions 2020-11-11 15:51:36 +03:00
Roman Khimov
06f3c34981 mempool: replace timeStamp with blockStamp
Time is not really relevant for us here and we don't use this timestamp in any
way. Yet it occupies 24 bytes and we do two clock_gettime calls to get it.

Replace it with blockStamp which is going to be used in the future for
transaction retransmissions.

It allows to improve single-node TPS by another 3%.
2020-11-11 15:48:13 +03:00
Evgenii Stratonikov
f3abbf34e3 mpt: fill cached fields when getting node from store
Node which has been got from store shouldn't be flushed again.
2020-11-11 13:22:53 +03:00
Evgenii Stratonikov
df1792c80b mpt: export func for decoding node with type
`NodeObject` can contain auxilliary fields and shouldn't be used from outside.
2020-11-11 13:22:53 +03:00
Roman Khimov
136e4b5886 core: move NotificationEvent->NEP5Transfer conversion to state
We'll need it as a separate function and it's cleaner this way.
2020-10-29 22:10:40 +03:00
Roman Khimov
14c39b2b26 storage: fix Get for BoltDB, fix #1482 2020-10-13 19:21:57 +03:00
Anna Shaleva
168366f33e core: copy storage item key in simple.GetStorageItems
Close #1468.

We should copy the key to avoid bytes substitution. Otherwise there's a
chance that at the end of dao.Store.Seek(...) execution some keys won't
be the same as the original keys found inside saveToMap function.
2020-10-10 12:24:08 +03:00
Roman Khimov
4f7fa732cf core|config: make max(free)txperblock height-dependent
See neo-project/neo-modules#336.
2020-09-14 22:57:03 +03:00
Roman Khimov
e14ba6c855 core: fix policy-enforcing network fee check
0.001 should be added to the 'extra' value as per https://neo.org/blog/details/4148
2020-09-11 16:52:58 +03:00
Roman Khimov
0ef65d1bb9 config: add minimum network fee setting
Follow neo-project/neo#1901.
2020-09-11 15:41:08 +03:00
Roman Khimov
8865d5b2c5
Merge pull request #1395 from nspcc-dev/add-limits-to-gettransfers
Add limits to get(nep5|utxo)transfers
2020-09-09 15:45:09 +03:00
Roman Khimov
5b05081525 config: allow configuring free gas depending on height
And update default mainnet/testnet settings. Follow neo-project/neo#1888 and
neo-project/neo-node#656.
2020-09-08 18:52:53 +03:00
Roman Khimov
b310ac051b core/rpc: add continue flag to iterating functions
Most of the time we don't need to get all transfers from the DB and
deserialize them.
2020-09-08 15:38:33 +03:00
Roman Khimov
d3e415d3bd core/state: reverse the order of ForEachTransfer
When using limits we're usually concerned about the most recent
transfers. Returning 3 transfers from the middle of the chain isn't very
helpful.
2020-09-08 12:57:45 +03:00
Roman Khimov
9aee3e5a34 transaction: set output position when decoding
We had a kludge for getrawtransaction to set this useless field, but
7e371588a7 broke it. Add it right into the
decoder now to fix all types of queries (getblock/getrawtransaction/gettxout).

Fixes #1392.
2020-09-07 15:37:57 +03:00
Roman Khimov
39897e811d core: fix NEO UTXO tracking, drop Fixed8 multiplier
When this vout:
         {
            "n" : 0,
            "address" : "ASkbjwosE3aKyGtDQkEgqhNq3Zpv8Xkt14",
            "asset" : "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
            "value" : "606"
         },

Becomes this transfer:
               {
                  "txid" : "0x192ab8e422aed6ac868cb329d6f9af20964134b591908c736d32a2fd8a51d7bf",
                  "amount" : "60600000000",
                  "block_index" : 6113653,
                  "timestamp" : 1599199074
               }

Something is wrong here.
2020-09-04 09:09:35 +03:00
Roman Khimov
43eb670d45 mpt: fix comment typo 2020-08-11 21:55:51 +03:00
Roman Khimov
ffbdcb202f
Merge pull request #1288 from nspcc-dev/fix/utxo
rpc: adjust `getutxotransfers` RPC
2020-08-07 19:08:46 +03:00
Evgenii Stratonikov
57a325b3d5 rpc: adjust getutxotransfers RPC 2020-08-07 18:12:40 +03:00
Anna Shaleva
4715efd531 core, rpc: invoke script with hashes for verifying
Closes #1275
2020-08-07 13:18:55 +03:00
Evgenii Stratonikov
7bd4488ff9 core: do not store NEP5 transfer log in memory
Traversing transfer log instead of accumulating and returning it
is faster and takes less memory.
2020-08-06 14:22:30 +03:00
Evgenii Stratonikov
022fb04077 rpc: implement getutxotransfers RPC 2020-08-06 14:16:28 +03:00
Evgenii Stratonikov
407e348cd5 core: save UTXO transfer info 2020-08-06 14:03:21 +03:00
Evgenii Stratonikov
e4fcd90b6d state: make NEP5Transfer log more generic 2020-08-04 16:57:36 +03:00
Evgenii Stratonikov
49f9c4ad7e dao: rename transfers to nep5transfers 2020-08-04 15:50:09 +03:00
Evgenii Stratonikov
c6d33c5841 core: save NEP5 transfer notify index
TransferNotifyIndex is the index of transfer event in the list
of all transfers in a transaction.
2020-08-03 10:58:23 +03:00
Evgenii Stratonikov
f0d75afc48 rpc: provide old transfers in getnep5transfers
Return performed transfers even if contract was migrated.
2020-07-24 14:39:16 +03:00
Evgenii Stratonikov
7cd1bca1e1 core,dao: save contract metadata on migration
After contract is migrated there is no way to retrieve it's state.
This commit implements some metadata for NEP5 contracts, so that
values important for diplaying transfer log aren't lost.
2020-07-24 14:38:10 +03:00
Roman Khimov
6d32751292 core: fix stateroot height update for testnet
When synchronizing with stateroot-enabled network from genesis and if
stateroot is not enabled in block zero we were failing to update state height
because initially it's updated with a jump from 0 to StateRootEnableIndex, so
we should allow that to happen to have correct state height.
2020-07-18 10:17:17 +03:00
Roman Khimov
1ab4f81fc3 dao: migrate nep5 balances with the contract
Fixes #1144. It's quite simple approach, we just update balance info right
upon contract migration. It will slow down migration transactions, but it
takes about 1-2 seconds to Seek through balances at mainnet's 3.8M, so the
approach should still work good enough. The other idea was to make lazy
updates (maintaining contract migration map), but it's more complicated to
implement (and implies that a balance get might also do a write).

There also is a concern about memory usage, it can give a spike of some tens
of megabytes, but that also is considered to be acceptable.
2020-07-07 19:55:55 +03:00
Roman Khimov
25b742d24e cache: prevent TestRelayCache_Add failures
random.Bytes() might be not random enough.
2020-06-26 12:42:12 +03:00
Roman Khimov
c489f975d4 core: fix TestSubscriptions occasional failures
panic: Log in goroutine after TestSubscriptions has completed

goroutine 1079 [running]:
testing.(*common).logDepth(0xc00057a100, 0xc00039e210, 0xa4, 0x3)
	/usr/local/go/src/testing/testing.go:634 +0x51a
testing.(*common).log(...)
	/usr/local/go/src/testing/testing.go:614
testing.(*common).Logf(0xc00057a100, 0xe32eaa, 0x2, 0xc0009560e0, 0x1, 0x1)
	/usr/local/go/src/testing/testing.go:649 +0x91
go.uber.org/zap/zaptest.testingWriter.Write(0xf64120, 0xc00057a100, 0x0, 0xc0003fe400, 0xa5, 0x400, 0xc000958e40, 0xc0009560d0, 0xc000958e60)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zaptest/logger.go:130 +0x120
go.uber.org/zap/zapcore.(*ioCore).Write(0xc0005cd050, 0x0, 0xbfb54ffc0626aba2, 0x916de700, 0x1485500, 0x0, 0x0, 0xe43fb0, 0x1c, 0x0, ...)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/core.go:90 +0x1c5
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000102d10, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/zapcore/entry.go:215 +0x1e8
go.uber.org/zap.(*Logger).Info(0xc00035eba0, 0xe43fb0, 0x1c, 0xc00039a000, 0x5, 0x5)
	/go/pkg/mod/go.uber.org/zap@v1.10.0/logger.go:187 +0x96
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).persist(0xc00000cb40, 0xc00017c2c0, 0xbe8a00)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:839 +0x6c9
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run.func2(0xc00000cb40, 0xc0005c6c30)
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:302 +0x54
created by github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).Run
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/blockchain.go:301 +0x25d
FAIL	github.com/nspcc-dev/neo-go/pkg/core	2.463s
2020-06-25 19:41:17 +03:00
Evgenii Stratonikov
2b21102c14 core,dao: use MPT only if it is enabled in config 2020-06-24 16:13:27 +03:00
Roman Khimov
25fdc62203 core: add state height to prometheus metrics 2020-06-24 14:47:08 +03:00
Roman Khimov
ea17793aee core: fix unconditional MPT collapsing
It should depend on EnableStateRoot.
2020-06-24 14:41:57 +03:00
Evgenii Stratonikov
9ee9cb8e39 core: provide key recovery interops only if neox is enabled 2020-06-24 12:37:27 +03:00
Evgenii Stratonikov
d128b55dbf *: add config flag for enabling state root feature 2020-06-24 12:37:27 +03:00
Evgenii Stratonikov
f8051da0bd core: extend Blockchainer interface with StateHeight()
Allow to query current verified state root height.
2020-06-22 10:53:13 +03:00
Evgenii Stratonikov
f665843887 core: update verified state root height 2020-06-22 10:53:12 +03:00
Roman Khimov
7d786fac79 core: fix key recovery interops return value
It's uncompressed coordinate, both X and Y, not just X. Fix #1043.
2020-06-10 19:20:57 +03:00
Roman Khimov
76f71ab1ef
Merge pull request #1012 from nspcc-dev/feature/mptrpc
rpc: implement MPT-related RPC (2.x)
2020-06-09 17:25:55 +03:00
Evgenii Stratonikov
1fd7938fd8 network: process state roots properly 2020-06-08 17:31:59 +03:00
Evgenii Stratonikov
8cbbddddaf rpc: implement getproof RPC 2020-06-08 17:31:58 +03:00
Roman Khimov
6437f4b32e core: respect MaxFreeTransactionsPerBlock setting
Fix #1019.
2020-06-08 12:35:39 +03:00
Roman Khimov
a1c4d7ce26 core: do MPT compaction every once in a while
We need to compact our in-memory MPT from time to time, otherwise it quickly
fills up all available memory. This raises two obvious quesions --- when to do
that and to what level do that.

As for 'when', I think it's quite easy to use our regular persistence interval
as an anchor (and it also frees up some memory), but we can't do that in the
persistence routine itself because of synchronization issues (adding some
synchronization primitives would add some cost that I'd also like to avoid),
so do it indirectly by comparing persisted and current height in `storeBlock`.

Choosing proper level is another problem, but if we're to roughly estimate one
full branch node to use 1K of memory (usually it's way less than that) then we
can easily store 1K of these nodes and that gives us a depth of 10 for our
trie.
2020-06-04 17:25:57 +03:00
Roman Khimov
e3af560d11 dao: optimize storage cache flushing
Items were serialized several times if there were several successful
transactions in a block, prevent that by using State field as a bitfield (as
it almost was intended to) and adding one more bit. It also eliminates useless
duplicate MPT traversions.

Confirmed to not break storage changes up to 3.3M on testnet.
2020-06-04 17:21:58 +03:00