Roman Khimov
2e58a14978
core: improve documentation a little
2020-05-12 17:20:31 +03:00
Roman Khimov
943d435cd2
core: ensure we produce correct blocks for tests
...
Blocks must have at least one transaction and we should check for correct
merkle root generation.
2020-05-07 21:46:28 +03:00
Anna Shaleva
54cdfe4a23
storage: add support of BadgerDB
...
closes #820
2020-04-09 13:55:59 +03:00
Evgenii Stratonikov
ee0ba9b1b4
core: make SpawnVM a method of context
...
spawnVMWithInterops is rather long too type and
it doesn't use Blockchain in any way.
2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
a71cd0961e
core/dao: remove unnecessary slice type
...
It is used only once, so a simple `sort.Slice`
invocation will suffice.
2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
030b7754ad
core: move DAO to a separate package
2020-04-08 08:38:44 +03:00
Roman Khimov
9997661998
Merge pull request #821 from nspcc-dev/fix-cache-propagation-to-invocations
...
core: wrap cached dao properly, don't miss cached data
2020-04-06 11:52:41 +03:00
Evgenii Stratonikov
1fcc019bf3
rpc: update test chain
...
Also provide info for getblockheader RPC while
generating test chain.
2020-04-06 09:31:09 +03:00
Roman Khimov
5f09381cf4
core: wrap cached dao properly, don't miss cached data
...
Fixes #817 where invoked contract missed updated account information because
it got it one layer below cachedDao used to process the block.
2020-04-03 10:15:11 +03:00
Roman Khimov
2d0ad30fcf
vm: rework Map with internal slice representation
...
Which makes iterating over map stable which is important for serialization and
and even fixes occasional test failures. We use the same ordering here as
NEO 3.0 uses, but it should also be fine for NEO 2.0 because it has no
defined order.
2020-04-01 19:33:53 +03:00
Roman Khimov
9b5dab57e8
Merge pull request #810 from nspcc-dev/refactor/config
...
config: move config.go out of config
2020-04-01 10:29:57 +03:00
Anna Shaleva
5a984fdf88
config: move config.go out of config/
...
closes #423
2020-03-31 17:55:59 +03:00
Roman Khimov
e41853d0a4
Merge pull request #789 from nspcc-dev/fix/tx_specific_data_marshalling
...
rpc: fix marshalling of type-specific tx data
2020-03-30 16:10:25 +03:00
Anna Shaleva
9c09ad9c89
rpc: fix marshalling of type-specific tx data
...
closes #585
2020-03-30 15:48:50 +03:00
Roman Khimov
237de73dec
Merge pull request #807 from nspcc-dev/optimize-memcached-persist
...
storage: optimize (*MemCachedStore).Persist for memory-backed ps
2020-03-29 13:41:21 +03:00
Roman Khimov
4758de71ec
storage: optimize (*MemCachedStore).Persist for memory-backed ps
...
Most of the time it's persisted into the MemoryStore or MemCachedStore, when
that's the case there is no real need to go through the Batch mechanism as it
incurs multiple copies of the data.
Importing 1.5M mainnet blocks with verification turned off, before:
real 12m39,484s
user 20m48,300s
sys 2m25,022s
After:
real 11m15,053s
user 18m2,755s
sys 2m4,162s
So it's around 10% improvement which looks good enough.
2020-03-28 17:21:50 +03:00
Roman Khimov
c738975b7b
Merge pull request #803 from nspcc-dev/feature/appcall
...
emit: implement AppCallWithOperationAndArgs
2020-03-27 13:30:48 +03:00
Evgenii Stratonikov
db2dccf7cb
emit: implement AppCallWithOperationAndArgs
...
It is nice to have a typical task of calling contract method
with specific arguments incapsulated inside some function.
2020-03-27 11:05:36 +03:00
Evgenii Stratonikov
0036b3e52b
random: make use or random package in tests
...
Also implement Bytes/Fill routines for generating byte slices.
2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
9abda40171
testserdes: implement helpers for encode/decode routines
...
Frequently one needs to check if struct serializes/deserializes
properly. This commit implements helpers for such cases including:
1. JSON
2. io.Serializable interface
2020-03-27 10:27:46 +03:00
Evgenii Stratonikov
fcc4877f43
go.mod: update boltdb to v1.3.4
...
Also import lib as declared in it's go.mod:
via `go.etcd.io/bbolt`.
2020-03-25 17:07:54 +03:00
Evgenii Stratonikov
c992d6c518
core: reuse buffer in (*cacheddao).Persist()
...
When serializing multiple accounts, cost of a buffer grow
can become significant. This commit tries to amortize it by
reusing the same buffer in a single `Persist()` call.
2020-03-25 13:13:54 +03:00
Evgenii Stratonikov
6ed2bd63b9
core: marshal AppExecResult.Stack as an array
...
Closes #782 .
2020-03-23 16:27:47 +03:00
Roman Khimov
8400f0add2
mempool: gofmt -s
2020-03-17 16:47:51 +03:00
Roman Khimov
ec76a0bf15
mempool: disallow more than one issue tx at once
...
Technically they could conflict for available asset amount, but as they're
very rare (and even can be considered obsolete) we can simplify this check.
2020-03-17 11:40:01 +03:00
Roman Khimov
d5d0479671
core: verify results of issue transaction
...
It shouldn't try to issue more tokens than there is available.
2020-03-17 11:40:01 +03:00
Roman Khimov
10601cb375
core: add issuer hashes into the verification list for Issue TX
...
As it should be done.
2020-03-17 11:40:00 +03:00
Roman Khimov
dca637d2d0
core: add owner hash to verified list for Register TX
...
As it should be done.
2020-03-17 11:37:02 +03:00
Roman Khimov
77a799f7d0
core: add missing state tx verifications
2020-03-16 20:14:59 +03:00
Roman Khimov
6ede65610d
core: add appropriate hashes to check for State TX
...
These checks are important for proper transaction verification.
2020-03-16 19:52:28 +03:00
Evgenii Stratonikov
b193e78def
core: remove duplication from IsDoubleClaim/IsDoubleSpend
2020-03-16 14:11:19 +03:00
Evgenii Stratonikov
cdf025bf89
transaction: implement AddVerificationHash() method
2020-03-16 14:11:19 +03:00
Roman Khimov
e9429374aa
Merge pull request #756 from nspcc-dev/feature/unclaimed
...
core/state: do not unmarshal Unclaimed balances in account
2020-03-16 13:21:23 +03:00
Evgenii Stratonikov
e503d1001d
core/state: do not unmarshal Unclaimed balances in account
2020-03-16 12:19:32 +03:00
Roman Khimov
e6e8761d35
core: fix contract's state migration and don't swallow errors
...
Fixes difference in state changes at mainnet's block 2442790 because contract
migration in b4eb2dc35226e6520ee4e09a56197dff91547b50a7f57edc82930fc18c75dffc
doesn't actually transfer the storage state, it only deletes the old one.
And add an error check just in case.
2020-03-16 11:52:09 +03:00
Roman Khimov
d9a83373ed
Merge pull request #748 from nspcc-dev/feature/splitnep5
...
core: store NEP5 balances separately from account
2020-03-12 18:31:25 +03:00
Evgenii Stratonikov
ac475940a0
core: cache NEP5Transfers in cached DAO
2020-03-12 17:16:11 +03:00
Evgenii Stratonikov
6fa2a998f4
core: cache NEP5Balances in cached DAO
2020-03-12 17:06:20 +03:00
Evgenii Stratonikov
32401a567e
core: store NEP5Transfers in batches
...
This is an append-only log which is read only during some RPCs.
It is rather slow to get it from base every time we need to append to
it. This commit stores all NEP5Transfers in batches, so that
only a last batch needs to be unmarshaled during block processing.
2020-03-12 14:51:12 +03:00
Evgenii Stratonikov
3c6d9653b0
core/state: add Size() method to NEP5TransferLog
2020-03-12 12:58:30 +03:00
Evgenii Stratonikov
df2598c8dc
core: store NEP5 balances separately
...
There is no need to take and unmarshal an account structure only
to get it's NEP5 balances.
2020-03-12 12:58:28 +03:00
Roman Khimov
bbd802681e
cli: make gas parameter to deployment add gas to the base price
...
That's how it was intended to behave originally. One thing questionable here
is contract price (policy thing, basically) being moved to smartcontract
package, but it's probably fine for NEO 2.0 (as it won't change) and we'll
make something better for NEO 3.0.
2020-03-11 20:34:36 +03:00
Roman Khimov
5f1868af28
Merge pull request #743 from nspcc-dev/optimize-some-db-accesses
...
Optimize some db accesses
2020-03-11 17:33:36 +03:00
Roman Khimov
abd7855890
Merge pull request #742 from nspcc-dev/add-dynamic-appcall
...
vm: add support for dynamic invocations in APPCALL
2020-03-11 17:27:05 +03:00
Roman Khimov
8c902a7223
core: cache UnspentCoins in cachedDao
...
1.5M block import time (VerifyBlocks disabled) on AMD Ryzen 5 1600/16GB/HDD,
before:
real 159m16.551s
user 69m58.279s
sys 7m34.334s
after:
real 139m41.836s
user 67m12.477s
sys 6m19.420s
12% which is even a bit more than could be expected from inputs analysis (that
has around 10% cache hits for a block-wide cache), worth doing.
2020-03-11 12:40:02 +03:00
Roman Khimov
23464401bc
core/state: merge spent and unspent coins state, use it to store more things
...
This change reduces pressure on DB by doing the following things:
* not storing additional KV pair for SpentCoin
* storing Output right in the UnspentCoin, thus eliminating the need to get a
full transaction from DB
At the same time it makes UnspentCoin more fat and hot, but it should probably
worth it.
Also drop `GetUnspentCoinStateOrNew` as it shouldn't ever existed, UTXOs
can't come out of nowhere.
1.5M block import time (VerifyBlocks disabled) on AMD Ryzen 5 1600/16GB/HDD,
before:
real 302m9.895s
user 96m17.200s
sys 13m37.084s
after:
real 159m16.551s
user 69m58.279s
sys 7m34.334s
So it's almost two-fold which is a great improvement.
2020-03-11 12:40:02 +03:00
Roman Khimov
e1f194ea7b
core: treat state.Coin as a bitfield
...
As it was intended to.
2020-03-11 12:22:57 +03:00
Roman Khimov
377fb382aa
core: move (un)SpentCoin structs into the state package
...
As they're all about the state.
2020-03-11 12:22:52 +03:00
Roman Khimov
8318adac56
vm: add support for dynamic invocations in APPCALL
...
Fixes #740 .
2020-03-10 17:17:36 +03:00
Evgenii Stratonikov
ebf867b11d
state: add a test for NEP5Transfer size
...
It's size is used in NEP5TransferLog so we need to be
sure it reflects reality.
2020-03-10 13:02:14 +03:00
Roman Khimov
ee57b96566
Merge pull request #733 from nspcc-dev/fix-getreferences-interop-regression
...
Fix GetReferences interop regression
2020-03-10 10:27:36 +03:00
Roman Khimov
0e2bda4f21
core: drop txHash from SpentCoinState
...
It's a key for it, makes no sense storing it as data.
2020-03-09 16:58:21 +03:00
Roman Khimov
eb404ceae3
core: fix max contract description limit check
...
It differs from other parameters in C# code. Fixes #735 .
2020-03-09 14:18:51 +03:00
Roman Khimov
4587121c7f
core: fix error propagation in contract-related interops
...
Obvious bug that hides failed contract deployments.
2020-03-09 14:18:00 +03:00
Roman Khimov
4b83e9a5cd
Merge pull request #732 from nspcc-dev/feature/getvalidators
...
rpc: implement getvalidators
2020-03-07 21:43:38 +03:00
Anna Shaleva
456a2d55fd
rpc: implement getvalidators
...
closes #714
2020-03-07 18:05:40 +03:00
Roman Khimov
baeaa3dbe6
core: optimize tx verification, only get references and results once
...
Getting references requires DB access and that is expensive.
2020-03-06 20:15:01 +03:00
Roman Khimov
fe4916f691
core: simplify GetTransactionResults() a bit
...
Make less movements.
2020-03-06 19:26:39 +03:00
Roman Khimov
ced5ddbb9e
core: fix wrong references ordering in interop function
...
Broken by 9f7018503a
. Almost the same problem as
in 01082a8988
(though it is deterministic now,
just not the way the contract expects).
2020-03-06 19:20:55 +03:00
Roman Khimov
0e8ff558d1
Merge pull request #727 from nspcc-dev/fix-wrong-endian-in-interop-hashes
...
core: fix wrong endian used in interop functions
2020-03-06 19:07:28 +03:00
Roman Khimov
f8eee778f4
Merge pull request #724 from nspcc-dev/feature/submitblock
...
rpc: implement submitblock
2020-03-06 12:08:45 +03:00
Anna Shaleva
a746d8e6e6
rpc: implement submitblock RPC
...
closes #344
2020-03-06 12:03:08 +03:00
Evgenii Stratonikov
40188c5400
core: get rid of unnecessary copies
...
There is no need to keep Balances and Unclaimed slices
sorted, we need only to remove a single element.
2020-03-05 20:42:20 +03:00
Anna Shaleva
d3063c26e1
core: add custom error to blockhain.go
...
Add InvalidBlockIndex error to AddBlock func
2020-03-05 20:30:19 +03:00
Roman Khimov
a9abd3d841
core: fix wrong endian used in interop functions
...
C# uses ToArray() or UintXXX(bytes) here which interprets hashes as they
should be interpreted (BE, although they always convert to LE when converting
to String just for the fun of it). It leads to state difference for us at
block 2025204 where even though we have the same value for the key, the key
itself differs, ours:
dd2b538e2a0c1db1ae5061c15be14f916bd1e678e512ffcda6d9499d8e7fe97ee71fd6b8004583d9afe09cc4dadbd5deb63d01e061009b7cffdaa674beae0f930ebe6085af900093e5fe56b34a5c220ccdcf6efc336fc5000000000000000000000000000000000010
theirs:
dd2b538e2a0c1db1ae5061c15be14f916bd1e67861e0013db6ded5dbdac49ce0afd9834500b8d61fe77ee97f8e9d49d9a6cdff12e5009b7cffdaa674beae0f930ebe6085af900093e5fe56b34a5c220ccdcf6efc336fc5000000000000000000000000000000000010
In this key there is a tx hash encoded
(e512ffcda6d9499d8e7fe97ee71fd6b84583d9afe09cc4dadbd5deb63d01e061 in LE used
by all the tools like neoscan).
I love Neo.
2020-03-05 19:44:09 +03:00
Evgenii Stratonikov
95a8fa234f
rpc: implement getnep5transfers RPC
2020-03-05 18:22:40 +03:00
Evgenii Stratonikov
6d270c4550
core,rpc: add NEP5 contract to testdata
...
Also transfer tokens between accounts.
2020-03-05 18:22:40 +03:00
Evgenii Stratonikov
f92fd3c948
core: track NEP5 transfers
2020-03-05 18:22:40 +03:00
Evgenii Stratonikov
e8c4179a9c
core: track NEP5 balances
2020-03-05 18:22:19 +03:00
Evgenii Stratonikov
3a510b9dad
core: allow transfer amount to be bytes
...
VM can produce both big.Int and []byte because they
are converted to each other on demand.
2020-03-05 15:18:38 +03:00
Roman Khimov
d03b2ef4a1
Merge pull request #706 from nspcc-dev/feature/transfer
...
cli: implement transfer from multisig accounts
2020-03-05 12:28:01 +03:00
Roman Khimov
5c781be08d
Merge pull request #718 from nspcc-dev/fix-invocation-tx-checks-and-fee
...
Fix Invocation TX checks and fee
2020-03-05 11:51:51 +03:00
Roman Khimov
9ac8001c7d
core: protect (*Blockchain).keyCache with a lock
...
Avoid failures like this:
fatal error: concurrent map writes
goroutine 103 [running]:
runtime.throw(0xca6fe3, 0x15)
/usr/lib64/go/1.12/src/runtime/panic.go:617 +0x72 fp=0xc000687988 sp=0xc000687958 pc=0x42d8d2
runtime.mapassign(0xb9d4a0, 0xc0000b1f80, 0xc000687a6c, 0x13bd940)
/usr/lib64/go/1.12/src/runtime/map.go:590 +0x5e3 fp=0xc000687a10 sp=0xc000687988 pc=0x40e2a3
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHashAgainstScript(0xc0000f8000, 0x1300d1f5a3d348be, 0x78607089e4fe9fab, 0xc0a21e4f71, 0xc000d16070, 0x1e2b8881f8178e92, 0xd589cfa965287bb8, 0x24a4e4f292ebc797, 0xb90d3120d878e619, 0xc0001d01e0, ...)
/home/rik/dev/neo-go/pkg/core/blockchain.go:1866 +0x371 fp=0xc000687ad8 sp=0xc000687a10 pc=0x984ed1
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeaderWitnesses(0xc0000f8000, 0xc000d16000, 0xc00042d950, 0x1, 0xc00008a6c0)
/home/rik/dev/neo-go/pkg/core/blockchain.go:1914 +0x209 fp=0xc000687c90 sp=0xc000687ad8 pc=0x985999
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeader(0xc0000f8000, 0xc000d16000, 0xc00042d950, 0xf0ee0049d611101f, 0xf8967398a8f9351b)
/home/rik/dev/neo-go/pkg/core/blockchain.go:1254 +0x235 fp=0xc000687d70 sp=0xc000687c90 pc=0x97e4e5
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).addHeaders(0xc0000f8000, 0x1, 0xc000d12000, 0x7d0, 0x7d0, 0x0, 0x1)
/home/rik/dev/neo-go/pkg/core/blockchain.go:369 +0x266 fp=0xc000687e68 sp=0xc000687d70 pc=0x973f46
github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddHeaders(0xc0000f8000, 0xc000d12000, 0x7d0, 0x7d0, 0x0, 0xc0003640c0)
/home/rik/dev/neo-go/pkg/core/blockchain.go:338 +0x51 fp=0xc000687eb0 sp=0xc000687e68 pc=0x973cb1
github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleHeadersCmd(0xc00022c140, 0xdd0ca0, 0xc000374000, 0xc00000c020)
/home/rik/dev/neo-go/pkg/network/server.go:431 +0x74 fp=0xc000687fc0 sp=0xc000687eb0 pc=0x9ce5e4
runtime.goexit()
/usr/lib64/go/1.12/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000687fc8 sp=0xc000687fc0 pc=0x45ca51
created by github.com/nspcc-dev/neo-go/pkg/network.(*Server).handleMessage
/home/rik/dev/neo-go/pkg/network/server.go:705 +0xa4a
...
2020-03-04 20:26:18 +03:00
Evgenii Stratonikov
7eaeb18f18
transaction: marshal ContractTX even if Data is nil
...
It contains no information so it doesn't matter anyway.
2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
634e9483d3
transaction: unmarshal Witness properly
...
Both verification and invocation scripts need to
be unmarshaled from hex.
Also fix failing RPC tests: block contains non-pointer
`transaction.Witness` field and (*Witness).MarshalJSON method
is not called.
2020-03-04 20:01:26 +03:00
Evgenii Stratonikov
33f99104e8
transaction: unmarshal Output properly
...
Address is marshaled in base58 and needs to be
decoded accordingly.
2020-03-04 20:01:26 +03:00
Roman Khimov
81d89cd502
core: fix SystemFee calculation for Invocation TXes
...
They have it specified right in the transaction. Unfortunately, this little
change rendered invalid our RPC test chain, but I think it became even better
after it, especially given that chain generation is a nice test by itself, so
it should be running as a regular test.
2020-03-04 19:23:23 +03:00
Roman Khimov
ae6edf0601
core: add a check for fractional gas in invocation TXes
...
It's not allowed in C# code and that's reasonable as it's a sysfee essentially
that can only be integer.
2020-03-04 19:23:23 +03:00
Roman Khimov
823798514a
util: rename Int64Value to IntegralValue, add FractionalValue
...
Makes it's interface more clear and allows to easily access fractional part.
2020-03-04 19:23:23 +03:00
Roman Khimov
7cc847d655
transaction: add some missing InvocationTX decoding checks
...
Script must have something inside and GAS can't be negative.
2020-03-04 19:23:23 +03:00
Roman Khimov
e41d434a49
*: move all packages from CityOfZion to nspcc-dev
2020-03-03 17:21:42 +03:00
Roman Khimov
59e6dac351
Merge pull request #708 from nspcc-dev/fix/trigger
...
core: convert trigger.Type to byte when pushing on stack
2020-03-03 16:17:01 +03:00
Evgenii Stratonikov
fe3bbbd78a
core: convert trigger.Type to byte when pushing on stack
2020-03-03 16:13:19 +03:00
Roman Khimov
c3e73c5b7d
core: don't reverify stale headers in addHeader
...
During networked synchronization we expect there to be a lot of duplicate
headers received and it makes no sense for us reverifying them.
2020-03-03 15:34:03 +03:00
Evgenii Stratonikov
05a3625b7d
wallet: implement (*Account).SignTx
...
It is used in both CLI and RPC.
2020-03-02 18:03:56 +03:00
Evgenii Stratonikov
6add4f3e50
transaction: disallow negative outputs
...
Otherwise it is possible to make outputs which will sum
to the expected value, but steal GAS from some other account.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
9e9d59b49a
core: set NetworkFee to 0 for Claim and Miner transactions
...
Claim tx have no GAS inputs and a positive output which
can lead to negative network fee.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
de5215a564
core: use GetAccountStateOrNew() in (*Blockchain).GetValidators()
...
Target of the transaction output may not yet exist in database.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
1b9968df67
core: optimize CalculateClaimable()
...
Because accumulated system fee is stored for every block,
it is easy to calculate sum with just to reads.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
258d8be1dd
core: store all accumulated SystemFee with every block
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
f69c8a763b
core: store system fee together with block
...
Recalculating system fee can be rather costly if done
frequently.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
972e0d8ad1
core: add one more Contract tx to the test chain
...
When testing CLI it is useful to have some spent coins
on an account with a known key.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
45b8669b42
core: verify Claim transactions
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
44792ed5f7
core: make test chain format compatible with mainnet
...
Write uint32 length before every block.
2020-03-02 18:01:49 +03:00
Evgenii Stratonikov
0e2a1f40ba
core: add ContractTX to a testchain generator
...
Simple transfer from multisig account to the account
of one of the validators.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
72d72296c3
core: implement (*Transaction).GetSignedPart()
...
Marshalling it and taking all but last byte violates incapsulation
and is just wrong in case transaction already contains any witnesses.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
95d9f36c98
core: implement UnclaimedBalance tracking
...
To make it easy to get unclaimed coins for the
specified account they must be tracked together.
2020-03-02 18:00:00 +03:00
Evgenii Stratonikov
7095ec6c51
core: implement (*Blockchain).CalculateClaimable
...
Calculating amount of GAS that can be claimed is required
for getclaimable RPC.
2020-03-02 18:00:00 +03:00
Roman Khimov
252a9f2f31
Merge pull request #690 from nspcc-dev/feature/getapplicationlog
...
rpc: implement getapplicationlog RPC
2020-03-02 17:41:32 +03:00
Anna Shaleva
ff4384d7ff
rpc: implement getapplicationlog RPC
...
Closes #500
2020-03-02 17:25:27 +03:00