Commit graph

937 commits

Author SHA1 Message Date
Evgenii Stratonikov
b1d9e1132d cli: dump storage changes into JSON files 2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
fb9af98179 storage: implement GetBatch() to view storage changes
GetBatch returns changes to be persisted.
2020-02-12 12:16:31 +03:00
Evgenii Stratonikov
3e84f2bdf8 compiler: rewrite jump targets properly
Old implementation could view 0x62 byte in
a script as a JMP instruction irregardless of whether it is
a real opcode or a part of a parameter of another instruction.
In this commit instructions are decoded together with parameters
during jump label rewriting.
2020-02-12 10:58:53 +03:00
Roman Khimov
d3badc3f97 consensus: log more like neo-cli, update dbft 2020-02-11 18:02:13 +03:00
Roman Khimov
38de2af073
Merge pull request #655 from nspcc-dev/fix-mempool-deadlock-on-reverification
core: get transactions from dao when mempool should not be used
2020-02-11 13:34:23 +03:00
Roman Khimov
292051d8df core: get transactions from dao when mempool should not be used
All of these places deal strictly with the chain and shouldn't ever be
bothered with mempool. It also fixes a deadlock on reverification of
non-standard tx:

1 @ 0x42f62f 0x43fbe9 0x43fbbf 0x43f95d 0x967059 0x966f66 0x972c7c 0x974e13 0x97a5d9 0x97bdf0 0x976147 0x966cc0 0x970f70 0x96c8cb 0x9ba858 0x45ca51
	0x43f95c	sync.runtime_SemacquireMutex+0x3c							/usr/local/go/src/runtime/sema.go:71
	0x967058	sync.(*RWMutex).RLock+0x128								/usr/local/go/src/sync/rwmutex.go:50
	0x966f65	github.com/CityOfZion/neo-go/pkg/core/mempool.(*Pool).TryGetValue+0x35			/go/src/github.com/CityOfZion/neo-go/pkg/core/mempool/mem_pool.go:229
	0x972c7b	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).GetTransaction+0x4b			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:782
	0x974e12	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).References+0x132			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:944
	0x97a5d8	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).GetScriptHashesForVerifying+0x58	/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1410
	0x97bdef	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).verifyTxWitnesses+0x4f		/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1545
	0x976146	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).isTxStillRelevant+0x216		/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:1067
	0x966cbf	github.com/CityOfZion/neo-go/pkg/core/mempool.(*Pool).RemoveStale+0xff			/go/src/github.com/CityOfZion/neo-go/pkg/core/mempool/mem_pool.go:208
	0x970f6f	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).storeBlock+0x2ecf			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:614
	0x96c8ca	github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).AddBlock+0xea			/go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:308
	0x9ba857	github.com/CityOfZion/neo-go/pkg/network.(*blockQueue).run+0x157			/go/src/github.com/CityOfZion/neo-go/pkg/network/blockqueue.go:48
2020-02-10 18:53:22 +03:00
Roman Khimov
0461827fa3
Merge pull request #653 from nspcc-dev/fix-net-locks-and-leaks
network: fix networking stalls caused by stale peers
2020-02-10 18:51:56 +03:00
Roman Khimov
f7749cea67
Merge pull request #652 from nspcc-dev/Fix_FreeGasLimit
core: fix FreeGasLimit constraint
2020-02-10 18:49:09 +03:00
Roman Khimov
7ee8f9c5d8 network: fix networking stalls caused by stale peers
We can leak sending goroutines and stall broadcasts because of already gone
peers that happened to be cached by some s.Peers() user (more than 800 of
these can be seen in nodoka log along with (*Server).run blocking on
CMDGetAddr send):

Feb 10 16:35:15 nodoka neo-go[1563]: goroutine 41 [chan send, 3320 minutes]:
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).putPacketIntoQueue(...)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:81
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*TCPPeer).EnqueueHPPacket(0xc0083d57a0, 0xc017206100, 0x18, 0x40, 0x136a240, 0xc018ef9720)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/tcp_peer.go:119 +0x98
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).iteratePeersWithSendMsg(0xc0000ca000, 0xc0001848a0, 0xcb4550, 0x0)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:720 +0x12a
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).broadcastHPMessage(...)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:731
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).run(0xc0000ca000)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:203 +0xee4
Feb 10 16:35:15 nodoka neo-go[1563]: github.com/CityOfZion/neo-go/pkg/network.(*Server).Start(0xc0000ca000, 0xc000072c60)
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/pkg/network/server.go:173 +0x2ec
Feb 10 16:35:15 nodoka neo-go[1563]: created by github.com/CityOfZion/neo-go/cli/server.startServer
Feb 10 16:35:15 nodoka neo-go[1563]:         /go/src/github.com/CityOfZion/neo-go/cli/server/server.go:331 +0x476
2020-02-10 18:47:52 +03:00
Anna Shaleva
9b03f968b1 core: fixe FreeGasLimit constraint
Lack of FreeGasLimit in privnet leads to gas limit exceeding in case of transactions with small amount of GAS to be used for invoke operation (< real cost of the transaction). Solution: Fixed constraint in case when FreeGasLimit == 0. So now we are able to perform transactions in privnet with FreeGasLimit = 0 for free.
2020-02-10 18:07:19 +03:00
Evgenii Stratonikov
895a8d9ebc compiler: reverse args in AppCall
Invoked contract is expecting first argument to be on top of the stack.
Change test to use non-commutative operation to catch this behaviour.
2020-02-10 10:53:58 +03:00
Evgenii Stratonikov
52d8d58593 compiler,interop: make AppCall accept varargs 2020-02-10 10:51:29 +03:00
Evgenii Stratonikov
1fc64d515f compiler: abstract out emitReverse
Extract logic of reversing top n items of the stack
in a separate function.
2020-02-10 10:43:31 +03:00
Roman Khimov
268cad5c06
Merge pull request #548 from nspcc-dev/feature/multisig
vm: implement parallel algorithm for CHECKMULTISIG
2020-02-08 16:10:34 +03:00
Roman Khimov
9b9adb28c1
Merge pull request #642 from nspcc-dev/feature/emit
vm, compiler: move Emit* functions to a separate package

Closes #449, #534.
2020-02-08 15:56:48 +03:00
Roman Khimov
c896e2b731 network: fix minor gofmt issues (spotted by goreportcard) 2020-02-08 15:53:08 +03:00
Roman Khimov
7ccf7974b6 core: substitute bad notifications with error messages
Tesnet sync failed with:
Feb 07 00:04:19 nodoka neo-go[1747]: 2020-02-07T00:04:19.838+0300        WARN        blockQueue: failed adding block into the blockchain        {"error": "failed to store notifications: not supported", "blockHeight": 713984, "nextIndex": 713985}

because some (not so) smart contract emitted a notification with an
InteropItem inside.
2020-02-07 15:34:59 +03:00
Roman Khimov
b805b1a71b vm: make SerializeItem/DeserializeItem public APIs
They're useful as wrappers around EncodeBinaryStackItem/DecodeBinaryStackItem.
2020-02-07 15:34:59 +03:00
Roman Khimov
7e7ce7f19f vm: rephrase serialization errors
Seeing some
    blockQueue: failed adding block into the blockchain        {"error": "failed to store notifications: not supported", "blockHeight": 713984, "nextIndex": 713985}
in logs is not very helpful.
2020-02-07 11:53:53 +03:00
Evgenii Stratonikov
a8dc704197 emit: add tests for Int 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
6fd3f0fa48 emit: add tests for Bytes 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
5e992d8cdd emit: add tests for Syscall, Jmp 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
dbc41b3044 compiler: replace emit* instructions with those from emit/ package 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
8243a8b3a7 emit: use io.BinWriter instead of bytes.Buffer 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
698c647f07 emit: refactor tests
Add structure and call Bytes() method on buffer once.
2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
1400ecfdde emit: fix Int to serialize integers in correct format
Related #605, #623.
2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
c821e1c4c8 vm: move IntToBytes and BytesToInt to emit package 2020-02-06 18:45:37 +03:00
Evgenii Stratonikov
4d8a3a359b vm: move Emit* functions to a separate package
Also strip 'Emit' prefix because 'emit' is now
in the package name.
2020-02-06 18:45:37 +03:00
Roman Khimov
ab14a4619d
Merge pull request #647 from nspcc-dev/fix-mempool-and-chain-locking
Fix mempool and chain locking

This allows us easily make 1000 Tx/s in 4-nodes privnet, fixes potential
double spends and improves mempool testing coverage.
2020-02-06 18:43:29 +03:00
Evgenii Stratonikov
9204b95d5b core: fix GAS price definitions 2020-02-06 18:19:08 +03:00
Roman Khimov
7445655437 consensus: switch test to using the new PoolTx API
Fixes GolangCI:
  Error return value of
  (*github.com/CityOfZion/neo-go/pkg/core/mempool.Pool).Add is not checked
  (from errcheck)

and allows us to almost completely forget about mempool here.
2020-02-06 17:50:11 +03:00
Roman Khimov
18695e660b mempool: drop RemoveOverCapacity(), handle it right in the Add()
Simplifies things a lot and removes useless code.
2020-02-06 17:50:11 +03:00
Roman Khimov
684cbf5bac mempool: make it almost 100% test-covered 2020-02-06 17:50:11 +03:00
Roman Khimov
6672f4b26f mempool: iterate over slice in GetVerifiedTransactions()
It's more efficient and keeps transactions sorted by priority.
2020-02-06 16:59:20 +03:00
Roman Khimov
e97396e56c mempool: test addition of conflicting tx to the pool 2020-02-06 15:57:20 +03:00
Roman Khimov
1133bbe584 mempool: remove unverified transactions pool
Our mempool only contains valid verified transactions all the time, it never
has any unverified ones. Unverified pool made some sense for quick unverifying
after the new block acceptance (and gradual background reverification), but
reverification needs some non-trivial locking between blockchain and mempool
and internal mempool state locking (reverifying tx and moving it between
unverified and verified pools must be atomic). But our current reverification
is fast enough (and has all the appropriate locks), so bothering with
unverified pool makes little sense.
2020-02-06 15:45:16 +03:00
Roman Khimov
b675903f52 mempool/core: redesign mempool dances on block acceptance
We not only need to remove transactions stored in the block, but also
invalidate some potential double spends caused by these transactions. Usually
new block contains a substantial number of transactions from the pool, so it's
easier to make one pass over it only keeping valid items rather than remove
them one by one and make an additional pass to recheck inputs/witnesses.
2020-02-06 15:45:16 +03:00
Roman Khimov
b567ce86ac mempool: implement insertion to sorted slice
Which is way faster than sort.Sort'ing things all the time.
2020-02-06 15:44:18 +03:00
Roman Khimov
35183b6dba mempool: reverse the order of sorted slice
Chopping off the last element of the slice if way easier than doing it with
the first one.
2020-02-06 15:44:16 +03:00
Roman Khimov
794027a90b mempool: use one slice for both priorities
It doesn't harm as we have transactions naturally ordered by fee anyway and it
makes managing them a little easier. This also makes slices store item itself
instead of pointers to it which reduces the pressure on the memory subsystem.
2020-02-06 15:43:45 +03:00
Roman Khimov
325bea3fa9 mempool: cache Feer invocation results in the item
They shouldn't depend on the chain state and for the same transaction they
should always produce the same result. Thus, it makes no sense recalculating
them over and over again.
2020-02-06 15:41:56 +03:00
Roman Khimov
e01bfeeb4d mempool: remove lock indirection from the Pool
After the f0bb886be3 with all methods of Pool
being pointer-based it makes no sense having this lock as a pointer.
2020-02-06 15:41:56 +03:00
Roman Khimov
a928ad9cfa mempool: make item an internal thing of mempool package
Nobody outside should care about these details, mempool operates on
transactions and that's it.
2020-02-06 15:41:56 +03:00
Roman Khimov
f0e3a31bc8 mempool: fix appending to sorted pools
Appending and not changing the real Items is utterly wrong.
2020-02-06 15:41:56 +03:00
Roman Khimov
b9b77ac1be network: fix block relaying, don't spit out useless errors
We can only add one block of the given height and we have two competing
goroutines to do that --- consensus and block queue. Whomever adds the block
first shouldn't trigger an error in another one.

Fix block relaying for blocks added via the block queue also, previously one
consensus-generated blocks were broadcasted.
2020-02-06 15:41:56 +03:00
Roman Khimov
f9963cca37 core: short-circuit verifyInputs when there are no inputs 2020-02-06 15:41:56 +03:00
Roman Khimov
70b3839fd0 core/mempool: fix AddBlock and tx pooling concurrency issues
Eliminate races between tx checks and adding them to the mempool, ensure the
chain doesn't change while we're working with the new tx. Ensure only one
block addition attempt could be in progress.
2020-02-06 15:41:52 +03:00
Evgenii Stratonikov
24bb66e606 util: fix a bug with not copying slice of len=1 in ArrayReverse
ArrayReverse copies it's argument only if it's len is > 1.
It needs to be consistent in all cases.
2020-02-06 12:15:35 +03:00
Evgenii Stratonikov
d07d6f3371 util: refactor tests for ArrayReverse 2020-02-06 12:15:35 +03:00
Evgenii Stratonikov
2a86149c82 vm: leave single CHECKMULTISIG implementation
Remove sequential implementation and benchmarks.
They will be still present in commit history.
2020-02-05 17:17:22 +03:00