dstepanov-yadro/neoneo-go
1
0
Fork 0
forked from TrueCloudLab/neoneo-go
Code Pull requests Activity
1140 commits 35 branches 134 tags 34 MiB
Commit graph

726 commits

Author SHA1 Message Date
Vsevolod Brekelov
ec17654986 core: refactoring blockchain state and storage 
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
 2019-12-11 13:05:31 +03:00
Roman Khimov
d0f9a28196 vm/core: improve block import speed with PublicKey caching 
This change (closely related to the neo-project/neo#1321 proposal) speeds up
1.4M mainnet blocks import by 30%. Basically, we're eliminating key decoding
for block's multisignature that has the same keys most of the time.

Things I don't like about this patch:
 * yet another parameter for verifyHashAgainstScript()
 * vm keys are not copied in/out

But it's rather simple and solves the problem for this particular case, so I
think it's worth it.
 2019-12-10 19:13:29 +03:00
Roman Khimov
35e368c241 io: add a note for WriteArray, fix #519 
It can't be really solved in many cases (it's used in P2P protocol and we have
to follow the usual conventions there) and in most of the cases we don't care
about the difference between nil slice and zero-length slice.
 2019-12-09 18:39:30 +03:00
Roman Khimov
f1856bfa8b core/tx: remove publickey indirection from assets and txes 
It makes very little sense having pointers here, these structures MUST have
some kind of key and this key is not gonna be wandering somewhere on its
own. Fixes a part of #519.
 2019-12-09 18:33:04 +03:00
Roman Khimov
5b6c5af704 *: implement EncodeBinary with pointer receivers where appropriate 
Everywhere except ParamType (which is just a byte), reduce copying things
around for no real reason.
 2019-12-09 18:25:15 +03:00
Roman Khimov
7e371588a7 core/tx: remove one layer of indirection for scripts and inouts 
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
 2019-12-09 17:14:10 +03:00
Roman Khimov
b542a5e7a0 io: add support for pointer receivers in WriteArray() 
It's actually preferable to have pointer receivers for serializable types, so
this should be supported.
 2019-12-09 16:57:25 +03:00
Roman Khimov
052ba1e94f
Merge pull request #545 from nspcc-dev/feat/optimizeio 
Remove some reflection from the io package
 2019-12-09 15:12:13 +03:00
Evgenii Stratonikov
fccb008594 io: implement ReadBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
838050f8b5 io: rename ReadBytes() to ReadVarBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
f01fc1cc29 io: optimize BinWriter.WriteArray() 
Replace reflect.MethodByName with a simple interface cast.
 2019-12-09 14:59:49 +03:00
Evgenii Stratonikov
1784a14148 io: optimize BinReader.ReadArray() 
reflect.MethodByName is a rather expensive function especially when
called on hot path. This became obvious during profiling of db restore.
This commit replaces reflection with a cast to an interface.
 2019-12-09 14:58:37 +03:00
Roman Khimov
3898aadcec rpc: simplify uint160 handling 
Use new Uint160DecodeStringLE() function.
 2019-12-06 19:47:58 +03:00
Roman Khimov
e4d821f32d
Merge pull request #546 from nspcc-dev/write-optimizations 
Write optimizations
 2019-12-06 19:40:38 +03:00
Roman Khimov
9992a98007 core: optimize balance utxo removal 
One of my samples had 8 (out of 30) seconds spent here, but values of 100ms
are more typical. After this change it becomes invisible.
 2019-12-06 18:46:24 +03:00
Roman Khimov
844491d365 *: use more efficient WriteBytes where appropriate 
Before this patch on block import we could easily be spending more than 6
seconds out of 30 in Uint256 encoding for UnspentBalance, now it's completely
off the radar.
 2019-12-06 18:22:21 +03:00
Roman Khimov
e7687d620d io: simplify WriteBytes() 
Which speeds it up at least twofold for a typical 32-bytes write (and that's
for a very naïve test that allocates new BufBinWriter on every iteration):

pkg: github.com/CityOfZion/neo-go/pkg/io
BenchmarkWriteBytes-8           10000000               124 ns/op
BenchmarkWriteBytesOld-8         5000000               251 ns/op
 2019-12-06 17:40:47 +03:00
Evgenii Stratonikov
aa20a95181 util: add Uint256DecodeStringBE() 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
72fe884faa util: add Uint160DecodeStringLE() 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
07e832f046 util: add Uint160DecodeBytesLE() 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
09b295d727 util: add Uint160.Reverse() 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
9e04e61533 util: make Uint160Size public 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
7179e4ba9f util: add LE suffix to Uint256 methods 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
57efad912c util: add LE suffix to Uint160 methods 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
138c94eda3 consensus: sign and verify consensus messages 2019-12-06 11:35:06 +03:00
Evgenii Stratonikov
765c354793 consensus: return signed messages from recovery.Get* 2019-12-06 11:33:32 +03:00
Roman Khimov
652ede03c8
Merge pull request #543 from nspcc-dev/force-neo-vm-tests-run 
vm: force neo-vm tests presence
 2019-12-05 12:42:57 +03:00
Roman Khimov
8beb135829
Merge pull request #542 from nspcc-dev/fix-storage-interop-reads 
core: fix wrong data being read in interops (part of #501)
 2019-12-05 09:08:38 +03:00
Roman Khimov
becd4f2333 vm: force neo-vm tests presence 
We succeeded this test without running anything real which is certainly not
good and I think we should always run this tests.
 2019-12-04 20:30:27 +03:00
Roman Khimov
c034aae378 core: fix wrong data being read in interops (part of #501) 
When 74590551 introduced this code we had no proper caching layer, so there
were these strange fallbacks in the code. fc0031e5 should'd removed them, but
failed to do so, so do it now and fix processing of transactions that touch
storage for the same key (address) in the same block.
 2019-12-04 19:51:57 +03:00
Vsevolod Brekelov
ff15a0acfd core: add trigger types as constants. Closes #509 2019-12-04 12:27:04 +03:00
Roman Khimov
e002d7b40c
Merge pull request #538 from nspcc-dev/opcode-and-related 
To use opcode definitions you have to import whole vm package that you might
not care about at all. So this moves opcodes to their own package under vm, fixes
and deduplicate related code and moves compiler package up one level.
 2019-12-03 19:31:11 +03:00
Roman Khimov
852e6a335b compiler: move it up from vm 
It really deserves it, I think. Especially given that it doesn't have any
direct usage of `vm` package now.
 2019-12-03 18:23:46 +03:00
Roman Khimov
31add423a8 core: replace open-coded PUSHBYTES64 with the one from opcode 2019-12-03 18:23:46 +03:00
Roman Khimov
141d27795e rpc: remove hardcoded pushbytes64 constant 
Using our new and shiny opcode package.
 2019-12-03 18:23:46 +03:00
Roman Khimov
138e125646 *: remove duplicate functions producing verification script 
Drop wif.GetVerificationScript(), drop
smartcontract.CreateSignatureRedeemScript(), add GetVerificationScript()
directly to the PublicKey and use it everywhere.
 2019-12-03 18:23:46 +03:00
Roman Khimov
8d4dd2d2e1 vm: move opcodes into their own package 
This allows easier reuse of opcodes and in some cases allows to eliminate
dependencies on the whole vm package, like in compiler that only needs opcodes
and doesn't care about VM for any other purpose.

And yes, they're opcodes because an instruction is a whole thing with
operands, that's what context.Next() returns.
 2019-12-03 18:22:14 +03:00
Vsevolod Brekelov
5a41da0e1a io: add missing test 2019-12-03 16:20:06 +03:00
Roman Khimov
f48228ef7d
Merge pull request #467 from nspcc-dev/errcheck_297 
This patchset closes #297 and #457.
 2019-12-03 15:06:11 +03:00
Vsevolod Brekelov
b08387efdb vm: use BinWriter instead of Buffered version 
In the future we could reuse emit interface without buf better
 2019-12-03 13:50:23 +03:00
Vsevolod Brekelov
f5e2401984 vm: removed logging to fix #457 2019-12-03 13:49:49 +03:00
Vsevolod Brekelov
03ff2976ed io: refactoring for using WriteVarBytes instead of WriteLE 
goal is to be consistent with C# implementation.
For writing []byte WriteBytes used and for byte - WriteVarByte.
 2019-12-03 13:49:33 +03:00
Roman Khimov
1c89c192ac core: optimize some accesses to unsortedTxn in mempool 
These don't need a full lock, they only read things from maps.
 2019-12-02 22:39:43 +03:00
Roman Khimov
0cd3493fa5 core: fix potential locking problem in mempool 
I think it should fix this issue mentioned in the #526:

INFO[1456] blockchain persist completed                  blockHeight=63480 headerHeight=1810000 persistedBlocks=1 persistedKeys=4 took=740.7113ms
fatal error: concurrent map read and map write

goroutine 322 [running]:
runtime.throw(0xc8a6dc, 0x21)
        /usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc003473958 sp=0xc003473928 pc=0x42e282
runtime.mapaccess2(0xb706a0, 0xc0001893b0, 0xc0034739c8, 0xc0028704e0, 0x3)
        /usr/local/go/src/runtime/map.go:470 +0x278 fp=0xc0034739a0 sp=0xc003473958 pc=0x40dc08
github.com/CityOfZion/neo-go/pkg/core.MemPool.ContainsKey(0xc0001d0d20, 0xc0001893b0, 0xc0001893e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /neo-go/pkg/core/mem_pool.go:92 +0xcb fp=0xc003473a30 sp=0xc0034739a0 pc=0x9326db
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).HasTransaction(0xc0001ca8c0, 0x49f0b45d430e441b, 0x553db79b7072821c, 0x28969518de11976, 0xba5100efddbe79d4, 0xc003473cd0)
        /neo-go/pkg/core/blockchain.go:803 +0xa1 fp=0xc003473b68 sp=0xc003473a30 pc=0x914b11
github.com/CityOfZion/neo-go/pkg/core.Blockchainer.HasTransaction-fm(0x49f0b45d430e441b, 0x553db79b7072821c, 0x28969518de11976, 0xba5100efddbe79d4, 0xc005a5d388)
        /neo-go/pkg/core/blockchainer.go:28 +0x46 fp=0xc003473ba8 sp=0xc003473b68 pc=0x997326
github.com/CityOfZion/neo-go/pkg/network.(*Server).handleInvCmd(0xc00018f680, 0xd9d980, 0xc00025e190, 0xc005a5d380, 0x0, 0x0)
        /neo-go/pkg/network/server.go:401 +0x3bb fp=0xc003473d38 sp=0xc003473ba8 pc=0x9924cb
github.com/CityOfZion/neo-go/pkg/network.(*Server).handleMessage(0xc00018f680, 0xd9d980, 0xc00025e190, 0xc007a0d050, 0x0, 0x0)
        /neo-go/pkg/network/server.go:582 +0x1ae fp=0xc003473da0 sp=0xc003473d38 pc=0x993bbe
github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).handleConn(0xc000228420, 0xd9b880, 0xc0001b6f00)
        /neo-go/pkg/network/tcp_transport.go:93 +0x202 fp=0xc003473fc8 sp=0xc003473da0 pc=0x996672
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc003473fd0 sp=0xc003473fc8 pc=0x45b3e1
created by github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).Dial
        /neo-go/pkg/network/tcp_transport.go:36 +0xb4

The problem is that we're modifying `unsortedTxn` under a reader lock.
 2019-12-02 22:36:59 +03:00
Roman Khimov
abc0ec33bd
Merge pull request #532 from nspcc-dev/fix/peer_count 
Consensus fixes.
 2019-12-02 18:08:36 +03:00
Evgenii Stratonikov
9ddebfdb8a consensus: add tests for recovery message 2019-12-02 17:00:17 +03:00
Evgenii Stratonikov
b649ffb88e consensus: set preparation hash in recovery message 2019-12-02 16:43:24 +03:00
Evgenii Stratonikov
7c900edd2d network: count only handshaked peers as connected 2019-12-02 15:09:09 +03:00
Vsevolod Brekelov
d02673c112 vm: add bufBinWriter to emit functions in order to catch errors 2019-12-02 13:04:33 +03:00
Roman Khimov
1da76a8eb4 network: rework inventory handling, check for item presence 
Don't ask peers about the items we already have.
 2019-12-02 11:02:52 +03:00
Roman Khimov
bc5beb438f *: gofmt 
Somehow these crept into the repository.
 2019-12-02 10:10:50 +03:00
Roman Khimov
dccf440dca rpc: skip input/outputs mangling if no gas is attached to invocation 2019-11-29 18:00:08 +03:00
Roman Khimov
8a5ac12df8 rpc: make generic SignAndPushInvocationTx out of DeployContract 
SignAndPushInvocationTx() is gonna be used for more than just deploying
contracts.
 2019-11-29 17:59:07 +03:00
Roman Khimov
65332f5e7f
Merge pull request #529 from nspcc-dev/peer-communication-fixes 
A set of fixes to make neo-go privnet more usable.
 2019-11-29 16:29:28 +03:00
Roman Khimov
b4d9935bda network: deduplicate a part of RelayTxn() 
We already have relayInventory() for this.
 2019-11-29 16:26:44 +03:00
Roman Khimov
2c0ed62680 core: fix blockchain init with (2000*N + 1) header hashes in DB 
Stored header count includes block number zero, so it's (height + 1).
 2019-11-29 16:26:44 +03:00
Roman Khimov
cd858ea5e6
Merge pull request #528 from nspcc-dev/fix/tx 
consensus: fix 2 bugs
 2019-11-29 16:12:53 +03:00
Evgenii Stratonikov
60f1648778 consensus: fix a bug with index out of range 2019-11-29 16:05:00 +03:00
Roman Khimov
efdcacca81
Merge pull request #527 from nspcc-dev/invoke-tx 
Invoke RPC
 2019-11-29 15:43:26 +03:00
Evgenii Stratonikov
189a708988 consensus: fix a bug with nil tx channel 2019-11-29 15:40:11 +03:00
Roman Khimov
293615ea5f network/consensus: add new block relaying 
Tell everyone about our new shiny blocks.
 2019-11-29 12:27:15 +03:00
Roman Khimov
4d286dcfeb network: check height before requesting headers 
Only request headers from the other peer if his height is bigger than
ours. Otherwise we routinely ask 0-height newcomers for some random headers
that they know nothing about.
 2019-11-29 11:48:17 +03:00
Roman Khimov
a730529b0c network: process incoming p2p transactions 
It's the same relaying as for RPC, but we don't need to return any result for
it.
 2019-11-29 11:09:54 +03:00
Roman Khimov
9f9cf4ae3f network: add getheaders message processing 
This one is essential for the consensus nodes as otherwise they won't give out
the blocks they generate making their generation almost useless. It also makes
our networking part more complete.
 2019-11-29 11:08:22 +03:00
Roman Khimov
734338ad70 network: move Version sending to the tcp transport 
We have a race between reader and writer goroutines for the same connection
that leads to handshake failures when reader is faster to read the incoming
version (and try to reply to it) than writer is to write our own Version:

WARN[0000] peer disconnected                             addr="172.200.0.4:20334" peerCount=5 reason="invalid handshake: tried to send VersionAck, but didn't send Version yet

Fix it by moving Version sending before the reader loop starts.
 2019-11-29 11:05:42 +03:00
Evgenii Stratonikov
0a9a742019 crypto: add invalid testcase 2019-11-29 11:03:03 +03:00
Evgenii Stratonikov
4d82419776 crypto: add missing tests for hash pkg 2019-11-29 11:03:03 +03:00
Roman Khimov
127f8418c8 rpc: refactor out runScriptInVM() from invokers 2019-11-28 19:13:58 +03:00
Roman Khimov
e216139108 rpc: implement server-side 'invoke' method, fix #346 2019-11-28 19:08:31 +03:00
Roman Khimov
2d41450ac9
Merge pull request #507 from nspcc-dev/dbft 
network: plug in dBFT library
 2019-11-27 15:22:40 +03:00
Roman Khimov
1b5da8401a
Merge pull request #521 from nspcc-dev/invokefunction 
Invokefunction
 2019-11-27 15:14:29 +03:00
Roman Khimov
e63b25d5ad smartcontract: add user-facing testinvokefunction command 
With a very special syntax.
 2019-11-27 15:12:15 +03:00
Roman Khimov
c5e2b3ce73
Merge pull request #523 from nspcc-dev/fix-interim-block-changes-store 
Fix interim block changes store
 2019-11-27 14:51:29 +03:00
Vsevolod
1a9e98caf3
Merge pull request #522 from nspcc-dev/fix-invalid-tx-type-handling 
Fix invalid tx type handling
 2019-11-27 13:40:52 +03:00
Roman Khimov
467c9c146d core: restore intermediate block-level store 
Commit c80ee952a1 removed temporary store used
to contain changes of the block being processed. It's wrong in that the block
changes should be applied to the database in a single transaction so that
there wouldn't be any intermediate state observed from the outside (which is
possible now). Also, this made changes commiting persist them to the
underlying store effectively making our persist loop a no-op (and not
producing `persist completed` log lines that we love so much).
 2019-11-27 13:28:13 +03:00
Roman Khimov
bb765ccfa7 core: gofmt 2019-11-27 13:27:47 +03:00
Roman Khimov
65bf0d01d5 transaction: don't hide decoding error by logging 
Unknown transaction types are wrong and should be treated as errors.
 2019-11-27 13:12:04 +03:00
Roman Khimov
05f3329ec0 vm: add IterBack to the Stack to iterate bottom-to-top 
And use it to build user-facing stack representation because that's the order
that C# VM uses.
 2019-11-27 13:00:11 +03:00
Roman Khimov
f3ed91a1f7 vm: properly convert arrays to stackItems 
They should be arrays of stackItems, not arrays of values.
 2019-11-27 13:00:11 +03:00
Roman Khimov
d04bc0cbe3 rpc: add marshaler for StackParamType 
Makes stack output look better, no one cares about numbers.
 2019-11-27 13:00:11 +03:00
Roman Khimov
3843451ec4 rpc: fix potential getblock handling problem, add a test for it 
It should only accept hex strings and numbers, anything else is wrong.
 2019-11-27 13:00:11 +03:00
Roman Khimov
1c883cc819 rpc: add invokescript tests 2019-11-27 13:00:11 +03:00
Roman Khimov
cf39171485 rpc: implement invokefunction, fix #347 
Param getters were redone to return errors because otherwise bad FuncParam
values could lead to panic. FuncParam itself might be not the most elegant
solution, but it works good enough for now.
 2019-11-27 13:00:11 +03:00
Evgenii Stratonikov
b5fb43bac9 network: don't reconnect to peers with identical ID 2019-11-27 11:58:17 +03:00
Evgenii Stratonikov
fdd5276d3e network: plug in dBFT library 2019-11-27 10:57:22 +03:00
Vsevolod
34a38ccce1
Merge pull request #518 from nspcc-dev/feat/noerrors 
core: do not return error in block.getHashableData
 2019-11-26 16:20:46 +03:00
Evgenii Stratonikov
af61571ba0 core: do not return error in block.getHashableData 2019-11-26 15:27:37 +03:00
Evgenii Stratonikov
ab922c1810 util: add missing tests for Uint256 2019-11-25 11:55:26 +03:00
Evgenii Stratonikov
d018dc4e56 util: add missing tests for Uint160 2019-11-25 11:55:26 +03:00
Evgenii Stratonikov
18075bc5c8 util: add missing tests for Fixed8 2019-11-25 11:54:59 +03:00
Evgenii Stratonikov
116d4c656e util: make tests use require 2019-11-25 11:53:49 +03:00
Roman Khimov
ea9bc22510
Merge pull request #513 from nspcc-dev/feat/rpc_param 
This PR does 3 things:

    adds array parameter unmarshalling
    extend Param with convenient methods
    refactor tests into using tables to make it easier add new tests

(part of #347 solution)
 2019-11-22 18:47:59 +03:00
Roman Khimov
eb84ae49da
Merge pull request #508 from nspcc-dev/feature/getvalidators_420 
core: Implement getvalidators interop
 2019-11-22 18:15:08 +03:00
Evgenii Stratonikov
c9d5f8b89c rpc: cover stack_param with more tests 
Use require/assert instead of builtin facilities.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2019-11-22 13:20:15 +03:00
Evgenii Stratonikov
0f9024d177 rpc: make client default values constants 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2019-11-22 12:57:10 +03:00
Evgenii Stratonikov
3afcd784f0 rpc: refactor tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2019-11-22 12:57:07 +03:00
Evgenii Stratonikov
d5fa31cecd rpc: trim spaces in tests once 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2019-11-22 12:57:03 +03:00
Evgenii Stratonikov
c8987eda32 rpc: add array param type and tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2019-11-22 12:56:58 +03:00
Evgenii Stratonikov
7331127556 rpc: make parameter type an enum 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2019-11-22 12:56:49 +03:00
Vsevolod Brekelov
38088b648a crypto: add unit tests for public key 2019-11-21 15:41:49 +03:00