Roman Khimov
54d888ba70
io: add type-specific read/write methods
...
This seriously improves the serialization/deserialization performance for
several reasons:
* no time spent in `binary` reflection
* no memory allocations being made on every read/write
* uses fast ReadBytes everywhere it's appropriate
It also makes Fixed8 Serializable just for convenience.
2019-12-12 20:19:50 +03:00
Roman Khimov
89d7f6d26e
core/tx: microoptimize block/tx hashing
...
Don't hash the data twice.
2019-12-12 18:01:30 +03:00
Roman Khimov
7e83078d13
hash: remove useless no-op decoding
...
It changes nothing here.
2019-12-12 17:58:34 +03:00
Vsevolod Brekelov
c93a8d2bc4
core: extracted same logic to separate methods
2019-12-11 13:14:45 +03:00
Vsevolod Brekelov
c1f39d5c7b
internal: moved testutil method to internal package
2019-12-11 13:14:43 +03:00
Vsevolod Brekelov
2d42b14a1d
core: renames entities-> state and removed State prefix
2019-12-11 13:14:18 +03:00
Vsevolod Brekelov
8809fe437d
core: unit tests for dao
2019-12-11 13:13:51 +03:00
Vsevolod Brekelov
c0e59ebd4e
core: unit tests for entities
2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
c7ac4b6dff
core: fix encoding and decoding for notification event
2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
ec17654986
core: refactoring blockchain state and storage
...
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
2019-12-11 13:05:31 +03:00
Roman Khimov
d0f9a28196
vm/core: improve block import speed with PublicKey caching
...
This change (closely related to the neo-project/neo#1321 proposal) speeds up
1.4M mainnet blocks import by 30%. Basically, we're eliminating key decoding
for block's multisignature that has the same keys most of the time.
Things I don't like about this patch:
* yet another parameter for verifyHashAgainstScript()
* vm keys are not copied in/out
But it's rather simple and solves the problem for this particular case, so I
think it's worth it.
2019-12-10 19:13:29 +03:00
Roman Khimov
35e368c241
io: add a note for WriteArray, fix #519
...
It can't be really solved in many cases (it's used in P2P protocol and we have
to follow the usual conventions there) and in most of the cases we don't care
about the difference between nil slice and zero-length slice.
2019-12-09 18:39:30 +03:00
Roman Khimov
f1856bfa8b
core/tx: remove publickey indirection from assets and txes
...
It makes very little sense having pointers here, these structures MUST have
some kind of key and this key is not gonna be wandering somewhere on its
own. Fixes a part of #519 .
2019-12-09 18:33:04 +03:00
Roman Khimov
5b6c5af704
*: implement EncodeBinary with pointer receivers where appropriate
...
Everywhere except ParamType (which is just a byte), reduce copying things
around for no real reason.
2019-12-09 18:25:15 +03:00
Roman Khimov
7e371588a7
core/tx: remove one layer of indirection for scripts and inouts
...
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
2019-12-09 17:14:10 +03:00
Roman Khimov
b542a5e7a0
io: add support for pointer receivers in WriteArray()
...
It's actually preferable to have pointer receivers for serializable types, so
this should be supported.
2019-12-09 16:57:25 +03:00
Roman Khimov
052ba1e94f
Merge pull request #545 from nspcc-dev/feat/optimizeio
...
Remove some reflection from the io package
2019-12-09 15:12:13 +03:00
Evgenii Stratonikov
fccb008594
io: implement ReadBytes()
2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
838050f8b5
io: rename ReadBytes() to ReadVarBytes()
2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
f01fc1cc29
io: optimize BinWriter.WriteArray()
...
Replace reflect.MethodByName with a simple interface cast.
2019-12-09 14:59:49 +03:00
Evgenii Stratonikov
1784a14148
io: optimize BinReader.ReadArray()
...
reflect.MethodByName is a rather expensive function especially when
called on hot path. This became obvious during profiling of db restore.
This commit replaces reflection with a cast to an interface.
2019-12-09 14:58:37 +03:00
Roman Khimov
3898aadcec
rpc: simplify uint160 handling
...
Use new Uint160DecodeStringLE() function.
2019-12-06 19:47:58 +03:00
Roman Khimov
e4d821f32d
Merge pull request #546 from nspcc-dev/write-optimizations
...
Write optimizations
2019-12-06 19:40:38 +03:00
Roman Khimov
9992a98007
core: optimize balance utxo removal
...
One of my samples had 8 (out of 30) seconds spent here, but values of 100ms
are more typical. After this change it becomes invisible.
2019-12-06 18:46:24 +03:00
Roman Khimov
844491d365
*: use more efficient WriteBytes where appropriate
...
Before this patch on block import we could easily be spending more than 6
seconds out of 30 in Uint256 encoding for UnspentBalance, now it's completely
off the radar.
2019-12-06 18:22:21 +03:00
Roman Khimov
e7687d620d
io: simplify WriteBytes()
...
Which speeds it up at least twofold for a typical 32-bytes write (and that's
for a very naïve test that allocates new BufBinWriter on every iteration):
pkg: github.com/CityOfZion/neo-go/pkg/io
BenchmarkWriteBytes-8 10000000 124 ns/op
BenchmarkWriteBytesOld-8 5000000 251 ns/op
2019-12-06 17:40:47 +03:00
Evgenii Stratonikov
aa20a95181
util: add Uint256DecodeStringBE()
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
72fe884faa
util: add Uint160DecodeStringLE()
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
07e832f046
util: add Uint160DecodeBytesLE()
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
09b295d727
util: add Uint160.Reverse()
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
9e04e61533
util: make Uint160Size public
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
7179e4ba9f
util: add LE suffix to Uint256 methods
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
57efad912c
util: add LE suffix to Uint160 methods
2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
138c94eda3
consensus: sign and verify consensus messages
2019-12-06 11:35:06 +03:00
Evgenii Stratonikov
765c354793
consensus: return signed messages from recovery.Get*
2019-12-06 11:33:32 +03:00
Roman Khimov
652ede03c8
Merge pull request #543 from nspcc-dev/force-neo-vm-tests-run
...
vm: force neo-vm tests presence
2019-12-05 12:42:57 +03:00
Roman Khimov
8beb135829
Merge pull request #542 from nspcc-dev/fix-storage-interop-reads
...
core: fix wrong data being read in interops (part of #501 )
2019-12-05 09:08:38 +03:00
Roman Khimov
becd4f2333
vm: force neo-vm tests presence
...
We succeeded this test without running anything real which is certainly not
good and I think we should always run this tests.
2019-12-04 20:30:27 +03:00
Roman Khimov
c034aae378
core: fix wrong data being read in interops (part of #501 )
...
When 74590551
introduced this code we had no proper caching layer, so there
were these strange fallbacks in the code. fc0031e5
should'd removed them, but
failed to do so, so do it now and fix processing of transactions that touch
storage for the same key (address) in the same block.
2019-12-04 19:51:57 +03:00
Vsevolod Brekelov
ff15a0acfd
core: add trigger types as constants. Closes #509
2019-12-04 12:27:04 +03:00
Roman Khimov
e002d7b40c
Merge pull request #538 from nspcc-dev/opcode-and-related
...
To use opcode definitions you have to import whole vm package that you might
not care about at all. So this moves opcodes to their own package under vm, fixes
and deduplicate related code and moves compiler package up one level.
2019-12-03 19:31:11 +03:00
Roman Khimov
852e6a335b
compiler: move it up from vm
...
It really deserves it, I think. Especially given that it doesn't have any
direct usage of `vm` package now.
2019-12-03 18:23:46 +03:00
Roman Khimov
31add423a8
core: replace open-coded PUSHBYTES64 with the one from opcode
2019-12-03 18:23:46 +03:00
Roman Khimov
141d27795e
rpc: remove hardcoded pushbytes64 constant
...
Using our new and shiny opcode package.
2019-12-03 18:23:46 +03:00
Roman Khimov
138e125646
*: remove duplicate functions producing verification script
...
Drop wif.GetVerificationScript(), drop
smartcontract.CreateSignatureRedeemScript(), add GetVerificationScript()
directly to the PublicKey and use it everywhere.
2019-12-03 18:23:46 +03:00
Roman Khimov
8d4dd2d2e1
vm: move opcodes into their own package
...
This allows easier reuse of opcodes and in some cases allows to eliminate
dependencies on the whole vm package, like in compiler that only needs opcodes
and doesn't care about VM for any other purpose.
And yes, they're opcodes because an instruction is a whole thing with
operands, that's what context.Next() returns.
2019-12-03 18:22:14 +03:00
Vsevolod Brekelov
5a41da0e1a
io: add missing test
2019-12-03 16:20:06 +03:00
Roman Khimov
f48228ef7d
Merge pull request #467 from nspcc-dev/errcheck_297
...
This patchset closes #297 and #457 .
2019-12-03 15:06:11 +03:00
Vsevolod Brekelov
b08387efdb
vm: use BinWriter instead of Buffered version
...
In the future we could reuse emit interface without buf better
2019-12-03 13:50:23 +03:00
Vsevolod Brekelov
f5e2401984
vm: removed logging to fix #457
2019-12-03 13:49:49 +03:00
Vsevolod Brekelov
03ff2976ed
io: refactoring for using WriteVarBytes instead of WriteLE
...
goal is to be consistent with C# implementation.
For writing []byte WriteBytes used and for byte - WriteVarByte.
2019-12-03 13:49:33 +03:00
Roman Khimov
1c89c192ac
core: optimize some accesses to unsortedTxn in mempool
...
These don't need a full lock, they only read things from maps.
2019-12-02 22:39:43 +03:00
Roman Khimov
0cd3493fa5
core: fix potential locking problem in mempool
...
I think it should fix this issue mentioned in the #526 :
INFO[1456] blockchain persist completed blockHeight=63480 headerHeight=1810000 persistedBlocks=1 persistedKeys=4 took=740.7113ms
fatal error: concurrent map read and map write
goroutine 322 [running]:
runtime.throw(0xc8a6dc, 0x21)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc003473958 sp=0xc003473928 pc=0x42e282
runtime.mapaccess2(0xb706a0, 0xc0001893b0, 0xc0034739c8, 0xc0028704e0, 0x3)
/usr/local/go/src/runtime/map.go:470 +0x278 fp=0xc0034739a0 sp=0xc003473958 pc=0x40dc08
github.com/CityOfZion/neo-go/pkg/core.MemPool.ContainsKey(0xc0001d0d20, 0xc0001893b0, 0xc0001893e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/neo-go/pkg/core/mem_pool.go:92 +0xcb fp=0xc003473a30 sp=0xc0034739a0 pc=0x9326db
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).HasTransaction(0xc0001ca8c0, 0x49f0b45d430e441b, 0x553db79b7072821c, 0x28969518de11976, 0xba5100efddbe79d4, 0xc003473cd0)
/neo-go/pkg/core/blockchain.go:803 +0xa1 fp=0xc003473b68 sp=0xc003473a30 pc=0x914b11
github.com/CityOfZion/neo-go/pkg/core.Blockchainer.HasTransaction-fm(0x49f0b45d430e441b, 0x553db79b7072821c, 0x28969518de11976, 0xba5100efddbe79d4, 0xc005a5d388)
/neo-go/pkg/core/blockchainer.go:28 +0x46 fp=0xc003473ba8 sp=0xc003473b68 pc=0x997326
github.com/CityOfZion/neo-go/pkg/network.(*Server).handleInvCmd(0xc00018f680, 0xd9d980, 0xc00025e190, 0xc005a5d380, 0x0, 0x0)
/neo-go/pkg/network/server.go:401 +0x3bb fp=0xc003473d38 sp=0xc003473ba8 pc=0x9924cb
github.com/CityOfZion/neo-go/pkg/network.(*Server).handleMessage(0xc00018f680, 0xd9d980, 0xc00025e190, 0xc007a0d050, 0x0, 0x0)
/neo-go/pkg/network/server.go:582 +0x1ae fp=0xc003473da0 sp=0xc003473d38 pc=0x993bbe
github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).handleConn(0xc000228420, 0xd9b880, 0xc0001b6f00)
/neo-go/pkg/network/tcp_transport.go:93 +0x202 fp=0xc003473fc8 sp=0xc003473da0 pc=0x996672
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc003473fd0 sp=0xc003473fc8 pc=0x45b3e1
created by github.com/CityOfZion/neo-go/pkg/network.(*TCPTransport).Dial
/neo-go/pkg/network/tcp_transport.go:36 +0xb4
The problem is that we're modifying `unsortedTxn` under a reader lock.
2019-12-02 22:36:59 +03:00
Roman Khimov
abc0ec33bd
Merge pull request #532 from nspcc-dev/fix/peer_count
...
Consensus fixes.
2019-12-02 18:08:36 +03:00
Evgenii Stratonikov
9ddebfdb8a
consensus: add tests for recovery message
2019-12-02 17:00:17 +03:00
Evgenii Stratonikov
b649ffb88e
consensus: set preparation hash in recovery message
2019-12-02 16:43:24 +03:00
Evgenii Stratonikov
7c900edd2d
network: count only handshaked peers as connected
2019-12-02 15:09:09 +03:00
Vsevolod Brekelov
d02673c112
vm: add bufBinWriter to emit functions in order to catch errors
2019-12-02 13:04:33 +03:00
Roman Khimov
1da76a8eb4
network: rework inventory handling, check for item presence
...
Don't ask peers about the items we already have.
2019-12-02 11:02:52 +03:00
Roman Khimov
bc5beb438f
*: gofmt
...
Somehow these crept into the repository.
2019-12-02 10:10:50 +03:00
Roman Khimov
dccf440dca
rpc: skip input/outputs mangling if no gas is attached to invocation
2019-11-29 18:00:08 +03:00
Roman Khimov
8a5ac12df8
rpc: make generic SignAndPushInvocationTx out of DeployContract
...
SignAndPushInvocationTx() is gonna be used for more than just deploying
contracts.
2019-11-29 17:59:07 +03:00
Roman Khimov
65332f5e7f
Merge pull request #529 from nspcc-dev/peer-communication-fixes
...
A set of fixes to make neo-go privnet more usable.
2019-11-29 16:29:28 +03:00
Roman Khimov
b4d9935bda
network: deduplicate a part of RelayTxn()
...
We already have relayInventory() for this.
2019-11-29 16:26:44 +03:00
Roman Khimov
2c0ed62680
core: fix blockchain init with (2000*N + 1) header hashes in DB
...
Stored header count includes block number zero, so it's (height + 1).
2019-11-29 16:26:44 +03:00
Roman Khimov
cd858ea5e6
Merge pull request #528 from nspcc-dev/fix/tx
...
consensus: fix 2 bugs
2019-11-29 16:12:53 +03:00
Evgenii Stratonikov
60f1648778
consensus: fix a bug with index out of range
2019-11-29 16:05:00 +03:00
Roman Khimov
efdcacca81
Merge pull request #527 from nspcc-dev/invoke-tx
...
Invoke RPC
2019-11-29 15:43:26 +03:00
Evgenii Stratonikov
189a708988
consensus: fix a bug with nil tx channel
2019-11-29 15:40:11 +03:00
Roman Khimov
293615ea5f
network/consensus: add new block relaying
...
Tell everyone about our new shiny blocks.
2019-11-29 12:27:15 +03:00
Roman Khimov
4d286dcfeb
network: check height before requesting headers
...
Only request headers from the other peer if his height is bigger than
ours. Otherwise we routinely ask 0-height newcomers for some random headers
that they know nothing about.
2019-11-29 11:48:17 +03:00
Roman Khimov
a730529b0c
network: process incoming p2p transactions
...
It's the same relaying as for RPC, but we don't need to return any result for
it.
2019-11-29 11:09:54 +03:00
Roman Khimov
9f9cf4ae3f
network: add getheaders message processing
...
This one is essential for the consensus nodes as otherwise they won't give out
the blocks they generate making their generation almost useless. It also makes
our networking part more complete.
2019-11-29 11:08:22 +03:00
Roman Khimov
734338ad70
network: move Version sending to the tcp transport
...
We have a race between reader and writer goroutines for the same connection
that leads to handshake failures when reader is faster to read the incoming
version (and try to reply to it) than writer is to write our own Version:
WARN[0000] peer disconnected addr="172.200.0.4:20334" peerCount=5 reason="invalid handshake: tried to send VersionAck, but didn't send Version yet
Fix it by moving Version sending before the reader loop starts.
2019-11-29 11:05:42 +03:00
Evgenii Stratonikov
0a9a742019
crypto: add invalid testcase
2019-11-29 11:03:03 +03:00
Evgenii Stratonikov
4d82419776
crypto: add missing tests for hash pkg
2019-11-29 11:03:03 +03:00
Roman Khimov
127f8418c8
rpc: refactor out runScriptInVM() from invokers
2019-11-28 19:13:58 +03:00
Roman Khimov
e216139108
rpc: implement server-side 'invoke' method, fix #346
2019-11-28 19:08:31 +03:00
Roman Khimov
2d41450ac9
Merge pull request #507 from nspcc-dev/dbft
...
network: plug in dBFT library
2019-11-27 15:22:40 +03:00
Roman Khimov
1b5da8401a
Merge pull request #521 from nspcc-dev/invokefunction
...
Invokefunction
2019-11-27 15:14:29 +03:00
Roman Khimov
e63b25d5ad
smartcontract: add user-facing testinvokefunction command
...
With a very special syntax.
2019-11-27 15:12:15 +03:00
Roman Khimov
c5e2b3ce73
Merge pull request #523 from nspcc-dev/fix-interim-block-changes-store
...
Fix interim block changes store
2019-11-27 14:51:29 +03:00
Vsevolod
1a9e98caf3
Merge pull request #522 from nspcc-dev/fix-invalid-tx-type-handling
...
Fix invalid tx type handling
2019-11-27 13:40:52 +03:00
Roman Khimov
467c9c146d
core: restore intermediate block-level store
...
Commit c80ee952a1
removed temporary store used
to contain changes of the block being processed. It's wrong in that the block
changes should be applied to the database in a single transaction so that
there wouldn't be any intermediate state observed from the outside (which is
possible now). Also, this made changes commiting persist them to the
underlying store effectively making our persist loop a no-op (and not
producing `persist completed` log lines that we love so much).
2019-11-27 13:28:13 +03:00
Roman Khimov
bb765ccfa7
core: gofmt
2019-11-27 13:27:47 +03:00
Roman Khimov
65bf0d01d5
transaction: don't hide decoding error by logging
...
Unknown transaction types are wrong and should be treated as errors.
2019-11-27 13:12:04 +03:00
Roman Khimov
05f3329ec0
vm: add IterBack to the Stack to iterate bottom-to-top
...
And use it to build user-facing stack representation because that's the order
that C# VM uses.
2019-11-27 13:00:11 +03:00
Roman Khimov
f3ed91a1f7
vm: properly convert arrays to stackItems
...
They should be arrays of stackItems, not arrays of values.
2019-11-27 13:00:11 +03:00
Roman Khimov
d04bc0cbe3
rpc: add marshaler for StackParamType
...
Makes stack output look better, no one cares about numbers.
2019-11-27 13:00:11 +03:00
Roman Khimov
3843451ec4
rpc: fix potential getblock handling problem, add a test for it
...
It should only accept hex strings and numbers, anything else is wrong.
2019-11-27 13:00:11 +03:00
Roman Khimov
1c883cc819
rpc: add invokescript tests
2019-11-27 13:00:11 +03:00
Roman Khimov
cf39171485
rpc: implement invokefunction, fix #347
...
Param getters were redone to return errors because otherwise bad FuncParam
values could lead to panic. FuncParam itself might be not the most elegant
solution, but it works good enough for now.
2019-11-27 13:00:11 +03:00
Evgenii Stratonikov
b5fb43bac9
network: don't reconnect to peers with identical ID
2019-11-27 11:58:17 +03:00
Evgenii Stratonikov
fdd5276d3e
network: plug in dBFT library
2019-11-27 10:57:22 +03:00
Vsevolod
34a38ccce1
Merge pull request #518 from nspcc-dev/feat/noerrors
...
core: do not return error in block.getHashableData
2019-11-26 16:20:46 +03:00
Evgenii Stratonikov
af61571ba0
core: do not return error in block.getHashableData
2019-11-26 15:27:37 +03:00
Evgenii Stratonikov
ab922c1810
util: add missing tests for Uint256
2019-11-25 11:55:26 +03:00
Evgenii Stratonikov
d018dc4e56
util: add missing tests for Uint160
2019-11-25 11:55:26 +03:00
Evgenii Stratonikov
18075bc5c8
util: add missing tests for Fixed8
2019-11-25 11:54:59 +03:00
Evgenii Stratonikov
116d4c656e
util: make tests use require
2019-11-25 11:53:49 +03:00
Roman Khimov
ea9bc22510
Merge pull request #513 from nspcc-dev/feat/rpc_param
...
This PR does 3 things:
adds array parameter unmarshalling
extend Param with convenient methods
refactor tests into using tables to make it easier add new tests
(part of #347 solution)
2019-11-22 18:47:59 +03:00
Roman Khimov
eb84ae49da
Merge pull request #508 from nspcc-dev/feature/getvalidators_420
...
core: Implement getvalidators interop
2019-11-22 18:15:08 +03:00
Evgenii Stratonikov
c9d5f8b89c
rpc: cover stack_param with more tests
...
Use require/assert instead of builtin facilities.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-22 13:20:15 +03:00
Evgenii Stratonikov
0f9024d177
rpc: make client default values constants
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-22 12:57:10 +03:00
Evgenii Stratonikov
3afcd784f0
rpc: refactor tests
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-22 12:57:07 +03:00
Evgenii Stratonikov
d5fa31cecd
rpc: trim spaces in tests once
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-22 12:57:03 +03:00
Evgenii Stratonikov
c8987eda32
rpc: add array param type and tests
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-22 12:56:58 +03:00
Evgenii Stratonikov
7331127556
rpc: make parameter type an enum
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-22 12:56:49 +03:00
Vsevolod Brekelov
38088b648a
crypto: add unit tests for public key
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
c03de88b41
core: unit tests blockchain state
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
d28f2dce05
core: unit test validator state
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
419b031e96
core: add weighted function for validators
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
672f314be0
core: nit refactoring and add godoc
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
8ae88593dd
core: blockchain fmt
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
3c4b0e955d
core: create blockchain state that represents state of the bc
...
move some util methods to blockchain state
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
d4e8846ed0
!squash core/crypto
2019-11-21 15:41:49 +03:00
Vsevolod Brekelov
c80ee952a1
core/crypto: add validators and interop for validators
...
add processing of validators while block persist;
add validator structure with decoding/encoding;
add validator get from store;
add EnrollmentTX and StateTX processing;
add pubkey decode bytes, unique and contains functions;
2019-11-21 15:23:35 +03:00
Roman Khimov
aae3e217a8
rpc/cli: implement contract deployment, fix #474
...
It's used like this:
./bin/neo-go contract deploy -i 1-print.avm -c neo-go.yml -e http://localhost:20331 -w KxDgvEKzgSBPPfuVfw67oPQBSjidEiqTHURKSDL1R7yGaGYAeYnr -g 100
2019-11-20 16:12:40 +03:00
Roman Khimov
310104667d
rpc: refactor out reusable parts of CreateRawContractTransaction()
...
Signing and inputs/outputs management is common for different transactions, so
make separate functions for them.
2019-11-20 16:12:40 +03:00
Roman Khimov
7d89ccdb6f
rpc: implement YAML marshaling/unmarshaling for StackParamType
2019-11-20 16:12:40 +03:00
Roman Khimov
34e2122e58
core: only check tx against mempool if it's not in the block
...
Fixes failure to process transaction from the block when it was relayed
initially:
WARN[0788] blockQueue: failed adding block into the blockchain blockHeight=7270 error="transaction 35088916403e5cf2152e16c3bc6e0fba20c955fba38543b9fa5c50a3d3a4ace5 failed to verify: invalid transaction due to conflicts with the memory pool" nextIndex=7271
WARN[0790] blockQueue: failed adding block into the blockchain blockHeight=7270 error="transaction 35088916403e5cf2152e16c3bc6e0fba20c955fba38543b9fa5c50a3d3a4ace5 failed to verify: invalid transaction due to conflicts with the memory pool" nextIndex=7271
WARN[0790] blockQueue: failed adding block into the blockchain blockHeight=7270 error="transaction 35088916403e5cf2152e16c3bc6e0fba20c955fba38543b9fa5c50a3d3a4ace5 failed to verify: invalid transaction due to conflicts with the memory pool" nextIndex=7271
2019-11-20 16:12:40 +03:00
Roman Khimov
27a57e1a2d
transaction: allow system fee specification in NewInvocationTX
...
It's not possible to create any deployment TX without it.
2019-11-20 16:12:40 +03:00
Roman Khimov
d93499cc6f
rpc: implement CalculateInputs for RPC client
...
Using getunspents RPC call.
2019-11-20 16:12:40 +03:00
Roman Khimov
826a29cc98
rpc: implement client-side getunspents
2019-11-20 16:12:40 +03:00
Roman Khimov
29882b076c
rpc: remove duplicating definition of UTXO
...
Port sorting methods to core.
2019-11-19 17:35:04 +03:00
Roman Khimov
d2bdae99e4
Merge pull request #492 from nspcc-dev/feature/interop_neo_tests1
...
core: add unit tests for interops_neo
2019-11-18 19:39:05 +03:00
Vsevolod Brekelov
d57b6e1091
core: add unit tests for interops_neo
2019-11-18 19:31:27 +03:00
Roman Khimov
5076704019
Merge pull request #502 from nspcc-dev/docker-compose-environment-for-neo-go
...
Create docker-compose environment
2019-11-18 17:21:11 +03:00
Evgeniy Kulikov
2f8b11dabf
reformat code at pkg/network/metrics/metrics.go
2019-11-18 17:17:21 +03:00
Evgeniy Kulikov
0a56d3ddbc
network: generate randomized server id
...
math/rand might generate same id's on one environment, so.. use crypto/rand for generation id's
2019-11-18 17:17:21 +03:00
Roman Khimov
53f666bc13
Merge pull request #504 from nspcc-dev/getunspents
...
Implement getunspents RPC API
2019-11-18 13:45:33 +03:00
Roman Khimov
5841d3931e
rpc: implement getunspents method, fix #473
2019-11-18 11:30:09 +03:00
Evgenii Stratonikov
e3098ed0f8
network: write messages atomically
...
Right now message can be written in several Write's so
concurrent calls of writeMsg() can in theory interleave.
This commit fixes it.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2019-11-18 09:31:00 +03:00
Evgenii Stratonikov
f65545023d
io: restrict ReadArray max array size
2019-11-15 22:39:21 +03:00
Roman Khimov
e4868cd429
core: implement EncodeBinary on value
...
Fixes panic:
panic: core.NotificationEvent does not have EncodeBinary(*BinWriter)
goroutine 97 [running]:
github.com/CityOfZion/neo-go/pkg/io.(*BinWriter).WriteArray(0xc004f7dda0, 0xabf820, 0xc004f7ddc0)
/home/rik/dev/neo-go/pkg/io/binaryWriter.go:45 +0x4b7
github.com/CityOfZion/neo-go/pkg/core.(*AppExecResult).EncodeBinary(0xc000635498, 0xc004f7dda0)
/home/rik/dev/neo-go/pkg/core/notification_event.go:73 +0x9c
github.com/CityOfZion/neo-go/pkg/core.putAppExecResultIntoStore(0xcf0020, 0xc001aafec0, 0xc000635498, 0xc004f81940, 0x40)
/home/rik/dev/neo-go/pkg/core/notification_event.go:33 +0xab
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).storeBlock(0xc000118000, 0xc006f89ea0, 0xc006f89ea0, 0x0)
/home/rik/dev/neo-go/pkg/core/blockchain.go:534 +0x2539
github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).AddBlock(0xc000118000, 0xc006f89ea0, 0xc004f368b0, 0x1)
/home/rik/dev/neo-go/pkg/core/blockchain.go:256 +0x91
github.com/CityOfZion/neo-go/pkg/network.(*blockQueue).run(0xc0048f4b80)
/home/rik/dev/neo-go/pkg/network/blockqueue.go:39 +0x14c
created by github.com/CityOfZion/neo-go/pkg/network.(*Server).Start
/home/rik/dev/neo-go/pkg/network/server.go:133 +0x272
2019-11-15 22:05:09 +03:00
Roman Khimov
1a5731e005
core: rework balance accounting
...
Store all unspents instead of balance values. Bump store version as this is an
incompatible change.
2019-11-15 22:05:09 +03:00
Roman Khimov
a16c2c3825
core: add NEP5 transfer tracking stub
...
See #498 .
2019-11-15 18:04:59 +03:00
Roman Khimov
01082a8988
core: fix nondeterministic txGetReferences() behavior
...
Ranging over map is nondeterministic and contracts may be unprepared for
that. Fixes #454 .
2019-11-15 18:04:59 +03:00
Roman Khimov
5e8122bfac
core: add contract hash into the error output for checkStorageContext()
...
Makes debugging things easier.
2019-11-15 18:04:59 +03:00
Roman Khimov
64e20508e0
vm: use hex for ByteArray JSON marshallization
...
It's way more convenient for different purposes.
2019-11-15 18:04:59 +03:00
Roman Khimov
5ce269c035
core: change notify implementation to save notifications
...
Fixes #453 and makes it possible to refer to these notifications later.
2019-11-15 18:04:59 +03:00
Evgenii Stratonikov
7080b78a6b
vm: fix a bug in collection serialize
...
When encountering already seen stack item we should fail
only if it is a collection. Duplicate Integers or ByteArrays are ok
because they can't lead to recursion.
2019-11-15 12:48:00 +03:00
Evgenii Stratonikov
1a08ad19aa
sc: implement io.Serializable for ParamType
2019-11-14 14:20:38 +03:00
Evgenii Stratonikov
2be18f91df
util: implement io.Serializable for Uint256
2019-11-14 14:20:38 +03:00
Evgenii Stratonikov
b16e56a47b
io: refactor BinReader.ReadArray()
...
Make it accept arbitrary slice pointer.
2019-11-14 14:19:58 +03:00
Evgenii Stratonikov
085ca7b770
network: implement Consensus payloads
2019-11-13 17:27:25 +03:00
Evgenii Stratonikov
ad9091d13d
io: implement generic array (de-)serialization
...
It is done through reflection and panics
in every unexpected situation.
2019-11-13 17:27:23 +03:00
Evgenii Stratonikov
61fdd5cde5
util: make Uint256Size public
2019-11-13 17:09:55 +03:00
Roman Khimov
394346fc26
core: add proper claim transactions processing
2019-11-11 15:35:57 +03:00
Roman Khimov
b05754deac
core: add Close() to blockchainer, implement it to properly close chain
...
Before it the deferred function in Run() was actually never able to properly
close the Store, so we weren't synching the latest state to the disk.
2019-11-08 12:19:54 +03:00
Evgenii Stratonikov
816d78b5ee
vm: redefine SHL/SHR limits based on integer size
2019-11-07 12:50:11 +03:00
Evgenii Stratonikov
439cd72294
vm: restrict BigInteger item size
2019-11-07 12:34:27 +03:00
Vsevolod
f686069f37
Merge pull request #481 from nspcc-dev/core-fix-init-corner-cases
...
Fix init corner cases
2019-11-06 18:36:40 +03:00
Vsevolod
11b372b45d
Merge pull request #480 from nspcc-dev/handshake-wait-for-version
...
network: wait for both Version messages before ACKing
2019-11-06 18:33:20 +03:00
Roman Khimov
c567307748
core: fix potential issue with header list init
...
If we're to receive some 500 headers (less than `headerBatchCount`) and quit
before receiving more of them we end up with clean `bc.headerList` that will
be inited going backwards to the `targetHash`, but code path doesn't add add
the `targetHash` itself which it should do in this particular case, otherwise
we end with no genesis block hash in the list.
2019-11-06 18:13:45 +03:00
Roman Khimov
d7f747fa9a
network: wait for both Version messages before ACKing
...
Otherwise the node might crash in `startProtocol` because of missing Version
field in the peer. And it also keeps the sequence correct, Version MUST be
sent first and ACKs can only follow it.
2019-11-06 18:05:50 +03:00
Roman Khimov
b5199625cd
core: init genesis block header if there are no other headers in DB
...
Fixes crash when restarted after the DB initialization and no blocks written
into the DB.
2019-11-06 17:58:19 +03:00
Roman Khimov
c16c2bf102
core: store current header reference on clean init
...
Missing it the following line could fail on subsequent restarts:
currHeaderHeight, currHeaderHash, err := storage.CurrentHeaderHeight(bc.store)
if the node was stopped before any headers had been received.
2019-11-06 17:56:06 +03:00
Roman Khimov
c062a8e288
Merge pull request #470 from nspcc-dev/config_updates_460
...
config: use logpath and address for server/rpc/monitoring, fix #460 .
2019-11-06 17:30:01 +03:00
Vsevolod Brekelov
d799c98cfe
io: move common function and add unit tests for it
2019-11-06 17:12:33 +03:00
Roman Khimov
aeef395c45
Merge pull request #476 from nspcc-dev/feature/neoUT
...
VM: Use JSON-based tests from neoVM
After the implementation of stack limits nothing is needed for us to pass reference JSON tests :)
The only thing that differs --- we do not compare stack in case of FAULT (which matches NEO 3 behavior).
Also two commits were reverted to match 2.x VM behavior.
2019-11-06 16:42:06 +03:00
Evgenii Stratonikov
6c002297cd
vm: implement json tests from neoVM
...
Add neo-vm submodule @master2.x .
Closes #196 .
2019-11-06 16:22:56 +03:00
Roman Khimov
9f7625d699
core: don't do useless DB reads if nothing was persisted
2019-11-06 16:10:37 +03:00
Roman Khimov
90d4299f0b
core: gofmt blockchain.go
2019-11-06 16:09:45 +03:00
Vsevolod Brekelov
11ce73af28
server: add log-path and address configuration
...
- LogPath can be configured through config
- node,rpc and monitoring address can be configured thought command line
or config
2019-11-06 15:58:54 +03:00
Roman Khimov
79d0c7446a
Merge pull request #478 from nspcc-dev/handshake-and-peers-fix
...
Fixes #458 , MaxPeers handling and some other related things.
2019-11-06 15:42:02 +03:00
Roman Khimov
7cf9a40468
network: fix MaxPeers, introduce AttemptConnPeers
...
Our node didn't respect the MaxPeers setting, fix it with a drop of random
connection when this limit is reached (to give a chance for newcomers to
communicate), but also introduce AttemptConnPeers setting to tune the number
of attempted connections.
This also raises the default MaxPeers for testnet/mainnet to 100, because
neo-go nodes love making friends.
2019-11-06 15:29:58 +03:00
Roman Khimov
31954bb20c
network: disallow double connections to the same peer
...
Makes no sense and C# node does it too.
2019-11-06 15:29:58 +03:00
Roman Khimov
d5a7ad2c47
network: fix data race in server peers map access
2019-11-06 15:29:58 +03:00
Roman Khimov
ec76ed23a5
network: rework peer handshaking, fix #458
...
This allows to start handshaking from both client and server (mainnet/testnet
nodes were seen to not care about string ordering for it), but still maintains
some sane checks in the process. It also makes functions thread-safe because
we have two goroutines servicing read and write side of the Peer connection,
so they can clash on access to the struct fields.
Add a test for it also.
2019-11-06 15:29:58 +03:00
Roman Khimov
e859e03240
network: split Peer's NetAddr into RemoteAddr and PeerAddr
...
As they are different things used for different purposes.
2019-11-06 15:26:24 +03:00
Roman Khimov
0296184da9
Merge pull request #475 from nspcc-dev/unitTests_wallet
...
Improve wallet unit testing.
2019-11-06 15:24:10 +03:00
Vsevolod Brekelov
75104a4a34
wallet: add unit tests
2019-11-06 13:15:47 +03:00
Evgenii Stratonikov
7d40d2f71e
vm: make StepOut/StepOver match original VM behavior
2019-11-06 12:25:04 +03:00
Evgenii Stratonikov
9ebb793009
vm: revert SUBSTR offset behavior to NEO 2.x
2019-11-06 12:15:55 +03:00
Evgenii Stratonikov
4c688355bc
vm: revert bool -> []byte conversion to NEO 2.x
2019-11-06 12:15:48 +03:00
067d9655bf
vm: restrict total stack item count
2019-11-06 11:03:43 +03:00
ed758458d5
vm: implement serialization interops in core package
2019-11-05 17:10:52 +03:00
412582dc78
vm: add more serialization tests
2019-11-05 17:04:14 +03:00
59f9c2bddc
vm: implement Array and Struct item serialization
2019-11-05 16:58:09 +03:00
e1d019e087
vm: implement Map item serialization
2019-11-05 16:57:48 +03:00
cd690803cf
vm: implement BigInteger item serialization
2019-11-05 15:15:43 +03:00
25f77257ce
vm: implement Boolean and ByteArray item serialization
2019-11-05 15:15:39 +03:00
Roman Khimov
de2eeb4671
rpc: add one to the block height for the getblockcount response
...
There is a difference in interpretation of what a block count is. neo-go nodes
currently respond to this request with the latest block number which is the
same number that neoscan.io shows. However, C# nodes deliberately do add one
to this number when answering to the getblockcount request to account for the
genesis block number 0.
This patch makes us consistent with C# nodes wrt to getblockcount behaviour.
2019-11-01 20:13:00 +03:00
Roman Khimov
70e4529848
rpc: add 0x
prefix to the getbestblockhash answer
...
For consistency with C# nodes that do add it to their answers.
2019-11-01 20:12:44 +03:00
Roman Khimov
e9f8b25776
network: add default MinPeers
...
If there is nothing specified in the configuration, set it to some reasonable
value.
2019-11-01 13:39:47 +03:00
Roman Khimov
b46dd295bc
rpc: add CORS workaround for RPC
...
This one enables our RPC to be called from the browser if there is a
need. It's insecure and not standards-compliant, thus this behaviour is
configurable is not enabled by default. It makes our node with this workaround
enabled compatible with neo-mon monitoring.
Originally debugged by @anatoly-bogatyrev in #464 .
2019-11-01 13:39:47 +03:00
Roman Khimov
b12a68994d
network: make minpeers configurable
...
Without it node in privnet tries to reconnect to four existing nodes
indefinitely for no good reason.
2019-10-31 15:10:00 +03:00
Roman Khimov
2f6e678a19
Merge pull request #463 from nspcc-dev/smartcontract-fixes
...
Smartcontract RPC fixes
2019-10-29 20:54:46 +03:00
Roman Khimov
6d1a09183b
Merge pull request #455 from nspcc-dev/prometheus_441
...
add prometheus monitoring, fix #441
2019-10-29 20:54:29 +03:00
Vsevolod Brekelov
d374175170
monitoring: add prometheus monitoring
...
add init metrics service which uses prometheus;
add configuration for metrics service;
add monitoring metrics for blockchain,rpc,server;
2019-10-29 20:51:17 +03:00
Roman Khimov
ebc1ba4f38
rpc/core: implement invokescript method, fix #348
...
Extend Blockchainer with one more method to spawn a VM for test runs and use
it to run scripts. Gas consumption is not counted or limited in any way at the
moment (see #424 ).
2019-10-29 18:31:39 +03:00
Roman Khimov
94776b8a1f
vm: add MarshalJSON to the Stack
...
To easily dump it in a known format.
2019-10-29 18:26:59 +03:00
Roman Khimov
47f66dfbf3
vm: add State() method to get a state description
2019-10-29 18:26:04 +03:00
Roman Khimov
a568740f56
core: deduplicate a part of VM setup into spawnVMWithInterops()
2019-10-29 17:10:50 +03:00
Roman Khimov
3d6cf3a647
rpc: drop useless cases from server
...
Let them be handled by `default` statement returning a proper error instead of
plaintext message.
2019-10-29 16:48:14 +03:00
c7f0b7bd68
vm: restrict max invocation stack size
2019-10-29 15:09:17 +03:00
Roman Khimov
579aa31ddd
compiler: drop useless options parameter to Compile()
...
It's not used in any way there.
2019-10-29 13:02:54 +03:00
Roman Khimov
e319c6c638
cli: move avm inspect from vm to contract command
...
Make inspect work with avms by default and with go files if told so. In the
end this makes our CLI interface more consistent and usable. Drop useless
CompileAndInspect() compiler method along the way.
2019-10-29 12:56:44 +03:00
Roman Khimov
ae7687422c
vm: clear state in Load()
...
Make VM usable after the first run.
2019-10-29 12:53:09 +03:00