Commit graph

652 commits

Author SHA1 Message Date
Roman Khimov
de2eeb4671 rpc: add one to the block height for the getblockcount response
There is a difference in interpretation of what a block count is. neo-go nodes
currently respond to this request with the latest block number which is the
same number that neoscan.io shows. However, C# nodes deliberately do add one
to this number when answering to the getblockcount request to account for the
genesis block number 0.

This patch makes us consistent with C# nodes wrt to getblockcount behaviour.
2019-11-01 20:13:00 +03:00
Roman Khimov
70e4529848 rpc: add 0x prefix to the getbestblockhash answer
For consistency with C# nodes that do add it to their answers.
2019-11-01 20:12:44 +03:00
Roman Khimov
e9f8b25776 network: add default MinPeers
If there is nothing specified in the configuration, set it to some reasonable
value.
2019-11-01 13:39:47 +03:00
Roman Khimov
b46dd295bc rpc: add CORS workaround for RPC
This one enables our RPC to be called from the browser if there is a
need. It's insecure and not standards-compliant, thus this behaviour is
configurable is not enabled by default. It makes our node with this workaround
enabled compatible with neo-mon monitoring.

Originally debugged by @anatoly-bogatyrev in #464.
2019-11-01 13:39:47 +03:00
Roman Khimov
b12a68994d network: make minpeers configurable
Without it node in privnet tries to reconnect to four existing nodes
indefinitely for no good reason.
2019-10-31 15:10:00 +03:00
Roman Khimov
2f6e678a19
Merge pull request #463 from nspcc-dev/smartcontract-fixes
Smartcontract RPC fixes
2019-10-29 20:54:46 +03:00
Roman Khimov
6d1a09183b
Merge pull request #455 from nspcc-dev/prometheus_441
add prometheus monitoring, fix #441
2019-10-29 20:54:29 +03:00
Vsevolod Brekelov
d374175170 monitoring: add prometheus monitoring
add init metrics service which uses prometheus;
add configuration for metrics service;
add monitoring metrics for blockchain,rpc,server;
2019-10-29 20:51:17 +03:00
Roman Khimov
ebc1ba4f38 rpc/core: implement invokescript method, fix #348
Extend Blockchainer with one more method to spawn a VM for test runs and use
it to run scripts. Gas consumption is not counted or limited in any way at the
moment (see #424).
2019-10-29 18:31:39 +03:00
Roman Khimov
94776b8a1f vm: add MarshalJSON to the Stack
To easily dump it in a known format.
2019-10-29 18:26:59 +03:00
Roman Khimov
47f66dfbf3 vm: add State() method to get a state description 2019-10-29 18:26:04 +03:00
Roman Khimov
a568740f56 core: deduplicate a part of VM setup into spawnVMWithInterops() 2019-10-29 17:10:50 +03:00
Roman Khimov
3d6cf3a647 rpc: drop useless cases from server
Let them be handled by `default` statement returning a proper error instead of
plaintext message.
2019-10-29 16:48:14 +03:00
c7f0b7bd68 vm: restrict max invocation stack size 2019-10-29 15:09:17 +03:00
Roman Khimov
579aa31ddd compiler: drop useless options parameter to Compile()
It's not used in any way there.
2019-10-29 13:02:54 +03:00
Roman Khimov
e319c6c638 cli: move avm inspect from vm to contract command
Make inspect work with avms by default and with go files if told so. In the
end this makes our CLI interface more consistent and usable. Drop useless
CompileAndInspect() compiler method along the way.
2019-10-29 12:56:44 +03:00
Roman Khimov
ae7687422c vm: clear state in Load()
Make VM usable after the first run.
2019-10-29 12:53:09 +03:00
Roman Khimov
006337b1f8 network: rework discovery with rwmutex, add test
Keeping run() as the owner of all maps would mean adding at least three more
channels to keep address getters with thread-safety. But then there also is a
race between requestToWork() and run() which is way harder to solve with
channels because there are lots of possibilities for deadlocks. So rework all
of this with good old mutexes.

While at it, fix `requestCh` handling in the inner select of run, it will waste
one loop to handle it, so we should add one to the `requested`.

Fixes #445.
2019-10-28 13:37:27 +03:00
Roman Khimov
77a50d6dc6 network: remove useless checks in discovery
These are useless.
2019-10-27 16:11:32 +03:00
Roman Khimov
a48e297252
Merge pull request #451 from nspcc-dev/call-isolation
vm: implement stack isolation opcodes, fix #192
2019-10-25 18:27:08 +03:00
Roman Khimov
f0cffe0d3f vm: implement stack isolation opcodes, fix #192
No tests yet.
2019-10-25 18:25:12 +03:00
Roman Khimov
99bbad331d core: add issue tx processing
Count available asset quantity, match C# code.
2019-10-25 17:20:12 +03:00
Roman Khimov
2611e9ab5c smartcontract: fix PropertyState definitions
Wrong bits were used to represent flags which is important for contracts
created via interop. Fixes contracts failing to store things:

WARN[16278] contract invocation failed                    block=3773025 err="error encountered at instruction 3435 (SYSCALL): failed to invoke syscall: contract c9d870d7857e956d82290d5df19de3133c107815 can't have storage" tx=fa695eea240b7b4dbb6f42ea6335447a764d8b629c40b7812ea3bca16b1f098d
WARN[16278] contract invocation failed                    block=3773025 err="error encountered at instruction 1279 (SYSCALL): failed to invoke syscall: contract 97210e7c98582151ceb37f9748c9a1d27d9ae6fd can't have storage" tx=0144d84038149fa0cf1f7912f7d5854fa5f3670f5b4217789c1441f9fd52d27b
2019-10-25 11:05:58 +03:00
Roman Khimov
9ff55ac02c network: move inventory semantic check into common code
As per @volekerb suggestion.
2019-10-24 13:12:02 +03:00
Roman Khimov
45cac07643 core: remove transactions from MemPool when they're stored in block
Extend MemPool with Remove() and use it. Fixes #446.
2019-10-24 13:12:02 +03:00
Roman Khimov
184d8a0180 core: fix missing unlock in MemPool TryAdd()
Fixes deadlock on subsequent MemPool operations.
2019-10-24 13:12:01 +03:00
Roman Khimov
257ba1c2ad network: implement getdata message handling in server
To properly implement relaying we need to answer these requests from other
nodes.
2019-10-24 13:12:01 +03:00
Roman Khimov
8d0ed9259c transaction: fix invocation TX consistency in NewInvocationTX()
NewInvocationTX() returned a version number one transaction that actually
failed to pass that version down to the invocation data which lead to
serialization/deserialization inconsistency.
2019-10-23 14:50:24 +03:00
Roman Khimov
1250bf9579
Merge pull request #443 from nspcc-dev/spellcheck
spellcheck and comments fix
2019-10-22 18:01:40 +03:00
Vsevolod Brekelov
8ee421db14 fix spelling and godoc comments 2019-10-22 17:56:03 +03:00
Roman Khimov
a9a2a3c711 vm: fix tests failing, follow-up to #426
Deal with input and transitional VM states properly.
2019-10-22 15:20:44 +03:00
Roman Khimov
3cbb699eb7
Merge pull request #426 from nspcc-dev/logger_247
Change fmt.Println to log, close #247.
2019-10-22 14:41:30 +03:00
Roman Khimov
558a25cbd2
Merge pull request #438 from nspcc-dev/fix/437
vm: copy slice in NEWARRAY/NEWSTRUCT

Fixes #437.
2019-10-22 14:36:32 +03:00
Vsevolod Brekelov
e2bfff8666 vm: removed mute mode and pushed logging to upper lvl
VM should be responsible for code execution and in case anyone interested in additional logging or errors they could handle them like we do it iin cli.
2019-10-22 13:44:14 +03:00
Evgenii Stratonikov
b2609786e9 vm: copy slice in NEWARRAY/NEWSTRUCT
When performing NEWARRAY on a Struct or NEWSTRUCT on a Array,
underlying slice needs to be copied, because when it's capacity
doesn't matches it's length, underlying storage will be used
for appends even if it is already pointed at by another slice.
2019-10-22 13:34:35 +03:00
Roman Khimov
f1750d117b
Merge pull request #440 from nspcc-dev/db-dump-restore
DB dump and restore, fixes #436.
2019-10-21 15:42:05 +03:00
Vsevolod Brekelov
f2805541cb vm: fix style and comments 2019-10-21 14:22:17 +03:00
Roman Khimov
07c2105aa5 core: log values from the store in persist()
We're about stored values here, so print those, which avoids blocking in
bc.HeaderHeight() and removes duplication between blockHeight and
persistedHeight. Fixes saving the blockchain on exit (deferred function in
Run() blocked in persist()).

Test modification was required because storeBlocks() doesn't actually save
headers and thus TestGetTransaction started to fail on persist().
2019-10-21 14:18:09 +03:00
Roman Khimov
70407f0c19 core: remove unused context parameter from persist() 2019-10-21 14:18:09 +03:00
Vsevolod Brekelov
063fe5be9d unspentcoinstate: fix branch never executed 2019-10-21 12:40:11 +03:00
Vsevolod Brekelov
05ea84454c contracttest: fix err checking branch 2019-10-21 12:31:17 +03:00
Roman Khimov
b533dfceba core: don't panic on init when there are less than 2000 hashes
If you're to sync less than 2000 headers no batched header key-value is
gonna be written into the DB and init() would panic because
bc.headerList.Len() would return 0. Use genesis block as a target in this
case.
2019-10-21 08:37:01 +03:00
Vsevolod Brekelov
fe39c565b9 codestyle: redundant type conversion 2019-10-18 18:39:31 +03:00
Vsevolod Brekelov
1afdb895a1 codestyle: declare empty slice
https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices
2019-10-18 18:38:33 +03:00
Vsevolod Brekelov
3560ada669 codestyle: use camel case 2019-10-18 18:36:54 +03:00
Vsevolod Brekelov
59e3bd2fa9 codestyle: fix error string should not be capitalized 2019-10-18 18:34:58 +03:00
Vsevolod Brekelov
0daef50b9d cli: fix for error wrapping 2019-10-17 18:23:49 +03:00
Vsevolod Brekelov
5b81110d16 cli: print logo via shell 2019-10-17 18:23:49 +03:00
Vsevolod Brekelov
ced2022273 storage: change println error for botldb to log 2019-10-17 18:23:49 +03:00
Vsevolod Brekelov
31212676f2 rpc: swap fmt to log for response handling 2019-10-17 18:23:49 +03:00
Evgenii Stratonikov
487570153b vm: restrict max item size in PUSHDATA4 2019-10-17 17:10:00 +03:00
Evgenii Stratonikov
cae431b844 vm: restrict max item size in CAT 2019-10-17 17:09:42 +03:00
Vsevolod
67219b9439
Merge pull request #433 from nspcc-dev/technical-fixes
Technical fixes
2019-10-17 12:41:18 +03:00
Roman Khimov
19a0d16751
Merge pull request #427 from nspcc-dev/feat/max_size
Arrays, Structs and Maps have maximum size defined as MaxArraySize.
We need to return an error in case collection becomes too big.
Part of #373.
2019-10-17 12:39:11 +03:00
Roman Khimov
3fc2bf5452 *: fix some misspellings
Goreport:
   neo-go/pkg/core/contract_state_test.go
        Line 21: warning: "Contracto" is a misspelling of "Contraction" (misspell)
        Line 64: warning: "Contracto" is a misspelling of "Contraction" (misspell)

   neo-go/pkg/core/interop_neo.go
        Line 420: warning: "succeedes" is a misspelling of "succeeds" (misspell)

   neo-go/pkg/network/discovery.go
        Line 118: warning: "succeded" is a misspelling of "succeeded" (misspell)
        Line 128: warning: "successfuly" is a misspelling of "successfully" (misspell)
2019-10-17 12:30:24 +03:00
Roman Khimov
1b5ea67737 core: fix ineffassign suggestion
Goreport:
  Line 467: warning: ineffectual assignment to persisted (ineffassign)
2019-10-17 12:27:15 +03:00
Roman Khimov
e5ed7a7eb7 io: fix lintian suggestions in test code
golint:
  pkg/io/binaryrw_test.go:25:11: should omit type []byte from declaration of var bin; it will be inferred from the right-hand side
  pkg/io/binaryrw_test.go:42:11: should omit type []byte from declaration of var bin; it will be inferred from the right-hand side
  pkg/io/binaryrw_test.go:118:7: should omit type string from declaration of var str; it will be inferred from the right-hand side
2019-10-17 12:25:36 +03:00
Roman Khimov
6029d5a888 payload: drop useless type declaration
golint suggests:
    pkg/network/payload/address.go:48:12: should omit type net.IP from declaration of var netip; it will be inferred from the right-hand side
2019-10-17 12:19:41 +03:00
Roman Khimov
288000a8af storage: drop useless nil assignment in leveldb code
golint suggests:
   Line 24: warning: should drop = nil from declaration of var opts; it is the zero value (golint)
2019-10-17 12:18:16 +03:00
Evgenii Stratonikov
6d7016c3b9 vm: restrict max size in SETITEM 2019-10-17 12:17:06 +03:00
Evgenii Stratonikov
6f1f9e56bb vm: restrict max size in PACK 2019-10-17 12:17:06 +03:00
Evgenii Stratonikov
2d56c66bde vm: restrict max size in APPEND 2019-10-17 12:17:01 +03:00
Roman Khimov
3f6104ef4d network: ignore consensus messages
It's a temporary stub until proper encoding/decoding is implemented. It's
useful for testnet/mainnet connections because without it consensus message
receival leads to peer disconnection.
2019-10-17 11:58:27 +03:00
Evgenii Stratonikov
8abcaeee6f vm: restrict max size in NEWARRAY/NEWSTRUCT 2019-10-17 11:48:48 +03:00
Roman Khimov
56459c6c63 network: remove port check from handleVersionCmd()
It's bogus and no other node implementation has anything like that. It fires
up for no good reason in the case when some other node connects to us and it
obviously doesn't use its listening port for it.
2019-10-17 11:19:24 +03:00
Roman Khimov
2245fedbb1 core: deduplicate state commit methods, use interim MemCachedStore
commit methods duplicated putSmthIntoStore functions, but have MemCachedStore
now that can easily substitute for a Batch, especially given that interop
needs something like that for its storage purposes anyway.
2019-10-16 17:33:45 +03:00
Roman Khimov
fc0031e5aa core: move write caching layer into MemCacheStore
Simplify Blockchain and associated functions, deduplicate code, fix Get() and
Seek() implementations.
2019-10-16 17:33:45 +03:00
Roman Khimov
4822c736bb
Merge pull request #418 from nspcc-dev/various-verification-fixes2
Transaction verification fixes, interops and block verification. Fixes #12.
2019-10-15 19:11:00 +03:00
Roman Khimov
8926cbe368 core: fix potential data race in logging code
Spotted in CircleCI build with Go 1.12:

WARNING: DATA RACE
Write at 0x00c00011095c by goroutine 88:
  sync/atomic.SwapInt32()
      /usr/local/go/src/runtime/race_amd64.s:249 +0xb
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).persist()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:483 +0x172
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run.func2()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:191 +0x50

Previous read at 0x00c00011095c by goroutine 64:
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).persist()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:492 +0x4be
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run.func2()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:191 +0x50

Goroutine 88 (running) created at:
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:190 +0x264

Goroutine 64 (finished) created at:
  github.com/CityOfZion/neo-go/pkg/core.(*Blockchain).Run()
      /go/src/github.com/CityOfZion/neo-go/pkg/core/blockchain.go:190 +0x264
2019-10-15 18:58:22 +03:00
Roman Khimov
a6610ba082 core: verify blocks, fix #12
This adds the following verifications:
 * merkleroot check
 * index check
 * timestamp check
 * witnesses verification

VerifyWitnesses is also renamed to verifyTxWitnesses here to not confuse it
with verifyBlockWitnesse and to hide it from external access (no users at the
moment).
2019-10-15 18:58:17 +03:00
Roman Khimov
03c20f1876 core: make ContractState receiver names consistent
Linter isn't happy with our recent changes:

pkg/core/contract_state.go:109:1: receiver name cs should be consistent with previous receiver name a for ContractState
pkg/core/contract_state.go:114:1: receiver name cs should be consistent with previous receiver name a for ContractState
pkg/core/contract_state.go:119:1: receiver name cs should be consistent with previous receiver name a for ContractState

But actually `a` here most probably is a copy-paste from AssetState methods,
so fit the old code to match the new one.
2019-10-15 12:56:25 +03:00
Roman Khimov
aec6a5f029 core/config: make block and transaction verification configurable
Enable transaction verification for privnets and tests, testnet can't
successfuly verify block number 316711 with it enabled and mainnet stops at
105829.
2019-10-15 12:56:25 +03:00
Roman Khimov
56dcff2894 core: invoke contracts for Invocation TXes
They can have some side-effects that future invocations rely on.
2019-10-15 12:56:25 +03:00
Roman Khimov
74590551c4 core: add some interops
This also changes Verify to VerifyTx and VerifyWitnesses, because there is a
need to pass a block for some interop functions.
2019-10-15 12:56:25 +03:00
Roman Khimov
667f346c04 core: improve error message in AddBlock test 2019-10-15 12:56:25 +03:00
Roman Khimov
8266a5ce19 core: export GetContractState/GetScriptHashesForVerifying via Blockchainer
These are gonna be used by interops and are also useful in general.
2019-10-15 12:56:25 +03:00
Roman Khimov
19fd7f844e core: add GetUnspentCoinState() for future interops 2019-10-15 12:56:25 +03:00
Roman Khimov
bfddf9b3f6 core: implement StorageItem for future interops 2019-10-15 12:56:25 +03:00
Roman Khimov
2156d5db73 core: add put/delete functions for ContractState
These are gonna be used by interops.
2019-10-15 12:56:25 +03:00
Roman Khimov
58d9b79fe3 core: avoid duplicates in GetScriptHashesForVerifyingClaim()
As they can break VerifyWitnesses().
2019-10-15 12:56:25 +03:00
Roman Khimov
35824728eb core: add putAssetStateIntoStore()
Will be used by interops.
2019-10-15 12:56:25 +03:00
Roman Khimov
96934cfeab core: return error from verifyOutputs()
Make debugging easier.
2019-10-15 12:56:25 +03:00
Roman Khimov
ab4ff79e02 core: fix typo in GetTransactionResults() 2019-10-15 12:56:25 +03:00
Roman Khimov
cffe8d0ee2 core: return error from verifyResults()
Make it easier to find out what the real problem is.
2019-10-15 12:56:25 +03:00
Roman Khimov
16bc5296cb block: return error from Verify
Don't hide real problem behind the bool value. Makes it easier to identify
problems when looking at log messages.
2019-10-15 12:56:25 +03:00
Roman Khimov
258f397b9a core: append transactions to the block in GetBlock()
We want to get a full block, so it has to have transactions
inside. Unfortunately our tests were used to this wrong behavior and utilized
completely bogus transactions without data that couldn't be persisted, so fix
that also.
2019-10-15 12:56:25 +03:00
Roman Khimov
d007cc00cc vm: add script check functions
These are required for future interops.
2019-10-15 12:56:25 +03:00
Roman Khimov
7ab58ff8cb keys: make public key's IsInfinity() public
It's gonna be used in interops for key validity check.
2019-10-15 12:56:25 +03:00
Roman Khimov
c5a4cfaebe smartcontract: add CreateSignatureRedeemScript()
It's very similar to the CreateMultiSigRedeemScript() and it will be used in
interop functions.
2019-10-15 12:56:25 +03:00
Roman Khimov
238c590ddb core: fix contract state's Properties to use PropertyState
PublishTX only had one of these flags, but newer contracts (created via the
interop function) can have more and these flags are aggregated into one field
that uses PropertyState enumeration (it's used to publish contract, so
supposedly it's also a nice choice for contract state storage).
2019-10-15 12:56:25 +03:00
Roman Khimov
acb7ef7fbd vm: support uint8 and uint64 in makeStackItem()
Convenience to avoid casts, uint64 is also a bit special in that it can't be
converted to int64 without data loss.
2019-10-15 12:56:25 +03:00
Roman Khimov
6b70c5f2bd keys: rename New*FromRawBytes to New*FromASN1
RawBytes is too confusing and may be read as being compatible with
NEO-serialized format.
2019-10-15 12:56:25 +03:00
Roman Khimov
782ca64d92 vm: accept uint16 in makeStackItem() 2019-10-15 12:56:25 +03:00
Roman Khimov
78861485b6 storage: simplify MemoryBatch
It's used a lot and it looks a lot like MemoryStore, it just needs not to
return errors from Put and Delete, so make it use MemoryStore internally with
adjusted interface.
2019-10-15 12:56:25 +03:00
Roman Khimov
13bf2618ef storage: improve PutBatch for MemoryStore
Make it look more like a real transaction, put/delete things with a single
lock. Make a copy of value in Put also, just for safety purposes, no one knows
how this value slice can be used after the Put.
2019-10-15 12:56:25 +03:00
Roman Khimov
3ada92944a storage: drop Len from the Batch interface
It's almost meaningless now and we can easily live without it.
2019-10-15 12:56:25 +03:00
Roman Khimov
e111892653 storage: redo DB testing
Make generic tests for all Store implementations, deduplicate tests. Implement
Delete() tests, test Seek() better, add LevelDB tests (finally!).
2019-10-15 12:56:25 +03:00
Vsevolod Brekelov
5109b2e3e7 vm: fix cli comment 2019-10-14 18:38:05 +03:00
Vsevolod Brekelov
591d5eafbe vm: add stepInto,stepOver,stepOut
Original C# vm debugger behavior
2019-10-14 18:37:11 +03:00
Roman Khimov
48b6a427cf storage: add Delete method for Batch and Store
It's gonna be used by Storage and Contract interops, both can delete their
data.
2019-10-07 19:40:11 +03:00
Roman Khimov
add9368e9d storage: use strings as keys for memory batch
Using pointers is just plain wrong here, because the batch can be updated with
newer values for the same keys.

Fixes Seek() to use HasPrefix also because this is the intended behavior.
2019-10-07 17:05:53 +03:00
Roman Khimov
dca332f333 vm: use new Context.Next() to properly dump programs
Fix #295, deduplicate code and add `inspect` parameter to the vm command to
dump AVMs (`contract inspect` works with Go code).
2019-10-04 16:13:39 +03:00
Roman Khimov
53a3b18652 vm: completely separate instruction read and execution phases
Make Context.Next() return both opcode and instruction parameter if any. This
simplifies some code and needed to deal with #295.
2019-10-04 16:13:39 +03:00
Roman Khimov
1bf232ad50 vm: introduce TryBool() for Element and use it in VerifyWitnesses
Script can return non-bool results that can still be converted to bool
according to the usual VM rules. Unfortunately Bool() panics if this
conversion fails which is OK for things done in vm.execute(), but certainly
not for VerifyWitnesses(), thus there is a need for TryBool() that will just
return an error in this case.
2019-10-04 16:13:39 +03:00
Roman Khimov
8441b31b4b vm: accept uint32 in makeStackItem()
Interop services routinely push such things (block index, blockchain height)
onto the stack.
2019-10-04 16:13:39 +03:00
Roman Khimov
d62a367900 vm: add Value() method to Element
It gives access to the internal value's Value() which is essential for interop
functions that need to get something from InteropItems. And it also simplifies
some already existing code along the way.
2019-10-04 16:13:39 +03:00
Roman Khimov
0c963875af vm: check for fault flag first in Run()
Switch cases are evaluated sequentially and the fault case is top-priority to
handle.
2019-10-04 16:13:39 +03:00
Roman Khimov
705c7f106f vm: don't panic if there is no result in PopResult()
This function is intended to be ran outside of the execute's panic recovery
mechanism, so it shouldn't panic if there is no result.
2019-10-04 16:13:39 +03:00
Roman Khimov
a357d99624 vm: introduce MaxArraySize constant
This is both for #373 and for interop functions that have to check some
inputs.
2019-10-04 16:13:39 +03:00
Roman Khimov
cfa0c13322 vm: add InteropItem type for interop data
This is an opaque data item that is to be used by the interop functions.
2019-10-04 16:13:39 +03:00
Roman Khimov
da2156f955 vm: add batched RegisterInteropFuncs 2019-10-04 16:13:39 +03:00
Roman Khimov
26e3b6abbe vm: extend interops to contain price
The same way C# node does.
2019-10-04 16:13:39 +03:00
Roman Khimov
ceca9cdb67 core/vm: implement contract storage and script retrieval
Fixes script invocations via the APPCALL instruction. Adjust contract state
field types accordingly.
2019-10-04 16:13:39 +03:00
Roman Khimov
b7c1efe7e3 core: fix References() result key type
If the block references two ouputs in some other transaction the code failed
to verify it because of key collision. C# code implements it properly by using
full CoinReference type as a key, so let's do it in a similar fashion.
2019-10-01 13:41:26 +03:00
Roman Khimov
8537700b7b core: sort hashes and witnesses in VerifyWitnesses()
Fixes failure to verify some multi-witnesses transactions in Testnet chain. C#
code contains similar logic.
2019-10-01 13:41:26 +03:00
Roman Khimov
dd0dd2d031 core: add claim-specific GetScriptHashesForVerifying()
Claim transactions have different logic in C# node, so we need to
implement it too. It's not the most elegant way to fix it, but let's make it
work first and then refactor if and where needed. Fixes verification of Claim
transactions.
2019-10-01 13:41:26 +03:00
Roman Khimov
854fb187a3 core: verify transactions in AddBlock() 2019-10-01 13:41:26 +03:00
Roman Khimov
1095abb04c core: fix default asset expiration to match C# code
Fixes bogus verification failures for non-native assets.
2019-10-01 13:41:26 +03:00
Roman Khimov
832c56f97d core: fix DutyFlag check in GetScriptHashesForVerifying()
It's a flag (used by Share and Invoice asset types), so it should be checked
like a flag, the same way C# node does.
2019-10-01 13:41:26 +03:00
Roman Khimov
4ae18e8ffc block: check for Transaction length before messing with txes
Fixes panic two lines below. Blocks without transactions are invalid by
definition, so there is a need to adjust tests accordingly.
2019-10-01 13:41:26 +03:00
Roman Khimov
c3591d8897 vm: fix CHECKMULTISIG to comply with NEO VM implementation
Testing with testnet quickly revealed that our code has an issue when the key
count doesn't equal signature count, fix it and add some comments.
2019-10-01 13:41:26 +03:00
Roman Khimov
fac778d3dd
Merge pull request #414 from nspcc-dev/persistence-rewamp
What started as an attempt to fix #366 ended up being quite substantial refactoring of the Blockchain->Store and Server->Blockchain interactions. As usually, some additional problems were noted and fixed along the way. It also accidentally fixes #410.
2019-09-27 17:55:43 +03:00
Roman Khimov
7779ad6c28 storage: always return ErrKeyNotFound when key is not found
It was true for MemoryStore and BoltDB, but not for LevelDB and Redis.
2019-09-27 15:42:36 +03:00
Roman Khimov
5ff1399d54 storage: unexport MemoryStore mutex
It shouldn't be available from outside.
2019-09-27 15:42:36 +03:00
Roman Khimov
9617a6f9e9 network: fix requestBlocks() for (headers == blocks + 1)
In the very specific case when the list of headers received is exactly one
block ahead of the chain of full blocks requestBlocks() failed to generate
request to get the next full block.
2019-09-27 15:42:35 +03:00
Roman Khimov
3c40a53c4d core: no need to Close() Blockchain in tests
It's a Store method actually and for every Blockchain that uses Run() the
Close() will be handled automatically on exit because of context magic.
2019-09-27 15:42:35 +03:00
Roman Khimov
0dbdbb9c2a storage: remove RedisBatch in favor of MemoryBatch
They're also almost the same.
2019-09-27 15:42:35 +03:00
Roman Khimov
af557cea19 storage: deduplicate BoltDBBatch/MemoryBatch
BoltDB doesn't have internal batching mechanism, thus we have a substitute for
it, but this substitute is absolutely identical to MemoryBatch, so it's better
to unify them and import ac5d2f94d3 fix into the
MemoryBatch.
2019-09-27 15:42:35 +03:00
Roman Khimov
8bf37f45fc storage: remove impossible error branch from MemoryStore
MemoryStore is always successful, so this branch makes no sense.
2019-09-27 15:42:35 +03:00
Roman Khimov
4a732d97ac storage: add comments about errors from MemoryStore
It never returns errors for some operations and this knowledge can be used by
other code.
2019-09-27 15:42:35 +03:00
Roman Khimov
920d7c610c core: remove blockCache, use MemoryStore, redesign persist()
Commit 578ac414d4 was wrong in that it saved
only a part of the block, so depending on how you use blockchain, you may
still see that the block was not really processed properly. To really fix it
this commit introduces intermediate storage layer in form of memStore, which
actually is a MemoryStore that supports full Store API (thus easily fitting
into the existing code) and one extension that allows it to flush its data to
some other Store.

It also changes AddBlock() semantics in that it only accepts now successive
blocks, but when it does it guarantees that they're properly added into the
Blockchain and can be referred to in any way. Pending block queing is now
moved into the server (see 8c0c055ac657813fe3ed10257bce199e9527d5ed).

So the only thing done with persist() now is just a move from memStore to
Store which probably should've always been the case (notice also that
previously headers and some other metadata was written into the Store
bypassing caching/batching mechanism thus leading to some inefficiency).
2019-09-27 15:42:35 +03:00
Roman Khimov
c531dc0bde network: add block queue
This one will replace blockCache in Blockchain itself as it can and should be
external from it. The idea is that we only feed successive blocks into the
Blockchain and it only stores valid proper Blockchain and nothing else.
2019-09-27 13:00:09 +03:00
Roman Khimov
ab8d9c59d6
Merge pull request #412 from nspcc-dev/feature/map
VM: implement maps, closes #359.
2019-09-25 19:00:26 +03:00
Evgenii Stratonikov
c7c4291774 vm: simplify APPEND implementation a bit 2019-09-25 17:53:36 +03:00
Evgenii Stratonikov
13d7770c68 vm: support Map in ARRAYSIZE
Also make logic the same as in reference implementation
and add tests.
2019-09-25 17:53:36 +03:00
Evgenii Stratonikov
c66f493017 vm: compare Map by reference in EQUAL 2019-09-25 17:53:36 +03:00
Evgenii Stratonikov
c0be2b2a99 vm: support Map in REMOVE 2019-09-25 17:53:36 +03:00
Evgenii Stratonikov
8b6bddca3c vm: support Map in PICKITEM and SETITEM 2019-09-25 17:53:36 +03:00
Evgenii Stratonikov
d7bb50c609 vm: implement VALUES opcode 2019-09-25 17:53:36 +03:00
Evgenii Stratonikov
2da64267b0 vm: implement KEYS opcode 2019-09-25 17:53:34 +03:00
Roman Khimov
903173c991 core: fix deadlock in headers list init
This probably is a problem since adc880d323,
because AddHeaders() uses headersOp channel and its reader is only ran in
Run().
2019-09-25 17:52:46 +03:00
Roman Khimov
578ac414d4 core: make unpersisted blocks/txs available in Blockchain
This changes the Blockchain to also return unpersisted (theoretically, verified
in the AddBlock!) blocks and transactions, making Add/Get interfaces
symmetrical. It allows to turn Persist into internal method again and makes it
possible to enable transaction check in GetBlock(), thus fixing #366.
2019-09-25 17:46:28 +03:00
Evgenii Stratonikov
0fb4bb05cf vm: implement HASKEY opcode 2019-09-25 12:08:41 +03:00
Roman Khimov
2daff0957a rpc: check for error responses in tests
sendrawtransaction_negative was actually wrong in trying to get
SendTXResponse, it received an error.
2019-09-24 18:47:23 +03:00
Evgenii Stratonikov
df18da0ac9 vm: implement NEWMAP opcode 2019-09-24 17:06:17 +03:00
Roman Khimov
8a457ef540 transaction: add a check for createHash() invocation
It should work in general.
2019-09-24 13:01:54 +03:00
Roman Khimov
679e01bd0f core: implement witness verification logic
Fix #368.
2019-09-24 13:01:54 +03:00
Roman Khimov
ca9c9be71f vm: add CHECKSIG/VERIFY/CHECKMULTISIG implementations
Fix #269.
2019-09-24 13:01:54 +03:00
Roman Khimov
3bbeb2476e vm: introduce HasFailed() method and use it where appropriate
It's gonna be used by external modules.
2019-09-23 20:19:07 +03:00
Evgenii Stratonikov
3c53beca82 vm: restrict SHL/SHR arguments to -256..256
Also unify SHL/SHR implementation.
2019-09-23 15:13:10 +03:00
Evgenii Stratonikov
4a8be486f0 vm: do not pop items in OVER 2019-09-23 14:54:59 +03:00
Evgenii Stratonikov
7cd69ea8e2 vm: truncate length in SUBSTR
Also fail on first error, without changing the stack.
2019-09-23 14:54:59 +03:00
Evgenii Stratonikov
a88a8e13fc vm: compare Array by reference in EQUAL 2019-09-23 13:25:59 +03:00
Evgenii Stratonikov
99f1d761ca vm: clone Struct on APPEND 2019-09-23 13:25:59 +03:00
Roman Khimov
ac5d2f94d3 storage: fix BoltDB batched Put()
It must copy both the value and the key because they can be reused for other
purposes between Put() and PutBatch(). This actually happens with values in
headers processing, leading to wrong data being written into the DB.

Extend the batch test to check for that.
2019-09-23 09:27:18 +03:00
Roman Khimov
5a0f08f2c0 network: fix SIGSEGV on unknown message acceptance
For example, at the moment our node can't handle `consensus` message, so when
it received it before the patch it just crashed because of uninitialized `p`.
2019-09-22 20:09:55 +03:00
Roman Khimov
4b4bac675b core: don't print persisting errors twice
As they're already printed in the calling goroutine. And they're alse not
always useful when printed by Persist().
2019-09-22 20:08:15 +03:00
Roman Khimov
102c926ef3 core: don't print useless persist messages with nil errors
Errors should be printed only when there are errors.
2019-09-22 20:07:49 +03:00
Roman Khimov
f69adc59ad io: drop getVarStringSize() completely
After unexport this doesn't make much sense at all as it has just one user.
2019-09-20 20:01:56 +03:00
Roman Khimov
d01b7740e2 io: improve GetVarSize() documentation 2019-09-20 20:00:45 +03:00
Roman Khimov
6dd37dd076 io: unexport getVarIntSize and getVarStringSize
All external users should just use GetVarSize().
2019-09-20 19:08:58 +03:00
Roman Khimov
5cddfe071b
Merge pull request #403 from nspcc-dev/fix_rpctest
Fix rpctest, fix #353, fix #305.
2019-09-18 23:14:37 +03:00
Vsevolod Brekelov
100fee164b unitTest: reworked RPC unit test
earlier we had an issue with failing test in #353 and other one #305.
Reworked these test to have in-memory database. This led to multiple
changes: made some functions like Hash and Persist public(otherwise
it's not possible to control state of the blockchain); removed
unit_tests storage package which was used mainly for leveldb in unit
tests.
I see these tests not really good since they look like e2e tests and
as for me should be run in separate step against dockerized env or
in case we want to check rpc handler we might want to rework it in order
to have interface for proper unit tests.
As for me this patchset at least makes as safe with not removing totally
previous tests and at the same time CircleCI will be happy now.
2019-09-18 18:21:16 +03:00
Vsevolod Brekelov
55dfc0bbbc storage: add seek implementation
during testing found missing Seek() implementation for inmemorydb
2019-09-18 18:20:41 +03:00
Roman Khimov
c68a254eba
Merge pull request #404 from nspcc-dev/move-user-docs
Move user docs, fix #339.
2019-09-18 12:50:06 +03:00
Roman Khimov
42df4c2f39 vm: update and move README, refs. #339 2019-09-18 12:10:12 +03:00
Roman Khimov
9441c5e77b rpc: split README into developer and user parts
Most of the document is written for developers and thus belongs to
godoc. User-related document is now moved to docs as per #339.
2019-09-18 12:10:12 +03:00
Roman Khimov
778d29f543 compiler: update and move README to the docs folder
Refs. #339.
2019-09-17 19:09:02 +03:00
Roman Khimov
cb4be2ae4a storage: drop useless README, refs. #339 2019-09-17 17:30:05 +03:00
Vsevolod
8e75674f30
Merge pull request #401 from nspcc-dev/refactor_bc_run
blockchain: server runs goroutine instead of blockchain init
2019-09-17 16:26:06 +03:00
Roman Khimov
bd1f70366a crypto: change files mode to 644 (some *.go had exec flags set) 2019-09-17 15:34:00 +03:00
Vsevolod Brekelov
adc880d323 blockchain: server runs goroutine instead of blockchain init
rework initBlockChain in order to have controllable way of running
blockchain;
remove context from initBlockChain func;
2019-09-17 15:27:40 +03:00
Roman Khimov
5bca4d2313 io: expand panic tests to reach 100% coverage 2019-09-17 13:23:24 +03:00
Roman Khimov
96618015cd network: implement EncodeBinary() for MerkleBlock 2019-09-17 13:21:52 +03:00
Roman Khimov
d1a4e43c48 io: redo Serializable to return errors in BinReader/BinWriter
Further simplifies error handling.
2019-09-17 13:21:52 +03:00
Roman Khimov
0bb8950f89 make TXer and Payload implement Serializable
Both are duplicating Serializable at the moment, but let's keep them for the
future.
2019-09-17 13:21:52 +03:00
Roman Khimov
e299a44983 io: drop Size() method from Serializable and associated
It's no longer needed after the io.GetVarSize() improvement. It's duplicating
a lot of EncodeBinary() logic also.
2019-09-17 13:21:45 +03:00
Roman Khimov
56c72b5c67 io: redo GetVarSize for Serializable things
Use writes to a fake io.Writer that counts the bytes. Allows us to kill Size()
methods as useless and duplicating lots of functionality.
2019-09-16 23:39:54 +03:00
Roman Khimov
0da9fe6946 io: move size calculator there
It's mostly used for Serializable and in other cases where one needs to
estimate binary-encoded size of the stucture. This also simplifies future
removal of the Size() from Serializable.
2019-09-16 23:39:54 +03:00
Roman Khimov
5bf00db2c9 io: move BinReader/BinWriter there, redo Serializable with it
The logic here is that we'll have all binary encoding/decoding done via our io
package, which simplifies error handling. This functionality doesn't belong to
util, so it's moved.

This also expands BufBinWriter with Reset() method to fit the needs of core
package.
2019-09-16 23:39:51 +03:00
Roman Khimov
f01354b5bd core: make TestHeaderEncodeDecode use assert for field comparisons 2019-09-16 23:38:48 +03:00
Roman Khimov
031a260cb1 util: add tests for BinaryReader/BinaryWriter
100% coverage.
2019-09-16 23:38:48 +03:00
Roman Khimov
72fc880182 util: remove bogus check from BinWriter.WriteVarUint()
val is uint64, it can't be less than zero by definition.
2019-09-16 23:38:48 +03:00
Roman Khimov
aacf58c9ab util: add 'constructors' for BinReader/BinWriter
And an additional BufBinWriter to ease buffer management.
2019-09-16 23:38:48 +03:00
Roman Khimov
0838948540
Merge pull request #394 from nspcc-dev/fix/json-tests-bugs
VM: fix some bugs from neo-vm JSON tests
2019-09-16 19:46:12 +03:00
Roman Khimov
c98a626871
Merge pull request #399 from nspcc-dev/refactor_store
storage: refactor store, add Close()
2019-09-16 18:59:34 +03:00
Vsevolod Brekelov
264dfef370 storage: close function
add close function to storage interface
add common defer function call which will close db connection
remove context as soon as it's not needed anymore
updated unit tests
2019-09-16 18:52:47 +03:00
Roman Khimov
adba9e11ee
Merge pull request #396 from nspcc-dev/network-reconnections-and-fixes
This one fixes #390 and some connected problems. After this patchset the node reconnects to some other nodes if anything goes wrong and it better senses when something goes wrong. It also fixes some block handling problems based on the testnet connection experience.
2019-09-16 16:57:10 +03:00
Roman Khimov
b9ff4d929c network: add getaddr message handling
Respond with known good addresses.
2019-09-16 16:32:04 +03:00
Roman Khimov
b2530a4c93 network: be more specific on port mismatch error 2019-09-16 16:32:04 +03:00
Roman Khimov
773ccc2b92 network: allow discoverer to reuse addresses
...and don't try to connect to the nodes we're already connected to.

Before this change we had a problem of discoverer throwing away good valid
addresses just because they are already known which lead to pool draining over
time (as address reuse was basically forbidden and getaddr may not get enough
new nodes).
2019-09-16 16:32:04 +03:00
Roman Khimov
46dc141c6c network: enlarge address request queue
Queuing one message is not reliable enough, the peer that gets it can fail to
actually make a request, so make this queue a bit deeper to have a higher
chance of success.
2019-09-16 16:32:04 +03:00
Roman Khimov
d3bb8ddf8f network: handle errors and connection close more correctly
This makes writer side handle errors properly and fixes communication between
reader and writer goroutine to always correctly unregister the peer. This is
especially important for the case where error occurs before handshake
completes as in this case we don't even have goroutine in startProtocol()
running.
2019-09-16 16:32:04 +03:00
Roman Khimov
76c7cff67f network: make node strictly follow handshake procedure
Don't accept other messages before handshake is completed, check handshake
message sequence.
2019-09-16 16:32:04 +03:00
Roman Khimov
c6487423ae network: close connection on disconnect
If it's already closed, this won't hurt, but in the case of logical error it
saves us from leaking this connection (and potentially, peer).
2019-09-16 16:26:30 +03:00
Roman Khimov
90e13debe4 network: implement getaddr sends and addr receival
Request new addresses if we're short on known addresses pool size.
2019-09-16 16:26:30 +03:00
Roman Khimov
2a49e68d77 network: start worker goroutine for every connection attempts
Prevents blocking on write to workCh which can be dangerous for the server.
2019-09-16 16:26:30 +03:00
Roman Khimov
b4e284f301 discovery: make pool management more reliable
Just drop excessive addresses, otherwise we can block for no good reason.
2019-09-16 16:26:30 +03:00
Roman Khimov
85f19936dd network: implement connection retries
It's worth to try a bit more than once.
2019-09-16 16:26:30 +03:00
Roman Khimov
be6c905e5d network: use and improve discovery mechanism for reconnections
This makes our node reconnect to other nodes if connection drops for some
reason. Fixes #390.
2019-09-16 16:26:30 +03:00