Commit graph

1401 commits

Author SHA1 Message Date
Roman Khimov
f0047b4055 interop/contract: update documentation, fix some interfaces
Some functions were just not correct in their interfaces.
2020-05-19 13:13:15 +03:00
Roman Khimov
a43f2234dd core: fix Neo.Contract.GetStorageContext security check
This syscall should only work for contracts created by current transaction and
that is what is supposed to be checked here. Do so by looking at the
differences between ic.dao and original lower DAO.
2020-05-19 13:13:15 +03:00
Roman Khimov
5d941364bd compiler/interop: expose GetTransactionHeight
And sort syscall names as they change the indentation anyway.
2020-05-19 13:13:15 +03:00
Roman Khimov
80b8b50f02 core: fix Neo.Blockchain.GetValidators implementation
It should return keys, attempting to push []*state.Validator to the stack
would probably lead to failure.
2020-05-19 13:13:15 +03:00
Roman Khimov
9e94895bb0 interop/blockchain: update documentation 2020-05-19 13:13:15 +03:00
Roman Khimov
d09c3b1e27 interop/block: update documentation, fix GetTransaction
GetTransaction never worked with hash, it works with indexes.
2020-05-19 13:13:15 +03:00
Roman Khimov
7abd35b358 compiler: add support for attribute syscalls 2020-05-19 13:13:15 +03:00
Roman Khimov
77c5f28b09 interop/attribute: update documentation 2020-05-19 13:13:14 +03:00
Roman Khimov
f69e654260 compiler: add missing asset syscalls, sort them 2020-05-19 13:13:14 +03:00
Roman Khimov
cbcc8e160f asset: update documentation and fix Create/Renew
Both Create and Renew have things returned from them.
2020-05-19 13:13:14 +03:00
Evgenii Stratonikov
231a5189a2 compiler: allow using OP= with struct fields and slice elements
Do it in a generic way, there is no need in restricting to only
variables.
2020-05-19 12:42:37 +03:00
Roman Khimov
9c46e79745 compiler: add support for account syscalls
Turns out, they never functioned correctly.
2020-05-18 19:08:28 +03:00
Roman Khimov
83df376d17 account: add missing IsStandard interop function
There is a Neo.Account.IsStandard syscall, but we didn't have a wrapper for
it.
2020-05-18 19:08:28 +03:00
Roman Khimov
6dff01672c interop/account: update and extend documentation 2020-05-18 19:08:28 +03:00
alexvanin
8db714785d rpc/client: handle client creation error in new wsclient
Client returns error if it can't parse endpoint string. WSClient
should check client error or there could be panic at `cl.cli = nil`
expression.
2020-05-18 16:25:36 +03:00
Roman Khimov
8cd7bc7e07 rpc/client: deduplicate block/header tests a bit
The same data is copied at least three times here.
2020-05-14 17:28:14 +03:00
Roman Khimov
9546e021a9 rpc/block: rework the way Block is JSONized
Our block.Block was JSONized in a bit different fashion than result.Block in
its Nonce and NextConsensus fields. It's not good for notifications because
third-party clients would probably expect to see the same format. Also, using
completely different Block representation in result is probably making our
client a bit weaker as this representation is harder to use with other neo-go
components.

So use the same approach we took for Transactions and wrap block.Base which is
to be serialized in proper way.
2020-05-14 17:28:14 +03:00
Roman Khimov
83febead59 transaction: add json.Unmarshaler to Attribute
It actually was missing and it might affect Transaction conversion to/from
JSON.
2020-05-14 17:28:14 +03:00
Roman Khimov
8f55f0ac76 rpc/server: add notification filters
And check state string correctness on unmarshaling.
2020-05-14 17:28:14 +03:00
Roman Khimov
78716c5335 rpc/client: add support for notification filters
Differing a bit from #895 draft specification, we won't add `verifier` (or
signer) for Neo 2, it's not worth doing so at the moment.
2020-05-14 00:56:26 +03:00
Roman Khimov
da32cff313 rpc/server: don't panic on test failure 2020-05-14 00:56:26 +03:00
Roman Khimov
966ff28091 rpc: add subscriber queue overflow check
It's not practical adding server-side tests for 2.0 (as it requires generating
more blocks), so we'll leave it for 3.0.
2020-05-14 00:56:26 +03:00
Roman Khimov
1d5734c882
Merge pull request #944 from nspcc-dev/notifications-2.x
Notifications 2.x
2020-05-13 17:22:55 +03:00
Roman Khimov
9454ef5c28 core: improve locking in storeBlock
Getting batch, updating Prometheus metrics and pushing events doesn't require
any locking: batch is a local cache batch that no one outside cares about,
Prometheus metrics are not critical to be in perfect sync and events are
asynchronous anyway.
2020-05-13 17:17:41 +03:00
Roman Khimov
5464f9c3ae rpc/client: add notifications support for WSClient
It differs from #895 design in that we have Notifications channel always
exposed as WSClient field, probably it simplifies things a little.
2020-05-13 17:17:41 +03:00
Roman Khimov
e1408b6525 rpc/server: add notification subscription
Note that the protocol differs a bit from #895 in its notifications format,
to avoid additional server-side processing we're omitting some metadata like:
 * block size and confirmations
 * transaction fees, confirmations, block hash and timestamp
 * application execution doesn't have ScriptHash populated

Some block fields may also differ in encoding compared to `getblock` results
(like nonce field).

I think these differences are unnoticieable for most use cases, so we can
leave them as is, but it can be changed in the future.
2020-05-13 17:17:41 +03:00
Roman Khimov
29ada4ca46 smartcontract: add JSON marshal/unmarshal for InteropType
We actually have to do that in order to answer getapplicationlog requests for
transactions that leave some interop items on the stack. It follows the same
logic our binary serializer/deserializes does leaving the type and stripping
the value (whatever that is).
2020-05-13 17:17:41 +03:00
Roman Khimov
d5df1212c2 rpc/server: start and shutdown Server in tests
It will be important for proper subscription testing and it doesn't hurt even
though technically we've got two http servers listening after this change (one
is a regular Server's http.Server and one is httptest's Server). Reusing
rpc.Server would be nice, but it requires some changes to Start sequence to
start Listener with net.Listen and then communicate back its resulting
Addr. It's not very convenient especially given that no other code needs it,
so doing these changes just for a bit cleaner testing seems like and
overkill.

Update config appropriately. Update Start comment along the way.
2020-05-12 17:42:34 +03:00
Roman Khimov
d686fe4e5d network: get blocks directly from the chain for rebroadcasting
Simplify network<->consensus relations, also broadcast blocks received by
other means like RPC.
2020-05-12 17:42:34 +03:00
Roman Khimov
dd8bcfae47 consensus: remove OnNewBlock(), use Blockchain subscription
Get new blocks directly from the Blockchain. It may lead to some duplications
(as we'll also receive our own blocks), but at the same time it's more
correct, because technically we can also get blocks via other means besides
network server like RPC (submitblock call). And it simplifies network server
at the same time.
2020-05-12 17:41:23 +03:00
Roman Khimov
1ac4f8528d core: add Blockchain event subscription mechanism
A deep internal part of #895. Blockchainer interface is also extended for
various uses of these methods.
2020-05-12 17:41:15 +03:00
Roman Khimov
2e58a14978 core: improve documentation a little 2020-05-12 17:20:31 +03:00
Roman Khimov
3db14b4699 wallet: check for t.GetSignedPart() result correctness
It can return nil easily and signing that is a big mistake.
2020-05-07 21:48:45 +03:00
Roman Khimov
943d435cd2 core: ensure we produce correct blocks for tests
Blocks must have at least one transaction and we should check for correct
merkle root generation.
2020-05-07 21:46:28 +03:00
Roman Khimov
a1d8206ef7
Merge pull request #935 from nspcc-dev/fix/slice
compiler: support implicit type in function arguments
2020-05-06 18:30:01 +03:00
Evgenii Stratonikov
37813f1020 compiler: support implicit type in function arguments
Go supports declaring multiple arguments of the same type without
duplicating type name. Now we support this too.
2020-05-06 18:20:13 +03:00
Evgenii Stratonikov
922aff1fcc compiler: support named returns 2020-05-06 18:15:52 +03:00
Evgenii Stratonikov
b2a3cee451 compiler: support using return in some branches
When `return` is used in one codepath, but not the other,
we do not clean altstack properly. This commit fixes it.
2020-05-06 18:15:50 +03:00
Evgenii Stratonikov
2a6be8ceef compiler: allow to use return with no arguments 2020-05-06 17:23:32 +03:00
Roman Khimov
e097e86bfa
Merge pull request #921 from nspcc-dev/rpc-websocket-2.x
RPC over websocket (2.x)
2020-05-04 13:55:19 +03:00
Anna Shaleva
99d0bafa2c compiler: add ability to generate .abi.json file
A part of integration with NEO Blockchain Toolkit (see #902). To be
able to deploy smart-contract compiled with neo-go compiler via NEO
Express, we have to generate additional .abi.json file. This file
contains the following information:
 - hash of the compiled contract
 - smart-contract metadata (title, description, version, author,
email, has-storage, has-dynamic-invoke, is-payable)
 - smart-contract entry point
 - functions
 - events

However, this .abi.json file is slightly different from the one,
described in manifest.go, so we have to add auxilaury stractures for
json marshalling. The .abi.json format used by NEO-Express is described
[here](https://github.com/neo-project/neo-devpack-dotnet/blob/master/src/Neo.Compiler.MSIL/FuncExport.cs#L66).
2020-05-04 08:31:14 +03:00
Anna Shaleva
592fd068b1 compiler: fix bug with missing methods parameters
Method `methodInfoFromScope(...)` always returned an empty parameters
set, so we were missing this information in both .abi.json and
.debug.json files. Fixed now.
2020-05-04 08:31:09 +03:00
Roman Khimov
556ab39a5a rpc/client: add minimalistic websocket client 2020-04-30 22:59:19 +03:00
Roman Khimov
6333060897 rpc/client: separate out http-related functionality 2020-04-30 22:59:19 +03:00
Roman Khimov
38f9b511ae rpc/client: drop Version from Options
It makes no sense at all, it's a JSON-RPC version.
2020-04-30 22:59:19 +03:00
Roman Khimov
e32367a1c2 rpc/client: fix some comments 2020-04-30 22:59:19 +03:00
Roman Khimov
a49f2bc47c client: remove Balancer getter/setter, make it an Option
Keep it internal to the client instance, it makes no sense exposing it to the
outside user.
2020-04-30 22:59:19 +03:00
Roman Khimov
00d6439eb2 client: make http.Client internal to the Client
Exposing it the outside users is strange, so incapsulate it completely.

Fix DialTimeout setting along the way, handle negative timeouts as invalid.
2020-04-30 22:59:01 +03:00
Roman Khimov
24b46c6041 rpc/server: add websockets support via '/ws' URL 2020-04-30 17:53:28 +03:00
Roman Khimov
3eb6d266ca rpc/server: use httptest.Server for testing
Which allows to reuse it for websockets.
2020-04-30 17:52:42 +03:00
Roman Khimov
3d2a2af575 rpc: shuffle handleHttpRequest/handleRequest responsibilities
Make handleRequest reusable in other contexts like websockets.
2020-04-30 17:52:15 +03:00
Roman Khimov
dbe7be5b00 rpc: change handlers to always return response.Error for errors
As it's expected by WriteErrorResponse() actually.
2020-04-30 17:52:15 +03:00
Roman Khimov
21c31d7d24 rpc/server: refactor handler methods a little
request.In is a natural request representation, one can always get
request.Params from it.
2020-04-30 17:52:13 +03:00
Anna Shaleva
87b0e76a8c rpc, smartcontract: move contract metadata to smartcontract package 2020-04-29 13:50:05 +03:00
Anna Shaleva
c813873577 consensus: added partial message decoding
closes #849
2020-04-16 17:16:29 +03:00
Roman Khimov
bfaa025a34
Merge pull request #839 from nspcc-dev/feature/badgerdb_support
storage: add support of BadgerDB
2020-04-09 15:57:50 +03:00
Anna Shaleva
54cdfe4a23 storage: add support of BadgerDB
closes #820
2020-04-09 13:55:59 +03:00
Roman Khimov
54a95810a4
Merge pull request #833 from nspcc-dev/test/vm_calls
vm: missing tests for CALL* instructions
2020-04-09 12:12:41 +03:00
Anna Shaleva
9f0bc2aaf5 vm: add tests to CALL* instructions
closes #452
2020-04-09 11:20:39 +03:00
Evgenii Stratonikov
c71ad6a5db cli: output contract after compiling only with --verbose flag
Also remove Debug from compiler option as it is used only in CLI.
2020-04-08 13:09:51 +03:00
Evgenii Stratonikov
ee0ba9b1b4 core: make SpawnVM a method of context
spawnVMWithInterops is rather long too type and
it doesn't use Blockchain in any way.
2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
a71cd0961e core/dao: remove unnecessary slice type
It is used only once, so a simple `sort.Slice`
invocation will suffice.
2020-04-08 08:38:45 +03:00
Evgenii Stratonikov
030b7754ad core: move DAO to a separate package 2020-04-08 08:38:44 +03:00
Anna Shaleva
495c1b0565 vm: add tests for missing bit and numeric operations
Added tests for:
  - bit operatoins: AND, OR, XOR
  - numeric operations: BOOLOR, MIN, MAX, WITHIN, NEGATE
2020-04-07 18:10:31 +03:00
Roman Khimov
4e0c3fab0f
Merge pull request #811 from nspcc-dev/feature/debug
compiler: support neo-debugger
2020-04-06 19:18:13 +03:00
Evgenii Stratonikov
da826522f8 compiler: set variable index on first declaration
This way variables will have indices corresponding to their
order of appearance in a source file.
2020-04-06 15:30:07 +03:00
Evgenii Stratonikov
457e7e006a compiler: support exporting method variables in debug info 2020-04-06 15:30:07 +03:00
Evgenii Stratonikov
5bdee683e6 cli,compiler: support emitting debug info in a file 2020-04-06 15:30:07 +03:00
Evgenii Stratonikov
5d3da26e1e compiler: support sequence points in debug info
Sequence points is a way to map a specific instruction offset
from a compiled contract to a text span in a source file.

This commit implements mapping only for `return` statements.
Further improvements are straight-forward.
2020-04-06 15:30:07 +03:00
Evgenii Stratonikov
24fef35ead compiler: split Compile info sub-functions
Also add tests for basic debug info.
2020-04-06 15:30:07 +03:00
Evgenii Stratonikov
00c40b58aa compiler: record basic debug info
Save info about method's byte-code sections.
2020-04-06 15:30:06 +03:00
Roman Khimov
9997661998
Merge pull request #821 from nspcc-dev/fix-cache-propagation-to-invocations
core: wrap cached dao properly, don't miss cached data
2020-04-06 11:52:41 +03:00
Evgenii Stratonikov
96806262bf vm: handle negative arguments in SHL/SHR
Do it as in reference implementation: a >> -b == a << b.
2020-04-06 10:46:35 +03:00
Roman Khimov
83a02f42f7
Merge pull request #825 from nspcc-dev/fix/compiler
compiler: accept varargs in `runtime.Notify`
2020-04-06 10:08:02 +03:00
Evgenii Stratonikov
1fcc019bf3 rpc: update test chain
Also provide info for getblockheader RPC while
generating test chain.
2020-04-06 09:31:09 +03:00
Evgenii Stratonikov
004023920e rpc: use Notify with varargs in the test contract 2020-04-06 09:31:09 +03:00
Evgenii Stratonikov
efad66aee1 compiler: make Notify accept varargs 2020-04-06 09:31:09 +03:00
Evgenii Stratonikov
0023c4f1f6 vm: use truncated division in MOD
Mimic C#'s `%` behavior.
Related 4b44190 (#773).
2020-04-06 09:30:48 +03:00
Roman Khimov
5f09381cf4 core: wrap cached dao properly, don't miss cached data
Fixes #817 where invoked contract missed updated account information because
it got it one layer below cachedDao used to process the block.
2020-04-03 10:15:11 +03:00
Roman Khimov
f64aa201c7
Merge pull request #819 from nspcc-dev/fix/util160_marshalling
smartcontract: fix uint160 marshalling in smartcontract.Parameter
2020-04-01 22:04:30 +03:00
Anna Shaleva
5a62eb923e smartcontract: fix uint160 marshalling in smartcontract.Parameter
There's a bug after #785: smartcontract.Parameter of type hash160 should
be marshalled in LE (as default marshaller for uint160 does) instead of
BE, so fixed.
2020-04-01 20:48:46 +03:00
Roman Khimov
2d0ad30fcf vm: rework Map with internal slice representation
Which makes iterating over map stable which is important for serialization and
and even fixes occasional test failures. We use the same ordering here as
NEO 3.0 uses, but it should also be fine for NEO 2.0 because it has no
defined order.
2020-04-01 19:33:53 +03:00
Roman Khimov
25201d480d smartcontract: simplify Array JSON marshalling 2020-04-01 19:21:00 +03:00
Roman Khimov
3dbe549a61 smartcontract: store MapType Parameter as a slice of KV pairs
Fixes #809.

Basically, there are three alternative approaches to fixing it:
 * allowing both []byte and string for ByteArrayType value
   minimal invasion into existing code, but ugly as hell and will probably
   backfire at some point
 * storing string values in ByteArrayType
   incurs quite a number of type conversions (and associated data copying),
   though note that these values are not changed usually, so dynamic
   properties of []byte are almost irrelevant here
 * storing only []byte values in ByteArrayType
   makes it impossible to use them as map keys which can be solved in several
   ways:
   - via an interface (Marshalable)
     which is good, but makes testing and comparing values in general harder,
     because of keys mismatch
   - using serialized Parameter as a key (in a string)
     which will need some additional marshaling/unmarshaling
   - converting MapType from map to a slice of key-value pairs
     not a bad idea as we don't use this map as a map really, the type
     itself is all about input/output for real VM types and this approach is
     also a bit closer to JSON representation of the Map
2020-04-01 19:21:00 +03:00
Evgenii Stratonikov
a43c9b9246 compiler: calculate stack size more precisely
When calculating number of local variables, only
defining assignments (i.e. via `:=`) must be taken into account.
2020-04-01 17:36:19 +03:00
Roman Khimov
9b5dab57e8
Merge pull request #810 from nspcc-dev/refactor/config
config: move config.go out of config
2020-04-01 10:29:57 +03:00
Anna Shaleva
5a984fdf88 config: move config.go out of config/
closes #423
2020-03-31 17:55:59 +03:00
Anna Shaleva
a8fa9c9244 vm: add tests to stack_item.go
closes #199
2020-03-31 17:18:13 +03:00
Roman Khimov
e41853d0a4
Merge pull request #789 from nspcc-dev/fix/tx_specific_data_marshalling
rpc: fix marshalling of type-specific tx data
2020-03-30 16:10:25 +03:00
Anna Shaleva
9c09ad9c89 rpc: fix marshalling of type-specific tx data
closes #585
2020-03-30 15:48:50 +03:00
Roman Khimov
6f13455cc9
Merge pull request #800 from nspcc-dev/fix/slice
compiler: declare compoud types with var keyword
2020-03-30 00:14:59 +03:00
Roman Khimov
92bafffcd8
Merge pull request #805 from nspcc-dev/fix-rpc-invocation-json
rpc: fix stack field name for invocation results
2020-03-29 13:41:46 +03:00
Roman Khimov
237de73dec
Merge pull request #807 from nspcc-dev/optimize-memcached-persist
storage: optimize (*MemCachedStore).Persist for memory-backed ps
2020-03-29 13:41:21 +03:00
Roman Khimov
0d482ceb7f
Merge pull request #808 from nspcc-dev/fix-map-serialization
vm: make map serialization more compatible with C#
2020-03-29 13:40:52 +03:00
Roman Khimov
665bf78d11 vm: make map serialization more compatible with C#
C# pushes value and key to the stack of non-serialized items, so key gets
serialized first followed by value. Fixes #806.

Notice though that neither IDictionary in C#, nor map in Go have elements
ordered, so we can easily get a difference in KV pairs order and it would be
impossible to fix.
2020-03-28 17:25:42 +03:00
Roman Khimov
4758de71ec storage: optimize (*MemCachedStore).Persist for memory-backed ps
Most of the time it's persisted into the MemoryStore or MemCachedStore, when
that's the case there is no real need to go through the Batch mechanism as it
incurs multiple copies of the data.

Importing 1.5M mainnet blocks with verification turned off, before:
real    12m39,484s
user    20m48,300s
sys     2m25,022s

After:
real    11m15,053s
user    18m2,755s
sys     2m4,162s

So it's around 10% improvement which looks good enough.
2020-03-28 17:21:50 +03:00
Roman Khimov
2316e2c4a7 rpc: fix stack field name for invocation results
Follow C# implementation:
https://docs.neo.org/docs/en-us/reference/rpc/latest-version/api/invokefunction.html
2020-03-28 13:59:42 +03:00
Anna Shaleva
0be237beb6 rpc: fix double-pointer bugs in RPC Client
Fixed double-pointer bug in following methods:
	- GetRawTransactionVerbose
	- GetBlockVerbose
	- GetBlockHeader
	- GetTxOut
	- ValidateAddress
2020-03-27 16:04:11 +03:00
Evgenii Stratonikov
3cbd138b67 compiler: allow to declare variables of struct type
Previously, struct variables were initialize with VM's nil value
which is of primitive type. Thus SETITEM used for struct's field
updating wasn't working.
2020-03-27 13:50:09 +03:00
Evgenii Stratonikov
6baed7a010 compiler: allow to declare slices of compound types
Previously this declarations were ignored which resulted
in runtime errors, because VM's nil is an element of primitive type
and can't be converted to an array.
2020-03-27 13:50:09 +03:00