Commit graph

601 commits

Author SHA1 Message Date
Roman Khimov
0583f252ab *: create real temporary dirs and files in tests
Improve reliability.
2021-07-20 12:51:11 +03:00
Roman Khimov
19717dd9a8 slice: introduce common Copy helper
It's a bit more convenient to use.
2021-07-19 22:57:55 +03:00
Roman Khimov
a54e3516d1 slice: add Reverse function, deduplicate code a bit 2021-07-19 22:57:55 +03:00
Roman Khimov
100e97d772 util: move ArrayReverse into package of its own
Leave just uint160 and uint256 types in util.
2021-07-19 22:57:55 +03:00
Roman Khimov
1568ebc513 stackitem: rework struct cloning protection
Count everything, fail early, make it more compatible with
neo-project/neo-vm#423.
2021-07-19 15:42:42 +03:00
Roman Khimov
5f13de3a76 *: simplify some integer checks with IsUint64()
If we need a positive number we can do `IsUint64()` instead of checking that
`Int64()` is `> 0`.
2021-07-19 15:42:42 +03:00
Roman Khimov
233307aca5 stackitem: completely drop MaxArraySize
Turns out C# VM doesn't have it since preview2, so our limiting of
MaxArraySize in incompatible with it. Removing this limit shouldn't be a
problem with the reference counter we have, both APPEND and SETITEM add things
to reference counter and we can't exceed MaxStackSize. PACK on the other hand
can't get more than MaxStackSize-1 of input elements.

Unify NEWSTRUCT with NEWARRAY* and use better integer checks at the same time.

Multisig limit is still 1024.
2021-07-19 15:42:42 +03:00
Roman Khimov
ee4a647934 stackitem: limit EQUAL for deeply nested structs
See neo-project/neo-vm#428.
2021-07-19 15:42:42 +03:00
Roman Khimov
f89f0300f6 stackitem: improve test coverage a bit 2021-07-19 15:42:42 +03:00
Roman Khimov
df2430d5e4 stackitem: limit deserialization to MaxDeserialized items
Follow neo-project/neo#2531. Even though it's not strictly required (our node
handles problematic script just fine) we better be compliant wrt
deserialization behavior. MaxDeserialized is introduced to avoid moving
MaxStackSize which is a VM parameter.
2021-07-19 15:42:42 +03:00
Roman Khimov
aab18c3083 stackitem: introduce Convertible interface
We have a lot of native contract types that are converted to stack items
before serialization, then deserialized as stack items and converted back to
regular structures. stackitem.Convertible allows to remove a lot of repetitive
io.Serializable code.

This also introduces to/from converter in testserdes which unfortunately
required to change util tests to avoid circular references.
2021-07-19 15:42:42 +03:00
Roman Khimov
fbe8bd2d9c stackitem: limit deserialized arrays/maps
See neo-project/neo#2531, even though they use MaxStackSize there resulting
element is not valid unless it has <=MaxArraySize elements.
2021-07-19 15:42:42 +03:00
Roman Khimov
654c4a6589 stackitem: properly pass allowInvalid on binary deserialization
Otherwise DecodeBinaryProtected() is broken for arrays/maps.
2021-07-19 15:42:41 +03:00
Roman Khimov
15be763bb3 vm: limit POW, fix #2060
Calculating pow(pow(2, 255), 0xffffffff) takes unknown amount of time. See
also neo-project/neo-vm#422.
2021-07-19 15:42:41 +03:00
Roman Khimov
d638aee7a1
Merge pull request #2053 from nspcc-dev/stackitem-json
Speedup stackitem json serialization
2021-07-13 11:27:33 +03:00
Evgeniy Stratonikov
0611031fd4 stackitem/test: improve coverage for serialization
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-13 11:06:19 +03:00
Evgeniy Stratonikov
8fdc7e32f5 stackitem: cache serialized results in EncodeBinary
```
name            old time/op    new time/op    delta
EncodeBinary-8    8.39ms ± 1%    0.05ms ±21%  -99.44%  (p=0.000 n=10+9)

name            old alloc/op   new alloc/op   delta
EncodeBinary-8    2.24MB ± 0%    0.33MB ± 0%  -85.29%  (p=0.000 n=9+10)

name            old allocs/op  new allocs/op  delta
EncodeBinary-8     65.6k ± 0%      0.0k ± 0%  -99.95%  (p=0.000 n=10+10)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-13 11:06:19 +03:00
Evgeniy Stratonikov
17a3f17c74 stackitem: use byte-slice directly during encoding
```
name            old time/op    new time/op    delta
EncodeBinary-8    10.6ms ± 1%     8.4ms ± 1%  -20.94%  (p=0.000 n=9+10)

name            old alloc/op   new alloc/op   delta
EncodeBinary-8    1.64MB ± 0%    2.24MB ± 0%  +36.18%  (p=0.000 n=8+9)

name            old allocs/op  new allocs/op  delta
EncodeBinary-8      131k ± 0%       66k ± 0%  -49.98%  (p=0.000 n=10+10)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-13 11:06:18 +03:00
Evgeniy Stratonikov
c4eb7db8a5 stackitem: access value of Array/Struct directly
```
name      old time/op    new time/op    delta
ToJSON-8    50.3µs ±34%    47.8µs ± 9%     ~     (p=0.971 n=10+10)

name      old alloc/op   new alloc/op   delta
ToJSON-8     396kB ± 0%     396kB ± 0%   -0.10%  (p=0.000 n=6+10)

name      old allocs/op  new allocs/op  delta
ToJSON-8      34.0 ± 0%      18.0 ± 0%  -47.06%  (p=0.000 n=10+10)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:40:21 +03:00
Evgeniy Stratonikov
dc0a17dd0e stackitem: cache visited items while marshaling json
```
name      old time/op    new time/op    delta
ToJSON-8    4.52ms ± 4%    0.05ms ±34%  -98.89%  (p=0.000 n=8+10)

name      old alloc/op   new alloc/op   delta
ToJSON-8    2.13MB ± 0%    0.40MB ± 0%  -81.44%  (p=0.000 n=9+6)

name      old allocs/op  new allocs/op  delta
ToJSON-8     65.6k ± 0%      0.0k ± 0%  -99.95%  (p=0.000 n=10+10)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:40:21 +03:00
Evgeniy Stratonikov
69cdd5252a stackitem: add benchmark for serialization routines
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:40:17 +03:00
Roman Khimov
fe3c68b92d vm: panic in cloneIfStruct
It's internal, so we can deduplicate code a bit.
2021-07-12 11:52:24 +03:00
Roman Khimov
4000dd692c vm: make cloning limits a bit more effective
Take actual reference counter value into account.
2021-07-11 19:37:06 +03:00
Roman Khimov
cfe41abd35 vm: fix OOM during nested structure cloning
Resulting item can't have more than MaxStackSize elements. Technically this
limits to MaxStackSize cloned elements but it's considered to be enough to
mitigate the issue (the next size check is going to happen during push to the
stack). See neo-project/neo#2534, thanks @vang1ong7ang.
2021-07-11 18:47:50 +03:00
Evgeniy Stratonikov
1863023e27 vm: copy argument in unary opcodes, fix #2051
We already do this in binary opcodes, let's be consistent.

This fixes state difference at height 275663
for tx 3c498317684d63849b03e4c58ad57ce4b19bb206b7b01bcc64233de3b3e207f4

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-08 17:09:04 +03:00
Roman Khimov
e62a766058 state: move nil check down to stackitem JSON processing
We want to return real errors, not some generic thing for any kind of thing
happening.
2021-07-07 00:38:19 +03:00
Roman Khimov
5a9efcc654 stackitem: rework error handling
Return good named errors everywhere, export appropriate constants, make
errors.Is() work.
2021-07-07 00:18:00 +03:00
Roman Khimov
0de949b575 stackitem: remove Item/StackItem from function names
They're useless in a package named 'stackitem', make this package a bit more
user-friendly.
2021-07-06 19:56:23 +03:00
Roman Khimov
b9ff07f32c stackitem: add limit to serialized items
Standard binary serialization/deserialization is mostly used in VM to put/get
elements into/from storage, so they never should exceed MaxSize (otherwise one
won't be able to deserialize these items).

This patch leaves EncodeBinaryStackItem unprotected, but that's a streaming
interface, so it's up to the user of it to ensure its appropriate use (and our
uses are mostly for native contract's data, so they're fine).
2021-07-06 19:34:02 +03:00
Roman Khimov
8472064bbc stackitem: refactor serialization code, add explicit context 2021-07-06 19:32:52 +03:00
Roman Khimov
1b7b7e4bec stackitem: don't overwrite error when serializing Item
We must get out as quickly as possible.
2021-07-06 17:51:46 +03:00
Roman Khimov
e34fa2e915 stackitem: limit JSONization nesting level
We can have very deep reference types and attempt to JSONize them can easily
lead to OOM (even though there are no recursive references inside). Therefore
we have to limit them. While regular ToJSON() is buffer size limited to
MaxSize, ToJSONWithTypes is not and limiting it to MaxSize output will require
substantial rewriting effort while not really providing fair result, MaxSize
is more about stack item size while its JSON representation can be much bigger
because of various overheads.

Initial thought was to limit it by element count based on
MaxIteratorResultItems, but the problem here is that we don't always have this
limit in contexts using ToJSONWithTypes (like notification event
marshaling). Thus we need some generic limit which would be fine for all
users.

We at the same time have maxJSONDepth used when deserializing from JSON, so
it can be used for marshaling as well, it's not often that we have deeper
structures in real results.

Inspired by neo-project/neo#2521.
2021-07-06 17:33:16 +03:00
Roman Khimov
6879cbcdcc stackitem: properly detect recursive references during serialization
If we're done with element it no longer can lead to recursion error, so fix
cases like `[arr, arr]` where we have two copies of `arr` trigger this error
for no good reason (there is no recursion there).
2021-07-06 17:10:31 +03:00
Evgeniy Stratonikov
8d67a03aec stackitem: fix Buffer deserialization
Fix incorrect application log for transaction in N3 testnet
8eb4076f1f1c07e693eda7e810779488a2d2b50aba9b727fd237cbc3adbec9e9

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-06-29 11:39:55 +03:00
Anna Shaleva
f95bdd9cb5 vm: serialize + rune as \u002B for byte-contained stackitems
This commit fixes the following Go vs C# state diff:

block 74613: value mismatch for key EwAAAHN0cmVhbXMvDg==: eyJpZCI6MTQsInN0YXJ0IjoxNjIyNTAwMjAwMDAwLCJzdG9wIjoxNjIyNTAyMDAwMDAwLCJkZXBvc2l0IjoxMDAwMDAwMDAsInJlbWFpbmluZyI6MTAwMDAwMDAwLCJzZW5kZXIiOiJmeEY4RDl2ZFU3K0gwcDV3NTlyWllMNytNSlE9IiwicmVjaXBpZW50IjoiSVV6c3pveFV0S1NGVnlZRGczSmdTQTFlbTFNPSJ9 vs eyJpZCI6MTQsInN0YXJ0IjoxNjIyNTAwMjAwMDAwLCJzdG9wIjoxNjIyNTAyMDAwMDAwLCJkZXBvc2l0IjoxMDAwMDAwMDAsInJlbWFpbmluZyI6MTAwMDAwMDAwLCJzZW5kZXIiOiJmeEY4RDl2ZFU3XHUwMDJCSDBwNXc1OXJaWUw3XHUwMDJCTUpRPSIsInJlY2lwaWVudCI6IklVenN6b3hVdEtTRlZ5WURnM0pnU0ExZW0xTT0ifQ==

I.e.:
```
{"id":14,"start":1622500200000,"stop":1622502000000,"deposit":100000000,"remaining":100000000,"sender":"fxF8D9vdU7+H0p5w59rZYL7+MJQ=","recipient":"IUzszoxUtKSFVyYDg3JgSA1em1M="}
```
vs
```
{"id":14,"start":1622500200000,"stop":1622502000000,"deposit":100000000,"remaining":100000000,"sender":"fxF8D9vdU7\u002BH0p5w59rZYL7\u002BMJQ=","recipient":"IUzszoxUtKSFVyYDg3JgSA1em1M="}
```
2021-06-08 16:40:38 +03:00
Anna Shaleva
b14fc5da7c vm: increase waiting time for vm cli program 2021-06-01 16:29:08 +03:00
Anna Shaleva
e3611bfa4c vm, cli: allow to specify flags while loading VM 2021-05-28 12:07:41 +03:00
Roman Khimov
9d2712573f *: enable godot linter and fix all its warnings
It's important for NeoGo to have clean documentation. No functional changes.
2021-05-12 23:17:03 +03:00
Roman Khimov
c4e084b0d8 *: fix whitespace errors
leading/trailing newlines
2021-05-12 22:51:41 +03:00
Roman Khimov
de5e61588d vm: simplify some test code
gosimple: S1017: should replace this `if` statement with an unconditional `strings.TrimPrefix`
2021-05-12 18:44:35 +03:00
Roman Khimov
7af67d2a3c vm/cli: simplify buffer-related code
gosimple: S1030: should use buf.String() instead of string(buf.Bytes())
2021-05-12 18:42:50 +03:00
Roman Khimov
07cdbb815c *: drop unnecessary fmt.Sprintf uses
gosimple: S1039: unnecessary use of fmt.Sprintf
2021-05-12 18:29:39 +03:00
Anna Shaleva
6d59689d9c core: rename Neo.Crypto.CheckMultisig interop 2021-05-11 18:38:14 +03:00
Anna Shaleva
366e79b9b8 core: rename Neo.Crypto.CheckSig interop 2021-05-11 18:37:55 +03:00
Evgeniy Stratonikov
23a4e25436 interop: remove System.Iterator.Create, fix #1935
There are now only storage iterators. Related #1933.
2021-05-11 12:13:30 +03:00
Anna Shaleva
9eeebf481c rpc: allow to marshal Iterators for invoke* results 2021-04-30 16:23:06 +03:00
Evgeniy Stratonikov
8fa4a576f4 vm: allow Null arguments for LT,LE,GT,GE opcodes 2021-04-29 16:10:51 +03:00
Evgeniy Stratonikov
dc393642a2 opcode: fix GTE, LTE string representations 2021-04-29 16:08:48 +03:00
Anna Shaleva
f7dcb7ae29 vm: allow emit.Array handle uint256 2021-04-16 16:20:30 +03:00
Anna Shaleva
f57187e611 vm: throw unhandled exception with message instead of panic
The result is the same HALT state, but the exception message is the real
one got from user. Changes ported from C#:

1. Throw exception: 59b8ac73d2/src/neo-vm/ExecutionEngine.cs (L1448)
2. Prettify message: https://github.com/neo-project/neo-vm/blob/master/src/neo-vm/VMUnhandledException.cs#L28

The result is that instead of
```
2021-03-31T17:02:54.508+0300	WARN	contract invocation failed	{"tx": "2aefeb705f3a609df8767d9b45e036b9dd1eb77407e5732375981915668889b8", "block": 30640, "error": "error encountered at instruction 970 (THROW): runtime error: invalid memory address or nil pointer dereference"}
```

we'll get
```
2021-03-31T17:33:56.299+0300	WARN	contract invocation failed	{"tx": "2aefeb705f3a609df8767d9b45e036b9dd1eb77407e5732375981915668889b8", "block": 30640, "error": "error encountered at instruction 970 (THROW): unhandled exception: No authorization."}
```
in the node logs.
2021-03-31 19:37:52 +03:00