Commit graph

1789 commits

Author SHA1 Message Date
Anna Shaleva
a276a85b72 core: unify code of state sync module initialization 2021-09-07 19:43:27 +03:00
Anna Shaleva
3b7807e897 network: request unknown MPT nodes
In this commit:

1. Request unknown MPT nodes from peers. Note, that StateSync module itself
shouldn't be responsible for nodes requests, that's a server duty.
2. Do not request the same node twice, check if it is in storage
already. If so, then the only thing remaining is to update refcounter.
2021-09-07 19:43:27 +03:00
Anna Shaleva
6a04880b49 core: collapse completed parts of Billet
Some kind of marker is needed to check whether node has been collapsed
or not. So introduce (HashNode).Collapsed
2021-09-07 19:43:27 +03:00
Anna Shaleva
74f1848d19 core: adjust LastUpdatedBlock calculation for NEP17 balances
...wrt P2PStateExchange setting.
2021-09-07 19:43:27 +03:00
Anna Shaleva
d67ff30704 core: implement statesync module
And support GetMPTData and MPTData P2P commands.
2021-09-07 19:43:27 +03:00
Anna Shaleva
a22b1caa3e core: implement MPT Billet structure for MPT restore
MPT restore process is much simpler then regular MPT maintaining: trie
has a fixed structure, we don't need to remove or rebuild MPT nodes. The
only thing we should do is to replace Hash nodes to their unhashed
counterparts and increment refcount. It's better not to touch the
regular MPT code and create a separate structure for this.
2021-09-07 19:43:27 +03:00
Roman Khimov
c9e62769a6
Merge pull request #2143 from nspcc-dev/mpt/add_empty_values
core: allow empty MPT Leaf values
2021-09-07 09:18:48 +03:00
Anna Shaleva
c95f2079d5 core: adjust comments on behaviour defferences for MPT TestCompatibility
C# node does not return empty proof enymore in case if path is bad. C#
node also throws an exception on bad Put.

Our node does not return an error on delete if the key is empty.
2021-09-03 13:46:52 +03:00
Anna Shaleva
f721384ead core: allow empty MPT Leaf values
Allow it for (*Trie).Put. And distinguish empty value and nil value for
(*Trie).PutBatch, because batch is already capable of handling both nil
and empty value. For (*Trie).PutBatch putting nil value means deletion,
while putting empty value means just putting LeafNode with an empty
value.
2021-09-03 13:46:48 +03:00
Evgeniy Stratonikov
7371593bdc native/policy: disallow blocking native contracts
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-09-03 11:11:06 +03:00
Roman Khimov
b07347e602 core: reuse PushItem for interops
Probably less critical here, but still let's push things faster.
2021-08-30 23:43:58 +03:00
Roman Khimov
7b9558d756
Merge pull request #2142 from nspcc-dev/fix-customgroups-witness-scope
runtime: fix CustomGroups witness
2021-08-26 18:13:08 +03:00
Roman Khimov
734eef3290
Merge pull request #2147 from nspcc-dev/drop-go-1.14
Drop Go 1.14, use 1.17
2021-08-26 17:59:21 +03:00
Roman Khimov
6d074a96e9 *: make tests use TempDir(), fix #1319
Simplify things, drop TempFile at the same time (refs. #1764)
2021-08-26 17:29:40 +03:00
Roman Khimov
40c6c065d2
Merge pull request #2140 from nspcc-dev/vm-optimize-stack
Optimize VM stack
2021-08-26 10:31:31 +03:00
Roman Khimov
de4ed7d020 runtime: fix CustomGroups witness
See neo-project/neo#2586.
2021-08-24 15:50:24 +03:00
Roman Khimov
930653418d vm: rework stack as a simple slice
Double-linked list is quite expensive to manage especially given that it
requires microallocations for each Element. It can be replaced by simple slice
which is much more effective for simple push/pop operations that are very
typical in a VM. I've worried a little about more complex operations like
XDROP/1024 or REVERSEN/1024 because these require copying quite substantial
number of elements, but turns out these work fine too.

At the moment Element is kept as a convenient wrapper for Bytes/BigInt/Bool/etc
methods, but it can be changed in future. Many other potential optimizations
are also possible now.

Complex scripts:
name                    old time/op    new time/op    delta
ScriptFibonacci-8         1.11ms ± 2%    0.85ms ± 2%  -23.40%  (p=0.000 n=10+10)
ScriptNestedRefCount-8    1.46ms ± 2%    1.16ms ± 1%  -20.65%  (p=0.000 n=10+10)
ScriptPushPop/4-8         1.81µs ± 1%    1.54µs ± 4%  -14.96%  (p=0.000 n=8+10)
ScriptPushPop/16-8        4.88µs ± 2%    3.91µs ± 2%  -19.87%  (p=0.000 n=9+9)
ScriptPushPop/128-8       31.9µs ± 9%    26.7µs ± 3%  -16.28%  (p=0.000 n=9+8)
ScriptPushPop/1024-8       235µs ± 1%     192µs ± 3%  -18.31%  (p=0.000 n=9+10)

name                    old alloc/op   new alloc/op   delta
ScriptFibonacci-8          392kB ± 0%     123kB ± 0%  -68.68%  (p=0.000 n=8+8)
ScriptNestedRefCount-8     535kB ± 0%     266kB ± 0%  -50.38%  (p=0.000 n=6+10)
ScriptPushPop/4-8           352B ± 0%      160B ± 0%  -54.55%  (p=0.000 n=10+10)
ScriptPushPop/16-8        1.41kB ± 0%    0.64kB ± 0%  -54.55%  (p=0.000 n=10+10)
ScriptPushPop/128-8       11.3kB ± 0%     8.7kB ± 0%  -22.73%  (p=0.000 n=10+10)
ScriptPushPop/1024-8      90.1kB ± 0%    73.2kB ± 0%  -18.75%  (p=0.000 n=10+10)

name                    old allocs/op  new allocs/op  delta
ScriptFibonacci-8          9.14k ± 0%     3.53k ± 0%  -61.41%  (p=0.000 n=10+10)
ScriptNestedRefCount-8     17.4k ± 0%     11.8k ± 0%  -32.35%  (p=0.000 n=10+10)
ScriptPushPop/4-8           12.0 ± 0%       8.0 ± 0%  -33.33%  (p=0.000 n=10+10)
ScriptPushPop/16-8          48.0 ± 0%      32.0 ± 0%  -33.33%  (p=0.000 n=10+10)
ScriptPushPop/128-8          384 ± 0%       259 ± 0%  -32.55%  (p=0.000 n=10+10)
ScriptPushPop/1024-8       3.07k ± 0%     2.05k ± 0%  -33.14%  (p=0.000 n=10+10)

Some stack-management opcodes:

name                                 old time/op    new time/op    delta
Opcodes/XDROP/0/1-8                     255ns ± 9%     273ns ±11%    +6.92%  (p=0.016 n=11+10)
Opcodes/XDROP/0/1024-8                  362ns ± 2%     365ns ± 8%      ~     (p=0.849 n=10+11)
Opcodes/XDROP/1024/1024-8              3.20µs ± 2%    1.99µs ±12%   -37.69%  (p=0.000 n=11+11)
Opcodes/XDROP/2047/2048-8              6.55µs ± 3%    1.75µs ± 5%   -73.26%  (p=0.000 n=10+11)
Opcodes/DUP/null-8                      414ns ± 6%     245ns ±12%   -40.88%  (p=0.000 n=11+11)
Opcodes/DUP/boolean-8                   411ns ± 8%     245ns ± 6%   -40.31%  (p=0.000 n=11+11)
Opcodes/DUP/integer/small-8             684ns ± 8%     574ns ± 3%   -16.02%  (p=0.000 n=11+10)
Opcodes/DUP/integer/big-8               675ns ± 6%     601ns ±10%   -10.98%  (p=0.000 n=11+11)
Opcodes/DUP/bytearray/small-8           675ns ±10%     566ns ±10%   -16.22%  (p=0.000 n=11+11)
Opcodes/DUP/bytearray/big-8            6.39µs ±11%    6.13µs ± 3%      ~     (p=0.148 n=10+10)
Opcodes/DUP/buffer/small-8              412ns ± 5%     261ns ± 8%   -36.55%  (p=0.000 n=9+11)
Opcodes/DUP/buffer/big-8                586ns ±10%     337ns ± 7%   -42.53%  (p=0.000 n=11+11)
Opcodes/DUP/struct/small-8              458ns ±12%     256ns ±12%   -44.09%  (p=0.000 n=11+11)
Opcodes/DUP/struct/big-8                489ns ± 7%     274ns ± 5%   -44.06%  (p=0.000 n=10+10)
Opcodes/DUP/pointer-8                   586ns ± 7%     494ns ± 7%   -15.67%  (p=0.000 n=11+11)
Opcodes/OVER/null-8                     450ns ±14%     264ns ±10%   -41.30%  (p=0.000 n=11+11)
Opcodes/OVER/boolean-8                  450ns ±14%     264ns ±10%   -41.31%  (p=0.000 n=11+11)
Opcodes/OVER/integer/small-8            716ns ± 9%     604ns ± 6%   -15.65%  (p=0.000 n=11+11)
Opcodes/OVER/integer/big-8              696ns ± 5%     634ns ± 6%    -8.89%  (p=0.000 n=10+11)
Opcodes/OVER/bytearray/small-8          693ns ± 1%     539ns ± 9%   -22.18%  (p=0.000 n=9+10)
Opcodes/OVER/bytearray/big-8           6.33µs ± 2%    6.16µs ± 4%    -2.79%  (p=0.004 n=8+10)
Opcodes/OVER/buffer/small-8             415ns ± 4%     263ns ± 8%   -36.76%  (p=0.000 n=9+11)
Opcodes/OVER/buffer/big-8               587ns ± 5%     342ns ± 7%   -41.70%  (p=0.000 n=11+11)
Opcodes/OVER/struct/small-8             446ns ±14%     257ns ± 8%   -42.42%  (p=0.000 n=11+11)
Opcodes/OVER/struct/big-8               607ns ±26%     278ns ± 7%   -54.25%  (p=0.000 n=11+11)
Opcodes/OVER/pointer-8                  645ns ±12%     476ns ±10%   -26.21%  (p=0.000 n=11+11)
Opcodes/PICK/2/null-8                   460ns ±11%     264ns ± 9%   -42.68%  (p=0.000 n=11+11)
Opcodes/PICK/2/boolean-8                460ns ± 4%     260ns ± 4%   -43.37%  (p=0.000 n=8+11)
Opcodes/PICK/2/integer/small-8          725ns ± 7%     557ns ± 4%   -23.19%  (p=0.000 n=11+10)
Opcodes/PICK/2/integer/big-8            722ns ±12%     582ns ± 6%   -19.51%  (p=0.000 n=11+11)
Opcodes/PICK/2/bytearray/small-8        705ns ± 6%     545ns ± 4%   -22.69%  (p=0.000 n=11+11)
Opcodes/PICK/2/bytearray/big-8         7.17µs ±36%    6.37µs ± 8%      ~     (p=0.065 n=11+11)
Opcodes/PICK/2/buffer/small-8           427ns ± 8%     253ns ± 8%   -40.82%  (p=0.000 n=11+11)
Opcodes/PICK/2/buffer/big-8             590ns ± 3%     331ns ± 6%   -43.83%  (p=0.000 n=11+11)
Opcodes/PICK/2/struct/small-8           428ns ± 8%     254ns ± 7%   -40.64%  (p=0.000 n=11+11)
Opcodes/PICK/2/struct/big-8             489ns ±15%     283ns ± 7%   -42.11%  (p=0.000 n=11+11)
Opcodes/PICK/2/pointer-8                553ns ± 7%     414ns ± 8%   -25.18%  (p=0.000 n=11+11)
Opcodes/PICK/1024/null-8                531ns ± 4%     327ns ± 6%   -38.49%  (p=0.000 n=10+10)
Opcodes/PICK/1024/boolean-8             527ns ± 5%     318ns ± 5%   -39.78%  (p=0.000 n=11+9)
Opcodes/PICK/1024/integer/small-8       861ns ± 4%     683ns ± 4%   -20.66%  (p=0.000 n=11+11)
Opcodes/PICK/1024/integer/big-8         882ns ± 4%    1060ns ±47%      ~     (p=0.748 n=11+11)
Opcodes/PICK/1024/bytearray/small-8     850ns ± 4%     671ns ± 5%   -21.12%  (p=0.000 n=10+11)
Opcodes/PICK/1024/bytearray/big-8      6.32µs ±26%    6.75µs ± 4%    +6.86%  (p=0.019 n=10+11)
Opcodes/PICK/1024/buffer/small-8        530ns ± 6%     324ns ± 5%   -38.86%  (p=0.000 n=10+11)
Opcodes/PICK/1024/buffer/big-8          570ns ± 4%     417ns ±45%   -26.82%  (p=0.001 n=11+10)
Opcodes/PICK/1024/struct/small-8      1.11µs ±122%    0.34µs ±11%   -69.38%  (p=0.000 n=11+10)
Opcodes/PICK/1024/pointer-8             693ns ± 5%     568ns ±31%   -18.10%  (p=0.002 n=10+10)
Opcodes/TUCK/null-8                     450ns ±10%     275ns ± 8%   -38.93%  (p=0.000 n=11+11)
Opcodes/TUCK/boolean-8                  449ns ±13%     268ns ± 9%   -40.16%  (p=0.000 n=11+10)
Opcodes/TUCK/integer/small-8            716ns ± 7%     599ns ± 7%   -16.30%  (p=0.000 n=11+11)
Opcodes/TUCK/integer/big-8              718ns ± 8%     613ns ±11%   -14.55%  (p=0.000 n=11+11)
Opcodes/TUCK/bytearray/small-8          700ns ±12%     558ns ± 7%   -20.39%  (p=0.000 n=11+11)
Opcodes/TUCK/bytearray/big-8           5.88µs ± 7%    6.37µs ± 3%    +8.31%  (p=0.000 n=10+11)
Opcodes/TUCK/buffer/small-8             425ns ± 6%     258ns ±12%   -39.28%  (p=0.000 n=11+11)
Opcodes/TUCK/buffer/big-8               553ns ±19%     334ns ± 6%   -39.57%  (p=0.000 n=11+11)
Opcodes/TUCK/struct/small-8             474ns ± 3%     263ns ±12%   -44.51%  (p=0.000 n=10+11)
Opcodes/TUCK/struct/big-8               641ns ±24%     284ns ± 8%   -55.63%  (p=0.000 n=11+11)
Opcodes/TUCK/pointer-8                  635ns ±13%     468ns ±16%   -26.31%  (p=0.000 n=11+11)
Opcodes/SWAP/null-8                     227ns ±31%     212ns ±11%      ~     (p=0.847 n=11+11)
Opcodes/SWAP/integer-8                  233ns ±32%     210ns ±14%      ~     (p=0.072 n=10+11)
Opcodes/SWAP/big_bytes-8                263ns ±39%     211ns ±11%      ~     (p=0.056 n=11+11)
Opcodes/ROT/null-8                      308ns ±68%     223ns ±12%      ~     (p=0.519 n=11+11)
Opcodes/ROT/integer-8                   226ns ±25%     228ns ± 9%      ~     (p=0.705 n=10+11)
Opcodes/ROT/big_bytes-8                 215ns ±18%     218ns ± 7%      ~     (p=0.756 n=10+11)
Opcodes/ROLL/4/null-8                   269ns ±10%     295ns ± 9%    +9.42%  (p=0.002 n=10+11)
Opcodes/ROLL/4/integer-8                344ns ±48%     280ns ± 2%      ~     (p=0.882 n=11+9)
Opcodes/ROLL/4/big_bytes-8              276ns ±13%     288ns ± 4%    +4.38%  (p=0.046 n=9+11)
Opcodes/ROLL/1024/null-8               4.21µs ±70%    1.01µs ± 9%   -76.15%  (p=0.000 n=11+11)
Opcodes/ROLL/1024/integer-8            4.78µs ±82%    0.71µs ± 3%   -85.06%  (p=0.000 n=11+11)
Opcodes/ROLL/1024/big_bytes-8          3.28µs ± 5%    1.35µs ±36%   -58.91%  (p=0.000 n=9+11)
Opcodes/REVERSE3/null-8                 219ns ± 9%     224ns ± 9%      ~     (p=0.401 n=11+11)
Opcodes/REVERSE3/integer-8              261ns ±28%     220ns ± 6%   -15.67%  (p=0.015 n=11+11)
Opcodes/REVERSE3/big_bytes-8            245ns ±31%     218ns ± 7%      ~     (p=0.051 n=10+11)
Opcodes/REVERSE4/null-8                 223ns ±10%     218ns ± 6%      ~     (p=0.300 n=11+11)
Opcodes/REVERSE4/integer-8              233ns ±10%     220ns ± 7%    -5.74%  (p=0.016 n=11+11)
Opcodes/REVERSE4/big_bytes-8            225ns ±10%     220ns ± 7%      ~     (p=0.157 n=10+11)
Opcodes/REVERSEN/5/null-8               281ns ±12%     277ns ± 4%      ~     (p=0.847 n=11+11)
Opcodes/REVERSEN/5/integer-8            280ns ±11%     275ns ± 5%      ~     (p=0.243 n=11+11)
Opcodes/REVERSEN/5/big_bytes-8          283ns ± 9%     276ns ± 7%      ~     (p=0.133 n=11+11)
Opcodes/REVERSEN/1024/null-8           4.85µs ± 6%    1.94µs ± 6%   -60.07%  (p=0.000 n=10+11)
Opcodes/REVERSEN/1024/integer-8        4.97µs ± 7%    1.99µs ±22%   -59.88%  (p=0.000 n=11+11)
Opcodes/REVERSEN/1024/big_bytes-8      5.11µs ±10%    2.00µs ± 4%   -60.87%  (p=0.000 n=10+9)
Opcodes/PACK/1-8                       1.22µs ± 7%    0.95µs ± 6%   -22.17%  (p=0.000 n=10+11)
Opcodes/PACK/255-8                     11.1µs ± 4%    10.2µs ± 6%    -7.96%  (p=0.000 n=11+11)
Opcodes/PACK/1024-8                    38.9µs ± 4%    37.4µs ± 9%      ~     (p=0.173 n=10+11)
Opcodes/UNPACK/1-8                     1.32µs ±34%    0.96µs ± 6%   -27.57%  (p=0.000 n=10+11)
Opcodes/UNPACK/255-8                   27.2µs ±14%    16.0µs ±13%   -41.04%  (p=0.000 n=11+11)
Opcodes/UNPACK/1024-8                   102µs ±10%      64µs ±16%   -37.33%  (p=0.000 n=10+11)

name                                 old alloc/op   new alloc/op   delta
Opcodes/XDROP/0/1-8                     0.00B          0.00B           ~     (all equal)
Opcodes/XDROP/0/1024-8                  0.00B          0.00B           ~     (all equal)
Opcodes/XDROP/1024/1024-8               0.00B          0.00B           ~     (all equal)
Opcodes/XDROP/2047/2048-8               0.00B          0.00B           ~     (all equal)
Opcodes/DUP/null-8                      48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/boolean-8                   48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/integer/small-8             96.0B ± 0%     48.0B ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/DUP/integer/big-8                104B ± 0%       56B ± 0%   -46.15%  (p=0.000 n=11+11)
Opcodes/DUP/bytearray/small-8           88.0B ± 0%     40.0B ± 0%   -54.55%  (p=0.000 n=11+11)
Opcodes/DUP/bytearray/big-8            65.6kB ± 0%    65.6kB ± 0%    -0.07%  (p=0.000 n=10+9)
Opcodes/DUP/buffer/small-8              48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/buffer/big-8                48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/struct/small-8              48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/struct/big-8                48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/pointer-8                    112B ± 0%       64B ± 0%   -42.86%  (p=0.000 n=11+11)
Opcodes/OVER/null-8                     48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/boolean-8                  48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/integer/small-8            96.0B ± 0%     48.0B ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/OVER/integer/big-8               104B ± 0%       56B ± 0%   -46.15%  (p=0.000 n=11+11)
Opcodes/OVER/bytearray/small-8          88.0B ± 0%     40.0B ± 0%   -54.55%  (p=0.000 n=11+11)
Opcodes/OVER/bytearray/big-8           65.6kB ± 0%    65.6kB ± 0%    -0.07%  (p=0.000 n=9+11)
Opcodes/OVER/buffer/small-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/buffer/big-8               48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/struct/small-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/struct/big-8               48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/pointer-8                   112B ± 0%       64B ± 0%   -42.86%  (p=0.000 n=11+11)
Opcodes/PICK/2/null-8                   48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/boolean-8                48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/integer/small-8          96.0B ± 0%     48.0B ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/integer/big-8             104B ± 0%       56B ± 0%   -46.15%  (p=0.000 n=11+11)
Opcodes/PICK/2/bytearray/small-8        88.0B ± 0%     40.0B ± 0%   -54.55%  (p=0.000 n=11+11)
Opcodes/PICK/2/bytearray/big-8         65.6kB ± 0%    65.6kB ± 0%    -0.07%  (p=0.001 n=9+11)
Opcodes/PICK/2/buffer/small-8           48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/buffer/big-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/struct/small-8           48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/struct/big-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/pointer-8                 112B ± 0%       64B ± 0%   -42.86%  (p=0.000 n=11+11)
Opcodes/PICK/1024/null-8                48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/boolean-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/integer/small-8       96.0B ± 0%     48.0B ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/integer/big-8          104B ± 0%       56B ± 0%   -46.15%  (p=0.000 n=11+11)
Opcodes/PICK/1024/bytearray/small-8     88.0B ± 0%     40.0B ± 0%   -54.55%  (p=0.000 n=11+11)
Opcodes/PICK/1024/bytearray/big-8      65.6kB ± 0%    65.6kB ± 0%    -0.07%  (p=0.000 n=11+11)
Opcodes/PICK/1024/buffer/small-8        48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/buffer/big-8          48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/struct/small-8        48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/pointer-8              112B ± 0%       64B ± 0%   -42.86%  (p=0.000 n=11+11)
Opcodes/TUCK/null-8                     48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/boolean-8                  48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/integer/small-8            96.0B ± 0%     48.0B ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/TUCK/integer/big-8               104B ± 0%       56B ± 0%   -46.15%  (p=0.000 n=11+11)
Opcodes/TUCK/bytearray/small-8          88.0B ± 0%     40.0B ± 0%   -54.55%  (p=0.000 n=11+11)
Opcodes/TUCK/bytearray/big-8           65.6kB ± 0%    65.6kB ± 0%    -0.07%  (p=0.000 n=10+11)
Opcodes/TUCK/buffer/small-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/buffer/big-8               48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/struct/small-8             48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/struct/big-8               48.0B ± 0%      0.0B       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/pointer-8                   112B ± 0%       64B ± 0%   -42.86%  (p=0.000 n=11+11)
Opcodes/SWAP/null-8                     0.00B          0.00B           ~     (all equal)
Opcodes/SWAP/integer-8                  0.00B          0.00B           ~     (all equal)
Opcodes/SWAP/big_bytes-8                0.00B          0.00B           ~     (all equal)
Opcodes/ROT/null-8                      0.00B          0.00B           ~     (all equal)
Opcodes/ROT/integer-8                   0.00B          0.00B           ~     (all equal)
Opcodes/ROT/big_bytes-8                 0.00B          0.00B           ~     (all equal)
Opcodes/ROLL/4/null-8                   0.00B          0.00B           ~     (all equal)
Opcodes/ROLL/4/integer-8                0.00B          0.00B           ~     (all equal)
Opcodes/ROLL/4/big_bytes-8              0.00B          0.00B           ~     (all equal)
Opcodes/ROLL/1024/null-8                0.00B          0.00B           ~     (all equal)
Opcodes/ROLL/1024/integer-8             0.00B          0.00B           ~     (all equal)
Opcodes/ROLL/1024/big_bytes-8           0.00B          0.00B           ~     (all equal)
Opcodes/REVERSE3/null-8                 0.00B          0.00B           ~     (all equal)
Opcodes/REVERSE3/integer-8              0.00B          0.00B           ~     (all equal)
Opcodes/REVERSE3/big_bytes-8            0.00B          0.00B           ~     (all equal)
Opcodes/REVERSE4/null-8                 0.00B          0.00B           ~     (all equal)
Opcodes/REVERSE4/integer-8              0.00B          0.00B           ~     (all equal)
Opcodes/REVERSE4/big_bytes-8            0.00B          0.00B           ~     (all equal)
Opcodes/REVERSEN/5/null-8               0.00B          0.00B           ~     (all equal)
Opcodes/REVERSEN/5/integer-8            0.00B          0.00B           ~     (all equal)
Opcodes/REVERSEN/5/big_bytes-8          0.00B          0.00B           ~     (all equal)
Opcodes/REVERSEN/1024/null-8            0.00B          0.00B           ~     (all equal)
Opcodes/REVERSEN/1024/integer-8         0.00B          0.00B           ~     (all equal)
Opcodes/REVERSEN/1024/big_bytes-8       0.00B          0.00B           ~     (all equal)
Opcodes/PACK/1-8                         144B ± 0%       96B ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PACK/255-8                     4.22kB ± 0%    4.18kB ± 0%    -1.14%  (p=0.000 n=11+11)
Opcodes/PACK/1024-8                    16.5kB ± 0%    16.5kB ± 0%    -0.29%  (p=0.000 n=11+11)
Opcodes/UNPACK/1-8                       168B ± 0%       72B ± 0%   -57.14%  (p=0.000 n=11+11)
Opcodes/UNPACK/255-8                   12.4kB ± 0%     7.8kB ± 0%   -37.28%  (p=0.000 n=11+11)
Opcodes/UNPACK/1024-8                  49.3kB ± 0%    52.8kB ± 0%    +7.18%  (p=0.000 n=11+11)

name                                 old allocs/op  new allocs/op  delta
Opcodes/XDROP/0/1-8                      0.00           0.00           ~     (all equal)
Opcodes/XDROP/0/1024-8                   0.00           0.00           ~     (all equal)
Opcodes/XDROP/1024/1024-8                0.00           0.00           ~     (all equal)
Opcodes/XDROP/2047/2048-8                0.00           0.00           ~     (all equal)
Opcodes/DUP/null-8                       1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/boolean-8                    1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/integer/small-8              3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/DUP/integer/big-8                3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/DUP/bytearray/small-8            3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/DUP/bytearray/big-8              3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/DUP/buffer/small-8               1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/buffer/big-8                 1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/struct/small-8               1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/struct/big-8                 1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/DUP/pointer-8                    2.00 ± 0%      1.00 ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/OVER/null-8                      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/boolean-8                   1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/integer/small-8             3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/OVER/integer/big-8               3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/OVER/bytearray/small-8           3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/OVER/bytearray/big-8             3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/OVER/buffer/small-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/buffer/big-8                1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/struct/small-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/struct/big-8                1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/OVER/pointer-8                   2.00 ± 0%      1.00 ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/null-8                    1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/boolean-8                 1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/integer/small-8           3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/2/integer/big-8             3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/2/bytearray/small-8         3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/2/bytearray/big-8           3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/2/buffer/small-8            1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/buffer/big-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/struct/small-8            1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/struct/big-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/2/pointer-8                 2.00 ± 0%      1.00 ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/null-8                 1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/boolean-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/integer/small-8        3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/1024/integer/big-8          3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/1024/bytearray/small-8      3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/1024/bytearray/big-8        3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/PICK/1024/buffer/small-8         1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/buffer/big-8           1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/struct/small-8         1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/PICK/1024/pointer-8              2.00 ± 0%      1.00 ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/TUCK/null-8                      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/boolean-8                   1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/integer/small-8             3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/TUCK/integer/big-8               3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/TUCK/bytearray/small-8           3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/TUCK/bytearray/big-8             3.00 ± 0%      2.00 ± 0%   -33.33%  (p=0.000 n=11+11)
Opcodes/TUCK/buffer/small-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/buffer/big-8                1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/struct/small-8              1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/struct/big-8                1.00 ± 0%      0.00       -100.00%  (p=0.000 n=11+11)
Opcodes/TUCK/pointer-8                   2.00 ± 0%      1.00 ± 0%   -50.00%  (p=0.000 n=11+11)
Opcodes/SWAP/null-8                      0.00           0.00           ~     (all equal)
Opcodes/SWAP/integer-8                   0.00           0.00           ~     (all equal)
Opcodes/SWAP/big_bytes-8                 0.00           0.00           ~     (all equal)
Opcodes/ROT/null-8                       0.00           0.00           ~     (all equal)
Opcodes/ROT/integer-8                    0.00           0.00           ~     (all equal)
Opcodes/ROT/big_bytes-8                  0.00           0.00           ~     (all equal)
Opcodes/ROLL/4/null-8                    0.00           0.00           ~     (all equal)
Opcodes/ROLL/4/integer-8                 0.00           0.00           ~     (all equal)
Opcodes/ROLL/4/big_bytes-8               0.00           0.00           ~     (all equal)
Opcodes/ROLL/1024/null-8                 0.00           0.00           ~     (all equal)
Opcodes/ROLL/1024/integer-8              0.00           0.00           ~     (all equal)
Opcodes/ROLL/1024/big_bytes-8            0.00           0.00           ~     (all equal)
Opcodes/REVERSE3/null-8                  0.00           0.00           ~     (all equal)
Opcodes/REVERSE3/integer-8               0.00           0.00           ~     (all equal)
Opcodes/REVERSE3/big_bytes-8             0.00           0.00           ~     (all equal)
Opcodes/REVERSE4/null-8                  0.00           0.00           ~     (all equal)
Opcodes/REVERSE4/integer-8               0.00           0.00           ~     (all equal)
Opcodes/REVERSE4/big_bytes-8             0.00           0.00           ~     (all equal)
Opcodes/REVERSEN/5/null-8                0.00           0.00           ~     (all equal)
Opcodes/REVERSEN/5/integer-8             0.00           0.00           ~     (all equal)
Opcodes/REVERSEN/5/big_bytes-8           0.00           0.00           ~     (all equal)
Opcodes/REVERSEN/1024/null-8             0.00           0.00           ~     (all equal)
Opcodes/REVERSEN/1024/integer-8          0.00           0.00           ~     (all equal)
Opcodes/REVERSEN/1024/big_bytes-8        0.00           0.00           ~     (all equal)
Opcodes/PACK/1-8                         5.00 ± 0%      4.00 ± 0%   -20.00%  (p=0.000 n=11+11)
Opcodes/PACK/255-8                       5.00 ± 0%      4.00 ± 0%   -20.00%  (p=0.000 n=11+11)
Opcodes/PACK/1024-8                      5.00 ± 0%      4.00 ± 0%   -20.00%  (p=0.000 n=11+11)
Opcodes/UNPACK/1-8                       5.00 ± 0%      3.00 ± 0%   -40.00%  (p=0.000 n=11+11)
Opcodes/UNPACK/255-8                      259 ± 0%         7 ± 0%   -97.30%  (p=0.000 n=11+11)
Opcodes/UNPACK/1024-8                   1.03k ± 0%     0.01k ± 0%   -98.93%  (p=0.000 n=11+11)
2021-08-24 15:28:14 +03:00
Roman Khimov
7808762ba0 transaction: avoid reencoding and reading what can't be read
name               old time/op    new time/op    delta
DecodeFromBytes-8    1.79µs ± 2%    1.46µs ± 4%  -18.44%  (p=0.000 n=10+10)

name               old alloc/op   new alloc/op   delta
DecodeFromBytes-8      800B ± 0%      624B ± 0%  -22.00%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
DecodeFromBytes-8      10.0 ± 0%       8.0 ± 0%  -20.00%  (p=0.000 n=10+10)
2021-08-23 21:41:38 +03:00
Roman Khimov
2808f6857d interop: don't allocate for Functions and Notifications in New
Functions are usually immediately replaced (and it's OK for them to be nil,
searching through an array with length of zero is fine), Notifications are
usually appended to (and are absolutely useless in verification contexts).
2021-08-20 11:56:28 +03:00
Roman Khimov
a68a8aa8fc core: simplify and correct notification handling
* both 'to' and 'from' are either Null or Hash160, there is no other
   possibility for valid NEP-17. So returning util.Uint160{} in case of
   parsing error is wrong.
 * but this is what allowed burns/mints to work at the expense of error
   allocation inside of util.Uint160DecodeBytesBE()
 * Uint160 can technically fit into regular VM integer, so even though it'd be
   quite surprising to see it there, TryBytes() is more correct (and easier!)
   to use
 * same thing with `amount`, we have `TryInteger()` that easily covers all
   possible cases and does appropriate error checking inside
2021-08-20 11:26:16 +03:00
Roman Khimov
abc48229a3 block: Grow buffer on Trim, avoid reallocations 2021-08-20 11:05:46 +03:00
Anna Shaleva
5f9d38f640 core: refactor (*DAO).StoreAsTransaction
Squash (*DAO).StoreAsTransaction and
(*DAO).StoreConflictingTransactions. It's better to keep them this way,
because StoreAsTransaction is always followed by
StoreConflictingTransactions, so it's an atomic operation.

The logic wasn't changed.
2021-08-18 13:39:28 +03:00
Anna Shaleva
4b35a1cf92 core: remove conflicting transactions wrt MaxTraceableBlocks 2021-08-18 13:31:47 +03:00
Roman Khimov
f477a48758 contract: block calls to contracts via Policy contract
See neo-project/neo#2567.
2021-08-17 15:24:06 +03:00
Roman Khimov
adc660c3e0
Merge pull request #2123 from nspcc-dev/store-better
Store better
2021-08-13 12:50:24 +03:00
Roman Khimov
ae071d4542 storage: introduce PutChangeSet and use it for Persist
We're using batches in wrong way during persist, we already have all changes
accumulated in two maps and then we move them to batch and then this is
applied. For some DBs like BoltDB this batch is just another MemoryStore, so
we essentially just shuffle the changeset from one map to another, for others
like LevelDB batch is just a serialized set of KV pairs, it doesn't help much
on subsequent PutBatch, we just duplicate the changeset again.

So introduce PutChangeSet that allows to take two maps with sets and deletes
directly. It also allows to simplify MemCachedStore logic.

neo-bench for single node with 10 workers, LevelDB:

  Reference:

  RPS    30189.132 30556.448 30390.482 ≈ 30379    ±  0.61%
  TPS    29427.344 29418.687 29434.273 ≈ 29427    ±  0.03%
  CPU %     33.304    27.179    33.860 ≈    31.45 ± 11.79%
  Mem MB   800.677   798.389   715.042 ≈   771    ±  6.33%

  Patched:

  RPS    30264.326 30386.364 30166.231 ≈ 30272    ± 0.36% ⇅
  TPS    29444.673 29407.440 29452.478 ≈ 29435    ± 0.08% ⇅
  CPU %     34.012    32.597    33.467 ≈   33.36  ± 2.14% ⇅
  Mem MB   549.126   523.656   517.684 ≈  530     ± 3.15% ↓ 31.26%

BoltDB:

  Reference:

  RPS    31937.647 31551.684 31850.408 ≈ 31780    ±  0.64%
  TPS    31292.049 30368.368 31307.724 ≈ 30989    ±  1.74%
  CPU %     33.792    22.339    35.887 ≈    30.67 ± 23.78%
  Mem MB  1271.687  1254.472  1215.639 ≈  1247    ±  2.30%

  Patched:

  RPS    31746.818 30859.485 31689.761 ≈ 31432    ± 1.58% ⇅
  TPS    31271.499 30340.726 30342.568 ≈ 30652    ± 1.75% ⇅
  CPU %     34.611    34.414    31.553 ≈    33.53 ± 5.11% ⇅
  Mem MB  1262.960  1231.389  1335.569 ≈  1277    ± 4.18% ⇅
2021-08-12 17:42:16 +03:00
Roman Khimov
5aff82aef4
Merge pull request #2119 from nspcc-dev/states-exchange/insole
core, network: prepare basis for Insole module
2021-08-12 10:35:02 +03:00
Roman Khimov
47f0f4c45f dao: completely drop Cached
It was very useful in 2.0 days, but today it only serves one purpose that
could easily (and more effectively!) be solved in another way.
2021-08-11 23:06:17 +03:00
Roman Khimov
3e60771175 core: deduplicate and simplify processNEP17Transfer a bit
Just refactoring, no functional changes.
2021-08-11 22:36:26 +03:00
Roman Khimov
50ee1a1f91 *: don't use dao.Cached in tests
There is no need to use it.
2021-08-11 21:02:50 +03:00
Roman Khimov
18682f2409 storage: don't use locks for memory batches
They're inherently single-threaded, so locking makes no sense for them.
2021-08-11 18:55:07 +03:00
Roman Khimov
13da1b62fb interop: fetch baseExecFee once and keep it in the Context
It never changes during single execution, so we can cache it and avoid going
to Policer via Chain for every instruction.
2021-08-11 15:42:23 +03:00
Anna Shaleva
0e3b9c48a2 core: add API to store StateSyncPoint and StateSyncCurrentBlockHeight
We need it in order not to mess up the blockchain which has its own
CurrentBlockHeight.
2021-08-10 14:06:28 +03:00
Anna Shaleva
cb01f533c0 core: store conflicting transactions in a separate method
(DAO).StoreConflictingTransactions will be reused from the state sync
module.
2021-08-10 13:47:13 +03:00
Anna Shaleva
72e654332e core: refactor block queue
It requires only two methods from Blockchainer: AddBlock and
BlockHeight. New interface will allow to easily reuse the block queue
for state exchange purposes.
2021-08-10 13:47:13 +03:00
Anna Shaleva
35501a281a core: remove untraceable blocks wrt StateSyncInterval 2021-08-10 13:47:10 +03:00
Anna Shaleva
76c687aaa1 config: add P2PStateExchangeExtensions and StateSyncInterval settings 2021-08-10 11:00:32 +03:00
Roman Khimov
1e0c70ecb0
Merge pull request #2117 from nspcc-dev/io-grow
Some io package improvements
2021-08-10 09:57:31 +03:00
Evgeniy Stratonikov
73e4040628 mpt: use BinWriter.Grow() instead of custom buffer
Also add benchmarks.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-10 09:34:05 +03:00
Evgeniy Stratonikov
620295efe3 transaction: add benchmark for transaction serialization
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-09 12:01:38 +03:00
Evgeniy Stratonikov
23adb1e2fc state: optimize NEP17TransferLog.Append
Do not allocate a separate buffer for the transfer.
```
name                       old time/op    new time/op    delta
NEP17TransferLog_Append-8    58.8µs ± 3%    32.1µs ± 1%  -45.40%  (p=0.000 n=10+9)

name                       old alloc/op   new alloc/op   delta
NEP17TransferLog_Append-8     118kB ± 1%      44kB ± 3%  -63.00%  (p=0.000 n=9+10)

name                       old allocs/op  new allocs/op  delta
NEP17TransferLog_Append-8       901 ± 1%       513 ± 3%  -43.08%  (p=0.000 n=9+8)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-09 11:09:49 +03:00
Evgeniy Stratonikov
403a4b75de state/test: add benchmark for NEP17TransferLog.Append
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-09 11:09:49 +03:00
Evgeniy Stratonikov
b210a34b1e state: optimize NEP17Balance deserialization
```
BenchmarkNEP17BalanceFromBytes/stackitem-8         	 2402318	       503.3 ns/op	     208 B/op	      10 allocs/op
BenchmarkNEP17BalanceFromBytes/from_bytes-8        	 7623139	       160.7 ns/op	      72 B/op	       3 allocs/op
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-09 11:09:49 +03:00
Evgeniy Stratonikov
3218b74ea5 state: optimize NEP17Balance serialization
Put to slice directly and allow to provide pre-allocated buffer.
```
BenchmarkNEP17BalanceBytes/stackitem-8         	 1712475	       673.4 ns/op	     448 B/op	       9 allocs/op
BenchmarkNEP17BalanceBytes/bytes-8             	13422715	        75.80 ns/op	      32 B/op	       2 allocs/op
BenchmarkNEP17BalanceBytes/bytes,_prealloc-8   	25990371	        46.46 ns/op	      16 B/op	       1 allocs/op
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-09 11:09:06 +03:00
Roman Khimov
b989504d74
Merge pull request #2108 from nspcc-dev/optimize-mpt
Some allocation optimizations
2021-08-06 14:51:10 +03:00
Evgeniy Stratonikov
bd2b1a0521 mpt: add Size method to trie nodes
Knowing serialized size of the node is useful for
preallocating byte-slice in advance.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 12:01:16 +03:00
Evgeniy Stratonikov
db80ef28df mpt: move empty hash node in a separate type
We use them quite frequently (consider children for a new branch
node) and it is better to get rid of unneeded allocations.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 12:01:16 +03:00
Evgeniy Stratonikov
291a29af1e *: do not use WriteArray for frequently used items
`WriteArray` involves reflection, it makes sense to optimize
serialization of transactions and application logs which are serialized
constantly. Adding case in a type switch in `WriteArray` is not an
option because we don't want new dependencies for `io` package.

```
name                          old time/op    new time/op    delta
AppExecResult_EncodeBinary-8     852ns ± 3%     656ns ± 2%  -22.94%  (p=0.000 n=10+9)

name                          old alloc/op   new alloc/op   delta
AppExecResult_EncodeBinary-8      448B ± 0%      376B ± 0%  -16.07%  (p=0.000 n=10+10)

name                          old allocs/op  new allocs/op  delta
AppExecResult_EncodeBinary-8      7.00 ± 0%      5.00 ± 0%  -28.57%  (p=0.000 n=10+10)
```

```
name                 old time/op    new time/op    delta
Transaction_Bytes-8    1.29µs ± 3%    0.76µs ± 5%  -41.52%  (p=0.000 n=9+10)

name                 old alloc/op   new alloc/op   delta
Transaction_Bytes-8    1.21kB ± 0%    1.01kB ± 0%  -16.56%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transaction_Bytes-8      12.0 ± 0%       7.0 ± 0%  -41.67%  (p=0.000 n=10+10)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 11:59:20 +03:00
Roman Khimov
39f874d03f core: don't recalculate witness script hash
We know it already, but with current loading code VM will hash it once
more. It doesn't help a lot and still it costs nothing to avoid this
overhead.

name             old time/op    new time/op    delta
VerifyWitness-8    93.4µs ± 3%    92.7µs ± 2%    ~     (p=0.353 n=10+10)

name             old alloc/op   new alloc/op   delta
VerifyWitness-8    3.43kB ± 0%    3.40kB ± 0%  -0.70%  (p=0.000 n=9+9)

name             old allocs/op  new allocs/op  delta
VerifyWitness-8      67.0 ± 0%      66.0 ± 0%  -1.49%  (p=0.000 n=10+10)
2021-08-06 11:25:09 +03:00
Evgeniy Stratonikov
43ee671f36 mpt: do not allocate NodeObject for serialization
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-06 10:28:19 +03:00
Roman Khimov
e41fc2fd1b
Merge pull request #2111 from nspcc-dev/drop-refuel
native: drop Refuel method from GAS
2021-08-05 16:42:29 +03:00
Roman Khimov
d6bd6b6888 native: drop Refuel method from GAS
It can be used to attack the network (amplifying DOS), so it's broken
beyond repair. This reverts ac601601c1.

See also neo-project/neo#2560 and neo-project/neo#2561.
2021-08-05 10:27:13 +03:00
Roman Khimov
892c9785ad transaction: don't allocate new buffer to calculate hash
We can write directly to hash.Hash.

name               old time/op    new time/op    delta
DecodeBinary-8       2.89µs ± 3%    2.82µs ± 5%     ~     (p=0.052 n=10+10)
DecodeJSON-8         13.0µs ± 1%    12.8µs ± 1%   -1.54%  (p=0.002 n=10+8)
DecodeFromBytes-8    2.37µs ± 1%    2.25µs ± 5%   -5.25%  (p=0.000 n=9+10)

name               old alloc/op   new alloc/op   delta
DecodeBinary-8       1.75kB ± 0%    1.53kB ± 0%  -12.79%  (p=0.000 n=10+10)
DecodeJSON-8         3.49kB ± 0%    3.26kB ± 0%   -6.42%  (p=0.000 n=10+10)
DecodeFromBytes-8    1.37kB ± 0%    1.14kB ± 0%  -16.37%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
DecodeBinary-8         26.0 ± 0%      23.0 ± 0%  -11.54%  (p=0.000 n=10+10)
DecodeJSON-8           58.0 ± 0%      55.0 ± 0%   -5.17%  (p=0.000 n=10+10)
DecodeFromBytes-8      18.0 ± 0%      15.0 ± 0%  -16.67%  (p=0.000 n=10+10)
2021-08-04 23:43:20 +03:00
Roman Khimov
6d10cdc2f6 transaction: avoid ReadArray()
Reflection adds some real cost to it:

name               old time/op    new time/op    delta
DecodeBinary-8       3.14µs ± 5%    2.89µs ± 3%   -8.19%  (p=0.000 n=10+10)
DecodeJSON-8         12.6µs ± 3%    13.0µs ± 1%   +3.77%  (p=0.000 n=10+10)
DecodeFromBytes-8    2.73µs ± 2%    2.37µs ± 1%  -13.12%  (p=0.000 n=9+9)

name               old alloc/op   new alloc/op   delta
DecodeBinary-8       1.82kB ± 0%    1.75kB ± 0%   -3.95%  (p=0.000 n=10+10)
DecodeJSON-8         3.49kB ± 0%    3.49kB ± 0%     ~     (all equal)
DecodeFromBytes-8    1.44kB ± 0%    1.37kB ± 0%   -5.00%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
DecodeBinary-8         29.0 ± 0%      26.0 ± 0%  -10.34%  (p=0.000 n=10+10)
DecodeJSON-8           58.0 ± 0%      58.0 ± 0%     ~     (all equal)
DecodeFromBytes-8      21.0 ± 0%      18.0 ± 0%  -14.29%  (p=0.000 n=10+10)
2021-08-04 23:34:57 +03:00
Roman Khimov
d2732a71d8 transaction: don't overwrite error and witnesses length check
ReadArray() can return some error and we shouldn't overwrite it. At the same
time limiting ReadArray() to the number of Signers can make it return wrong
error if the number of witnesses actually is bigger than the number of
signers, so use MaxAttributes.
2021-08-04 23:17:50 +03:00
Roman Khimov
d487b54612 transaction: don't recalculate size when decoding from buffer
name               old time/op    new time/op    delta
DecodeBinary-8       3.17µs ± 6%    3.14µs ± 5%     ~     (p=0.579 n=10+10)
DecodeJSON-8         12.8µs ± 3%    12.6µs ± 3%     ~     (p=0.105 n=10+10)
DecodeFromBytes-8    3.45µs ± 4%    2.73µs ± 2%  -20.70%  (p=0.000 n=10+9)

name               old alloc/op   new alloc/op   delta
DecodeBinary-8       1.82kB ± 0%    1.82kB ± 0%     ~     (all equal)
DecodeJSON-8         3.49kB ± 0%    3.49kB ± 0%     ~     (all equal)
DecodeFromBytes-8    1.82kB ± 0%    1.44kB ± 0%  -21.05%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
DecodeBinary-8         29.0 ± 0%      29.0 ± 0%     ~     (all equal)
DecodeJSON-8           58.0 ± 0%      58.0 ± 0%     ~     (all equal)
DecodeFromBytes-8      29.0 ± 0%      21.0 ± 0%  -27.59%  (p=0.000 n=10+10)
2021-08-04 23:13:58 +03:00
Roman Khimov
64c780ad7a native: optimize totalSupply operations during token burn/mint
We burn GAS in OnPersist for every transaction so some buffer reuse here is
quite natural.

This also doesn't change a lot in the overall TPS picture, maybe adding some
1%.
2021-08-03 17:59:38 +03:00
Roman Khimov
dede4fa7b1 state: convert NEO balance to stack item directly
Avoid calling Append() that will reallocate the slice, we know the length of
the slice exactly.
2021-08-03 17:59:38 +03:00
Roman Khimov
5c65d33439 native: move required balance check to token contracts
Which duplicates the check, but deduplicates error path. This check forced
double balance deserialization which is quite costly operation, so we better
do it once.

It's hardly noticeable as of TPS metrics though, maybe some 1-2%%.
2021-08-03 17:59:38 +03:00
Roman Khimov
3c1325035e fee: use array for opcodes
Use less memory and have faster access.

name       old time/op  new time/op  delta
Opcode1-8  22.4ns ± 6%   3.0ns ± 6%  -86.63%  (p=0.000 n=10+10)
2021-08-02 20:18:33 +03:00
Roman Khimov
dfc514eda0
Merge pull request #2102 from nspcc-dev/store4
Improve (*MemCachedStore).Persist
2021-08-02 20:10:05 +03:00
Roman Khimov
82f481e143
Merge pull request #2105 from nspcc-dev/json-restrict
native/std: restrint amount of items in JSON deserialization
2021-08-02 19:41:54 +03:00
Evgeniy Stratonikov
bdb9748c1b native/std: restrict amount of items in JSON deserialization
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-08-02 18:57:47 +03:00
Roman Khimov
f8174ca64c core: ensure data logged is from persistent store
Using bc.dao here is wrong, it can contain unpersisted data.
2021-08-02 16:33:09 +03:00
Roman Khimov
8277b7a19a core: don't spawn goroutine for persist function
It doesn't make any sense, in some situations it leads to a number of
goroutines created that will Persist one after another (as we can't Persist
concurrently). We can manage it better in a single thread.

This doesn't change performance in any way, but somewhat reduces resource
consumption. It was tested neo-bench (single node, 10 workers, LevelDB) on two
machines and block dump processing (RC4 testnet up to 62800 with VerifyBlocks
set to false) on i7-8565U.

Reference (b9be892bf9):

Ryzen 9 5950X:
RPS     27747.349 27407.726 27520.210  ≈ 27558   ± 0.63%
TPS     26992.010 26993.468 27010.966  ≈ 26999   ± 0.04%
CPU %      28.928    28.096    29.105  ≈    28.7 ± 1.88%
Mem MB    760.385   726.320   756.118  ≈   748   ± 2.48%

Core i7-8565U:
RPS     7783.229 7628.409 7542.340  ≈ 7651   ± 1.60%
TPS     7708.436 7607.397 7489.459  ≈ 7602   ± 1.44%
CPU %     74.899   71.020   72.697  ≈   72.9 ± 2.67%
Mem MB   438.047  436.967  416.350  ≈  430   ± 2.84%

DB restore:
real    0m20.838s 0m21.895s 0m21.794s  ≈ 21.51 ± 2.71%
user    0m39.091s 0m40.565s 0m41.493s  ≈ 40.38 ± 3.00%
sys      0m3.184s  0m2.923s  0m3.062s  ≈  3.06 ± 4.27%

Patched:

Ryzen 9 5950X:
RPS     27636.957 27246.911 27462.036  ≈ 27449   ±  0.71%  ↓ 0.40%
TPS     27003.672 26993.468 27011.696  ≈ 27003   ±  0.03%  ↑ 0.01%
CPU %      28.562    28.475    28.012  ≈    28.3 ±  1.04%  ↓ 1.39%
Mem MB    627.007   648.110   794.895  ≈   690   ± 13.25%  ↓ 7.75%

Core i7-8565U:
RPS     7497.210 7527.797 7897.532  ≈ 7641   ±  2.92%  ↓ 0.13%
TPS     7461.128 7482.678 7841.723  ≈ 7595   ±  2.81%  ↓ 0.09%
CPU %     71.559   73.423   69.005  ≈   71.3 ±  3.11%  ↓ 2.19%
Mem MB   393.090  395.899  482.264  ≈  424   ± 11.96%  ↓ 1.40%

DB restore:
real    0m20.773s 0m21.583s 0m20.522s  ≈ 20.96 ±  2.65%  ↓ 2.56%
user    0m39.322s 0m42.268s 0m38.626s  ≈ 40.07 ±  4.82%  ↓ 0.77%
sys      0m3.006s  0m3.597s  0m3.042s  ≈  3.22 ± 10.31%  ↑ 5.23%
2021-08-02 16:33:00 +03:00
Roman Khimov
b9be892bf9 storage: allow accessing MemCachedStore during Persist
Persist by its definition doesn't change MemCachedStore visible state, all KV
pairs that were acessible via it before Persist remain accessible after
Persist. The only thing it does is flushing of the current set of KV pairs
from memory to peristent store. To do that it needs read-only access to the
current KV pair set, but technically it then replaces maps, so we have to use
full write lock which makes MemCachedStore inaccessible for the duration of
Persist. And Persist can take a lot of time, it's about disk access for
regular DBs.

What we do here is we create new in-memory maps for MemCachedStore before
flushing old ones to the persistent store. Then a fake persistent store is
created which actually is a MemCachedStore with old maps, so it has exactly
the same visible state. This Store is never accessed for writes, so we can
read it without taking any internal locks and at the same time we no longer
need write locks for original MemCachedStore, we're not using it. All of this
makes it possible to use MemCachedStore as normally reads are handled going
down to whatever level is needed and writes are handled by new maps. So while
Persist for (*Blockchain).dao does its most time-consuming work we can process
other blocks (reading data for transactions and persisting storeBlock caches
to (*Blockchain).dao).

The change was tested for performance with neo-bench (single node, 10 workers,
LevelDB) on two machines and block dump processing (RC4 testnet up to 62800
with VerifyBlocks set to false) on i7-8565U.

Reference results (bbe4e9cd7b):

Ryzen 9 5950X:
RPS     23616.969 22817.086 23222.378  ≈ 23218   ± 1.72%
TPS     23047.316 22608.578 22735.540  ≈ 22797   ± 0.99%
CPU %      23.434    25.553    23.848  ≈    24.3 ± 4.63%
Mem MB    600.636   503.060   582.043  ≈   562   ± 9.22%

Core i7-8565U:
RPS     6594.007 6499.501 6572.902  ≈ 6555   ± 0.76%
TPS     6561.680 6444.545 6510.120  ≈ 6505   ± 0.90%
CPU %     58.452   60.568   62.474    ≈ 60.5 ± 3.33%
Mem MB   234.893  285.067  269.081   ≈ 263   ± 9.75%

DB restore:
real    0m22.237s 0m23.471s 0m23.409s  ≈ 23.04 ± 3.02%
user    0m35.435s 0m38.943s 0m39.247s  ≈ 37.88 ± 5.59%
sys      0m3.085s  0m3.360s  0m3.144s  ≈  3.20 ± 4.53%

After the change:

Ryzen 9 5950X:
RPS     27747.349 27407.726 27520.210  ≈ 27558   ± 0.63%  ↑ 18.69%
TPS     26992.010 26993.468 27010.966  ≈ 26999   ± 0.04%  ↑ 18.43%
CPU %      28.928    28.096    29.105  ≈    28.7 ± 1.88%  ↑ 18.1%
Mem MB    760.385   726.320   756.118  ≈   748   ± 2.48%  ↑ 33.10%

Core i7-8565U:
RPS     7783.229 7628.409 7542.340  ≈ 7651   ± 1.60%  ↑ 16.72%
TPS     7708.436 7607.397 7489.459  ≈ 7602   ± 1.44%  ↑ 16.85%
CPU %     74.899   71.020   72.697  ≈   72.9 ± 2.67%  ↑ 20.50%
Mem MB   438.047  436.967  416.350  ≈  430   ± 2.84%  ↑ 63.50%

DB restore:
real    0m20.838s 0m21.895s 0m21.794s  ≈ 21.51 ± 2.71%  ↓ 6.64%
user    0m39.091s 0m40.565s 0m41.493s  ≈ 40.38 ± 3.00%  ↑ 6.60%
sys      0m3.184s  0m2.923s  0m3.062s  ≈  3.06 ± 4.27%  ↓ 4.38%

It obviously uses more memory now and utilizes CPU more aggressively, but at
the same time it allows to improve all relevant metrics and finally reach a
situation where we process 50K transactions in less than second on Ryzen 9
5950X (going higher than 25K TPS). The other observation is much more stable
block time, on Ryzen 9 it's as close to 1 second as it could be.
2021-08-02 16:33:00 +03:00
Roman Khimov
3cebd2b129 interop: use non-Cached wrapped DAO
Cached only caches NEP-17 tracking data now, it makes no sense here.
2021-07-30 15:45:17 +03:00
Roman Khimov
fa7314ea90 dao: drop dropNEP17Cache from Cached
It's not used now.
2021-07-30 15:45:17 +03:00
Roman Khimov
49be753850 core: spread storeBlock actions to three goroutines
Block processing consists of:
 * saving block/transactions to the DB
 * executing blocks/transactions
 * processing notifications/saving AERs
 * updating MPT
 * atomically updating Blockchain state

Of these the first one is completely independent of others, it can be done in
a separate routine easily. The third one technically depends on the second,
it just doesn't have data until something is executed. At the same time it
doesn't affect future executions in any way, so we can offload
AER/notification processing to separate goroutine (while the main thread
proceeds with other transactions).

MPT update depends on all executions, so it can't be offloaded, but it can be
done concurrently to AER processing. And only the last thing actually needs
all previous ones to be finished, so it's a natural synchronization point.

So we spawn two additional routines and let the main one execute transactions
and update MPT as fast as it can. While technically all of these routines
could share single DAO (they are working with different KV sets) benchmarking
shows that using separate DAOs and then persisting them to lower one actually
works about 7-8%% better. At the same time we can simplify DAOs used, Cached
one is only relevant for AER processing because it caches NEP-17 tracking
data, everything else can do just fine with Simple.

The change was tested for performance with neo-bench (single node, 10 workers,
LevelDB) on two machines and block dump processing (RC4 testnet up to 50825
with VerifyBlocks set to false) on i7-8565U. neo-bench creates huge blocks
with lots of transactions while RC4 dump mostly consists of empty blocks.

Reference results (06c3dda5d1):

Ryzen 9 5950X:
RPS ≈ 20059.569   21186.328   20158.983   ≈ 20468   ±  3.05%
TPS ≈ 19544.993   20585.450   19658.338   ≈ 19930   ±  2.86%
CPU ≈    18.682%     23.877%     22.852%  ≈    21.8 ± 12.62%
Mem ≈   618.981MB   559.246MB   541.539MB ≈   573   ±  7.08%

Core i7-8565U:
RPS ≈ 5927.082   6526.739   6372.115   ≈ 6275   ± 4.96%
TPS ≈ 5899.531   6477.187   6329.515   ≈ 6235   ± 4.81%
CPU ≈   56.346%    61.955%    58.125%  ≈   58.8 ± 4.87%
Mem ≈  212.191MB  224.974MB  205.479MB ≈  214   ± 4.62%

DB restore:
real    0m12.683s 0m13.222s 0m13.382s  ≈ 13.096 ±  2.80%
user    0m18.501s 0m19.163s 0m19.489s  ≈ 19.051 ±  2.64%
sys      0m1.404s  0m1.396s  0m1.666s  ≈  1.489 ± 10.32%

After the change:

Ryzen 9 5950X:
RPS ≈ 23056.899   22822.015   23006.543   ≈ 22962   ± 0.54%
TPS ≈ 22594.785   22292.071   22800.857   ≈ 22562   ± 1.13%
CPU ≈    24.262%     23.185%     25.921%  ≈    24.5 ± 5.65%
Mem ≈   614.254MB   613.204MB   555.491MB ≈   594   ± 5.66%

Core i7-8565U:
RPS ≈ 6378.702   6423.927   6363.788      ≈ 6389   ± 0.49%
TPS ≈ 6327.072   6372.552   6311.179      ≈ 6337   ± 0.50%
CPU ≈   57.599%    58.622%    59.737%     ≈   58.7 ± 1.82%
Mem ≈  198.697MB  188.746MB  200.235MB    ≈  196   ± 3.18%

DB restore:
real    0m13.576s 0m13.334s 0m12.757s  ≈  13.222 ±  3.18%
user    0m19.113s 0m19.490s 0m20.197s  ≈  19.600 ±  2.81%
sys      0m2.211s  0m1.558s  0m1.559s  ≈   1.776 ± 21.21%

On Ryzen 9 we've got 12% better RPS, 13% better TPS with 12% CPU and 3% RAM
more used. Core i7-8565U changes don't seem to be statistically significant:
1.8% more RPS, 1.6% more TPS with about the same CPU and 8.5% less RAM
used. It also is 1% worse in DB restore time.

The result is somewhat expected, on a powerful machine with lots of spare
cores we get 10%+ better results while on average resource-constrained laptop it
doesn't change much (the machine is already saturated). Overall, this seems to
be worthwhile.
2021-07-30 15:45:17 +03:00
Roman Khimov
06c3dda5d1
Merge pull request #2093 from nspcc-dev/states-exchange/drop-nep17-balance-state
core: implement dynamic NEP17 balances tracking
2021-07-29 19:08:42 +03:00
Anna Shaleva
a30e48ff90 core: increment the DB version
DB scheme has been changed.
2021-07-29 10:23:13 +03:00
Anna Shaleva
e8bed184d5 core: implement dynamic NEP17 balances tracking
Request NEP17 balances from a set of NEP17 contracts instead of getting
them from storage. LastUpdatedBlock tracking remains untouched, because
there's no way to retrieve it dynamically.
2021-07-29 10:23:01 +03:00
Anna Shaleva
e46d76d7aa core: rename state.NEP17Balances to state.NEP17TransferInfo
Balances are to be removed from state.NEP17TransferInfo, so the remnant
fields are NextTransferBatch, NewBatch and a map of LastUpdatedBlocks.
These fields are more staff-related.

Also rename dao.[Get, Put, put]NEP17Balances and STNEP17Balances
preffix.

Also rename NEP17TransferInfo.Trackers to LastUpdatedBlockTrackers
because NEP17TransferInfo.Balances are to be removed.
2021-07-28 13:22:53 +03:00
Anna Shaleva
c0a2c74e0c core: maintain a set of NEP17-compliant contracts 2021-07-28 13:22:53 +03:00
Roman Khimov
50d99464e0
Merge pull request #2064 from nspcc-dev/fix-remove-stale-hang
mempool: send events in a separate goroutine
2021-07-23 18:16:14 +03:00
Evgeniy Stratonikov
3507f52c32 notary: process new transactions in a separate goroutine
Related #2063.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-23 14:48:00 +03:00
Roman Khimov
1d8ad5b84a *: simplify some error messages
Log:
2021-07-23T09:59:18.948+0300    WARN    contract invocation failed      {"tx": "de3e3c1f1d37e4528990f894dea5583fd320485ad3862a95eb0e8823eecf4a5f", "block": 9643, "error": "error encountered at instruction 1 (SYSCALL): error during call from native: error encountered at instruction 745 (CAT): invalid conversion: Map/ByteString"}

The word "error" appears 4 times here.
2021-07-23 10:08:09 +03:00
Roman Khimov
002ad9dfee go.mod: update miniredis to 2.15.1
It's only used for testing purposes and this version doesn't change anything
for us, but still better be current.
2021-07-21 23:28:26 +03:00
Roman Khimov
df07ba505a config: update mainnet magic
It's NEO3, see neo-project/neo-node#795.
2021-07-21 14:42:26 +03:00
Roman Khimov
35c2c3ae8e
Merge pull request #2078 from nspcc-dev/configurable-initial-gas
config: add InitialGASSupply, fix #2073
2021-07-20 17:10:25 +03:00
Roman Khimov
36d486a664 config: add InitialGASSupply, fix #2073
We now have 52M by default.
2021-07-20 16:59:54 +03:00
Roman Khimov
0583f252ab *: create real temporary dirs and files in tests
Improve reliability.
2021-07-20 12:51:11 +03:00
Roman Khimov
3b19b34122 storage: fix memcached test with boltdb store
Everything was wrong here, wrong file used, wrong cleanup procedure, the net
result is this (and some failing tests from time to time):

  $ ls -l /tmp/test_bolt_db* | wc -l
  30939
2021-07-20 12:35:24 +03:00
Roman Khimov
c88ebaede9
Merge pull request #2075 from nspcc-dev/small-refactoring
Array util refactoring and naming improvement
2021-07-20 11:29:59 +03:00
Roman Khimov
7477e2cd9f
Merge pull request #2074 from nspcc-dev/fix-oracle-service-behaviour
Fix oracle service behaviour
2021-07-20 11:29:39 +03:00
Roman Khimov
19717dd9a8 slice: introduce common Copy helper
It's a bit more convenient to use.
2021-07-19 22:57:55 +03:00
Roman Khimov
a54e3516d1 slice: add Reverse function, deduplicate code a bit 2021-07-19 22:57:55 +03:00
Roman Khimov
3a93977b7b oracle: only process new requests after initial sync
If an oracle node is resynchronized from the genesis the service receives all
requests from all blocks via AddRequests() invoked from the native
contract. Almost all of them are long obsolete and need to be removed, native
oracle contract will try to do that with RemoveRequests() calls, but they
won't change anything.

So queue up all "initial" requests in special map and manage it directly
before the module is Run() which happens after synchronization
completion. Then process any requests that are still active and work with new
blocks as usual.
2021-07-19 22:52:59 +03:00
Roman Khimov
588f3fbbd3 state: drop State from NEOBalance and NEP17Balance
We're in the `state` package already.
2021-07-19 22:01:07 +03:00
Roman Khimov
f4a9139a05
Merge pull request #2071 from nspcc-dev/fix-more-vm-bugs
Fix more VM bugs
2021-07-19 22:00:17 +03:00
Roman Khimov
79b1bf72aa native: check that oracle request GAS IsInt64()
We use int64 value down below, so check for IsInt64() while MinimumResponseGas
check still covers for values less than zero.
2021-07-19 15:42:42 +03:00
Roman Khimov
5f13de3a76 *: simplify some integer checks with IsUint64()
If we need a positive number we can do `IsUint64()` instead of checking that
`Int64()` is `> 0`.
2021-07-19 15:42:42 +03:00
Roman Khimov
233307aca5 stackitem: completely drop MaxArraySize
Turns out C# VM doesn't have it since preview2, so our limiting of
MaxArraySize in incompatible with it. Removing this limit shouldn't be a
problem with the reference counter we have, both APPEND and SETITEM add things
to reference counter and we can't exceed MaxStackSize. PACK on the other hand
can't get more than MaxStackSize-1 of input elements.

Unify NEWSTRUCT with NEWARRAY* and use better integer checks at the same time.

Multisig limit is still 1024.
2021-07-19 15:42:42 +03:00
Roman Khimov
aab18c3083 stackitem: introduce Convertible interface
We have a lot of native contract types that are converted to stack items
before serialization, then deserialized as stack items and converted back to
regular structures. stackitem.Convertible allows to remove a lot of repetitive
io.Serializable code.

This also introduces to/from converter in testserdes which unfortunately
required to change util tests to avoid circular references.
2021-07-19 15:42:42 +03:00
Roman Khimov
2d993d0da5 state: store notary deposit as stackitem
Which is less effective, but makes it more similar to other native contracts
that are supposed to be contracts anyway.
2021-07-19 15:42:42 +03:00
Roman Khimov
70ddbf7180 native: reuse stackitem.(De)Serialize more for data structures
Less code bloat, no functional changes.
2021-07-19 15:42:42 +03:00
Roman Khimov
4775b513f9 native: do proper error handling when deserializing user data 2021-07-19 15:42:42 +03:00
Roman Khimov
3a991abb62 fee: adjust SQRT price
See neo-project/neo#2540.
2021-07-19 15:42:41 +03:00
Roman Khimov
21e05f5779
Merge pull request #2055 from nspcc-dev/fix-gas-limits
Tune fixed GAS limits
2021-07-16 16:53:06 +03:00
Evgeniy Stratonikov
8077f9232d interop: implement System.Runtime.GetRandom
Our murmur3 implementation is architecture independent and optimized in
assembly.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-15 16:00:01 +03:00
Evgeniy Stratonikov
fdb54f2dc3 core/test: get rid of empty tx scripts
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-15 15:58:49 +03:00
Evgeniy Stratonikov
3a4e0caeb8 core/block: add Nonce field to header
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-15 15:58:49 +03:00
Evgeniy Stratonikov
2cd5b63f0b policy: fix max exec fee
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-14 10:27:09 +03:00
Evgeniy Stratonikov
451b02122a *: increase GAS for verification
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-14 10:27:09 +03:00
Roman Khimov
2b7abd20e7
Merge pull request #2057 from nspcc-dev/optimize-storage-find
Optimize `storageFind`
2021-07-13 12:51:10 +03:00
Roman Khimov
83a557f0eb
Merge pull request #2042 from nspcc-dev/oracle-not-supported
Check oracle response Content-Type header
2021-07-13 11:31:37 +03:00
Evgeniy Stratonikov
eb7bd7b99b core: optimize storageFind
Because `Map` stores elements in arbitrary order, addition of new
element takes linear time (`Index` iterates over all keys). Thus our
`storageFind` is actually quadratic in time. Optimize this by creating
map from sorted slice.

```
name           old time/op    new time/op    delta
StorageFind-8     157µs ± 2%     112µs ± 1%  -28.60%  (p=0.000 n=10+10)

name           old alloc/op   new alloc/op   delta
StorageFind-8    69.4kB ± 0%    60.5kB ± 0%  -12.90%  (p=0.000 n=9+10)

name           old allocs/op  new allocs/op  delta
StorageFind-8     2.21k ± 0%     2.00k ± 0%   -9.37%  (p=0.000 n=10+7)
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:22:17 +03:00
Evgeniy Stratonikov
b8b272c8c4 core/test: add benchmark for storageFind
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:20:45 +03:00
Evgeniy Stratonikov
0d5ede0bff core/test: use testing.TB in constructors
This allows to use chain in benchmarks.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:20:45 +03:00
Evgeniy Stratonikov
9f377cde12 storage: convert key once in MemoryStore.seek
There is no need in additional allocations.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 14:14:32 +03:00
Evgeniy Stratonikov
8e9302f40b oracle: check response Content-Type
If not specified everything is allowed.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-12 13:13:48 +03:00
Roman Khimov
bc5a1b7bcd
Merge pull request #2043 from nspcc-dev/runtime-getnetwork
interop/runtime: add `System.Runtime.GetNetwork`, fix #2038
2021-07-12 10:46:41 +03:00
Roman Khimov
1853d0c713 core: don't copy config in ApplyPolicyToTxSet
Makes no sense, GetConfig() is for external users.
2021-07-09 13:01:42 +03:00
Evgeniy Stratonikov
96a42cd011 interop: add System.Runtime.GetNetwork, fix #2038
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-08 17:02:37 +03:00
Roman Khimov
7bc7457fbe
Merge pull request #2050 from nspcc-dev/fix-network-test-race
Fix some test failures
2021-07-08 14:50:50 +03:00
Roman Khimov
bd400dfe20 native: copy contract on update
It can be a pointer to cached data:

2021-07-07T20:05:24.8508647Z ==================
2021-07-07T20:05:24.8509294Z WARNING: DATA RACE
2021-07-07T20:05:24.8510243Z Write at 0x00c000142cc8 by goroutine 51:
2021-07-07T20:05:24.8511350Z   github.com/nspcc-dev/neo-go/pkg/core/native.(*Management).Update()
2021-07-07T20:05:24.8529622Z       /home/runner/work/neo-go/neo-go/pkg/core/native/management.go:335 +0x70a
2021-07-07T20:05:24.8531266Z   github.com/nspcc-dev/neo-go/pkg/core/native.(*Management).updateWithData()
2021-07-07T20:05:24.8532544Z       /home/runner/work/neo-go/neo-go/pkg/core/native/management.go:316 +0x1e9
2021-07-07T20:05:24.8533726Z   github.com/nspcc-dev/neo-go/pkg/core/native.(*Management).update()
2021-07-07T20:05:24.8534928Z       /home/runner/work/neo-go/neo-go/pkg/core/native/management.go:303 +0x111
2021-07-07T20:05:24.8536123Z   github.com/nspcc-dev/neo-go/pkg/core/native.(*Management).update-fm()
2021-07-07T20:05:24.8537389Z       /home/runner/work/neo-go/neo-go/pkg/core/native/management.go:302 +0x34
2021-07-07T20:05:24.8538462Z   github.com/nspcc-dev/neo-go/pkg/core/native.Call()
2021-07-07T20:05:24.8539565Z       /home/runner/work/neo-go/neo-go/pkg/core/native/interop.go:54 +0x90c
2021-07-07T20:05:24.8540751Z   github.com/nspcc-dev/neo-go/pkg/core/interop.(*Context).SyscallHandler()
2021-07-07T20:05:24.8542177Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/context.go:262 +0x216
2021-07-07T20:05:24.8543328Z   github.com/nspcc-dev/neo-go/pkg/core/interop.(*Context).SyscallHandler-fm()
2021-07-07T20:05:24.8544586Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/context.go:250 +0x53
2021-07-07T20:05:24.8545648Z   github.com/nspcc-dev/neo-go/pkg/vm.(*VM).execute()
2021-07-07T20:05:24.8546635Z       /home/runner/work/neo-go/neo-go/pkg/vm/vm.go:1312 +0xb2d5
2021-07-07T20:05:24.8547632Z   github.com/nspcc-dev/neo-go/pkg/vm.(*VM).Step()
2021-07-07T20:05:24.8548640Z       /home/runner/work/neo-go/neo-go/pkg/vm/vm.go:416 +0x1f2
2021-07-07T20:05:24.8549836Z   github.com/nspcc-dev/neo-go/pkg/vm.(*VM).Run()
2021-07-07T20:05:24.8550852Z       /home/runner/work/neo-go/neo-go/pkg/vm/vm.go:393 +0x195
2021-07-07T20:05:24.8551989Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).runScriptInVM()
2021-07-07T20:05:24.8553174Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:1326 +0x7eb
2021-07-07T20:05:24.8554345Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).invokeFunction()
2021-07-07T20:05:24.8555560Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:1219 +0x50c
2021-07-07T20:05:24.8556616Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).handleIn()
2021-07-07T20:05:24.8557823Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:345 +0xecb
2021-07-07T20:05:24.8558987Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).handleRequest()
2021-07-07T20:05:24.8560356Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:315 +0x364
2021-07-07T20:05:24.8561599Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).handleHTTPRequest()
2021-07-07T20:05:24.8562976Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:309 +0xf9b
2021-07-07T20:05:24.8564208Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).handleHTTPRequest-fm()
2021-07-07T20:05:24.8565523Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:257 +0x64
2021-07-07T20:05:24.8566348Z   net/http.HandlerFunc.ServeHTTP()
2021-07-07T20:05:24.8567290Z       /opt/hostedtoolcache/go/1.15.13/x64/src/net/http/server.go:2042 +0x51
2021-07-07T20:05:24.8568185Z   net/http.serverHandler.ServeHTTP()
2021-07-07T20:05:24.8569130Z       /opt/hostedtoolcache/go/1.15.13/x64/src/net/http/server.go:2843 +0xca
2021-07-07T20:05:24.8569861Z   net/http.(*conn).serve()
2021-07-07T20:05:24.8570634Z       /opt/hostedtoolcache/go/1.15.13/x64/src/net/http/server.go:1925 +0x84c
2021-07-07T20:05:24.8571139Z
2021-07-07T20:05:24.8571639Z Previous read at 0x00c000142cc8 by goroutine 41:
2021-07-07T20:05:24.8572892Z   github.com/nspcc-dev/neo-go/pkg/core/interop/contract.callExFromNative()
2021-07-07T20:05:24.8574265Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/contract/call.go:101 +0x2d4
2021-07-07T20:05:24.8575581Z   github.com/nspcc-dev/neo-go/pkg/core/interop/contract.callInternal()
2021-07-07T20:05:24.8576951Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/contract/call.go:85 +0x1f5
2021-07-07T20:05:24.8578094Z   github.com/nspcc-dev/neo-go/pkg/core/interop/contract.Call()
2021-07-07T20:05:24.8579306Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/contract/call.go:69 +0x804
2021-07-07T20:05:24.8580514Z   github.com/nspcc-dev/neo-go/pkg/core/interop.(*Context).SyscallHandler()
2021-07-07T20:05:24.8581755Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/context.go:262 +0x216
2021-07-07T20:05:24.8582907Z   github.com/nspcc-dev/neo-go/pkg/core/interop.(*Context).SyscallHandler-fm()
2021-07-07T20:05:24.8584300Z       /home/runner/work/neo-go/neo-go/pkg/core/interop/context.go:250 +0x53
2021-07-07T20:05:24.8585422Z   github.com/nspcc-dev/neo-go/pkg/vm.(*VM).execute()
2021-07-07T20:05:24.8586403Z       /home/runner/work/neo-go/neo-go/pkg/vm/vm.go:1312 +0xb2d5
2021-07-07T20:05:24.8590451Z   github.com/nspcc-dev/neo-go/pkg/vm.(*VM).Step()
2021-07-07T20:05:24.8591549Z       /home/runner/work/neo-go/neo-go/pkg/vm/vm.go:416 +0x1f2
2021-07-07T20:05:24.8592466Z   github.com/nspcc-dev/neo-go/pkg/vm.(*VM).Run()
2021-07-07T20:05:24.8593550Z       /home/runner/work/neo-go/neo-go/pkg/vm/vm.go:393 +0x195
2021-07-07T20:05:24.8594531Z   github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).storeBlock()
2021-07-07T20:05:24.8595708Z       /home/runner/work/neo-go/neo-go/pkg/core/blockchain.go:717 +0x1694
2021-07-07T20:05:24.8596793Z   github.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddBlock()
2021-07-07T20:05:24.8598142Z       /home/runner/work/neo-go/neo-go/pkg/core/blockchain.go:579 +0x544
2021-07-07T20:05:24.8599419Z   github.com/nspcc-dev/neo-go/pkg/consensus.(*service).processBlock()
2021-07-07T20:05:24.8601093Z       /home/runner/work/neo-go/neo-go/pkg/consensus/consensus.go:521 +0x141
2021-07-07T20:05:24.8602485Z   github.com/nspcc-dev/neo-go/pkg/consensus.(*service).processBlock-fm()
2021-07-07T20:05:24.8603809Z       /home/runner/work/neo-go/neo-go/pkg/consensus/consensus.go:517 +0x55
2021-07-07T20:05:24.8604910Z   github.com/nspcc-dev/dbft.(*DBFT).checkCommit()
2021-07-07T20:05:24.8606098Z       /home/runner/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/check.go:73 +0xd8a
2021-07-07T20:05:24.8607235Z   github.com/nspcc-dev/dbft.(*DBFT).checkPrepare()
2021-07-07T20:05:24.8608551Z       /home/runner/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/check.go:36 +0x677
2021-07-07T20:05:24.8609681Z   github.com/nspcc-dev/dbft.(*DBFT).sendPrepareRequest()
2021-07-07T20:05:24.8611016Z       /home/runner/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/send.go:42 +0x47c
2021-07-07T20:05:24.8612107Z   github.com/nspcc-dev/dbft.(*DBFT).OnTimeout()
2021-07-07T20:05:24.8613257Z       /home/runner/go/pkg/mod/github.com/nspcc-dev/dbft@v0.0.0-20210302103605-cc75991b7cfb/dbft.go:188 +0x68c
2021-07-07T20:05:24.8614499Z   github.com/nspcc-dev/neo-go/pkg/consensus.(*service).eventLoop()
2021-07-07T20:05:24.8615712Z       /home/runner/work/neo-go/neo-go/pkg/consensus/consensus.go:274 +0x17ed
2021-07-07T20:05:24.8616284Z
2021-07-07T20:05:24.8616757Z Goroutine 51 (running) created at:
2021-07-07T20:05:24.8617367Z   net/http.(*Server).Serve()
2021-07-07T20:05:24.8618159Z       /opt/hostedtoolcache/go/1.15.13/x64/src/net/http/server.go:2969 +0x5d3
2021-07-07T20:05:24.8619253Z   github.com/nspcc-dev/neo-go/pkg/rpc/server.(*Server).Start.func2()
2021-07-07T20:05:24.8620437Z       /home/runner/work/neo-go/neo-go/pkg/rpc/server/server.go:224 +0x85
2021-07-07T20:05:24.8620936Z
2021-07-07T20:05:24.8621405Z Goroutine 41 (running) created at:
2021-07-07T20:05:24.8622393Z   github.com/nspcc-dev/neo-go/pkg/consensus.(*service).Start()
2021-07-07T20:05:24.8623562Z       /home/runner/work/neo-go/neo-go/pkg/consensus/consensus.go:250 +0x17c
2021-07-07T20:05:24.8624682Z   github.com/nspcc-dev/neo-go/pkg/network.(*Server).tryStartServices()
2021-07-07T20:05:24.8626078Z       /home/runner/work/neo-go/neo-go/pkg/network/server.go:445 +0x2e1
2021-07-07T20:05:24.8627226Z   github.com/nspcc-dev/neo-go/pkg/network.(*Server).Start()
2021-07-07T20:05:24.8628333Z       /home/runner/work/neo-go/neo-go/pkg/network/server.go:268 +0x2af
2021-07-07T20:05:24.8628973Z ==================
2021-07-08 10:40:54 +03:00
Roman Khimov
ea9dde257c
Merge pull request #2047 from nspcc-dev/fix-state
Don't create an extension node for the last branch child
2021-07-08 10:20:59 +03:00
Evgeniy Stratonikov
a33e422b79 mpt: do not create an Extension for the last child of a Branch
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-07 15:49:53 +03:00
Evgeniy Stratonikov
25428d53f1 mpt: strip branch node in a simple implementation
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-07 15:47:07 +03:00
Roman Khimov
0cd9cd0c80 state/result: save/return real JSONization errors
Don't hide/obfuscate real problems.
2021-07-07 00:42:36 +03:00
Roman Khimov
e62a766058 state: move nil check down to stackitem JSON processing
We want to return real errors, not some generic thing for any kind of thing
happening.
2021-07-07 00:38:19 +03:00
Roman Khimov
0de949b575 stackitem: remove Item/StackItem from function names
They're useless in a package named 'stackitem', make this package a bit more
user-friendly.
2021-07-06 19:56:23 +03:00
Evgeniy Stratonikov
9691eee10c mpt: strip branch if 1 child is left
If the child left is a hash node, we should retrieve it from store.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-05 11:04:20 +03:00
Evgeniy Stratonikov
d2e8254848 native: fix typo in comments
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-05 11:04:19 +03:00
Evgeniy Stratonikov
a7722f0fa5 native: save balance value in storage for failed transfers
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-05 11:03:42 +03:00
Roman Khimov
e3323faf69
Merge pull request #2032 from nspcc-dev/optimize-restore
native: cache GAS per vote values
2021-07-01 15:17:37 +03:00
Evgeniy Stratonikov
0591366f85 native: cache GAS per vote values
`storage.Seek()` is rather expensive and we need only last updated value
of gas per block in `PostPersist()`.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-07-01 10:34:04 +03:00
Anna Shaleva
9673a04009 core: allow to add several headers with StateRootInHeader on
Problem: with StateRootInHeader setting on only one header of height N+1
can be added to the chain of height N, because we need local stateroot
to verify headers (which is calculated for the last stored block N).
Thus, adding chunk of headers starting from the current chain's heigh
is impossible and (*Blockchain).AddHeaders doesn't have much sense.

Solution: verify header.PrevStateRoot only for header N+1. Rest of the
headers should be added without PrevStateRoot verification.
2021-06-30 11:56:05 +03:00
Anna Shaleva
d49294b046 core: fix the way candidates are sorted
(*NEO).GetCandidates along with getCandidatesCall use candidates sorted
by serialized ECPoint bytes.

(*NEO).computeCommitteeMembers use candidates sorted by votes, and then
by deserialized ECPoint, i.e. using default ECPoint comparator.
2021-06-16 17:35:04 +03:00
Anna Shaleva
50296975e2 crypto: allow to provide non-default scrypt parameters 2021-06-11 15:22:28 +03:00
Anna Shaleva
6ff64153db core: check stored native states against autogenerated ones on start 2021-06-10 22:16:05 +03:00
Anna Shaleva
be902afe9e core: do not allow NEP17 roundtrip in case of insufficient funds
NEP17 roundtrip is prohibited if from account doesn't have enough funds.

This commit fixes states diff in block 92057 where account
NfuwpaQ1A2xaeVbxWe8FRtaRgaMa8yF3YM initiates two NEO roundtrips with
amount exceeding the account's balance:

block 92057: value mismatch for key +////xTbYWBH3r5qhRKZAPFPHabKfb2vhQ==: QQMhAkwBIQOZZwEA vs QQMhAkwBIQN/ZwEA
block 92057: value mismatch for key +v///ws=: kqlddcitCg== vs tphddcitCg==
block 92057: value mismatch for key +v///xTbYWBH3r5qhRKZAPFPHabKfb2vhQ==: QQEhBUWyDu0W vs QQEhBWmhDu0W

C#'s applog (contains False and False on stack for both transfers):
```
{
   "id" : 1,
   "jsonrpc" : "2.0",
   "result" : {
      "executions" : [
         {
            "gasconsumed" : "11955500",
            "exception" : null,
            "stack" : [
               {
                  "value" : false,
                  "type" : "Boolean"
               },
               {
                  "value" : false,
                  "type" : "Boolean"
               }
            ],
            "vmstate" : "HALT",
            "trigger" : "Application",
            "notifications" : []
         }
      ],
      "txid" : "0x8e73a7e9a566a514813907272ad65fc965002c3b098eacc5bdda529af19d7688"
   }
}
```

Go's applog (both transfers succeeded and GAS minted):
```
{
   "result" : {
      "executions" : [
         {
            "gasconsumed" : "11955500",
            "trigger" : "Application",
            "stack" : [
               {
                  "type" : "Boolean",
                  "value" : true
               },
               {
                  "type" : "Boolean",
                  "value" : true
               }
            ],
            "vmstate" : "HALT",
            "notifications" : [
               {
                  "eventname" : "Transfer",
                  "contract" : "0xd2a4cff31913016155e38e474a2c06d08be276cf",
                  "state" : {
                     "value" : [
                        {
                           "type" : "Any"
                        },
                        {
                           "value" : "22FgR96+aoUSmQDxTx2myn29r4U=",
                           "type" : "ByteString"
                        },
                        {
                           "value" : "4316",
                           "type" : "Integer"
                        }
                     ],
                     "type" : "Array"
                  }
               },
               {
                  "state" : {
                     "type" : "Array",
                     "value" : [
                        {
                           "value" : "22FgR96+aoUSmQDxTx2myn29r4U=",
                           "type" : "ByteString"
                        },
                        {
                           "type" : "ByteString",
                           "value" : "22FgR96+aoUSmQDxTx2myn29r4U="
                        },
                        {
                           "type" : "Integer",
                           "value" : "1111111111"
                        }
                     ]
                  },
                  "contract" : "0xef4073a0f2b305a38ec4050e4d3d28bc40ea63f5",
                  "eventname" : "Transfer"
               },
               {
                  "contract" : "0xef4073a0f2b305a38ec4050e4d3d28bc40ea63f5",
                  "state" : {
                     "type" : "Array",
                     "value" : [
                        {
                           "value" : "22FgR96+aoUSmQDxTx2myn29r4U=",
                           "type" : "ByteString"
                        },
                        {
                           "type" : "ByteString",
                           "value" : "22FgR96+aoUSmQDxTx2myn29r4U="
                        },
                        {
                           "type" : "Integer",
                           "value" : "1111111"
                        }
                     ]
                  },
                  "eventname" : "Transfer"
               }
            ]
         }
      ],
      "txid" : "0x8e73a7e9a566a514813907272ad65fc965002c3b098eacc5bdda529af19d7688"
   },
   "id" : 1,
   "jsonrpc" : "2.0"
}
```
2021-06-09 13:36:50 +03:00
Anna Shaleva
20e0a0d746 core: add price for CALLT instruction 2021-06-07 19:17:22 +03:00
Evgeniy Stratonikov
f3d02b9fef mpt/tests: add compatibility tests 2021-06-04 11:09:17 +03:00
Evgeniy Stratonikov
e833d333fe mpt: disallow empty keys
This is not a problem in practice, as all keys are prefixed
by a contract ID. However in theory it can lead to a different
state root after new portion of changes thus this fix.
2021-06-04 11:09:17 +03:00
Evgeniy Stratonikov
5b1f6207de mpt: reduce max key length
The constant used is actually a nibbled-path restriction.
Clarify this and make them private.
2021-06-04 11:09:17 +03:00
Anna Shaleva
b8e96ac82b core: move mempool.Event to a separate package
And write a marshaller for EventType, it'll be reused by the Notification
subsystem.
2021-06-01 12:24:28 +03:00
Anna Shaleva
999fba81cd core: export (*NEOBalanceState).FromStackItem 2021-05-31 11:18:54 +03:00
Roman Khimov
b858be63b4
Merge pull request #1969 from nspcc-dev/examples/nns-fix
examples: fix IPv6 bounds check
2021-05-28 14:29:54 +03:00
Anna Shaleva
2a9ce03376 examples: fix IPv6 bounds check 2021-05-28 11:31:09 +03:00
Evgeniy Stratonikov
233996eec0 native/neo: add getAccountState method, close #1975 2021-05-27 10:59:38 +03:00
Evgeniy Stratonikov
192771a58a native/std: add base58Check* methods, close #1974 2021-05-25 17:32:19 +03:00
Roman Khimov
e0779f2d6e
Merge pull request #1965 from nspcc-dev/native/remove_nns
native: remove NNS contract
2021-05-17 22:51:58 +03:00
Anna Shaleva
2d20b0fa23 core: remove nnsrecords package from native
Use nns from examples instead.
2021-05-17 22:24:56 +03:00
Anna Shaleva
c9099fa555 core: remove native nonfungible token 2021-05-17 22:22:07 +03:00
Anna Shaleva
99b37efc31 core: remove native NNS 2021-05-17 22:22:07 +03:00
Anna Shaleva
4c4361b2c6 native: move\remove native NNS tests
Record types test is moved to non-native NNS tests and adjusted
according to the non-native implementation behavior.

NEP11-compliance test is removed (non-native NNS is checked against
NEP11 standard during examples compilation test).

Domain name test is removed (we already have the same non-native
NNS tests).
2021-05-17 22:22:03 +03:00
Anna Shaleva
bfda758221 core: replace native NNS with non-native NNS in core tests 2021-05-17 22:12:38 +03:00
Anna Shaleva
7180b2ce06 rpc: use non-native NNS instead of native NNS in testchain 2021-05-17 22:08:15 +03:00
Anna Shaleva
4f1c50373f core: detalize errors during contract.LoadToken and contract.Call 2021-05-17 22:08:15 +03:00
Evgeniy Stratonikov
33e1e61343 config: make MaxValidUntilBlockIncrement configurable 2021-05-17 13:43:03 +03:00
Roman Khimov
f2f9ec1a05
Merge pull request #1916 from nspcc-dev/fix/oraclefilter
oracle: make JSONPath compatible with C# implementation
2021-05-13 15:34:52 +03:00
Evgeniy Stratonikov
1c30d8c395 oracle: make JSONPath compatible with C# implementation
C# node uses simplified implementation which is easy to port.
2021-05-13 11:53:28 +03:00
Roman Khimov
9d2712573f *: enable godot linter and fix all its warnings
It's important for NeoGo to have clean documentation. No functional changes.
2021-05-12 23:17:03 +03:00
Roman Khimov
c4e084b0d8 *: fix whitespace errors
leading/trailing newlines
2021-05-12 22:51:41 +03:00
Roman Khimov
b1710bebaa native: fix errcheck errors in nns contract 2021-05-12 21:49:42 +03:00
Roman Khimov
e9cefc4bfc *: fix all errcheck warnings in testing code 2021-05-12 21:45:12 +03:00
Roman Khimov
4cd3da99b4 core: fix ineffassign warning in test code
ineffectual assignment to err (ineffassign)
2021-05-12 19:37:14 +03:00
Roman Khimov
b5ff87c2bd native: fix minimum deployment fee setting
SA4003: no value of type uint32 is less than 0 (staticcheck)

But it's more involved than that, we should allow any positive big.Int here.
2021-05-12 19:27:02 +03:00
Roman Khimov
333f778aa6 *: drop empty branches
Fix this warning from staticcheck: SA9003: empty branch
2021-05-12 19:10:31 +03:00
Roman Khimov
92dbb3c4b9 *: fix all unused warnings
From golangci-lint.
2021-05-12 18:53:12 +03:00
Roman Khimov
bd48454c72 mempool: simpilfy boolean comparisons in test code
gosimple: S1008: should use 'return t == tx2' instead of 'if t == tx2 { return true }; return false'
2021-05-12 18:40:41 +03:00
Roman Khimov
cf59bd878d native: simplify escaping in regexp.MustCompile()
gosimple: S1007: should use raw string (`...`) with regexp.MustCompile to avoid having to escape twice
2021-05-12 18:38:24 +03:00
Roman Khimov
0178594850 *: simpilfy make() invocations
gosimple: S1019: should use make([]byte, 64) instead
2021-05-12 18:36:45 +03:00
Roman Khimov
cfc067dd24 *: remove dead code
Found by deadcode via golangci-lint.
2021-05-12 18:13:14 +03:00
Roman Khimov
75a55d910e
Merge pull request #1956 from nspcc-dev/interop/checksig-rename
core: rename Neo.Crypto.[CheckSig CheckMultisig] interops
2021-05-12 17:06:39 +03:00
Anna Shaleva
f824789116 core: remove neointerops-related files
And move their content to systeminterops-related files.
2021-05-12 13:30:04 +03:00
Anna Shaleva
4b933f88a7 core: simplify interop functions
We now have the only interop table (system interops).
2021-05-12 13:30:01 +03:00
Roman Khimov
fec214055f
Merge pull request #1948 from nspcc-dev/extensible-dos
Prevent network DoS attack using Extensible payloads
2021-05-12 10:58:51 +03:00
Anna Shaleva
6d59689d9c core: rename Neo.Crypto.CheckMultisig interop 2021-05-11 18:38:14 +03:00
Anna Shaleva
366e79b9b8 core: rename Neo.Crypto.CheckSig interop 2021-05-11 18:37:55 +03:00
Roman Khimov
3a21d8f44f
Merge pull request #1947 from nspcc-dev/iterator-remove
interop: remove `System.Iterator.Create`
2021-05-11 18:29:17 +03:00
Evgeniy Stratonikov
fbf7d9edce core: drop oracles from extensible whitelist 2021-05-11 12:16:24 +03:00
Evgeniy Stratonikov
23a4e25436 interop: remove System.Iterator.Create, fix #1935
There are now only storage iterators. Related #1933.
2021-05-11 12:13:30 +03:00
Evgeniy Stratonikov
40d1dd0e0d native/std: add stringSplit method 2021-05-11 12:11:38 +03:00
Evgeniy Stratonikov
dadfe2b9ab native/std: add memorySearch method 2021-05-11 12:11:38 +03:00
Evgeniy Stratonikov
978f4dfbc5 native/std: add memoryCompare method 2021-05-11 12:11:38 +03:00
Evgeniy Stratonikov
82a6c3266c native/std: limit input size for some methods
Also fix prices accordingly.
2021-05-11 12:11:38 +03:00
Evgeniy Stratonikov
e4b34833da native/std: add overloads for itoa and atoi 2021-05-11 12:11:38 +03:00
Roman Khimov
3402a3ec30 oracle: send user-agent header with our requests, fix #1923 2021-05-07 16:40:45 +03:00
Roman Khimov
3d49f7d99a stateroot: only send stateroot message once per validators count
Actually fix #1922.
2021-05-07 14:31:20 +03:00
Anna Shaleva
d0c64347ab *: add data to NEP11 Transfer 2021-05-05 19:44:29 +03:00
Evgeniy Stratonikov
ac601601c1 native: add refuel method to GAS contract 2021-04-29 18:26:34 +03:00
Evgeniy Stratonikov
0114f9a912 interop: add System.Runtime.BurnGas 2021-04-29 18:23:33 +03:00
Roman Khimov
5924123927
Merge pull request #1939 from nspcc-dev/vm-arith-null
Allow NULL items for LT/LE/GT/GE instructions
2021-04-29 17:55:13 +03:00
Roman Khimov
62810bb510
Merge pull request #1941 from nspcc-dev/storage-find-prefix
interop: strip full prefix in `System.Storage.Find`
2021-04-29 17:55:00 +03:00
Evgeniy Stratonikov
ffaae0f773 interop: strip full prefix in System.Storage.Find 2021-04-29 17:29:26 +03:00
Evgeniy Stratonikov
dc393642a2 opcode: fix GTE, LTE string representations 2021-04-29 16:08:48 +03:00
Evgeniy Stratonikov
8c3998593e native: emit Designation event on role update 2021-04-29 14:12:48 +03:00
Anna Shaleva
0632e15f06 core: add NEP11 to nonfungible supported standards list 2021-04-26 17:56:02 +03:00
Anna Shaleva
f7c65f2b55 core: fix Condition never satisfied test error
This test is failing from time to time.
2021-04-21 11:19:12 +03:00
Roman Khimov
508ad51a86 core: ignore non-numeric "amounts" in Transfer events
Refs. #1900.
2021-04-12 15:09:37 +03:00
Roman Khimov
46d866965d
Merge pull request #1889 from nspcc-dev/tune-logging
Tune logging
2021-04-07 20:58:52 +03:00
Roman Khimov
1785a333d2 core: tune "blockchain persist completed" message
It's misleading, we're writing it on disk-level persistence, it's not the same
as block "persistence" in C#, so we better write it clear.
2021-04-07 18:14:30 +03:00
Roman Khimov
453012ff16 runtime: tune runtime.Log messages
Add transaction hash, print contract hash in LE (as all user-facing functions
do) and don't double-quote the message, zap already does quoting of its own.
2021-04-07 18:14:30 +03:00
Roman Khimov
e2121ff583 core: fix key clashes in Storage.Find
Fixes #1873. NewByteArray() doesn't copy the slice, so we need to do it
ourselves.
2021-04-07 14:52:50 +03:00
Roman Khimov
9b15ab48ad core: restore MPT collapsing logic
Fix #1867, revert part of bf20db09e0.
2021-04-07 11:14:18 +03:00
Roman Khimov
ed358b4c4c native: expose MinimumResponseGas, add specific error for it 2021-04-06 22:50:42 +03:00
Roman Khimov
bba22cb736 transaction: marshal oracle response codes as strings
C# node does it this way.
2021-04-06 22:50:42 +03:00
Roman Khimov
29b79db998 oracle: we only work with https, forget http 2021-04-06 16:53:18 +03:00
Roman Khimov
a01636a1b0 stateroot: set networking callback in a more straightforward way 2021-04-02 12:12:36 +03:00
Anna Shaleva
58ab3b5ad1 core: get native nep17 balance from contract storage
Not from DAO's items with storage.STNEP17Balances prefix, because
changes are reflected there only after notifications processing. And
this happens only after the transaction script is executed, but there
might be cases when we need to get the balance that was updated
earlier during the same transaction processing.

Affects storage dumps.
2021-04-01 18:37:44 +03:00
Roman Khimov
2c8f8bf3ec
Merge pull request #1871 from nspcc-dev/mpt/serialisation-fix
mpt: refactor serialisation
2021-04-01 11:01:38 +03:00
Anna Shaleva
8c358aa556 mpt: change MaxKeyLength for LeafNode 2021-04-01 10:20:16 +03:00
Anna Shaleva
7f038bd465 mpt: split HashNode in two types
First type is non-empty HashNode, and the second one is an Empty node.
2021-04-01 10:19:57 +03:00
Anna Shaleva
b9927c39ee mpt: refactor nodes serialisation
It should be serialised with type in case if it's a children node.
The type can be either HashT or EmptyT.
2021-04-01 10:19:57 +03:00
Anna Shaleva
6e836d325e mpt: increase max leaf value length 2021-04-01 10:17:08 +03:00
Anna Shaleva
f0c222b385 core: move StorageItem-related constants to storage pkg
Need this to avoid import cycle problem.
2021-04-01 10:17:08 +03:00
Anna Shaleva
ec19a087bb core: remove StorageFlag remnant 2021-04-01 10:17:08 +03:00
Anna Shaleva
7dc9f0bde2 mpt: swap Leaf and Node types 2021-04-01 10:17:08 +03:00
Anna Shaleva
93530fa8fa core: pop all args from stack before validation checks in Notify
We should match the C# behaviour.
2021-03-31 19:01:05 +03:00
Anna Shaleva
b7194ec608 core: put candidate account to the storage during (*NEO).vote
C# node uses Snapshot.GetAndChange to get it from storage, so it's the
same problem as with voting account. Affects dumps.
2021-03-29 17:06:37 +03:00
Anna Shaleva
c63528db44 core: put voiting account to the storage during (*NEO).vote
After getting account from the storage there are a lot of checks which
may fail. But C# node uses Snapshot.GetAndChange, so the account state
should be put in the storage even in case of vote failure. Affects
dumps.
2021-03-29 17:04:20 +03:00
Anna Shaleva
3d8d927178 core: check candidate is registered at the start of voiting
C# node returns an error if the candidate is not registered. We have this
check inside the ModifyAccountVotes, but (*NEO).vote doesn't panics and
just return bool on stack, so we should match exactly the C#'s
algorithm.
2021-03-29 17:04:15 +03:00
Roman Khimov
5b4f6d255f
Merge pull request #1857 from nspcc-dev/rpc/nep11
rpc: add NEP11 and NNS interfaces to RPC Client
2021-03-27 18:27:29 +03:00
Roman Khimov
afb4c1e5e2
Merge pull request #1858 from nspcc-dev/rpc/calculatenetworkfee
rpc: add `calculatenetworkfee` RPC method
2021-03-27 15:08:42 +03:00
Roman Khimov
216513e14f stateroot: reject validated root if it doesn't match local one
Prevent

2021-03-27T00:05:23.382+0300    WARN    blockQueue: failed adding block into the blockchain     {"error": "error while trying to apply MPT changes: key not found", "blockHeight": 12757, "nextIndex": 12758}

after node restart (node reads "local" root hash that it doesn't have).
2021-03-27 00:33:11 +03:00
Roman Khimov
d143696328 stateroot: try to fix MPT caching/updating
It was completely ruined by bf20db09e0. MPT was
updating bc.dao directly which shouldn't ever happen, it must write into the
same cache and then Persist these KVs as usual.
2021-03-27 00:33:11 +03:00
Roman Khimov
56fd375c6d core: move stateroot check into header check
As it's a part of the header.
2021-03-27 00:33:11 +03:00
Anna Shaleva
ad9ede455e rpc: add NNS-connected transactions to the test chain
It is needed for RPC client integration tests.
2021-03-26 20:44:32 +03:00
Anna Shaleva
a2a9d7ff05 core: move NNS record types to a separate package
We need this to avoid `native` dependency in RPC client.
2021-03-26 20:44:32 +03:00
Anna Shaleva
252e03bc34 rpc: add CalculateNetworkFee RPC method 2021-03-26 19:14:46 +03:00
Roman Khimov
5f7ec6e784 state: fix MPTRoot JSONization
Match C# implementation. It's easier to do this by changing the structure
itself.
2021-03-26 13:45:18 +03:00
Roman Khimov
b3f9cd1541 dao: drop network from DAO
Not used any more.
2021-03-26 13:45:18 +03:00
Roman Khimov
c789431402 payload: drop Network from P2PNotaryRequest
It's not needed now.
2021-03-26 13:45:18 +03:00
Roman Khimov
8c110a6147 state: drop Network from MPTRoot structure
It's only needed to verify/sign.
2021-03-26 13:45:18 +03:00
Roman Khimov
95c279325a block: drop Network from the Header
It's not network-tied any more, network is only needed to
sign/verify. Unfortunately we still have to keep network in consensus data
structures because of dbft library interface.
2021-03-26 13:45:18 +03:00
Roman Khimov
d314f82db3 transaction: drop Network from Transaction
We only need it when signing/verifying.
2021-03-26 13:45:18 +03:00
Roman Khimov
df12adaa9e crypto: remove crypto.Verifiable interface
We can now verify any hash.Hashable thing.
2021-03-26 13:45:18 +03:00
Roman Khimov
0248e2c0d2 state: add network to MPTRoot, change hashing
Fix RC1 incompatibility.
2021-03-25 22:37:30 +03:00
Anna Shaleva
1e649bc9a0 core: move NotaryVerificationPrice to a separate package
It is needed to avoid `native` dependency in RPC client.
2021-03-23 13:56:39 +03:00
Anna Shaleva
681bd89cc7 core: move native node roles to a separate pkg
We need this to avoid `native` dependency in the RPC client.
2021-03-23 13:56:34 +03:00
Roman Khimov
28da00f057
Merge pull request #1852 from nspcc-dev/tests/fix-rubles-contract
rpc: refactor Rubl test contract
2021-03-22 15:55:51 +03:00
Anna Shaleva
6b36590181 core: ensure all transactions are HALT while creating test chain 2021-03-22 15:35:22 +03:00
Anna Shaleva
0a5072a1da core: allow to compile test contracts with yaml config
And refactored Rubl test contract (it should support NEP-17 and
onNEP17Payment).
2021-03-22 15:32:26 +03:00
Roman Khimov
20144e6137 *: fix some misspellings spotted by GoReportCard 2021-03-22 12:13:08 +03:00
Anna Shaleva
96e198de97 core: fix Management.GetContract 2021-03-19 20:44:21 +03:00
Roman Khimov
57d4990fe6
Merge pull request #1850 from nspcc-dev/fex-storage-price
core: fix storage.Put price
2021-03-19 19:10:22 +03:00
Anna Shaleva
7118eae467 core: remove useless code from putWithContext 2021-03-19 18:57:29 +03:00
Anna Shaleva
5b18f5974f core: fix storage.Delete price 2021-03-19 18:57:00 +03:00
Anna Shaleva
0bca63b907 core: fix storage.Put price 2021-03-19 18:41:33 +03:00
Roman Khimov
b56e028733 *: add more package-specific documentation
For the most important packages at least.
2021-03-19 16:18:45 +03:00
Anna Shaleva
e5cdecfa9f core: fix transaction hashes 2021-03-18 17:57:54 +03:00
Roman Khimov
42465dd002
Merge pull request #1825 from nspcc-dev/fix-invokecontractverify
rpc: refactor invokecontractverify
2021-03-17 18:08:23 +03:00
Roman Khimov
0de72f1dc9 core: fix call flags for Native(On|Post)Persist
See neo-project/neo#2339.
2021-03-16 22:49:14 +03:00
Roman Khimov
d1251b8daf core: don't limit on/postPersist methods
They read/write/call/notify, so they need everything.
2021-03-16 22:48:32 +03:00
Roman Khimov
304b9ad3d4 contract: disable notifications for safe methods
See neo-project/neo#2339.
2021-03-16 22:47:49 +03:00
Roman Khimov
025330f132
Merge pull request #1832 from nspcc-dev/consensus/microfixes
consensus: add MaxBlockSize and MaxBlockSysFee settings
2021-03-16 18:11:33 +03:00
Anna Shaleva
0d9266d407 core: remove System.Contract.IsStandard 2021-03-16 16:59:23 +03:00
Roman Khimov
a18fbc7bb1
Merge pull request #1827 from nspcc-dev/native/update_history
config: add NativeUpdateHistory
2021-03-16 12:47:22 +03:00