Commit graph

2070 commits

Author SHA1 Message Date
Anna Shaleva
0b0591fc34 network: allow to use -1 to specify GetBlockByIndex.Count 2020-08-04 17:52:34 +03:00
Anna Shaleva
0b856033b0 network: use MaxHeadersAllowed to restrict GetBlockByIndex 2020-08-04 17:52:34 +03:00
Anna Shaleva
737ba700e9 network: rename GetBlockData command
GetBlockData -> GetBlockByIndex
2020-08-04 17:52:34 +03:00
Anna Shaleva
7c6cdcbcc9 core: fix bug with policy.MaxVerificationGas
In 121c9664b we should take into account isValid flag of
NativePolicy contract while retrieving MaxVerificationGas native
policy value. Otherwise we won't be able to get MaxVerificationGas
after the node was restarted, because this value is not truly
stored along with the other native policy values.

This commit fixes bug with headers verification after the  node
restarting with an existing storage:

```
2020-08-03T12:52:56.158+0300	WARN	failed processing headers	{"error": "vm failed to execute the script with error: error encountered at instruction 0 (PUSHDATA1): gas limit is exceeded", "errorVerbose": "vm failed to execute the script with error: error encountered at instruction 0 (PUSHDATA1): gas limit is exceeded\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHashAgainstScript\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1454\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeaderWitnesses\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1517\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeader\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1175\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).addHeaders\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:484\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddHeaders\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:453\ngithub.com/nspcc-dev/neo-go/pkg/network.(*Server).handleHeadersCmd\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/network/server.go:454\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373"}
```
2020-08-04 17:52:34 +03:00
Anna Shaleva
7865bc5925 core: fix native policy tests
That was a merge problem.
2020-08-04 15:05:31 +03:00
Roman Khimov
62f5aa8eb4
Merge pull request #1254 from nspcc-dev/native/policy/maxblocksystemfee
core, consensus: add maxBlockSystemFee to native Policy
2020-08-04 13:13:06 +03:00
Roman Khimov
fa5ff8dd95
Merge pull request #1242 from nspcc-dev/fix/json
Implement lossless stackitem to json conversion
2020-08-04 12:59:47 +03:00
Anna Shaleva
db65ed04d9 consensus: apply policy during verifyBlock
To follow C# implementation we should also check proposed block on
policy matching.
2020-08-04 10:59:09 +03:00
Anna Shaleva
4a1c8464f9 core: use maxBlockSize to apply policy to tx set
We should also take into account maxBlockSize in
(bc *Blockchain).ApplyPolicyToTxSet
2020-08-04 10:59:09 +03:00
Anna Shaleva
c647f8e4ed core: add maxBlockSystemFee to native Policy
Closes #1195
2020-08-04 10:59:00 +03:00
Roman Khimov
3e192b11b2
Merge pull request #1255 from nspcc-dev/p2p-handle-decompression-length-mismatch
network: handle length mismatch in decompression routine
2020-08-04 10:24:57 +03:00
Roman Khimov
1f46f73d12 network: handle length mismatch in decompression routine
It's a protocol level error when uncompressed payload size doesn't match the
one specified in the header.
2020-08-03 22:38:55 +03:00
Roman Khimov
00671deb8f
Merge pull request #1241 from nspcc-dev/fix/string
Ensure strings are valid UTF-8 where appropriate
2020-08-03 18:10:27 +03:00
Anna Shaleva
13336f9dba core: adjust calling flags of native Policy methods 2020-08-03 16:09:10 +03:00
Anna Shaleva
60f9abae40 core: chenge maxBlockSizeKey of native Policy 2020-08-03 16:00:15 +03:00
Anna Shaleva
b2aef6acbf smartcontract: calculate double-SHA256 for .nef files
Closes #1203
2020-08-03 14:25:27 +03:00
Evgenii Stratonikov
54590062e6 vm: use raw stack items when printing stack
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
e5d538ed21 core: use raw stack items in application logs
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
9cba25616d rpc: use raw stack items in invoke* RPC
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
7b4acd5a7f vm: allow to convert stack to a slice
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
805f746f15 rpc: change NotificationEvent format
Name is now a separate field.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
9d6ce4bdcc stackitem: implement unmarshaling from JSON with types
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
ee5ff40b17 stackitem: allow to convert type from string
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
c16bb466a0 stackitem: implement marshaling to JSON with types
There are 2 kinds of JSON marshaling:
1. Lossy raw marshaling, when type information is lost and
    map keys are expected to be valid utf-8 strings.
2. Almost lossless marshaling, which can handle any non-recursive item.
    Interop value preserves only type.

This commit implements the second.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
b53f0257f5 stackitem: change ByteArray type to ByteString
Adjust only string representation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
a781d299e0 compiler: use fully-qualified names for tracking functions
Function name now consists of 3 parts:
1) full package path
2) method receiver type (if any)
3) function name itself .

Fix #1150.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:07:06 +03:00
Evgenii Stratonikov
528c184f00 compiler: allow to use exported constants
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
6df019913d compiler: allow to use exported variables
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
b8d7e93459 compiler: make ForEachFile accept a package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
ac040a6f22 compiler: remove unused resolveEntryPoint
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
7009417325 compiler: allow to declare global variables in multiple files
Traverse and count globals across all used files.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
fyrchik
babd84ec10
Merge pull request #1243 from nspcc-dev/protocol/uncompressed_payload_size
network: add uncompressed payload size
2020-07-31 10:03:55 +03:00
fyrchik
e4fc655115
Merge pull request #1239 from nspcc-dev/store_nep5_with_id
core: store contract IDs instead of hashes for NEP5Balances and Transfers
2020-07-30 12:56:19 +03:00
Anna Shaleva
2574b65b74 network: add uncompressed payload size
Closes #1212
2020-07-30 12:47:42 +03:00
Roman Khimov
e21c65c59f core: add state height to prometheus metrics
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
5ee3ecf381 core: update verified state root height
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
ab802cdd5f state: implement JSON marshaling for MPT* items
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
236438d799 core: do MPT compaction every once in a while
We need to compact our in-memory MPT from time to time, otherwise it quickly
fills up all available memory. This raises two obvious quesions --- when to do
that and to what level do that.

As for 'when', I think it's quite easy to use our regular persistence interval
as an anchor (and it also frees up some memory), but we can't do that in the
persistence routine itself because of synchronization issues (adding some
synchronization primitives would add some cost that I'd also like to avoid),
so do it indirectly by comparing persisted and current height in `storeBlock`.

Choosing proper level is another problem, but if we're to roughly estimate one
full branch node to use 1K of memory (usually it's way less than that) then we
can easily store 1K of these nodes and that gives us a depth of 10 for our
trie.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
58b7e16e0e core: fix PrevHash calculation for MPTRoot
This was differing from C# notion of PrevHash. It's not a previous root, but
rather a hash of the previous serialized MPTRoot structure (that is to be
signed by CNs).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
caea6d6ca8 mpt: fix extension node cache invalidation
It should always be invalidated if something changes in the `next` (below the
extension node).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
0e29382035 core: update MPT during block processing
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
2b53877dff mpt: don't flush nodes already present in the DB
It's just a waste of time.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
475bf2445a mpt: restructure nodes a bit, implement serialization and hash cache
It drastically reduces the number of allocations and hash calculations.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
6ca22027d5 mpt: implement (*Trie).Collapse()
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
f0b85f8af7 mpt: implement JSON marshaling/unmarshaling
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
9b328240dd mpt: implement MPT proof Get and Verify
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
dc6741bce7 mpt: implement MPT trie
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
5500195d58 stackitem: ensure map keys are a valid UTF-8 2020-07-30 12:37:32 +03:00
Evgenii Stratonikov
3c99393bef core: simplify System.Contract.Call* parameter handling 2020-07-30 12:37:32 +03:00
Evgenii Stratonikov
f40aba4cd0 vm: convert items to UTF-8 strings
Add `stackitem.ToString` for seamless string conversion.
2020-07-30 12:37:31 +03:00