Commit graph

2163 commits

Author SHA1 Message Date
Roman Khimov
5ef08f60ae remove github.com/pkg/errors from dependencies
It's not needed any more with Go 1.13 as we have wrapping/unwrapping in base
packages. All errors.Wrap calls are replaced with fmt.Errorf, some strings are
improved along the way.
2020-08-07 12:21:52 +03:00
Evgenii Stratonikov
bfda60c683 native: add missing witness checks to NEO.(Un)registerCandidate 2020-08-07 09:17:02 +03:00
Evgenii Stratonikov
8af3f05358 native: implement NEO.UnregisterCandidate method 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
38a92323c9 native: fill votes when registering unregistered candidate
If a candidate was registered, then unregistered and then again
registered, it's votes must not be lost.
2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
27169d140f core: implement (*Blockchain).GetStandByCommitee() 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
05c24d9401 cli: support voting
Closes #1206.
2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
a3f419f8df emit: allow to emit Null 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
e5d973d3a4 config: rename StandbyValidators to StandbyCommittee 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
b14b047c78 native: add tests for delegated voting 2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
9bc731b3b1 native: implement delegated voting
Close #867.
2020-08-06 20:39:13 +03:00
Evgenii Stratonikov
f0b62cdaa6 native: make *totalSupply accept DAO
There is no need to provide full interop context.
2020-08-06 14:26:24 +03:00
Evgenii Stratonikov
3e39f0f211 native: rename Validator to Candidate 2020-08-06 14:26:24 +03:00
Roman Khimov
e1d3223505
Merge pull request #1247 from nspcc-dev/feature/pointers
Support pointers in compiler
2020-08-06 13:53:08 +03:00
Roman Khimov
5e53aceb0f
Merge pull request #1277 from nspcc-dev/restrict-verification-call-flags
core: restrict verification scripts with CallFlags.None
2020-08-06 13:52:14 +03:00
Roman Khimov
57ee8b80e5
Merge pull request #1246 from nspcc-dev/protocol/getblockbyindex
protocol: request blocks by index
2020-08-05 23:21:07 +03:00
Roman Khimov
f5fbef165a
Merge pull request #1259 from nspcc-dev/fix/notifications
Disallow changing notifications
2020-08-05 19:50:45 +03:00
Roman Khimov
b7f89b0544 core: restrict verification scripts with CallFlags.None
Follow neo-project/neo#1776. Looks very suspicious to me as it severely limits
verification scripts, but we have to be compatible at this point.
2020-08-05 19:17:48 +03:00
Roman Khimov
fced917b71
Merge pull request #1261 from nspcc-dev/feature/init
compiler: allow to use `init` function
2020-08-05 19:04:51 +03:00
Evgenii Stratonikov
f2107bfbc4 compiler: copy structs when passing as arguments
In Go structs must be copied when used as arguments.
To do so we must clone struct on VM level.
This is done by appending this struct to an intermediate array.
2020-08-05 13:14:38 +03:00
Evgenii Stratonikov
4488f61777 compiler: compile init even if there are no globals
`init` can be useful even if no globals are present,
e.g. we can use some syscall inside.
2020-08-05 12:59:53 +03:00
Evgenii Stratonikov
194da64975 core: deepcopy notification in System.Runtime.Notify
Make it impossible to change already emitted notifications via
`System.Runtime.GetNotifications`.
2020-08-05 11:19:41 +03:00
Evgenii Stratonikov
b8cc33d0b2 stackitem: implement DeepCopy 2020-08-05 11:19:41 +03:00
Evgenii Stratonikov
439d9ff94d compiler: initialize packages according to go spec
`init()` functions should be called during package initialization,
after global variables were processed.
2020-08-05 11:15:07 +03:00
Evgenii Stratonikov
6f2759be3a compiler: process packages in deterministic order 2020-08-05 11:00:25 +03:00
Evgenii Stratonikov
b771d2d024 compiler: allow to use init function
Process `init()` functions after global variables has been processed.
It's body is saved into the `_initialize` method where all
initialization is performed.
2020-08-05 11:00:25 +03:00
Roman Khimov
4bbe863904
Merge pull request #1266 from nspcc-dev/notifications/filter_by_name
rpc: filter subscriptions' notifications by name
2020-08-05 10:00:15 +03:00
Evgenii Stratonikov
7e3813a88f cli: use TAB as delimiter in vm parse
Make it simpler to use result in pipes:

neo-go vm parse 123 | rg "Integer to Hex" | cut -f1- | sed -r 's/\t//g'
2020-08-05 09:40:50 +03:00
Evgenii Stratonikov
227c6a2caa cli: allow to use vm parse directly 2020-08-05 09:40:50 +03:00
Roman Khimov
ef53a45e7a
Merge pull request #1264 from nspcc-dev/smartcontract/manifest/supported_standards
smartcontract: add list of supported standards to manifest
2020-08-04 22:17:00 +03:00
Roman Khimov
49688de75c
Merge pull request #1260 from nspcc-dev/feature/sendmany
cli: allow to transfer multiple tokens in `multitransfer`
2020-08-04 19:35:03 +03:00
Anna Shaleva
6c8accf18c core, network: request blocks instead of headers
Closes #1192

1. We now have CMDGetBlockByIndex, so there's no need to request headers
   first when we can just ask for blocks.
2. We don't ask for headers (i.e. we don't send CMDGetHeaders),
   consequently, we shouldn't react on CMDHeaders.
3. But we still keep on reacting on CMDGetHeaders command as
   there could be a node which needs headers.
2020-08-04 17:52:34 +03:00
Anna Shaleva
f6f3863e0e network: allow to GetHeaders by index
Use GetBlockByIndex payload for GetHeaders command instead of GetBlocks
payload.
2020-08-04 17:52:34 +03:00
Anna Shaleva
7b1c305000 network: fix handleGetBlockByIndexCmd method
It returned an error in case if block wasn't found (it might be when our
chain is lower). Fixed. It also should return all requested blocks, not
the first one.
2020-08-04 17:52:34 +03:00
Anna Shaleva
0b0591fc34 network: allow to use -1 to specify GetBlockByIndex.Count 2020-08-04 17:52:34 +03:00
Anna Shaleva
0b856033b0 network: use MaxHeadersAllowed to restrict GetBlockByIndex 2020-08-04 17:52:34 +03:00
Anna Shaleva
737ba700e9 network: rename GetBlockData command
GetBlockData -> GetBlockByIndex
2020-08-04 17:52:34 +03:00
Anna Shaleva
7c6cdcbcc9 core: fix bug with policy.MaxVerificationGas
In 121c9664b we should take into account isValid flag of
NativePolicy contract while retrieving MaxVerificationGas native
policy value. Otherwise we won't be able to get MaxVerificationGas
after the node was restarted, because this value is not truly
stored along with the other native policy values.

This commit fixes bug with headers verification after the  node
restarting with an existing storage:

```
2020-08-03T12:52:56.158+0300	WARN	failed processing headers	{"error": "vm failed to execute the script with error: error encountered at instruction 0 (PUSHDATA1): gas limit is exceeded", "errorVerbose": "vm failed to execute the script with error: error encountered at instruction 0 (PUSHDATA1): gas limit is exceeded\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHashAgainstScript\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1454\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeaderWitnesses\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1517\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).verifyHeader\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:1175\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).addHeaders\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:484\ngithub.com/nspcc-dev/neo-go/pkg/core.(*Blockchain).AddHeaders\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/core/blockchain.go:453\ngithub.com/nspcc-dev/neo-go/pkg/network.(*Server).handleHeadersCmd\n\t/home/neospcc/Documents/GitProjects/nspcc-dev/neo-go/pkg/network/server.go:454\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373"}
```
2020-08-04 17:52:34 +03:00
Anna Shaleva
ba08a9b6ad core: move all transaction.Decode checks to separate method
We should perform the same checks during UnmarshalJSON.
2020-08-04 17:34:06 +03:00
Anna Shaleva
ead6bdb25e core: remove unnecessary comment 2020-08-04 17:34:06 +03:00
Anna Shaleva
90825efa16 core: move transaction's sender to cosigners
Closes #1184

Ported changes from https://github.com/neo-project/neo/pull/1752
2020-08-04 17:33:50 +03:00
Anna Shaleva
66ceaa6b75 smartcontract: add list of supported standards to manifest
Closes #1204
2020-08-04 17:29:44 +03:00
Anna Shaleva
4ff3a9e9a7 rpc: filter subscriptions' notifications by name
Closes #1263
2020-08-04 16:29:13 +03:00
Anna Shaleva
8697582b23 core: add FeeOnly witness scope 2020-08-04 15:08:59 +03:00
Anna Shaleva
75f1c2d6f2 core: add scopes check to cosigners in DecodeBinary 2020-08-04 15:08:59 +03:00
Anna Shaleva
7865bc5925 core: fix native policy tests
That was a merge problem.
2020-08-04 15:05:31 +03:00
Roman Khimov
62f5aa8eb4
Merge pull request #1254 from nspcc-dev/native/policy/maxblocksystemfee
core, consensus: add maxBlockSystemFee to native Policy
2020-08-04 13:13:06 +03:00
Roman Khimov
fa5ff8dd95
Merge pull request #1242 from nspcc-dev/fix/json
Implement lossless stackitem to json conversion
2020-08-04 12:59:47 +03:00
Evgenii Stratonikov
d54c60ded3 compiler: support pointer dereferencing for structs
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-08-04 11:14:07 +03:00
Evgenii Stratonikov
eb047b12a7 compiler: support creating pointers to struct
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-08-04 11:14:07 +03:00
Anna Shaleva
db65ed04d9 consensus: apply policy during verifyBlock
To follow C# implementation we should also check proposed block on
policy matching.
2020-08-04 10:59:09 +03:00
Anna Shaleva
4a1c8464f9 core: use maxBlockSize to apply policy to tx set
We should also take into account maxBlockSize in
(bc *Blockchain).ApplyPolicyToTxSet
2020-08-04 10:59:09 +03:00
Anna Shaleva
c647f8e4ed core: add maxBlockSystemFee to native Policy
Closes #1195
2020-08-04 10:59:00 +03:00
Evgenii Stratonikov
2dd6ef964e cli: allow to transfer multiple tokens in multitransfer
Like this:
./neogo wallet nep5 multitransfer -w wallet1_solo.json
  --from NVNvVRW5Q5naSx2k2iZm7xRgtRNGuZppAK -r http://127.0.0.1:30333
    neo:NYqxsNMHxDg3T19APYP27mBZFfauC4zngR:2
    neo:NVNvVRW5Q5naSx2k2iZm7xRgtRNGuZppAK:3
    gas:NYqxsNMHxDg3T19APYP27mBZFfauC4zngR:2
2020-08-04 10:36:06 +03:00
Roman Khimov
3e192b11b2
Merge pull request #1255 from nspcc-dev/p2p-handle-decompression-length-mismatch
network: handle length mismatch in decompression routine
2020-08-04 10:24:57 +03:00
Roman Khimov
1f46f73d12 network: handle length mismatch in decompression routine
It's a protocol level error when uncompressed payload size doesn't match the
one specified in the header.
2020-08-03 22:38:55 +03:00
Roman Khimov
00671deb8f
Merge pull request #1241 from nspcc-dev/fix/string
Ensure strings are valid UTF-8 where appropriate
2020-08-03 18:10:27 +03:00
Anna Shaleva
13336f9dba core: adjust calling flags of native Policy methods 2020-08-03 16:09:10 +03:00
Anna Shaleva
60f9abae40 core: chenge maxBlockSizeKey of native Policy 2020-08-03 16:00:15 +03:00
Anna Shaleva
b2aef6acbf smartcontract: calculate double-SHA256 for .nef files
Closes #1203
2020-08-03 14:25:27 +03:00
Evgenii Stratonikov
54590062e6 vm: use raw stack items when printing stack
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
e5d538ed21 core: use raw stack items in application logs
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
9cba25616d rpc: use raw stack items in invoke* RPC
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
7b4acd5a7f vm: allow to convert stack to a slice
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
805f746f15 rpc: change NotificationEvent format
Name is now a separate field.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
9d6ce4bdcc stackitem: implement unmarshaling from JSON with types
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
ee5ff40b17 stackitem: allow to convert type from string
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
c16bb466a0 stackitem: implement marshaling to JSON with types
There are 2 kinds of JSON marshaling:
1. Lossy raw marshaling, when type information is lost and
    map keys are expected to be valid utf-8 strings.
2. Almost lossless marshaling, which can handle any non-recursive item.
    Interop value preserves only type.

This commit implements the second.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
b53f0257f5 stackitem: change ByteArray type to ByteString
Adjust only string representation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 16:07:16 +03:00
Evgenii Stratonikov
a781d299e0 compiler: use fully-qualified names for tracking functions
Function name now consists of 3 parts:
1) full package path
2) method receiver type (if any)
3) function name itself .

Fix #1150.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:07:06 +03:00
Evgenii Stratonikov
528c184f00 compiler: allow to use exported constants
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
6df019913d compiler: allow to use exported variables
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
b8d7e93459 compiler: make ForEachFile accept a package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
ac040a6f22 compiler: remove unused resolveEntryPoint
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
Evgenii Stratonikov
7009417325 compiler: allow to declare global variables in multiple files
Traverse and count globals across all used files.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-31 12:06:42 +03:00
fyrchik
babd84ec10
Merge pull request #1243 from nspcc-dev/protocol/uncompressed_payload_size
network: add uncompressed payload size
2020-07-31 10:03:55 +03:00
fyrchik
e4fc655115
Merge pull request #1239 from nspcc-dev/store_nep5_with_id
core: store contract IDs instead of hashes for NEP5Balances and Transfers
2020-07-30 12:56:19 +03:00
Anna Shaleva
2574b65b74 network: add uncompressed payload size
Closes #1212
2020-07-30 12:47:42 +03:00
Roman Khimov
e21c65c59f core: add state height to prometheus metrics
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
5ee3ecf381 core: update verified state root height
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
ab802cdd5f state: implement JSON marshaling for MPT* items
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
236438d799 core: do MPT compaction every once in a while
We need to compact our in-memory MPT from time to time, otherwise it quickly
fills up all available memory. This raises two obvious quesions --- when to do
that and to what level do that.

As for 'when', I think it's quite easy to use our regular persistence interval
as an anchor (and it also frees up some memory), but we can't do that in the
persistence routine itself because of synchronization issues (adding some
synchronization primitives would add some cost that I'd also like to avoid),
so do it indirectly by comparing persisted and current height in `storeBlock`.

Choosing proper level is another problem, but if we're to roughly estimate one
full branch node to use 1K of memory (usually it's way less than that) then we
can easily store 1K of these nodes and that gives us a depth of 10 for our
trie.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
58b7e16e0e core: fix PrevHash calculation for MPTRoot
This was differing from C# notion of PrevHash. It's not a previous root, but
rather a hash of the previous serialized MPTRoot structure (that is to be
signed by CNs).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
caea6d6ca8 mpt: fix extension node cache invalidation
It should always be invalidated if something changes in the `next` (below the
extension node).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
0e29382035 core: update MPT during block processing
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
2b53877dff mpt: don't flush nodes already present in the DB
It's just a waste of time.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Roman Khimov
475bf2445a mpt: restructure nodes a bit, implement serialization and hash cache
It drastically reduces the number of allocations and hash calculations.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
6ca22027d5 mpt: implement (*Trie).Collapse()
Because trie size is rather big, it can't be stored in memory.
Thus some form of caching should also be implemented. To avoid
marshaling/unmarshaling of items which are close to root and are used
very frequenly we can save them across the persists.
This commit implements pruning items at the specified depth,
replacing them by hash nodes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
f0b85f8af7 mpt: implement JSON marshaling/unmarshaling
Because there is no distinct type field in JSONized nodes, distinction
is made via payload itself, thus all unmarshaling is done via
NodeObject.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
9b328240dd mpt: implement MPT proof Get and Verify
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
dc6741bce7 mpt: implement MPT trie
MPT is a trie with a branching factor = 16, i.e. it consists of sequences in
16-element alphabet.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-30 12:42:15 +03:00
Evgenii Stratonikov
5500195d58 stackitem: ensure map keys are a valid UTF-8 2020-07-30 12:37:32 +03:00
Evgenii Stratonikov
3c99393bef core: simplify System.Contract.Call* parameter handling 2020-07-30 12:37:32 +03:00
Evgenii Stratonikov
f40aba4cd0 vm: convert items to UTF-8 strings
Add `stackitem.ToString` for seamless string conversion.
2020-07-30 12:37:31 +03:00
Anna Shaleva
167bd7d9df rpc: store decimals by contract ID instead of hash
And we should also cache contract scripthash to fill Asset field.
2020-07-29 15:15:48 +03:00
Anna Shaleva
0dd1730632 core: store NEP5Transfer asset by ID instead of hash
To avoid problems with retrieving decimals for migrated contracts.
2020-07-29 15:15:48 +03:00
Anna Shaleva
b9bdab8ec8 core: store nep5balances using contract id
Closes #1194
2020-07-29 15:15:48 +03:00
Anna Shaleva
1f5794b316 rpc: make getDecimals return a standard error
`getDecimals` is an internal method, wo there's no need to return
response.Error.
2020-07-29 15:14:24 +03:00
Anna Shaleva
dbd460d883 core: retrieve contract hash by ID
We'll need this ability further to retrieve contracts hashes for Nep5Balances.
2020-07-29 15:14:24 +03:00
Evgenii Stratonikov
9a992865b1 interop: add Secp2k to the list of crypto interops
Allow to use them during verification.
2020-07-29 13:43:33 +03:00
Evgenii Stratonikov
23a1430395 core: allow to restrict creating callbacks from syscalls
Specify DisallowCallback flag if syscall is not allowed to be used in a
callback.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-07-29 13:41:08 +03:00