TrueCloudLab/neo-go
8
0
Fork 0
mirror of https://github.com/nspcc-dev/neo-go.git synced 2024-11-26 19:42:23 +00:00
Code Issues Projects Releases Packages Wiki Activity
6702 commits 39 branches 136 tags 43 MiB
Commit graph

234 commits

Author SHA1 Message Date
Anna Shaleva
75d7891ca1 services: add ability to combine notary signers 
Notes for witnesses:
* [N sig + M multisig + K contract] combination is possible where N, M, K >=0.
* Each verification script should be properly filled in.
* Each invocation script should either be empty or contain exactly one
signature.
 2021-10-25 12:22:13 +03:00
Anna Shaleva
fcc7f7349d services: refactor the logic of notary signatures counting 
Maintain the number of signatures left to collect instead of maintaining
the number of already collected signatures and overall number of
signatures.
 2021-10-22 17:06:46 +03:00
Anna Shaleva
07cbe4d253 core: add finalizer functions to interop context 
These functions are aimed to free the resources occupied by storage
iterator by the end of script execution or whenever Finilize is called.
 2021-10-21 10:05:12 +03:00
Anna Shaleva
dfc0b25cfe gomod: use nspcc-dev's fork of go-ordered-json 
Escape non-ASCII characters while JSON encoding.
 2021-09-15 15:01:01 +03:00
Evgeniy Stratonikov
e2f2addf95 notary: fix possible deadlock in UpdateNotaryNodes 
`UpdateNotaryNodes` takes account then request mutex, and `PostPersist` takes
them in a different order. Because they are executed concurrently a deadlock
can appear.

```
2021-07-23T11:06:58.3732405Z panic: test timed out after 10m0s
2021-07-23T11:06:58.3732642Z
2021-07-23T11:06:58.3742610Z goroutine 7351 [semacquire, 9 minutes]:
2021-07-23T11:06:58.3743140Z sync.runtime_SemacquireMutex(0xc00010e4dc, 0x1100000000, 0x1)
2021-07-23T11:06:58.3743747Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/runtime/sema.go:71 +0x47
2021-07-23T11:06:58.3744222Z sync.(*Mutex).lockSlow(0xc00010e4d8)
2021-07-23T11:06:58.3744742Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/sync/mutex.go:138 +0x1c1
2021-07-23T11:06:58.3745209Z sync.(*Mutex).Lock(0xc00010e4d8)
2021-07-23T11:06:58.3745692Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/sync/mutex.go:81 +0x7d
2021-07-23T11:06:58.3746162Z sync.(*RWMutex).Lock(0xc00010e4d8)
2021-07-23T11:06:58.3746764Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/sync/rwmutex.go:98 +0x4a
2021-07-23T11:06:58.3747699Z github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).UpdateNotaryNodes(0xc00010e480, 0xc000105b90, 0x1, 0x1)
2021-07-23T11:06:58.3748621Z 	/home/runner/work/neo-go/neo-go/pkg/services/notary/node.go:44 +0x3ba
2021-07-23T11:06:58.3749367Z github.com/nspcc-dev/neo-go/pkg/core.TestNotary(0xc0003677a0)
2021-07-23T11:06:58.3750116Z 	/home/runner/work/neo-go/neo-go/pkg/core/notary_test.go:594 +0x2dba
2021-07-23T11:06:58.3750641Z testing.tRunner(0xc0003677a0, 0x16f3308)
2021-07-23T11:06:58.3751202Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/testing/testing.go:1050 +0x1ec
2021-07-23T11:06:58.3751696Z created by testing.(*T).Run
2021-07-23T11:06:58.3752225Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/testing/testing.go:1095 +0x538
2021-07-23T11:06:58.3752573Z
2021-07-23T11:06:58.3771319Z goroutine 7340 [semacquire, 9 minutes]:
2021-07-23T11:06:58.3772048Z sync.runtime_SemacquireMutex(0xc00010e504, 0x0, 0x0)
2021-07-23T11:06:58.3772889Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/runtime/sema.go:71 +0x47
2021-07-23T11:06:58.3773581Z sync.(*RWMutex).RLock(0xc00010e4f8)
2021-07-23T11:06:58.3774310Z 	/opt/hostedtoolcache/go/1.14.15/x64/src/sync/rwmutex.go:50 +0xa4
2021-07-23T11:06:58.3775449Z github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).getAccount(0xc00010e480, 0x0)
2021-07-23T11:06:58.3776626Z 	/home/runner/work/neo-go/neo-go/pkg/services/notary/node.go:51 +0x51
2021-07-23T11:06:58.3778270Z github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).finalize(0xc00010e480, 0xc0003b2630, 0xa97df1bc78dd5787, 0xcc8a4d69e7f5d62a, 0x1a4d7981bd86b087, 0xbafdb720c93480b3, 0x0, 0x0)
2021-07-23T11:06:58.3779845Z 	/home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:306 +0x54
2021-07-23T11:06:58.3781022Z github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).PostPersist(0xc00010e480)
2021-07-23T11:06:58.3782232Z 	/home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:297 +0x662
2021-07-23T11:06:58.3782989Z github.com/nspcc-dev/neo-go/pkg/services/notary.(*Notary).Run(0xc00010e480)
2021-07-23T11:06:58.3783941Z 	/home/runner/work/neo-go/neo-go/pkg/services/notary/notary.go:148 +0x3cb
2021-07-23T11:06:58.3784702Z created by github.com/nspcc-dev/neo-go/pkg/core.TestNotary
2021-07-23T11:06:58.3785451Z 	/home/runner/work/neo-go/neo-go/pkg/core/notary_test.go:132 +0x6e0
```

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
 2021-07-23 14:48:00 +03:00
Evgeniy Stratonikov
3507f52c32 notary: process new transactions in a separate goroutine 
Related #2063.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
 2021-07-23 14:48:00 +03:00
Roman Khimov
c88ebaede9
Merge pull request #2075 from nspcc-dev/small-refactoring 
Array util refactoring and naming improvement
 2021-07-20 11:29:59 +03:00
Roman Khimov
19717dd9a8 slice: introduce common Copy helper 
It's a bit more convenient to use.
 2021-07-19 22:57:55 +03:00
Roman Khimov
3a93977b7b oracle: only process new requests after initial sync 
If an oracle node is resynchronized from the genesis the service receives all
requests from all blocks via AddRequests() invoked from the native
contract. Almost all of them are long obsolete and need to be removed, native
oracle contract will try to do that with RemoveRequests() calls, but they
won't change anything.

So queue up all "initial" requests in special map and manage it directly
before the module is Run() which happens after synchronization
completion. Then process any requests that are still active and work with new
blocks as usual.
 2021-07-19 22:52:59 +03:00
Roman Khimov
0bad413e23 oracle: ensure proper ValidUntilBlock for backup tx 
See neo-project/neo-modules#608.
 2021-07-19 21:58:10 +03:00
Evgeniy Stratonikov
e936e5e295 oracle: set request Content-Type 
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
 2021-07-12 13:13:48 +03:00
Evgeniy Stratonikov
8e9302f40b oracle: check response Content-Type 
If not specified everything is allowed.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
 2021-07-12 13:13:48 +03:00
Evgeniy Stratonikov
a4c7b3d0cf jsonpath: add tests for null objects, fix #2039 
Turns out, nothing should be changed in our implementation because
nil-check is done implicitly by type casts or type switches.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
 2021-07-07 12:27:10 +03:00
Anna Shaleva
472a44a464 *: regenerate test wallets with resources-friendly scrypt parameters 2021-06-11 15:22:32 +03:00
Anna Shaleva
50296975e2 crypto: allow to provide non-default scrypt parameters 2021-06-11 15:22:28 +03:00
Pavel Karpy
874c56c77b oracle/neofs: stop using deprecated types 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-06-10 14:16:07 +03:00
Pavel Karpy
c4f8c7a9b2 oracle/neofs: update neofs-api-go 
Update neofs-api-go to `v1.27.1`.
Add `WithURIAddress` client option.
Stop using `V2` abstractions outside
of the `api-go` library.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-06-10 14:16:02 +03:00
Anna Shaleva
b8e96ac82b core: move mempool.Event to a separate package 
And write a marshaller for EventType, it'll be reused by the Notification
subsystem.
 2021-06-01 12:24:28 +03:00
Evgeniy Stratonikov
33e1e61343 config: make MaxValidUntilBlockIncrement configurable 2021-05-17 13:43:03 +03:00
Evgeniy Stratonikov
ea49c6b637 wallet: rename isdefault to isDefault 
Follow neo-project/neo#2451 .
 2021-05-14 10:31:31 +03:00
Roman Khimov
4131e208e5 jsonpath: drop unnecessary assignment, fix lint fail 
pkg/services/oracle/jsonpath/jsonpath.go:354:8  ineffassign  ineffectual assignment to val
 2021-05-13 17:26:38 +03:00
Roman Khimov
f2f9ec1a05
Merge pull request #1916 from nspcc-dev/fix/oraclefilter 
oracle: make JSONPath compatible with C# implementation
 2021-05-13 15:34:52 +03:00
Evgeniy Stratonikov
28a0d68c05 jsonpath: allow empty paths 2021-05-13 11:53:49 +03:00
Evgeniy Stratonikov
5fc81c787b jsonpath: use ordered map json unmarshaler 2021-05-13 11:53:49 +03:00
Evgeniy Stratonikov
6890688b8f oracle: add max nesting depth to JSONPath filter 2021-05-13 11:53:49 +03:00
Evgeniy Stratonikov
1c30d8c395 oracle: make JSONPath compatible with C# implementation 
C# node uses simplified implementation which is easy to port.
 2021-05-13 11:53:28 +03:00
Roman Khimov
c4e084b0d8 *: fix whitespace errors 
leading/trailing newlines
 2021-05-12 22:51:41 +03:00
Roman Khimov
92dbb3c4b9 *: fix all unused warnings 
From golangci-lint.
 2021-05-12 18:53:12 +03:00
Roman Khimov
cfc067dd24 *: remove dead code 
Found by deadcode via golangci-lint.
 2021-05-12 18:13:14 +03:00
Anna Shaleva
366e79b9b8 core: rename Neo.Crypto.CheckSig interop 2021-05-11 18:37:55 +03:00
Roman Khimov
3402a3ec30 oracle: send user-agent header with our requests, fix #1923 2021-05-07 16:40:45 +03:00
Roman Khimov
39571745cb stateroot: fix incompleteRoot leak 
Drop old incomplete roots from the map.
 2021-05-07 14:31:20 +03:00
Roman Khimov
3d49f7d99a stateroot: only send stateroot message once per validators count 
Actually fix #1922.
 2021-05-07 14:31:20 +03:00
Roman Khimov
4a45abe3e0 stateroot: make vote/root extensibles live less 
These are arbitrary numbers, but that's what C# node uses. maxRetries could be
set lower because we have exponential backoff anyway, but this works too.
 2021-05-07 14:31:20 +03:00
Roman Khimov
7c902669bf stateroot: implement exponential vote resends 2021-05-07 14:31:20 +03:00
Roman Khimov
d5c7a40db9 stateroot: try finalizing stateroot when adding our signature 
And don't add/resend it multiple times.

1. We can be in a setup with one SV only and no AddSignature() called at all.
2. AddSignature() might add M-1 signatures and our signature should be the
   last one to complete MPTRoot, but we'll never do that.
 2021-05-06 23:29:43 +03:00
Roman Khimov
847927af74 stateroot: s/onValidatedRoot/relayExtensible/ 
Makes more sense because it's not just roots that we're relaying.
 2021-05-06 22:32:13 +03:00
Roman Khimov
222b4dc920 stateroot: simplify account locking, deduplicate code 2021-05-06 22:17:09 +03:00
Roman Khimov
fc800dcbc1 stateroot: protect incomplete root manipulations in signAndSend 
We're modifying it here, so there can be a race between this method and
AddSignature().
 2021-05-06 21:53:34 +03:00
Roman Khimov
7a1c1638e4 oracle: specify neofs timeout as timeout, set default if 0 
There has to be some sane default and we'd like configuration to be consistent
with other timeout values.
 2021-04-06 17:08:04 +03:00
Roman Khimov
f69fd34220 oracle: add logging, refactor request processing a bit 
And return more appropriate errors in some cases.
 2021-04-06 17:02:32 +03:00
Roman Khimov
d2f7f00997 oracle: change neofs URI scheme 
Strip '//', see neo-project/neo-modules#518.
 2021-04-06 16:56:19 +03:00
Roman Khimov
bd9a303e29 oracle: URI host validation is only relevant for https 
NeoFS doesn't have hosts.
 2021-04-06 16:53:32 +03:00
Roman Khimov
29b79db998 oracle: we only work with https, forget http 2021-04-06 16:53:18 +03:00
Roman Khimov
84b240cc9a services/consensus: add some logging on start 2021-04-02 13:13:26 +03:00
Roman Khimov
a01636a1b0 stateroot: set networking callback in a more straightforward way 2021-04-02 12:12:36 +03:00
Roman Khimov
4d0cbebb5a stateroot: make it shutdown properly, not locking up whole system 2021-04-02 11:38:49 +03:00
Roman Khimov
863b9550e6 rpcbroadcaster: don't mess with the URL from the config 
It can be http://, it can be https://, it's not up to rpcbroadcaster to change
it in any way.
 2021-04-02 11:13:38 +03:00
Roman Khimov
3b4dde05c3 stateroot: handle ErrStateMismatch internally 
It's a local problem and returning error from here leads to peer disconnect
that isn't solving anything.
 2021-03-27 00:33:11 +03:00
Roman Khimov
1fdd406234 stateroot: drop unused function 2021-03-27 00:33:11 +03:00
Roman Khimov
042aef452d stateroot: Extensible messages must have proper category 2021-03-27 00:33:11 +03:00
Roman Khimov
5569512ce9 stateroot: fix missing verification script 
Stateroot payload must have it.
 2021-03-26 13:45:18 +03:00
Roman Khimov
5f7ec6e784 state: fix MPTRoot JSONization 
Match C# implementation. It's easier to do this by changing the structure
itself.
 2021-03-26 13:45:18 +03:00
Roman Khimov
8c110a6147 state: drop Network from MPTRoot structure 
It's only needed to verify/sign.
 2021-03-26 13:45:18 +03:00
Roman Khimov
f91ff78918 payload: drop Network from Extensible 
It's only used to sign/verify it and is not a part of the structure. It's
still neded in consensus.Payload though because that's the way dbft library
is.
 2021-03-26 13:45:18 +03:00
Roman Khimov
d314f82db3 transaction: drop Network from Transaction 
We only need it when signing/verifying.
 2021-03-26 13:45:18 +03:00
Roman Khimov
04e0ea2c0f stateroot: sign extensible payloads being sent! 
Unsigned payloads are considered to be invalid for some reason.
 2021-03-26 13:45:18 +03:00
Roman Khimov
0248e2c0d2 state: add network to MPTRoot, change hashing 
Fix RC1 incompatibility.
 2021-03-25 22:37:30 +03:00
Evgeniy Stratonikov
e8b95ebf3e oracle/neofs: change result types 
1. `GetHash` returnes '0x'-prefixed little-endian hex-string.
2. `GetPayload` and `GetRange` assume result is UTF-8.
 2021-03-15 10:54:08 +03:00
Evgeniy Stratonikov
540c6c13a9 oracle/neofs: allow to use filters on NeoFS responses 2021-03-12 14:15:40 +03:00
Anna Shaleva
cdaca7be3e core: use Neo.Crypto.CheckSig for standard signature verification 2021-03-10 21:45:58 +03:00
Roman Khimov
731adf8448
Merge pull request #1484 from nspcc-dev/oracle/neofs 
Integrate NeoFS into oracle module
 2021-03-09 17:30:15 +03:00
Evgenii Stratonikov
22b4edd58e oracle: support NeoFS 2021-03-09 14:24:57 +03:00
Evgeniy Stratonikov
2f3abf95a2 stateroot: broadcast state on new blocks 2021-03-09 13:51:11 +03:00
Evgeniy Stratonikov
3c65ed1507 stateroot: allow to sign new roots 2021-03-09 13:51:11 +03:00
Evgeniy Stratonikov
ac227a80fe stateroot: use RoleStateValidator for verification 2021-03-09 13:51:10 +03:00
Evgeniy Stratonikov
717be43a5d oracle: split broadcaster into parts 
1. Generic parallel sending part can be reused by state service.
2. Specific oracle marshaling is implemented on top of (1).
 2021-03-09 13:46:44 +03:00
Evgeniy Stratonikov
61ce4a7f79 core: set native script and hash in SetOracle 2021-02-26 10:59:09 +03:00
Evgeniy Stratonikov
f9f1fe03b2 core: refactor native call 
1. `System.Contract.CallNative` expects version on stack.
2. Actual method is determined based on current
   instruction pointer.
3. Native hashes don't longer depend on NEF checksum.
 2021-02-26 10:59:09 +03:00
Anna Shaleva
988dfcc7fe services: update Notary transaction size after completion 
Close #1766
 2021-02-24 12:38:32 +03:00
Roman Khimov
608df7fb21
Merge pull request #1747 from nspcc-dev/default-notary-config 
notary: usability improvements
 2021-02-17 14:21:47 +03:00
Anna Shaleva
5569f7ad6c services: optimize notary transaction verification 2021-02-17 13:19:26 +03:00
Anna Shaleva
bcb82b457d config: move notary module config to ApplicationConfiguration 2021-02-16 13:58:25 +03:00
Anna Shaleva
8444f3d816 network: refactor notary service's PostBlock 
There was a deadlock while trying to finalize transaction during
PostBlock:
	1) (*Notary).PostBlock is called under the blockchain lock
	2) (*Notary).onTransaction is called inside the PostBlock
	3) (*Notary).onTransaction needs to RLock the blockchain to add
completed transaction to the memory pool (and the blockchain is Lock'ed
by this moment)

The problem is fixed by using notifications subsistem, because it's not
required to call (*Notary).PostBlock under the blockchain lock.
 2021-02-11 17:11:36 +03:00
Anna Shaleva
b0563ad0d7 services: fix Oracle responces mutex 
Solves the following problem:
=== RUN   TestOracleFull
    logger.go:130: 2021-02-04T09:25:16.305Z	INFO	P2PNotaryRequestPayloadPool size is not set or wrong, setting default value	{"P2PNotaryRequestPayloadPoolSize": 1000}
    logger.go:130: 2021-02-04T09:25:16.306Z	INFO	no storage version found! creating genesis block
    logger.go:130: 2021-02-04T09:25:27.687Z	DEBUG	done processing headers	{"headerIndex": 1, "blockHeight": 0, "took": "2.413398ms"}
    logger.go:130: 2021-02-04T09:25:27.696Z	DEBUG	done processing headers	{"headerIndex": 2, "blockHeight": 1, "took": "1.138196ms"}
    logger.go:130: 2021-02-04T09:25:28.680Z	INFO	blockchain persist completed	{"persistedBlocks": 2, "persistedKeys": 173, "headerHeight": 2, "blockHeight": 2, "took": "166.793µs"}
fatal error: sync: Unlock of unlocked RWMutex

goroutine 6157 [running]:
runtime.throw(0x115dfdb, 0x20)
	/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc000297ca0 sp=0xc000297c70 pc=0x44f432
sync.throw(0x115dfdb, 0x20)
	/usr/local/go/src/runtime/panic.go:1102 +0x35 fp=0xc000297cc0 sp=0xc000297ca0 pc=0x44f3b5
sync.(*RWMutex).Unlock(0xc000135300)
	/usr/local/go/src/sync/rwmutex.go:129 +0xf3 fp=0xc000297d00 sp=0xc000297cc0 pc=0x4a1ac3
github.com/nspcc-dev/neo-go/pkg/services/oracle.(*Oracle).Run(0xc000135180)
	/go/src/github.com/nspcc-dev/neo-go/pkg/services/oracle/oracle.go:189 +0x82b fp=0xc000297fd8 sp=0xc000297d00 pc=0xe13b0b
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc000297fe0 sp=0xc000297fd8 pc=0x4834d1
created by github.com/nspcc-dev/neo-go/pkg/core.TestOracleFull
	/go/src/github.com/nspcc-dev/neo-go/pkg/core/oracle_test.go:276 +0x3f1
 2021-02-05 11:37:12 +03:00
Anna Shaleva
df26ca1c7f core: add Notary module test 2021-02-02 22:01:32 +03:00
Anna Shaleva
bfbd096fed core: introduce mempool notifications 2021-02-02 22:01:32 +03:00
Anna Shaleva
19fa0daaa6 core, network: add Notary module 2021-02-02 22:01:20 +03:00
Evgeniy Stratonikov
c420014cb5 oracle: implement filters 2021-01-28 13:00:59 +03:00
Evgenii Stratonikov
e4528e59dc oracle: reprocess request on fail 2021-01-28 13:00:59 +03:00
Evgenii Stratonikov
aa852aaaac oracle: submit responses concurrently 2021-01-28 13:00:59 +03:00
Evgenii Stratonikov
25d734cbad oracle: process requests concurrently 2021-01-28 13:00:59 +03:00
Evgenii Stratonikov
43e4d3af88 oracle: integrate module in core and RPC 
1. Initialization is performed via `Blockchain` methods.
2. Native Oracle contract updates list of oracle nodes
  and in-fly requests in `PostPersist`.
3. RPC uses Oracle module directly.
 2021-01-28 13:00:58 +03:00
Evgenii Stratonikov
7e16bea126 network: implement Oracle module 2021-01-28 12:43:20 +03:00