TrueCloudLab/neoneo-go
7
0
Fork 2
Code Issues Pull requests Projects Releases Packages Wiki Activity
5317 commits 34 branches 134 tags 35 MiB
Commit graph

32 commits

Author SHA1 Message Date
Roman Khimov
cda1c75db3 core: drop GetStandBy* methods 
They're misleading now that we have variable number of committee
members/validators. The standby list can be seen in the configuration and the
appropriate numbers can be received from it also.
 2022-01-31 23:14:38 +03:00
Anna Shaleva
9b841b9b8f core, rpc: use Seek to iterate over NEP* transfers 
The results are controversial a bit. MemoryPS is a special storage
and the new approach doesn't help it to iterate through NEP17 transfers.
However, on long distances (if transfers are requested for more than
1000 blocks, which is ~4.1 hours of mainnet chain) both LevelDB and
BoltDB perform good enough with the new approach. Taking into account
the fact that default value for the query period of `getnep17transfers`
RPC handler is a week (~40K mainnet blocks) we can sacrifice MemoryPS
performance in favour of BoltDB and LevelDB optimisation.

Close #2263.

Benchmark results:

name                                                                           old time/op    new time/op    delta
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1_Take100Blocks-8           783µs ±13%    1762µs ± 4%  +125.12%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1_Take1000Blocks-8         6.91ms ± 2%    9.00ms ± 2%   +30.28%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-100_Take100Blocks-8        1.43ms ± 8%    1.79ms ± 4%   +24.93%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-100_Take1000Blocks-8       7.78ms ± 3%    8.93ms ± 2%   +14.78%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1000_Take100Blocks-8       7.69ms ± 3%    1.73ms ±10%   -77.50%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1000_Take1000Blocks-8      14.1ms ± 2%     9.0ms ± 2%   -36.53%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1_Take100Blocks-8          768µs ± 3%     801µs ± 2%    +4.24%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1_Take1000Blocks-8        8.03ms ± 2%    8.05ms ± 8%      ~     (p=0.436 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-100_Take100Blocks-8       1.70ms ±16%    0.85ms ± 7%   -49.85%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-100_Take1000Blocks-8      10.8ms ± 2%     8.1ms ± 2%   -25.21%  (p=0.000 n=10+8)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1000_Take100Blocks-8      10.8ms ± 2%     0.9ms ± 7%   -92.12%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1000_Take1000Blocks-8     18.2ms ±13%     8.2ms ± 6%   -54.95%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1_Take100Blocks-8         874µs ± 6%     823µs ± 2%    -5.81%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1_Take1000Blocks-8       9.35ms ± 2%    8.14ms ± 2%   -12.91%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-100_Take100Blocks-8      1.85ms ± 4%    0.89ms ±10%   -51.68%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-100_Take1000Blocks-8     10.6ms ± 4%     8.2ms ± 2%   -22.44%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1000_Take100Blocks-8     10.7ms ± 2%     0.9ms ± 2%   -91.90%  (p=0.000 n=8+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1000_Take1000Blocks-8    21.8ms ±15%     8.2ms ± 1%   -62.25%  (p=0.000 n=10+10)

name                                                                           old alloc/op   new alloc/op   delta
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1_Take100Blocks-8           650kB ± 0%    1010kB ± 7%   +55.45%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1_Take1000Blocks-8         6.39MB ± 0%    9.89MB ± 1%   +54.74%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-100_Take100Blocks-8        1.28MB ± 0%    1.14MB ± 0%   -11.25%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-100_Take1000Blocks-8       7.02MB ± 0%   10.01MB ± 0%   +42.54%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1000_Take100Blocks-8       7.02MB ± 0%    1.08MB ± 0%   -84.64%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1000_Take1000Blocks-8      12.8MB ± 0%    10.0MB ± 0%   -22.04%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1_Take100Blocks-8          823kB ± 3%     866kB ± 5%    +5.19%  (p=0.001 n=9+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1_Take1000Blocks-8        9.92MB ± 0%    9.85MB ± 0%    -0.71%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-100_Take100Blocks-8       1.84MB ± 4%    1.01MB ± 5%   -44.75%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-100_Take1000Blocks-8      11.0MB ± 0%    10.0MB ± 1%    -8.45%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1000_Take100Blocks-8      11.0MB ± 0%     1.0MB ± 1%   -90.55%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1000_Take1000Blocks-8     20.0MB ± 0%    10.1MB ± 0%   -49.69%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1_Take100Blocks-8         913kB ± 5%     907kB ± 6%      ~     (p=0.497 n=9+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1_Take1000Blocks-8       10.0MB ± 1%    10.0MB ± 1%    -0.63%  (p=0.001 n=10+8)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-100_Take100Blocks-8      1.92MB ± 2%    1.04MB ± 0%   -45.53%  (p=0.000 n=9+8)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-100_Take1000Blocks-8     11.1MB ± 1%    10.1MB ± 0%    -9.22%  (p=0.000 n=10+7)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1000_Take100Blocks-8     11.1MB ± 1%     1.0MB ± 0%   -90.61%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1000_Take1000Blocks-8    20.4MB ± 1%    10.1MB ± 0%   -50.46%  (p=0.000 n=10+10)

name                                                                           old allocs/op  new allocs/op  delta
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1_Take100Blocks-8           11.1k ± 0%     11.7k ± 0%    +5.35%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1_Take1000Blocks-8           110k ± 0%      110k ± 0%    +0.55%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-100_Take100Blocks-8         22.0k ± 0%     11.9k ± 0%   -46.14%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-100_Take1000Blocks-8         120k ± 0%      110k ± 0%    -8.44%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1000_Take100Blocks-8         120k ± 0%       12k ± 0%   -90.36%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/MemPS_StartFromBlockN-1000_Take1000Blocks-8        219k ± 0%      110k ± 0%   -49.73%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1_Take100Blocks-8          11.3k ± 0%     11.2k ± 0%    -1.06%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1_Take1000Blocks-8          112k ± 0%      110k ± 0%    -2.36%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-100_Take100Blocks-8        22.5k ± 0%     11.3k ± 0%   -49.65%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-100_Take1000Blocks-8        124k ± 0%      110k ± 0%   -11.08%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1000_Take100Blocks-8        124k ± 0%       11k ± 0%   -90.84%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/BoltPS_StartFromBlockN-1000_Take1000Blocks-8       225k ± 0%      110k ± 0%   -51.07%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1_Take100Blocks-8         11.4k ± 0%     11.3k ± 0%    -1.36%  (p=0.000 n=9+9)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1_Take1000Blocks-8         114k ± 0%      111k ± 0%    -2.33%  (p=0.000 n=10+9)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-100_Take100Blocks-8       22.7k ± 0%     11.5k ± 0%   -49.56%  (p=0.000 n=9+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-100_Take1000Blocks-8       125k ± 0%      111k ± 0%   -11.12%  (p=0.000 n=10+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1000_Take100Blocks-8       125k ± 0%       11k ± 0%   -90.82%  (p=0.000 n=8+10)
Blockchain_ForEachNEP17Transfer/LevelPS_StartFromBlockN-1000_Take1000Blocks-8      229k ± 1%      111k ± 0%   -51.42%  (p=0.000 n=10+10)
 2022-01-19 20:55:08 +03:00
Roman Khimov
746644a4eb network: decouple it from blockchainer.Blockchainer 
We don't need all of it.
 2022-01-14 19:57:16 +03:00
Roman Khimov
fc45d3b132 blockchain: properly set invocation counter for verification ctx 
Fix problem noted in #2270.
 2022-01-14 19:57:16 +03:00
Roman Khimov
9f9bd7261c core: return *interop.Context from GetTestVM 
Return everything needed, fix #2273.
 2022-01-14 19:57:16 +03:00
Roman Khimov
bf1604454c blockchainer/network: move StateSync interface to the user 
Only network package cares about it.
 2022-01-14 19:57:14 +03:00
Roman Khimov
c942402957 blockchainer: drop Policer interface 
We never use it as a proper interface, so it makes no sense keeping it this
way.
 2022-01-12 00:58:03 +03:00
Roman Khimov
125e4231b0 core: store NEP-11 transfers, add accessor functions 2021-11-18 00:09:10 +03:00
Roman Khimov
8bb1ecb45a network: remove priority queue from block queue 
Use circular buffer which is a bit more appropriate. The problem is that
priority queue accepts and stores equal items which wastes memory even in
normal usage scenario, but it's especially dangerous if the node is stuck for
some reason. In this case it'll accept from peers and put into queue the same
blocks again and again leaking memory up to OOM condition.

Notice that queue length calculation might be wrong in case circular buffer
wraps, but it's not very likely to happen (usually blocks not coming from the
queue are added by consensus and it's not very fast in doing so).
 2021-11-01 11:49:01 +03:00
AnnaShaleva
2d196b3f35 rpc: refactor calculatenetworkfee handler 
Use (Blockchainer).VerifyWitness() to calculate network fee for
contract-based witnesses.
 2021-10-25 19:07:25 +03:00
Anna Shaleva
07cbe4d253 core: add finalizer functions to interop context 
These functions are aimed to free the resources occupied by storage
iterator by the end of script execution or whenever Finilize is called.
 2021-10-21 10:05:12 +03:00
Anna Shaleva
72726d46d3 core: refactor callers of MemCachedStore.Seek 
MemCachedStore.Seek now sorts results, so its callers may omit sorting.
 2021-10-21 10:05:12 +03:00
Evgeniy Stratonikov
4dd3a0d503 network: request headers in parallel, fix #2158 
Do this similarly to how blocks are requested.
See also 4aa1a37.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
 2021-10-06 15:25:54 +03:00
Anna Shaleva
b3ea7504cb subscriptions: add container hash to notification event 
External users make use of it. Close #2190.
 2021-09-24 17:42:02 +03:00
Anna Shaleva
6357af0bb0 network: fix race in TestHandleGetMPTData 
Init server config before server start. Fixes the following data race:

```
WARNING: DATA RACE
Write at 0x00c00032ef20 by goroutine 26:
  github.com/nspcc-dev/neo-go/pkg/network.TestHandleGetMPTData.func2()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server_test.go:755 +0x10a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1193 +0x202

Previous read at 0x00c00032ef20 by goroutine 24:
  github.com/nspcc-dev/neo-go/internal/fakechain.(*FakeChain).GetConfig()
      /go/src/github.com/nspcc-dev/neo-go/internal/fakechain/fakechain.go:167 +0x6f
  github.com/nspcc-dev/neo-go/pkg/network.(*Server).initStaleMemPools()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:1433 +0x89
  github.com/nspcc-dev/neo-go/pkg/network.(*Server).Start()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server.go:284 +0x288
  github.com/nspcc-dev/neo-go/pkg/network.startWithChannel.func1()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server_test.go:91 +0x44

Goroutine 26 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:1238 +0x5d7
  github.com/nspcc-dev/neo-go/pkg/network.TestHandleGetMPTData()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server_test.go:752 +0x8c
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1193 +0x202

Goroutine 24 (running) created at:
  github.com/nspcc-dev/neo-go/pkg/network.startWithChannel()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server_test.go:90 +0x78
  github.com/nspcc-dev/neo-go/pkg/network.startTestServer()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server_test.go:384 +0xbd
  github.com/nspcc-dev/neo-go/pkg/network.TestHandleGetMPTData.func2()
      /go/src/github.com/nspcc-dev/neo-go/pkg/network/server_test.go:753 +0x55
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1193 +0x202
```
 2021-09-13 11:45:48 +03:00
Anna Shaleva
0fa48691f7 network: do not duplicate MPT nodes in GetMPTNodes response 
Also tests are added.
 2021-09-08 14:25:54 +03:00
Anna Shaleva
0e0b55350a core: convert (*Blockchain).JumpToState to a callback 
We don't need this method to be exposed, the only its user is the
StateSync module. At the same time StateSync module manages its state by
itself which guarantees that (*Blockchain).jumpToState will be called
with proper StateSync stage.
 2021-09-07 19:43:27 +03:00
Anna Shaleva
3b7807e897 network: request unknown MPT nodes 
In this commit:

1. Request unknown MPT nodes from peers. Note, that StateSync module itself
shouldn't be responsible for nodes requests, that's a server duty.
2. Do not request the same node twice, check if it is in storage
already. If so, then the only thing remaining is to update refcounter.
 2021-09-07 19:43:27 +03:00
Anna Shaleva
d67ff30704 core: implement statesync module 
And support GetMPTData and MPTData P2P commands.
 2021-09-07 19:43:27 +03:00
Anna Shaleva
e8bed184d5 core: implement dynamic NEP17 balances tracking 
Request NEP17 balances from a set of NEP17 contracts instead of getting
them from storage. LastUpdatedBlock tracking remains untouched, because
there's no way to retrieve it dynamically.
 2021-07-29 10:23:01 +03:00
Anna Shaleva
e46d76d7aa core: rename state.NEP17Balances to state.NEP17TransferInfo 
Balances are to be removed from state.NEP17TransferInfo, so the remnant
fields are NextTransferBatch, NewBatch and a map of LastUpdatedBlocks.
These fields are more staff-related.

Also rename dao.[Get, Put, put]NEP17Balances and STNEP17Balances
preffix.

Also rename NEP17TransferInfo.Trackers to LastUpdatedBlockTrackers
because NEP17TransferInfo.Balances are to be removed.
 2021-07-28 13:22:53 +03:00
Anna Shaleva
ae36523a61 rpc, internal: fix CodeQL int conversions warnings 
```
Incorrect conversion of an integer with architecture-dependent bit
size from to a lower bit size type int32 without an upper bound
check.
```
 2021-04-19 11:30:28 +03:00
Roman Khimov
df12adaa9e crypto: remove crypto.Verifiable interface 
We can now verify any hash.Hashable thing.
 2021-03-26 13:45:18 +03:00
Anna Shaleva
1261dd5306 core: refactor initVerificationVM 
It needs only VM and function to get contract state. Also exposed this
method and extended Blockchainer. These changes are needed for the
next commit.
 2021-03-12 18:55:27 +03:00
Evgeniy Stratonikov
f83b376181 block: replace Base with Header 2021-03-10 13:38:44 +03:00
Evgeniy Stratonikov
ac227a80fe stateroot: use RoleStateValidator for verification 2021-03-09 13:51:10 +03:00
Evgeniy Stratonikov
bf20db09e0 stateroot: move state-root related logic to core/stateroot 2021-03-09 13:48:29 +03:00
Evgeniy Stratonikov
55698d0426 dao: use raw state.StorageItem instead of pointer 
It is now a slice, there is no need for additional indirection.
 2021-03-09 12:11:25 +03:00
Evgeniy Stratonikov
ffd85dd51d native/policy: remove MaxBlockSize and MaxBlockSystemFee 2021-03-04 16:59:19 +03:00
Evgeniy Stratonikov
f0d8652bcd rpc: implement getnativecontracts RPC 2021-02-11 12:27:43 +03:00
Anna Shaleva
260bcd373c core: store initial native values into DAO 2021-02-05 10:43:17 +03:00
Anna Shaleva
4ad9c7929b internals: move testchain from network to internals 
It'll be useful for tests outside of the network pkg.
 2021-02-02 22:01:32 +03:00