Commit graph

710 commits

Author SHA1 Message Date
Evgenii Stratonikov
884779e501 *: goimports 2019-12-17 14:51:28 +03:00
Evgenii Stratonikov
014be31a0a network: fix link in pprof 2019-12-17 14:49:28 +03:00
Vsevolod
27a5825d42
Merge pull request #536 from nspcc-dev/feature/pprof
network: add Pprof metrics
2019-12-17 14:21:25 +03:00
Vsevolod Brekelov
0085831ec5 network: add Pprof metrics
Since we have some perf issues from time to time it is good to have pprof debugger. Disabled by default.
2019-12-17 14:13:07 +03:00
Roman Khimov
587cfc7c66 vm: optimize ROLL/ROT, refactor common code
Add `Roll` method to Stack that doesn't pop and push values and use it for
ROLL and ROT.

1.4M->1.5M 100K block import test before:
real    3m44,292s
user    5m43,494s
sys     0m34,741s

After:
real    3m40,449s
user    5m42,701s
sys     0m35,500s
2019-12-16 19:53:21 +03:00
Roman Khimov
2627628387 vm: optimize SWAP instruction, refactor common code
Add `Swap` method to the Stack and use it for both SWAP and XSWAP. Avoid
element popping and pushing (and associated accounting costs).

1.4M->1.5M 100K block import test before:
real    3m51,885s
user    5m54,744s
sys     0m38,444s

After:
real    3m44,292s
user    5m43,494s
sys     0m34,741s
2019-12-16 19:02:40 +03:00
Roman Khimov
7a5e995030 core: add MemPool Verify() test 2019-12-14 00:05:45 +03:00
Roman Khimov
5d68f88196 core: fix and speed up mempool Verify()
First of all, it was wrong, it was not checking for inputs really, it compared
tx hashes for some reason, second, when it did compare inputs it compared only
the PrevIndex part of them which is also wrong.

Also, there is absolutely no reason to go through GetVerifiedTransactions()
here, we don't need this copy of pointers and it can also be outdated by the
time we're to finish our check.

Before:
BenchmarkTXPerformanceTest-4
    5000            485506 ns/op           65886 B/op        409 allocs/op
ok      github.com/CityOfZion/neo-go/integration        3.212s

After:
enchmarkTXPerformanceTest-4
    5000            371104 ns/op           44367 B/op        408 allocs/op
ok      github.com/CityOfZion/neo-go/integration        2.712s
2019-12-13 23:23:33 +03:00
Roman Khimov
35f7f4137e
Merge pull request #556 from nspcc-dev/optimize-dao-and-things
Optimize dao and things, this set improves 1.4M->1.5M import of 100K blocks
by more than 43%.
2019-12-13 22:43:41 +03:00
Roman Khimov
e631e75718 core: optimize GetVerifiedTransactions()
This simple change improves our BenchmarkTXPerformanceTest by 14%, just
because we don't waste time on reallocations during append().

Before:
   10000            439754 ns/op          218859 B/op        428 allocs/op
ok      github.com/CityOfZion/neo-go/integration        5.423s

After:
   10000            369833 ns/op           87209 B/op        412 allocs/op
ok      github.com/CityOfZion/neo-go/integration        4.612s
2019-12-13 19:57:17 +03:00
Roman Khimov
c9257c3de4 vm: optimize Next() in Context
Creating a new BinReader for every instruction is a bit too much and it adds
about 1% overhead on block import (and actually is quite visible in the VM
profiling statistics). So use a bit more ugly but efficient method.
2019-12-13 19:57:17 +03:00
Roman Khimov
a9401e2ec7 core: do not write new dao elements into DB
It's useless work being done before it's actually needed. These (updated with
new values) are going to be written with some kind of Put anyway, so writing
them here is just a waste of time.
2019-12-13 19:57:17 +03:00
Roman Khimov
6896b40dee state: use open-coded array (de)serialization for accounts
We're spending a lot of time here, 100K blocks import starting at 1.4M, before
this patch:
real    4m17,748s
user    6m23,316s
sys     0m37,866s

After:
real    3m54,968s
user    5m56,547s
sys     0m39,398s

9% is quite a substantial improvement to justify this change.
2019-12-13 19:57:17 +03:00
Roman Khimov
0d0a27d271 core: add cachedDao to cache accounts and contracts
Importing 100K blocks starting at 1.4M, before this patch:
real    6m0,356s
user    8m52,293s
sys     0m47,372s

After this patch:
real    4m17,748s
user    6m23,316s
sys     0m37,866s

Almost 30% better.
2019-12-13 19:57:17 +03:00
Roman Khimov
212cf44e26 core: get data from dao in interops
It's more logically correct as Blockchain can be several transactions behind
in terms of its state.
2019-12-13 17:17:14 +03:00
Roman Khimov
0abd55c2c2 vm: add cached ScriptHash() to Context
Avoid recalculating it over and over again in interop.
2019-12-13 17:05:03 +03:00
Roman Khimov
0afaff2f79 vm: microoptimize RegisterInteropFuncs()
Avoid useless copying.
2019-12-13 17:02:28 +03:00
Evgenii Stratonikov
0efd9a6062 consensus: fix payload sign test
Do not fill verification script randomly as there is a probability
for it to be executed sucessfully.

time="2019-12-12T17:24:22+03:00" level=info msg="blockchain persist completed" blockHeight=0 headerHeight=0 persistedBlocks=0 persistedKeys=15 took="54.474µs"
time="2019-12-12T17:24:23+03:00" level=info msg="blockchain persist completed" blockHeight=0 headerHeight=0 persistedBlocks=0 persistedKeys=15 took="49.312µs"
2019-12-12T17:24:24.026+0300    DEBUG   can't verify payload from #%d1  {"module": "dbft"}
--- FAIL: TestPayload_Sign (0.00s)
    payload_test.go:302:
                Error Trace:    payload_test.go:302
                Error:          Should be false
                Test:           TestPayload_Sign
FAIL
coverage: 75.8% of statements
FAIL    github.com/CityOfZion/neo-go/pkg/consensus      2.145s
2019-12-13 12:09:51 +03:00
Evgenii Stratonikov
8a2e1c3d0a io: remove ReadLE/BE and WriteLE/BE
It was replaced with faster specialized versions earlier.
2019-12-13 11:38:28 +03:00
Roman Khimov
09223236f9 core: add Persist() to dao()
Hide its internals a little, which is gonna be useful for future composition.
2019-12-12 21:17:13 +03:00
Roman Khimov
eb1749d190 core: remove persisting from GetValidators()
It's a getter function and even though it's quite fancy with its transactions
processing (for consensus operation) it shouldn't ever change the state of the
Blockchain. If we're to change anything here these changes may conflict with
the actual block processing later or may lead to broken state (if transactions
won't be approved for some reason).
2019-12-12 21:14:33 +03:00
Roman Khimov
31dbc35279 core: gofmt dao_test.go 2019-12-12 21:05:36 +03:00
Roman Khimov
c5ed3b788b core: simplify dao creation with newDao() 2019-12-12 21:04:55 +03:00
Roman Khimov
8b3080b972 io: rename Read/WriteBytes to Read/WriteB
go vet is not happy about them:
  pkg/io/binaryReader.go:92:21: method ReadByte() byte should have signature ReadByte() (byte, error)
  pkg/io/binaryWriter.go:75:21: method WriteByte(u8 byte) should have signature WriteByte(byte) error
2019-12-12 20:19:50 +03:00
Roman Khimov
0b14916d79 io: use optimized Read/WriteUXX for Read/WriteVarUint() 2019-12-12 20:19:50 +03:00
Roman Khimov
54d888ba70 io: add type-specific read/write methods
This seriously improves the serialization/deserialization performance for
several reasons:
 * no time spent in `binary` reflection
 * no memory allocations being made on every read/write
 * uses fast ReadBytes everywhere it's appropriate

It also makes Fixed8 Serializable just for convenience.
2019-12-12 20:19:50 +03:00
Roman Khimov
89d7f6d26e core/tx: microoptimize block/tx hashing
Don't hash the data twice.
2019-12-12 18:01:30 +03:00
Roman Khimov
7e83078d13 hash: remove useless no-op decoding
It changes nothing here.
2019-12-12 17:58:34 +03:00
Vsevolod Brekelov
c93a8d2bc4 core: extracted same logic to separate methods 2019-12-11 13:14:45 +03:00
Vsevolod Brekelov
c1f39d5c7b internal: moved testutil method to internal package 2019-12-11 13:14:43 +03:00
Vsevolod Brekelov
2d42b14a1d core: renames entities-> state and removed State prefix 2019-12-11 13:14:18 +03:00
Vsevolod Brekelov
8809fe437d core: unit tests for dao 2019-12-11 13:13:51 +03:00
Vsevolod Brekelov
c0e59ebd4e core: unit tests for entities 2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
c7ac4b6dff core: fix encoding and decoding for notification event 2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
ec17654986 core: refactoring blockchain state and storage
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
2019-12-11 13:05:31 +03:00
Roman Khimov
d0f9a28196 vm/core: improve block import speed with PublicKey caching
This change (closely related to the neo-project/neo#1321 proposal) speeds up
1.4M mainnet blocks import by 30%. Basically, we're eliminating key decoding
for block's multisignature that has the same keys most of the time.

Things I don't like about this patch:
 * yet another parameter for verifyHashAgainstScript()
 * vm keys are not copied in/out

But it's rather simple and solves the problem for this particular case, so I
think it's worth it.
2019-12-10 19:13:29 +03:00
Roman Khimov
35e368c241 io: add a note for WriteArray, fix #519
It can't be really solved in many cases (it's used in P2P protocol and we have
to follow the usual conventions there) and in most of the cases we don't care
about the difference between nil slice and zero-length slice.
2019-12-09 18:39:30 +03:00
Roman Khimov
f1856bfa8b core/tx: remove publickey indirection from assets and txes
It makes very little sense having pointers here, these structures MUST have
some kind of key and this key is not gonna be wandering somewhere on its
own. Fixes a part of #519.
2019-12-09 18:33:04 +03:00
Roman Khimov
5b6c5af704 *: implement EncodeBinary with pointer receivers where appropriate
Everywhere except ParamType (which is just a byte), reduce copying things
around for no real reason.
2019-12-09 18:25:15 +03:00
Roman Khimov
7e371588a7 core/tx: remove one layer of indirection for scripts and inouts
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
2019-12-09 17:14:10 +03:00
Roman Khimov
b542a5e7a0 io: add support for pointer receivers in WriteArray()
It's actually preferable to have pointer receivers for serializable types, so
this should be supported.
2019-12-09 16:57:25 +03:00
Roman Khimov
052ba1e94f
Merge pull request #545 from nspcc-dev/feat/optimizeio
Remove some reflection from the io package
2019-12-09 15:12:13 +03:00
Evgenii Stratonikov
fccb008594 io: implement ReadBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
838050f8b5 io: rename ReadBytes() to ReadVarBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
f01fc1cc29 io: optimize BinWriter.WriteArray()
Replace reflect.MethodByName with a simple interface cast.
2019-12-09 14:59:49 +03:00
Evgenii Stratonikov
1784a14148 io: optimize BinReader.ReadArray()
reflect.MethodByName is a rather expensive function especially when
called on hot path. This became obvious during profiling of db restore.
This commit replaces reflection with a cast to an interface.
2019-12-09 14:58:37 +03:00
Roman Khimov
3898aadcec rpc: simplify uint160 handling
Use new Uint160DecodeStringLE() function.
2019-12-06 19:47:58 +03:00
Roman Khimov
e4d821f32d
Merge pull request #546 from nspcc-dev/write-optimizations
Write optimizations
2019-12-06 19:40:38 +03:00
Roman Khimov
9992a98007 core: optimize balance utxo removal
One of my samples had 8 (out of 30) seconds spent here, but values of 100ms
are more typical. After this change it becomes invisible.
2019-12-06 18:46:24 +03:00
Roman Khimov
844491d365 *: use more efficient WriteBytes where appropriate
Before this patch on block import we could easily be spending more than 6
seconds out of 30 in Uint256 encoding for UnspentBalance, now it's completely
off the radar.
2019-12-06 18:22:21 +03:00