Commit graph

1197 commits

Author SHA1 Message Date
Roman Khimov
16d9ba2297
Merge pull request #551 from nspcc-dev/feature/performance_test
performance: add performance test
2019-12-13 16:57:48 +03:00
Vsevolod Brekelov
6ef7837614 performance: removed require call from benchmark test 2019-12-13 16:13:01 +03:00
Evgenii Stratonikov
0efd9a6062 consensus: fix payload sign test
Do not fill verification script randomly as there is a probability
for it to be executed sucessfully.

time="2019-12-12T17:24:22+03:00" level=info msg="blockchain persist completed" blockHeight=0 headerHeight=0 persistedBlocks=0 persistedKeys=15 took="54.474µs"
time="2019-12-12T17:24:23+03:00" level=info msg="blockchain persist completed" blockHeight=0 headerHeight=0 persistedBlocks=0 persistedKeys=15 took="49.312µs"
2019-12-12T17:24:24.026+0300    DEBUG   can't verify payload from #%d1  {"module": "dbft"}
--- FAIL: TestPayload_Sign (0.00s)
    payload_test.go:302:
                Error Trace:    payload_test.go:302
                Error:          Should be false
                Test:           TestPayload_Sign
FAIL
coverage: 75.8% of statements
FAIL    github.com/CityOfZion/neo-go/pkg/consensus      2.145s
2019-12-13 12:09:51 +03:00
Roman Khimov
170d020081
Merge pull request #554 from nspcc-dev/fix/removeio
io: remove Read/Write LE/BE

They were replaced with faster specialized versions earlier. The only obstacle
on the way to removing them completely was dbft library, which is also updated
in this PR.
2019-12-13 11:45:15 +03:00
Evgenii Stratonikov
8a2e1c3d0a io: remove ReadLE/BE and WriteLE/BE
It was replaced with faster specialized versions earlier.
2019-12-13 11:38:28 +03:00
Evgenii Stratonikov
07b88796fb consensus: update dbft version 2019-12-13 11:28:57 +03:00
Roman Khimov
09223236f9 core: add Persist() to dao()
Hide its internals a little, which is gonna be useful for future composition.
2019-12-12 21:17:13 +03:00
Roman Khimov
eb1749d190 core: remove persisting from GetValidators()
It's a getter function and even though it's quite fancy with its transactions
processing (for consensus operation) it shouldn't ever change the state of the
Blockchain. If we're to change anything here these changes may conflict with
the actual block processing later or may lead to broken state (if transactions
won't be approved for some reason).
2019-12-12 21:14:33 +03:00
Vsevolod Brekelov
eeb2fb7004 integration: add signing tx 2019-12-12 21:13:54 +03:00
Roman Khimov
31dbc35279 core: gofmt dao_test.go 2019-12-12 21:05:36 +03:00
Roman Khimov
c5ed3b788b core: simplify dao creation with newDao() 2019-12-12 21:04:55 +03:00
Roman Khimov
32ac01130d
Merge pull request #553 from nspcc-dev/optimize-io-and-hashes
This patchset improves 100K blocks (starting at 1.4M) import time by 25% (from ~8m to ~6m).
2019-12-12 20:31:17 +03:00
Roman Khimov
8b3080b972 io: rename Read/WriteBytes to Read/WriteB
go vet is not happy about them:
  pkg/io/binaryReader.go:92:21: method ReadByte() byte should have signature ReadByte() (byte, error)
  pkg/io/binaryWriter.go:75:21: method WriteByte(u8 byte) should have signature WriteByte(byte) error
2019-12-12 20:19:50 +03:00
Roman Khimov
0b14916d79 io: use optimized Read/WriteUXX for Read/WriteVarUint() 2019-12-12 20:19:50 +03:00
Roman Khimov
54d888ba70 io: add type-specific read/write methods
This seriously improves the serialization/deserialization performance for
several reasons:
 * no time spent in `binary` reflection
 * no memory allocations being made on every read/write
 * uses fast ReadBytes everywhere it's appropriate

It also makes Fixed8 Serializable just for convenience.
2019-12-12 20:19:50 +03:00
Roman Khimov
89d7f6d26e core/tx: microoptimize block/tx hashing
Don't hash the data twice.
2019-12-12 18:01:30 +03:00
Roman Khimov
7e83078d13 hash: remove useless no-op decoding
It changes nothing here.
2019-12-12 17:58:34 +03:00
Vsevolod Brekelov
a2dac4507b performance: add performance test
In order to run it use:
go test -bench=. -benchmem
2019-12-12 16:01:22 +03:00
Roman Khimov
710520a999
Merge pull request #517 from nspcc-dev/refactor_blockchain_storage
core: refactoring blockchain state and storage
2019-12-11 16:14:28 +03:00
Vsevolod Brekelov
c93a8d2bc4 core: extracted same logic to separate methods 2019-12-11 13:14:45 +03:00
Vsevolod Brekelov
c1f39d5c7b internal: moved testutil method to internal package 2019-12-11 13:14:43 +03:00
Vsevolod Brekelov
2d42b14a1d core: renames entities-> state and removed State prefix 2019-12-11 13:14:18 +03:00
Vsevolod Brekelov
8809fe437d core: unit tests for dao 2019-12-11 13:13:51 +03:00
Vsevolod Brekelov
c0e59ebd4e core: unit tests for entities 2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
c7ac4b6dff core: fix encoding and decoding for notification event 2019-12-11 13:05:31 +03:00
Vsevolod Brekelov
ec17654986 core: refactoring blockchain state and storage
add dao which takes care about all CRUD operations on storage
remove blockchain state since everything is stored on change
remove storage operations from structs(entities)
move structs to entities package
2019-12-11 13:05:31 +03:00
Roman Khimov
c43ff15c78
Merge pull request #550 from nspcc-dev/cache-publickeys
vm/core: improve block import speed with PublicKey caching
2019-12-11 11:21:27 +03:00
Roman Khimov
d0f9a28196 vm/core: improve block import speed with PublicKey caching
This change (closely related to the neo-project/neo#1321 proposal) speeds up
1.4M mainnet blocks import by 30%. Basically, we're eliminating key decoding
for block's multisignature that has the same keys most of the time.

Things I don't like about this patch:
 * yet another parameter for verifyHashAgainstScript()
 * vm keys are not copied in/out

But it's rather simple and solves the problem for this particular case, so I
think it's worth it.
2019-12-10 19:13:29 +03:00
Roman Khimov
36df81bf20
Merge pull request #549 from nspcc-dev/serialization-and-struct-improvements
Serialization and struct improvements

This set improves serialization/deserialization performance and, more
importantly, simplifies memory management for some structures avoiding
useless copying.

It adds some percents to the 1,4M blocks import test.
2019-12-10 12:51:30 +03:00
Roman Khimov
35e368c241 io: add a note for WriteArray, fix #519
It can't be really solved in many cases (it's used in P2P protocol and we have
to follow the usual conventions there) and in most of the cases we don't care
about the difference between nil slice and zero-length slice.
2019-12-09 18:39:30 +03:00
Roman Khimov
f1856bfa8b core/tx: remove publickey indirection from assets and txes
It makes very little sense having pointers here, these structures MUST have
some kind of key and this key is not gonna be wandering somewhere on its
own. Fixes a part of #519.
2019-12-09 18:33:04 +03:00
Roman Khimov
5b6c5af704 *: implement EncodeBinary with pointer receivers where appropriate
Everywhere except ParamType (which is just a byte), reduce copying things
around for no real reason.
2019-12-09 18:25:15 +03:00
Roman Khimov
7e371588a7 core/tx: remove one layer of indirection for scripts and inouts
It reduces heap pressure a little for these elements as we don't have to
allocate/free them individually. And they're directly tied to transactions or
block, not being shared or anything like that, so it makes little sense for
them to be pointer-based. It only makes building transactions a little easier,
but that's obviously a minor usecase.
2019-12-09 17:14:10 +03:00
Roman Khimov
b542a5e7a0 io: add support for pointer receivers in WriteArray()
It's actually preferable to have pointer receivers for serializable types, so
this should be supported.
2019-12-09 16:57:25 +03:00
Roman Khimov
052ba1e94f
Merge pull request #545 from nspcc-dev/feat/optimizeio
Remove some reflection from the io package
2019-12-09 15:12:13 +03:00
Evgenii Stratonikov
0e1621a0da go.mod: update dbft and tidy 2019-12-09 15:03:38 +03:00
Evgenii Stratonikov
fccb008594 io: implement ReadBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
838050f8b5 io: rename ReadBytes() to ReadVarBytes() 2019-12-09 15:00:15 +03:00
Evgenii Stratonikov
f01fc1cc29 io: optimize BinWriter.WriteArray()
Replace reflect.MethodByName with a simple interface cast.
2019-12-09 14:59:49 +03:00
Evgenii Stratonikov
1784a14148 io: optimize BinReader.ReadArray()
reflect.MethodByName is a rather expensive function especially when
called on hot path. This became obvious during profiling of db restore.
This commit replaces reflection with a cast to an interface.
2019-12-09 14:58:37 +03:00
Roman Khimov
1217c4f62a
Merge pull request #547 from nspcc-dev/rpc-uint160
rpc: simplify uint160 handling
2019-12-09 11:37:37 +03:00
Roman Khimov
3898aadcec rpc: simplify uint160 handling
Use new Uint160DecodeStringLE() function.
2019-12-06 19:47:58 +03:00
Roman Khimov
e4d821f32d
Merge pull request #546 from nspcc-dev/write-optimizations
Write optimizations
2019-12-06 19:40:38 +03:00
Roman Khimov
9992a98007 core: optimize balance utxo removal
One of my samples had 8 (out of 30) seconds spent here, but values of 100ms
are more typical. After this change it becomes invisible.
2019-12-06 18:46:24 +03:00
Roman Khimov
844491d365 *: use more efficient WriteBytes where appropriate
Before this patch on block import we could easily be spending more than 6
seconds out of 30 in Uint256 encoding for UnspentBalance, now it's completely
off the radar.
2019-12-06 18:22:21 +03:00
Roman Khimov
e7687d620d io: simplify WriteBytes()
Which speeds it up at least twofold for a typical 32-bytes write (and that's
for a very naïve test that allocates new BufBinWriter on every iteration):

pkg: github.com/CityOfZion/neo-go/pkg/io
BenchmarkWriteBytes-8           10000000               124 ns/op
BenchmarkWriteBytesOld-8         5000000               251 ns/op
2019-12-06 17:40:47 +03:00
Roman Khimov
86239dd668
Merge pull request #520 from nspcc-dev/fix/uint
util: refactor uint160/uint256 for endianness consistency 

Closes #314.
2019-12-06 15:52:00 +03:00
Evgenii Stratonikov
aa20a95181 util: add Uint256DecodeStringBE() 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
72fe884faa util: add Uint160DecodeStringLE() 2019-12-06 12:16:55 +03:00
Evgenii Stratonikov
07e832f046 util: add Uint160DecodeBytesLE() 2019-12-06 12:16:55 +03:00