Commit graph

5017 commits

Author SHA1 Message Date
Anna Shaleva
0a4f45c9b0 core: add ability to free storage.Iterator resources 2021-10-21 10:05:12 +03:00
Anna Shaleva
89ee2e7720 core: refactor storage.Find and storage.Iterator to work with channel
Add SeekAsync methods in order to fetch matching storage items
on demand. Refactor storage.Find and storage.Iterator wrt these changes.
2021-10-21 10:05:12 +03:00
Anna Shaleva
f2ac07a3c0 core: do not copy key/value results got from MemCachedStore.Seek
They are already copied inside the MemCachedStore.Seek, so that
persistent storage  can't change them anymore.
2021-10-21 10:05:12 +03:00
Anna Shaleva
72726d46d3 core: refactor callers of MemCachedStore.Seek
MemCachedStore.Seek now sorts results, so its callers may omit sorting.
2021-10-21 10:05:12 +03:00
Anna Shaleva
7ba88e98e2 core: optimize (*MemCachedStore).Seek operation
Real persistent storage guarantees that result of Seek is sorted
by keys. The idea of optimisation is to merge two sorted seek
results into one (memStore+persistentStore), so that
(*MemCachedStore).Seek will return sorted list. The only thing
that remains is to sort items got from (*MemoryStore).Seek.
2021-10-21 10:05:12 +03:00
Anna Shaleva
191cc45032 core: sort items in MemoryStore.Seek
MemoryStore is used in a MemCachedStore as a persistent layer in tests.
Further commits suppose that persistent storage returns sorted values
from Seek, so sort the result of MemoryStore.Seek.

Benchmark results for 10000 matching items in MemoryStore compared to
master:
name          old time/op    new time/op    delta
MemorySeek-8     712µs ± 0%    3850µs ± 0%   +440.52%  (p=0.000 n=8+8)

name          old alloc/op   new alloc/op   delta
MemorySeek-8     160kB ± 0%    2724kB ± 0%  +1602.61%  (p=0.000 n=10+8)

name          old allocs/op  new allocs/op  delta
MemorySeek-8     10.0k ± 0%     10.0k ± 0%     +0.24%  (p=0.000 n=10+10)

For details on implementation efficiency see the
https://github.com/nspcc-dev/neo-go/pull/2193#discussion_r722993358.
2021-10-21 10:05:12 +03:00
Anna Shaleva
d8210c0137 core: add benchmarks for iterator.Next, MemCached.Seek, Mem.Seek 2021-10-21 10:05:12 +03:00
Anna Shaleva
8d8071f97e core: distinguish storage.KeyValue and storage.KeyValueExists
We need Exists field for storage batch related code; other cases may go
without Exists, so add new KeyValue structure and refactor related code.
2021-10-21 10:05:12 +03:00
Roman Khimov
af9e39ced2
Merge pull request #2205 from nspcc-dev/skip-unemitted-events
compiler: do not emit code for unused imported functions
2021-10-14 10:03:28 +03:00
Roman Khimov
40dfca3eda
Merge pull request #2213 from nspcc-dev/deploy-scope
cli/smartcontract: allow to deploy contracts with scope
2021-10-13 16:02:09 +03:00
Roman Khimov
9e7b334953 docs: add a note for wallet conversion 2021-10-13 15:58:41 +03:00
Evgeniy Stratonikov
0bc81aecf4 compiler: do not emit code for unused imported functions
Our current algorithm marks function as used if it is called
at least ones, even if the callee function is itself unused.
This commit implements more clever traversal to collect usage
information more precisely.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-13 15:56:07 +03:00
Evgeniy Stratonikov
0db5dd5fd7 cli/smartcontract: allow to deploy contracts with scope
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-13 15:55:42 +03:00
Roman Khimov
a340e11b0b CHANGELOG: release 0.97.3 2021-10-13 15:42:21 +03:00
Roman Khimov
2bec43511d
Merge pull request #2207 from nspcc-dev/rpc/gethistoricalstate
rpc: implement `getstate` and `findstates` RPC methods
2021-10-13 15:27:51 +03:00
Anna Shaleva
495aba4174 core: try to collapse traversed Billet part on errStop 2021-10-13 14:47:59 +03:00
Anna Shaleva
360bb220b0 rpc: remove unnecessary base64 encoding
It's default encoding for []byte.
2021-10-13 14:47:59 +03:00
Anna Shaleva
892eadf86d core: mandate passing from as a subprefix for (*Trie).Find
However, we need to distinguish empty subprefix and nil subprefix (no
start specified) to match the C# behaviour.
2021-10-13 14:47:41 +03:00
Anna Shaleva
8e7c76827b core: optimise (*Billet).Traverse and (*Trie).Find
(*Billet).Traverse changes:
1. Get rid of the `offset` argument. We can cut `from` and pass just the
part that remains. This implies that node with path matching `from` will
also be included in the result, so additional check needs to be added to
the callback function.
2. Pass `path` and `from` without search prefix. Append prefix to the
result inside the callback.
3. Remove duplicating code.

(*Trie).Find changes:
1. Properly prepare `from` argument for traversing function. It closly
depends on the `path` argument.
2021-10-13 13:52:33 +03:00
Anna Shaleva
43ac4e1517 rpc: implement findstates RPC handler 2021-10-13 11:41:05 +03:00
Anna Shaleva
7da394fd3f core: check MPT node is not requested twice by StateSync module
This check prevents infinite loop if something goes wrong with MPT nodes
restore process.
2021-10-12 14:08:56 +03:00
Roman Khimov
e2c4b6d393
Merge pull request #2209 from nspcc-dev/vm-exceptions
Catch out-of-bounds exceptions in VM
2021-10-12 09:30:49 +03:00
Roman Khimov
38ba6d735e
Merge pull request #2212 from nspcc-dev/fix-nep17-paging
rpc: use uint64 for timestamp boundaries in GetNEP17Transfers
2021-10-11 17:42:59 +03:00
Anna Shaleva
6eb3cad6d5 rpc: use uint64 for timestamp boundaries in GetNEP17Transfers
Block's timestamp is in milliseconds, so it overflows uint32.
2021-10-11 17:34:05 +03:00
Evgeniy Stratonikov
5c470a6e3d vm: catch out-of-bounds exceptions, fix #2208
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-11 17:28:00 +03:00
Evgeniy Stratonikov
435463e620 vm: allow big string keys in PICKITEM
Because `MaxKeySize` is bigger than integer size, we fail on integer
cast while retreiving items from map. SETITEM is not affected.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-11 17:14:28 +03:00
Roman Khimov
fa2ca3f25a
Merge pull request #2204 from nspcc-dev/compiler-underscore
compiler: allow to use multiple underscores in func arguments
2021-10-11 17:08:36 +03:00
Anna Shaleva
01143da621 rpc: add getstate RPC handler 2021-10-11 16:43:44 +03:00
Roman Khimov
daf1e7ebe5
Merge pull request #2211 from nspcc-dev/cli-missing-parameter
cli/wallet: do not panic if receiver is missing, fix #2210
2021-10-11 11:35:24 +03:00
Evgeniy Stratonikov
e3ddbd28ae cli/wallet: do not panic if receiver is missing, fix #2210
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-11 11:04:23 +03:00
Evgeniy Stratonikov
8d562cef99 compiler: allow to use multiple underscores in func arguments
It should still be present in the argument array in VM so just don't
save them in the map.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-09 13:23:11 +03:00
Roman Khimov
fb31a81fd2
Merge pull request #2206 from nspcc-dev/compiler-safe-methods
compiler: check that safe methods exist
2021-10-07 14:53:37 +03:00
Roman Khimov
5ff475383b
Merge pull request #2200 from nspcc-dev/optimize-headers
network: request headers in parallel
2021-10-07 14:51:38 +03:00
Roman Khimov
aa043ea788
Merge pull request #2184 from nspcc-dev/wallet-ro
wallet: allow to open in read-only mode
2021-10-07 14:46:44 +03:00
Anna Shaleva
c8120a139d core: add MaxKeyLength restrictions to MPT's operations 2021-10-06 16:37:23 +03:00
Evgeniy Stratonikov
cb5b075e7d wallet: do not store file handle
Close immediately after read/write. This can be a bit slower but
we store everything in memory anyway and code size is smaller.
`wallet.Close()` method is now a no-op.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-06 15:29:52 +03:00
Evgeniy Stratonikov
4dd3a0d503 network: request headers in parallel, fix #2158
Do this similarly to how blocks are requested.
See also 4aa1a37.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-06 15:25:54 +03:00
Evgeniy Stratonikov
0c01d89827 compiler: check that safe methods exist
If a method is missing from the manifest, it is most likely a typo
or regression after refactoring. There is no "turn-off" flag
for this error because we can do this precisely.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-06 15:18:58 +03:00
Evgeniy Stratonikov
7b6203d0d4 wallet: allow to open in read-only mode, fix #2182
If wallet was opened via `NewWalletFromFile`, open it as read-only first
and re-open for write if needed.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-10-05 16:25:50 +03:00
Roman Khimov
b4e24bef14
Merge pull request #2202 from nspcc-dev/validatorscount
rpc: add validatorscount to getversion response
2021-09-29 11:55:59 +03:00
Roman Khimov
0ddc7be74d
Merge pull request #2199 from nspcc-dev/conflicts-message
Reword some error messages
2021-09-28 10:17:46 +03:00
Roman Khimov
5a2b77238c rpc: add validatorscount to getversion response
See neo-project/neo-modules#642.
2021-09-28 10:10:26 +03:00
Evgeniy Stratonikov
45976a4111 rpc/response: beautify error message
If `Error.Cause` is nil, omit ugly `%s!<nil>`.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-09-28 09:57:42 +03:00
Evgeniy Stratonikov
e6aa9b76ee mempool: reword ErrConflict message, fix #2197
Make it more user-friendly.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-09-28 09:57:42 +03:00
fyrchik
cfeaeb0f07
Merge pull request #2194 from nspcc-dev/fix-config-types
config: fix duration parameter types
2021-09-27 10:20:15 +03:00
Roman Khimov
e34c47a0c6
Merge pull request #2192 from nspcc-dev/rpc/container_hash
subscriptions: add container hash to notification event
2021-09-25 22:33:16 +03:00
Evgeniy Stratonikov
7fa6c8dcf6 config: fix duration parameter types
These parameters denote seconds and are thus unitless integers, not
durations.

Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-09-25 13:13:51 +03:00
Anna Shaleva
5c97e0dcf2 rpc: move NotaryRequestEvent to the subscriptions pkg
It is used for subscriptions only, so move it to the subscriptions pkg.
2021-09-24 17:42:59 +03:00
Anna Shaleva
b3ea7504cb subscriptions: add container hash to notification event
External users make use of it. Close #2190.
2021-09-24 17:42:02 +03:00
Roman Khimov
63c939b9fa
Merge pull request #2191 from nspcc-dev/port-nef-source
nef: add Source field
2021-09-24 10:42:49 +03:00