Commit graph

4749 commits

Author SHA1 Message Date
Pavel Karpy
7cd4e409eb [#476] cmd/cfg: Add worker pools
Add worker pools to Netmap and Container
config structures. Add its initialization
that depends on environmental variables(
sync/async; worker pool size).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Alex Vanin
9a961e21b1 [#477] metabase: Move MergeSplitInfo to storage/util pkg
This function already reused in different storage engine parts
so it makes sense to keep it in separate package.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 11:18:39 +03:00
Alex Vanin
bc09e29bfd [#477] engine: Reuse SplitInfoError fix for engine.Range
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 11:18:39 +03:00
Alex Vanin
7e435db292 [#477] engine: Reuse SplitInfoError fix for engine.Get
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 11:18:39 +03:00
Alex Vanin
7547592ce3 [#477] engine: Iterate over all shards to return complete SplitInfoError
Different SplitInfo parts may be stored in different shards. Storage
engine must not stop at first SplitInfoError and should make
best effort to complete SplitInfo structure if needed.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 11:18:39 +03:00
Alex Vanin
e020fe5597 [#477] engine: Add test for missing link issue
There were no unit tests of storage engine. This commit
adds first test to reproduce missing link ID in split info
at `engine.Head(raw)` request.

Engine tests uses some constructors from metabase tests,
so it is better to locate such functions in common
package at local_object_storage.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 11:18:39 +03:00
Alex Vanin
2716000b08 [#477] engine: Make generateShardID function independent
This function does not use any engine internal states.
This way it can be reused in tests.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 11:18:39 +03:00
Pavel Karpy
bd5ce4535a [#475] morph/client: Ignore "already on chain"
Add checking if returned error contains
"already on chain" substring. Do not
consider that behavior to be erroneous.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-13 20:59:27 +03:00
Alex Vanin
7a25d45f02 [#453] Update config for NEO3 testnet RC1
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-13 11:17:19 +03:00
Pavel Karpy
4a4aee82e0 [#474] morph/client: Add FaultException to error msg
If non-"HALT" `State` occurs after
calling `InvokeFunction` NeoGo client
method, add `FaultException` information
to returning error. Add returning state
check to `NotaryInvoke` method of the
morph/client.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-13 11:15:59 +03:00
Alex Vanin
f6f2e4d10f [#469] cmd/neofs-node: Don't send UpdateState tx at shutdown by default
if `shutdown.offline.enabled` config option set to true, then
send netmap.UpdateState(offline) tx at the shutdown. This config
option is not set by default.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-13 09:19:36 +03:00
Leonard Lyubich
a97e08cfd7 [#473] Implement EigenTrust calculations
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-12 13:25:28 +03:00
Leonard Lyubich
0ec8bcf6b4 Update neofs-api-go library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-12 13:25:28 +03:00
Alex Vanin
05b45270c8 [#468] Update neo-go to v0.94.1
New neo-go version provides:
- new type for roles in `RoleManagement` contract,
- methods to get keys from `RoleManagement` contract,
- new way to sign notary transaction.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-09 11:32:05 +03:00
Pavel Karpy
60cc3b3e16 [#460] reputation: Add Router to reputation server
Add `Router` to the reputation server. `Router` is
called on every incoming request and inits `Writer`
that sends `Trust`s to the next route point or
handle(logs in that implementation) them if current
node is the end point of the route.

Rename `onlyKeyRemoteServerInfo` struct for container
to separate it from the same implementation of the
same `ServerInfo` interface for reputation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Pavel Karpy
39f55611c6 [#460] reputation/local: Implement dependencies
Implement dependencies for local trust router
in `main` package. Managers are defined as
next node after the current one in HRW sorted
list of current NetMap.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Pavel Karpy
5970a5e6ae [#460] reputation/local: Declare Managers
Declare interfaces for building list of
managers of certain peer in certain epoch.
Implement `Builder` struct that aggregates
fields that are necessary for building
list of managers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Pavel Karpy
91825a0162 [#460] reputation/local: Implement local trust router
Implement reputation `Router` and its constructor,
designed to define where to send local trusts.
Router is based on dependencies that are hidden
behind interfaces, that are declared in the router's
package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Pavel Karpy
e6c9fb283c [#460] Fix typos in commentaries
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Alex Vanin
0f01a69fd3 [#465] processor/neofs: Don't modify tx details
Mint, Burn and Cheque details should contain hash of
main net tx of Deposit and Withdraw invocation. They will
be formatted inside the contracts so alphabet nodes must
not modify them.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-08 17:01:00 +03:00
Alex Vanin
3e9c578e62 [#465] settlement: Use unified details format for all asset transfers
Unified format uses transfer type as the first byte
and extra details next. List of transfer types used in
contracts defined in `details.go`. It includes:
- audit settlement,
- basic income collection,
- basic income distribution.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-08 17:01:00 +03:00
Alex Vanin
1d68e74636 [#465] innerring: Add audit fee fetcher to settlement processor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-08 17:01:00 +03:00
Alex Vanin
0816f7b63b [#465] settlement/audit: Transfer audit fee to inner ring nodes
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-08 17:01:00 +03:00
Alex Vanin
e11b5bd32b [#465] morph/client: Add audit fee getter in netmap wrapper
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-08 17:01:00 +03:00
Leonard Lyubich
df909984bd [#461] metabase: Delete graves regardless of the presence of objects
`Inhume` operation can be performed on already deleted objects, and in this
case the entry will be added to the graveyard. `Delete` operation finishes
with error if object is not presented in metabase. However, the entry in the
cemetery must be deleted regardless of the presence of the object.

Additionally, now `Delete` does not return an error in the absence of an
object.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-06 17:34:01 +03:00
Alex Vanin
27cf6f5112 [#452] Use API structures for reputation PeerID and GlobalTrust
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
7cf48d4d91 [#452] Update neo-go to latest master
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
b18da34b55 [#452] innerring: Use reputation processor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
09e4479d44 [#452] innerring: Add reputation processor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
335b31206f [#452] morph/event: Support reputationPut event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
b9a1aaec23 [#452] morph/client: Add reputation contract client wrapper
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
9f1a49a562 [#452] morph/client: Add reputation contract client
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Alex Vanin
ee29ee2c47 [#456] morph/client: Print transaction hashes in LE
neo-go JSON RPC server expect all hashes in LE, e.g.
in `getapplicationlog` method.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 08:03:19 +03:00
Alex Vanin
3aae60d517 [#454] innerring: Increase duration of notary deposit
Extra blocks for notary deposit must not be less than extra blocks
at notary tx rounding.

Consider you make notary deposit every 1000 block for next
1100 blocks. At block 555 you made notary deposit up to 1655.

At block 1554 you want to send notary tx. Notary client uses rounding
to calculate `until` value. By default notary client rounds with up
to 150 block ahead, thus for tx at 1554 `until` will be 1700.

1700 is bigger than deposit limit at 1655 and tx will fail. However
if extra blocks for notary deposit will be 200, then this case
won't be possible.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-02 11:26:37 +03:00
Leonard Lyubich
732038ac28 [#444] cmd/node: Serve ReputationService on the node
Register reputation service server that:
  * verifies signatures of the requests and signs the responses;
  * attaches response meta header to the responses;
  * logs body structures of the requests (only as initial implementation, in
    future it will be replaces with local trust storage).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Leonard Lyubich
3f9c8f2f59 [#444] reputation: Implement sign and response servers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Leonard Lyubich
b37b608150 [#444] reputation/grpc: Implement gRPC ReputationServiceServer
Implement gRPC ReputationServiceServer on structure that forwards requests
to underlying NeoFS API v2 ReputationService `Server`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Leonard Lyubich
df97e35f30 [#444] reputation: Define RPC Server interface
Define `Server` interface of NeoFS API V2 Reputation service server.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Leonard Lyubich
747cfa44e5 [#444] Pull neofs-api-go with reputation changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Alex Vanin
bf7b8679bf [#450] Fix go.sum changes at builds with go1.16
There are changes in `go mod download` in go1.16 and it
adds checksums for all module contents. Therefore we should
perform `go mod tidy` after `go mod download`.

Read more at https://github.com/golang/go/issues/43994

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-31 11:49:53 +03:00
Alex Vanin
576ed4075e [#450] Add go1.16 in tests
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-31 11:49:53 +03:00
Alex Vanin
7880438cca Update changelog for v0.18.0 release
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 17:19:00 +03:00
Alex Vanin
70dcb920b7 [#448] util/keyer: Prioritize hex decoding over base58
Hex encoded values are often may be misinterpreted as base58
values. Reverse case is quite rare, so it is better to try
hex decoding first.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 15:27:55 +03:00
Alex Vanin
3fe5962e92 [#447] innerring: Use governance processor in inner ring
Network map processors creates sync event once per epoch.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 12:38:42 +03:00
Alex Vanin
d0d1731af7 [#447] innerring: Define governance processor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 12:38:42 +03:00
Alex Vanin
3965cc2296 [#447] innerring: Use alias type for slice of public keys
Alias type provide sort function so it is better to use it
everywhere where list of public keys is presented.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 12:38:42 +03:00
Alex Vanin
34f25adc8c [#447] invoke: Add alphabet update method
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 12:38:42 +03:00
Alex Vanin
999ad5e1c0 [#447] innerring: Do not handle or call InnerRingUpdate method
This method has been removed from netmap contract. Corresponding
event from neofs contract renamed to AlphabetUpdate and should not
be processed, because alphabet updated from `RoleManagement`
contract.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 12:38:42 +03:00
Alex Vanin
1d1fc04ac9 [#446] innerring: Use less mutex locks in indexer
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 11:53:10 +03:00
Alex Vanin
e05f1e1394 [#446] innerring: Use alphabet state in processors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 11:53:10 +03:00