Commit graph

272 commits

Author SHA1 Message Date
Leonard Lyubich
acd63afd48 [#493] node: Replace configuration of API clients to a separated section
Replace dial timeout of API client connection from `grpc` section to
`apiclient` since this value is semantically decoupled from the transport
protocol.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
31eb6617fc [#493] node: Abolish non-usable gRPC server configurations
Reflect service is disabled, max message size frozen by constant.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
8d17dab86e [#493] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
6339f1a468 [#493] node: Connect to main chain
Establish client connection with main chain node on storage node startup.
Client is configured simlarly to morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 19:41:51 +03:00
Leonard Lyubich
1fa49bca9c [#493] logger: Write app version once on start
Writing application version in each log message seems pretty redundant.

Remove global `app_version` field from Node/IR loggers. Write version, build
time and debug flag once on start.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
bec4507997 [#493] logger: Do not write app name in each log message of Node/IR
Writing application name in each log message seems pretty redundant.

Remove no longer used `misc.NodeName` and `misc.InnerRingName` constants.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
db2e43387b [#493] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
2a970b0125 [#493] logger: Do not set global fields in constructor
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
d17526f8ac [#493] logger: Abolish non-usable options
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.

Remove default values of no longer existing options in node/ir config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
d49bd4b94a Update neofs-api-go to v1.26.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-10 09:45:11 +03:00
Pavel Karpy
bd6d5e7f7b [#504] reputation/intermediate: Add reading alpha from global cfg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-06 13:52:43 +03:00
Pavel Karpy
20b7295087 [#488] reputation/router: Add commentaries and TODO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
25ea5fea90 [#488] reputation: Initial Trusts using netMap
Make initial trust values depend on NetMap:
initial trust equals 1 / (`amount of storage nodes`).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d1db54acf8 [#488] reputation: Change Writer interface
Includes:
- Delete first `ctx` argument in `Write` method.
- Move intermediate Initial trust struct and method
to `calculator` file.
- Change Alpha to 0.1.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
0d34d7c508 [#488] cmd/reputation: Collect all reputation wrappers
Init all wrappers eigenTrust algorithm needed in
`main` packages. Implement `SendIntermediateResult`
GRPC method of `reputationServer`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
eb74a9cafc [#488] reputation: Add commentaries
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
2c8c9f69c8 [#488] storage/calculator: Make alpha dynamic
Delete reading `alpha` from env var. Cover
retrieving `alpha` behind interface in
intermediate calculator. Add TODO to decide
if it is necessary to receive that param from
global config or not.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ad4a55468e [#488] storage/config: Add wrapper and scripthash
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ea781664cf [#488] reputation/eigentrust/calculator: Implement calc wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d3c1fc7dda [#488] reputation/local/storage: Add trusting peer
Fix iteration logic after adding trusting peer
in `Trust` struct.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
b75074991b [#488] reputation/cmd: Add implementation
Change existing implementations and
add new in `main` package after defining
new interfaces.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e69917b27a [#488] reputation/common: Move ServerInfo to common pkg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f7aa79f0b6 [#488] cmd/reputation/storage: Add consumers storage
Add consumer storage wrapper that implements
`WriterProvider` interface. Change field naming
in daughters storage package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e8885d72f4 [#488] reputation/remoteProvider: Implement intermediate remote
Move common remoteProvider code to cmd/reputation/common.
Hide WriterProvider initialization behind interface and
add implementation of that interface to local and
intermediate packages in cmd/reputation directory.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
49d477f466 [#488] reputation/managers: Implement route builders
Add implementation of Builder interface for
intermediate trusts. Move all code associated
with managers to `common` directory in `cmd`
and `pkg/services/reputation`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
b293ea3ba1 [#488] reputation/intermediate: Add implementation of
`DaughterTrustIteratorProvider` interface

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
35ec694b04 [#488] cmd/cfg: Update cfgReputation
Add `alpha` and `workerPool` to
`cfgReputation` structure.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f6783f4f81 [#488] cmd/reputation: Add DaughterStorage
Add `DaughterStorage` init in main pkg
and start write all received daughters'
trusts to it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Evgenii Stratonikov
59de521fd1 [#472] blobstor: implement write-cache
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
a8a9f88d90 [#471] cmd: allow to start node as Relay node
Disable bootstrapping completely.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-27 17:15:36 +03:00
Alex Vanin
e9b3488aab [#490] cmd/neofs-node: Do not send rebootstrap after node went offline
Control service turns storage node to online and offline modes. We can't
reuse `cfgNetmap.reBootstrapEnabled` because it represents configuration
value while control service manages state in runtime.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-22 09:02:58 +03:00
Pavel Karpy
ff7a8ae677 [#484] cmd/reputation: split it into subpackages
Split cmd/reputation into subpackages
for future refactoring.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-20 10:14:28 +03:00
Pavel Karpy
477682adb7 [#482] reputation/router: Move router to ./common
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
ac8441b718 [#482] reputation/router: Make route pkg independent
Make route package independent from
controller package. Add common interfaces
to `./common` directory.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
0a16aaacb1 [#482] reputation/router: Use trusting peer
In route building use `trusting` peer
field of `reputation.Trust` struct
instead of `reputation.Peer`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Alex Vanin
66ddff3498 [#479] cmd/neofs-node: Add eigen trust block timer
Eigen trust block timer ticks to start new round of
eigen trust calculations. Every epoch this timer
recalculates duration and starts again.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-19 09:35:54 +03:00
Pavel Karpy
f02de33f8b [#476] cmd/containter: Make network handlers async
Make container handlers that do network
communication execute asynchronously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
a04f40d616 [#476] cmd/netmap: Make network handlers async
Make netmap handlers that do network
communication execute asynchronously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
91e0f4b370 [#476] cmd/container: Add async handlers
Add `addContainerAsyncNotificationHandler`
func that passed handler to container's
worker pool similar to how it is done in
reputation handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
72699b4c26 [#476] reputation: Make reputation report async
Add handler closure over worker pool
in the event package.
Add `addNewEpochAsyncNotificationHandler`
function that uses that closure. Pass
the reputation report handler to worker
pool via using that function.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
7cd4e409eb [#476] cmd/cfg: Add worker pools
Add worker pools to Netmap and Container
config structures. Add its initialization
that depends on environmental variables(
sync/async; worker pool size).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Alex Vanin
f6f2e4d10f [#469] cmd/neofs-node: Don't send UpdateState tx at shutdown by default
if `shutdown.offline.enabled` config option set to true, then
send netmap.UpdateState(offline) tx at the shutdown. This config
option is not set by default.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-13 09:19:36 +03:00
Pavel Karpy
60cc3b3e16 [#460] reputation: Add Router to reputation server
Add `Router` to the reputation server. `Router` is
called on every incoming request and inits `Writer`
that sends `Trust`s to the next route point or
handle(logs in that implementation) them if current
node is the end point of the route.

Rename `onlyKeyRemoteServerInfo` struct for container
to separate it from the same implementation of the
same `ServerInfo` interface for reputation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Pavel Karpy
39f55611c6 [#460] reputation/local: Implement dependencies
Implement dependencies for local trust router
in `main` package. Managers are defined as
next node after the current one in HRW sorted
list of current NetMap.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Alex Vanin
7cf48d4d91 [#452] Update neo-go to latest master
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Leonard Lyubich
732038ac28 [#444] cmd/node: Serve ReputationService on the node
Register reputation service server that:
  * verifies signatures of the requests and signs the responses;
  * attaches response meta header to the responses;
  * logs body structures of the requests (only as initial implementation, in
    future it will be replaces with local trust storage).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Alex Vanin
97fe50d452 [#446] cmd/neofs-node: Get inner ring list from RoleManagement contract
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 11:53:10 +03:00
Leonard Lyubich
28fb8c971c [#428] cmd/node: Overload client constructor for Object service
There is a need to track the results of Object interactions with the remote
peers during node's lifetime. Each successful operation should increment the
number of satisfactory interactions with the remote peer, and the failed
ones, on the contrary, decrement. Collected numbers of satisfactory
interactions are going to be used for calculation of the normalized local
trust values according to original EigenTrust algorithm.

Implement wrapper over local trust storage that calls `Update` method on it
with the result of any object operation. The wrapper provides interface of
the remote client constructor. It is used as a `ClientConstructor` component
in Object service server of the app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-24 10:11:52 +03:00
Leonard Lyubich
a6f2394dec [#428] cmd/node: Trigger local trust controller on new epoch
Implement and call `initReputationService` func that constructs local trust
storage and controller, and subscribes the controller on new epoch
notification. Event handler calls `Controller.Report` method to process
collected values.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-24 10:11:52 +03:00
Leonard Lyubich
c4a1c70089 [#428] cmd/node: Implement logging writer provider
Implement simple `WriterProvider` building a `Writer` that writes incoming
data to the log. In the future, this action will be replaced by sending the
value to the manager nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-24 10:11:52 +03:00
Leonard Lyubich
d9f1491566 [#428] cmd/node: Implement iterator provider on local trust storage
Implement iterator provider on wrapper over the local trust storage.
Provided iterator passes normalized local trust values according to original
EigenTrust algorithm description. In the borderline case, normalized values
are set as 1/N, N - number of remote nodes in network map.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-24 10:11:52 +03:00
Leonard Lyubich
106884fc40 [#428] client: Hide client cache behind interface in dependent packages
Replace usage of `cache.ClientCache` type with interface with similar
signature. This will further allow overloading clients without affecting the
logic of dependent packages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-24 10:11:52 +03:00
Alex Vanin
518f375dfd [#441] cmd/neofs-node: Don't update access time in ttlNetCache
ttlNetCache should evict records after TTL duration. However if
data is often accessed and there are no LRU eviction (cache used
with small number of keys), then data will not be evicted ever.

This is a invalid behaviour for mutable data such as eACL.

Solution is to not update access time on every get, so the data
will be guarantee evicted after TTL duration.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-23 20:19:16 +03:00
Leonard Lyubich
383d2494eb [#425] Adapt the Client's refactoring to the interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Leonard Lyubich
718a2fad26 [#425] services: Define service interfaces that was removed from API lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Alex Vanin
7131e6f339 [#426] cmd/neofs-node: Fix metric option in local storage
`WithMetrics` can't make nil check without reflection so we
have to explicitly check if metrics enabled outside of
engine constructor.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin
b8e10571c6 [#426] Put prometheus behind pkg/metrics
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin
3c0e47e6fd [#426] cmd/neofs-node: Add metrics collector in node
Disabled by default.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin
534ce03421 [#426] cmd/neofs-node: Refactor object service pipeline set up
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Evgenii Stratonikov
6679d59e89 [#422] pkg/services: Provide client options on cache creation
Because options are not used when client is already in cache
providing them to shared cache is misleading at best.
In the worst case `dial_timeout` is set randomly (because of race
condition) which can lead to one service having `dial_timeout` of
another. Thus we set default client creation options when cache is
created.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-03-15 14:07:53 +03:00
Evgenii Stratonikov
cc7287d6f7 [#422] pkg/services: Cache clients by address only
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-03-15 14:07:53 +03:00
Leonard Lyubich
55dec28bbb [#419] cmd/node: Cache network maps read from sidechain
Implement LRU cache of eACL tables read from sidechain. Use it as a netmap
storage in neofs-node app. Current cache size is set to 10 (constant).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich
f9342aa562 [#419] cmd/node: Cache eACL tables read from sidechain
Implement TTL cache of eACL tables read from sidechain. Use it as a eACL
storage in neofs-node app. Current cache size is set to 100, TTL is 30s
(constants).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich
34a51ed38b [#419] cmd/node: Cache containers read from sidechain
Implement LRU cache for values read from the network. Implement LRU cache of
containers read from sidechain. Use it as a container storage in neofs-node
app. Current cache size is set to 100, TTL is 30s (constants).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich
ed808c3f1b [#419] eacl: Replace eACL storage implementation to app-side
Replace `eacl.Storage` interface implementation from eACL lib to neofs-node
app package. Remove `eacl.WithMorphClient` option in order to abstract eACL
validator from eACL storage implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich
f1a4b8d322 [#217] cmd/node: Use local Inhume operation as a redundant copy callback
Exec `StorageEngine.Inhume` operation on redundant copy callback from Object
Policer with `MarkAsGarbage` parameter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 20:59:14 +03:00
Leonard Lyubich
7a4e3efa95 [#217] storage engine: Transform Inhume to batch operation
Make `StorageEngine.Inhume` to process list of objects per single operation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 20:59:14 +03:00
Leonard Lyubich
aa24702ebe [#401] control: Serve DropObjects RPC
Re-compile protobuf definition of Control service. Implement required
messages on DropObjects RPC request and response messages. Implement
`DropObjects` method on Control service server of the node. Use
`StorageEngine.Delete` method as a deleted object handler on server.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 13:07:53 +03:00
Leonard Lyubich
9073e198b9 [#398] cmd/node: Serve NetmapService.NetworkInfo RPC
Implement `NetworkInfo` calls on full stack of Netmap services. Current
epoch is read from node local state, magic number is read via `MagicNumber`
call of morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 19:16:40 +03:00
Leonard Lyubich
b644589bb4 [#378] cmd/node: Configure tombstone lifetime
Add `TOMBSTONE_LIFETIME` configuration value of the node which is measured
in NeoFS epoch and is set to 5 by default.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 11:56:32 +03:00
Leonard Lyubich
b961570766 [#378] cmd/node: Subscribe shards on new epoch events
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 11:56:32 +03:00
Leonard Lyubich
18add24727 [#378] cmd/node: Configure GC options
Add `GC_REMOVER_BATCH_SIZE` and `GC_REMOVER_SLEEP_INTERVAL` config values to
shard's config section.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 11:56:32 +03:00
Leonard Lyubich
3b9d4b4df1 [#362] cmd/node: Use const key to UN/LOCODE attribute from the API lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-11 14:01:35 +03:00
Leonard Lyubich
37915b53cf [#362] Pull latest neofs-api-go lib
Adopt new names of constant keys to node attributes from API lib.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-11 14:01:35 +03:00
Leonard Lyubich
e63f9faab8 [#316] cmd/node: Make UN-LOCODE attribute mandatory in the configuration
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 14:05:03 +03:00
Leonard Lyubich
7a9b02049f [#316] cmd/node: Refactor working with well-known node attributes
Split the attributes into those that must be explicitly set in the
configuration, and those that, if absent, will be assigned a default value.
Support this logic in `addWellKnownAttributes` function. If no explicit
attribute is set, the application will panic.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 14:05:03 +03:00
Alex Vanin
9618c3e268 [#367] cmd/neofs-node: Don't update local node info for bootstrap
At startup, update only node info status. Leave all other
attributes from node configuration, so user can update them.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-02-04 14:16:22 +03:00
Leonard Lyubich
bd5c70131b [#328] cmd/node: Serve Container.AnnounceUsedSpace RPC
Register recently implemented handler of AnnounceUsedSpace RPC in node app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
b270c49b5c [#328] container/load: Process Start/Stop notification events from contract
Construct used space Controller on node's app-side. Call Controller.Start on
StartEstimation event from sidechain. Call Controller.Stop on
StopEstimation event from sidechain.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
e9cbdc4a0d [#328] cmd/node: Add the ability to listen Container contract's events
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
8419294f22 [#328] container/load: Implement AnnounceUsedSpace NeoFS API v2 RPC handler
Implement processing of AnnounceUsedSpace RPC from NeoFS API v2 method on
the app-side according to the following algorithm (except common steps for
each RPC):

 * check if the request sender is presented in container;
 * check route of the request (compose from signatures);
 * pass the value to Writer returned by WriterProvider.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
10902c0b82 [#328] container/load: Implement RemoteWriterProvider for controller
Define component on app-side that implements RemoteWriterProvider with next
logic:

 * if ServerInfo is nil, call internal dead end provider;
 * if address in ServerInfo equals to local node's address, do nothing;
 * otherwise create API client instance via client cache and return the
   wrapper over it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
1bed03995b [#328] container/load: Implement Iterator on local storage engine
Implement wrapper over StorageEngine on app-side that iterates over the
values returned by ListContainers+ContainerSize method pair. Component will
be used as the source of local metrics.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
910017b72f [#328] container/load: Implement PlacementBuilder for used space router
Implement wrapper over netmap and container storages on app-side that builds
placement vectors by next steps:

 * read container from storage;
 * read network map from storage by epoch number;
 * build container nodes according to placement policy;
 * build placement vectors with epoch-dependent pivot.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
6f552c8599 [#328] container/load: Implement WriterProvider on API client
Implement wrapper over API client on app-side that passes used space values
to it. Values are buffered on Put call and sent on Close call.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
fa685160f9 [#328] container/load: Implement Writer on Container contract's client
Implement wrapper over Container contract's client on app-side that passes
used space values to it. Uses a fixed public key, whose value will be the
key of the local storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Alex Vanin
e252f93dbf [#350] object/acl: Make best effort to classify request
Classifier looks at list of inner ring nodes and container
nodes from current and previous epoch to classify request.
Sometimes these checks might return error.

Consider there is a request from unknown key and container's
placement policy valid for current epoch and invalid for past
epoch. Classifier tries to find if key belongs to container
node from current epoch -- it is not. Then it tries to find if
key belongs to container node from past epoch and it throws
error, because placement policy is invalid for past epoch.

This is a legit case and classifier should ignore such errors
to provide best effort in matching. The only error classifier
should return is an error when request does not contain
public key to classify it.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-27 20:18:33 +03:00
Leonard Lyubich
198eb06032 [#339] cmd/node: Change the way the local NodeInfo structure is stored
Implement NodeState interface required by Netmap service. Make a single
point of updating the state of the node (for both Netmap and Control
services). Protect node info structure from data race.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-25 17:13:46 +03:00
Leonard Lyubich
3bf1519b1d [#323] cmd/node: Set BoltDB timeout option of Metabase component
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-18 18:53:39 +03:00
Leonard Lyubich
cf00682065 [#323] cmd/node: Call Close method of the storage engine on app shutdown
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-18 18:53:39 +03:00
Leonard Lyubich
9f41192bff [#323] cmd/node: Compose functions for closing components in app
Closing callback can be registered in app through onShutdown method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-18 18:53:39 +03:00
Leonard Lyubich
707434efa9 [#323] cmd/node: Set default value of Blobovnicza's storage size limit
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-18 18:53:39 +03:00
Leonard Lyubich
53f8f06f2d [#323] cmd/node: Set default value of small size limit cfg of BlobStor
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-18 18:53:39 +03:00
Leonard Lyubich
fa0ea35f83 [#315] cmd/node: Implement node status updating in application
Call Netmap contract on SetNetmapStatus rpc of Control service (AddPeer
method if new status is ONLINE, UpdatePeerState method otherwise).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 17:46:12 +03:00
Leonard Lyubich
4666a7797b [#312] cmd/node: Switch health status while the application is running
Switch health status to:
  * STARTING after the initialization of app resources and before bootup;
  * READY after bootup and until shutting down;
  * SHUTTING_DOWN before exiting the application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
147399f547 [#312] cmd/node: Implement HealthStatus method on node application
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
71fcbd3ed4 [#313] cmd/node: Update netmap status in the right way
Update node network status via update status by getting the latest network
map and finding the node in it. This step is performed at the start of the
application and upon notification of a new epoch.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
83d4420a30 [#313] control: Rename HealthStatus enum to NetmapStatus
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
3e882d963f [#311] services/control: Implement NetmapSnapshot method
Implement methods required for signatures. Receive network map from netmap
storage, convert it to Control service message and return in response.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
73e99fa590 [#306] cmd/node: Rename permitted_keys config value to authorized_keys
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00