Depending on having notary contract in sidechain get
IR list either from NeoFSAlphabet role either from
netmap contract.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Some users want to specify only wallet section in the SN. It is not
possible if `Key` throws panic on empty value. Instead it should
fallback to wallet section. Panic is suitable if node's key is provided
but invalid.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add `errf`, `exitOnErr` and `exitOnErrCode` functions
that works with errors and exits with non-zero exit
codes on non-nil errors.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Extended ACL input can have version later than CLI one, and it should not be
downgraded. But the version should be still adequate.
Set `pkg.SDKVersion` on false return of `version.IsValid`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Changes:
* replace `iotuil` elements with the ones from `os` package;
* replace `os.Filemode` with `fs.FileMode`;
* use `signal.NotifyContext` instead of `NewGracefulContext` (removed).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to support multiple network addresses of the storage nodes.
Make `BootstrapAddress` to return `network.AddressGroup` (and rename).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Generalize single gRPC interface of the storage node to a group of
interfaces. Each interface calls the same RPC handler.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation if Control listen endpoint was omitted in config
the gRPC listening endpoint was used instead.
Consider empty address as an option to disable the service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to support multiple server endpoints for reputation and
container transmission.
Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Head service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Search service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Get service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Put service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Change type of the `ClientCache.Get` method's parameter to `AddressGroup`.
Use `GroupFromAddress` to call the method from the wrappers in order to no
change their interface.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add group-address `Client` implementation. Return instances of this
implementation from `ClientCache.Get` method.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to generalize single-address client to group-address client.
To do this, we can re-implement `Client` interface from NeoFS API Go library
and still use it in the application code. There is a problem with method
`Raw` which must return single-address raw client. So as not to make changes
to API library we need to overload Client interface in order to support
`Raw` method in group-address client implementation.
Define `Client` interface in new `pkg/core/client` package. Completely
inherit API `Client` interface. Add `RawForAddress` method to build raw
client for the single node address. Adopt the application code that used Raw
method to work with new `Client`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Obtain key once in every cobra command to
pass it to `getOwnerID` and `initSession`
and do not ask to enter password more than
one time in `put` and `putSG` operations.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Change usage function to `Run`(does not return
err). Log errors with `cmd.PrintErrln`. Change
all `fmt.Print*` to `cmd.Print*`.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
`network.Address` structure in most cases created once and used read-only.
Replace `AddressFromString` function with `Address.FromString` method with
the same purpose and implementation. Make all libraries to work with value.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement `Address.WriteToNodeInfo` method which sets address of `NodeInfo`
structure. Use it in storage node application.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Return tyype is clear from the method's signature and docs, there is no
point in reflecting it in the name.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Panic if internal `manet.DialArgs` call returns error since this is
unexpected according to `AddressFromString` implementation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Serve `ControlService` instance on configured endpoint (do not serve if not
specified). Read allowed keys from config.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Also use neo-go private key wrapper where possible, as it
already has methods for (un)marshaling.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Config `NodeInfo` should be used for bootstrap. Separete local node info and
netmap one. Return configured `NodeInfo` if structure from netmap is
missing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In testnet and neofs-dev-env we use timers with these
settings and they are looking good. So it makes sense
to make them default.
Storage node now can't configure bootstrap healthcheck
length and can't disable it. Inner ring default values
now adopt these changes too.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
New config package in storage node does not use
application prefix from misc package. Therefore
inner ring node can define prefix in local scope.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Container commands in NeoFS CLI can use signed session token
to create, delete container and change extended ACL table.
This token should be signed the same way we sign bearer tokens.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
containerSessionToken() actually parses any session token.
We can reuse this function to resign session token when it
takes filepath as an argument, instead of public variable.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation `Config.Sub` method could lead to the violation
of the internal `path` slice because of `append`. This has been observed on
deeply nested subsections.
Fix `Config.Sub` to copy internal slice in order to prevent violations.
Cover problem case in test config files and unit test.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Dedicated type `LoggerSection` turned out to be redundant since it doesn't
do a hidden logic and just uses `config.Config` API.
Remove `LoggerSection` type and implement `Level` which do the same.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation ENV values were read only if config filepath is
specified. From now ENVs are always read to `Config`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In some cases viper doesn't interpret `section.value` as a subsection with
`section` name, but value is value still can be accessed through full
pathname.
Fix `Config.Sub` method implementation in order to always interpret
configuration like described above as a subsection. From now method never
returns nil, therefore an additional check has been removed from the `Value`
method.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Create `logger` sub-package of `config` package. Implement `LoggerSection`
type of logger sub-section. Add `Level` method to read logger level config
value. Default level is `info`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Create `config` package nearby storage node application. Implement `Config`
as a wrapper over `viper.Viper` that provides the minimum functionality
required by the application.
The constructor allows you to read the config from the file. Methods are
provided for reading subsections and values from the config tree. Helper
functions are implemented to cast a value to native Go types.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Container sessions allow to perform some operations on behalf of another
user. There is a need to to attach session tokens to commands.
Add `session` flag to `put`, `delete` and `set-eacl` commands from
`container` section. It should be a path to the JSON-encoded session token.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement function `fatalOnErrDetails` similar to `fatalOnErr` but accepting
string details that are written to log output. Use the function everywhere
in application code without wrapping in an if-else statement.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation wrapper over the Container contract's client
returned the signature of the eACL table in addition to itself. After recent
changes in API Go lib table carries its signature. Thus, it is redundant to
return the table signature separately.
Make `Wrapper.GetEACL` method to return only `eacl.Table` with error.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make `Get` method of the wrapper over Container contract's client to
accept binary container ID. Create `Get` function similar to the previous
`Get` variation. Use this function in Container service server in the place
where `Get` method was used.
Additionally implement `AsContainerSource` function which allows
to simply compose container Source interface from the wrapper.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Wrap functions at `pkg/errors` return nil if error argument
was nil. fmt.Errorf always returns error so we need to add
missing error checks to the code.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
When user creates a container using neofs-cli, he can set a pre-defined ACL.
Current keywords are confusing: for example, `public` really means that
everyone can write to the container, while the expectation is just public
visibility. Perform the following renames for a better understanding of the
purpose:
* `readonly` -> `public-read`;
* `public` -> `public-read-write`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define constants for `public`/`private`/`readonly` keywords of predefined
basic ACL values. Use constants instead of repeating string literals.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
To enable TLS support we can't operate with IP addresses directly.
Certificates are issued with host names so it is required to
pass them into RPC client. DNS resolving should be done by transport
layer and not be a part of node. Therefore `IPAddrString` usage is
removed from code.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Currently main chain's client is unused, therefore, it is not advisable to
require mandatory setting of addresses.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
If default smart contract addresses are valid 20 byte strings, then omitting
the values in configuration will not be tracked at startup.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
From now non-relay node always sends re-bootstrap transaction every 2 epochs
starting from the boot-up epoch.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Application can be shut down often, it is undesirable to send status changes
on a transaction every time. Control API should be used before shutdown for
this purpose.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make work scope and expansion rate the app-side constants since Policer
independently regulates the amount of work performed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is no need to use synchronous execution of notification handlers. Also
there is no understanding of how to assess the need to change the size of
the pools.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Calls to contracts by storage nodes do not lead to the accumulation of
multisignatures in the contract memory, so the call cost can always be
accurately calculated in advance without additional fee.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace dial timeout of API client connection from `grpc` section to
`apiclient` since this value is semantically decoupled from the transport
protocol.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Although the default values for configurable addresses are empty strings,
explicitly specifying the default values allows you to know about all
possible configurations.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Rename `util/profiler` package to `httputil` and refactor it:
* simplify utility HTTP server;
* make more generic server's parameters in order to remove `viper.Viper`
dependency;
* use single constructor for creating the pprof and prometheus servers;
* replace `enabled` config value with empty-check of the network address.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Establish client connection with main chain node on storage node startup.
Client is configured simlarly to morph client.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Writing application version in each log message seems pretty redundant.
Remove global `app_version` field from Node/IR loggers. Write version, build
time and debug flag once on start.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Writing application name in each log message seems pretty redundant.
Remove no longer used `misc.NodeName` and `misc.InnerRingName` constants.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.
Remove default values of no longer existing options in node/ir config.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
For N3 Testnet RC2 release inner ring app supports three modes:
- notary enabled in all chains (default),
- notary disabled in all chains,
- notary enabled only in side chain.
All notary related functions are moved to notary.go
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
When notary disabled, inner ring should be able to
configure extra fee for vote collections inside the
contracts. Previously these values were hardcoded,
however we might want to change them depending on
a environment.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
With `mainnet.notary_deposit=false` inner ring will ignore
notary deposit \ awaiting routines in the application start,
so it can run on the environments without notary support.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Includes:
- Delete first `ctx` argument in `Write` method.
- Move intermediate Initial trust struct and method
to `calculator` file.
- Change Alpha to 0.1.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Init all wrappers eigenTrust algorithm needed in
`main` packages. Implement `SendIntermediateResult`
GRPC method of `reputationServer`.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Delete reading `alpha` from env var. Cover
retrieving `alpha` behind interface in
intermediate calculator. Add TODO to decide
if it is necessary to receive that param from
global config or not.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add consumer storage wrapper that implements
`WriterProvider` interface. Change field naming
in daughters storage package.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Move common remoteProvider code to cmd/reputation/common.
Hide WriterProvider initialization behind interface and
add implementation of that interface to local and
intermediate packages in cmd/reputation directory.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add implementation of Builder interface for
intermediate trusts. Move all code associated
with managers to `common` directory in `cmd`
and `pkg/services/reputation`
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Control service turns storage node to online and offline modes. We can't
reuse `cfgNetmap.reBootstrapEnabled` because it represents configuration
value while control service manages state in runtime.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Eigen trust block timer ticks to start new round of
eigen trust calculations. Every epoch this timer
recalculates duration and starts again.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add `addContainerAsyncNotificationHandler`
func that passed handler to container's
worker pool similar to how it is done in
reputation handlers.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add handler closure over worker pool
in the event package.
Add `addNewEpochAsyncNotificationHandler`
function that uses that closure. Pass
the reputation report handler to worker
pool via using that function.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add worker pools to Netmap and Container
config structures. Add its initialization
that depends on environmental variables(
sync/async; worker pool size).
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
if `shutdown.offline.enabled` config option set to true, then
send netmap.UpdateState(offline) tx at the shutdown. This config
option is not set by default.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add `Router` to the reputation server. `Router` is
called on every incoming request and inits `Writer`
that sends `Trust`s to the next route point or
handle(logs in that implementation) them if current
node is the end point of the route.
Rename `onlyKeyRemoteServerInfo` struct for container
to separate it from the same implementation of the
same `ServerInfo` interface for reputation.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Implement dependencies for local trust router
in `main` package. Managers are defined as
next node after the current one in HRW sorted
list of current NetMap.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Register reputation service server that:
* verifies signatures of the requests and signs the responses;
* attaches response meta header to the responses;
* logs body structures of the requests (only as initial implementation, in
future it will be replaces with local trust storage).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Alias type provide sort function so it is better to use it
everywhere where list of public keys is presented.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
There is a need to track the results of Object interactions with the remote
peers during node's lifetime. Each successful operation should increment the
number of satisfactory interactions with the remote peer, and the failed
ones, on the contrary, decrement. Collected numbers of satisfactory
interactions are going to be used for calculation of the normalized local
trust values according to original EigenTrust algorithm.
Implement wrapper over local trust storage that calls `Update` method on it
with the result of any object operation. The wrapper provides interface of
the remote client constructor. It is used as a `ClientConstructor` component
in Object service server of the app.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement and call `initReputationService` func that constructs local trust
storage and controller, and subscribes the controller on new epoch
notification. Event handler calls `Controller.Report` method to process
collected values.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement simple `WriterProvider` building a `Writer` that writes incoming
data to the log. In the future, this action will be replaced by sending the
value to the manager nodes.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement iterator provider on wrapper over the local trust storage.
Provided iterator passes normalized local trust values according to original
EigenTrust algorithm description. In the borderline case, normalized values
are set as 1/N, N - number of remote nodes in network map.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace usage of `cache.ClientCache` type with interface with similar
signature. This will further allow overloading clients without affecting the
logic of dependent packages.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
ttlNetCache should evict records after TTL duration. However if
data is often accessed and there are no LRU eviction (cache used
with small number of keys), then data will not be evicted ever.
This is a invalid behaviour for mutable data such as eACL.
Solution is to not update access time on every get, so the data
will be guarantee evicted after TTL duration.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
On the `Deposit` events add gas balance check.
Make transfer only if the balance is greater
than the `GasTransferThreshold` that is defined
with environmental variable.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
`WithMetrics` can't make nil check without reflection so we
have to explicitly check if metrics enabled outside of
engine constructor.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Because options are not used when client is already in cache
providing them to shared cache is misleading at best.
In the worst case `dial_timeout` is set randomly (because of race
condition) which can lead to one service having `dial_timeout` of
another. Thus we set default client creation options when cache is
created.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Implement LRU cache of eACL tables read from sidechain. Use it as a netmap
storage in neofs-node app. Current cache size is set to 10 (constant).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement TTL cache of eACL tables read from sidechain. Use it as a eACL
storage in neofs-node app. Current cache size is set to 100, TTL is 30s
(constants).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement LRU cache for values read from the network. Implement LRU cache of
containers read from sidechain. Use it as a container storage in neofs-node
app. Current cache size is set to 100, TTL is 30s (constants).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace `eacl.Storage` interface implementation from eACL lib to neofs-node
app package. Remove `eacl.WithMorphClient` option in order to abstract eACL
validator from eACL storage implementation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Exec `StorageEngine.Inhume` operation on redundant copy callback from Object
Policer with `MarkAsGarbage` parameter.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
NeoFS contract can be deployed in sidechain instead of main chain.
Add `without_mainnet` config flag that can switch IR node to work with
sidechain only. By default this flag is unset.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Previously we set default addresses of alphabetic contracts to empty strings
in inner ring application config. Recently number of using alphabetic
contracts became dynamic. In order to not enumerate default address for each
glagolitic letter, they are removed from config defaults.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add `drop-objects` sub-command to `control` command that drops objects from
the node via `ControlService.DropObjects` API. Address list argument is
passed in NeoFS API string format of the object address.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Re-compile protobuf definition of Control service. Implement required
messages on DropObjects RPC request and response messages. Implement
`DropObjects` method on Control service server of the node. Use
`StorageEngine.Delete` method as a deleted object handler on server.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add `--expires-on` flag to `put` sub-command of `object` command that
indicates object's expiration epoch. Set corresponding object attributes if
flag value is set.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add `netinfo` sub-cmd of `netmap` cmd that read recent information about
NeoFS network. Info is read via NeoFS API NetmapService.NetworkInfo RPC.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
