Commit graph

427 commits

Author SHA1 Message Date
Pavel Karpy
6638136d11 [#711] node/config/test: Adapt test to new config examples
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-23 11:09:02 +03:00
Pavel Karpy
863633e6a5 [#705] cmd/neofs-node/object: Add fetcher without Notary
Depending on having notary contract in sidechain get
IR list either from NeoFSAlphabet role either from
netmap contract.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Alex Vanin
fd24a99533 [#694] cmd/neofs-node: Reuse single instance of client cache in all components
This will reduce amount of open connections up to 3 times.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-13 15:17:48 +03:00
Pavel Karpy
53391f057e [#693] node/reputation: Log keys in hex format
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-13 15:12:21 +03:00
Alex Vanin
e2f7b3f1cc [#690] cmd/neofs-node: Fallback to wallet section if node key is not set
Some users want to specify only wallet section in the SN. It is not
possible if `Key` throws panic on empty value. Instead it should
fallback to wallet section. Panic is suitable if node's key is provided
but invalid.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-13 14:49:31 +03:00
Pavel Karpy
75632a7d83 [#667] node: Add --version flag support
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-06 18:07:35 +03:00
Leonard Lyubich
56d4410913 [#638] Update to Go 1.16
Changes:

  * replace `iotuil` elements with the ones from `os` package;
  * replace `os.Filemode` with `fs.FileMode`;
  * use `signal.NotifyContext` instead of `NewGracefulContext` (removed).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-29 13:44:59 +03:00
Leonard Lyubich
81ddaeca16 [#607] config/example: Add all formats of node's network addresses
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
119031c8c7 [#607] network: Make AddressGroup.WriteToNodeInfo method a function
Method implementation doesn't use any private logic.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
163c24a2d2 [#607] cmd/node: Configure group of bootstrap addresses
There is a need to support multiple network addresses of the storage nodes.

Make `BootstrapAddress` to return `network.AddressGroup` (and rename).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8060735732 [#607] cmd/node: Serve gRPC on multiple interfaces
Generalize single gRPC interface of the storage node to a group of
interfaces. Each interface calls the same RPC handler.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
d1eb9c3b0f [#607] cmd/node: Empty Control server address as disable
In previous implementation if Control listen endpoint was omitted in config
the gRPC listening endpoint was used instead.

Consider empty address as an option to disable the service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
c444118f47 [#607] cmd/node: Make reputationClientConstructor to accept AddressGroup
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
cede2b4ed7 [#607] reputation,container: Support address groups in ServerInfo
There is a need to support multiple server endpoints for reputation and
container transmission.

Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8ac3c62518 [#607] object/head: Make client constructor to work with group address
Make Object Head service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
d0e48c949b [#607] object/search: Make client constructor to work with group address
Make Object Search service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
ad14df07f6 [#607] object/get: Make client constructor to work with group address
Make Object Get service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8972f84672 [#607] object/put: Make client constructor to work with group address
Make Object Put service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
6e5d7f84af [#607] network: Generalize LocalAddressSource to address group
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
e11f50ec8e [#607] network: Make ClientCache to accept AddressGroup
Change type of the `ClientCache.Get` method's parameter to `AddressGroup`.
Use `GroupFromAddress` to call the method from the wrappers in order to no
change their interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
c82615667d [#607] network: Return group-address client from ClientCache
Add group-address `Client` implementation. Return instances of this
implementation from `ClientCache.Get` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
3805b0f638 [#607] client: Overload Client interface
There is a need to generalize single-address client to group-address client.
To do this, we can re-implement `Client` interface from NeoFS API Go library
and still use it in the application code. There is a problem with method
`Raw` which must return single-address raw client. So as not to make changes
to API library we need to overload Client interface in order to support
`Raw` method in group-address client implementation.

Define `Client` interface in new `pkg/core/client` package. Completely
inherit API `Client` interface. Add `RawForAddress` method to build raw
client for the single node address. Adopt the application code that used Raw
method to work with new `Client`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Pavel Karpy
48827f42d3 [#643] pkg: Sync method names and commentaries to them
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 16:10:44 +03:00
Leonard Lyubich
adbbad0beb [#607] network: Do not work with Address pointers
`network.Address` structure in most cases created once and used read-only.

Replace `AddressFromString` function with `Address.FromString` method with
the same purpose and implementation. Make all libraries to work with value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
5de074f24c [#607] network: Implement WriteToNodeInfo method on Address
Implement `Address.WriteToNodeInfo` method which sets address of `NodeInfo`
structure. Use it in storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
e5504c7130 [#607] network: Do not use Address.String for address comparison
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
fdd123ad98 [#607] network: Prevent potential panic in Address.Equal method
Make `Address.Equal` method to accept value instead of pointer in order to
prevent NPE.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Pavel Karpy
6b176e8769 [#613] pkg/reputation: Move manager building to pkg
Move `managers` package to `pkg` since
it can be reused in other packages.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-17 18:30:06 +03:00
Evgenii Stratonikov
41a30d6ec2 [#562] config/node: fallback to wallet if raw key is invalid
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Evgenii Stratonikov
2f020a500d [#562] config: parse key on config load
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Evgenii Stratonikov
5cab0026c3 [#562] pkg/morph: remove neofs-crypto uses
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Evgenii Stratonikov
3f07313604 [#562] cmd/neofs-node: use NEP-6 wallet for keys
Also use neo-go private key wrapper where possible, as it
already has methods for (un)marshaling.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Leonard Lyubich
946d4c4253 [#603] cmd/node: Remove no longer used BootstrapType enum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-11 15:36:17 +03:00
Leonard Lyubich
6279b6343f [#603] cmd/node: Separate configuration and netmap node info
Config `NodeInfo` should be used for bootstrap. Separete local node info and
netmap one. Return configured `NodeInfo` if structure from netmap is
missing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-11 15:36:17 +03:00
Leonard Lyubich
5a4c3dfddf [#603] cmd/node: Parse node attributes when node info is constructed
Remove no longer needed `cfgNodeInfo.attributes` field.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-11 15:36:17 +03:00
Leonard Lyubich
4859bb2e1c [#603] cmd/node: Remove unused bootType field of cfgNodeInfo struct
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-11 15:36:17 +03:00
Alex Vanin
a2547da5ae [#493] cmd/node: Fix linter errors in config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 16:07:42 +03:00
Alex Vanin
e9e986ac71 [#493] cmd/node: Remove viper from storage node
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 16:07:42 +03:00
Alex Vanin
cd947bb580 [#493] cmd/node: Add object service section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 16:07:42 +03:00
Alex Vanin
f40b84c99e [#493] cmd/node: Add replicator section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 16:07:42 +03:00
Alex Vanin
8c96494da0 [#493] cmd/node: Add policer section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 16:07:42 +03:00
Alex Vanin
27b4781e95 [#493] cmd/node: Add apiclient section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 16:07:42 +03:00
Alex Vanin
0f4e8d2362 [#496] cmd/node: Use new config for morph and mainchain configuration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 15:12:08 +03:00
Alex Vanin
4ef968aa06 [#496] cmd/node: Add mainchain section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 15:12:08 +03:00
Alex Vanin
cf5e371590 [#496] cmd/node: Add morph section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 15:12:08 +03:00
Alex Vanin
44b19c145f [#493] cmd/node: Use new config for control service configuration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 14:58:22 +03:00
Alex Vanin
c828848024 [#493] cmd/node: Add control section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 14:58:22 +03:00
Alex Vanin
161fca58eb [#493] cmd/node: Use new config for contracts configuration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 14:58:22 +03:00
Alex Vanin
0eea25375e [#493] cmd/node: Add contracts section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-02 14:58:22 +03:00
Pavel Karpy
8a0a75a6b2 [#579] cmd/node: Use new config for GRPC configuration
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-02 14:02:33 +03:00
Pavel Karpy
7dbeb08c58 [#579] cmd/node: Add grpc section to config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-02 14:02:33 +03:00
Pavel Karpy
6ab7efb358 [#577] cmd/node: Use new config for node configuration
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-02 13:28:15 +03:00
Pavel Karpy
42d18f1322 [#577] cmd/node: Add node section to config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-02 13:28:15 +03:00
Pavel Karpy
52c82ef46a [#577] cmd/node: Add tests for boolean type casting in config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-02 13:28:15 +03:00
Evgenii Stratonikov
b8a7c11e57 [#501] object/put: reduce TTL of the relayed request
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-02 10:31:14 +03:00
Leonard Lyubich
6ca7f4511c [#493] cmd/node: Use engineconfig pkg for storage engine construction
Use `engineconfig.IterateShards` in order to compose options of the shards.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 09:54:00 +03:00
Leonard Lyubich
e149eae7b2 [#493] node/config: Test storage engine's config with config/example
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 09:54:00 +03:00
Leonard Lyubich
f663a1c125 [#493] node/config: Implement sections of local object storage
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 09:54:00 +03:00
Leonard Lyubich
e26dc0a6e3 [#493] node/config: Fix corrupting of path to the subsection
In previous implementation `Config.Sub` method could lead to the violation
of the internal `path` slice because of `append`. This has been observed on
deeply nested subsections.

Fix `Config.Sub` to copy internal slice in order to prevent violations.
Cover problem case in test config files and unit test.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 09:54:00 +03:00
Leonard Lyubich
2bbd4d0ee3 [#493] node/config: Implement integer casters
Implement `Int` / `Uint` functions which casts value to `int64` / `uint64`.
Implement safe functions `IntSafe` / `UintSafe`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 09:54:00 +03:00
Alex Vanin
efcd12c71e [#493] cmd/node: Use new config for metrics configuration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-01 15:29:08 +03:00
Alex Vanin
3fbf5e05b2 [#493] cmd/node: Add metrics section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-01 15:29:08 +03:00
Alex Vanin
da8310f0e3 [#493] cmd/node: Use new config for profiler configuration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-01 15:29:08 +03:00
Alex Vanin
561c809fa5 [#493] cmd/node: Add profiler section to config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-01 15:29:08 +03:00
Alex Vanin
922d29ff88 [#493] cmd/node: Support duration type casting in config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-01 15:29:08 +03:00
Alex Vanin
25a13d3611 [#493] cmd/node: Add .env file for tests
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-01 15:29:08 +03:00
Leonard Lyubich
db7a7f9bd3 [#493] cmd/node: Remove no longer needed cfgLogLevel constant
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
56109f941b [#493] cmd/node: Use new config implementation for logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
3b797d7957 [#493] node/config/logger: Simplify approach to read the level
Dedicated type `LoggerSection` turned out to be redundant since it doesn't
do a hidden logic and just uses `config.Config` API.

Remove `LoggerSection` type and implement `Level` which do the same.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
be6898a51d [#493] node/config/logger: Cover ENV variables with unit test
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
3fd4b32a4f [#493] node/config: Cover ENV variables with unit test
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
099ceeae50 [#493] node/config: Implement ENV variable key constructor
Add `internal.Env` function which converts path to config value to ENV
variable key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
270b147205 [#493] node/config: Export ENV constants from internal package
Replace ENV prefix and separator to `internal` package in order to reuse
them for testing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
32dc80e7bc [#493] node/config: Always read the values of ENV variables
In previous implementation ENV values were read only if config filepath is
specified. From now ENVs are always read to `Config`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
1aa88159ca [#493] node/config: Change Sub implementation
In some cases viper doesn't interpret `section.value` as a subsection with
`section` name, but value is value still can be accessed through full
pathname.

Fix `Config.Sub` method implementation in order to always interpret
configuration like described above as a subsection. From now method never
returns nil, therefore an additional check has been removed from the `Value`
method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
e5c014bbfb [#493] node/config/logger: Write unit tests
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
b8a5f09174 [#493] node/config: Export useful functions into a separate test package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
c645781b98 [#493] node/config: Implement logger section
Create `logger` sub-package of `config` package. Implement `LoggerSection`
type of logger sub-section. Add `Level` method to read logger level config
value. Default level is `info`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
cbe3e0a271 [#493] node/config: Implement string caster
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
7e11bf9a55 [#493] cmd/node: Implement a basic configuration component
Create `config` package nearby storage node application. Implement `Config`
as a wrapper over `viper.Viper` that provides the minimum functionality
required by the application.

The constructor allows you to read the config from the file. Methods are
provided for reading subsections and values from the config tree. Helper
functions are implemented to cast a value to native Go types.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-01 11:45:38 +03:00
Leonard Lyubich
3dd10b6795 [#570] *: Remove usage of deprecated elements from API Go library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
9b2939d538 [#567] cmd/node: Close cached clients on shutdown
Call `CloseAll` on all `ClientCache` instances on application shutdown.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 13:11:37 +03:00
Leonard Lyubich
6b1916a529 [#552] cmd/node: Implement error checker with details
Implement function `fatalOnErrDetails` similar to `fatalOnErr` but accepting
string details that are written to log output. Use the function everywhere
in application code without wrapping in an if-else statement.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 10:09:40 +03:00
Pavel Karpy
077f1af5a7 [#549] node: Add TLS to bootstrap address
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Pavel Karpy
33bef46f31 [#549] network/cache: Change Get signature
Make network cache's `Get` method accept
`network.Address` argument instead of
string.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Pavel Karpy
e10981a7d3 [#549] grpc/server: Add TLS encryption
Add TLS to config. Add server side encryption
if it is configured so.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Leonard Lyubich
df197dc38b [#525] morph/container: Do not return signature from GetEACL method
In previous implementation wrapper over the Container contract's client
returned the signature of the eACL table in addition to itself. After recent
changes in API Go lib table carries its signature. Thus, it is redundant to
return the table signature separately.

Make `Wrapper.GetEACL` method to return only `eacl.Table` with error.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
83c27f6e8a [#505] morph/container: Change get container API
Make `Get` method of the wrapper over Container contract's client to
accept binary container ID. Create `Get` function similar to the previous
`Get` variation. Use this function in Container service server in the place
where `Get` method was used.

Additionally implement `AsContainerSource` function which allows
to simply compose container Source interface from the wrapper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Evgenii Stratonikov
2b2b2c2c45 [#496] Use single contract wrapper constructor
There is no need in a separate `New()` or `WrapClient()`

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Alex Vanin
b5256ccf4c [#521] Fix issues with transition from pkg/errors pkg
Wrap functions at `pkg/errors` return nil if error argument
was nil. fmt.Errorf always returns error so we need to add
missing error checks to the code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-19 19:09:35 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-19 16:42:54 +03:00
Alex Vanin
16f13bc0a5 [#522] Use HostAddrString as RPC endpoint instead of IPAddrString
To enable TLS support we can't operate with IP addresses directly.
Certificates are issued with host names so it is required to
pass them into RPC client. DNS resolving should be done by transport
layer and not be a part of node. Therefore `IPAddrString` usage is
removed from code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Pavel Karpy
9fbc6f5efe [#527] reputation: Fix EigenTrust algorithm
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-18 10:29:25 +03:00
Leonard Lyubich
3622e4165a [#493] node: Make list of main chain's RPC endpoints optional
Currently main chain's client is unused, therefore, it is not advisable to
require mandatory setting of addresses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-14 11:02:16 +03:00
Leonard Lyubich
c36bca1ddc [#493] node: Remove unused worker pools for reading object operations
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
f582475ea2 [#493] node: Replace object pool configurations to object section
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
9e2b47d663 [#493] node: Set default contract addresses to empty strings
If default smart contract addresses are valid 20 byte strings, then omitting
the values in configuration will not be tracked at startup.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
9920425db9 [#493] node: Abolish tombstone lifetime configuration
There is no need to change this value for now.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
d36084c26a [#493] node: Replace relay config value to node section
Set default value to `false`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
f90fc86ec8 [#493] node: Abolish re-bootstrap configurations
From now non-relay node always sends re-bootstrap transaction every 2 epochs
starting from the boot-up epoch.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
7a64a70b12 [#492] node: Never go offline on application shutdown
Application can be shut down often, it is undesirable to send status changes
on a transaction every time. Control API should be used before shutdown for
this purpose.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
10308101df [#493] node: Abolish some of the Policer's configurations
Make work scope and expansion rate the app-side constants since Policer
independently regulates the amount of work performed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
80ef4492c1 [#493] node: Get rid of outdated object GC worker
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
1f817d1cd2 [#493] node: Abolish configuration of pools of notification handlers
There is no need to use synchronous execution of notification handlers. Also
there is no understanding of how to assess the need to change the size of
the pools.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
3ef5b0ff9c [#493] node: Do not add fee in smart contract calls
Calls to contracts by storage nodes do not lead to the accumulation of
multisignatures in the contract memory, so the call cost can always be
accurately calculated in advance without additional fee.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
acd63afd48 [#493] node: Replace configuration of API clients to a separated section
Replace dial timeout of API client connection from `grpc` section to
`apiclient` since this value is semantically decoupled from the transport
protocol.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
31eb6617fc [#493] node: Abolish non-usable gRPC server configurations
Reflect service is disabled, max message size frozen by constant.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
8d17dab86e [#493] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
6339f1a468 [#493] node: Connect to main chain
Establish client connection with main chain node on storage node startup.
Client is configured simlarly to morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 19:41:51 +03:00
Leonard Lyubich
1fa49bca9c [#493] logger: Write app version once on start
Writing application version in each log message seems pretty redundant.

Remove global `app_version` field from Node/IR loggers. Write version, build
time and debug flag once on start.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
bec4507997 [#493] logger: Do not write app name in each log message of Node/IR
Writing application name in each log message seems pretty redundant.

Remove no longer used `misc.NodeName` and `misc.InnerRingName` constants.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
db2e43387b [#493] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
2a970b0125 [#493] logger: Do not set global fields in constructor
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
d17526f8ac [#493] logger: Abolish non-usable options
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.

Remove default values of no longer existing options in node/ir config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
d49bd4b94a Update neofs-api-go to v1.26.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-10 09:45:11 +03:00
Pavel Karpy
bd6d5e7f7b [#504] reputation/intermediate: Add reading alpha from global cfg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-06 13:52:43 +03:00
Pavel Karpy
20b7295087 [#488] reputation/router: Add commentaries and TODO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
25ea5fea90 [#488] reputation: Initial Trusts using netMap
Make initial trust values depend on NetMap:
initial trust equals 1 / (`amount of storage nodes`).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d1db54acf8 [#488] reputation: Change Writer interface
Includes:
- Delete first `ctx` argument in `Write` method.
- Move intermediate Initial trust struct and method
to `calculator` file.
- Change Alpha to 0.1.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
0d34d7c508 [#488] cmd/reputation: Collect all reputation wrappers
Init all wrappers eigenTrust algorithm needed in
`main` packages. Implement `SendIntermediateResult`
GRPC method of `reputationServer`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
eb74a9cafc [#488] reputation: Add commentaries
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
2c8c9f69c8 [#488] storage/calculator: Make alpha dynamic
Delete reading `alpha` from env var. Cover
retrieving `alpha` behind interface in
intermediate calculator. Add TODO to decide
if it is necessary to receive that param from
global config or not.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ad4a55468e [#488] storage/config: Add wrapper and scripthash
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ea781664cf [#488] reputation/eigentrust/calculator: Implement calc wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d3c1fc7dda [#488] reputation/local/storage: Add trusting peer
Fix iteration logic after adding trusting peer
in `Trust` struct.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
b75074991b [#488] reputation/cmd: Add implementation
Change existing implementations and
add new in `main` package after defining
new interfaces.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e69917b27a [#488] reputation/common: Move ServerInfo to common pkg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f7aa79f0b6 [#488] cmd/reputation/storage: Add consumers storage
Add consumer storage wrapper that implements
`WriterProvider` interface. Change field naming
in daughters storage package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e8885d72f4 [#488] reputation/remoteProvider: Implement intermediate remote
Move common remoteProvider code to cmd/reputation/common.
Hide WriterProvider initialization behind interface and
add implementation of that interface to local and
intermediate packages in cmd/reputation directory.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
49d477f466 [#488] reputation/managers: Implement route builders
Add implementation of Builder interface for
intermediate trusts. Move all code associated
with managers to `common` directory in `cmd`
and `pkg/services/reputation`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
b293ea3ba1 [#488] reputation/intermediate: Add implementation of
`DaughterTrustIteratorProvider` interface

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
35ec694b04 [#488] cmd/cfg: Update cfgReputation
Add `alpha` and `workerPool` to
`cfgReputation` structure.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f6783f4f81 [#488] cmd/reputation: Add DaughterStorage
Add `DaughterStorage` init in main pkg
and start write all received daughters'
trusts to it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Evgenii Stratonikov
59de521fd1 [#472] blobstor: implement write-cache
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
a8a9f88d90 [#471] cmd: allow to start node as Relay node
Disable bootstrapping completely.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-27 17:15:36 +03:00
Alex Vanin
e9b3488aab [#490] cmd/neofs-node: Do not send rebootstrap after node went offline
Control service turns storage node to online and offline modes. We can't
reuse `cfgNetmap.reBootstrapEnabled` because it represents configuration
value while control service manages state in runtime.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-22 09:02:58 +03:00
Pavel Karpy
ff7a8ae677 [#484] cmd/reputation: split it into subpackages
Split cmd/reputation into subpackages
for future refactoring.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-20 10:14:28 +03:00
Pavel Karpy
477682adb7 [#482] reputation/router: Move router to ./common
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
ac8441b718 [#482] reputation/router: Make route pkg independent
Make route package independent from
controller package. Add common interfaces
to `./common` directory.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
0a16aaacb1 [#482] reputation/router: Use trusting peer
In route building use `trusting` peer
field of `reputation.Trust` struct
instead of `reputation.Peer`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Alex Vanin
66ddff3498 [#479] cmd/neofs-node: Add eigen trust block timer
Eigen trust block timer ticks to start new round of
eigen trust calculations. Every epoch this timer
recalculates duration and starts again.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-19 09:35:54 +03:00
Pavel Karpy
f02de33f8b [#476] cmd/containter: Make network handlers async
Make container handlers that do network
communication execute asynchronously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
a04f40d616 [#476] cmd/netmap: Make network handlers async
Make netmap handlers that do network
communication execute asynchronously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
91e0f4b370 [#476] cmd/container: Add async handlers
Add `addContainerAsyncNotificationHandler`
func that passed handler to container's
worker pool similar to how it is done in
reputation handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
72699b4c26 [#476] reputation: Make reputation report async
Add handler closure over worker pool
in the event package.
Add `addNewEpochAsyncNotificationHandler`
function that uses that closure. Pass
the reputation report handler to worker
pool via using that function.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
7cd4e409eb [#476] cmd/cfg: Add worker pools
Add worker pools to Netmap and Container
config structures. Add its initialization
that depends on environmental variables(
sync/async; worker pool size).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Alex Vanin
f6f2e4d10f [#469] cmd/neofs-node: Don't send UpdateState tx at shutdown by default
if `shutdown.offline.enabled` config option set to true, then
send netmap.UpdateState(offline) tx at the shutdown. This config
option is not set by default.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-13 09:19:36 +03:00
Pavel Karpy
60cc3b3e16 [#460] reputation: Add Router to reputation server
Add `Router` to the reputation server. `Router` is
called on every incoming request and inits `Writer`
that sends `Trust`s to the next route point or
handle(logs in that implementation) them if current
node is the end point of the route.

Rename `onlyKeyRemoteServerInfo` struct for container
to separate it from the same implementation of the
same `ServerInfo` interface for reputation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Pavel Karpy
39f55611c6 [#460] reputation/local: Implement dependencies
Implement dependencies for local trust router
in `main` package. Managers are defined as
next node after the current one in HRW sorted
list of current NetMap.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Alex Vanin
7cf48d4d91 [#452] Update neo-go to latest master
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00