Commit graph

51 commits

Author SHA1 Message Date
Leonard Lyubich
e473f3ac91 [#645] *: Use helper functions to build client.NodeInfo structures
Helper functions from core/client package allow to set public keys of
storage nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
7b228b7603 [#645] *: Construct clients from client.NodeInfo in API client cache
There is a need to have the ability to expand the data needed for client
construction.

Replace `network.AddressGroup` parameter of client cache interfaces with
`client.NodeInfo`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Alex Vanin
01df4ffa61 [#811] service/container: Hide cache invalidation logic in Writer interface
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:50:18 +03:00
Alex Vanin
49b6b5b49d [#811] cmd/node: Use cache invalidator in container service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:50:18 +03:00
Pavel Karpy
19e97e4d7c [#807] node: Fix notary container invocation
Create non-notary container wrapper and
use it in load announcing since it shouldn't
be invoked as notary request.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-09 18:37:45 +03:00
Pavel Karpy
2f343a15e5 [#770] node/cmd: Add TryNotary to SN container wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
d252aa4a3e [#770] pkg/morph: Rename all parsers and handlers structs/interfaces
Prepare all listening structures for notary events:
rename(add prefix/suffix 'notification') all
notification specific handlers/parsers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Leonard Lyubich
358e3ed8c4 [#645] *: Change the locality condition of the node from the placement
Some software components regulate the way of working with placement arrays
when a local node enters it. In the previous implementation, the locality
criterion was the correspondence between the announced network address
(group) and the address with which the node was configured. However, by
design, network addresses are not unique identifiers of storage nodes in the
system.

Change comparisons by network addresses to comparisons by keys in all
packages with the logic described above. Implement `netmap.AnnouncedKeys`
interface on `cfg` type in the storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-07 09:53:18 +03:00
Leonard Lyubich
e738699fcc [#676] services/container: Cache the results of read operations
In previous implementation Container service handlers didn't cache the
results of `Get` / `GetEACL` / `List` operations. As a consequence of this,
high load on the service caused neo-go client's connection errors. To avoid
this there is a need to use cache. Object service already uses `Get` and
`GetEACL` caches.

Implement cache of `List` results. Share already implemented cache of Object
service with the Container one. Provide new instance of read-only container
storage (defined as an interface)to morph executor's constructor on which
container service is based. Write operations remained unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-30 18:01:26 +03:00
Pavel Karpy
e8665f6cef [#730] node/morph: Use disable_cache config param
Do not init caches for eACL, containers and netmap
if `disable_cache` config options is `true`, use
direct RPC calls instead.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-03 09:40:10 +03:00
Alex Vanin
fd24a99533 [#694] cmd/neofs-node: Reuse single instance of client cache in all components
This will reduce amount of open connections up to 3 times.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-13 15:17:48 +03:00
Leonard Lyubich
8060735732 [#607] cmd/node: Serve gRPC on multiple interfaces
Generalize single gRPC interface of the storage node to a group of
interfaces. Each interface calls the same RPC handler.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
cede2b4ed7 [#607] reputation,container: Support address groups in ServerInfo
There is a need to support multiple server endpoints for reputation and
container transmission.

Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
6e5d7f84af [#607] network: Generalize LocalAddressSource to address group
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
e11f50ec8e [#607] network: Make ClientCache to accept AddressGroup
Change type of the `ClientCache.Get` method's parameter to `AddressGroup`.
Use `GroupFromAddress` to call the method from the wrappers in order to no
change their interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
adbbad0beb [#607] network: Do not work with Address pointers
`network.Address` structure in most cases created once and used read-only.

Replace `AddressFromString` function with `Address.FromString` method with
the same purpose and implementation. Make all libraries to work with value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
e5504c7130 [#607] network: Do not use Address.String for address comparison
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Evgenii Stratonikov
3f07313604 [#562] cmd/neofs-node: use NEP-6 wallet for keys
Also use neo-go private key wrapper where possible, as it
already has methods for (un)marshaling.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Leonard Lyubich
6279b6343f [#603] cmd/node: Separate configuration and netmap node info
Config `NodeInfo` should be used for bootstrap. Separete local node info and
netmap one. Return configured `NodeInfo` if structure from netmap is
missing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-11 15:36:17 +03:00
Leonard Lyubich
3dd10b6795 [#570] *: Remove usage of deprecated elements from API Go library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
9b2939d538 [#567] cmd/node: Close cached clients on shutdown
Call `CloseAll` on all `ClientCache` instances on application shutdown.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 13:11:37 +03:00
Pavel Karpy
33bef46f31 [#549] network/cache: Change Get signature
Make network cache's `Get` method accept
`network.Address` argument instead of
string.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Leonard Lyubich
83c27f6e8a [#505] morph/container: Change get container API
Make `Get` method of the wrapper over Container contract's client to
accept binary container ID. Create `Get` function similar to the previous
`Get` variation. Use this function in Container service server in the place
where `Get` method was used.

Additionally implement `AsContainerSource` function which allows
to simply compose container Source interface from the wrapper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Evgenii Stratonikov
2b2b2c2c45 [#496] Use single contract wrapper constructor
There is no need in a separate `New()` or `WrapClient()`

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-19 16:42:54 +03:00
Alex Vanin
16f13bc0a5 [#522] Use HostAddrString as RPC endpoint instead of IPAddrString
To enable TLS support we can't operate with IP addresses directly.
Certificates are issued with host names so it is required to
pass them into RPC client. DNS resolving should be done by transport
layer and not be a part of node. Therefore `IPAddrString` usage is
removed from code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Leonard Lyubich
3ef5b0ff9c [#493] node: Do not add fee in smart contract calls
Calls to contracts by storage nodes do not lead to the accumulation of
multisignatures in the contract memory, so the call cost can always be
accurately calculated in advance without additional fee.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Pavel Karpy
f02de33f8b [#476] cmd/containter: Make network handlers async
Make container handlers that do network
communication execute asynchronously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
91e0f4b370 [#476] cmd/container: Add async handlers
Add `addContainerAsyncNotificationHandler`
func that passed handler to container's
worker pool similar to how it is done in
reputation handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-15 15:11:34 +03:00
Pavel Karpy
60cc3b3e16 [#460] reputation: Add Router to reputation server
Add `Router` to the reputation server. `Router` is
called on every incoming request and inits `Writer`
that sends `Trust`s to the next route point or
handle(logs in that implementation) them if current
node is the end point of the route.

Rename `onlyKeyRemoteServerInfo` struct for container
to separate it from the same implementation of the
same `ServerInfo` interface for reputation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-08 17:29:08 +03:00
Leonard Lyubich
106884fc40 [#428] client: Hide client cache behind interface in dependent packages
Replace usage of `cache.ClientCache` type with interface with similar
signature. This will further allow overloading clients without affecting the
logic of dependent packages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-24 10:11:52 +03:00
Leonard Lyubich
383d2494eb [#425] Adapt the Client's refactoring to the interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Leonard Lyubich
718a2fad26 [#425] services: Define service interfaces that was removed from API lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Evgenii Stratonikov
cc7287d6f7 [#422] pkg/services: Cache clients by address only
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-03-15 14:07:53 +03:00
Leonard Lyubich
34a51ed38b [#419] cmd/node: Cache containers read from sidechain
Implement LRU cache for values read from the network. Implement LRU cache of
containers read from sidechain. Use it as a container storage in neofs-node
app. Current cache size is set to 100, TTL is 30s (constants).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich
bd5c70131b [#328] cmd/node: Serve Container.AnnounceUsedSpace RPC
Register recently implemented handler of AnnounceUsedSpace RPC in node app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
b270c49b5c [#328] container/load: Process Start/Stop notification events from contract
Construct used space Controller on node's app-side. Call Controller.Start on
StartEstimation event from sidechain. Call Controller.Stop on
StopEstimation event from sidechain.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
e9cbdc4a0d [#328] cmd/node: Add the ability to listen Container contract's events
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
8419294f22 [#328] container/load: Implement AnnounceUsedSpace NeoFS API v2 RPC handler
Implement processing of AnnounceUsedSpace RPC from NeoFS API v2 method on
the app-side according to the following algorithm (except common steps for
each RPC):

 * check if the request sender is presented in container;
 * check route of the request (compose from signatures);
 * pass the value to Writer returned by WriterProvider.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
10902c0b82 [#328] container/load: Implement RemoteWriterProvider for controller
Define component on app-side that implements RemoteWriterProvider with next
logic:

 * if ServerInfo is nil, call internal dead end provider;
 * if address in ServerInfo equals to local node's address, do nothing;
 * otherwise create API client instance via client cache and return the
   wrapper over it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
1bed03995b [#328] container/load: Implement Iterator on local storage engine
Implement wrapper over StorageEngine on app-side that iterates over the
values returned by ListContainers+ContainerSize method pair. Component will
be used as the source of local metrics.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
910017b72f [#328] container/load: Implement PlacementBuilder for used space router
Implement wrapper over netmap and container storages on app-side that builds
placement vectors by next steps:

 * read container from storage;
 * read network map from storage by epoch number;
 * build container nodes according to placement policy;
 * build placement vectors with epoch-dependent pivot.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
6f552c8599 [#328] container/load: Implement WriterProvider on API client
Implement wrapper over API client on app-side that passes used space values
to it. Values are buffered on Put call and sent on Close call.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
fa685160f9 [#328] container/load: Implement Writer on Container contract's client
Implement wrapper over Container contract's client on app-side that passes
used space values to it. Uses a fixed public key, whose value will be the
key of the local storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
19f9c7eacb [#83] services: Remove setting of meta header from executing services
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
Leonard Lyubich
3065a7363b [#83] cmd/neofs-node: Use response services in application
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
Alex Vanin
0e7e0bd2d6 [#84] Remove mocks and debug code from neofs-node services
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-09 09:15:18 +03:00
Leonard Lyubich
1d676fcfb2 [#69] object/acl: Add eACL components to service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
Alex Vanin
8585e147d0 [#43] cmd/neofs-node: Use morph wrappers as data sources
Morph wrappers are being uses as sources for containers and
snapshots of network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin
86b9aefcae [#28] Make storage node configurable
To run storage node at dev-env environment it should have
configurable parameters. To keep `cfg` structures we can
read configuration from env and yml config file with viper
and parse values such as script hashes, fees, keys into
`cfg` structures.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00