In previous implementation Object service's handler returned const error in
case of failure (full or partial) of PUT operation. This did not even allow
us to roughly guess what the reason is. Not as a complete solution, but to
alleviate some cases where all nodes in a container return the same error,
it is suggested to return the error of the last server that responded.
Return latest server error from placement loop of `iteratePlacement` method
of `distributedTarget` type.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation sticky bit could disrupt access of container
nodes to replication. According to NeoFS specification sticky bit should not
affect the requests sent by nodes from SYSTEM group.
Add role check to `stickyBitCheck`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Some software components regulate the way of working with placement arrays
when a local node enters it. In the previous implementation, the locality
criterion was the correspondence between the announced network address
(group) and the address with which the node was configured. However, by
design, network addresses are not unique identifiers of storage nodes in the
system.
Change comparisons by network addresses to comparisons by keys in all
packages with the logic described above. Implement `netmap.AnnouncedKeys`
interface on `cfg` type in the storage node application.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to process public keys of the placement result.
Implement `Node.PublicKey` method which returns storage node's key announced
in netmap.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation `placement.Traverser.Next` method returned slice
of `network.AddressGroup` elements. There is a need to process keys of
storage nodes besides network addresses for intra-container communication.
Wrap `network.AddressGroup` in a new type `placement.Node` that summarizes
the storage node information required for communication. Return slice of
`Node` instances from `Traverser.Next` method. Fix compilation breaks in
dependent packages.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation Container service handlers didn't cache the
results of `Get` / `GetEACL` / `List` operations. As a consequence of this,
high load on the service caused neo-go client's connection errors. To avoid
this there is a need to use cache. Object service already uses `Get` and
`GetEACL` caches.
Implement cache of `List` results. Share already implemented cache of Object
service with the Container one. Provide new instance of read-only container
storage (defined as an interface)to morph executor's constructor on which
container service is based. Write operations remained unchanged.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation of Policer's job queue the same list of objects for
processing was selected at each iteration. This was caused by consistent
return of `engine.List` function.
Use `rand.Shuffle` function to compose pseudo-random list of all objects in
order to approximately evenly distribute objects to work.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Do not init caches for eACL, containers and netmap
if `disable_cache` config options is `true`, use
direct RPC calls instead.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Storage node should only reflect container Put requests to sidechain w/o any
validation. All verification steps are performed by IR.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Netmap service to return single network address on `LocalNodeInfo` RPC
in order to older clients.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to support multiple server endpoints for reputation and
container transmission.
Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make placement `Traverser.Next` method to return ``[]network.AddressGroup`
in order to support multiple addresses of the storeage nodes.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Head service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Search service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Get service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Put service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to generalize single-address client to group-address client.
To do this, we can re-implement `Client` interface from NeoFS API Go library
and still use it in the application code. There is a problem with method
`Raw` which must return single-address raw client. So as not to make changes
to API library we need to overload Client interface in order to support
`Raw` method in group-address client implementation.
Define `Client` interface in new `pkg/core/client` package. Completely
inherit API `Client` interface. Add `RawForAddress` method to build raw
client for the single node address. Adopt the application code that used Raw
method to work with new `Client`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
`network.Address` structure in most cases created once and used read-only.
Replace `AddressFromString` function with `Address.FromString` method with
the same purpose and implementation. Make all libraries to work with value.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Return tyype is clear from the method's signature and docs, there is no
point in reflecting it in the name.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Panic if internal `manet.DialArgs` call returns error since this is
unexpected according to `AddressFromString` implementation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Serve `ControlService` instance on configured endpoint (do not serve if not
specified). Read allowed keys from config.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement `ControlServiceServer` on `Server` type. The `Server` requires all
requests to be signed with keys from the so-called whitelist. To obtain
health status, it uses the abstraction in the form of `HealthChecker`
interface.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define `ControlService` for IR similar to the one from storage node. Add
`HealthStatus` RPC which returns health status of the IR application.
Implement getters, setters and methods to sign/verify the messages.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
NeoFS network dictates the limitation of the maximum size of the "finished"
objects. The payload size of an object is recorded in its header. When
writing finished objects, the nodes must check:
* satisfying the constraint;
* matching the value in the header to the number of payload bytes.
Provide value returned by `MaxSizeSource` component to `validatingTarget`.
Check max size value during the stream of the "finished" objects. Check
header value during the streaming and on-close. Check payload size in v2
relay scenario.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
If container is removed via session, then session token should be included
in removal witness.
Write session token from request meta header to `container.RemovalWitness`
structure which is passed to `wrapper.Delete` function.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
