Announced keys of storage nodes are required for many application components
to function.
Define a unified interface for the utility for working with public keys of
nodes. Add a method to check if the key has been advertised by the local
node in the application context.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to process public keys of the placement result.
Implement `Node.PublicKey` method which returns storage node's key announced
in netmap.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation `placement.Traverser.Next` method returned slice
of `network.AddressGroup` elements. There is a need to process keys of
storage nodes besides network addresses for intra-container communication.
Wrap `network.AddressGroup` in a new type `placement.Node` that summarizes
the storage node information required for communication. Return slice of
`Node` instances from `Traverser.Next` method. Fix compilation breaks in
dependent packages.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
IDs come from NeoFS contract in big endian, but it is customary to write in
the node logs in little endian.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Shard should try to read object headers from write-cache if it is enabled.
Extend `writecache.Cache` interface with `Head` method. Call the method in
`Shard.Head` if `Shard.hasWriteCache` returns true.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Write cache should be able to execute HEAD operations according to spec.
Add simple implementation of `Head` method through the `Get` one. Leave
notes for future optimization.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Caching is performed inside `GetNativeContractHash` method of neo-go client,
so the additional cache level is redundant.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is no need to continue iterating over Neo RPC endpoints in case of
some address-independent error (e.g. NeoFS logic error).
Unwrap and immediately return `neofsError` errors from loop in
`iterateClients`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement `error` interface on new `neofsError` type which is a wrapper over
NeoFS-specific error. Wrap all `Client` errors except neo-go client API ones
into `neofsError`. Wrapped errors are going to be used for multi-endpoint
loop control.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Since morph `Client` works in multi-client mode, there is an error case when
we can not get network magic when all endpoints are unavailable.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to work with a set of Neo RPC nodes in order not to depend
on the failure of some nodes while others are active.
Support "multi-client" mode of morph `Client` entity. If instance is not
"multi-client", it works as before. Constructor `New` creates multi-client,
and each method performs iterating over the fixed set of endpoints until
success. Opened client connections are cached (without eviction for now).
Storage (as earlier) and IR (from now) nodes can be configured with multiple
Neo endpoints. As above, `New` creates multi-client instance, so we don't
need initialization changes on app-side.
`Wait` and `GetDesignateHash` methods of `Client` return an error from now
to detect connection errors. `NotaryEnabled` method is removed as unused.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
As soon as resulting list and cache operate with the attribute pointer,
we can add attribute structure immediately and set restored key and
value of the attribute later.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Attributes are linked to each other through parents, so they can
be returned in any order. However, it will be better to return
the list in consistent order to reduce entropy.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Parser should reuse existing attributes from the cache to update list
of the parent links in it. Parent links should be unique.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation Container service handlers didn't cache the
results of `Get` / `GetEACL` / `List` operations. As a consequence of this,
high load on the service caused neo-go client's connection errors. To avoid
this there is a need to use cache. Object service already uses `Get` and
`GetEACL` caches.
Implement cache of `List` results. Share already implemented cache of Object
service with the Container one. Provide new instance of read-only container
storage (defined as an interface)to morph executor's constructor on which
container service is based. Write operations remained unchanged.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation of Policer's job queue the same list of objects for
processing was selected at each iteration. This was caused by consistent
return of `engine.List` function.
Use `rand.Shuffle` function to compose pseudo-random list of all objects in
order to approximately evenly distribute objects to work.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
To encode attributes with semicolon or slash, use
backslash as escaped character.
Example:
User-Agent:NeoFS\/0.23
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Do not init caches for eACL, containers and netmap
if `disable_cache` config options is `true`, use
direct RPC calls instead.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Do not read `without_notary` config value from env.
Make morph client constructor return client without
notary support. Enabling notary support should be done
with public `EnableNotarySupport` method separately.
Notary availability is deducted with client. Further,
if notary is presented on chain its support is
enabled at the corresponding client.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add `MkdirAllX` function that wraps `os.MkdirAll`
and adds `x` permission for user and group.
This is used to make all created dirs openable
for user and group regardless of the read config
permissions.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
When Inner Ring node starts, it should sync nearest epoch tick
event based on the block of the latest epoch. Otherwise epoch
ticking can be stopped, because ballots or notary transactions
are valid for limited period of time.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Get innerring list and committee in innerring
indexer from interfaces, not from structs.
Passing IR fetcher to indexer constructor depends
on having Notary contract in sidechain(use NeoFS
role or netmap contract).
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add `IrFetcherWithNotary` and `IrFetcherWithoutNotary`
that can fetch IR keys with and without usage of
notary contract.
Both can be hidden behind
`InnerRingKeys() (keys.PublicKeys, error)` interface.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add initialization of `updateInnerRing` method name
in config on startup.
Rename var `setInnerRing` => `updateInnerRing`.
Rename method `SetInnerRing` => `UpdateInnerRing`
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
We should go through every key in main chain list to merget lists
as fast as possible. Previously we drop main chain traversing as
soon as we have no more new keys to add. Instead we should try
to go for old keys in the list and add it as more as we can.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add sorting of passed public keys in `newAlphabetList`
function before Alphabet list merge to make returned
results more expected.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
If `WITHOUT_MAINNET` environmental variable is
`true`:
- Do not read `NeoFS` and `processing`
script-hashes from envs;
- Do not init Governance processor;
- Do not init NeoFS processor.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Storage node should only reflect container Put requests to sidechain w/o any
validation. All verification steps are performed by IR.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Container can have version later than the current lib version, and it should
not be considered as invalid. But it should still have adequate version.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Create `version` package and implement `IsValid` function which checks if
version is not earlier than the genesis one.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Audit client is used to send audit results and they should be
signed by inner ring node itself on order to be saved in
smart contract.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add `netmapCandidates` method to `netmap` client
wrapper. Method parses node storages candidates
for the next epoch.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Changes:
* replace `iotuil` elements with the ones from `os` package;
* replace `os.Filemode` with `fs.FileMode`;
* use `signal.NotifyContext` instead of `NewGracefulContext` (removed).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Netmap service to return single network address on `LocalNodeInfo` RPC
in order to older clients.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement `sort.Interface` interface on `AddressGroup` and perform sorting
in `AddressGroup.FromIterator` method. Addresses with enabled TLS are "less"
in terms of slice position.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to support multiple server endpoints for reputation and
container transmission.
Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make placement `Traverser.Next` method to return ``[]network.AddressGroup`
in order to support multiple addresses of the storeage nodes.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Head service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Search service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Get service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Object Put service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
