There are two notifications:
- start estimation notification produced at the beginning of the
epoch,
- stop estimation notifications should be produced before
basic audit settlement starts.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
This small refactoring adds `blocktimer.go` file with
all timer related function and constructors. This way
we can create all timers in one place (at the end of
innerring.Server constructor).
To do that we had to move timer reset into global
server state so it can be accessed by netmap
processor.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
There is no point of making separate `Fee` and `NoFee`
wrappers because all reading operations are free disregarding
of fee value in static client. However we can use these same
wrappers so send transaction.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Pass handler of audit settlement event to netmap event processor. Generate
AuditEvent in during new epoch processing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define a processor of events related to monetary transactions. Define
audit-related event. Provide an interface for processing the audit payout
event.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Use client with extraFee instead of readOnlyFee. Rename
NewNoFeeBalanceClient to NewBalanceClient since no-fee client is no longer
used.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Do not pass zero transfers from the calculation table to Exchanger. Revert
transfers with negative amount since Exchanger interface requires positive
amounts of funds.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Move control of the minimum cost of payment for the audit from the
implementation of the Exchanger to the place where the amount is calculated.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Calculate payments to storage nodes for the passed audit when changing the
epoch. The calculation results are wrapped in a call to the Balance contract
(one transaction per user-to-user transfer).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement component that analyzes audit results and generates transactions
for payment of awards for successfully passed audit. When calculating the
total fee, the declared price of the node (attribute) and the total volume
of storage groups, which were successfully audited by the container, are
taken into account. In one call the calculator processes all audit results
for the previous epoch (relative to the calculated parameter).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation handler was not called more the one time after
Reset. It was caused by tick counter not being reset inside Reset method.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Container listing already supported in the metabase for `engine.List`
operation. To get container statistics engine should provide both the
option to get container volume estimation and list of all containers.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Objects of one container can be split among shards, so engine
should iterate over all available shards to sum all size
estimations.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Storage nodes keep container size estimation so they
can announce this info and hope for some basic income
settlements. This is also useful for monitoring.
Container size does not include non regular or inhumed
object sizes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Listen to new blocks from mainnet until the required fix in neo-go lib is
released (https://github.com/nspcc-dev/neo-go/pull/1687).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace WSClient.Init call from the body of BlockNotifications method to
constructor New since Init should be called before working with client
according to neo-go docs.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Write error returned by BlockNotifications() call ins listenLoop method body
to error channel only if it is procided. Otherwise, write debug log message.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace block channel's initialization from BlockNotifications method body
to Subscriber's constructor in order to prevent potential writing to nil
channel.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Call handler of the fractional block interval once between base interval
ticks by default. Add option to call handler of fractional block interval
multiple times (N times if fractional interval == BASE_INTERVAL / N).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Extend Listener with RegisterBlockHandler method. All block handlers are
called on each block read from Subscriber.BlockNotifications channel.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Call SubscribeForNewBlocks in Subscriber's constructor. Provide
BlockNotifications interface method that returns block channel. Write new
blocks to the channel on notification events with BlockEventID type.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Classifier looks at list of inner ring nodes and container
nodes from current and previous epoch to classify request.
Sometimes these checks might return error.
Consider there is a request from unknown key and container's
placement policy valid for current epoch and invalid for past
epoch. Classifier tries to find if key belongs to container
node from current epoch -- it is not. Then it tries to find if
key belongs to container node from past epoch and it throws
error, because placement policy is invalid for past epoch.
This is a legit case and classifier should ignore such errors
to provide best effort in matching. The only error classifier
should return is an error when request does not contain
public key to classify it.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Pivot used to shuffle nodes in the CRUSH tree. This is
required argument. We use container ID value to select
container nodes, so `nil` value produces incorrect placements.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Replace static NodeInfo structure with NodeState interface that provides
method to read node information in runtime.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a codecov issue because objects are not placed
in the engine the same way every unit test. Therefore
sometimes there are more coverage, sometimes there are
less. Seeded RNG should solve this issue for engine tests.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Netmap snapshot table caches bootstrap `AddPeer` txs so inner ring
node does not produce redundant approval txs for bootstrapped nodes.
However if node updates states to `Offline`, then such node should
be flagged in snapshot table, so re-bootstrap will actually produce
approval tx.
`ev.PublicKey.String()` returns uncompressed representation of the
node's public key, while snapshot contains compressed values.
Therefore the node was not flagged and re-bootstrap tx was not
approved by inner ring nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add HealthStatus method to HealthChecker interface that should return
current health status of the node application.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement methods required for signatures. Receive network map from netmap
storage, convert it to Control service message and return in response.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add NetmapSnapshot rpc to ControlService protobuf definition. Recompile
proto files. Add required method to server structure.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Support processing of NetmapEpoch and NetmapLookupDepth X-headers when
processing object read operations. Placement for operations
Get/Head/GetRange/GetRangeHash/Search is built for the epoch specified in
NetmapEpoch X-header (by default latest). Also the specified operations are
processed until success is achieved for network maps from the past up to
NetmapLookupDepth value. Behavior for default values (zero or missing) left
unchanged.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add binary salt field to RangeHashPrm struct. Implement field setter. Set
salt from the request in v2 service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
We can't use enum values from SDK library directly, they can be
different from API specification. Therefore we need to convert them
into protocol level format.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation of eACL service v2 the response X-headers were
validated at the stage of re-checking eACL. This provoked a mismatch of
records in the eACL table with requests. Fix this behavior by checking the
headers from the request, not the response.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Forward request X-headers to client calls during internal processing of
Object operations on the node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Since PoR audit check uses object.Head with large TTL values to
make real proof of retrievability, we may use random remote
nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Audit task manager should not discard tasks if all workers are
busy, therefore pools should not be non-blocking.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
PDP audit check is not quite working with very small objects, so
we try to build coverage with bigger objects.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
This function converts 2-dimension array of container nodes
into single dimension array. Useful when we need to iterate
over whole container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add numeric return from TaskManager.Reset method that shows the number of
canceled tasks. This values will be used for assessment of the progress of
the audit.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define interface of the container communicator which methods are going to be
used in audit checks. Make innerring Server to implement this interface.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement Reset method on audit task manager that cleans task queue.
Extended TaskManager interface with Reset method on IR side. Call Reset
method in audit processor before new audit start.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
