Pivot used to shuffle nodes in the CRUSH tree. This is
required argument. We use container ID value to select
container nodes, so `nil` value produces incorrect placements.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Replace static NodeInfo structure with NodeState interface that provides
method to read node information in runtime.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a codecov issue because objects are not placed
in the engine the same way every unit test. Therefore
sometimes there are more coverage, sometimes there are
less. Seeded RNG should solve this issue for engine tests.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Netmap snapshot table caches bootstrap `AddPeer` txs so inner ring
node does not produce redundant approval txs for bootstrapped nodes.
However if node updates states to `Offline`, then such node should
be flagged in snapshot table, so re-bootstrap will actually produce
approval tx.
`ev.PublicKey.String()` returns uncompressed representation of the
node's public key, while snapshot contains compressed values.
Therefore the node was not flagged and re-bootstrap tx was not
approved by inner ring nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add HealthStatus method to HealthChecker interface that should return
current health status of the node application.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement methods required for signatures. Receive network map from netmap
storage, convert it to Control service message and return in response.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add NetmapSnapshot rpc to ControlService protobuf definition. Recompile
proto files. Add required method to server structure.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Support processing of NetmapEpoch and NetmapLookupDepth X-headers when
processing object read operations. Placement for operations
Get/Head/GetRange/GetRangeHash/Search is built for the epoch specified in
NetmapEpoch X-header (by default latest). Also the specified operations are
processed until success is achieved for network maps from the past up to
NetmapLookupDepth value. Behavior for default values (zero or missing) left
unchanged.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add binary salt field to RangeHashPrm struct. Implement field setter. Set
salt from the request in v2 service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
We can't use enum values from SDK library directly, they can be
different from API specification. Therefore we need to convert them
into protocol level format.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation of eACL service v2 the response X-headers were
validated at the stage of re-checking eACL. This provoked a mismatch of
records in the eACL table with requests. Fix this behavior by checking the
headers from the request, not the response.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Forward request X-headers to client calls during internal processing of
Object operations on the node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Since PoR audit check uses object.Head with large TTL values to
make real proof of retrievability, we may use random remote
nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Audit task manager should not discard tasks if all workers are
busy, therefore pools should not be non-blocking.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
PDP audit check is not quite working with very small objects, so
we try to build coverage with bigger objects.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
This function converts 2-dimension array of container nodes
into single dimension array. Useful when we need to iterate
over whole container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add numeric return from TaskManager.Reset method that shows the number of
canceled tasks. This values will be used for assessment of the progress of
the audit.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define interface of the container communicator which methods are going to be
used in audit checks. Make innerring Server to implement this interface.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement Reset method on audit task manager that cleans task queue.
Extended TaskManager interface with Reset method on IR side. Call Reset
method in audit processor before new audit start.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation parent object header finalized twice in size
limiter + formatter. On the one hand, this added redundant action, on the
other hand, it could provoke a difference in the headers of the linking and
the last part. Change formatter to finalize parent header if it does not
container the signature. Change size limiter to reuse parent header after
last child finalization in linking child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation DB.Containers method could return an error about
invalid container ID string format. This could happen if some of top-level
buckets had name w/o "_" substring.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation eACL validator didn't take into account container
and object ID fields of request bodies.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Every unknown error must not decrease shortage counter and must not
exclude faulty node from the node list, because this list will be used
later for replication.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Deadlock occurs when `getActivate` function opens new blobovnicza and that
invokes evict in LRU cache of open blobovniczas. `getActivate` makes
`activeMtx.Lock()` and then cache evict makes `activeMtx.RLock()` and deadlock
happens.
Fix contains two steps:
- add separate mutex to open blobovniczas (1),
- split single Lock outside of `updateAndGet` (2).
As for the (1) `bbolt.Open()` locks when it tries to open the same file from
two threads. So separate mutex will prevent that.
As for the (2) `updateAndGet` function contains from two parts. At first it
checks if required blobovnicza is ready and it returns it. In this case we can
use the simple RLock. But then there is an option when we should open new
blobovnicza and update map of active blobovniczas.
In this case we call `openBlobovnicza` without activeMtx lock. Cache evict
happens there and it won't cause deadlock.
Then we lock activeMtx to update the map of active blobovniczas. Concurrency can
happen there. However `openBlobovnicza` will not open the same blobovnicza twice,
so we can make one more check if opened blobovnicza was activated while thread was
locked in activeMtx. If so, then return active blobovnicza, else finish activation.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation Blobovnicza could incorrectly initialize
dimensional buckets: if SmallSizeLimit = 2 ^ X + Y && Y < 2 ^ X, then
largest dimensional bucket was [2 ^ (X - 1) : 2 ^ X]. This was caused by an
incorrect condition for stopping the iterator along the dimensional
boundaries.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
All parameters and resulting values of all metabase operations are
structured in new types. The most popular scenarios for using operations are
moved to auxiliary functions.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
With the update of the local storage engine, the headers of virtual objects
are directly given. In this regard, the step with obtaining the the right
child header is removed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation Blobovnicza's stored objects in protocol format
which did not allow working with externally compressed objects. To achieve
this goal, operations Get and Put no longer work with the structure of the
object, but only with abstract binary data. Operation GetRange has become
incorrect in its original purpose to receive the payload range. In this
regard, BlobStor receives the payload range of the object through Get
operation. In the future either Blobovnicza will learn to compress objects
by itself, or the GetRange operation will be eliminated.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Blobovnicza returns object, so we can't put compressed
data there. Compressed data won't be deserialized correctly.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Replace ErrNotFound and ErrRangeOutOfBounds to core/object package in order
to share them across the libraries.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add blobovnicza instance to BlobStor structure. Create blobovnicza tree in
BlobStor constructor. Implement Open/Init/Close methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a need to support single blobovnicza in blobovnicza tree. This can
be achieved with a width of 1, and a depth of 0 or 1. With depth = 1 one
redundant directory is created, inside which there is a blobovnicza. If the
depth is zero, the blobobnivza will be in the root path. Fix negative
capacity in iterateDeepest method with zero depth.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
With exist check we should index parent first, because
as soon as child will be added to metabase, exist on
parent will return true even if it was not indexed yet.
Also this commit makes one db.Update instead of two for
parent and child.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Place the root of blobovnicza tree in a subdirectory of BlobStor with same
permissions. Abolish WithBlobovniczaRootPath and WithBlobovniczaPersmissions
options.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Blobovnicza ID parameter provides the ability to specify particular
blobovnicza to delete object from. In this case only specified blobovnicza
is processed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation objects were classified by size according to
payload size. From now they are classified by the size of their binary
representation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement "big or small" property classifier (only the size of the payload
is temporarily considered). Save "big" objects in shallow dir. Save "small"
objects in shallow dir until the moment of implementation of blobovnicza.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement Put/Get/GetRange/Select/SelectAll functions over storage
engine. These functions are going to be used by Object service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation Shard accessed the BlobStor to get the
object header. However, the shard must take headers from the metabase.
From now zero length of the requested payload range seens as object
header request. In this case shard calls metabase to get the header.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation WithCompressObjects returned Option
than could panic if zstd (de)compressor creation failed with error.
From now errors with (de)compressor creation result in an option
without using data compression. In this case, the error is written
to the log passed to the option constructor.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation each operation on local storage
locked engine mutex. This was done under the assumption that
the weights of the shards change as a result of write operations.
With the transition to static weights of shards, it is no longer
necessary to lock the global mutex during the execution of operations.
However, since the set of engine shards is dynamic, there is still a
need to control multiple access to this set. The same mutex is used
for synchronization.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation each shard operation locked
RW shard mutex. With this approach RW operations were executed
one-by-one and blocked the execution of RO operations.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Correct the calculation of maximum value of fs tree depth. Fix check
of the max depth overflow in WithShallowDepth function.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
GetPrm has WithPayloadRange option to specify the requested
payload range. In previous implementation StorageEngine.Get
method ignored this option. From now zero length matches
full payload request.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Now alphabet contracts use signature collection to make
a vote. All inner ring nodes votes for some list of validators
the same way as they vote fore new epoch or new container.
As soon as list is accepted by alphabet contracts, each of them
votes for one candidate from the list.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
With this argument neofs-ir will be run as an CLI
application that initialize inner ring server, invokes
`vote` method on corresponding alphabet contract and
exits.
User can provide a comma-separated list of validator
public keys.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
This function calculates validator key based on inner ring index
and invokes `Vote` method of corresponding alphabet contract.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation linking object inherited only ID of the parent
object. From now it inherits full header like last child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation child objects inherited parent attributes after
split-transformation, which was redundant. From now attributes are not
inherited.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation placement traverser processed incorrectly with
local placement build. Also entity incorrectly traversed the placement
vectors for fixed number read operations until success. The erroneous
behavior was due to the use of a vector number of successes instead of
a scalar number in these scenarios.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
With updated specification of object related operation
we don't have this search attribute any more and we
should not use functions related to this attribute.
This commit breaks object service logic, however it will
be fixed later.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
ErrEACLNotFound error was defined in implementation package. EACL validator
checked this error after the call of eACL storage interface method. Replace
ErrEACLNotFound to core container library. in order to: on the one hand not
use an implementation error, on the other hand, to be able to reuse a
generic type error (404).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Now morph library returns error if there is not eACL in
sidechain storage. However in this case eACL check should
be passed since it is the same as having empty eACL table.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Sticky bit checks if object owner and request owner are the
same. Container owner should not used in this check.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
`owner` field may be misused as request sender owner, however
it is a owner of a container for that request. New naming
should be clear.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
With this package we can reuse already created connections.
Later on neofs-api-go will support checking connection health
and `Close` operation, so this cache could run worker and remove
unhealthy clients.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Now root and phy (leaf) filters work like flags. They work with
any matcher and any value. So meta-storage sets `true` value for
all root and phy objects and puts them into separate bucket.
We also do not work with inversion anymore, so it either added
to the bucket or not. We don't need to store both options.
This is the reason `selectAll` function is changed a bit. Now
it performs some low-level parsing from primary bucket and root
bucket.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Keyer prints information about private key, public key,
NEO3 Wallet, scripthash. It can generate new private key
or generate multisig address.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Revert commit 0faa40e4 to increase the disk space consumed by the
metabase in favor of the speed of index updates.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In the previous implementation of the metabase, it was necessary to write
virtual objects to the primary index to be able to select them. In this
approach, virtual objects can be obtained directly using Head operation.
This has a side effect in handling object operations that do not expect to
receive a virtual object header in a single operation. With recent changes,
it is no longer necessary to have records of virtual objects in the primary
index, so this no longer happens for system integrity.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define ErrRelationNotFound error in searchsvc package. Return
ErrRelationNotFound from RelationSearcher.SearchRelation method if search
result is empty.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Refactored pkg/netmap package provides JSON converters for
NodeInfo and PlacementPolicy structures, that has been used
by client applications.
It also updates Node structure itself so it is a part of
grpc <-> v2 <-> pkg conversion chain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
NeoFS contract produces event with three arguments:
- update ID,
- config key,
- config value.
Update ID is a unique shared by inner ring holders
byte sequence that is used to update NeoFS runtime
configuration.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Fix a bug in the selection when removed object that matches search query
provoked the return of an empty result.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Fix a bug in the selection when an object could be added to the result after
a mismatch in the previous filter.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In the previous implementation of the metabase, the unique value of the
header was assigned a bucket, the elements of which were leaves with a
key-address and an empty value. This approach was relatively efficient in
terms of write speed. However, a large number of buckets led to a rapid
increase in the database volume (~4GB for 100K objects with unique
attributes). An approach is presented with storing indexes on the value of
headers in the leaves of the tree, where the keys are the unique values of
the header, and the values are a serialized list of addresses (gob
encoding is temporarily used for serialization).
The new approach gave a good result in saving space (~350MB), however, it
significantly reduced the write speed with an increase in the number of
objects (~ 80x after 100K objects).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
The previous metabase implementation took an exclusionary approach: filters
narrowed the set of all objects to those that match all filters. An
inclusive approach is presented. In it, when traversing the indexed headers,
the object becomes a candidate for selection. If at least one of the
subsequent filters is not passed, the object ceases to be a candidate. At
the end of the traversal, the remaining candidates are added to the
resulting sample. The borderline case of no filters is handled in a special
way: all stored objects are added to the resulting selection.
Presented inclusive approach showed better performance in most scenarios
(although not all).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous metabase implementation the absence of an attribute presented in
the search filter did not exclude the object from the result. Change this
behavior to exclude the object from the result.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Replace meta Bucket with meta.DB instance in local storage implementation.
Adopt all dependent components to new local storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Process parent objects in Put method. Headers of parent object are stored as
regular leaf objects in metabase from now. Build indexes for ROOT, LEAF and
CHILDFREE properties.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Call CheckFormat function in container.Put handler for conducting initial
checks of the structure that are not performed by the smart contract.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
With neo-project/neo-modules#358 nodes support RPC to
return used magic number. Therefore client doesn't need
that configuration value any more.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Neo-go does not use smartcontract.Parameter to return values
anymore, so it's convertes partly removed from neofs-node.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
There is an optimization to send ApprovePeer tx only in
case if `touch` returns false. It returns false if node
is not in the cache or it was flagged to removal.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
This event will be produced by new epoch handler. All
cleanups should be synchronous because smart contract can't
store inner ring votes forever. Therefore voting should be done
in similar time interval for all inner ring nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Cleanup table is a cache for inner ring node to look for netmap
snapshot. It updates access time of bootstrapped nodes and will
be used to send `updateState` txs to clean netmap from unresponsive
nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Implement Delete method on DB structure that adds deleted addresses to
tombstone index. Do not attach addresses from tombstone index to Select
result. Return error from Get method if address is presented in tombstone
index.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement bolt-based metabase that is going to be used in local object
storage. Implement Put/Get/Select methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Make Policer to call AddTask method of Replicator when an insufficient
number of copies of an object is detected in the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement Replicator service that performs background work to replicate
local object to remote nodes in the container. Replicator is going to be
used by Policer.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
There is a bit to allow or deny bearer token check for
each object service method. If this bit is not set then
ignore bearer token and use extended ACL table from
sidechain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
If bearer token is presented in the request then check
if it is a valid one and then use it to process extended
ACL checks.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Malicious user can stole public session key and use
it by sending request from it's own scope. To prevent
this each session token is signed and signature private
key must be corresponded with owner id in token. Therefore
malicious node cannot impersonate request without private
key to sign token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Implement Policer service that performs background work to check compliance
with the placement policy for local objects in the container. In the initial
implementation, the selection of the working queue of objects is
simplified, and there is no transfer of the result to the replicator.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define ErrNotFound error in headsvc package. Return ErrNotFound from Head
method if the header was not found in the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Wrong boolean operation order made matcher return false
on `non-root` search query with non-regular objects. Instead
it should return true for `non-root` query and false for `root`
query.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
If object service produces new request, the should contain
session token. This is the only way for node to grant access
for a private container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Listen and ListenWithError methods check if subscriber channel
has been closed. If so, ListenWithError passes error message
into provided channel.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
RPC node closes websocket notification channel if it was terminated
or something wrong happened. Subscriber has to check this condition
and alert about this in upper context by closing it's own channel.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Inner ring index will be used to access alphabet contracts.
First seven inner ring nods communicating with one alphabet
contract, depending on their index.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In #37 we've decided to remove length check, because smart contract would
fail on casting `nil` value from storage to `[]byte` producing FAULT state.
Apparently it does not fail, so we have to check length explicitly.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Classifier fetches public key of the request owner
and owner itself. Extended ACL check should rely on
this public key, because it might be extracted from
session token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
With one tombstone for split objects we can't simply
place it in container. We should inform all nodes that
store split objects of removed original object.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
