Implement Policer service that performs background work to check compliance
with the placement policy for local objects in the container. In the initial
implementation, the selection of the working queue of objects is
simplified, and there is no transfer of the result to the replicator.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define ErrNotFound error in headsvc package. Return ErrNotFound from Head
method if the header was not found in the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Wrong boolean operation order made matcher return false
on `non-root` search query with non-regular objects. Instead
it should return true for `non-root` query and false for `root`
query.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
If object service produces new request, the should contain
session token. This is the only way for node to grant access
for a private container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Listen and ListenWithError methods check if subscriber channel
has been closed. If so, ListenWithError passes error message
into provided channel.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
RPC node closes websocket notification channel if it was terminated
or something wrong happened. Subscriber has to check this condition
and alert about this in upper context by closing it's own channel.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Inner ring index will be used to access alphabet contracts.
First seven inner ring nods communicating with one alphabet
contract, depending on their index.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In #37 we've decided to remove length check, because smart contract would
fail on casting `nil` value from storage to `[]byte` producing FAULT state.
Apparently it does not fail, so we have to check length explicitly.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Classifier fetches public key of the request owner
and owner itself. Extended ACL check should rely on
this public key, because it might be extracted from
session token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
With one tombstone for split objects we can't simply
place it in container. We should inform all nodes that
store split objects of removed original object.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Owner of the request is stored in session token most of the times.
Put request contains session token in the object body, so we have
to fetch it from there.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
These buckets can be used to store blobs and metadata.
They will be removed as enhanced blob storage will be
implemented for neofs-node. To setup storage type
use `storage.object.type` and `storage.meta.type` params.
Available options:
- inmemory (default)
- boltdb
- filesystem
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
GRPC has default message limit of 4MiB. Since every transmitted
neofs message has to be signed, then original message should
be split into transfer fit structures before signature service.
This commit introduce transport payload splitter for object
service pipeline. This splitter works with stream response
for methods:
- object.Get
- object.Range
- object.Search
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation InitRelations call in payload size limiter was
called in write chunk method. This provoked clearing the split header in
children starting from the second.
Replace InitRelations call to the 1st child allocating stage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Refactor query to match object and its parents in a single call. Support
KeyRoot and KeyLeaf filters.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation object.Search services allowed to search only
physically stored objects. This limitation did not allow getting the ID of
the split object.
Extend search execution logic with parent object filtering. Parent objects
that passed filters are now included in the result
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Use key storage in object services in order to sign requests with private
session key within user session.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Linking object carries only identifier of the parent object. Fix Validate
method of FormatValidator to not validate parent object of the linking
object.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add session token argument to object formatter constructor which is written
to the object. Pass session token from trusted object Put.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Basic ACL checker gets request field via getters that are
NPE-free, therefore we don't need to worry about function
invocations on nil structures.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
ACL has to classify request senders by roles:
- owner of the container,
- request from container or inner ring node,
- any other request.
According to this roles ACL checker use different
bits of basic ACL to grant or deny access.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementation payload size limiter panicked in case of payload
emptiness. It was caused by the component waiting for at least one write of
a part of the payload.
Fix NPE occurrence with internal initialization after the WriteHeader call.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add SetSeekRange method to RangeTraverser that switches traverser to work
with provided object payload range.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation one excess element could be added to the chain.
Add previous sibling presence check to prevent this.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add FullRange option to get range operation parameters that allows to get
payload range [0:object_size] w/o the actual knowledge of the object size.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Head service receives right child of the processing object in some cases.
Add right child to Head result in order to use it as needed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Large inner ring requires more gas to make container registration.
Container contract makes balance transfers for each inner ring node
and it require extra gas to execute. This estimation should be enough
for seven inner ring nodes.
Later there should be heuristic evaluations for this: #47
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
In previous implementations object size limiter left checksums of parent
object payload unset. This was caused by the closure that written calculated
checksums to the child object instead of parent one.
Construct payload checksum writers in separate function in order to prevent
closure bug.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Container get wrapper implements container.Source interface
so it can be used in object service as container storage.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Attribute parser converts strings of attribute chain into
one-dimension array of NodeInfo attributes, that used
in network map.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Netmap source interface used by object service to build
placement based on current or previous network map.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Implement read-only local network address source. Implement function for
check if network address is local.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation session Storage.Get method took a v2 owner
identifier as an argument.
Change method signature to as an argument owner identifier from NeoFS SDK.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation traverser worked like all counts of all selectors
are equal to counts of corresponding replicas. Make traverser to take into
account select count of all replicas.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define object writer interface. Implement payload size limiter that
restricts payload size of the object. Implement object format moulder that
fill verification fields and finalizes object structure.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Move the serialization of the address and object into separate functions and
use them in methods. Use recently implemented RawObject constructor and
method in metaFromObject function.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Remove Address type. Makes Address method of the Object to return NeoFS SDK
Address type. Makes local storage to work with NeoFS SDK object address.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
JSON format is rather verbose an inconvenient to be
edited by hand. This commit implements SQL-like
language for representing placement policy.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
