Commit graph

409 commits

Author SHA1 Message Date
Leonard Lyubich
ed808c3f1b [#419] eacl: Replace eACL storage implementation to app-side
Replace `eacl.Storage` interface implementation from eACL lib to neofs-node
app package. Remove `eacl.WithMorphClient` option in order to abstract eACL
validator from eACL storage implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich
5900975d58 [#217] object/policer: Leave readability instead of performance comment
Right now we pass redundant copy to callback outside the for-loop through
the helpful boolean variable instead of calling it deeply nested.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 20:59:14 +03:00
Leonard Lyubich
277e3ca20a [#217] policer: Handler redundant local copy of the object
Detect redundant local copy of the object in Object Policer. Add redundant
copy callback (`WithRedundantCopyCallback` option). Pass address of the
redundant copy to callback.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 20:59:14 +03:00
Leonard Lyubich
aa24702ebe [#401] control: Serve DropObjects RPC
Re-compile protobuf definition of Control service. Implement required
messages on DropObjects RPC request and response messages. Implement
`DropObjects` method on Control service server of the node. Use
`StorageEngine.Delete` method as a deleted object handler on server.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 13:07:53 +03:00
Leonard Lyubich
f2337baedc [#401] control: Add RPC to mark objects to be removed from the node
There is a need to define storage node's interface to remove objects from
the local storage.

Define `DropObjects` RPC in `ControlService`. Define `DropObjectsRequest` /
`DropObjectsResponse` structure of the request / response messages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-24 13:07:53 +03:00
Leonard Lyubich
9073e198b9 [#398] cmd/node: Serve NetmapService.NetworkInfo RPC
Implement `NetworkInfo` calls on full stack of Netmap services. Current
epoch is read from node local state, magic number is read via `MagicNumber`
call of morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 19:16:40 +03:00
Leonard Lyubich
b8d1144839 [#378] object/delete: Set expiration epoch of the created tombstones
Make object delete service to use network information to calculate and set
expiration of the created tombstone.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 11:56:32 +03:00
Leonard Lyubich
38727c2930 [#368] object: Reject expired objects
The lifetime of an object can be limited by specifying a correspondin
well-known attribute. Node should refuse to save expired objects.

Checking objects in FormatValidator is extended with an expiration attribute
parsing step.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-15 19:04:33 +03:00
Leonard Lyubich
f85e5f0238 [#202] object/eacl: Fix algorithm for checking eACL table
Add SignWithRFC6979 option to signature verification function since eACL
table are signed by users with this option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-03 15:14:59 +03:00
Leonard Lyubich
ad348afcd6 [#202] object/eacl: Verify signature of eACL table
Since the contract started returning the table signature, it became
necessary to check its correctness.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-03 15:14:59 +03:00
Leonard Lyubich
bd5c70131b [#328] cmd/node: Serve Container.AnnounceUsedSpace RPC
Register recently implemented handler of AnnounceUsedSpace RPC in node app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
21133aee2f [#328] container/load: Implement route builder based on placement
Implement route Builder interface on wrapper over the container placement
builder, Component implies exactly one transfer to each of the most weight
nodes of the container (according to some weighing algorithm).
Implementation is planned for use when transferring local estimates of
storage nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
d48fb81193 [#328] container/load: Implement route controller
Implement a component for transmitting the value of the used container space
along a route defined in the system. Implement WriterProvider interface on
it. By implementation, it is the link between the route planner and the
point-to-point transmitter, and abstracts from the implementation of both of
them. In the future, this implementation will be used as a transmitter of
local estimates of storage nodes among themselves.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
4415f8dc5b [#328] container/load: Implement simple Iterator and Writer providers
Implement functions to wrap Writer or Iterator. The resulting wrapper
provides WriterProvider or IteratorProvider interface respectively.
Such a wrapper can be used as a single storage instance provider
regardless of context.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
2622e11ae3 [#328] container/load: Implement local storage of used space announcements
Implement a component that stores the values of the used space of
containers. The storage allows you to write several values for a fixed
container and epoch number, and read the averaged estimates of all
accumulated values. All values are stored in memory. This component is
planned to be used as an accumulator of opinions from various network
participants about the fullness of the container.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Leonard Lyubich
cc88320d6b [#328] container/load: Implement a metrics exchange controller
Implement a component that connects the value stores of the used space of
containers. Implement the Start/Stop operations on it, which will later
become the application handlers of the corresponding events from the
sidechain. The main task of the controller is to temporarily synchronize the
stages of calculating the global estimate of the used space in the
container. The details of the score calculation (the way of collecting /
transmitting local scores, the final score formula and writing to the
contract) are encapsulated in the dependency components, the controller is
abstracted from them.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Alex Vanin
e252f93dbf [#350] object/acl: Make best effort to classify request
Classifier looks at list of inner ring nodes and container
nodes from current and previous epoch to classify request.
Sometimes these checks might return error.

Consider there is a request from unknown key and container's
placement policy valid for current epoch and invalid for past
epoch. Classifier tries to find if key belongs to container
node from current epoch -- it is not. Then it tries to find if
key belongs to container node from past epoch and it throws
error, because placement policy is invalid for past epoch.

This is a legit case and classifier should ignore such errors
to provide best effort in matching. The only error classifier
should return is an error when request does not contain
public key to classify it.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-27 20:18:33 +03:00
Leonard Lyubich
c77d346016 [#339] services/netmap: Use dynamic node information
Replace static NodeInfo structure with NodeState interface that provides
method to read node information in runtime.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-25 17:13:46 +03:00
Leonard Lyubich
619f8826e1 [#315] control: Implement SetNetmapStatus on Server
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 17:46:12 +03:00
Leonard Lyubich
f39d08bda7 [#315] control: Add SetNetmapStatus rpc
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 17:46:12 +03:00
Leonard Lyubich
e5dc4ede57 [#312] control: Add newline at the end of proto file
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
f327024ee7 [#312] control/server: Extended HealthChecker interface
Add HealthStatus method to HealthChecker interface that should return
current health status of the node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
6ae51e41f9 [#312] control: Add health status field to health check response body
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
0abb8acef3 [#312] control: Define HealthStatus enum
Define enumeration of the statuses of storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Leonard Lyubich
83d4420a30 [#313] control: Rename HealthStatus enum to NetmapStatus
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:52:08 +03:00
Alex Vanin
a89567a88d [#317] morph/client: Return complete eACL signature from contract
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-15 16:25:49 +03:00
Leonard Lyubich
c75a828adf [#311] control: Fix loss of parent attribute keys in netmap converter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
6c9dab481c [#311] control: Cover StableMarshal methods with unit tests
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
5f22e7fac7 Add newline at end of Control service proto file
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
36d8e2d3b7 [#311] control: Implement JSON marshaler on Netmap message
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
dde24bc9e5 [#311] control: Rewrite methods for binary encoding and signatures
Rewrite marshalers and sizers using proto library from API repo.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
e9d4b2d25b [#311] Recompile protobuf files
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
3e882d963f [#311] services/control: Implement NetmapSnapshot method
Implement methods required for signatures. Receive network map from netmap
storage, convert it to Control service message and return in response.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
28777911fb [#311] services/control: Define NetmapSnapshot rpc
Add NetmapSnapshot rpc to ControlService protobuf definition. Recompile
proto files. Add required method to server structure.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
88023f3655 [#311] services/control: Define Netmap structure
Define NodeInfo protobuf type. Define Netmap protobuf type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
f311be3ab6 [#234] Fix ineffassign linter remark
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
0a0ee89665 [#306] control: Rename WithAllowedKeys function to WithAuthorizedKeys
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
abd9952e46 [#306] Rename Private service to Control service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
44a0fb5a69 [#306] private: Define and use HealthChecker interface to get the status
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
c1b8a4815f [#306] private: Implement server of gRPC private node service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
e75ddb0549 [#306] private: Implement setters on generated proto messages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
85ec633938 [#306] Define and compile proto files for private node service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
1e170c3812 [#234] services/object: Support netmap epoch and lookup dead in read ops
Support processing of NetmapEpoch and NetmapLookupDepth X-headers when
processing object read operations. Placement for operations
Get/Head/GetRange/GetRangeHash/Search is built for the epoch specified in
NetmapEpoch X-header (by default latest). Also the specified operations are
processed until success is achieved for network maps from the past up to
NetmapLookupDepth value. Behavior for default values (zero or missing) left
unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Leonard Lyubich
2f4d90025f [#234] core/netmap: Extend Source interface
Add GetNetMapByEpoch method. Add Epoch method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Alex Vanin
e88c1f750d [#302] Fix golint else/return linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
74a68af5c2 [#302] Fix golint underscore linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
04c0c1b8f5 [#302] Fix errcheck linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
5e231c515a [#302] Fix whitespace linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Leonard Lyubich
3a7f6701d8 [#290] object/rangehash: Apply salt to data before hashing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich
0e2c761481 [#290] object/rangehash: Add salt to RangeHashPrm structure
Add binary salt field to RangeHashPrm struct. Implement field setter. Set
salt from the request in v2 service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich
9a86fff7e0 [#291] Remove some unused code from repository
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
a51211eda7 [#291] Remove unused code from pkg/services/object/head
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
3260e9263e [#291] Remove unused pkg/services/object/rangehash packages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
d299d94049 [#291] Remove unused pkg/services/object/search/query packages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
1dd1762163 [#285] object/put: Fix the loss of X-headers when forwarding objects
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Leonard Lyubich
2897e83fb2 [#285] object/eacl: Validate X-headers from the requests, not the responses
In previous implementation of eACL service v2 the response X-headers were
validated at the stage of re-checking eACL. This provoked a mismatch of
records in the eACL table with requests. Fix this behavior by checking the
headers from the request, not the response.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Leonard Lyubich
c69f867af1 [#285] services/object: Add X-Headers to client call options
Forward request X-headers to client calls during internal processing of
Object operations on the node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Alex Vanin
7174abcc7c [#281] service/audit: Fix typos
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
e5108cf135 [#281] service/audit: Run each SG check in separate routing at PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
0d7832f5e9 [#281] service/audit: Add separate pool for SG checks in PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
8e72abaab7 [#281] service/audit: Make report structure threadsafe
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
44d9f95137 [#276] service/audit: Count Head requests and retries in PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
9273a89f61 [#276] service/audit: Use random nodes for object.Head in PoR
Since PoR audit check uses object.Head with large TTL values to
make real proof of retrievability, we may use random remote
nodes.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
10b548275a [#276] innerring: Use Head with TTL in PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
3ec342b2c3 [#259] services/audit: Implement PDP check
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
6977adec66 [#273] service/audit: Ignore coverage of very small objects
PDP audit check is not quite working with very small objects, so
we try to build coverage with bigger objects.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
afb83c610c [#273] service/audit: Add cache of head responses in PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
9212864f42 [#258] services/audit: Implement PoP check
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
a5320408a5 [#271] service/audit: Implement PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
5f65ec0265 [#271] audit/report: Add PoR related setters
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
85773c419b [#271] service/object: Fill all fields of new short header structure
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
af7ae0ec4c [#271] acl: Add permission to get objects for inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
dd5e67ff0c [#271] service/audit: Add netmap structure to audit task
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
64b022a5ed [#271] placement: Add FlattenNodes function
This function converts 2-dimension array of container nodes
into single dimension array. Useful when we need to iterate
over whole container.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
58033a3adc [#271] placement: Add BuildObjectPlacement function
This function allows to shuffle prepared container nodes
according to HRW weights from network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
2ebcbe70fb [#255] ir/audit: Make task manager to return number of skipped tasks
Add numeric return from TaskManager.Reset method that shows the number of
canceled tasks. This values will be used for assessment of the progress of
the audit.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
03e3afb0e8 [#255] services/audit: Define interface of container communicator
Define interface of the container communicator which methods are going to be
used in audit checks. Make innerring Server to implement this interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
76d4e53ea0 [#255] services/audit: Skip all tasks from previous epoch in audit processor
Implement Reset method on audit task manager that cleans task queue.
Extended TaskManager interface with Reset method on IR side. Call Reset
method in audit processor before new audit start.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
33804e024d [#255] services/audit: Complete audit report after the last check
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
580c9c974a [#255] services/audit: Implement task manager
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
0f0be2377b [#255] services/audit: Implement audit executor without checks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
babfbc18f2 [#255] services/audit: Implement Task structure
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
076f201807 [#255] services/audit: Define Report structure and Reporter interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
43ee94a572 [#80] storageroup: Implement function to collect structure from container
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
44466f4a29 [#80] storagegroup: Implement search query generator
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
6245e58d1f [#80] services/object: Implement iterators over split-tree leaves
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
e53bf574b5 [#279] container: Use new methods to work with container format
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
Leonard Lyubich
bbe8f34ffd [#253] object/eacl: Process only CID/OID object headers in RANGEHASH/DELETE
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich
9e27263df3 [#253] object/put: Fix loss of session token
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich
8e661d5c37 [#264] object/get: Prevent receiving payloads through GETRANGE when GET
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
66f9532857 [#264] object/get: Check parent address in linking/last child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
ec21040542 [#264] transformer: Finalize parent header once
In previous implementation parent object header finalized twice in size
limiter + formatter. On the one hand, this added redundant action, on the
other hand, it could provoke a difference in the headers of the linking and
the last part. Change formatter to finalize parent header if it does not
container the signature. Change size limiter to reuse parent header after
last child finalization in linking child.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
49131f1bc7 [#247] object/eacl: Use address from session token in request validation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich
dba88c79b4 [#247] object/range: Process only object_id and container_id headers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich
8654458b19 [#247] object/eacl: Use object ID from session token context
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich
168dcbdccd [#247] object/eacl: Process object address from request
In previous implementation eACL validator didn't take into account container
and object ID fields of request bodies.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Alex Vanin
bbe700fa37 [#250] service/policer: Don't shrink node list at unknown error
Every unknown error must not decrease shortage counter and must not
exclude faulty node from the node list, because this list will be used
later for replication.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-14 21:49:50 +03:00
Leonard Lyubich
fb50362dcc [#243] services/object: Share common parameters across services
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
a01262d8bd [#243] service/object: Fix object chain assembly for tombstone body
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
9265e31e65 [#243] services/object: Fix lost tokens when generating requests
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
fe3906c295 [#243] object/delete: Implement new service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
510e9ff2ec [#243] core/object: Replace test content of tombstone with an API structure
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
627012e4fe [#241] object/search: Set container ID parameter of storage engine call
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
611a29f682 [#241] object/search: Refactor service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
3bfb18386b object/get: Replace code with function call
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
9fb7190358 [#242] service/object: Support ContainerID field in local search
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
0e1f05ff45 [#239] object/head: Implement new service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
173d34a8a2 [#235] object/get: Set common operation parameters
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
7e56427534 [#235] object/getrangehash: Implement new service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
0fc5ea674c [#235] object/getrange: Fix incorrect processing of range from last child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
397d912e19 [#235] object/head: Support raw flag in service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1d23483828 [#235] services/object: Implement new GetRange algorithm
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
869d9e571c [#233] object/head: Remove getting the right child header
With the update of the local storage engine, the headers of virtual objects
are directly given. In this regard, the step with obtaining the the right
child header is removed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
f24daa10ff [#233] services/object: Implement new Get algorithm
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
e0350efe00 [#231] services/policer: Use engine.List method
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
351e4b4592 [#222] Support Inhume and Delete in object service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
953387a1e5 [#186] object/acl: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
046206f670 [#186] object/head: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2be8f154a0 [#186] object/put: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
14442a0801 [#186] object/range: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
9b32f131c0 [#186] object/rangehash: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1bce2a4e55 [#186] object/search: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
aa9eb2eaf2 [#186] policer: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1b867e53d7 [#186] replicator: Use new storage engine for work
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
6cddc91cbf [#227] transformer: Inherit parent header in linking object
In previous implementation linking object inherited only ID of the parent
object. From now it inherits full header like last child.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-02 14:53:04 +03:00
Leonard Lyubich
b8e8e1e80d [#227] transformer: Do not inherit attribute in generated objects
In previous implementation child objects inherited parent attributes after
split-transformation, which was redundant. From now attributes are not
inherited.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-02 14:53:04 +03:00
Leonard Lyubich
5470d94416 [#223] placement: Fix local and single-success placement traversal
In previous implementation placement traverser processed incorrectly with
local placement build. Also entity incorrectly traversed the placement
vectors for fixed number read operations until success. The erroneous
behavior was due to the use of a vector number of successes instead of
a scalar number in these scenarios.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 19:36:46 +03:00
Leonard Lyubich
d94a0eb25e [#221] eacl: Get rid of deprecated methods usage
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 11:08:47 +03:00
Alex Vanin
f96630581a [#208] Set unique splitID in children
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-30 10:44:15 +03:00
Alex Vanin
6280d075b9 [#208] Remove childfree search attribute
With updated specification of object related operation
we don't have this search attribute any more and we
should not use functions related to this attribute.

This commit breaks object service logic, however it will
be fixed later.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-30 10:44:15 +03:00
Leonard Lyubich
706bdf736e [#209] eacl: Support object ID filter in eACL mechanism
Add object ID header to the list of processing object headers in eACL
validation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-25 11:59:15 +03:00
Leonard Lyubich
fddc50fd85 [#203] Replace ErrEACLNotFound to core library
ErrEACLNotFound error was defined in implementation package. EACL validator
checked this error after the call of eACL storage interface method. Replace
ErrEACLNotFound to core container library. in order to: on the one hand not
use an implementation error, on the other hand, to be able to reuse a
generic type error (404).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 20:19:20 +03:00
Leonard Lyubich
9148980bd0 [#193] services/object: Support client options in all Object services
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 15:34:30 +03:00
Alex Vanin
cf1ea983e5 [#203] Do not fail eACL check on EACLNotFound error
Now morph library returns error if there is not eACL in
sidechain storage. However in this case eACL check should
be passed since it is the same as having empty eACL table.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-24 15:32:26 +03:00
Leonard Lyubich
46dab77705 [#195] services/object: Write debug log messages on worker pool errors
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
fa6e4a3ca4 [#195] services/object: Write debug log messages on internal service errors
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
6a5c37d592 [#195] object/search: Add option to set logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
cb46e4b154 [#195] object/rangehash: Add option to set logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
af6484e3b1 [#195] object/range: Add option to set logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
cf2dc37a42 [#195] object/put: Add option to set logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
9fbfc0b5e4 [#195] object/head: Add option to set logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Leonard Lyubich
e1e5a590e9 [#195] object/delete: Add option to set logger
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
Alex Vanin
dbf6c9efef [#190] Use request sender owner in sticky bit check
Sticky bit checks if object owner and request owner are the
same. Container owner should not used in this check.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 19:07:16 +03:00
Alex Vanin
2148e282ec [#190] Rename owner to cnrOwner in object request info
`owner` field may be misused as request sender owner, however
it is a owner of a container for that request. New naming
should be clear.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 19:07:16 +03:00
Alex Vanin
f0537b35c1 [#190] Add isOwnerFromKey helper function in ACL
This function takes public key and returns true if
owner id was produced by this key.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 19:07:16 +03:00
Alex Vanin
a14bb6292b [#182] Reuse search filter in policer
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 17:59:46 +03:00
Alex Vanin
2e605b2435 [#182] Limit policer object filter to physical stored objects
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 17:59:46 +03:00
Alex Vanin
cf85fa9fab [#180] Return isInnerRing flag in request classifier
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 15:17:18 +03:00
Alex Vanin
6f841e319d [#180] Make separate basic ACL check for inner ring requests
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 15:17:18 +03:00
Alex Vanin
e8fe07edd0 [#184] Use SDK client cache in object.Rangehash
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-18 18:18:07 +03:00