611a29f682
[ #241 ] object/search: Refactor service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3bfb18386b
object/get: Replace code with function call
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
9fb7190358
[ #242 ] service/object: Support ContainerID field in local search
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
0e1f05ff45
[ #239 ] object/head: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
173d34a8a2
[ #235 ] object/get: Set common operation parameters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7e56427534
[ #235 ] object/getrangehash: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
0fc5ea674c
[ #235 ] object/getrange: Fix incorrect processing of range from last child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
397d912e19
[ #235 ] object/head: Support raw flag in service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1d23483828
[ #235 ] services/object: Implement new GetRange algorithm
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
869d9e571c
[ #233 ] object/head: Remove getting the right child header
...
With the update of the local storage engine, the headers of virtual objects
are directly given. In this regard, the step with obtaining the the right
child header is removed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
f24daa10ff
[ #233 ] services/object: Implement new Get algorithm
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
e0350efe00
[ #231 ] services/policer: Use engine.List method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
351e4b4592
[ #222 ] Support Inhume and Delete in object service
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
953387a1e5
[ #186 ] object/acl: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
046206f670
[ #186 ] object/head: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2be8f154a0
[ #186 ] object/put: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
14442a0801
[ #186 ] object/range: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
9b32f131c0
[ #186 ] object/rangehash: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1bce2a4e55
[ #186 ] object/search: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
aa9eb2eaf2
[ #186 ] policer: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1b867e53d7
[ #186 ] replicator: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
6cddc91cbf
[ #227 ] transformer: Inherit parent header in linking object
...
In previous implementation linking object inherited only ID of the parent
object. From now it inherits full header like last child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-02 14:53:04 +03:00
b8e8e1e80d
[ #227 ] transformer: Do not inherit attribute in generated objects
...
In previous implementation child objects inherited parent attributes after
split-transformation, which was redundant. From now attributes are not
inherited.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-02 14:53:04 +03:00
5470d94416
[ #223 ] placement: Fix local and single-success placement traversal
...
In previous implementation placement traverser processed incorrectly with
local placement build. Also entity incorrectly traversed the placement
vectors for fixed number read operations until success. The erroneous
behavior was due to the use of a vector number of successes instead of
a scalar number in these scenarios.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 19:36:46 +03:00
d94a0eb25e
[ #221 ] eacl: Get rid of deprecated methods usage
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 11:08:47 +03:00
f96630581a
[ #208 ] Set unique splitID in children
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-30 10:44:15 +03:00
6280d075b9
[ #208 ] Remove `childfree` search attribute
...
With updated specification of object related operation
we don't have this search attribute any more and we
should not use functions related to this attribute.
This commit breaks object service logic, however it will
be fixed later.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-30 10:44:15 +03:00
706bdf736e
[ #209 ] eacl: Support object ID filter in eACL mechanism
...
Add object ID header to the list of processing object headers in eACL
validation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-25 11:59:15 +03:00
fddc50fd85
[ #203 ] Replace ErrEACLNotFound to core library
...
ErrEACLNotFound error was defined in implementation package. EACL validator
checked this error after the call of eACL storage interface method. Replace
ErrEACLNotFound to core container library. in order to: on the one hand not
use an implementation error, on the other hand, to be able to reuse a
generic type error (404).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 20:19:20 +03:00
9148980bd0
[ #193 ] services/object: Support client options in all Object services
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 15:34:30 +03:00
cf1ea983e5
[ #203 ] Do not fail eACL check on EACLNotFound error
...
Now morph library returns error if there is not eACL in
sidechain storage. However in this case eACL check should
be passed since it is the same as having empty eACL table.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-24 15:32:26 +03:00
46dab77705
[ #195 ] services/object: Write debug log messages on worker pool errors
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
fa6e4a3ca4
[ #195 ] services/object: Write debug log messages on internal service errors
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
6a5c37d592
[ #195 ] object/search: Add option to set logger
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
cb46e4b154
[ #195 ] object/rangehash: Add option to set logger
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
af6484e3b1
[ #195 ] object/range: Add option to set logger
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
cf2dc37a42
[ #195 ] object/put: Add option to set logger
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
9fbfc0b5e4
[ #195 ] object/head: Add option to set logger
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
e1e5a590e9
[ #195 ] object/delete: Add option to set logger
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 13:38:06 +03:00
dbf6c9efef
[ #190 ] Use request sender owner in sticky bit check
...
Sticky bit checks if object owner and request owner are the
same. Container owner should not used in this check.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 19:07:16 +03:00
2148e282ec
[ #190 ] Rename owner to cnrOwner in object request info
...
`owner` field may be misused as request sender owner, however
it is a owner of a container for that request. New naming
should be clear.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 19:07:16 +03:00
f0537b35c1
[ #190 ] Add isOwnerFromKey helper function in ACL
...
This function takes public key and returns true if
owner id was produced by this key.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 19:07:16 +03:00
a14bb6292b
[ #182 ] Reuse search filter in policer
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 17:59:46 +03:00
2e605b2435
[ #182 ] Limit policer object filter to physical stored objects
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 17:59:46 +03:00
cf85fa9fab
[ #180 ] Return isInnerRing flag in request classifier
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 15:17:18 +03:00
6f841e319d
[ #180 ] Make separate basic ACL check for inner ring requests
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-19 15:17:18 +03:00
e8fe07edd0
[ #184 ] Use SDK client cache in object.Rangehash
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-18 18:18:07 +03:00
f85e88c4f8
[ #184 ] Use SDK client cache in object.Range
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-18 18:18:07 +03:00
d485a5967d
[ #184 ] Use SDK client cache in object.Search
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-18 18:18:07 +03:00
7ba95dd5fc
[ #184 ] Use SDK client cache in object.Put
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-18 18:18:07 +03:00
e9a6365333
[ #184 ] Use SDK client cache in object.Head
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-18 18:18:07 +03:00
1caf15463e
[ #174 ] Update to neofs-api-go v1.20.0
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
58fcb35fb0
[ #174 ] Use Marshal(JSON)/Unmarshal(JSON) methods for encoding/decoding
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
3de8febe57
[ #174 ] Update to latest neofs-api-go changes
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
32219bb294
[ #160 ] Remove query match function
...
This function duplicates query processing that
is done in meta-storage now.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-16 10:02:12 +03:00
3c42f5b452
[ #161 ] object/head: Inherit common parameters in HeadRelation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-09 17:19:34 +03:00
d38633e047
[ #161 ] object/delete: Add address from request body to tombstone content
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-09 15:53:58 +03:00
5ad013c10b
[ #149 ] object/search: Return fixed error if relation not found
...
Define ErrRelationNotFound error in searchsvc package. Return
ErrRelationNotFound from RelationSearcher.SearchRelation method if search
result is empty.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 14:01:01 +03:00
65be09d3db
[ #155 ] Update neofs-api-go with refactored pkg/netmap
...
Refactored pkg/netmap package provides JSON converters for
NodeInfo and PlacementPolicy structures, that has been used
by client applications.
It also updates Node structure itself so it is a part of
grpc <-> v2 <-> pkg conversion chain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-06 09:55:05 +03:00
c0aa892161
[ #136 ] localstorage: Make local storage to use new metabase
...
Replace meta Bucket with meta.DB instance in local storage implementation.
Adopt all dependent components to new local storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-03 18:42:32 +03:00
766eea4c8c
[ #85 ] services/container: Check container format in Put
...
Call CheckFormat function in container.Put handler for conducting initial
checks of the structure that are not performed by the smart contract.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-03 14:14:38 +03:00
b48a4ede02
[ #125 ] services/eacl: Use latest object header keys
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 19:25:54 +03:00
8d931b81a6
[ #125 ] object/search: Use latest search filter keys
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 19:25:54 +03:00
f34ad9e730
[ #125 ] services/eacl: Fix undefined method usage
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 19:25:54 +03:00
d08c1c76c1
[ #122 ] Reduce precision from balance contract to Fixed8.
...
Fixed8 won't overflow int64 for values less than 92 billion
that is suitable for GAS.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-29 10:14:59 +03:00
f66c7958e7
[ #109 ] services/policer: Assign tasks to Replicator
...
Make Policer to call AddTask method of Replicator when an insufficient
number of copies of an object is detected in the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 15:23:22 +03:00
2d46baa4a5
[ #109 ] services: Implement Replicator service
...
Implement Replicator service that performs background work to replicate
local object to remote nodes in the container. Replicator is going to be
used by Policer.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 15:23:22 +03:00
53efa18e14
[ #109 ] object/put: Implement remote object sender
...
Define RemoteSender structure with PutObject method that puts object to a
remote node locally.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 15:23:22 +03:00
968033deed
[ #40 ] object/put: Assign zero return of MaxObjectSize invalid
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 14:03:25 +03:00
7fdb14cf8a
[ #83 ] services/response: Set epoch number from network state
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
19f9c7eacb
[ #83 ] services: Remove setting of meta header from executing services
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
0341773318
[ #83 ] services: Implement response sub-service for each service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
6bede7d836
[ #83 ] services/util: Implement response service
...
Create response package. Implement response Service that sets values of
response meta header.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
1cc7983c4e
[ #83 ] services/util: Add meta header methods to ResponseMessage
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
71a06f9e01
[ #83 ] services/util: Define type of response message interface
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
7464254680
[ #106 ] Put simplest bearer token check first
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
23ec33e821
[ #106 ] Check bearer token lifetime
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
bb455af05f
[ #106 ] Ignore bearer token if basic ACL restrict it
...
There is a bit to allow or deny bearer token check for
each object service method. If this bit is not set then
ignore bearer token and use extended ACL table from
sidechain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
89cd2ad463
[ #106 ] Process bearer token in ACL service
...
If bearer token is presented in the request then check
if it is a valid one and then use it to process extended
ACL checks.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
094248690b
[ #115 ] Make ACL classifier errors transparent for client
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 11:55:28 +03:00
ca552f53c6
[ #115 ] Check session token validity
...
Malicious user can stole public session key and use
it by sending request from it's own scope. To prevent
this each session token is signed and signature private
key must be corresponded with owner id in token. Therefore
malicious node cannot impersonate request without private
key to sign token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 11:55:28 +03:00
16a5107ef1
[ #60 ] object/put: Provide network State interface to formatter
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-22 11:54:08 +03:00
b627814dd8
[ #60 ] object/transformer: Set creation epoch number in new objects
...
Set value of CreationEpoch object field to the value from network State.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-22 11:54:08 +03:00
4a56f82571
[ #60 ] object/transformer: Group parameters of NewFormatTarget func
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-22 11:54:08 +03:00
2541ed4b8f
[ #88 ] object/eacl: Use String() methods to calculate ID values
...
Replace hex encoding of IDs with String() call (base58) in eACL processing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 18:53:04 +03:00
5318abcf38
[ #88 ] object/search: Use String() methods to calculate ID values
...
Replace hex encoding of IDs with String() call (base58) in search query
processing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 18:53:04 +03:00
0dab4b7581
[ #108 ] services: Implement Policer service
...
Implement Policer service that performs background work to check compliance
with the placement policy for local objects in the container. In the initial
implementation, the selection of the working queue of objects is
simplified, and there is no transfer of the result to the replicator.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
f6e56aa956
[ #108 ] placement: Implement Builder from netmap source
...
Implement placement.Builder interface on netmap.Source wrapper.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
5017ff0e4a
[ #108 ] object/head: Export remote header retrieval utility
...
Export remote head functionality in headsvc package. Refactor head service
to use RemoteHeader.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
5ad0df7794
[ #108 ] object/head: Return 404 error if header was not found
...
Define ErrNotFound error in headsvc package. Return ErrNotFound from Head
method if the header was not found in the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
ae0dd9e051
[ #106 ] Pass bearer token through generated requests
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-20 18:05:29 +03:00
9e08b41a6f
[ #102 ] Set split header in left object
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-20 09:59:09 +03:00
719075ca97
[ #99 ] Fix no-root search matcher
...
Wrong boolean operation order made matcher return false
on `non-root` search query with non-regular objects. Instead
it should return true for `non-root` query and false for `root`
query.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-16 13:45:35 +03:00
1332a6d3a8
[ #92 ] Provide session token to all produced requests
...
If object service produces new request, the should contain
session token. This is the only way for node to grant access
for a private container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-15 10:20:10 +03:00
2d5cb378a7
[ #84 ] Add netmap service executor and signer
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-09 09:15:18 +03:00
0e7e0bd2d6
[ #84 ] Remove mocks and debug code from neofs-node services
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-09 09:15:18 +03:00
87fc4f5df7
[ #82 ] Use morph wrapper in container service
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-08 11:22:50 +03:00
cd34145969
[ #73 ] Use request owner public key in eACL check
...
Classifier fetches public key of the request owner
and owner itself. Extended ACL check should rely on
this public key, because it might be extracted from
session token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-05 18:02:37 +03:00
7a2654719e
[ #71 ] Return only regular objects in root object search
...
Root search applies for user objects, so it should not
return tombstones and storage groups.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-05 14:03:55 +03:00
11262bed4a
[ #71 ] Broadcast tombstone to container
...
With one tombstone for split objects we can't simply
place it in container. We should inform all nodes that
store split objects of removed original object.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-05 14:03:55 +03:00
9cdf7d3896
[ #69 ] object/acl: Check eACL rules in ACL service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
1d676fcfb2
[ #69 ] object/acl: Add eACL components to service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
a7782cf1f9
[ #69 ] object/acl: Extended requestInfo structure
...
Add container identifier field. Add send public key field.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
6c3c872ee4
[ #69 ] object/acl: Define access denied error
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
0f52444ae9
[ #69 ] object/acl: Change basic ACL type in requestInfo
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
30e6912c7b
[ #69 ] object/acl: Construct service from options
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
e5898c9ca8
[ #69 ] object/acl: Rename BasicChecker to Service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
0d5495e997
[ #70 ] object manager: Implement an example object garbage collector
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
798fca9354
[ #70 ] core/object: Process a delete group at tombstone
...
Send object group to delete queue processor after tombstone content
validation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
2b16edebc9
[ #70 ] object/put: Fix NPE caused by nil FormatValidator
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
801999c577
[ #66 ] Impersonate object service verb from session token
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 19:47:49 +03:00
afeebd310c
[ #66 ] Use session token of object header at put ACL check
...
Owner of the request is stored in session token most of the times.
Put request contains session token in the object body, so we have
to fetch it from there.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 19:47:49 +03:00
69a69cdbee
[ #67 ] object/eacl: Implement eACL validator
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 19:46:45 +03:00
44fcd2f212
[ #64 ] object/delete: Change the formation of tombstone
...
Make delete service to write list of child object addresses to tombstone
payload.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 19:46:27 +03:00
861bac3892
[ #59 ] Use max msg size in transport server and splitter
...
For GRPC it is about 4 MiB.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
d2009c8731
[ #59 ] Add grpc payload splitter in object service chain
...
GRPC has default message limit of 4MiB. Since every transmitted
neofs message has to be signed, then original message should
be split into transfer fit structures before signature service.
This commit introduce transport payload splitter for object
service pipeline. This splitter works with stream response
for methods:
- object.Get
- object.Range
- object.Search
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
64691e6248
[ #62 ] object/transformer: Fix incorrect relation init stage
...
In previous implementation InitRelations call in payload size limiter was
called in write chunk method. This provoked clearing the split header in
children starting from the second.
Replace InitRelations call to the 1st child allocating stage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
e158497560
[ #43 ] cmd/neofs-node: Support hostnames with dns, ipv4 and ipv6 addresses
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
f930993e3a
[ #43 ] pkg/network: Do not panic at multiaddr to net.Addr conversion
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
6824a6f67b
[ #61 ] object/search: Support non-root and non-leaf filters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
51e373c3f0
[ #61 ] object/search: Support latest search filters
...
Refactor query to match object and its parents in a single call. Support
KeyRoot and KeyLeaf filters.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
f89c848e84
[ #61 ] object/search: Filter objects by container ID from request
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
1654df4d97
[ #61 ] Update to latest neofs-api-go changes
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
f251645def
[ #58 ] object/delete: Process linking object in Delete service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
16252ad09a
[ #58 ] object/search: Add object-with-children filter
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
624e8cd3cb
[ #58 ] object/search: Refactor RelationSearcher implementation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
4bcfed37ca
[ #58 ] object/head: Generalize RelationSearcher interface method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
6eb353c804
[ #58 ] object/put: Validate payload content after filling
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
017afbf0e3
[ #58 ] services/object: Implement Delete service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
b24adeae89
[ #58 ] object/transformer: Inherit type of parent object
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
39ddb3a3f4
[ #45 ] object/search: Fix double write of local result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
08b9ae547a
[ #45 ] object/search: Add filtering parent objects
...
In previous implementation object.Search services allowed to search only
physically stored objects. This limitation did not allow getting the ID of
the split object.
Extend search execution logic with parent object filtering. Parent objects
that passed filters are now included in the result
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
88459963fb
[ #57 ] services/object: Sign requests with session key
...
Use key storage in object services in order to sign requests with private
session key within user session.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
be322835af
[ #57 ] services/object: Implement private key storage
...
Implement storage that provides access to local node key and session keys
through session token.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
2da323c4b9
[ #57 ] services/object: Add session token to common parameters
...
Add session token field to CommonPrm. Remove session token field from Put
parameters.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
39c17253be
[ #57 ] services/object: Combine common service parameters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
8cddbe58a6
[ #56 ] object/transformer: Write session token to object body
...
Add session token argument to object formatter constructor which is written
to the object. Pass session token from trusted object Put.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
a4b9560ef6
[ #56 ] object/put: Validate object format in untrusted Put
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
12d57af998
[ #56 ] Update to latest neofs-api-go changes
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
fc74e9b40c
[ #32 ] Remove recover from basic ACL checks
...
Basic ACL checker gets request field via getters that are
NPE-free, therefore we don't need to worry about function
invocations on nil structures.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
4a8de3263d
[ #32 ] Use less v2 specific structures in basic ACL checker
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
c5a44e0a05
[ #32 ] Add tests for basic ACL helper
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
91fef72bb6
[ #32 ] Make basic ACL check in all object request
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
49ee9a14a1
[ #32 ] Add basic ACL helper
...
Basic ACL helper provides functions for simple access to
bit fields of basic ACL.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
f6904db84f
[ #32 ] Use pkg/core interfaces to fetch container and netmap
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
ad36a2cd8f
[ #32 ] Use classifier in basic ACL check
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
5045b0c3d4
[ #32 ] Add request sender classifier
...
ACL has to classify request senders by roles:
- owner of the container,
- request from container or inner ring node,
- any other request.
According to this roles ACL checker use different
bits of basic ACL to grant or deny access.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
ab565b1862
[ #32 ] Add basis of basic ACL check service
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
1b5ac0f2ae
[ #55 ] object/transformer: Fix NPE in case of empty payload
...
In previous implementation payload size limiter panicked in case of payload
emptiness. It was caused by the component waiting for at least one write of
a part of the payload.
Fix NPE occurrence with internal initialization after the WriteHeader call.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
867f1d772d
[ #53 ] Revert "[ #51 ] object/put: Fix incorrect splited object streaming"
...
Revert commit 803ea345 that which spawned a duplication of the payload of
objects.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
0a51263e72
[ #53 ] services/object: Implement GetRangeHash service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
834a8597c5
[ #53 ] object/util: Add seek range function to range traverser
...
Add SetSeekRange method to RangeTraverser that switches traverser to work
with provided object payload range.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
8791c4f0a5
[ #53 ] object/range: Share object range traverser between services
...
Replace object range chain utilities to object/util package in order to
reuse it in other services.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
759605410a
[ #53 ] object/range: Do not add chain element without the need
...
In previous implementation one excess element could be added to the chain.
Add previous sibling presence check to prevent this.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
bf2c33d7a9
[ #52 ] services/object: Implement Get service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
3880315a3f
[ #52 ] object/range: Add full range option to parameters
...
Add FullRange option to get range operation parameters that allows to get
payload range [0:object_size] w/o the actual knowledge of the object size.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
abf9ad3573
[ #52 ] object/range: Attach received object header to result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
0490107165
[ #50 ] services/object: Implement GetRange service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
dd16f568c3
[ #50 ] object/head: Add right child to Head operation result
...
Head service receives right child of the processing object in some cases.
Add right child to Head result in order to use it as needed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
3692f708ca
[ #51 ] object/put: Fix incorrect splited object streaming
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
6f4b477195
[ #44 ] services/object: Refactor for minor optimization
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
3692032e30
[ #44 ] object/head: Fix incorrect placement traversal
...
Add object identifier option to placement traverser of object.Head service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
a433f9dd85
[ #42 ] object/transformer: Fix incorrect checksums of parent object
...
In previous implementations object size limiter left checksums of parent
object payload unset. This was caused by the closure that written calculated
checksums to the child object instead of parent one.
Construct payload checksum writers in separate function in order to prevent
closure bug.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
d6a9c06c25
[ #39 ] service/object: Complicate Head service logic
...
Add a header recovery step through finding and getting the header of the
rightmost child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
753a6a2de5
[ #39 ] service/object: Implement right child searcher on Search service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
39c324bd6d
[ #39 ] service/object: Add query filters to search right child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
05f3963975
[ #38 ] service/object: Implement simplified object Head service
...
Implement Head service w/o linking object processing and restoration from
split-chain.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
09084a7bff
[ #34 ] service/object: Implement object Search distributed service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
a5ebdd1891
[ #34 ] services/object: Replace local placement to util
...
Replace local placement tool in order to reuse it in different object
services.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
57f8d3745d
[ #33 ] service/object: Implement object Put distributed service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
dcfb6a6b3a
[ #33 ] placement: Remote UseNetworkMap option from traverser
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
12e5e4e2d8
[ #33 ] placement: Add public constructor for netmap placement builder
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
d3b5ff9526
[ #33 ] session/storage: Change Get method signature
...
In previous implementation session Storage.Get method took a v2 owner
identifier as an argument.
Change method signature to as an argument owner identifier from NeoFS SDK.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
4024b3c30e
[ #33 ] services/session: Fix signature service constructor
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
73ee5105ff
[ #31 ] placement: Fix confused selectors and replicas
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
f7c685f682
[ #31 ] placement: Fix incorrect selectors processing
...
In previous implementation traverser worked like all counts of all selectors
are equal to counts of corresponding replicas. Make traverser to take into
account select count of all replicas.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
e7925fbc1c
[ #31 ] placement: Implement container placement traverser
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
121137c62b
[ #30 ] object_manager: Implement object transformers
...
Define object writer interface. Implement payload size limiter that
restricts payload size of the object. Implement object format moulder that
fill verification fields and finalizes object structure.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
9c1c023f05
[ #30 ] Update to latest neofs-api-go and fix conflicts
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
8f5ea75eb6
[ #19 ] Update eACL service methods
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
03b170237f
Remove broken packages
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
da92f2944f
[ #7 ] Fix container service according to APIv2 contracts
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
80f10dab7b
[ #16 ] Fix balanceOf wrapper for api request
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
b7702349dc
[ #13 ] services/util: Support client-side stream in SignService
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
d54a5d4f66
[ #13 ] services/util: Rename UnarySignService to SingService
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
75162d1f8d
[ #13 ] services/util: Support server-side stream in UnarySignService
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
6b4fb3a0aa
[ #13 ] services: Refactor UnarySignService
...
Replace UnaryHandler from structure to method arguments in order to reuse
single instance for different service methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
0c83e84613
[ #13 ] services/object: Implement GetRangeHash handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
7a6b62c552
[ #13 ] services/object: Implement Delete handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
881f49814a
[ #13 ] services/object: Implement Head handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
652f64ae85
[ #13 ] services/object: Implement GetRange handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
62d0c50b15
[ #13 ] services/object: Implement Put handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
f14d1fd65f
[ #13 ] services/object: Implement Get handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
486aeef5df
[ #13 ] services/object: Define and implement Object executor
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
276ec2e1e0
[ #13 ] services: Define signing Object service and implement Search
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
da1961cdf9
[ #13 ] services/util: Support server-side stream request verify
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
f5a0e354ef
[ #12 ] services/session: Implement service executor
...
Implement Session service executor that generates private token and stores
it to map.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:30 +03:00
de12d751e9
[ #12 ] services/session: Implement execution service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:24 +03:00
7b56633185
[ #12 ] services/session: Implement Sign/Verify service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:18 +03:00
8539f5c2cd
[ #11 ] services/container: Implement Neo:Morph executor and service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:24:45 +03:00
f71d64435e
[ #11 ] services: Implement universal Sign/Verify service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:24:30 +03:00
5022362c1a
[ #11 ] accounting: Implement ServiceExecutor on Neo:Morph client
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:48 +03:00
7cc1a2a85a
[ #11 ] accounting: Define service executor
...
Defines ServiceExecutor interface of the executor that handles service
request bodies and returns service response bodies. Adds Service
implementation that writes the result of ServiceExecutor to response body
and attaches the internal ResponseMetaHeader.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:35 +03:00
5cf622881f
[ #11 ] accounting: Implement signing service
...
Adds accounting Service implementation that verifiers request signatures and
signs responses from internal Service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:29 +03:00
a87fdab324
[ #11 ] Trim the old functionality
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:22:55 +03:00
b7b5079934
Add Inner Ring code
2020-07-24 17:07:37 +03:00
Leonard Lyubich
Alex Vanin
Stanislav Bogatyrev