c0aa892161
[ #136 ] localstorage: Make local storage to use new metabase
...
Replace meta Bucket with meta.DB instance in local storage implementation.
Adopt all dependent components to new local storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-03 18:42:32 +03:00
b48a4ede02
[ #125 ] services/eacl: Use latest object header keys
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 19:25:54 +03:00
8d931b81a6
[ #125 ] object/search: Use latest search filter keys
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 19:25:54 +03:00
f34ad9e730
[ #125 ] services/eacl: Fix undefined method usage
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 19:25:54 +03:00
53efa18e14
[ #109 ] object/put: Implement remote object sender
...
Define RemoteSender structure with PutObject method that puts object to a
remote node locally.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 15:23:22 +03:00
968033deed
[ #40 ] object/put: Assign zero return of MaxObjectSize invalid
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 14:03:25 +03:00
0341773318
[ #83 ] services: Implement response sub-service for each service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
71a06f9e01
[ #83 ] services/util: Define type of response message interface
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-23 10:54:48 +03:00
7464254680
[ #106 ] Put simplest bearer token check first
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
23ec33e821
[ #106 ] Check bearer token lifetime
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
bb455af05f
[ #106 ] Ignore bearer token if basic ACL restrict it
...
There is a bit to allow or deny bearer token check for
each object service method. If this bit is not set then
ignore bearer token and use extended ACL table from
sidechain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
89cd2ad463
[ #106 ] Process bearer token in ACL service
...
If bearer token is presented in the request then check
if it is a valid one and then use it to process extended
ACL checks.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 18:02:11 +03:00
094248690b
[ #115 ] Make ACL classifier errors transparent for client
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 11:55:28 +03:00
ca552f53c6
[ #115 ] Check session token validity
...
Malicious user can stole public session key and use
it by sending request from it's own scope. To prevent
this each session token is signed and signature private
key must be corresponded with owner id in token. Therefore
malicious node cannot impersonate request without private
key to sign token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-22 11:55:28 +03:00
16a5107ef1
[ #60 ] object/put: Provide network State interface to formatter
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-22 11:54:08 +03:00
4a56f82571
[ #60 ] object/transformer: Group parameters of NewFormatTarget func
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-22 11:54:08 +03:00
2541ed4b8f
[ #88 ] object/eacl: Use String() methods to calculate ID values
...
Replace hex encoding of IDs with String() call (base58) in eACL processing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 18:53:04 +03:00
5318abcf38
[ #88 ] object/search: Use String() methods to calculate ID values
...
Replace hex encoding of IDs with String() call (base58) in search query
processing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 18:53:04 +03:00
5017ff0e4a
[ #108 ] object/head: Export remote header retrieval utility
...
Export remote head functionality in headsvc package. Refactor head service
to use RemoteHeader.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
5ad0df7794
[ #108 ] object/head: Return 404 error if header was not found
...
Define ErrNotFound error in headsvc package. Return ErrNotFound from Head
method if the header was not found in the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
ae0dd9e051
[ #106 ] Pass bearer token through generated requests
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-20 18:05:29 +03:00
719075ca97
[ #99 ] Fix no-root search matcher
...
Wrong boolean operation order made matcher return false
on `non-root` search query with non-regular objects. Instead
it should return true for `non-root` query and false for `root`
query.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-16 13:45:35 +03:00
1332a6d3a8
[ #92 ] Provide session token to all produced requests
...
If object service produces new request, the should contain
session token. This is the only way for node to grant access
for a private container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-15 10:20:10 +03:00
0e7e0bd2d6
[ #84 ] Remove mocks and debug code from neofs-node services
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-09 09:15:18 +03:00
cd34145969
[ #73 ] Use request owner public key in eACL check
...
Classifier fetches public key of the request owner
and owner itself. Extended ACL check should rely on
this public key, because it might be extracted from
session token.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-05 18:02:37 +03:00
7a2654719e
[ #71 ] Return only regular objects in root object search
...
Root search applies for user objects, so it should not
return tombstones and storage groups.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-05 14:03:55 +03:00
11262bed4a
[ #71 ] Broadcast tombstone to container
...
With one tombstone for split objects we can't simply
place it in container. We should inform all nodes that
store split objects of removed original object.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-05 14:03:55 +03:00
9cdf7d3896
[ #69 ] object/acl: Check eACL rules in ACL service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
1d676fcfb2
[ #69 ] object/acl: Add eACL components to service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
a7782cf1f9
[ #69 ] object/acl: Extended requestInfo structure
...
Add container identifier field. Add send public key field.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
6c3c872ee4
[ #69 ] object/acl: Define access denied error
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
0f52444ae9
[ #69 ] object/acl: Change basic ACL type in requestInfo
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
30e6912c7b
[ #69 ] object/acl: Construct service from options
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
e5898c9ca8
[ #69 ] object/acl: Rename BasicChecker to Service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 14:02:14 +03:00
798fca9354
[ #70 ] core/object: Process a delete group at tombstone
...
Send object group to delete queue processor after tombstone content
validation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
2b16edebc9
[ #70 ] object/put: Fix NPE caused by nil FormatValidator
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
801999c577
[ #66 ] Impersonate object service verb from session token
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 19:47:49 +03:00
afeebd310c
[ #66 ] Use session token of object header at put ACL check
...
Owner of the request is stored in session token most of the times.
Put request contains session token in the object body, so we have
to fetch it from there.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 19:47:49 +03:00
69a69cdbee
[ #67 ] object/eacl: Implement eACL validator
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 19:46:45 +03:00
44fcd2f212
[ #64 ] object/delete: Change the formation of tombstone
...
Make delete service to write list of child object addresses to tombstone
payload.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 19:46:27 +03:00
861bac3892
[ #59 ] Use max msg size in transport server and splitter
...
For GRPC it is about 4 MiB.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
d2009c8731
[ #59 ] Add grpc payload splitter in object service chain
...
GRPC has default message limit of 4MiB. Since every transmitted
neofs message has to be signed, then original message should
be split into transfer fit structures before signature service.
This commit introduce transport payload splitter for object
service pipeline. This splitter works with stream response
for methods:
- object.Get
- object.Range
- object.Search
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
e158497560
[ #43 ] cmd/neofs-node: Support hostnames with dns, ipv4 and ipv6 addresses
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
f930993e3a
[ #43 ] pkg/network: Do not panic at multiaddr to net.Addr conversion
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
6824a6f67b
[ #61 ] object/search: Support non-root and non-leaf filters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
51e373c3f0
[ #61 ] object/search: Support latest search filters
...
Refactor query to match object and its parents in a single call. Support
KeyRoot and KeyLeaf filters.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
f89c848e84
[ #61 ] object/search: Filter objects by container ID from request
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
1654df4d97
[ #61 ] Update to latest neofs-api-go changes
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
f251645def
[ #58 ] object/delete: Process linking object in Delete service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
16252ad09a
[ #58 ] object/search: Add object-with-children filter
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
624e8cd3cb
[ #58 ] object/search: Refactor RelationSearcher implementation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
4bcfed37ca
[ #58 ] object/head: Generalize RelationSearcher interface method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
6eb353c804
[ #58 ] object/put: Validate payload content after filling
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
017afbf0e3
[ #58 ] services/object: Implement Delete service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
39ddb3a3f4
[ #45 ] object/search: Fix double write of local result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
08b9ae547a
[ #45 ] object/search: Add filtering parent objects
...
In previous implementation object.Search services allowed to search only
physically stored objects. This limitation did not allow getting the ID of
the split object.
Extend search execution logic with parent object filtering. Parent objects
that passed filters are now included in the result
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
88459963fb
[ #57 ] services/object: Sign requests with session key
...
Use key storage in object services in order to sign requests with private
session key within user session.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
be322835af
[ #57 ] services/object: Implement private key storage
...
Implement storage that provides access to local node key and session keys
through session token.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
2da323c4b9
[ #57 ] services/object: Add session token to common parameters
...
Add session token field to CommonPrm. Remove session token field from Put
parameters.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
39c17253be
[ #57 ] services/object: Combine common service parameters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
8cddbe58a6
[ #56 ] object/transformer: Write session token to object body
...
Add session token argument to object formatter constructor which is written
to the object. Pass session token from trusted object Put.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
a4b9560ef6
[ #56 ] object/put: Validate object format in untrusted Put
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
fc74e9b40c
[ #32 ] Remove recover from basic ACL checks
...
Basic ACL checker gets request field via getters that are
NPE-free, therefore we don't need to worry about function
invocations on nil structures.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
4a8de3263d
[ #32 ] Use less v2 specific structures in basic ACL checker
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
c5a44e0a05
[ #32 ] Add tests for basic ACL helper
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
91fef72bb6
[ #32 ] Make basic ACL check in all object request
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
49ee9a14a1
[ #32 ] Add basic ACL helper
...
Basic ACL helper provides functions for simple access to
bit fields of basic ACL.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
f6904db84f
[ #32 ] Use pkg/core interfaces to fetch container and netmap
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
ad36a2cd8f
[ #32 ] Use classifier in basic ACL check
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
5045b0c3d4
[ #32 ] Add request sender classifier
...
ACL has to classify request senders by roles:
- owner of the container,
- request from container or inner ring node,
- any other request.
According to this roles ACL checker use different
bits of basic ACL to grant or deny access.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
ab565b1862
[ #32 ] Add basis of basic ACL check service
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
867f1d772d
[ #53 ] Revert "[ #51 ] object/put: Fix incorrect splited object streaming"
...
Revert commit 803ea345 that which spawned a duplication of the payload of
objects.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
0a51263e72
[ #53 ] services/object: Implement GetRangeHash service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
834a8597c5
[ #53 ] object/util: Add seek range function to range traverser
...
Add SetSeekRange method to RangeTraverser that switches traverser to work
with provided object payload range.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
8791c4f0a5
[ #53 ] object/range: Share object range traverser between services
...
Replace object range chain utilities to object/util package in order to
reuse it in other services.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
759605410a
[ #53 ] object/range: Do not add chain element without the need
...
In previous implementation one excess element could be added to the chain.
Add previous sibling presence check to prevent this.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
bf2c33d7a9
[ #52 ] services/object: Implement Get service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
3880315a3f
[ #52 ] object/range: Add full range option to parameters
...
Add FullRange option to get range operation parameters that allows to get
payload range [0:object_size] w/o the actual knowledge of the object size.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
abf9ad3573
[ #52 ] object/range: Attach received object header to result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
0490107165
[ #50 ] services/object: Implement GetRange service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
dd16f568c3
[ #50 ] object/head: Add right child to Head operation result
...
Head service receives right child of the processing object in some cases.
Add right child to Head result in order to use it as needed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
3692f708ca
[ #51 ] object/put: Fix incorrect splited object streaming
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
6f4b477195
[ #44 ] services/object: Refactor for minor optimization
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
3692032e30
[ #44 ] object/head: Fix incorrect placement traversal
...
Add object identifier option to placement traverser of object.Head service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
d6a9c06c25
[ #39 ] service/object: Complicate Head service logic
...
Add a header recovery step through finding and getting the header of the
rightmost child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
753a6a2de5
[ #39 ] service/object: Implement right child searcher on Search service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
39c324bd6d
[ #39 ] service/object: Add query filters to search right child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
05f3963975
[ #38 ] service/object: Implement simplified object Head service
...
Implement Head service w/o linking object processing and restoration from
split-chain.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
09084a7bff
[ #34 ] service/object: Implement object Search distributed service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
a5ebdd1891
[ #34 ] services/object: Replace local placement to util
...
Replace local placement tool in order to reuse it in different object
services.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
57f8d3745d
[ #33 ] service/object: Implement object Put distributed service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
b7702349dc
[ #13 ] services/util: Support client-side stream in SignService
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
d54a5d4f66
[ #13 ] services/util: Rename UnarySignService to SingService
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
75162d1f8d
[ #13 ] services/util: Support server-side stream in UnarySignService
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
6b4fb3a0aa
[ #13 ] services: Refactor UnarySignService
...
Replace UnaryHandler from structure to method arguments in order to reuse
single instance for different service methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
0c83e84613
[ #13 ] services/object: Implement GetRangeHash handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
7a6b62c552
[ #13 ] services/object: Implement Delete handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
881f49814a
[ #13 ] services/object: Implement Head handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
652f64ae85
[ #13 ] services/object: Implement GetRange handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
62d0c50b15
[ #13 ] services/object: Implement Put handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
f14d1fd65f
[ #13 ] services/object: Implement Get handlers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
486aeef5df
[ #13 ] services/object: Define and implement Object executor
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
276ec2e1e0
[ #13 ] services: Define signing Object service and implement Search
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
Alex Vanin