Commit Graph

4265 Commits (f04806ccd342be552cf172b4ebf856899fbc7a70)

Author SHA1 Message Date
Alex Vanin 49ee9a14a1 [#32] Add basic ACL helper
Basic ACL helper provides functions for simple access to
bit fields of basic ACL.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin f6904db84f [#32] Use pkg/core interfaces to fetch container and netmap
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin ad36a2cd8f [#32] Use classifier in basic ACL check
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 5045b0c3d4 [#32] Add request sender classifier
ACL has to classify request senders by roles:
- owner of the container,
- request from container or inner ring node,
- any other request.

According to this roles ACL checker use different
bits of basic ACL to grant or deny access.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin ab565b1862 [#32] Add basis of basic ACL check service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 1b5ac0f2ae [#55] object/transformer: Fix NPE in case of empty payload
In previous implementation payload size limiter panicked in case of payload
emptiness. It was caused by the component waiting for at least one write of
a part of the payload.

Fix NPE occurrence with internal initialization after the WriteHeader call.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 867f1d772d [#53] Revert "[#51] object/put: Fix incorrect splited object streaming"
Revert commit 803ea345 that which spawned a duplication of the payload of
objects.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 0a51263e72 [#53] services/object: Implement GetRangeHash service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 9d8576d397 [#53] util: Add SaltXOR function
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 834a8597c5 [#53] object/util: Add seek range function to range traverser
Add SetSeekRange method to RangeTraverser that switches traverser to work
with provided object payload range.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 8791c4f0a5 [#53] object/range: Share object range traverser between services
Replace object range chain utilities to object/util package in order to
reuse it in other services.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 759605410a [#53] object/range: Do not add chain element without the need
In previous implementation one excess element could be added to the chain.
Add previous sibling presence check to prevent this.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 7813b90554 [#52] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich bf2c33d7a9 [#52] services/object: Implement Get service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 3880315a3f [#52] object/range: Add full range option to parameters
Add FullRange option to get range operation parameters that allows to get
payload range [0:object_size] w/o the actual knowledge of the object size.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich abf9ad3573 [#52] object/range: Attach received object header to result
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 0490107165 [#50] services/object: Implement GetRange service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich dd16f568c3 [#50] object/head: Add right child to Head operation result
Head service receives right child of the processing object in some cases.
Add right child to Head result in order to use it as needed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 3692f708ca [#51] object/put: Fix incorrect splited object streaming
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 1a169a1a9d [#49] Use zap.Logger in neofs-storage
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 2fc2f8792e [#48] Make IR initial config errors more readable
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin be353ad69f [#48] Use more extra fee at container registration
Large inner ring requires more gas to make container registration.
Container contract makes balance transfers for each inner ring node
and it require extra gas to execute. This estimation should be enough
for seven inner ring nodes.

Later there should be heuristic evaluations for this: #47

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 6f4b477195 [#44] services/object: Refactor for minor optimization
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 3692032e30 [#44] object/head: Fix incorrect placement traversal
Add object identifier option to placement traverser of object.Head service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich a433f9dd85 [#42] object/transformer: Fix incorrect checksums of parent object
In previous implementations object size limiter left checksums of parent
object payload unset. This was caused by the closure that written calculated
checksums to the child object instead of parent one.

Construct payload checksum writers in separate function in order to prevent
closure bug.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 3038c6cc5c [#37] Implement eACL related morph wrapper functions
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 14f85184ab [#37] Implement container related morph wrapper functions
Container get wrapper implements container.Source interface
so it can be used in object service as container storage.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich d6a9c06c25 [#39] service/object: Complicate Head service logic
Add a header recovery step through finding and getting the header of the
rightmost child.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 753a6a2de5 [#39] service/object: Implement right child searcher on Search service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich 39c324bd6d [#39] service/object: Add query filters to search right child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin df5fb23ba8 [#36] Move node attribute code into separate file
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 8e4416ad4b [#36] Add default capacity and price attributes
Default values for capacity and price will be 0, so storage node
won't be present in placement. Add these attributes if they are
not explicitly set up by user config.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 1759ff57f1 [#36] Parse attributes from node config
Well-known attributes set up with explicit configuration
values, other attributes set up from chain of attributes.

Chain of attributes is a string, that contains keys-value pairs,
divided by semicolon. Pairs itself divided by slash.

E.g. "StorageType:HDD/RPM:7200"

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin 28534a509d [#36] Add attribute parser
Attribute parser converts strings of attribute chain into
one-dimension array of NodeInfo attributes, that used
in network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 05f3963975 [#38] service/object: Implement simplified object Head service
Implement Head service w/o linking object processing and restoration from
split-chain.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 21fc85540a [#38] localstore: Finalize object to meta converter
Use CutPayload method of RawObject in order to make object meta structure
from source object.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 2f57855775 [#38] sdk/object: Add CutPayload method to RawObject
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich becb9148c1 [#38] sdk/object: Add Object to RawObject converter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich dc88934034 [#34] Update go.sum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 09084a7bff [#34] service/object: Implement object Search distributed service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich a5ebdd1891 [#34] services/object: Replace local placement to util
Replace local placement tool in order to reuse it in different object
services.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich ded0350c5e [#34] localstore: Store object identifier in meta storage
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin 99d380c20b [#35] Implement netmap source interface in morph wrapper
Netmap source interface used by object service to build
placement based on current or previous network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin 92fb384c09 [#35] Add snapshot method for morph/netmap client
With snapshot method node can get previous network
maps from previous epoch.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich c410b23c97 [#33] Update to latest neofs-api-go version
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 57f8d3745d [#33] service/object: Implement object Put distributed service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich dcfb6a6b3a [#33] placement: Remote UseNetworkMap option from traverser
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 12e5e4e2d8 [#33] placement: Add public constructor for netmap placement builder
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich 900949c648 [#33] network: Implement tool to work with local address
Implement read-only local network address source. Implement function for
check if network address is local.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich d3349a293a [#33] core/object: Add to-SDK object converting method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00