Commit graph

1657 commits

Author SHA1 Message Date
Alex Vanin
14f85184ab [] Implement container related morph wrapper functions
Container get wrapper implements container.Source interface
so it can be used in object service as container storage.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
d6a9c06c25 [] service/object: Complicate Head service logic
Add a header recovery step through finding and getting the header of the
rightmost child.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
753a6a2de5 [] service/object: Implement right child searcher on Search service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
39c324bd6d [] service/object: Add query filters to search right child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin
28534a509d [] Add attribute parser
Attribute parser converts strings of attribute chain into
one-dimension array of NodeInfo attributes, that used
in network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
05f3963975 [] service/object: Implement simplified object Head service
Implement Head service w/o linking object processing and restoration from
split-chain.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
21fc85540a [] localstore: Finalize object to meta converter
Use CutPayload method of RawObject in order to make object meta structure
from source object.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
2f57855775 [] sdk/object: Add CutPayload method to RawObject
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
becb9148c1 [] sdk/object: Add Object to RawObject converter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
09084a7bff [] service/object: Implement object Search distributed service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
a5ebdd1891 [] services/object: Replace local placement to util
Replace local placement tool in order to reuse it in different object
services.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
ded0350c5e [] localstore: Store object identifier in meta storage
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
99d380c20b [] Implement netmap source interface in morph wrapper
Netmap source interface used by object service to build
placement based on current or previous network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
92fb384c09 [] Add snapshot method for morph/netmap client
With snapshot method node can get previous network
maps from previous epoch.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
57f8d3745d [] service/object: Implement object Put distributed service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
dcfb6a6b3a [] placement: Remote UseNetworkMap option from traverser
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
12e5e4e2d8 [] placement: Add public constructor for netmap placement builder
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
900949c648 [] network: Implement tool to work with local address
Implement read-only local network address source. Implement function for
check if network address is local.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d3349a293a [] core/object: Add to-SDK object converting method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
3d25d6c217 [] core/netmap: Define read-only storage interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
030375af63 [] container: Define read-only storage interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d52bdb27c5 [] util: Define worker pool interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d3b5ff9526 [] session/storage: Change Get method signature
In previous implementation session Storage.Get method took a v2 owner
identifier as an argument.

Change method signature to as an argument owner identifier from NeoFS SDK.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
4024b3c30e [] services/session: Fix signature service constructor
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
c312e11f6e [] network: Add network endpoint getter to Address
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
73ee5105ff [] placement: Fix confused selectors and replicas
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
f7c685f682 [] placement: Fix incorrect selectors processing
In previous implementation traverser worked like all counts of all selectors
are equal to counts of corresponding replicas. Make traverser to take into
account select count of all replicas.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
e7925fbc1c [] placement: Implement container placement traverser
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
44def45ff4 [] network: Make the address a separate type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d6a4965841 [] network: Remove unused dial functions
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
121137c62b [] object_manager: Implement object transformers
Define object writer interface. Implement payload size limiter that
restricts payload size of the object. Implement object format moulder that
fill verification fields and finalizes object structure.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
0a6130a924 [] core/object: Make Object constructors similar to RawObject
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
2975b61abd [] core/object: Remove no longer used MarshalStableV2 method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
87487cde00 [] localstorage: Refactor source code
Move the serialization of the address and object into separate functions and
use them in methods. Use recently implemented RawObject constructor and
method in metaFromObject function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
4326ff56a7 [] core/object: Remove redundant Address type
Remove Address type. Makes Address method of the Object to return NeoFS SDK
Address type. Makes local storage to work with NeoFS SDK object address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
8366c146d7 [] core/object: Implement raw object
Define RawObject type that inherits RawObject from NeoFS SDK. Implement
converters.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
9c1c023f05 [] Update to latest neofs-api-go and fix conflicts
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Evgenii Stratonikov
84b4ff0755 [] Support JSON format for placement policy
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
778dd54244 [] Update neofs-api-go to build images
These hotfixes will be altered with later neofs-api-go
updates.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
001a23eb31 [] Support SQL-like queries for placement
JSON format is rather verbose an inconvenient to be
edited by hand. This commit implements SQL-like
language for representing placement policy.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
f23d8a5f4a [] ir: Remove excess todo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
4aee3de24e [] ir: Add inner ring list relay processor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
f11ae1035d [] ir: Add config relay processor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
be9e7664f3 [] ir: Provide netmap contract script hash into neofs processor
Netmap contract script hash is used to synchronize global config
and inner ring list in both chains.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
a19e9c1f00 [] Add SetConfig and UpdateInnerRing method invoke helpers
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
2a142155d9 [] Add neofs inner ring update event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
7d154f8659 [] Add neofs config event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
f412d20523 [] Add neofs unbind event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
66eafcd558 [] Add neofs bind event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
df3564eaf2 [] Add [][]byte type in toStackParameter converter
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
aa98592ac4 [] Fix wrapper for add peer contract method
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
ee9551992c [] Use api-go definition of NodeInfo in node
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
f0ee275ce6 [] Use lowercase methods in netmap client
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
c299de5de9 [] Parse StackItem structs as arrays
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
8f5ea75eb6 [] Update eACL service methods
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
03b170237f Remove broken packages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
92f448f303 [] Add handlers for AddPeer and UpdatePeer events
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
7bce9a3d87 [] Add invoke helpers for AddPeer and UpdatePeer methods
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
eef20c53df [] Add AddPeer and UpdatePeer morph events
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
da92f2944f [] Fix container service according to APIv2 contracts
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
2ab855b2ec [] Allow to get bytes from buffer stackitem
Smart-contract can return slice of bytes as buffer type.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
5e5e431534 [] Add container delete notification handler
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
480b3fd1a9 [] Add container delete event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
2422d8bbfe [] Use lowercase methods in container client of storage node
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
7e099c7921 [] Use container processor in inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
c4cdfe3ec2 [] Add container processor for inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
055f7be6d1 [] Add new container invocation wrapper for inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
5f3978b66c [] Add put container event parser for morph
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
9f51c85054 [] localstorage: Adopt local object storage for new types
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
edcaef8478 [] core/object: Implement Object and Address types
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
80f10dab7b [] Fix balanceOf wrapper for api request
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
58cb90966a [] ir: Use mint and burn methods in balance contract
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
27fcf2cc1d [] Update neo-go and neofs-api-go dependencies
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
44b55740a8 [] Use notification name directly from neo-go
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
3d0064cb4c [] Use lower case in first letter of contract methods
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
b6913394a2 [] Update event subscriber constructor call
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
6520d00ff0 [] Adopt stackitems in inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
29d6ababae [] Adopt stackitems in morph requests
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
0c06eafc60 [] Use stackitems in morph/client converters
In neo-go v0.91.0 testinvoke returns stackitems.Item
interface, so converters should work with this type.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
b7702349dc [] services/util: Support client-side stream in SignService
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d54a5d4f66 [] services/util: Rename UnarySignService to SingService
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
75162d1f8d [] services/util: Support server-side stream in UnarySignService
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
6b4fb3a0aa [] services: Refactor UnarySignService
Replace UnaryHandler from structure to method arguments in order to reuse
single instance for different service methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
0c83e84613 [] services/object: Implement GetRangeHash handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
7a6b62c552 [] services/object: Implement Delete handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
881f49814a [] services/object: Implement Head handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
652f64ae85 [] services/object: Implement GetRange handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
62d0c50b15 [] services/object: Implement Put handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d03b153e09 [] object/grpc: Fix Put call implementation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
f14d1fd65f [] services/object: Implement Get handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
486aeef5df [] services/object: Define and implement Object executor
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
276ec2e1e0 [] services: Define signing Object service and implement Search
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
a0aa175fcc [] transport/object: Fix io.EOF check in Object gRPC handlers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
da1961cdf9 [] services/util: Support server-side stream request verify
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
f5a0e354ef [] services/session: Implement service executor
Implement Session service executor that generates private token and stores
it to map.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:30 +03:00
Leonard Lyubich
de12d751e9 [] services/session: Implement execution service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:24 +03:00
Leonard Lyubich
7b56633185 [] services/session: Implement Sign/Verify service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:18 +03:00
Leonard Lyubich
8539f5c2cd [] services/container: Implement Neo:Morph executor and service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:24:45 +03:00
Leonard Lyubich
9e30a87022 [] morph: Return nil on Any stack item array type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:24:36 +03:00
Leonard Lyubich
f71d64435e [] services: Implement universal Sign/Verify service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:24:30 +03:00
Leonard Lyubich
21c750016b [] transport/session: Fix package name
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:24:04 +03:00
Leonard Lyubich
5022362c1a [] accounting: Implement ServiceExecutor on Neo:Morph client
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:48 +03:00
Leonard Lyubich
7cc1a2a85a [] accounting: Define service executor
Defines ServiceExecutor interface of the executor that handles service
request bodies and returns service response bodies. Adds Service
implementation that writes the result of ServiceExecutor to response body
and attaches the internal ResponseMetaHeader.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:35 +03:00
Leonard Lyubich
5cf622881f [] accounting: Implement signing service
Adds accounting Service implementation that verifiers request signatures and
signs responses from internal Service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:29 +03:00
Leonard Lyubich
c2954aab20 [] transport: Implement gRPC server handlers of API service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:23:09 +03:00
Leonard Lyubich
a87fdab324 [] Trim the old functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:22:55 +03:00
Stanislav Bogatyrev
b7b5079934 Add Inner Ring code 2020-07-24 17:07:37 +03:00