Commit graph

258 commits

Author SHA1 Message Date
Alex Vanin
c35e15a758 [#249] pkg/netmap: Add CBF field in placement context
If CBF value is not set, then netmap package uses default CBF
value. However it modifies placement policy structure in
`GetContainerNodes()` because policy passed as a pointer.

Instead package can store CBF value in internal context and use
it without policy modification.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-27 15:47:06 +03:00
Alex Vanin
9401724b9c [#246] pkg/client: Use io.Reader wrapper at object put
Object payload transferred in chunks. Size of the
chunks may be limited by transport protocols. To
split it we use `io.CopyBuffer` with pre-allocated
buffer size of one chunk. However this function may
ignore buffer if reader or writer implements
`WriteTo` or `ReadFrom` methods. Unfortunately
`bytes.Reader` implements `WriteTo` function.

To fix this we wrap reader so wrapper implements
only `io.Reader` interface.

Related to github.com/nspcc-dev/neofs-node/issues/338

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-25 13:25:52 +03:00
Alex Vanin
8a82400451 [#243] Fix golint linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:22:07 +03:00
Alex Vanin
c2db2a54a1 [#243] Fix exhaustive linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:22:07 +03:00
Alex Vanin
aafeb70bdb [#235] v2/audit: Regenerate protobuf
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:20:34 +03:00
Alex Vanin
1c25c3904b [#234] Update audit result struct definition
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:20:34 +03:00
Leonard Lyubich
9986a4ecd1 [#233] Implement data audit result type with basic methods
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:20:34 +03:00
Leonard Lyubich
f0ebe80a78 [#231] object/search: Implement method to add search filter by type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:20:26 +03:00
Leonard Lyubich
53e2756762 [#231] object: Implement string encode/decode methods on Type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:20:26 +03:00
Leonard Lyubich
0f04087543 [#230] pkg: Implement string encode/decode methods on Checksum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:20:26 +03:00
Leonard Lyubich
6861de042b [#230] pkg: Implement storage group type with basic methods
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:20:26 +03:00
Leonard Lyubich
37e2cab93b [#229] pkg/client: Set hash fields to result of short HEAD operation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:20:26 +03:00
Leonard Lyubich
c4f7be19ea [#194] pkg/client: Verify container format in GetContainer method
Make Client.GetContainer method to return an error if received container
structure does not meet NeoFS API specification.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 12:54:53 +03:00
Leonard Lyubich
7988405753 [#194] pkg/container: Implement container constructor with format check
Implement NewVerifiedFromV2 function that verifies format of NeoFS API V2
Container message.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 12:54:53 +03:00
Leonard Lyubich
70c29ca3e5 [#194] pkg/container: Implement container nonce getter/setter as UUID
Implement NonceUUID/SetNonceUUID methods on container structure. Change
implementation of Nonce/SetNonce methods and mark them deprecated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 12:54:53 +03:00
Leonard Lyubich
3550e128bb [#236] Fix SDK minor version number
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 10:35:33 +03:00
Leonard Lyubich
5395988efc [#208] pkg/client: Add function to get container and verify ID
GetContainer method reads container structure by identifier from the
network. In some cases it is required to additionally check the
correspondence of the container structure to the identifier as a hash from
the binary representation. To do this, a new function
GetVerifiedContainerStructure is defined to execute the check after
receiving the container.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-17 14:52:04 +03:00
Leonard Lyubich
84839b09c2 [#208] pkg/container: Define standard error for mismatch identifiers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-17 14:52:04 +03:00
Leonard Lyubich
1bc91466aa sdk/client: Fix setter of objectAddressWriter structure value
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 11:19:45 +03:00
Leonard Lyubich
803c91b3eb [#226] sdk/client: Add tombstone address to the return of DeleteObject
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-10 18:26:23 +03:00
Leonard Lyubich
c064760f5d [#226] sdk/object: Implement Tombstone type with field access and encoding
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-10 18:26:23 +03:00
Alex Vanin
139660c6ff [#225] pkg/client: Update object methods
- Support errors with SplitInfo in head and range methods.
- Support raw flat in range method.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-08 10:23:27 +03:00
Alex Vanin
28aad71860 [#225] Do not use wrappers on oneof object.HeadResponse types
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-08 10:23:27 +03:00
Alex Vanin
6db6b569e0 [#223] pkg/client: Add getters for object params
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 18:07:42 +03:00
Alex Vanin
1fe0307711 [#223] pkg/object: Work with SplitInfoError via pointer
Closes #221

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 18:07:42 +03:00
Alex Vanin
b683dbe7e6 [#218] pkg/object: Add marshal operations for SplitInfo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 16:09:38 +03:00
Alex Vanin
664ebfd8a7 [#218] pkg/client: Add raw flag for object.Head
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 16:09:38 +03:00
Alex Vanin
582cdd4ba3 [#218] pkg/client: Add raw flag for object.Get
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 16:09:38 +03:00
Alex Vanin
e11b1728be [#218] pkg/object: Define custom error for SplitInfo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 16:09:38 +03:00
Alex Vanin
1fdeca84e1 [#218] Support SplitInfo structure
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-03 16:09:38 +03:00
Evgenii Stratonikov
707a0bcb35 [#220] netmap: process REP X policies correctly
For `REP X` select X nodes from the default filter
and fail if it cannot be done.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-12-03 15:39:49 +03:00
Evgenii Stratonikov
576841e0e0 [#215] netmap: add test with multiple replicas
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-12-01 13:33:11 +03:00
Evgenii Stratonikov
4e0a11b71a [#156] netmap: Use default value for CBF if unset
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-12-01 13:33:11 +03:00
Evgenii Stratonikov
4e65fa41b8 [#213] netmap: Do not append not twice if selector is unnamed
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-12-01 13:33:11 +03:00
Evgenii Stratonikov
ea6b6adfe0 [#157] netmap: Fallback to using minimal backup factor
Use CBF=1 when strict policy can't be satisfied.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-12-01 13:33:11 +03:00
Leonard Lyubich
c01024b553 [#217] pkg/eacl: Change interface for working with keys on Target and Record
In previous implementation Target provided Keys/SetKeys methods which
allowed working with ECDSA keys. There was also a bug in the NewTargetFromV2
function when the binary key differed in format from the ECDSA key. New
BinaryKeys/SetBinaryKeys methods work with binary keys. To work with ECDSA
keys added functions TargetECDSAKeys/SetTargetECDSAKeys. Old methods are
left and marked deprecated.

Type Record provided an interface for adding a Target by Role and a list of
ECDSA keys. New SetTargets method allows to set the list of Target's,
AddTarget function allows to add a single Target. AddFormedTarget works like
old AddTarget method, which is now deprecated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 10:11:32 +03:00
Alex Vanin
2a94fdc5e7 [#209] Support nil value in SplitID
SplitID is not set on non-split and virtual
objects, so we should support this state in
library.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-26 13:26:34 +03:00
Alex Vanin
3d08d8140f [#209] object: Support splitID search attribute
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-26 11:56:30 +03:00
Alex Vanin
79c76e87e4 [#209] object: Remove childfree search attribute
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-26 11:56:30 +03:00
Alex Vanin
6b0bd42f25 [#209] pkg/object: Support splitID field in SDK library
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-26 11:56:30 +03:00
Leonard Lyubich
8c99d9d54c [#207] sdk/client: Implement GetEACLWithSignature method
GetEACL method of Client receives eACL table with signature, verifies the
signature and return the table. In some cases we need to receive table and
signature regardless of their correctness. New method GetEACLWithSignature
provides such an opportunity.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 15:16:42 +03:00
Leonard Lyubich
3ebfef9f94 [#207] sdk/client: Fix signature verification in GetEACL method
Extended ACL tables should be signed with RFC-6979 standard. Fix Client
.GetEACL implementation to verify eACL signature with SignRFC6979 option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 15:16:42 +03:00
Leonard Lyubich
5ee500bb43 [#205] sdk/client: Support option to set dial timeout
There is a need to set dial timeout in SDK client that is used in case of
internal connection opening. Add DialTimeout option constructor to support
this feature.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-23 14:15:42 +03:00
Leonard Lyubich
20ede88fe7 [#204] sdk/object: Add SearchFilters method to filter by object ID
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-20 10:41:17 +03:00
Alex Vanin
9ceba98198 [#202] pkg/client: Fix signature check in object header getter
According to API object.Head response contains signature of
object ID rather than signature of object header itself.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-17 16:23:44 +03:00
Leonard Lyubich
0620a3b1eb [#199] sdk/client: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
00ce980d82 [#199] sdk/token: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
c0de2bf9e5 [#199] sdk/owner: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
1ff8b3fd94 [#199] sdk/object: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
3a966ee5df [#199] sdk/netmap: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
459d295788 [#199] sdk/container: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
7611c218e3 [#199] sdk/client: Verify eACL table signature in GetEACL method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
3097059f85 [#199] sdk/client: Verify header signature in GetObjectHeader method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
c6884f9823 [#199] sdk/eacl: Correct linter's remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
2b3502d6c0 [#199] sdk/acl: Add comments for basic ACL constants
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
154e09f4e4 [#199] sdk/accounting: Rename test file package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
79f72e10c9 [#199] sdk: Refactor string parse errors
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 18:51:14 +03:00
Leonard Lyubich
fe336fd5ba [#198] sdk/client: Use XHeader type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 12:44:37 +03:00
Leonard Lyubich
5452554b58 [#198] sdk: Implement XHeader type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 12:44:37 +03:00
Leonard Lyubich
5f5e5ac5dd [#197] sdk/object: Rename getters of Attribute and Object types
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 11:52:21 +03:00
Leonard Lyubich
67bcf6eb4d [#197] sdk/object: Rename getters of Address type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 11:52:21 +03:00
Leonard Lyubich
9f0bc50e53 [#197] sdk: Rename getters of Checksum type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 11:52:21 +03:00
Leonard Lyubich
2afc684313 [#197] sdk: Rename getters of Signature type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 11:52:21 +03:00
Leonard Lyubich
2bcbd48972 [#197] sdk: Rename getters of Version type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-16 11:52:21 +03:00
Alex Vanin
f886d75560 [#168] pkg/object: Fix lint error on attribute receiver
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
85d4713348 [#168] session: Implement binary/JSON encoders/decoders on SessionToken
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
93df2fd765 [#168] refs: Implement binary/JSON encoders/decoders on Checksum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
4b55b06780 [#168] refs: Implement binary/JSON encoders/decoders on Signature
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
1519a02d63 [#168] refs: Implement binary/JSON encoders/decoders on Version
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
9ec57ee9f8 [#168] refs: Implement binary/JSON encoders/decoders on OwnerID
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
c0f3ac51d4 [#168] refs: Implement binary/JSON encoders/decoders on ContainerID
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
7289ff6c64 [#168] refs: Implement binary/JSON encoders/decoders on ObjectID
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
4ce751f13c [#168] refs: Implement binary/JSON encoders/decoders on Address
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
9325e22871 [#168] object: Implement binary/JSON encoders/decoders on Object
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
0caff85fb7 [#168] object: Implement binary/JSON encoders/decoders on Attribute
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
a684da6118 [#168] container: Implement binary/JSON encoders/decoders on Container
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
ec957be60c [#168] acl: Implement binary/JSON encoders/decoders on BearerToken
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
233756ca8f [#168] acl: Implement binary/JSON encoders/decoders on Table
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
ae68790bbd [#168] acl: Implement binary/JSON encoders/decoders on Record
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
9ddc4c1f48 [#168] acl: Implement binary/JSON encoders/decoders on Target
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
7f42156201 [#168] acl: Implement binary/JSON encoders/decoders on HeaderFilter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
9bebc1247d [#168] accounting: Implement binary/JSON encoders/decoders on Decimal
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
52fae76533 [#168] sdk/accounting: Refactor Decimal type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Leonard Lyubich
e721734599 [#168] netmap: Implement binary and JSON encoders/decoders on Filter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-13 18:07:26 +03:00
Alex Vanin
874a4b937f [#193] pkg/client: Fix error message typos
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-13 12:28:00 +03:00
Alex Vanin
d442b204ca [#193] pkg/client: Update container structure
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-13 12:28:00 +03:00
Alex Vanin
5ff7b5ba6b [#193] pkg/container: Increase test coverage
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-13 12:28:00 +03:00
Alex Vanin
f16a420bed [#193] pkg/container: Wrap container structure with SDK functions
All setters and getters should work with SDK types.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-13 12:28:00 +03:00
Alex Vanin
abed8a9156 [#191] v2: Add ObjectID filters for ACL and Object packages
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-11 10:35:34 +03:00
Alex Vanin
fe48e3ab5b [#191] pkg/object: Use root and phy filters as flags
Inverted values for `root` and `phy` filters now are not supported.
Therefore these filters works like flags. It either presented in
search query or not.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-11 10:35:34 +03:00
Alex Vanin
91eade02cd [#191] v2/object: Rename leaf filter to phy
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-11 10:35:34 +03:00
Alex Vanin
d704795dcc [#189] sdk/netmap: Define unspecified clause
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
162b0de725 [#189] sdk/client: Return NodeInfo SDK type from EndpointInfo method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
9834ed4e68 [#189] sdk/netmap: Get rid of the use of v2 types in the placement code
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
40d7c9cd93 [#189] sdk/netmap: Implement NodeInfo JSON encode/decode functions
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
8367b498d9 [#189] sdk/netmap: Implement NodeInfo type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
6347f846eb [#189] sdk/netmap: Implement NodeAttribute type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
d4f5c27d47 [#189] sdk/netmap: Implement NodeState enum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
53697081ce [#189] sdk: Fix type conflicts after netmap package refactoring
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00
Leonard Lyubich
9f19139999 [#189] sdk/netmap: Refactor PlacementPolicy type
Replace alias to v2 type PlacementPolicy with v2-compatible implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-06 09:28:50 +03:00