Commit graph

123 commits

Author SHA1 Message Date
Leonard Lyubich
e0dce1043a [#1195] Adopt recent changes in NeoFS SDK
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-02-28 10:43:03 +03:00
Evgenii Stratonikov
050a4bb2b0 [#1115] *: link TODOs to corresponding issues
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-02-08 09:45:38 +03:00
Leonard Lyubich
4f3323f084 [#1101] *: Adopt interface changes of API client from SDK library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-02-03 14:00:14 +03:00
Pavel Karpy
fee2f5a330 [#1100] *: Adopt new SDK's owner.ID API
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-01-21 17:58:30 +03:00
Pavel Karpy
7c02a2e251 [#1087] *: Adopt SDK changes
- Update `neofs-sdk-go`:
v0.0.0-20211230072947-1fe37df88f80 => v0.0.0-20220113123743-7f3162110659

- Add client interface that duplicates SDK's client behaviour and new
`MultiAddressClient` interface that has method that iterates over wrapped
clients.

- Also start using simple client mode that does not require parsing statuses
outside the SDK library.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-01-14 17:29:03 +03:00
Leonard Lyubich
6f41cd86ed [#979] Upgrade SDK NeoFS Go dependency
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
7f5fb130c0 [#961] *: Support NeoFS API status returns
Upgrade NeoFS API Go library to version with status returns. Make all API
clients to pull out and return errors from failed statuses. Make signature
service to respond with status if client version supports it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 09:26:40 +03:00
Alex Vanin
1f911830a5 [#948] engine: Define EndOfListing error in component packages
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-11-19 16:57:19 +03:00
Alex Vanin
8d471c7e36 [#948] core/object: Move listing objects error to core
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-11-19 16:57:19 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-12 17:29:09 +03:00
Pavel Karpy
b148b85af4 [#919] pkg/object: Do not accept objects with empty owner
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-18 12:15:06 +03:00
Leonard Lyubich
4661f65975 [#645] client/cache: Check response public key in all client operations
There is a need to check if public key in the RPC response matches the
public key of the related storage node declared in network map.

Define `ErrWrongPublicKey` error. Implement RPC response handler's
constructor `AssertKeyResponseCallback` which checks public key. Construct
handler and pass it to client's option `WithResponseInfoHandler`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
3d3d30560a [#645] core/client: Implement helper functions to fill NodeInfo
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
4ec7e24b85 [#645] core/client: Add public key to NodeInfo structure
There is a need to process announced public keys of storage nodes for
intra-container communication.

Implement `PublicKey` / `SetPublicKey` methods of `client.NodeInfo` type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
91cc33bdb9 [#645] core/client: Define node information structure
Define `client.NodeInfo` structure which groups data required for API client
construction.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
f14987c06d [#645] core: Define interface of the tool for working with netmap keys
Announced keys of storage nodes are required for many application components
to function.

Define a unified interface for the utility for working with public keys of
nodes. Add a method to check if the key has been advertised by the local
node in the application context.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-07 09:53:18 +03:00
Leonard Lyubich
54d6f3de55 [#660] core/container: Use version.IsValid in CheckFormat
Container can have version later than the current lib version, and it should
not be considered as invalid. But it should still have adequate version.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-07-05 11:05:44 +03:00
Leonard Lyubich
69826ebd90 [#660] core: Implement function to check protocol version adequacy
Create `version` package and implement `IsValid` function which checks if
version is not earlier than the genesis one.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-07-05 11:05:44 +03:00
Leonard Lyubich
3805b0f638 [#607] client: Overload Client interface
There is a need to generalize single-address client to group-address client.
To do this, we can re-implement `Client` interface from NeoFS API Go library
and still use it in the application code. There is a problem with method
`Raw` which must return single-address raw client. So as not to make changes
to API library we need to overload Client interface in order to support
`Raw` method in group-address client implementation.

Define `Client` interface in new `pkg/core/client` package. Completely
inherit API `Client` interface. Add `RawForAddress` method to build raw
client for the single node address. Adopt the application code that used Raw
method to work with new `Client`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
25391111ad [#633] object: Disallow empty attribute values
Values of object attributes must not be empty according to NeoFS
specification.

Make `FormatValidator.Validate` method to return an error if at least one
attribute has empty value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-24 11:12:23 +03:00
Leonard Lyubich
01dd17e30a [#633] object: Prevent duplicate attributes by key
Keys of object attributes must be unique according to NeoFS specification.

Make `FormatValidator.Validate` method to return an error if at least one
attribute is duplicated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-24 11:12:23 +03:00
Pavel Karpy
6efeee5ce0 [#641] ir/container: Add unique attributes check
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 11:11:13 +03:00
Evgenii Stratonikov
9142c778c7 [#562] pkg: remove nspcc-dev/neofs-crypto dependency
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Leonard Lyubich
d34de558f0 [#570] *: Use generator of test owner IDs from API Go lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
3e1463cc76 [#570] *: Use generator of test container IDs from API Go lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
3dd10b6795 [#570] *: Remove usage of deprecated elements from API Go library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
70a7354e9d [#570] *: Use new Equal method of owner.ID type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
a1e1350db0 [#525] core/container: Extend removal witness with session token
NeoFS containers can be removed within a trusted session. There is a need to
take this into account during removal inspection.

Define `SessionToken` / `SetSessionToken` methods on `RemovalWitness` struct
in order to embed `session.Token` to it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
d691a20d52 [#525] core/container: Define removal witness
Define `RemovalWitness` structure which groups the information required to
prove and verify the removal of a container. This type is going to be used
in container Delete-related methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Alex Vanin
b5256ccf4c [#521] Fix issues with transition from pkg/errors pkg
Wrap functions at `pkg/errors` return nil if error argument
was nil. fmt.Errorf always returns error so we need to add
missing error checks to the code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-19 19:09:35 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-19 16:42:54 +03:00
Alex Vanin
5f47580ff7 [#399] object/fmt: Fix test for tombstone epoch expiration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-02-19 16:20:10 +03:00
Alex Vanin
e6cdf3fbf5 [#399] object/fmt: Check expiration epoch in tombstone body and header
According to nspcc-dev/neofs-api#136 tombstone body should store the same
attribute as in object header. If they are different, then check is failed
with `errTombstoneExpiration`.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-02-19 16:20:10 +03:00
Leonard Lyubich
38727c2930 [#368] object: Reject expired objects
The lifetime of an object can be limited by specifying a correspondin
well-known attribute. Node should refuse to save expired objects.

Checking objects in FormatValidator is extended with an expiration attribute
parsing step.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-15 19:04:33 +03:00
Alex Vanin
62de9f327e [#319] Do not return payload formatting error on regular objects
Regular objects by definition have valid payload, so there should
be no errors.

Related to #302

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-15 16:01:29 +03:00
Leonard Lyubich
2f4d90025f [#234] core/netmap: Extend Source interface
Add GetNetMapByEpoch method. Add Epoch method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Alex Vanin
7e30b0b9c8 [#302] Fix exhaustive linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Leonard Lyubich
9a86fff7e0 [#291] Remove some unused code from repository
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
8c4e033db3 [#80] core/object: Add SG-object format validation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
e53bf574b5 [#279] container: Use new methods to work with container format
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
Leonard Lyubich
510e9ff2ec [#243] core/object: Replace test content of tombstone with an API structure
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
351e4b4592 [#222] Support Inhume and Delete in object service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
67d4b0db12 [#222] Move ErrAlreadyRemoved to core package
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
47d2239332 [#220] localstorage: Replace basic errors to core library
Replace ErrNotFound and ErrRangeOutOfBounds to core/object package in order
to share them across the libraries.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
fddc50fd85 [#203] Replace ErrEACLNotFound to core library
ErrEACLNotFound error was defined in implementation package. EACL validator
checked this error after the call of eACL storage interface method. Replace
ErrEACLNotFound to core container library. in order to: on the one hand not
use an implementation error, on the other hand, to be able to reuse a
generic type error (404).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 20:19:20 +03:00
Leonard Lyubich
1caf15463e [#174] Update to neofs-api-go v1.20.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
Leonard Lyubich
3de8febe57 [#174] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
Alex Vanin
65be09d3db [#155] Update neofs-api-go with refactored pkg/netmap
Refactored pkg/netmap package provides JSON converters for
NodeInfo and PlacementPolicy structures, that has been used
by client applications.

It also updates Node structure itself so it is a part of
grpc <-> v2 <-> pkg conversion chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-06 09:55:05 +03:00
Leonard Lyubich
a3e0a9f74c [#85] core/container: Implement CheckFormat function
Implement function that conducts sanity checks of the container data.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-03 14:14:38 +03:00
Leonard Lyubich
7c081e4f15 [#60] core/netmap: Define interface of current network properties
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-22 11:54:08 +03:00
Leonard Lyubich
798fca9354 [#70] core/object: Process a delete group at tombstone
Send object group to delete queue processor after tombstone content
validation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
Leonard Lyubich
33ca88f85f [#64] core/object: Claim TombstoneContent in tombstone payload
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 19:46:27 +03:00
Leonard Lyubich
21bdc82fb5 [#64] core/object: Implement tombstone content messages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 19:46:27 +03:00
Leonard Lyubich
1654df4d97 [#61] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
2abb03dbd1 [#58] core/object: Decouple the payload content validation method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
107f3097e4 [#58] core/object: Add content validation to FormatValidator
Add content validation step to FormatValidator. Check tombstone payload
correctness.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
68178b8d74 [#57] core/object: Fix validation of linking object
Linking object carries only identifier of the parent object. Fix Validate
method of FormatValidator to not validate parent object of the linking
object.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
ede033256d [#56] core/object: Implement format validator
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
2f57855775 [#38] sdk/object: Add CutPayload method to RawObject
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
becb9148c1 [#38] sdk/object: Add Object to RawObject converter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
d3349a293a [#33] core/object: Add to-SDK object converting method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
3d25d6c217 [#33] core/netmap: Define read-only storage interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
030375af63 [#33] container: Define read-only storage interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
0a6130a924 [#30] core/object: Make Object constructors similar to RawObject
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
2975b61abd [#30] core/object: Remove no longer used MarshalStableV2 method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
4326ff56a7 [#30] core/object: Remove redundant Address type
Remove Address type. Makes Address method of the Object to return NeoFS SDK
Address type. Makes local storage to work with NeoFS SDK object address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
8366c146d7 [#30] core/object: Implement raw object
Define RawObject type that inherits RawObject from NeoFS SDK. Implement
converters.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
9c1c023f05 [#30] Update to latest neofs-api-go and fix conflicts
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
778dd54244 [#28] Update neofs-api-go to build images
These hotfixes will be altered with later neofs-api-go
updates.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
edcaef8478 [#17] core/object: Implement Object and Address types
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
a87fdab324 [#11] Trim the old functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:22:55 +03:00
Stanislav Bogatyrev
b7b5079934 Add Inner Ring code 2020-07-24 17:07:37 +03:00