TrueCloudLab/frostfs-node
19
1
Fork 27
Code Issues 91 Pull requests 6 Projects Releases 1 Activity Actions
5031 commits 8 branches 81 tags 52 MiB
Commit graph

181 commits

Author SHA1 Message Date
Evgenii Stratonikov
4b005d3178 [#1840] blobstor: Return info about all components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
51b8f26a31 [#1806] services/control: Allow to flush write-cache 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Leonard Lyubich
485a5418d2 [#1793] node: Serve NetmapService.NetmapSnapshot RPC 
There is no more need to serve the same request on Control API.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Evgenii Stratonikov
898689ec14 [#1731] services/replicator: Unify Task interface with other parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Evgenii Stratonikov
4e043a801c [#1731] services/control: Replicate object over network in EvacuateShard RPC 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Evgenii Stratonikov
091d7d30f6 [#1731] services/control: Allow to evacuate data from shard 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
b064fb24d8 [#1616] engine: Do not use batches in delete 
Use a simple loop at the callsite. This way we remove as much as we can.
Also, `Delete` metrics is more meaningful now.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Evgenii Stratonikov
cda8f9df2e [#1786] services/control: Remove WithDeletedObjectHandler option 
Use storage engine directly instead. It is already provided in the
options.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Leonard Lyubich
ae655b74f0 *: Exec make fmts 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-14 17:52:49 +04:00
Pavel Karpy
30341f2192 [#1687] *: Perform go fmt using go v1.19 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Evgenii Stratonikov
10c855efef [#1624] go.mod: Update dependencies 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-25 16:56:02 +03:00
Evgenii Stratonikov
4944490ffb [#1559] local_object_storage: Move shard to the DegradedReadOnly mode 
`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
339864b720 [#1559] local_object_storage: Move shard.Mode to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
bfdd68dcb3 [#1333] services/control: Allow to synchronize local trees 
Do not check that a node indeed belongs to the container, because the
synchronization will fail in this case anyway.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
5e843a73f9 [#1333] services/control: Return pilorama info in ListShards RPC 
Do not return backend type from the service for now, because memory
backend is expected to vanish.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
0408acc40e services/control: Autogenerate stable marshalers 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-20 16:52:58 +03:00
Leonard Lyubich
21d2f8f861 [#1513] Upgrade NeoFS SDK Go with changed netmap package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 15:53:18 +03:00
Pavel Karpy
6e752f36dc [#1418] shard: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
aeb9884218 [#1389] crypto: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-19 10:12:14 +03:00
Evgenii Stratonikov
0f376a5d83 [#1388] services/control: Extend tests for ShardInfo marshaling 
It is nice to have different paths for different components and also
check that the information returned is different for different shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-17 13:48:07 +03:00
Elizaveta Chichindaeva
cc7a723d77 [#1320] English Check 
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
 2022-05-11 10:40:02 +03:00
Evgenii Stratonikov
6472a170eb [#1143] shard: Introduce explicit Degraded mode 
`Degraded` mode is set automatically after error counter is over the
threshold. `ReadOnly` mode can still be set by an administrator.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Alex Vanin
7f3195b197 [#1238] Adopt neofs-cli for non pointer slices in SDK 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-16 15:29:31 +03:00
Pavel Karpy
cf17686321 [#1111] control: Rename aliaces 
`objectSDKAddress` -> `addressSDK`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-05 10:06:14 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes 
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-02-08 09:45:38 +03:00
Evgenii Stratonikov
ed50cf6207 [#1118] services/control: return error counter in ListShards 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-03 15:14:27 +03:00
Evgenii Stratonikov
7fb15fa1d0 [#1118] services/control: allow to reset error counter in SetShardMode 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-03 15:14:27 +03:00
Evgenii Stratonikov
18cfd8b042 [#1086] services/control: implement RestoreShard RPC 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-01-31 12:36:14 +03:00
Evgenii Stratonikov
2ec4a3c897 [#1086] neofs-cli: add command for shard dump 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-01-31 12:36:14 +03:00
Evgenii Stratonikov
0e60b1d6c9 [#1086] services/control: implement DumpShard RPC 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-01-31 12:36:14 +03:00
Evgenii Stratonikov
aa53418119 [#1086] services/control: fix comments 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-01-31 12:36:14 +03:00
Pavel Karpy
97fd330fae [#1059] control: Fix ListShards wrapper name 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
77808c7b41 [#1059] control: Provide shard's real mode 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
88bb8e2df9 [#1059] control: Add SetShardMode implementation to ctrl svc 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
fa78a37787 [#1059] control: Add unit test for SetShardMode 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
29c69f37eb [#1059] control: Add SetShardMode RPC wrappers 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
25665cb0fd [#1059] control: Add marshalling and setters to SetShardMode 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
4533784e11 [#1059] control: Recompile proto files 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
41dd385b7b [#1059] control: Add SetShardMode rpc to proto files 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
876b0c53de [#1048] control: Rename Blobstore to Blobstor 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
f5a9735e1c [#1048] cli: Add shard list operation 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
0e5410603e [#1048] control: Add ListShards implementation to ctrl svc 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
4e989e7133 [#1048] control: Add unit test for ListShards 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
f88a12eaa7 [#1048] control: Add ListShards RPC wrappers 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
a6b3e16975 [#1048] control: Add marshalling and setters to ListShards 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
a42b3d37f6 [#1048] control: Recompile proto files 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Pavel Karpy
20f11c88ca [#1048] control: Add ListShards rpc to proto files 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-28 13:21:44 +03:00
Leonard Lyubich
e8f8e58e90 [#979] adm: Add subnet-related commands 
Add `subnet` command which contains all subnet-related commands. Add
sub-commands:
  * `create` for creation;
  * `remove` for removal;
  * `get` for reading;
  * `admin` for admin management;
  * `client` for client management.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-12-01 17:31:04 +03:00
Leonard Lyubich
7f5fb130c0 [#961] *: Support NeoFS API status returns 
Upgrade NeoFS API Go library to version with status returns. Make all API
clients to pull out and return errors from failed statuses. Make signature
service to respond with status if client version supports it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-24 09:26:40 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-12 17:29:09 +03:00
Leonard Lyubich
8f8b638b95 [#922] control: Add MAINTENANCE netmap status 
Add `MAINTENANCE` value to `NetmapStatus` enum in Control API. The status is
going to be used to toggle maintenance mode of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-12 17:28:38 +03:00
Leonard Lyubich
be05bed0b3 [#860] Re-compile protobuf files after make protoc target update 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-09-29 17:51:33 +03:00
Leonard Lyubich
8eadf18e01 [#607] services/control: Fix incorrect allocation in NetmapSnapshot 
Fix make with length + subsequent append.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-06-28 15:52:50 +03:00
Leonard Lyubich
1e52e86bbc [#607] node/control: Make group address in NodeInfo message 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-06-28 15:52:50 +03:00
Leonard Lyubich
455fd952dd [#414] ir: Serve ControlService 
Serve `ControlService` instance on configured endpoint (do not serve if not
specified). Read allowed keys from config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-06-15 15:21:52 +03:00
Leonard Lyubich
dcfe9a6504 [#414] ir/control: Implement service server 
Implement `ControlServiceServer` on `Server` type. The `Server` requires all
requests to be signed with keys from the so-called whitelist. To obtain
health status, it uses the abstraction in the form of `HealthChecker`
interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-06-15 15:21:52 +03:00
Leonard Lyubich
93803b1a90 [#414] ir: Define Control service 
Define `ControlService` for IR similar to the one from storage node. Add
`HealthStatus` RPC which returns health status of the IR application.
Implement getters, setters and methods to sign/verify the messages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-06-15 15:21:52 +03:00
Alex Vanin
89b147ebe7 [#521] Recompile protobuf files after transition from pkg/errors 
Autogenerated files were accidentally modified. This commit returns
them back to the original.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-05-19 19:09:35 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-05-19 16:42:54 +03:00
Leonard Lyubich
4640099a67 [#425] control: Implement service RPCs 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-03-17 15:00:37 +03:00
Leonard Lyubich
aa24702ebe [#401] control: Serve DropObjects RPC 
Re-compile protobuf definition of Control service. Implement required
messages on DropObjects RPC request and response messages. Implement
`DropObjects` method on Control service server of the node. Use
`StorageEngine.Delete` method as a deleted object handler on server.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-02-24 13:07:53 +03:00
Leonard Lyubich
f2337baedc [#401] control: Add RPC to mark objects to be removed from the node 
There is a need to define storage node's interface to remove objects from
the local storage.

Define `DropObjects` RPC in `ControlService`. Define `DropObjectsRequest` /
`DropObjectsResponse` structure of the request / response messages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-02-24 13:07:53 +03:00
Leonard Lyubich
619f8826e1 [#315] control: Implement SetNetmapStatus on Server 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 17:46:12 +03:00
Leonard Lyubich
f39d08bda7 [#315] control: Add SetNetmapStatus rpc 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 17:46:12 +03:00
Leonard Lyubich
e5dc4ede57 [#312] control: Add newline at the end of proto file 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:52:08 +03:00
Leonard Lyubich
f327024ee7 [#312] control/server: Extended HealthChecker interface 
Add HealthStatus method to HealthChecker interface that should return
current health status of the node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:52:08 +03:00
Leonard Lyubich
6ae51e41f9 [#312] control: Add health status field to health check response body 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:52:08 +03:00
Leonard Lyubich
0abb8acef3 [#312] control: Define HealthStatus enum 
Define enumeration of the statuses of storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:52:08 +03:00
Leonard Lyubich
83d4420a30 [#313] control: Rename HealthStatus enum to NetmapStatus 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:52:08 +03:00
Leonard Lyubich
c75a828adf [#311] control: Fix loss of parent attribute keys in netmap converter 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
6c9dab481c [#311] control: Cover StableMarshal methods with unit tests 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
5f22e7fac7 Add newline at end of Control service proto file 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
36d8e2d3b7 [#311] control: Implement JSON marshaler on Netmap message 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
dde24bc9e5 [#311] control: Rewrite methods for binary encoding and signatures 
Rewrite marshalers and sizers using proto library from API repo.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
e9d4b2d25b [#311] Recompile protobuf files 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
3e882d963f [#311] services/control: Implement NetmapSnapshot method 
Implement methods required for signatures. Receive network map from netmap
storage, convert it to Control service message and return in response.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
28777911fb [#311] services/control: Define NetmapSnapshot rpc 
Add NetmapSnapshot rpc to ControlService protobuf definition. Recompile
proto files. Add required method to server structure.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
88023f3655 [#311] services/control: Define Netmap structure 
Define NodeInfo protobuf type. Define Netmap protobuf type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-15 16:11:50 +03:00
Leonard Lyubich
0a0ee89665 [#306] control: Rename WithAllowedKeys function to WithAuthorizedKeys 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-14 12:13:49 +03:00
Leonard Lyubich
abd9952e46 [#306] Rename Private service to Control service 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-14 12:13:49 +03:00