EliChin/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
795 commits 12 branches 26 tags 25 MiB
Commit graph

210 commits

Author SHA1 Message Date
Leonard Lyubich
611a29f682 [#241] object/search: Refactor service processing 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
0e1f05ff45 [#239] object/head: Implement new service processing 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
7e56427534 [#235] object/getrangehash: Implement new service processing 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
1d23483828 [#235] services/object: Implement new GetRange algorithm 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
a875d80491 Remove outdated code of metabase and localstore 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
869d9e571c [#233] object/head: Remove getting the right child header 
With the update of the local storage engine, the headers of virtual objects
are directly given. In this regard, the step with obtaining the the right
child header is removed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
f24daa10ff [#233] services/object: Implement new Get algorithm 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
a488d202e1 [#230] engine: Rename WithAddress to WithAddresses for delete operations 
Delete works with multiple addresses at once so it is convenient to
show it in parameter setter function.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
3d77fdb347 [#222] Set write cache specific config on application level 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
4114028a63 [#222] Add shard amount check 
There should be at least one correctly set up shard in node config.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
351e4b4592 [#222] Support Inhume and Delete in object service 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
f45b7a048d [#222] Rename WithTreeRoot.. to WithRoot.. in blobstor opts 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
92c95150cd [#222] Use write cache options in storage app 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
2eb83624cf [#220] cmd/neofs-node: Configure Shard's write cache usage 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
cd2228cd99 [#220] cmd/neofs-node: Support new configuration of storage components 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
a1164b2a5c [#211] cmd/neofs-node: Add SmallSizeLimit config value of BlobStor 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
4ca1035aa4 [#186] cmd/neofs-node: Integrate new storage engine into application 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
49666f87f1 [#170] Implement --vote argument in neofs-ir 
With this argument neofs-ir will be run as an CLI
application that initialize inner ring server, invokes
`vote` method on corresponding alphabet contract and
exits.

User can provide a comma-separated list of validator
public keys.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Alex Vanin
0b65c64e98 [#170] Vote for validators on inner ring startup 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Alex Vanin
4c0ca23b34 [#208] Print split ID in object.Head output of neofs-cli 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-30 10:44:15 +03:00
Leonard Lyubich
0168740c9a [#206] cli/container: Set default version of eACL table in set-eacl 
Set version of parsed eACL table to using SDK version if unsupported version
was set in source data.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-25 14:35:27 +03:00
Leonard Lyubich
b4699b1e9f [#206] cli/container: Use GetEACLWithSignature in set-eacl polling 
Replace GetEACL call with GetEACLWithSignature one in polling step of
set-eacl command of container CLI.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-25 14:35:27 +03:00
Leonard Lyubich
e8bc03d92f [#206] cli/container: Use GetEACLWithSignature client method 
Use GetEACLWithSignature client method in get-eacl cmd of container CLI in
order to print eACL table and signature regardless of their correctness. The
ability to check the correctness of the signature will be added later.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-25 14:35:27 +03:00
Leonard Lyubich
621840d542 [#193] cmd/neofs-node: Add dial timeouts to node configuration 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 15:34:30 +03:00
Leonard Lyubich
400b65801d [#201] cli/container: Do not check eACL version 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:56:03 +03:00
Leonard Lyubich
d286824d1d [#194] cmd/neofs-node: Use non-blocking worker pools for Object services 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:56 +03:00
Leonard Lyubich
6a5c37d592 [#195] object/search: Add option to set logger 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:06 +03:00
Leonard Lyubich
cb46e4b154 [#195] object/rangehash: Add option to set logger 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:06 +03:00
Leonard Lyubich
af6484e3b1 [#195] object/range: Add option to set logger 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:06 +03:00
Leonard Lyubich
cf2dc37a42 [#195] object/put: Add option to set logger 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:06 +03:00
Leonard Lyubich
9fbfc0b5e4 [#195] object/head: Add option to set logger 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:06 +03:00
Leonard Lyubich
e1e5a590e9 [#195] object/delete: Add option to set logger 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-24 13:38:06 +03:00
Leonard Lyubich
ea36789b51 [#179] cmd/neofs-cli: Implement range command in Object section 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-23 13:23:31 +03:00
Leonard Lyubich
dbfb6fadd6 [#177] cmd/neofs-cli: Add object ID filter flag to search cmd 
Add "oid" flag that allows to filter objects by identifier.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-23 09:54:59 +03:00
Alex Vanin
e8fe07edd0 [#184] Use SDK client cache in object.Rangehash 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-18 18:18:07 +03:00
Alex Vanin
f85e88c4f8 [#184] Use SDK client cache in object.Range 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-18 18:18:07 +03:00
Alex Vanin
d485a5967d [#184] Use SDK client cache in object.Search 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-18 18:18:07 +03:00
Alex Vanin
7ba95dd5fc [#184] Use SDK client cache in object.Put 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-18 18:18:07 +03:00
Alex Vanin
e9a6365333 [#184] Use SDK client cache in object.Head 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-18 18:18:07 +03:00
Leonard Lyubich
58fcb35fb0 [#174] Use Marshal(JSON)/Unmarshal(JSON) methods for encoding/decoding 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-17 11:56:00 +03:00
Leonard Lyubich
3de8febe57 [#174] Update to latest neofs-api-go changes 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-17 11:56:00 +03:00
Alex Vanin
e47c866d6a [#169] Add --ttl argument in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-16 12:50:21 +03:00
Alex Vanin
e580f2afd6 [#160] Update object CLI flags 
* remove no-leaf
* rename leaf to phy
* remove no-root

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-16 10:02:12 +03:00
Alex Vanin
eb13322bf0 [#166] Use async pools in object service handlers 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-13 15:54:00 +03:00
Alex Vanin
ded45e1fbc [#159] Add keyer to neofs-cli 
Keyer prints information about private key, public key,
NEO3 Wallet, scripthash. It can generate new private key
or generate multisig address.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-09 18:59:21 +03:00
Leonard Lyubich
3c17e871e1 [#158] cmd/neofs-node: Handle objects from tombstone by metabase 
Handle tombstone content by metabase instead of object GC queue.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-09 14:07:20 +03:00
Leonard Lyubich
36d5af812e [#158] metabase: Add logger to DB configuration 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-09 14:07:20 +03:00
Leonard Lyubich
60e4b5ddff [#158] metabase: Construct DB using options 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-09 14:07:20 +03:00
Alex Vanin
0021f1cc5f [#81] cmd/neofs-cli: Use hex encoding of node's public key 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-06 16:37:48 +03:00
Leonard Lyubich
25398262fc [#81] cmd/neofs-cli: Implement "nodeinfo" sub-cmd of "netmap" cmd 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-06 16:37:48 +03:00
Leonard Lyubich
5d94594007 [#81] cmd/neofs-cli: Implement "epoch" sub-cmd of "netmap" cmd 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-06 16:37:48 +03:00
Alex Vanin
65be09d3db [#155] Update neofs-api-go with refactored pkg/netmap 
Refactored pkg/netmap package provides JSON converters for
NodeInfo and PlacementPolicy structures, that has been used
by client applications.

It also updates Node structure itself so it is a part of
grpc <-> v2 <-> pkg conversion chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-06 09:55:05 +03:00
Alex Vanin
49da96d006 [#146] Set stdout as default output for CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-05 18:37:24 +03:00
Alex Vanin
64caa15556 [#148] Use base name as FileName attribute value 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-05 18:37:06 +03:00
Alex Vanin
54818d5a11 [#144] Support well-known application attributes for object in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-05 09:28:13 +03:00
Alex Vanin
83926647d7 [#144] Support well-known application attributes for container in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-05 09:28:13 +03:00
Leonard Lyubich
8125b544b4 [#136] cmd/neofs-node: Use new metabase in app 
Remove BoltDB bucket package. Construct meta.DB instance in node app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 18:42:32 +03:00
Alex Vanin
97077294fc [#141] Fix double sidechain GAS emission on asset mint 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-03 16:28:01 +03:00
Alex Vanin
0c49c08609 [#139] Use storage node GAS emission in inner ring app 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-03 16:20:41 +03:00
Alex Vanin
83119c00ec [#127] Remove magic number from config 
With neo-project/neo-modules#358 nodes support RPC to
return used magic number. Therefore client doesn't need
that configuration value any more.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 13:54:31 +03:00
Alex Vanin
2bd827a478 [#134] Resend bootstrap txs periodically 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:53:11 +03:00
Alex Vanin
976ba06300 [#132] Use netmap cleaner in inner ring server 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
b12d26f974 Update neofs-api-go to latest version 
Handle errors provided by JSON encoders.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-30 11:28:49 +03:00
Alex Vanin
b93c146824 [#26] Add extnded ACL table converter in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-28 13:36:25 +03:00
Alex Vanin
c4e9e258fd [#120] Add utility command to sign bearer token in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-23 15:31:04 +03:00
Alex Vanin
e7f9da9139 [#120] Add --bearer argument in CLI 
Bearer token should be stored in file and correctly
signed.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-23 15:31:04 +03:00
Leonard Lyubich
334e0e6f0f [#109] cmd/neofs-node: Activate Replicator 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-23 15:23:22 +03:00
Alex Vanin
0c3e6c7a04 [#40] cmd/neofs-node: Remove max object size value from config 
Now max object size value is taken from global config in sidechain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-23 14:18:00 +03:00
Leonard Lyubich
23f00a3cd2 [#40] cmd/neofs-node: Re-implement max object size source 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-23 14:03:25 +03:00
Leonard Lyubich
d05b2ff500 [#83] cmd/neofs-node: Add network state option to response service 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-23 10:54:48 +03:00
Leonard Lyubich
19f9c7eacb [#83] services: Remove setting of meta header from executing services 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-23 10:54:48 +03:00
Leonard Lyubich
3065a7363b [#83] cmd/neofs-node: Use response services in application 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-23 10:54:48 +03:00
Alex Vanin
23ec33e821 [#106] Check bearer token lifetime 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-22 18:02:11 +03:00
Leonard Lyubich
0bdfd18469 [#60] cmd/neofs-node: Initialize network state on boot 
Get current epoch number from morph client and store the value in local
state structure.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-22 11:54:08 +03:00
Leonard Lyubich
a8481223d6 [#60] cmd/neofs-node: Implement primary network State instance 
Define networkState structure and implement netmap.State interface on it.
Set epoch number on NewEpoch notification from chain. Provide
WithNetworkState option to Put service constructor.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-22 11:54:08 +03:00
Alex Vanin
c28641d290 [#95] Support --json flag in container.get command 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-21 18:22:59 +03:00
Alex Vanin
e40dc0412d [#26] Add get and set EACL commands in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-21 18:22:59 +03:00
Alex Vanin
1482b0c5d0 [#79] Simplify getCID and getOID functions in CLI 
Parser from string is available in neofs-api-go.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-21 18:22:59 +03:00
Leonard Lyubich
f52f643fe4 [#108] cmd/neofs-node: Add Policer worker to application 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-21 14:42:51 +03:00
Leonard Lyubich
07fe49088c [#108] cmd/neofs-node: Listen new epoch notifications from morph 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-10-21 14:42:51 +03:00
Alex Vanin
e6f04f7785 [#104] Update neofs-api-go with new protobuf API 
Also update contains JSON converters for neofs-cli
and fixes bug in container.set-acl command of SDK.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-20 17:31:59 +03:00
Evgenii Stratonikov
4d2ff842a2 [#79] Implement object commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2020-10-19 14:35:39 +03:00
Alex Vanin
b7dfc73bbc [#25] Use comma separated container attributes in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-17 13:16:59 +03:00
Alex Vanin
feac279ced [#25] Add container.get command in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-17 13:16:59 +03:00
Alex Vanin
26e13f4b28 [#25] Add container.list-objects command in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-17 13:16:59 +03:00
Alex Vanin
c0a4343282 [#25] Use api-go stringers and parsers for ID types in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-17 13:16:59 +03:00
Alex Vanin
db0dd6363f [#25] Add container.create, list, delete commands to CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-15 10:25:41 +03:00
Alex Vanin
d83411016a [#25] Use printVerbose wrapper for one-line messages 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-15 10:25:41 +03:00
Evgeniy Kulikov
53f7b58130 [#93] Add gRPC Reflection service 
Added gRPC Reflection service that can be enabled
by settings `grpc.enable_reflect_service`.

Read more about at
https://github.com/grpc/grpc-go/blob/master/Documentation/server-reflection-tutorial.md

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-10-15 10:20:58 +03:00
Alex Vanin
dd48666357 [#72] Shutdown inner ring app if RPC node has been terminated 
Adopt error channel from Listener interface.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-14 09:37:48 +03:00
Alex Vanin
a9d296faa8 [#87] Use viper to configure key and endpoint in CLI 
With viper we can set up key and endpoint with config
and environmental variables with `NEOFS_CLI` prefix.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
69dd95d748 [#78] Generate random key in CLI with --key new argument 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
c2083d773c [#76] Add owner argument for accounting.balance command 
With `--owner` argument user can look for balances of other
nodes by knowing their owner ID which is NEO3 compatible
address.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
945bb723ed [#76] Do not print error message twice in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
80cd4039a8 [#76] Implement accounting.balance command in CLI 
It prints result as a float type value. With verbose flag
it prints decimal precision and value.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
84928527d7 [#87] Add verbose global flag in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
cb1ca511cb [#77] Add endpoint global argument in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
fc700cbbdc [#78] Add private key global argument in CLI 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-13 14:02:48 +03:00
Alex Vanin
f1f20b49db [#86] Add alphabet contract processor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
4308a6f522 [#86] Run timer for gas emission event 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00