s.makhov/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
4187 commits 8 branches 70 tags 41 MiB
Commit graph

1479 commits

Author SHA1 Message Date
Evgenii Stratonikov
59db66cdb6 [#2091] neofs-adm: Sign blocks properly in tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
6882887bdd [#2091] neofs-adm: Do not query hashes via network 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
cfefebd5b3 [#2091] neofs-adm: Do not use deprecated methods 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:45:13 +03:00
Anton Nikiforov
51e886dd67 [#2090] neofs-cli: Remove --header from object get 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-30 11:46:02 +03:00
Anton Nikiforov
79130f781e [#2089] neofs-cli: Remove -g option from neofs-cli control ... and neofs-cli container create commands 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-30 09:14:50 +03:00
Pavel Karpy
ca3596dd05 [#2104] cli: OID signature in output 
OID signature should always be present in an object; it does not relate
to the object split.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-29 08:52:09 +03:00
Vladimir Domnich
71853348b9 [#2063] morph/client: Support new hash format in morph nns client 
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2022-11-19 11:01:04 +03:00
Anton Nikiforov
9a20498f34 [#1940] Removing all trees by container ID if tree ID is empty in pilorama.Forest.TreeDrop 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
426fe97990 [#2026] neofs-adm: Make contract update idempotent 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
7ef0303e13 [#2003] neofs-node: Allow to configure replicator pool size 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
597ed18269 [#2048] neofs-node: Use a separate client cache for client operations 
Background workers can prevent user operations to complete because of
locking in cache.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
c85bea15ef [#1978] cli: Add children to the static session on DELETE 
If an external session is provided and is not opened by CLI itself, add
children objects to it too. It fixes "not found" errors when removing a big
object with a predefined session (from a file).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
f48b1de54b [#2029] cli: Fix panic caused by flag redefinition 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
09a59fef56 [#2029] cli: Allow attaching static session to object hash 
All the other object commands already have it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
37f813604f [#2000] cli: Provide a bearer token to spawned HEAD by DELETE 
If a `neofs-cli object delete` operation is performing using a bearer token,
add it to the new `HEAD` requests that collects children OIDs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
aab398f4f5 [#1972] node: Do not save objects if node not in a container 
Do not use node's local storage if it is clear that an object will be
removed anyway as a redundant. It requires moving the changing local storage
logic from the validation step to the local target implementation.
It allows performing any relations checks (e.g. object locking) only if a
node is considered as a valid container member and is expected to store
(stored previously) all the helper objects (e.g. `LOCK`, `TOMBSTONE`, etc).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
a77392e9ce [#1972] cli: Fix lifetime flag in the lock command 
That part of the code was refactored incorrectly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Anton Nikiforov
bb52857b2b [#1338] neofs-cli: Add support to store/restore/delete binary objects 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-01 15:30:00 +03:00
Pavel Karpy
1f82c583e3 [#1971] cli: Unify CID and OID flags provision 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-01 15:07:19 +03:00
Pavel Karpy
7daa57d4d2 [#1991] cli: Refine container placement description 
Not to confuse a user by mixing a replication vector number with its copy
number.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-01 15:00:00 +03:00
Anton Nikiforov
51e3810285 [#1689] Add new command morph list-containers in neofs-adm 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-01 09:43:02 +03:00
Evgenii Stratonikov
98a152256b [#1992] writecache: Allow to open in NOSYNC mode 
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
b6930f2219 [#1992] neofs-node: Allow to open fstree in NOSYNC mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Leonard Lyubich
98034005f1 [#1900] node: Fix loosing the "maintenance" status 
In previous implementation node lost maintenance status after successful
switching to it. For example, after some period of time node sent
bootstrap requests with the "online" state instead of "maintenance".

Make `startMaintenance` method to set maintenance status in the
`networkState`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
db92e96e40 [#1900] node: Refactor bootstrap methods 
In previous implementation bootstrapping state was chosen according to
bool flag which was not convenient.

Create separate method to boostrap with "online" and the current state.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
98ac525272 [#1978] cli/object: Gather all related object in delete session 
Object removal session should reflect all objects related to the
removing one.

Make `OpenSessionViaClient` to gather the split members of the original
object in order to spread the session to them.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 13:49:41 +03:00
Evgenii Stratonikov
280e56f4bb [#1893] neofs-node: Do not fail unless all gRPC endpoints are unavailable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:48:38 +03:00
Leonard Lyubich
3c6daa2995 [#1883] cli/shards: Exclude degraded-read-write mode from help 
The mode is not expected to be used by the user. It is supported only
for developers.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
c4a2a283ae [#1883] cli/shards: Make supporting new modes easier 
Add new shard modes as a map entry to automatically parse them in
`set-mode` command. The change also automatically adds new modes to help
message.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
7f54462e60 [#1920] cli: Print used RPC timeout in verbose exec 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:20:36 +03:00
Pavel Karpy
9a0824766c [#1713] node: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Pavel Karpy
0631b38da1 [#1713] ir: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Pavel Karpy
d09ec2e3fe [#1615] node/config: Remove unused default constants 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Evgenii Stratonikov
4fe1aad30f [#1966] neofs-adm: Fix signature creation for multiactor scenario 
`SignTx` from neo-go appended verification script only if `Scripts`
array was not completely filled.

```
can't persist transactions: transaction b9cd02fb81721c7316c908687301eeaf8ef384e381476632c24a77bdf9ffacd5 failed to verify: witness #1: unknown verification contract
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-25 11:22:00 +03:00
Evgenii Stratonikov
2dcf8d51be [#1966] neofs-adm: Set VerifyBlocks: true and fix NetworkFee calculation 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-25 11:22:00 +03:00
Evgenii Stratonikov
6c9423cfb0 [#1959] neofs-adm: Add tests for more commands 
Also, fix `restore-containers`, but it is hard to test it now.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
7c3ac7150d [#1959] neofs-adm: Remove newTempDir function 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
2404a267b9 [#1959] neofs-adm: Move testdata generation to a separate function 
Later we could make it a separate tests an run neofs-adm tests on CI.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
6c21e2cc28 [#1959] neofs-adm: Fix set-config for >4 nodes 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
c87c3d3721 [#1959] neofs-adm: Refactor tests 
Make it easier to test both common deployment scenarios.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
02bea52f40 [#1959] neofs-adm: Allow to test all commands with a local dump 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
8648b102db [#1959] neofs-adm: Dump all blocks in file 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
23622ed283 [#1959] neofs-adm: Check transaction validity in tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
942155be6e [#1959] neofs-adm: Fix double multisig 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
2484c1d35e [#1959] neofs-adm: Verify blocks and transactions in tests 
Also update to the actual contracts version.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Leonard Lyubich
a4573d5026 [#1933] cli: Document sessions 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 17:45:22 +03:00
Leonard Lyubich
7b418c36b4 [#1930] services/session: Log calling Create RPC 
There is a need to check if session is opened during system
testing/debug.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 17:45:22 +03:00
Leonard Lyubich
8bba490c30 [#1933] cli: Support binary sessions 
There is a need to support NeoFS-binary sessions along with JSON ones in
NeoFS CLI.

Provide generic `common.ReadBinaryOrJSON` functions which tries to
decode NeoFS-binary structure and falls back to JSON format. Use this
function in all places with token reading.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 17:45:22 +03:00
Evgenii Stratonikov
04b67f3ba5 [#1949] neofs-adm: Fix epoch tick for >4 node committee 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 13:36:08 +03:00
Leonard Lyubich
60e9de8d63 [#1916] control: Check maintenance allowance on Control server 
In previous implementation turning to maintenance mode using NeoFS CLI
required NeoFS API endpoint. This was not convenient from the user
perspective. It's worth to move networks settings' check to the server
side.

Add `force_maintenance` field to `SetNetmapStatusRequest.Body` message
of Control API. Add `force` flag to `neofs-cli control set-status`
command which sets corresponding field in the requests body if status is
`maintenance`. Force flag is ignored for any other status.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 09:20:24 +04:00
Leonard Lyubich
c50603494b [#1916] cli/control: Pre-check maintenance mode in set-status 
Node's maintenance state can be set only if the network allows it.

Make `neofs-cli control set-status` to read current network settings and
check is maintenance state is allowed. Fail the execution if the mode is
not allowed.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 09:20:24 +04:00
Pavel Karpy
e1be0180f6 [#1329] tree: Sync trees when a node first time appears in a container 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-21 18:47:56 +03:00
Leonard Lyubich
217b030d20 [#1946] cli/internal: Fix test by wallet non-closing 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-21 15:21:59 +03:00
Leonard Lyubich
810087d8b9 [#1946] Bump neo-go module to v0.99.4 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-21 15:21:59 +03:00
Evgenii Stratonikov
de055f27c7 [#1942] neofs-adm: Fix multisig checks in tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-21 10:16:51 +03:00
Pavel Karpy
aa37078570 [#1902] node: Sync all the trees on bootstrap 
Do not limit synchronization by "system" and "version" tree IDs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
1766ca2039 [#1902] tree: Allow synchronize all the container trees 
Add `SynchronizeAllTrees` method of the Tree service. It allows fetching
tree IDs and sync all of them. Share common logic b/w the new method and
the `SynchronizeTree`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
1805c6606d [#1332] cli: Add tree list 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
3aa9938b8f [#1902] Update protoc to v3.21.7 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Evgenii Stratonikov
0d14ef69f0 [#1922] neofs-node: Allow to go online after maintenance 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-19 10:14:56 +03:00
Evgenii Stratonikov
1980ed968a [#1926] neofs-node: Use writecache.max_object_size from the config 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 16:44:28 +03:00
Evgenii Stratonikov
1cb892c579 [#1910] .golangci.yml: Add misspell linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Evgenii Stratonikov
d772e35aba [#1910] .golangci.yml: Add godot linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Evgenii Stratonikov
0a1530afa0 [#1917] neofs-cli: Support timeout in control commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 10:58:10 +03:00
Pavel Karpy
c16a31ef3e [#1329] node: Set undefined netmap status by default 
It allows general usage of `controlNetmapStatus` getter: does not matter of
handler (`NewEpoch` handler of initial setter).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 20:53:34 +03:00
Pavel Karpy
f4a3fa2977 [#1329] tree: Sync tree on startup 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 20:53:34 +03:00
Leonard Lyubich
d96139b201 [#1900] node: Do not override maintenance state on bootstrap 
In previous implementation node always sent bootstrap requests with
"online" network state. This provoked switching to online of the nodes
under maintenance.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-17 18:59:22 +03:00
Evgenii Stratonikov
8145e79d7f [#1903] neofs-adm: Expand update parameters in MakeCall 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-17 13:37:48 +03:00
Leonard Lyubich
1406d096a2 [#1680] service/object: Fail all operations in maintenance mode 
Storage node should not provide NeoFS Object API service when it is
under maintenance.

Declare `Common` service that unifies behavior of all object operations.
The implementation pre-checks if node is under maintenance and returns
`apistatus.NodeUnderMaintenance` if so. Use `Common` service as a first
logical processor in object service pipeline.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-14 13:54:32 +04:00
Evgenii Stratonikov
2050a623ac [#1896] neofs-node: Set MAINTENANCE status on update 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-14 11:46:19 +03:00
Pavel Karpy
beb1998ed1 [#1332] cli: Print missing nodes message verbosely 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Pavel Karpy
6e4f7180fe [#1332] cli: Implement tree add-by-path command 
It is `TreeService.AddNodeByPath` method implementation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Pavel Karpy
3d8349d7f8 [#1332] cli: Implement tree get-by-path command 
It is `TreeService.GetNodeByPath` method implementation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Pavel Karpy
2c02e66939 [#1332] cli: Implement tree add command 
It is `TreeService.Add` method implementation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Evgenii Stratonikov
80d3c7f9d6 [#1892] neofs-adm: Support MaintenanceModeAllowed network setting 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-13 17:45:47 +03:00
Evgenii Stratonikov
f2793060c5 [#1892] neofs-cli: Display maintenance status in netmap nodeinfo 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-13 17:45:47 +03:00
Anton Nikiforov
8714fc42b5 [#1765] Use hex format to print storage node ID 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-13 12:55:21 +03:00
Pavel Karpy
b6806ea6b9 [#1770] node: Support logger config rereading 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
8c75cb1dad [#1770] node: Validate logger config section 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
d7c7022bbd [#1770] logger: Support runtime level reloading 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component 
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Leonard Lyubich
9a006ac14f [#1865] cli/object: Do not open remote sessions in reading ops 
It does not make sense to open remote sessions with the storage node in
`get`, `head`, `search`, `range` and `hash` sub-commands of `neofs-cli
object` command.

Do not use NeoFS API `SessionService` in mentioned commands. Decode
object session from JSON file specified `--session` flag. Perform some
sanity checks instantly on CLI side.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-12 13:22:19 +03:00
Evgenii Stratonikov
c0199dee93 [#1867] services/control: Interpret empty list of IDs as all shards 
In neofs-cli the flag is still required, but `all` can be used to
process all shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Evgenii Stratonikov
b632260995 [#1867] neofs-cli: Support multiple shard IDs in control subcommands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Evgenii Stratonikov
19c0a74e94 [#1867] services/control: Allow to provide multiple shard IDs to some commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Anton Nikiforov
74d2f2c8d3 [#1854] cli: Unify help messages 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-12 09:53:11 +03:00
Pavel Karpy
9e2edfedc6 [#1833] node: Update local node info on nil updates 
If the contract returns a netmap that does not contain the node, update
local `NodeInfo`. It fixes `neofs-cli netmap nodeinfo` command that printed
"state: online" previously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 09:52:51 +03:00
Evgenii Stratonikov
2d43892fc9 [#1840] neofs-node: Use blobstor paths to identify shard 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
4b005d3178 [#1840] blobstor: Return info about all components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
9b241e4a17 [#1840] neofs-node: Allow to use mode: disabled in config 
Currently, when removing shard special care must be taken with respect
to shard numbering. `mode: disabled` allows to leave shard configuration
in place while also ignoring it during initialization. This makes
disk replacement much more convenient.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Pavel Karpy
1360273fec [#1856] node: Fix config rereading 
Config was reread from the file only once in two SIGHUPs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-10 10:35:52 +03:00
Anton Nikiforov
5e493b7f1c [#1704] Add command container nodes to output list of nodes for container, grouped by replica (#1704) 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-07 14:58:01 +03:00
Evgenii Stratonikov
90bfe0bad9 [#1826] neofs-cli: Add --timeout flag 
Allow to specify it everywhere `--rpc-endpoint` flag is present.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-07 13:29:15 +03:00
Evgenii Stratonikov
ca8dc872b2 [#1846] neofs-node: Make morph.cache_ttl equal to block time by default 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-07 10:34:55 +03:00
Leonard Lyubich
e54b52ec03 [#1420] object/acl: Fix correlation of object session to request 
In previous implementation of `neofs-node` app object session was not
checked for substitution of the object related to it. Also, for access
checks, the session object was substituted instead of the one from the
request. This, on the one hand, made it possible to inherit the session
from the parent object for authorization for certain actions. On the
other hand, it covered the mentioned object substitution, which is a
critical vulnerability.

Next changes are applied to processing of all Object service requests:
 - check if object session relates to the requested object
 - use requested object in access checks.

Disclosed problem of object context inheritance will be solved within

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-07 10:34:38 +03:00
Pavel Karpy
01ddb3f8e6 [#1770] node: Refactor application configuration 
Split all the fields in `cfg` structure on:
1. `applicationConfiguration`;
2. `internals`; // shared entities for an application work, such as
`context.Context`
3. `shared`; // holder for the shared entities b/w;
4. `cfgXXX`; // configuration for internal services.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-05 14:25:33 +04:00
Leonard Lyubich
713aea06fa [#1681] node: Block only Object service ops under maintenance 
In previous implementation node blocked any operation of local object
storage in maintenance mode. There is a need to perform some storage
operations like data evacuation or restoration.

Do not call block storage engine in maintenance mode. Make all Object
service operations to return `apistatus.NodeUnderMaintenance` error from
each local op.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
a7668618c9 [#1680] cli/netmap: Support MAINTENANCE mode 
Make `netmap snapshot` command to print `MAINTENANCE` state of the nodes
with `IsMaintenance()` flag set.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
eb1fba5182 [#1680] morph/netmap: Adopt to recent contract changes 
After recent Netmap contract changes all read methods which return
network map (either candidates or snapshots) encode node descriptors
into same structure.

Decode `netmap.Node` contract-side structure from the call results.
Replace node state with the value from the `netmap.Node.State` field.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
d6c01199c8 [#1680] node: Return MAINTENANCE status in corresponding mode 
Make storage node to return `NODE_UNDER_MAINTENANCE` status
error on each local object operation if the node is in `MAINTENANCE`
mode.

Pass `apistatus.NodeUnderMaintenance` to `StorageEngine.BlockExecution`
during `ControlService.SetNetmapStatus` RPC processing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
9fcc80ea38 [#1680] node/control: Send request to switch to MAINTENANCE state 
After recent changes `MAINTENANCE` state is reflected in the Sidechain.
Storage node should switch its state to "maintenance" during serving the
`ControlService.SetNetmapStatus` RPC with correspoding status in the
request.

Call `UpdatePeerState` operation of Netmap contract's client in
`control.NodeState` provider on Storage node app side. The op is
executed if `BlockExecution` on local object storage is succeeded.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
876e132d22 [#1680] cli/netmap: Print MaintenanceModeAllowed config 
After recent changes network configuration provided by NeoFS storage
nodes contains `MaintenanceModeAllowed` flag. There is
a need to support this value in NeoFS CLI application.

Print `MaintenanceModeAllowed` flag in `netmap netinfo` command.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
7a3a827d35 [#1680] node/netmap: Support MaintenanceModeAllowed config in RPC 
After recent changes network configuration stored in the Netmap contract
of the NeoFS Sidechain contains `MaintenanceModeAllowed` flag. There is
a need to support this value in Storage node application.

Make `NetmapService.NetworkInfo` RPC server of the storage node to set
`MaintenanceModeAllowed` flag according to corresponding value in the
Sidechain.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
d2d4191868 [#1680] morph/netmap: Support HomomorphicHashingDisabled config 
`NetworkConfiguration` represents NeoFS network configuration stored in
the Sidechain. In previous implementation the configuration missed flag
of disabled homomorphic hashing.

Add `NetworkConfiguration.HomomorphicHashingDisabled` boolean field.
Decode the field in `Client.ReadNetworkConfiguration` method. Print this
value in `netmap netinfo` command of NeoFS CLI.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
4c94faac67 [#1797] node/control: Verify states in SetNetmapStatus RPC server 
In previous implementation storage node interpreted all status values
sent in `SetNetmapStatus` RPC as `OFFLINE` except `ONLINE` and
`MAINTENANCE`. This could lead to incorrect processing of new values,
and also didn't allow detection of problems with sending garbage values.

Make implementation of `NodeState` interface used by Control API server
to deny requests with statuses other than protocol-declared enum.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Anton Nikiforov
8bf82d738b [#1704] cli: Add force option to the command container create 
Validate policy before container creation

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-05 09:14:13 +03:00
Evgenii Stratonikov
236414df49 [#1817] network: Allow to use network addresses from the iterator 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 12:12:18 +03:00
Pavel Karpy
1e35c12cc1 [#1770] node: Validate config before apply it 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
ab3ef7110e [#1770] node: Reread config files on SIGHUP 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
6fc3268ebf [#1770] node: Support configuration reread on SIGHUP 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
fbd5bc8c38 [#1770] engine: Support configuration reload 
Currently, it only supports changing the compound of the shards.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
91b56ad3e8 [#1770] node: Read storage config in a separate struct 
It will allow rereading config values and will simplify distinguishing them
from the custom values in the `cfg` structure.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
5c69e19016 [#1770] fstree: Depth parameter int -> uint64 
Negative values have no sense. On the other hand it differs from the
blobovnicza's configuration and prevents unification.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
4aa4694152 [#1770] node: Do not init local storage in reading config routine 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
2d7166f8d0 [#1770] shard: Move NewEpoch event routing on SE level 
It will allow dynamic shard management. Closing a shard does not allow
removing event handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
9374823950 [#1770] node: Do not create meta dir on config read 
It is created in `Open` anyway.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Alex Vanin
76cfcc242c [#1820] neofs-adm: Add wallet-address flag in refill command 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-10-03 10:11:38 +03:00
Evgenii Stratonikov
1edc048870 [#1697] services/object: Return proper error if session token is missing 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-30 16:13:01 +03:00
Vladimir Domnich
5284ac53f9 [#1821] neofs-cli: Fix description of cli commands 
1. nodeinfo - updated description of node.
2. eacl - fixed misprint.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2022-09-28 16:09:24 +03:00
Evgenii Stratonikov
1c62f1b2c4 [#1806] neofs-cli: Add control flush-cache command 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
36ab1a2472 [#1731] neofs-cli: Make --endpoint flag optional 
Allow to provide it using config file.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Leonard Lyubich
93742d37b7 [#1793] node/netmap: Change interface of the latest network map reader 
Replace `ProcessCurrentNetMap` method of `NodeState` interface with
`ReadCurrentNetMap` one with two changes:
 * Replace network map type from NeoFS SDK package with the
   protocol-generated message. This replaces all the business logic to
   the application layer.
 * Support error return. This allows to cover problem node states.

Return an error from `NodeState.ReadCurrentNetMap` method implemeted
through `atomic.Value` if `Store` method has not been called yet.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Leonard Lyubich
485a5418d2 [#1793] node: Serve NetmapService.NetmapSnapshot RPC 
There is no more need to serve the same request on Control API.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Evgenii Stratonikov
59de20fbba [#1764] neofs-node: Allow to check configuration 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
0fb5c51ac9 [#1764] neofs-node: Validate config before usage 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
6f45cc81fc [#1764] neofs-node: Use constants for storage types 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
9113793688 [#1764] neofs-node: Allow to return error from IterateShards 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
4e043a801c [#1731] services/control: Replicate object over network in EvacuateShard RPC 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Anton Nikiforov
c7f85994e5 [nspcc-dev#1692] cli: Remove --generate-key option in neofs-cli container delete 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-09-20 07:19:46 +04:00
Anton Nikiforov
bb02913c39 [nspcc-dev#1128] cli: Remove WIF and NEP2 support in --wallet argument 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-09-19 14:21:37 +03:00
Evgenii Stratonikov
8fc88487db [#1731] neofs-cli: Add control shards evacuate command 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
4208f7c0cf [#1741] neofs-adm: Use ValidUntilBlock to wait for tx persist 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 14:27:16 +03:00
Evgenii Stratonikov
cda8f9df2e [#1786] services/control: Remove WithDeletedObjectHandler option 
Use storage engine directly instead. It is already provided in the
options.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Leonard Lyubich
ae655b74f0 *: Exec make fmts 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-14 17:52:49 +04:00
Leonard Lyubich
d25b7e177b [#1189] node/config: Fix PersistentSessionsConfig.Path method's docs 
Method never returns `PersistentStatePathDefault` value, and this is
correct.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-13 21:29:04 +04:00
Pavel Karpy
876e014b5d [#1628] tree: Make ACL checks the same way as for object requests 
1. Do not require a request to be signed by the container owner if a
bearer token is missing
2. Do not check the system role since public requests are not expected to
be signed by IR or a container node (unlike the object requests)

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
4f18893d9b [#1628] node: Move common EACLSource interface to core pkg 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Evgenii Stratonikov
d25ec52459 [#1778] neofs-node: Panic if config type is invalid 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov
60b2930417 [#1778] neofs-node: Fix blobovnicza config type 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov
9da5d784cb [#1630] neofs-node: Remove trees on container removal event 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 09:54:15 +03:00
Leonard Lyubich
699b534416 [#1763] node/netmap: Write log message about parsed NewEpoch event 
There is a need to have the ability to track NeoFS timeline on storage
nodes. Epochs tick on notifications receipt, so the most obvious way to
know about received epochs is logging the events.

Wrap `morphEvent.ParseNewEpoch` event parser into function which writes
log message about new epoch number.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-12 09:53:05 +04:00
Pavel Karpy
8e6e89aca3 [#1711] amd: Cache committee actor in the init context 
Also simplify the code using `invoker.Invoker`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Pavel Karpy
60aa53651b [#1711] adm: Append witnesses only if they are missing 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Pavel Karpy
1e57565f6c [#1711] adm: Implement local client's methods 
Implement methods that `actor.Actor` requires:
1. `InvokeFunction` -- wrapper over `InvokeScript`
2. `GetVersion` returns default struct
3. `CalculateNetworkFee` copied and simplified from Neo-go server side.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Pavel Karpy
4bd8608b37 [#1711] *: Simplify code using neo-go actors 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Evgenii Stratonikov
4354359aed [#1746] network: Set timeout for streaming operations 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 16:00:51 +04:00
Alex Vanin
c4c2840b52 [#1632] neofs-adm: Update autogenerated storage node config 
Replace `diable_cache` with `cache_ttl` value.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-09-07 10:34:46 +03:00
Evgenii Stratonikov
e4dcc4d6a9 [#1749] neofs-adm: Register contract hashes in NNS in 2 formats 
NNS proposal describes string N3 address format, however we must also
have hex-string for backwards compatibility.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 09:58:05 +03:00
Evgenii Stratonikov
fe0c6db67d [#1749] neofs-adm: Delete NNS records before updating 
In case we already have the record and it is invalid, we should
overwrite it instead of having several conflicting records.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 09:58:05 +03:00
Pavel Karpy
fa18100489 [#1714] lens: Add open*COMPONENT* funcs 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-06 18:09:18 +04:00
Pavel Karpy
2132b78aba [#1714] lens: Add meta subcommand 
Includes:
1. `inspect`
2. `list-garbage`
3. `list-graveyard`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-06 18:09:18 +04:00
Pavel Karpy
adff08ad02 [#1714] lens: Separate commands 
Separate `inspect` and `list` on `write-cache` and `blobovnicza`
subcommands.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-06 18:09:18 +04:00
Evgenii Stratonikov
fef4f6d155 [#1748] neofs-adm: Allow to dump hashes from a custom zone 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
d4ac5bdb97 [#1748] neofs-adm: Allow to parse hashes in multiple formats 
NEO NNS proposal uses addresses. We should eventually use the same,
but must stay compatible now.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
4ccacb89e8 [#1748] neofs-adm: Set custom contract hashes as N3 address 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
11e6f03aa2 [#1748] neofs-adm: Refactor NNS price handling 
Make `nnsRegisterDomainScript` simpler.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
9290a7e1fe [#1748] neofs-adm: Remove addRecord from nnsRegisterDomainScript 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
eb5e1121c5 [#1748] neofs-adm: Move dump-hashes command to a separate file 
I have just lost 3 minutes trying to find it.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Leonard Lyubich
33143b18a9 [#1652] cli/container: Invert pre-check flag of set-eacl command 
Flag `--pre-check` of `set-eacl` command found to be in demand in most
cases. based on this, it makes sense to add its action to the default
behavior.

Pre-check container extensibility by default. Rename flag to
`--no-precheck` and invert its action.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 16:22:16 +04:00
Leonard Lyubich
46b815c863 [#1652] cli/container: Move eaclPathFrom flag var into the struct 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 16:22:16 +04:00
Leonard Lyubich
491a908af1 [#1652] cli/container: Pre-check ACL extensibility in set-eacl 
Container ACL in NeoFS can be extended only for container in which the
corresponding option is enabled. In previous implementation command
`set-eacl` could hang up on modifying eACL of the non-existent or
non-extendable container. To improve UX, there is a need to pre-check
the availability of `SETEACL` operation.

Add boolean `precheck` flag to `set-eacl` cmd which reads the container
before the actual transaction formation. If flag is set, command fails
on non-extendable container ACL.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 16:22:16 +04:00
Evgenii Stratonikov
177e8e01b1 [#1745] neofs-node: Remove memcache_capacity from the configuration 
It is unused since ddaed283e9 .

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 11:59:53 +03:00
Pavel Karpy
62b293b7ab [#1711] adm: Add error context 
To make all the calls in the command to be in the same form.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-02 10:41:18 +03:00
Pavel Karpy
a99474c40e [#1711] adm: Drop deprecated neo-go calls 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-02 10:41:18 +03:00
Leonard Lyubich
ae6ebccd9c [#1632] node/container: Don't add new container to missed cache record 
If container listing cache on node's side is missing (for particular
owner), then updating it as a reaction to successful container creation
leads to potentially invalid cache value for a period of time equivalent
to cache TTL.

Immediately return from `ttlContainerLister.update` method if owner's
container list isn't cached.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
fa3124fc33 [#1632] node: Configure TTL of Sidechain caches 
From now cache TTL can be parameterized in the `neofs-node` app using
`cache_ttl` config key. `disable_cache` value is no longer supported.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
5ce8315cd8 [#1632] node/container: Update LIST cache on successful writes 
In previous implementation storage node responded with the outdated
container list after successful creation/removal up until cache
invalidation due to TTL. In order to decrease the probability of
outdated responses node should update its cache on event receipts.

Implement `ttlContainerLister.update` method which actualizes cached
list of the owner's containers. Make node to call `update` method
on `PutSuccess`/`DeleteSuccess` notifications from the `Container`
contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
d15a7d8d3d [#1632] node/container: Update GET cache on successful removal 
In previous implementation storage node responded with the removed
container up until cache invalidation due to TTL. In order to avoid
false-positive responses node should update its cache on `DeleteSuccess`
events.

Make node to call `handleRemoval` method of the container cache which
leads to subsequent `apistatus.ErrContainerNotFound` errors.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
a529149e6f [#1632] node: Cache Sidechain errors along with the values 
In previous implementation failed requests to the Sidechain weren't
cached. It makes sense to cache errors along with the values in order to
decrease potential load spikes onto Sidechain nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
7c1babb7d6 [#1632] node: Subscribe on the successful container creations/removals 
There is a need to sync container-related caching mechanism with the
actual Sidechain changes. To do this, node should be able to listen
incoming notifications about container ops.

Define `PutSuccess` / `DeleteSuccess` notification event's parsers.
Subscribe to these events in node app. As initial implementation node
will log event receipts. Later handling is going to be practically
complicated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Evgenii Stratonikov
df88afeef3 [#1738] neofs-adm: Clean tx hash list after awaiting 
Do not query application log for the same transaction more than once.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 16:53:00 +03:00
Evgenii Stratonikov
180e5e938f [#1735] go.mod: Update API and SDK 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 12:27:46 +03:00
Evgenii Stratonikov
61f0d85834 [#1727] neofs-adm: Allow to register custom TLD for contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Evgenii Stratonikov
ddbe129cb2 [#1729] neofs-adm: Change default domain expiration time 
It is a noop for current contract versions, but will matter in the next
release.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Alex Vanin
e68c80df0a [#1728] Don't describe default value twice 
Cobra will mention default value automatically.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-24 16:44:44 +03:00
Alex Vanin
31b6af5667 [#1728] Update neofs-adm morph deploy command description 
Add more details about expected file names in contract dir.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-24 16:44:44 +03:00
Pavel Karpy
b54f34d7df [#1683] adm: Do not deploy custom contract with a group 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-24 13:52:55 +03:00
Pavel Karpy
bbd6b9780f [#1683] adm: Minimize registration price 
Register NNS domain in one TX:
1. Set minimal (`1`) registration price;
2. Register domain;
3. Return registration price back.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-24 13:52:55 +03:00
Evgenii Stratonikov
d57c57010f [#1726] neofs-cli: Truncate file before writing the object 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 18:07:12 +03:00
Evgenii Stratonikov
40f1cf36e1 [#1719] neofs-cli: Save object with proper permissions 
`curl` uses 0644, for example.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 18:07:12 +03:00
Evgenii Stratonikov
e4370c4129 [#1702] neofs-adm: Set HighPriority attribute for committee transactions 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 11:59:23 +03:00
Pavel Karpy
0720d96c9d [#1687] go.mod: Update neofs-sdk-go 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
30341f2192 [#1687] *: Perform go fmt using go v1.19 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
37ab26bfa9 [#1296] cli: "ID" -> "OID" 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 14:29:50 +04:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration 
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Pavel Karpy
5139dc9864 [#1706] cli: Do not duplicate payload on SG put 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-19 09:49:47 +04:00
Evgenii Stratonikov
f8b106ac85 [#1684] *: Fix linter warnings 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-15 10:57:31 +03:00
Leonard Lyubich
67af4c89a8 [#1649] cli: Refactor --owner flag definition in container list cmd 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-08-15 10:18:51 +03:00
Leonard Lyubich
7a26b2a57a [#1649] cli: Add option to print attributes in list 
Define `--with-attr` flag of `container list` which makes the command to
request and print user attributes for each found element.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-08-15 10:18:51 +03:00
Leonard Lyubich
2740bf7ee4 [#1649] cli: Add option to print attributes in list-objects 
Define `--with-attr` flag of `container list-objects` which makes the
command to request and print user attributes for each object from the
container.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-08-15 10:18:51 +03:00
a.y.volkov
81684b6f04 [#1665] neofs-lens: Use stdout as default output 
Signed-off-by: a.y.volkov <a.y.volkov@yadro.com>
 2022-08-09 13:23:32 +03:00
Pavel Karpy
9aba0ba512 [#1634] meta: Add epoch state 
It allows performing expiration checks on the stored objects.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 16:31:49 +03:00
Pavel Karpy
a97dee008c [#1648] morph: Change endpoint priority order 
The lowest value means the highest priority.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 16:11:24 +03:00
Pavel Karpy
8f44335925 [#1651] cli: Fix CID setting in eACL creation 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 17:04:42 +04:00
Pavel Karpy
0a60524a9c [#1654] ir, node: Drop deprecated profiler and metrics config sections 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 15:38:35 +03:00
Pavel Karpy
713cfa5610 [#1655] cli: Do not force specifying session lifetime 
We have the default value which is also printed in the help messages but any
call that does not specify that flag leads to an error.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 15:37:50 +03:00
Evgenii Stratonikov
7ecf54c6df [#1631] neofs-adm: Set group for updated alphabet contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:09:48 +03:00
Evgenii Stratonikov
d3eac7626d [#1631] neofs-adm: Set group for a proper contract 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:09:48 +03:00
Evgenii Stratonikov
5d02613c33 [#1633] neofs-adm: Allow to set custom NNS zone for deployed contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
6146df4998 [#1633] neofs-adm: Allow to provide arguments to deployed contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
dd30703a6b [#1633] neofs-adm: Allow to update arbitrary contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
fc06b6e89a [#1633] neofs-adm: Allow to deploy arbitrary contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
38558a3238 [#1633] neofs-adm: Start chain only on happy path 
If the error is returned, it will not be closed properly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
642d7328ab [#1633] neofs-adm: Fix dump generation in init 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
07465849a4 [#1637] go.mod: Update neo-go to v0.99.1 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-28 20:11:45 +03:00
Pavel Karpy
8162b27264 [#1627] node: Add tree service to the config framework 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-27 12:15:56 +03:00
Evgenii Stratonikov
10c855efef [#1624] go.mod: Update dependencies 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-25 16:56:02 +03:00
Evgenii Stratonikov
6049022f7e [#1612] neofs-cli: Remove RunE functions from commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-22 16:18:09 +03:00
Evgenii Stratonikov
a52e7c2c99 [#1612] neofs-cli: Unify expiration flags 
Use `expire-at` everywhere expiration epoch is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-22 16:18:09 +03:00
Evgenii Stratonikov
50e28f22f9 [#1559] shard: Change Degraded mode string representation 
It is a flag, but is a `degraded-read-write` mode for a user.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
4944490ffb [#1559] local_object_storage: Move shard to the DegradedReadOnly mode 
`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
1e786233bf [#1559] local_object_storage: Provide readOnly flag to Open 
We should be able to reopen storage in readonly in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
339864b720 [#1559] local_object_storage: Move shard.Mode to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
7410827db8 [#1609] config: Allow to prioritize N3 endpoints 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 16:08:42 +03:00
Evgenii Stratonikov
aed83d1660 [#1609] config: Remove mainchain section 
It is deprecated and unused, there is no need to have it in the
documentation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 16:08:42 +03:00
Pavel Karpy
a4ed91f9b5 [#1614] node: Update health metric on its change 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-21 16:07:47 +03:00
Pavel Karpy
581a9901c9 [#1614] metrics: Add health metrics 
Also, rename metrics structure since it collects not only storage metrics
now.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-21 16:07:47 +03:00
Evgenii Stratonikov
b549cc314c [#1607] services/tree: allow to customize some parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
2334e94fdb [#1607] neofs-node: Initialize storage before other services 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
a4adb79db7 [#1607] pilorama: Enable tree service explicitly 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
26041f18bf [#1505] pilorama: Allow to customize database parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
3904b0e017 [#1333] neofs-cli: Add control synchronize-tree command 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
bfdd68dcb3 [#1333] services/control: Allow to synchronize local trees 
Do not check that a node indeed belongs to the container, because the
synchronization will fail in this case anyway.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
0dc1a4e336 [#1333] neofs-node: Initialize tree service before the control one 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
5e843a73f9 [#1333] services/control: Return pilorama info in ListShards RPC 
Do not return backend type from the service for now, because memory
backend is expected to vanish.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
36e24f7f78 [#1342] neofs-node: Use the default endpoint for tree service 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
62154da17c [#1324] services/tree: Implement Object Tree Service 
Object Tree Service allows changing trees assotiated with
the container in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
198beae703 [#1613] neofs-cli: Print lock object ID 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 17:15:15 +03:00
Evgenii Stratonikov
5c8d725447 [#1610] neofs-adm: Fix generated storage config 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 16:19:41 +03:00
Evgenii Stratonikov
ccba07fe19 [#1608] neofs-adm: Register candidates in a single transaction 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 13:28:17 +03:00
Evgenii Stratonikov
59d08b8dae [#1608] neofs-adm: Print info about already performed steps 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 13:28:17 +03:00
Evgenii Stratonikov
e5c0338f8c [#1602] neofs-ir: Fix defaults for pprof and prometheus 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
3a0b06d5da [#1602] config: Rename metrics and profiler sections 
Depracate old names and remove them in the next release.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
ac46d1a11f [#1602] config: Enable metrics and profiler services with a flag 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
12dc5c3395 [#1563] config: Replace 127.0.0.1 with localhost 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
4a8dfd0ce4 [#1606] Remove deprecated functions 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 12:41:06 +03:00
Evgenii Stratonikov
6cc180391e [#1606] go.mod: Update neo-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 12:41:06 +03:00
Pavel Karpy
b1e54dec9f [#1461] cli: Add context to GetCurrentEpoch helper 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Pavel Karpy
9c5ef3bab8 [#1461] node: Allow force LOCK removal 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Pavel Karpy
de31bb27b6 [#1461] cli: Require LOCK object to be expired 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Pavel Karpy
595fc60d72 [#1461] cli: Reorganize common functions 
Make `ParseEpoch` and `GetCurrentEpoch` funcs public and move the first one
to the `common` package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Evgenii Stratonikov
6eb5260562 [#1500] neofs-cli: Check if container contains LOCK objects before removal 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-15 13:25:56 +03:00
Evgenii Stratonikov
c4b86cf1f1 [#1592] Remove debug builds 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
b8508585a5 [#1592] cmd: Ignore error when fetching a defined flag 
There is an error only if the flag is not defined, such errors should be
caught during debugging.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
f7d5045876 [#1592] neofs-lens: Add --version flag 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
5569ff82ef [#1587] Do not print build time in version 
This makes our build more reproducible.
Also print `Component` and `GoVersion`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Pavel Karpy
b3272e7cf1 [#1367] adm: Support homomorphic hashing config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
9bd1951ca4 [#1365] adm: Add homomorphic hash disabling option 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
ae86d2907c [#1365] cli: Calculate homomorphic hash flexibly 
Do not use homomorphic hash in storage group for containers that have
`homomorphic_hashing_disabled` set to `true`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
68e583f143 [#1365] cli: Sync container with network config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
49ba1ef1b6 [#1365] cli: Add SyncContainerSettings func to internal client 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Evgenii Stratonikov
1119014e53 [#1579] config: Move storage.default to shard.default 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-13 12:31:42 +03:00
Evgenii Stratonikov
f4a3fc5f11 [#1594] neofs-adm: Support better completion for set-policy 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-13 11:00:35 +03:00
Pavel Karpy
466b9b4273 [#1593] adm: Support changing network config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-12 21:03:43 +03:00
Pavel Karpy
96a24b5721 [#1490] cli: Require SG members to be unique 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-12 17:35:46 +03:00
Leonard Lyubich
9a11a75b77 [#1570] Upgrade NeoFS SDK Go with changed reputation API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-06 18:21:24 +03:00
Pavel Karpy
12d41918e6 [#1402] cli: Support storage group expiration 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-06 09:56:26 +03:00
Pavel Karpy
cfca88fc5c [#1402] cli: object <-> SG conversion via SDK 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-06 09:56:26 +03:00
Pavel Karpy
bc5882fc89 [#1402] cli: Add cid flag constant 
Also, move common flag constants in `root` file of the `storagegroup`
package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-06 09:56:26 +03:00
Leonard Lyubich
c165d1a9b5 [#1556] Upgrade NeoFS SDK Go with changed container API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-05 11:26:06 +03:00
Evgenii Stratonikov
ad81b6c90a [#1555] neofs-adm: Remove trailing dots from help messages 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-28 13:46:02 +03:00
Evgenii Stratonikov
b41658db04 [#1555] neofs-adm: Allow to remove nodes from the netmap 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-28 13:46:02 +03:00
Evgenii Stratonikov
32fd3692bf [#1555] neofs-adm: Fix cached group signer 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-28 13:46:02 +03:00
Leonard Lyubich
305dd7598f [#1533] acl: Upgrade NeoFS SDK Go with refactored basic ACL 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-25 13:57:21 +03:00
Evgenii Stratonikov
451343c751 [#1539] neofs-adm: Retrieve storage wallet passwords from config 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-24 13:40:21 +03:00
Pavel Karpy
fc7940a5ea [#1544] cli: Init bearer flags in list-objects cmd 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-24 13:40:07 +03:00
Evgenii Stratonikov
9b3e1bd27b [#1307] neofs-adm: Add tests for morph init command 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-24 10:33:34 +03:00
Evgenii Stratonikov
189507dc89 [#1541] neofs-cli: allow to print shards list output in JSON 
Eventually more parameters will be supported (#1390) and after blobstor
configuration refactoring the output will certainly change. Implement
the simplest approach now.

Marshaling the result directly results in too ugly names and they cannot
be easily customized. Marshaling the results via `jsonpb` is better but
is not that flexible in terms of what we want to output.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-24 10:33:22 +03:00
Evgenii Stratonikov
9816d59ec0 [#1323] neofs-cli: Reuse JSON flag for multiple commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-24 10:33:22 +03:00
Leonard Lyubich
b67974a8d3 [#xxx] Upgrade NeoFS SDK Go with changed container sessions 
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.

Group value, session token and signature in a structure for container
and eACL.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-22 16:38:57 +03:00
Evgenii Stratonikov
972ca83e23 [#1524] writecache: Add some bolt parameters to the configuration 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
07e06249d5 [#1524] metabase: Add some bolt parameters to the configuration 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
d6043abc24 [#1524] neofs-node: Reuse boltdb config for different components 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
0408acc40e services/control: Autogenerate stable marshalers 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-20 16:52:58 +03:00
Leonard Lyubich
d2d1694113 [#1513] cli/storagegroup: Fix linter remarks 
`staticcheck` linter scolds on potential nil-dereference since it
doesn't know `common.ExitOnErr` definitely exists. Explicit `return`
fixes linter's remark and prevents potential bugs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 15:53:18 +03:00
Leonard Lyubich
eb7b8bf7c4 [#1513] node: Encode node's public key into binary format once 
Avoid per-iteration key encoding due to the immutability of the key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 15:53:18 +03:00
Leonard Lyubich
4574ba646d [#1513] cli/storagegroup: Fix parsing container and SG flags 
Read `id` flag with the storage group ID. Prevent NPE-panic if flag is
missing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 15:53:18 +03:00
Leonard Lyubich
21d2f8f861 [#1513] Upgrade NeoFS SDK Go with changed netmap package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 15:53:18 +03:00
Evgenii Stratonikov
24b4c1ecf4 [#1484] neofs-cli: Print error on a separate line from a progress bar 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-16 16:38:33 +03:00
Evgenii Stratonikov
e42b7f6a65 [#1484] neofs-cli: Print progress bar after the header has been sent 
Make it similar to `object get`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-16 16:38:33 +03:00
Evgenii Stratonikov
b68778ea90 [#1380] neofs-cli: move container command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-16 09:27:07 +03:00
Evgenii Stratonikov
fd48b96082 [#1384] neofs-cli: move storagegroup command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-16 09:27:07 +03:00
Evgenii Stratonikov
a219e3a667 [#1383] neofs-cli: move object command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-16 09:27:07 +03:00
Evgenii Stratonikov
12bc5607f7 [#1382] neofs-cli: move netmap command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-16 09:27:07 +03:00
Pavel Karpy
df8a3807fe [#1508] node: Remove unused replicator code 
The node does not support asynchronous object replication anymore, so it
does not need to have replicator worker, channel and `AddTask` function.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-15 20:33:04 +03:00
Leonard Lyubich
2e4a1cb6df [#1518] Upgrade NeoFS SDK Go with changed subnet package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-15 17:02:49 +03:00
Evgenii Stratonikov
f602d05b0a [#1494] *: Fix linter warnings 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-15 12:26:10 +03:00
Evgenii Stratonikov
aab6094a7c [#1515] neofs-node: Cache max object size 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-15 11:14:37 +03:00
Evgenii Stratonikov
3db4526437 [#1498] neofs-adm: Save wallets as pretty JSON 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-09 16:16:46 +03:00
Pavel Karpy
4ed3063212 [#1466] adm: Remove notary flag in for subnet command 
It is possible to deduce that information from the network.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-08 11:06:19 +03:00
Pavel Karpy
815e02f185 [#1466] adm: Do not use morph client in adm 
Using morph client leads to requiring WebSocket endpoints.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-08 11:06:19 +03:00
Evgenii Stratonikov
60789dd475 [#1489] neofs-adm: Include alphabet contracts to the group 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-08 11:01:16 +03:00
Evgenii Stratonikov
a46f585fb3 [#1074] neofs-cli: Inline some of the functions 
They are quite small, however will produce problems during subsequent refactoring.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-06 13:54:29 +03:00
Evgenii Stratonikov
9efec21d34 [#1074] neofs-cli: Move session preparation to modules/session package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-06 13:54:29 +03:00
Evgenii Stratonikov
a2bcb3e0ce [#1074] neofs-cli: Move common API flags to a separate package 
TTL and XHeader flags are reused between multiple commands.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-06 13:54:29 +03:00
Evgenii Stratonikov
736e09a70d [#1074] neofs-cli: Move PrettyPrintUnixTime to the common package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-06 13:54:29 +03:00
Leonard Lyubich
72708296cc Upgrade NeoFS SDK Go to v1.0.0-rc.4 and NeoFS API Go to v2.12.2 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-03 17:17:50 +03:00
Evgenii Stratonikov
c41d9c3fbe [#1474] neofs-node/config: Remove "shard_num" from the "grpc" section 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:44:06 +03:00
Evgenii Stratonikov
73a88c2965 [#1474] neofs-node/config: Remove num from the "grpc" section 
It isn't needed and can be replaced with additional checks during the
iteration.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:44:06 +03:00
Pavel Karpy
281befec67 [#1418] blobstor: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
babd382ba5 [#1418] engine: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
da3ae202f0 [#1436] node: Log service initialization/boot up 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-02 16:16:39 +03:00
Leonard Lyubich
2da8396a9f [#1454] cli/util: Support container sessions in sign session-token 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
4c8ec20e32 [#1423] session: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Evgenii Stratonikov
dda56f1319 [#1437] neofs-cli: Move util command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-26 12:03:39 +03:00
Evgenii Stratonikov
6cb9c13c5e [#1424] neofs-cli: Fail immediately if a client can't be created 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
295ec3700a [#1424] neofs-cli: Fail immediately if a key can't be fetched 
If the key can't be fetched, an error is always returned, so it makes
sense to fail the whole command inside of a `key.Get*()`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
d9c5ca5e77 [#1381] neofs-cli: Move control command to a separate module 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
71d823f192 [#1381] neofs-cli: Move JSON and verbose printing to a common package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Leonard Lyubich
bb25ecbd15 [#1400] owner: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:33:14 +03:00
Evgenii Stratonikov
4761857fb3 [#1396] cmd: Allow to autogenerate documentation for all commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:35:22 +03:00
Evgenii Stratonikov
c6325fdc91 [#1379] neofs-cli: Move accounting command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Evgenii Stratonikov
56f33436dd [#1379] neofs-cli: Move exitOnErr to internal package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Evgenii Stratonikov
094534e31a [#1379] neofs-cli: Add key.GetOrGenerate helper 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Evgenii Stratonikov
cd46a7478e [#1379] neofs-cli: Bind key-related arguments to viper 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Evgenii Stratonikov
a198a4858c [#1379] neofs-cli: Move client creation to internal/client 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Evgenii Stratonikov
cbc4ca800d [#1379] neofs-cli: Move common flags to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Leonard Lyubich
aeb9884218 [#1389] crypto: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-19 10:12:14 +03:00
Leonard Lyubich
5e50ddd7f5 [#1389] storagegroup: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-19 10:12:14 +03:00
Leonard Lyubich
f15e6e888f [#1377] oid, cid: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-16 15:33:22 +03:00
Evgenii Stratonikov
1c7195666c [#722] neofs-adm: Unify pasword retrieval functions 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-16 15:07:31 +03:00
Evgenii Stratonikov
b685af487f [#722] neofs-adm: Replace path with filepath package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-16 15:07:31 +03:00
Evgenii Stratonikov
5bd6624eb6 [#722] neofs-adm: Provide contract state to getContractDeployParameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-16 15:07:31 +03:00
Evgenii Stratonikov
efb6545bfe [#722] neofs-adm: Allow to initialize local dump 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-16 15:07:31 +03:00
Evgenii Stratonikov
c55950bd70 [#722] neofs-adm: Check returned tx hash 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-16 15:07:31 +03:00
Evgenii Stratonikov
37f9d083fb [#722] neofs-adm: Hide N3 client behind an interface 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-16 15:07:31 +03:00
Evgenii Stratonikov
57200e18cd [#1375] neofs-adm: Do not update NNS group if the key is the same 
If the group key is already set, do not send any transactions.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-13 12:31:46 +03:00
Leonard Lyubich
3a188bb2e5 [#1371] bearer: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-12 15:16:17 +03:00
Leonard Lyubich
ae92074272 [#1369] checksum: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-12 09:48:31 +03:00
Leonard Lyubich
088df0e2a9 [#1369] version: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-12 09:48:31 +03:00
Evgenii Stratonikov
875f0e79a2 [#1346] neofs-adm: do not ask password twice 
When a contract wallet is created, there is no need to decrypt
newly-created accounts.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-11 12:01:11 +03:00
tasselsd
379c70cc90 [#1363] Use network package to parse and verify the announced address. 
Signed-off-by: tasselsd <tasselsd@outlook.com>
 2022-05-11 11:22:34 +03:00
Alex Vanin
d6be5d4087 [#1358] example: Add configs for inner ring application 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-05-11 11:15:54 +03:00
Elizaveta Chichindaeva
cc7a723d77 [#1320] English Check 
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
 2022-05-11 10:40:02 +03:00
Leonard Lyubich
cd545f0160 [#1351] cli: Fix connection scheme loss during endpoint parsing 
In previous implementation NeoFS CLI app used `network.Address.HostAddr`
as a server URI, which caused scheme loss since host address doesn't
contain it.

Rename `HostAddr` to `URIAddr` and make it to return URI address with
`grpcs` scheme if TLS is enabled. Make `TLSEnabled` unexported since it
was used to provide default `tls.Config` only (it is used by default in
SDK).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-05 10:43:27 +03:00
Evgenii Stratonikov
a4769d8624 *: use require.ErrorIs where possible 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-29 16:47:57 +03:00
Pavel Karpy
a275a71a87 [#1318] node: Use new tombstone handling 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-04-29 16:38:52 +03:00
Evgenii Stratonikov
b74fb2b932 [#1316] neofs-cli: Print object ID in base-58 
It was missed from dd9bd05ba.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-25 10:54:18 +03:00
Evgenii Stratonikov
a254fd6bc8 [#1216] neofs-cli: Allow to use session token from file 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-25 10:47:43 +03:00
Evgenii Stratonikov
a95bdb1811 [#1216] neofs-cli: Allow to create and save session token 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-25 10:47:43 +03:00
Evgenii Stratonikov
8d79168129 [#1216] neofs-cli: Reuse key retrieving code between modules 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-25 10:47:43 +03:00
Evgenii Stratonikov
006d6e8b48 [#1216] neofs-cli: Reuse GetSDKClient between CLI modules 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-25 10:47:43 +03:00
Evgenii Stratonikov
fd8c00400d [#1216] neofs-cli: Rename token subcommand to bearer 
It works only with the bearer token.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-25 10:47:43 +03:00
Pavel Karpy
bd27837364 [#1321] node: Register GC event channel before shard init 
Morph "NewEpoch" event handling was registered in a closure over
`addNewEpochNotificationHandler` func. That may lead to the data race:
if a shard was initialized before the event registration, everything works
as planned, but if registration was made earlier, it was not able to
include GC handlers since a shard has not called `eventChanInit` yet and,
therefore, it has not registered handler yet.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-04-25 10:39:37 +03:00
Leonard Lyubich
d4569946c5 [#1313] container: Do not invalidate cache on Container.Delete 
In previous implementation `Container.Delete` operation caused local
node's cache invalidation (container itself, eACL and listings). Any
subsequent `Container.Get` operation reversed invalidation. Given the
low latency sensitivity of deleting a container, there is no need to
touch the cache. With this approach, all pending deletion operations on
the node via the NeoFS API protocol will be delayed by the cache TTL.

Do not call cache invalidation ops in `morphContainerWriter.Delete`.
Remove no longer needed `InvalidateContainerListByCID` and
`InvalidateContainer` methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-04-25 10:34:12 +03:00
Evgenii Stratonikov
606dfa3414 [#1308] neofs-adm: Add command to dump GAS balances 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-12 17:37:40 +03:00
Evgenii Stratonikov
f1e91313db [#1311] neofs-adm: Set stdout as default output 
Cobra `Command` prints to stderr by default.
`neofs-cli` already sets the output properly.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-12 17:37:26 +03:00
Alex Vanin
8054311eee [#1170] neofs-adm: Update storage config template 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-12 17:36:37 +03:00
Pavel Karpy
01e69f2f7a [#1170] pkg/morph: Add worker pool 
Add worker pool to the listener to prevent blocking. It is used only for
notary notifications and new block events handling since it uses RPC
calls. That may lead to the deadlock state: neo-go cannot send RPC until
notification channel is read but notification channel cannot be read since
neo-go client cannot send RPC.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-04-08 11:52:18 +03:00