Commit graph

1046 commits

Author SHA1 Message Date
Pavel Karpy
37ab26bfa9 [#1296] cli: "ID" -> "OID"
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 14:29:50 +04:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Pavel Karpy
5139dc9864 [#1706] cli: Do not duplicate payload on SG put
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-19 09:49:47 +04:00
Evgenii Stratonikov
f8b106ac85 [#1684] *: Fix linter warnings
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-15 10:57:31 +03:00
Leonard Lyubich
67af4c89a8 [#1649] cli: Refactor --owner flag definition in container list cmd
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-08-15 10:18:51 +03:00
Leonard Lyubich
7a26b2a57a [#1649] cli: Add option to print attributes in list
Define `--with-attr` flag of `container list` which makes the command to
request and print user attributes for each found element.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-08-15 10:18:51 +03:00
Leonard Lyubich
2740bf7ee4 [#1649] cli: Add option to print attributes in list-objects
Define `--with-attr` flag of `container list-objects` which makes the
command to request and print user attributes for each object from the
container.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-08-15 10:18:51 +03:00
a.y.volkov
81684b6f04 [#1665] neofs-lens: Use stdout as default output
Signed-off-by: a.y.volkov <a.y.volkov@yadro.com>
2022-08-09 13:23:32 +03:00
Pavel Karpy
9aba0ba512 [#1634] meta: Add epoch state
It allows performing expiration checks on the stored objects.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:31:49 +03:00
Pavel Karpy
a97dee008c [#1648] morph: Change endpoint priority order
The lowest value means the highest priority.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:11:24 +03:00
Pavel Karpy
8f44335925 [#1651] cli: Fix CID setting in eACL creation
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 17:04:42 +04:00
Pavel Karpy
0a60524a9c [#1654] ir, node: Drop deprecated profiler and metrics config sections
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 15:38:35 +03:00
Pavel Karpy
713cfa5610 [#1655] cli: Do not force specifying session lifetime
We have the default value which is also printed in the help messages but any
call that does not specify that flag leads to an error.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 15:37:50 +03:00
Evgenii Stratonikov
7ecf54c6df [#1631] neofs-adm: Set group for updated alphabet contracts
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:09:48 +03:00
Evgenii Stratonikov
d3eac7626d [#1631] neofs-adm: Set group for a proper contract
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:09:48 +03:00
Evgenii Stratonikov
5d02613c33 [#1633] neofs-adm: Allow to set custom NNS zone for deployed contracts
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
6146df4998 [#1633] neofs-adm: Allow to provide arguments to deployed contracts
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
dd30703a6b [#1633] neofs-adm: Allow to update arbitrary contracts
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
fc06b6e89a [#1633] neofs-adm: Allow to deploy arbitrary contracts
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
38558a3238 [#1633] neofs-adm: Start chain only on happy path
If the error is returned, it will not be closed properly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
642d7328ab [#1633] neofs-adm: Fix dump generation in init
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
07465849a4 [#1637] go.mod: Update neo-go to v0.99.1
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-28 20:11:45 +03:00
Pavel Karpy
8162b27264 [#1627] node: Add tree service to the config framework
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-27 12:15:56 +03:00
Evgenii Stratonikov
10c855efef [#1624] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-25 16:56:02 +03:00
Evgenii Stratonikov
6049022f7e [#1612] neofs-cli: Remove RunE functions from commands
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-22 16:18:09 +03:00
Evgenii Stratonikov
a52e7c2c99 [#1612] neofs-cli: Unify expiration flags
Use `expire-at` everywhere expiration epoch is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-22 16:18:09 +03:00
Evgenii Stratonikov
50e28f22f9 [#1559] shard: Change Degraded mode string representation
It is a flag, but is a `degraded-read-write` mode for a user.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
4944490ffb [#1559] local_object_storage: Move shard to the DegradedReadOnly mode
`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
1e786233bf [#1559] local_object_storage: Provide readOnly flag to Open
We should be able to reopen storage in readonly in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
339864b720 [#1559] local_object_storage: Move shard.Mode to a separate package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
7410827db8 [#1609] config: Allow to prioritize N3 endpoints
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 16:08:42 +03:00
Evgenii Stratonikov
aed83d1660 [#1609] config: Remove mainchain section
It is deprecated and unused, there is no need to have it in the
documentation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 16:08:42 +03:00
Pavel Karpy
a4ed91f9b5 [#1614] node: Update health metric on its change
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-21 16:07:47 +03:00
Pavel Karpy
581a9901c9 [#1614] metrics: Add health metrics
Also, rename metrics structure since it collects not only storage metrics
now.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-21 16:07:47 +03:00
Evgenii Stratonikov
b549cc314c [#1607] services/tree: allow to customize some parameters
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
2334e94fdb [#1607] neofs-node: Initialize storage before other services
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
a4adb79db7 [#1607] pilorama: Enable tree service explicitly
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
26041f18bf [#1505] pilorama: Allow to customize database parameters
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
3904b0e017 [#1333] neofs-cli: Add control synchronize-tree command
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
bfdd68dcb3 [#1333] services/control: Allow to synchronize local trees
Do not check that a node indeed belongs to the container, because the
synchronization will fail in this case anyway.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
0dc1a4e336 [#1333] neofs-node: Initialize tree service before the control one
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
5e843a73f9 [#1333] services/control: Return pilorama info in ListShards RPC
Do not return backend type from the service for now, because memory
backend is expected to vanish.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
36e24f7f78 [#1342] neofs-node: Use the default endpoint for tree service
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
62154da17c [#1324] services/tree: Implement Object Tree Service
Object Tree Service allows changing trees assotiated with
the container in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
198beae703 [#1613] neofs-cli: Print lock object ID
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 17:15:15 +03:00
Evgenii Stratonikov
5c8d725447 [#1610] neofs-adm: Fix generated storage config
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 16:19:41 +03:00
Evgenii Stratonikov
ccba07fe19 [#1608] neofs-adm: Register candidates in a single transaction
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 13:28:17 +03:00
Evgenii Stratonikov
59d08b8dae [#1608] neofs-adm: Print info about already performed steps
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 13:28:17 +03:00
Evgenii Stratonikov
e5c0338f8c [#1602] neofs-ir: Fix defaults for pprof and prometheus
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
3a0b06d5da [#1602] config: Rename metrics and profiler sections
Depracate old names and remove them in the next release.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
ac46d1a11f [#1602] config: Enable metrics and profiler services with a flag
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
12dc5c3395 [#1563] config: Replace 127.0.0.1 with localhost
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
4a8dfd0ce4 [#1606] Remove deprecated functions
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 12:41:06 +03:00
Evgenii Stratonikov
6cc180391e [#1606] go.mod: Update neo-go
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 12:41:06 +03:00
Pavel Karpy
b1e54dec9f [#1461] cli: Add context to GetCurrentEpoch helper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
9c5ef3bab8 [#1461] node: Allow force LOCK removal
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
de31bb27b6 [#1461] cli: Require LOCK object to be expired
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
595fc60d72 [#1461] cli: Reorganize common functions
Make `ParseEpoch` and `GetCurrentEpoch` funcs public and move the first one
to the `common` package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Evgenii Stratonikov
6eb5260562 [#1500] neofs-cli: Check if container contains LOCK objects before removal
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-15 13:25:56 +03:00
Evgenii Stratonikov
c4b86cf1f1 [#1592] Remove debug builds
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
b8508585a5 [#1592] cmd: Ignore error when fetching a defined flag
There is an error only if the flag is not defined, such errors should be
caught during debugging.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
f7d5045876 [#1592] neofs-lens: Add --version flag
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
5569ff82ef [#1587] Do not print build time in version
This makes our build more reproducible.
Also print `Component` and `GoVersion`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-14 10:39:51 +03:00
Pavel Karpy
b3272e7cf1 [#1367] adm: Support homomorphic hashing config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-13 17:10:06 +03:00
Pavel Karpy
9bd1951ca4 [#1365] adm: Add homomorphic hash disabling option
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-13 17:10:06 +03:00
Pavel Karpy
ae86d2907c [#1365] cli: Calculate homomorphic hash flexibly
Do not use homomorphic hash in storage group for containers that have
`homomorphic_hashing_disabled` set to `true`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-13 17:10:06 +03:00
Pavel Karpy
68e583f143 [#1365] cli: Sync container with network config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-13 17:10:06 +03:00
Pavel Karpy
49ba1ef1b6 [#1365] cli: Add SyncContainerSettings func to internal client
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-13 17:10:06 +03:00
Evgenii Stratonikov
1119014e53 [#1579] config: Move storage.default to shard.default
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-13 12:31:42 +03:00
Evgenii Stratonikov
f4a3fc5f11 [#1594] neofs-adm: Support better completion for set-policy
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-13 11:00:35 +03:00
Pavel Karpy
466b9b4273 [#1593] adm: Support changing network config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-12 21:03:43 +03:00
Pavel Karpy
96a24b5721 [#1490] cli: Require SG members to be unique
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-12 17:35:46 +03:00
Leonard Lyubich
9a11a75b77 [#1570] Upgrade NeoFS SDK Go with changed reputation API
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-06 18:21:24 +03:00
Pavel Karpy
12d41918e6 [#1402] cli: Support storage group expiration
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-06 09:56:26 +03:00
Pavel Karpy
cfca88fc5c [#1402] cli: object <-> SG conversion via SDK
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-06 09:56:26 +03:00
Pavel Karpy
bc5882fc89 [#1402] cli: Add cid flag constant
Also, move common flag constants in `root` file of the `storagegroup`
package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-06 09:56:26 +03:00
Leonard Lyubich
c165d1a9b5 [#1556] Upgrade NeoFS SDK Go with changed container API
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-05 11:26:06 +03:00
Evgenii Stratonikov
ad81b6c90a [#1555] neofs-adm: Remove trailing dots from help messages
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
Evgenii Stratonikov
b41658db04 [#1555] neofs-adm: Allow to remove nodes from the netmap
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
Evgenii Stratonikov
32fd3692bf [#1555] neofs-adm: Fix cached group signer
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
Leonard Lyubich
305dd7598f [#1533] acl: Upgrade NeoFS SDK Go with refactored basic ACL
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-25 13:57:21 +03:00
Evgenii Stratonikov
451343c751 [#1539] neofs-adm: Retrieve storage wallet passwords from config
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 13:40:21 +03:00
Pavel Karpy
fc7940a5ea [#1544] cli: Init bearer flags in list-objects cmd
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-24 13:40:07 +03:00
Evgenii Stratonikov
9b3e1bd27b [#1307] neofs-adm: Add tests for morph init command
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 10:33:34 +03:00
Evgenii Stratonikov
189507dc89 [#1541] neofs-cli: allow to print shards list output in JSON
Eventually more parameters will be supported (#1390) and after blobstor
configuration refactoring the output will certainly change. Implement
the simplest approach now.

Marshaling the result directly results in too ugly names and they cannot
be easily customized. Marshaling the results via `jsonpb` is better but
is not that flexible in terms of what we want to output.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 10:33:22 +03:00
Evgenii Stratonikov
9816d59ec0 [#1323] neofs-cli: Reuse JSON flag for multiple commands
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 10:33:22 +03:00
Leonard Lyubich
b67974a8d3 [#xxx] Upgrade NeoFS SDK Go with changed container sessions
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.

Group value, session token and signature in a structure for container
and eACL.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-22 16:38:57 +03:00
Evgenii Stratonikov
972ca83e23 [#1524] writecache: Add some bolt parameters to the configuration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
07e06249d5 [#1524] metabase: Add some bolt parameters to the configuration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
d6043abc24 [#1524] neofs-node: Reuse boltdb config for different components
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
0408acc40e services/control: Autogenerate stable marshalers
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 16:52:58 +03:00
Leonard Lyubich
d2d1694113 [#1513] cli/storagegroup: Fix linter remarks
`staticcheck` linter scolds on potential nil-dereference since it
doesn't know `common.ExitOnErr` definitely exists. Explicit `return`
fixes linter's remark and prevents potential bugs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
Leonard Lyubich
eb7b8bf7c4 [#1513] node: Encode node's public key into binary format once
Avoid per-iteration key encoding due to the immutability of the key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
Leonard Lyubich
4574ba646d [#1513] cli/storagegroup: Fix parsing container and SG flags
Read `id` flag with the storage group ID. Prevent NPE-panic if flag is
missing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
Leonard Lyubich
21d2f8f861 [#1513] Upgrade NeoFS SDK Go with changed netmap package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
Evgenii Stratonikov
24b4c1ecf4 [#1484] neofs-cli: Print error on a separate line from a progress bar
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 16:38:33 +03:00
Evgenii Stratonikov
e42b7f6a65 [#1484] neofs-cli: Print progress bar after the header has been sent
Make it similar to `object get`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 16:38:33 +03:00
Evgenii Stratonikov
b68778ea90 [#1380] neofs-cli: move container command to a separate package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00
Evgenii Stratonikov
fd48b96082 [#1384] neofs-cli: move storagegroup command to a separate package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00
Evgenii Stratonikov
a219e3a667 [#1383] neofs-cli: move object command to a separate package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00