Commit graph

816 commits

Author SHA1 Message Date
Pavel Karpy
7c02a2e251 [#1087] *: Adopt SDK changes
- Update `neofs-sdk-go`:
v0.0.0-20211230072947-1fe37df88f80 => v0.0.0-20220113123743-7f3162110659

- Add client interface that duplicates SDK's client behaviour and new
`MultiAddressClient` interface that has method that iterates over wrapped
clients.

- Also start using simple client mode that does not require parsing statuses
outside the SDK library.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-01-14 17:29:03 +03:00
Evgenii Stratonikov
0d969d7a06 [#1060] blobstor: allow to disable compression based on content-type
For some data compression makes little sense, as it is already compressed.
This commit allows to leave such data unchanged based on `Content-Type`
attribute. Currently exact, prefix and suffix matching are supported.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-13 14:20:36 +03:00
Evgenii Stratonikov
5828f43e52 [#851] util/rand: use single random source
It is much more convenient to skip source creation.
Also fix some bugs:
1. `cryptoSource.Int63()` now returns number in [0, 1<<63) as required
   by `rand.Source` interface.
2. Replace `cryptoSource.Uint63()` with `cryptoSource.Uint64` to allow
   generate uint64 numbers directly (see rand.Source64 docs).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 19:29:27 +03:00
Evgenii Stratonikov
efabffbfd0 [#1041] cmd/*: factor out autocomplet command generation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Leonard Lyubich
e74ff7c15a [#1062] node/object: Change the sequence of servers
In previous implementation ACL server was the 1st (except metric server
in some cases) server in pipeline of Object service servers. This led to
the fact that errors of this handler could not be reduced to status
responses.

Nest object ACL server into signature and response servers to support
common response format.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-11 11:08:55 +03:00
Pavel Karpy
86b90eb944 [#1059] cli: Add SetShardMode operation
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Alex Vanin
90f05d4448 [#1066] cli: Support basic ACL constants without final flag
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-30 14:07:18 +03:00
Evgenii Stratonikov
56b8793520 [#1056] neofs-adm: deploy proxy contract without arguments
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-30 13:54:07 +03:00
Evgenii Stratonikov
185a2827cc Revert "[#1038] adm: Do not cut data args in NNS"
This reverts commit acb4a9e5b4.
2021-12-30 10:20:54 +03:00
Evgenii Stratonikov
549546dea1 [#1035] neofs-adm: reuse NEF and manifest during alphabet contract update
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-30 10:20:54 +03:00
Evgenii Stratonikov
4673c81451 [#1035] neofs-adm: update contracts in a single transaction
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-30 10:20:54 +03:00
Pavel Karpy
a8ba573ec8 [#1057] node/config: Use mode from config in shard/s constructor
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-29 18:06:15 +03:00
Pavel Karpy
8b8a815fb3 [#1057] node/config: Add shard mode config param
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-29 18:06:15 +03:00
Evgenii Stratonikov
e96eb3e00b [#1054] neofs-node: add epoch metric
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 15:34:20 +03:00
Pavel Karpy
f5a9735e1c [#1048] cli: Add shard list operation
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
0e5410603e [#1048] control: Add ListShards implementation to ctrl svc
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
acb4a9e5b4 [#1038] adm: Do not cut data args in NNS
The updated version of the `NNS` contract now supports the third `data`
argument that may provide additional information.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-20 14:43:29 +03:00
Pavel Karpy
11466ea3cc [#1031] adm: Use 10 as maxConnPerHost in neo-go client
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 18:55:53 +03:00
Pavel Karpy
c49e53ba9d [#1031] morph: Add maxConnPerHost option
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 18:55:53 +03:00
Pavel Karpy
63e035bd8a [#1031] node: Add maxConnPerHost to config
It allows configuring number of neo-go client opened connections per one
host.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 18:55:53 +03:00
Pavel Karpy
90dbf3d944 [#1039] cli: Make return codes depend on errors
Cast client errors to well-known public `sdk` errors and return appropriate
exit code.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 16:03:41 +03:00
Alex Vanin
2a031c5542 [#1013] Support autocomplete in neofs-adm
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-14 17:34:10 +03:00
Stanislav Bogatyrev
bd4d17ef9e [#1034] Add shorthand for generate-key flag
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
2021-12-10 19:54:41 +03:00
Pavel Karpy
cb246dfab7 [#1015] docs: Add subnetwork managing HOWTO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-09 09:47:50 +03:00
Pavel Karpy
14f49df658 [#1015] docs: Add subnetwork creation HOWTO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-09 09:47:50 +03:00
Alex Vanin
d89ffbfbdf [#1020] neofs-adm: Make key flag non global in subnet
Key flag should not be global because `subnet get` command
does not require any credentials.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-08 18:49:41 +03:00
Alex Vanin
3bb67075f4 [#1020] neofs-adm: Do not ask key in morph subnet get command
There is no need to provide credential to fetch public data from
side chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-08 18:49:41 +03:00
Pavel Karpy
15ff98c229 [#1024] adm: Change private key to wallet in subnet flags
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-08 15:57:22 +03:00
Leonard Lyubich
feb0a65efb [#1008] ir/container: Customize fee for named container registration
In notary disabled environment, approval of container creation with nice
name attribute takes much more additional GAS than other operations
(due to NNS invocation).

Morph library changes:
  * add the ability to specify per-op fees using `StaticClient` options;
  * add the ability to customize fee for `Put` operation with named
    container in container morph client.

Inner Ring changes:
  * add `fee.named_container_register` config value which specifies
    additional GAS fee for the approvals of the named container
    registrations;
  * pass the config value to `WithCustomFeeForNamedPut` option of
    container morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-08 13:57:01 +03:00
Pavel Karpy
b4c36a109d [#1019] adm: Fetch subnet SH from NNS
Also, delete `-a` (`--contract`) flag.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-08 13:33:36 +03:00
Alex Vanin
c383fc6929 [#1014] neofs-adm: Print NNS contract version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-07 18:55:44 +03:00
Evgenii Stratonikov
a5e952cc72 [#1006] neofs-adm: skip missing entries in dump-hashes
We have `subnet` contract in list, but it isn't currently deployed.
This commit skips missing NNS entries to handle such situation. In
future we may optimize this to be done in 1 round-trip.

As a nice side-effect, dump-hashes for notary-enabled environment works
even if notary is disabled -- it just prints zero hash for proxy
contract.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-03 13:24:17 +03:00
Sergio Nemirowski
4ad2961c51 [#1005] neofs-node: fix usage message
Signed-off-by: Sergio Nemirowski <sergio@nspcc.ru>
2021-12-03 10:37:24 +03:00
Evgenii Stratonikov
ca894fee23 [#749] neofs-adm: make alphabet size check more robust
We can now store wallet for signing manifest groups in the same dir.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-02 17:24:16 +03:00
Evgenii Stratonikov
40b51b3586 [#995] neofs-adm: implement deposit-notary command
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-02 17:24:16 +03:00
Evgenii Stratonikov
fba8890224 [#995] neofs-adm: split out tx context
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-02 17:24:16 +03:00
Evgenii Stratonikov
d1be5b5f9e [#878] neofs-node: default to secure TLS settings
Support TLS >=1.2 only and strong cipher suites.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-02 12:05:41 +03:00
Leonard Lyubich
1b698867a7 [#979] adm/subnet: Fix typos
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
dd2998d724 [#979] adm/subnet: Rename id flag to node in node cmd
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
5f915dfb43 [#979] adm/subnet: Rename id flag to admin in admin cmd
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
cc8a0037e0 [#979] adm/subnet: Fix print out of subnet info in get cmd
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
3fcd7bc948 [#979] adm/subnet: Always set group ID parameter in client cmds
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
4d79195ede [#979] adm/subnet: Fix notary issues
Only `subnet create` command can generate notary requests.

Remove global `non-notary` flag. Add `notary` flag to `create` cmd.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
98e0792b08 [#979] adm/subnet: Add commands to add/remove nodes
Implement `ManageNodes` operation on morph subnet client.
 Add `node add` and `node remove` commands to `subnet` section.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
579c3717a5 [#979] adm/subnet: Inherit pre-run functions in child commands
Cobra commands don't call `PreRun` functions on parent by default. We
need to do it in `subnet` command of admin utility in order to inherit
viper bindings.

Add `inheritPreRun` function which makes sub-commands to call `PreRun`
functions before its own `PreRun`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
6facc26cb9 [#979] adm/subnet: Fix handling of non-notary flag in initClient
`initSubnetClient` must call `EnableNotarySupport` only if non-notary
flag is not set.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
4b8ca75274 [#979] adm: Fix problems after testing
Use persistent flags on parent command in order to inherit flags in
sub-commands. Turn on notary mode of morph client in `subnet` command of
admin utility for notary environments.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
9622c9c858 [#979] adm: Register NNS domain of the deployed contracts
Make `deployContracts` method to call `nnsRegisterDomain` in order to
register NNS domain for deployed contracts.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
2f2ca2e0bd [#979] adm: Deploy missing contracts during updating
Make `nnsResolveHash` function to return declared error on `token not
found` fault exception. Catch this error in `deployContracts` method,
and switch to deployment if updating contract is missing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
e8f8e58e90 [#979] adm: Add subnet-related commands
Add `subnet` command which contains all subnet-related commands. Add
sub-commands:
  * `create` for creation;
  * `remove` for removal;
  * `get` for reading;
  * `admin` for admin management;
  * `client` for client management.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
70f17dc778 [#979] adm: Deploy and update Subnet contract
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Evgenii Stratonikov
98f288c946 [#984] neofs-adm: fix contract hash check in NNS
Don't update if the hash is already there.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-29 11:19:36 +03:00
Evgenii Stratonikov
7765ab3f02 [#984] neofs-adm: replace addRoot with register for NNS
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-29 11:19:36 +03:00
Evgenii Stratonikov
63a8b530f6 [#984] neofs-adm: compare NEF.Checksum to check if contract was updated
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-29 11:19:36 +03:00
Pavel Karpy
6207a2f261 [#988] cli: Add subnet flag to container create
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-11-29 11:03:56 +03:00
Leonard Lyubich
0fb838b169 [#983] node: Configure subnets
Add `subnet` sub-section to `node` section of storage node config. Add
`entries` value which allows to enumerate subnets for entrance. Add
`exit_zero` value which allows to not enter zero subnet.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-28 16:42:46 +03:00
Leonard Lyubich
41eaa1e246 [#973] ir: Listen and process Put/Delete events of Subnet contract
Define notification events, implement parsers. Add morph client of
Subnet contract. Listen, verify and approve events in Inner Ring app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-28 16:15:18 +03:00
Alex Vanin
a74a402a7d [#965] policer: Implement continuous replication
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-11-26 15:39:38 +03:00
Leonard Lyubich
7f5fb130c0 [#961] *: Support NeoFS API status returns
Upgrade NeoFS API Go library to version with status returns. Make all API
clients to pull out and return errors from failed statuses. Make signature
service to respond with status if client version supports it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 09:26:40 +03:00
Leonard Lyubich
44d0d453ef [#961] client: Clarify error returns in docs
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 09:26:40 +03:00
Alex Vanin
378474b02c [#950] neofs-cli: Attach x-headers in object range
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-11-19 17:10:01 +03:00
Pavel Karpy
dbf3a2f2fb [#971] *: Add optional parameters to container morph client calls
Adapt all container wrapper calls to new
structures.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-11-19 09:58:03 +03:00
Pavel Karpy
bad739258e [#971] *: Add notification TX hash to neofs/netmap morph client calls
Add hash of the TX that generated notification
to neofs/netmap event structures. Adapt all
neofs/netmap wrapper calls to new structures.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-11-19 09:58:03 +03:00
Pavel Karpy
8e1f187822 [#971] ir: Drop CLI validators voting functionality
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-11-19 09:58:03 +03:00
Evgenii Stratonikov
aba09bb853 [#903] config: support floating-point sizes
Rounding is done using the suffix boundary (i.e. floating point value
with `gb` suffix will return size which is an integer number of
megabytes).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-16 12:43:55 +03:00
Evgenii Stratonikov
e7fd980951 [#859] config: rename writecache.size_limit to writecache.capacity
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Evgenii Stratonikov
c015b04ed5 [#859] config: rename shallow_* to width and depth
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Evgenii Stratonikov
684e3e0ae0 [#859] config: rename mem_size to memcache_capacity
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Evgenii Stratonikov
a180f09523 [#859] config: rename max_size to max_object_size
`_size` suffix by itself makes me think about restrictions on database
size.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Evgenii Stratonikov
da51c119d5 [#859] config: rename opened_cache_size to opened_cache_capacity
Capacity is a common term to use with cache, see e.g. leveldb
configuration.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Evgenii Stratonikov
0759d8d0e5 [#859] config: rename small_size_limit to small_object_size
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Evgenii Stratonikov
cbf8ca12b0 [#859] config: rename refill_metabase to resync_metabase
This flag enables metabase content synchronization with blobstor on
start.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-15 13:14:24 +03:00
Leonard Lyubich
fb89d29574 [#969] node: Do not require shard config in relay mode
Relay storage node doesn't exec local object operations, so it doesn't need
shard configuration.

Add `required` bool parameter to `engineconfig.IterateShards`. Make it to
panic if it is `true`, and immediately return otherwise. Pass `false` if
node is configured as relay in app (it also prevents panic).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-15 06:47:25 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-12 17:29:09 +03:00
Leonard Lyubich
10f0bd91d6 [#922] engine: Change interface of container operations
Add `error` to return. Improve docs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-12 17:28:38 +03:00
Leonard Lyubich
6b1ce99c35 [#922] node: Block data ops of storage engine via Control API
There is a need to block execution of local object storage operations if
node is put into maintenance mode (resume if the node is taken out of
maintenance mode).

Call `BlockExecution` method if `ControlService.SetNetmapStatus` was called
with `MAINTENANCE` status. Call `ResumeExecution` if it was called with
another status.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-12 17:28:38 +03:00
Leonard Lyubich
29644e9bc2 [#922] cli: Support MAINTENANCE netmap status
Accept `maintenance` netmap status in `control set-status` command and
resolve it to `control.NetmapStatus_MAINTENANCE` value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-12 17:28:38 +03:00
Leonard Lyubich
0ec8f529ab [#842] object/delete: Set tombstone local node as tombstone owner
All objects in NeoFS must have owner ID. In previous implementation Object
Delete service handler set owner ID from request session token. If removal
was executed w/o a session, object with tombstone was prepared incorrectly.
In order to fix this node should set its own ID and become an owner of the
tombstone object.

Extend `NetworkInfo` interface required by Object.Delete handler with
`LocalNodeID` method which returns `owner.ID` of the local node. Implement
the method on `networkState` component of storage node application which is
updated on each node state change in NeoFS network map. Set owner returned
by `LocalNodeID` call as tombstone object's owner in Delete handler.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-10 16:18:02 +03:00
Pavel Karpy
6cab1635d4 [#936] ir: Do not require number of Alphabet contracts to be set
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-11-10 16:11:57 +03:00
Leonard Lyubich
e500efb9b3 [#957] services/reputation: Refactor usage of NeoFS API client
The client needs of the Reputation service are limited and change not often.
Interface changes of the client library should not affect the operation of
various service packages, if they do not change their requirements for
the provided functionality. To localize the use of the base client and
facilitate further support, an auxiliary package is implemented that will
only be used by the Reputation service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-03 18:30:09 +03:00
Leonard Lyubich
bbc2b873ab [#950] cli: Refactor usage of NeoFS API client
The client needs of the CLI application are limited and change not often.
Interface changes of the client library should not affect the operation of
various application packages, if they do not change their requirements for
the provided functionality. To localize the use of the base client and
facilitate further support, an auxiliary package is implemented that will
only be used by the CLI application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-03 18:26:36 +03:00
Leonard Lyubich
a3414b36dd [#958] cli/locode: Open RO database in info command
CLI `util locode info` command does not write operations. Opening a
`locodebolt.DB` instance in RW mode is redundant.

Provide `locodebolt.ReadOnly()` option to `locodebolt.DB` constructor in
order to create READ flock over BoltDB file.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-03 10:19:45 +03:00
Evgenii Stratonikov
088c894f44 [#733] neofs-adm: read contracts on start
Fail early and reduce disk operations when reading from archive.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-01 15:17:18 +03:00
Evgenii Stratonikov
0b6350d463 [#733] neofs-adm: fetch contracts release from Github
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-01 15:17:18 +03:00
Evgenii Stratonikov
c1e1b65ad9 [#733] neofs-adm: allow to read contract from tar archive
Assume contracts path is tar.gz archive if it is not a directory.
In theory we could support raw tar but it has little value as our
distribution format is tar.gz and this is the primary use-case for this
feature (download and use).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-01 15:17:18 +03:00
Evgenii Stratonikov
49c9dbfba8 [#877] config/engine: use default section for shards
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-28 15:04:14 +03:00
Evgenii Stratonikov
cb9bf00ceb [#877] neofs-node/config: allow to provide default values
In case we have multiple sections with similar structure (e.g. shards)
having defaults in a single place is easier to work with.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-28 15:04:14 +03:00
Evgenii Stratonikov
118c3b3cfe [#940] neofs-adm: print contract version in dump-hashes
```
nns         (unknown):  7061fbd31562664b58f422c3dee4acfd70dba8af
alphabet 0  (v0.12.1):  2392438eb31100857c0f161c66791872b249aa13
alphabet 1  (v0.12.1):  83ef4226d5d6519ca9c99a5de13b1b5ca223a6ad
alphabet 2  (v0.12.1):  6250927beaa9aa5a00171379dcb7187b0c91d17d
alphabet 3  (v0.12.1):  1d6a2519ba41a139b2ced1bfd5013938271a7578
alphabet 4  (v0.12.1):  b65fc7a3c31cf57a90d7eb1c0e9909e4ca69133c
alphabet 5  (v0.12.1):  f95b6ff8cd3b027c9911c18115518ad8c5d2f591
alphabet 6  (v0.12.1):  5b17c579bf56884fd68af152432b3b5aee7aee76
audit       (v0.12.1):  85fe181f4aa3cbdc94023d97c69001ece0730398
balance     (v0.12.1):  dc1ec98d9d0c5f9dfade16144defe08cffc5ca55
container   (v0.12.1):  1b6e68d299b570e1cb7e86eadfdc06aa2e8e0cc5
neofsid     (v0.12.1):  0a64ce753653cc97c0467e1334d9d3678ca8c682
netmap      (v0.12.1):  7c5bdb23e36cc7cce95bf42f3ab9e452c2501df1
reputation  (v0.12.1):  7ad824fd1eeb1565be2cee3889214b9aa605d2fc
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-28 09:35:52 +03:00
Alex Vanin
fbed86da2c [#943] service/object: Remove KeyStore from CommonPrm
There is no point to pass key storage in parameters because
it can be defined on the service level of application.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-27 20:25:25 +03:00
Alex Vanin
2fbdcbdee1 [#943] service/object: Check session token expiration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-27 20:25:25 +03:00
Alex Vanin
0f598289f6 [#942] neofs-cli: Return endpoint flag to control netmap-snapshot command
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-27 19:37:19 +03:00
Evgenii Stratonikov
41a9261069 [#935] config/example: quote spaces in node.env
```
> source config/example/node.env
config/example/node.env: строка 14: /dns4/s02.neofs.devenv/tcp/8081: Нет такого файла или каталога
config/example/node.env: строка 16: MSK: команда не найдена
config/example/node.env: строка 35: 028f42cfcb74499d7b15b35d9bff260a1c8d27de4f446a627406a382d8961486d6: команда не найдена
config/example/node.env: строка 48: https://rpc2.morph.fs.neo.org:40341: Нет такого файла или каталога
config/example/node.env: строка 49: wss://rpc2.morph.fs.neo.org:40341/ws: Нет такого файла или каталога
config/example/node.env: строка 53: https://rpc2.n3.nspcc.ru:30341: Нет такого файла или каталога
```

New implementation is still basic, but we don't need to parse any env
file, just have a correct example.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-25 11:10:01 +03:00
Pavel Karpy
208d150500 [#937] node: Init node state before gRPC server start
Fix panic when node processes RPC requests
before state initialization is finished.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-22 15:05:00 +03:00
Alex Vanin
eaecc438f8 [#932] neofs-cli: Support ContainerAliasFee global config parameter
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-19 19:15:56 +03:00
Evgenii Stratonikov
751147793f [#927] neofs-adm: provide better error message if endpoint is missing
Note that we cannot mark `--rpc-endpoint` flag as required because
it can be taken from config.

Before:
```
Error: can't create N3 client: failed to get network magic: Post "": unsupported protocol scheme ""
```

Now:
```
Error: can't create N3 client: missing endpoint
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Evgenii Stratonikov
f1202a5738 [#927] neofs-adm: fix dump-config command
```
Error: can't to initialize context: can't read alphabet wallets dir: open : no such file or directory
```
It needs neither contracts nor wallets, just N3 client.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Evgenii Stratonikov
c93c5e726c [#927] neofs-adm: add fee flags to morph init
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Evgenii Stratonikov
26e11a732d [#927] neofs-adm: provide container alias fee on deploy
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Pavel Karpy
0866c1fb90 [#920] cli: Rename RPC flag for control service
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-18 12:19:02 +03:00
Pavel Karpy
f0252e00c1 [#920] cli: Do not add rpc flags to util
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-18 12:19:02 +03:00
Evgenii Stratonikov
1a1435be3d [#791] neofs-lens: inspect objects by address
```
> neofs-lens list --path ./blob/blobovnicza/1/1/0
6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek/9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6
H3VBttoLQoknzMDgnVNyLZ8EpkDnQjnaxDr9fnAWeEHA/Hw1titdGh7BrTe2yLotiYbVh9FQaRRNhoNzXTyetpFgt

> neofs-lens inspect --path ./blob/blobovnicza/1/1/0 \
    --address 6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek/9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6 \
    --header --out payload
Version: v2.1
Type: REGULAR
CID: 6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek
ID: 9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6
Owner: 2dokPzmmcLnnR21jQB3qPppTQRgwMNMKEWD
CreatedAt: 0
PayloadSize: 32
Attributes:
  foo: bar

> hexdump -C payload
00000000  ff 6c d4 71 c4 83 f1 5f  b9 0b ad b3 7c 58 21 b6  |.l.q..._....|X!.|
00000010  d9 55 26 a4 1a 95 04 68  0b 4e 7c 8b 76 3a 1b 1d  |.U&....h.N|.v:..|
00000020
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-18 11:55:02 +03:00
Leonard Lyubich
e5e5395830 [#833] cli/netmap: Beautify output of MillisecondsPerBlock parameter
Convert milliseconds to `time.Duration` value and use its stringer in
`netinfo` command.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-15 13:56:19 +03:00
Leonard Lyubich
8f9b1fe090 [#833] cli: Write MillisecondsPerBlock and NeoFS network config
Print MillisecondsPerBlock and NeoFS network parameters in `netinfo` command
of `netmap` section.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-15 13:56:19 +03:00
Leonard Lyubich
283ccc04b4 [#833] services/netmap: Support new fields of netmap.NetworkInfo
Make the implementation of network info source (Netmap V2 service
dependency) to read MillisecondsPerBlock sidechain parameter and NeoFS
network parameters depending on the client version.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-15 13:56:19 +03:00
Pavel Karpy
fae2c21165 [#873] innerring/config: Delete useless defaults
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
2f2258733b [#873] node: Delete useless notary deposit Amount, Duration envs
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
8374c3d2f5 [#873] node: Delete useless notary timer
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
2a3a6cc0ba [#873] node: Use dynamic notary deposit duration
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
5e816dc01a [#873] node: Make deposits depend on epoch and balances
SN tries to keep 1:3 proportion of GAS and
notary balances respectively. If that proportion
has been messed(means that notary balance is
lower than required) it sends half of its
GAS balance to the notary service.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Leonard Lyubich
40a4a7faa2 [#674] object/put: Use pseudo worker pool for local operations
After storage engine started to limit number of PUT operations there is no
need to limited worker pool in Object Put service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-14 10:20:39 +03:00
Leonard Lyubich
2126235f0e [#674] node: Configure size of per-shard worker pools
Add `shard_pool_size` config to `storage` section. Set app default to 20.
Pass the value to `WithShardPoolSize` option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-14 10:20:39 +03:00
Evgenii Stratonikov
f836e7c1dc [#888] neofs-adm: use constant for update methods
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
31b3c71457 [#888] neofs-adm: allow to work with multiple NNS versions
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
6571f9214f [#888] neofs-adm: update container contract deploy parameters
It supports NNS domain and zone now.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
8ddd0aab55 [#888] neofs-adm: set contract hashes on read
Non-alphabet contract are always deployed from committee
so it makes sense to calculate hashes in one place.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
6bcd4811e2 [#888] neofs-adm: update NNS wrappers
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
bca41f87af [#888] neofs-adm: deploy NNS contract first
Container contract uses actual NNS interface. This also aleviates some
pain related to update as neofs-adm code itself uses current NNS
version.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
1177f0ca78 [#888] neofs-adm: deploy contracts with Global scope
Container contract now calls NNS contract to register TLD for nice
names. This must be witnessed by a committee.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
ec27a96b8c [#888] neofs-adm: dump NNS contract hash
This can be useful for debugging.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Alex Vanin
e500bd13a9 [#906] neofs-adm/docs: Fix notes from Fyrchik
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-11 18:33:45 +03:00
Elizaveta Chichindaeva
6c155f62c4 [#906] neofs-adm/docs: Fix language
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2021-10-11 18:33:45 +03:00
Alex Vanin
d50e8f2e72 [#906] neofs-adm/docs: Add step-by-step deploy guide
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-11 18:33:45 +03:00
Alex Vanin
3c47cff717 [#906] neofs-adm: Add README file
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-11 18:33:45 +03:00
Alex Vanin
0126f18531 [#886] cli: Fix RPC flags for control service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-08 17:18:18 +03:00
Evgenii Stratonikov
cc377b34d2 [#868] blobstor: initialize (de-)compressors in Init
Do not log in options constructors. Also failure to
initialize compression module (possibly due to invalid options) is
certainly an error deserving proper treatment.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-08 16:01:45 +03:00
Evgenii Stratonikov
00f14d4dcd [#857] config: use size suffixes where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-08 11:26:24 +03:00
Evgenii Stratonikov
a0abf10c8e [#857] config: allow to parse sizes with suffix
The code is taken from viper as `cast` package that we use
doesn't have needed converter.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-08 11:26:24 +03:00
Pavel Karpy
4615ff1392 [#886] cli: Fix API flags
Bind API cobra flags(`ttl' and xhdr')
to the Viper.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-07 18:54:34 +03:00
Alex Vanin
e41e74b5fc [#872] services/container: Ignore passed routes in load router
Higher level solution for original issue with lost announcements
from single node container.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-05 19:29:27 +03:00
Alex Vanin
f08636c518 [#872] cmd/neofs-node: Don't lose local container size estimations
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-05 19:29:27 +03:00
Leonard Lyubich
4d7915a366 [#791] neofs-lens: Implement object listing
Implement `list` command for object listing. Support listing of the objects
from write-cache and blobovnicza.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-05 10:51:10 +03:00
Leonard Lyubich
522cbab47c [#791] cmd: Implement application observing storage engine
Implement skeleton of `neofs-lens` app which is going to be used for working
with storage engine's data.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-05 10:51:10 +03:00
Pavel Karpy
8c59ade4ed [#854] cli: Do not use global flags
Also delete `ttl` and `xhdr` flags from
`accounting balance` command and refactor
command initialization.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-04 12:12:21 +03:00
Evgenii Stratonikov
4ccb3d05d8 [#835] neofs-adm: rename contract update method
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-09-30 20:57:24 +03:00
Alex Vanin
f2a61451b7 [#867] governance: Add option to disable governance sync
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-30 20:57:13 +03:00
Leonard Lyubich
312e356a75 [#645] object: Compare public keys in reputation client constructor
In previous implementation of reputation client constructor compared network
addresses to check the membership of the network map.

Replace network addresses comparison with public key comparison.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
e473f3ac91 [#645] *: Use helper functions to build client.NodeInfo structures
Helper functions from core/client package allow to set public keys of
storage nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
7b228b7603 [#645] *: Construct clients from client.NodeInfo in API client cache
There is a need to have the ability to expand the data needed for client
construction.

Replace `network.AddressGroup` parameter of client cache interfaces with
`client.NodeInfo`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-30 20:57:00 +03:00
Leonard Lyubich
287683c590 [#863] node: Do not open main chain client connection
In current implementation storage node doesn't use main chain client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-29 12:55:48 +03:00
Leonard Lyubich
50c4c5116d [#863] node: Prevent panic about missing Neo RPC endpoints
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-29 12:55:48 +03:00
Alex Vanin
409b72cb5b [#786] neofs-node: Ignore proxy contract in notary disabled environment
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-27 16:20:31 +03:00
Leonard Lyubich
0e3e8db5c0 [#845] node: Configure Object PUT remote and local pools separately
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-27 16:20:19 +03:00
Leonard Lyubich
ee20200c2e [#845] object/put: Separate pools for local and remote operations
In previous implementation Object PUT used single pool of workers for local
and remote ops, but these ops are heterogeneous.

Use remote/local pool for remote/local operations in PUT service. At first
the pools are configured with the same size.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-27 16:20:19 +03:00
Alex Vanin
3e5c7e0ade [#836] neofs-adm: Fix proxy contract deploy arguments
Proxy contract does not include notary flag because
contract is useless without notary subsystem.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-23 09:55:46 +03:00
Alex Vanin
4c30757439 [#836] neofs-adm: Do not define contract owner
Contract owners are removed in neofs-contract v0.11.0.
Now side chain committee has rights to update contracts.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-23 09:55:46 +03:00
Alex Vanin
ce8a906bb5 [#834] neofs-adm: Update NNS contract during contract update
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Alex Vanin
d996004d80 [#834] neofs-adm: Use committee signature when updating alphabet contract
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Alex Vanin
6b3a9e6fcc [#834] neofs-adm: Update contract update if it exists
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Alex Vanin
60636d4c1d [#834] neofs-adm: Throw error if deploy produce non-HALT state
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Leonard Lyubich
d938c7267b [#789] node/config: Add Shard's refill_metabase config to examples
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-15 18:22:09 +03:00