Commit graph

1457 commits

Author SHA1 Message Date
Evgenii Stratonikov
0d969d7a06 [#1060] blobstor: allow to disable compression based on content-type
For some data compression makes little sense, as it is already compressed.
This commit allows to leave such data unchanged based on `Content-Type`
attribute. Currently exact, prefix and suffix matching are supported.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-13 14:20:36 +03:00
Alex Vanin
0f1eb743af [#1077] service/object: Don't override status of payload overtake
Reverse payload overtake triggers direct payload overtake that
sets status and error. We should not override that.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-01-13 10:30:18 +03:00
Evgenii Stratonikov
5828f43e52 [#851] util/rand: use single random source
It is much more convenient to skip source creation.
Also fix some bugs:
1. `cryptoSource.Int63()` now returns number in [0, 1<<63) as required
   by `rand.Source` interface.
2. Replace `cryptoSource.Uint63()` with `cryptoSource.Uint64` to allow
   generate uint64 numbers directly (see rand.Source64 docs).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 19:29:27 +03:00
Evgenii Stratonikov
c35cdb3684 [#1041] autocomplete: fetch output stream from provided command
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Evgenii Stratonikov
c01acba3ce [#1041] autocomplete: beautify help message
Indent sections corresponding to different shells for faster navigation
and remove `#` at the line beginning.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Evgenii Stratonikov
efabffbfd0 [#1041] cmd/*: factor out autocomplet command generation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Leonard Lyubich
fe29080ebb [#1025] ir: Read IR list from role management contract w/o mainchain
If mainchain is disabled in IR config then the node should read inner
ring list via role management contract.

Use `NeoFSAlphabetList` method of morph client as IR lister if
`withoutMainNet` flag is set in IR app.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-12 18:30:36 +03:00
Leonard Lyubich
0b0c61f4a5 [#1063] node/object: Do not skip empty search responses in splitter
In previous implementation `Search` method of transport splitter skipped
responses with empty ID list.

Replace while-loop with do-while one in `TransportSplitter.Search`
method implementation in order to send responses with empty result too.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-11 12:02:54 +03:00
ZhangTao1596
6830d54a3c [#1069] morph/balance: Fix transfer to
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2022-01-11 12:02:11 +03:00
Pavel Karpy
97fd330fae [#1059] control: Fix ListShards wrapper name
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
77808c7b41 [#1059] control: Provide shard's real mode
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
375394dc99 [#1059] shard: Add shard mode to shard Info
Provide shard mode information via `DumpInfo()`. Delete atomic field from
Shard structure since it duplicates new field.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
88bb8e2df9 [#1059] control: Add SetShardMode implementation to ctrl svc
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
fa78a37787 [#1059] control: Add unit test for SetShardMode
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
29c69f37eb [#1059] control: Add SetShardMode RPC wrappers
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
25665cb0fd [#1059] control: Add marshalling and setters to SetShardMode
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
4533784e11 [#1059] control: Recompile proto files
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
41dd385b7b [#1059] control: Add SetShardMode rpc to proto files
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-30 14:14:48 +03:00
Pavel Karpy
93bd6be743 [#1057] shard: Do not allow memory change operations in "read-only"
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-29 18:06:15 +03:00
Pavel Karpy
4f756bf121 [#1057] shard: Make shard to have only two mode
Shard's mode was not used in the Node, so added only two modes whose roles
are clear. More modes will be added in the future.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-29 18:06:15 +03:00
Evgenii Stratonikov
e96eb3e00b [#1054] neofs-node: add epoch metric
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 15:34:20 +03:00
Evgenii Stratonikov
e1137aa09f [#1054] innerring: add epoch metric
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 15:34:20 +03:00
Pavel Karpy
876b0c53de [#1048] control: Rename Blobstore to Blobstor
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
f5a9735e1c [#1048] cli: Add shard list operation
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
284188f8f9 [#1048] node: Fill shard's info with its components' infos
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
0e5410603e [#1048] control: Add ListShards implementation to ctrl svc
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
4e989e7133 [#1048] control: Add unit test for ListShards
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
f88a12eaa7 [#1048] control: Add ListShards RPC wrappers
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
a6b3e16975 [#1048] control: Add marshalling and setters to ListShards
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
a42b3d37f6 [#1048] control: Recompile proto files
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Pavel Karpy
20f11c88ca [#1048] control: Add ListShards rpc to proto files
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-28 13:21:44 +03:00
Alex Vanin
5d46035ae8 [#1052] Tidy INFO logs
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-27 14:28:01 +03:00
Alex Vanin
e610d1ea5f [#1045] Provide sanity check of session token in container service
Without sanity check, container service provides successful response,
even though such request will never be approved by Alphabet nodes.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-20 14:51:07 +03:00
Alex Vanin
bca7cf9470 [#1047] policer: Check context before job selection
When application is being terminated, replicator routine
might be on the object picking phase. Storage is terminated
asynchronously, thus `Select()` may return corresponding
error. If we don't process `context.Done()` in this case,
then application freezes on shutdown.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-12-20 14:42:57 +03:00
Pavel Karpy
c49e53ba9d [#1031] morph: Add maxConnPerHost option
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 18:55:53 +03:00
Pavel Karpy
df0d76dc19 [#1032] node/container/router: Do not check nil route points
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 16:02:29 +03:00
Pavel Karpy
fd9514f90e [#1032] node/container/router: Fix interface description
After #872 `NextStage` method description becomes outdated.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-16 16:02:29 +03:00
ZhangTao1596
1be0c42726 [#1022] writecache: Fix db counter decrement and flushed big object
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2021-12-15 14:23:49 +03:00
Leonard Lyubich
fc42844fbf [#1028] writecache: Evict flushed big objects
Make `flushBigObjects` routine to call `evictObjects` which was
successfully written to `BlobStor`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-14 13:49:47 +03:00
Leonard Lyubich
d246e5193b [#1028] writecache: Mark big objects as flushed after write
Make `flushBigObjects` routine to mark objects which are written to
`BlobStor`. This prevents already flushed objects from being written on
the next iterator tick.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-14 13:49:47 +03:00
Leonard Lyubich
90fd883e32 [#915] blobovnicza: Use mean bucket size multiplier instead of min
For fullness estimation of `Blobovnicza` we use number of object stored
in each size bucket. In previous implementation we multiplied the number
by the difference in bucket boundaries. This expression rather
estimated the minimum volume (and for the smallest bucket, the maximum)
of objects in the bucket.

Multiply number of objects by mean bucket size.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-08 14:24:49 +03:00
Leonard Lyubich
2920c5203b [#915] blobovnicza: Re-use Bolt transaction in syncFullnessCounter
Make `syncFullnessCounter` to accept `bbolt.Tx` argument of Bolt
transaction within which counter should be synchronized. Pass
corresponding transaction during `Init`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-08 14:24:49 +03:00
Leonard Lyubich
feb0a65efb [#1008] ir/container: Customize fee for named container registration
In notary disabled environment, approval of container creation with nice
name attribute takes much more additional GAS than other operations
(due to NNS invocation).

Morph library changes:
  * add the ability to specify per-op fees using `StaticClient` options;
  * add the ability to customize fee for `Put` operation with named
    container in container morph client.

Inner Ring changes:
  * add `fee.named_container_register` config value which specifies
    additional GAS fee for the approvals of the named container
    registrations;
  * pass the config value to `WithCustomFeeForNamedPut` option of
    container morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-08 13:57:01 +03:00
Leonard Lyubich
f9bcb6f5e3 [#1007] metabase: Copy BoltDB value returned by IsSmall
According to BoltDB documentation bucket `value is only valid for the
life of the transaction`.

Make `DB.IsSmall` copy value slice in order to prevent potential memory
corruptions (e.g. `runtime.stringtobyteslice` cast).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-07 18:59:42 +03:00
Leonard Lyubich
3d5eb2a396 [#1008] ir/netmap: Call Register method on peer approval
After https://github.com/nspcc-dev/neofs-contract/issues/154 alphabet
nodes should call `Register` method for approval of the notary
notifications spawned by `AddPeer` method.

Call `register` method for peer approval in Netmap processor.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-07 17:15:44 +03:00
Evgenii Stratonikov
02be6c83a6 morph/client: update morph container wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-02 14:16:23 +03:00
Leonard Lyubich
973e50ad72 [#979] ir/subnet: Register parser and handler of RemoveNode event
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 19:27:16 +03:00
Leonard Lyubich
6f41cd86ed [#979] Upgrade SDK NeoFS Go dependency
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
3550ed9fe4 [#979] ir/subnet: Fix notification event names
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00
Leonard Lyubich
98e0792b08 [#979] adm/subnet: Add commands to add/remove nodes
Implement `ManageNodes` operation on morph subnet client.
 Add `node add` and `node remove` commands to `subnet` section.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-12-01 17:31:04 +03:00