Commit graph

578 commits

Author SHA1 Message Date
Leonard Lyubich
287683c590 [#863] node: Do not open main chain client connection
In current implementation storage node doesn't use main chain client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-29 12:55:48 +03:00
Leonard Lyubich
50c4c5116d [#863] node: Prevent panic about missing Neo RPC endpoints
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-29 12:55:48 +03:00
Alex Vanin
409b72cb5b [#786] neofs-node: Ignore proxy contract in notary disabled environment
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-27 16:20:31 +03:00
Leonard Lyubich
0e3e8db5c0 [#845] node: Configure Object PUT remote and local pools separately
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-27 16:20:19 +03:00
Leonard Lyubich
ee20200c2e [#845] object/put: Separate pools for local and remote operations
In previous implementation Object PUT used single pool of workers for local
and remote ops, but these ops are heterogeneous.

Use remote/local pool for remote/local operations in PUT service. At first
the pools are configured with the same size.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-27 16:20:19 +03:00
Alex Vanin
3e5c7e0ade [#836] neofs-adm: Fix proxy contract deploy arguments
Proxy contract does not include notary flag because
contract is useless without notary subsystem.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-23 09:55:46 +03:00
Alex Vanin
4c30757439 [#836] neofs-adm: Do not define contract owner
Contract owners are removed in neofs-contract v0.11.0.
Now side chain committee has rights to update contracts.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-23 09:55:46 +03:00
Alex Vanin
ce8a906bb5 [#834] neofs-adm: Update NNS contract during contract update
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Alex Vanin
d996004d80 [#834] neofs-adm: Use committee signature when updating alphabet contract
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Alex Vanin
6b3a9e6fcc [#834] neofs-adm: Update contract update if it exists
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Alex Vanin
60636d4c1d [#834] neofs-adm: Throw error if deploy produce non-HALT state
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-22 18:23:59 +03:00
Leonard Lyubich
d938c7267b [#789] node/config: Add Shard's refill_metabase config to examples
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-15 18:22:09 +03:00
Leonard Lyubich
b5fff810f4 [#789] cmd/node: Add refill_metabase config to shard section
Implement `RefillMetabase` method to `shardconfig.Config` type which reads
`refill_metabase` config value. Pass the result to `WithRefillMetabase`
option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-15 18:22:09 +03:00
Leonard Lyubich
368e280413 [#776] cmd/node: Configure write-cache size limit
Add `size_limit` config value to write-cache section. Add `SizeLimit` method
of `writecache.Config` type. Pass its value to `WithMaxCacheSize` option on
app construction.

Reflect config update in examples. Cover new value in unit test.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-15 18:07:36 +03:00
Alex Vanin
d840627816 [#786] cmd/neofs-node: Use NNS to find contract script hashes missing in config
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:51:35 +03:00
Alex Vanin
affae68061 [#812] cmd/neofs-node: Support notary notifications in reputation service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:51:07 +03:00
Alex Vanin
90823a7d05 [#812] cmd/neofs-node: Fix reputation wrapper import typo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:51:07 +03:00
Pavel Karpy
288a8784d0 [#815] node: Make notary deposit before bootstrap
Bootstrap process involves `addPeer` contract
call => it is necessary to make deposit before
booting up.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-13 14:50:52 +03:00
Pavel Karpy
b303e49408 [#815] morph/event/netmap: Add updateState notary notification support
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-13 14:50:52 +03:00
Alex Vanin
01df4ffa61 [#811] service/container: Hide cache invalidation logic in Writer interface
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:50:18 +03:00
Alex Vanin
49b6b5b49d [#811] cmd/node: Use cache invalidator in container service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-13 14:50:18 +03:00
Pavel Karpy
19e97e4d7c [#807] node: Fix notary container invocation
Create non-notary container wrapper and
use it in load announcing since it shouldn't
be invoked as notary request.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-09 18:37:45 +03:00
Alex Vanin
1edf40f4d6 [#798] morph/subscriber: Remove RPC Init timeout
More convenient way is to fail straight away and
expect external restart.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-07 17:22:25 +03:00
Alex Vanin
4874b4ae92 [#798] cmd/neofs-node: Save latest processed block number
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-07 17:22:25 +03:00
Alex Vanin
cdb3b71070 [#798] neofs-node/config: Add persistent_state section
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-07 17:22:25 +03:00
Alex Vanin
005f54e61e [#798] pkg/innerring: Save latest processed block number
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-07 17:22:25 +03:00
Pavel Karpy
4a81781c0c [#770] logger: Delete spaces in log message keys
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
2f343a15e5 [#770] node/cmd: Add TryNotary to SN container wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
722b844aa2 [#770] node: Add notary deposit timer
Storage Node needs to have notary deposit
for successful notary request sending.
Add notary deposit on startup(and wait for
its acceptance). Add notary deposit timer,
its config in `morph` section and env vars
for its tuning.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
e29bcd98e2 [#770] node/config: Add notary deposit config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
2ef5e86aaf [#770] node/config: Add uint32 casting
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
d77b2d1b76 [#770] node/config: Add proxy contract to config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Pavel Karpy
d252aa4a3e [#770] pkg/morph: Rename all parsers and handlers structs/interfaces
Prepare all listening structures for notary events:
rename(add prefix/suffix 'notification') all
notification specific handlers/parsers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 12:55:01 +03:00
Leonard Lyubich
358e3ed8c4 [#645] *: Change the locality condition of the node from the placement
Some software components regulate the way of working with placement arrays
when a local node enters it. In the previous implementation, the locality
criterion was the correspondence between the announced network address
(group) and the address with which the node was configured. However, by
design, network addresses are not unique identifiers of storage nodes in the
system.

Change comparisons by network addresses to comparisons by keys in all
packages with the logic described above. Implement `netmap.AnnouncedKeys`
interface on `cfg` type in the storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-07 09:53:18 +03:00
Pavel Karpy
3c848b2cad [#788] cli: Wrap/sync errors
Add context to error messages. Sync error
messages for errors with the same context.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-09-07 09:52:53 +03:00
Leonard Lyubich
85bd2a1cdf [#746] morph: Add error return of MagicNumber method
Since morph `Client` works in multi-client mode, there is an error case when
we can not get network magic when all endpoints are unavailable.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
ad7ad12a0c [#746] morph: Implement and use multi-client
There is a need to work with a set of Neo RPC nodes in order not to depend
on the failure of some nodes while others are active.

Support "multi-client" mode of morph `Client` entity. If instance is not
"multi-client", it works as before. Constructor `New` creates multi-client,
and each method performs iterating over the fixed set of endpoints until
success. Opened client connections are cached (without eviction for now).

Storage (as earlier) and IR (from now) nodes can be configured with multiple
Neo endpoints. As above, `New` creates multi-client instance, so we don't
need initialization changes on app-side.

`Wait` and `GetDesignateHash` methods of `Client` return an error from now
to detect connection errors. `NotaryEnabled` method is removed as unused.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
e738699fcc [#676] services/container: Cache the results of read operations
In previous implementation Container service handlers didn't cache the
results of `Get` / `GetEACL` / `List` operations. As a consequence of this,
high load on the service caused neo-go client's connection errors. To avoid
this there is a need to use cache. Object service already uses `Get` and
`GetEACL` caches.

Implement cache of `List` results. Share already implemented cache of Object
service with the Container one. Provide new instance of read-only container
storage (defined as an interface)to morph executor's constructor on which
container service is based. Write operations remained unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-30 18:01:26 +03:00
Pavel Karpy
c54f524df9 [#773] writecache: Delete unused dbSize param
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-27 15:05:53 +03:00
Pavel Karpy
9b32b5523d [#760] cli: Support COMMON_PREFIX matchtype
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-27 15:03:15 +03:00
Leonard Lyubich
7a5729ea2b [#761] cmd/node: Do not perform bootstrap procedure in relay mode
Storage node should not try to register itself in network in relay mode.

Implement `needBootstrap` method which checks if node need to bootstrap.
Call `bootstrap` method in `bootstrapNode` function only on true return.
Skip re-bootstrap logic in new epoch event handler on false return.
Return an error if `ControlService.SetNetmapStatus` is called on relay
node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-26 18:54:55 +03:00
Leonard Lyubich
05d5b724a9 [#761] cmd/node: Rename reBootstrapEnabled to needBootstrap
`reBootstrapEnabled` state var is not used. It is going to be used to decide
whether to bootstrap node or not.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-26 18:54:55 +03:00
Leonard Lyubich
3c78890b97 [#761] cmd/node: Add bootstrap method docs
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-26 18:54:55 +03:00
Alex Vanin
cadd94f08f [#766] Fix stylecheck import linter error
Remove redundant imports

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-26 17:57:15 +03:00
Alex Vanin
53f031e98c [#766] Fix misspell linter error
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-26 17:57:15 +03:00
Leonard Lyubich
4b7cc6e293 [#758] cmd/neofs-adm: Implement command to refill storage node's GAS
Add `refill-gas` sub-command to `morph` command which provides the ability
to refill storage node's GAS.

Command performs some actions from `generate-storage-wallet` runner, so
common code is moved to a separate function `refillGas`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-25 17:07:23 +03:00
Evgenii Stratonikov
f9d9f33461 neofs-adm: use nnsResolveHash instead of custom code
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-08-24 12:35:52 +03:00
Evgenii Stratonikov
72b8d919fe [#755] neofs-adm: allow to filter containers by ID
`--cid <cid1> --cid <cid2>` as well as `--cid <cid1>,<cid2>` is supported.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-08-24 12:35:52 +03:00
Evgenii Stratonikov
00a299c1a4 [#755] neofs-adm: add contract hash flag to dump-containers
`--container-contract` flag must be used for deployments without NNS.
Our current testnet sidechain is like this, for example.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-08-24 12:35:52 +03:00
Evgenii Stratonikov
5072b703bc [#755] neofs-adm: allow to restore containers
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-08-24 12:35:52 +03:00