Commit graph

921 commits

Author SHA1 Message Date
Evgenii Stratonikov
213bbcbf2b [#541] blobstor/fstree: fix a bug in Iterate()
Be able to recover address from the path. Also add tests.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-20 19:24:42 +03:00
Leonard Lyubich
1b51dcf8a4 [#546] engine/inhume: Fix incorrect Inhume behavior for root objects
If object to be inhumed is root we need to continue first traverse over the
shards. In case when several children are stored in different shards,
inhuming object in a single shard leads to appearance of inhumed object in
subsequent selections. Also, any object can be already inhumed, and this
case is equivalent to successful inhume.

Do not fail on `object.ErrAlreadyRemoved` error. Continue first iterating
over shards if we detected root object (`SplitInfoError`).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 17:39:38 +03:00
Leonard Lyubich
c5dae76c7d [#546] engine/inhume: Write unit tests
Write unit tests of `StorageEngine.Inhume` which assert that inhumed objects
don't appear in `Select` result.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 17:39:38 +03:00
Leonard Lyubich
d1d846cf4d [#546] engine/inhume: Fix incorrect reaction on successful traverse
In previous implementation storage engine false-negatively reacted to a
successful Inhume operation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 17:39:38 +03:00
Leonard Lyubich
b587b23e79 [#543] object/search: Fix potential NPE on request forwarder
Request forwarding callback should be called only if set since it is an
optional parameter.

Call `forwarder` function only if it is non-nil.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 11:33:24 +03:00
Leonard Lyubich
b2002ffc6d [#543] object/get: Fix potential NPE on request forwarder
Request forwarding callback should be called only if set since it is an
optional parameter. In GetRangeHash forwarder is never set.

Call `forwarder` function only if it is non-nil. Remove no longer needed
`hashOnly` option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 11:33:24 +03:00
Alex Vanin
89b147ebe7 [#521] Recompile protobuf files after transition from pkg/errors
Autogenerated files were accidentally modified. This commit returns
them back to the original.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-19 19:09:35 +03:00
Alex Vanin
b5256ccf4c [#521] Fix issues with transition from pkg/errors pkg
Wrap functions at `pkg/errors` return nil if error argument
was nil. fmt.Errorf always returns error so we need to add
missing error checks to the code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-19 19:09:35 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-19 16:42:54 +03:00
Leonard Lyubich
eb26f92678 [#505] neofsid: Implement wrapper over contract client
Implement wrapper over NeoFS ID contact's client which allows to which
allows you to conveniently interact with the contract. Implement
`AccountKeys` method for getting a list of keys by account ID.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-19 13:55:10 +03:00
Leonard Lyubich
68b469a79d [#505] neofsid: Implement contract client
Implement NeoFS ID contact's client which is responsible for collecting call
arguments and parsing stack items. Initially only key listing method is
supported.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-19 13:55:10 +03:00
Leonard Lyubich
72e2bc8fb6 [#532] object/hash: Fix NPE during request forwarding
In current implementation `Object.GetRangeHash` RPC handler forwards range
requests for payload data. Missing request forwarder of the original request
caused NPE during execution.

Do not call request forwarder if payload range hash is requested.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-18 18:05:18 +03:00
Evgenii Stratonikov
9ea8d11ec3 [#523] go.mod: update go-multiraddr library
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-18 15:50:47 +03:00
Alex Vanin
8d8d9eccbd [#522] network: Fix issue with empty endpoint in multiaddr
Before fix `:8080` host address was parsed as `/dns4/tcp/8080`
multiaddress. However such multiaddress is not correct. In this
case `dns4` section should be omitted, but it breaks `manet.DialArgs`.

To solve this issue we explicitly set 0.0.0.0 address.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Alex Vanin
16f13bc0a5 [#522] Use HostAddrString as RPC endpoint instead of IPAddrString
To enable TLS support we can't operate with IP addresses directly.
Certificates are issued with host names so it is required to
pass them into RPC client. DNS resolving should be done by transport
layer and not be a part of node. Therefore `IPAddrString` usage is
removed from code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Alex Vanin
2456873473 [#522] network: Add host address stringer
network.Address is a wrapper around multiaddress. Multiaddress
can be resolved to ip netaddr with `IPAddrString` function:
  /dns4/localhost/tcp/8080 => 127.0.0.1:8080
With `HostAddrString` multi address will be transformed to
host address:
  /dns4/localhost/tcp8080 => localhost:8080

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Pavel Karpy
9fbc6f5efe [#527] reputation: Fix EigenTrust algorithm
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-18 10:29:25 +03:00
Alex Vanin
e1e4a61ba7 [#520] Update neo-go to pre N3 testnet RC2 compatible version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-17 10:20:08 +03:00
Leonard Lyubich
80ef4492c1 [#493] node: Get rid of outdated object GC worker
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
8d17dab86e [#493] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
db2e43387b [#493] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
2a970b0125 [#493] logger: Do not set global fields in constructor
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
d17526f8ac [#493] logger: Abolish non-usable options
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.

Remove default values of no longer existing options in node/ir config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
d49bd4b94a Update neofs-api-go to v1.26.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-10 09:45:11 +03:00
Pavel Karpy
7319ca5a00 [#504] morph/client: Add more global config value getters
Including:
- typo fix for `StringFromStackItem` error msg
- EigenTrust alpha getter.
- renaming local var in reading uin64 values
from global cfg function

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-06 13:52:43 +03:00
Alex Vanin
dddbf0368c [#486] innerring: Add option to disable only main chain notary support
For N3 Testnet RC2 release inner ring app supports three modes:
- notary enabled in all chains (default),
- notary disabled in all chains,
- notary enabled only in side chain.

All notary related functions are moved to notary.go

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
94a1947482 Fix typo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
5a167f3991 [#486] innerring: Adopt disabled notary work flow
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
f2562e8c47 [#486] innerring: Use fee provider and notary disabled flag in processors
Processors that use `invoke` package to make chain invocation should provide
fee config and client with enabled or disabled notary support. If notary
support is disabled, then functions from `invoke` package will perform
ordinary method invocation with extra fee.

Processors that use `morph/client` wrappers should check `notaryDisabled`
flag to call corresponding wrapper function.

Netmap processor omits some actions during validator syncronization
if notary is disabled.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
91a1896b8b [#486] innerring: Use fee provider interface in invoke package
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
1f3bb33db8 [#486] innerring: Add fee configuration
When notary disabled, inner ring should be able to
configure extra fee for vote collections inside the
contracts. Previously these values were hardcoded,
however we might want to change them depending on
a environment.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
c4f4381b13 [#486] morph/client: Separate container estimation functions
Inner ring should be able to invoke contract methods both notary
and non notary way.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
7c559aaeaa [#486] morph/client: Add notary enabled check function
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
fd16b5701f [#486] morph/client: Add more global config getters
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
bd65e41257 [#486] innerring: Add notary support in main chain client
With `mainnet.notary_deposit=false` inner ring will ignore
notary deposit \ awaiting routines in the application start,
so it can run on the environments without notary support.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
dd1ace12f7 [#486] morph/client: Add option to setup custom alphabet source in notary
To enable notary support in main chain, notary subsystem should not get
alphabet keys from (main chain) committee. This key fetcher is now
separated and may be overwritten by `WithAlphabetSource` option.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
c84fe1360e [#486] morph/client: Remove unused fields in notary
With neo-go v0.94.1 verification fee can be calculated
precisely and alphabet keys are fetched from committee
instead of network map contract.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Pavel Karpy
20b7295087 [#488] reputation/router: Add commentaries and TODO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d1db54acf8 [#488] reputation: Change Writer interface
Includes:
- Delete first `ctx` argument in `Write` method.
- Move intermediate Initial trust struct and method
to `calculator` file.
- Change Alpha to 0.1.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
eb74a9cafc [#488] reputation: Add commentaries
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
2c8c9f69c8 [#488] storage/calculator: Make alpha dynamic
Delete reading `alpha` from env var. Cover
retrieving `alpha` behind interface in
intermediate calculator. Add TODO to decide
if it is necessary to receive that param from
global config or not.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ea781664cf [#488] reputation/eigentrust/calculator: Implement calc wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d3c1fc7dda [#488] reputation/local/storage: Add trusting peer
Fix iteration logic after adding trusting peer
in `Trust` struct.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e69917b27a [#488] reputation/common: Move ServerInfo to common pkg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
49d477f466 [#488] reputation/managers: Implement route builders
Add implementation of Builder interface for
intermediate trusts. Move all code associated
with managers to `common` directory in `cmd`
and `pkg/services/reputation`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e2a1b0e0ee [#488] reputation/eigentrust/storages: Fix args
Change anonymous func arg for `Iterate`
methods of Storages to `PeerTrustsHandler`
type for implementing corresponding
interface.
Implement missing `Iterate` method for
daughter Storage.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f6783f4f81 [#488] cmd/reputation: Add DaughterStorage
Add `DaughterStorage` init in main pkg
and start write all received daughters'
trusts to it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Leonard Lyubich
56b3e35779 [#480] ir: Do not sync alphabet keys in sidechain-only configuration
Alphabet nodes synchronize list of alphabet keys from main chain
`NeoFSAlphabet` role once per epoch. This can lead to a wrong behavior in
single chain deployment (`without_mainnet` config parameter). Alphabet node
in single chain environment will try to get NeoFSAlphabet role from main
chain client, but it'll get result from side chain instead. Side chain
stores list of all inner ring nodes in this role. Therefore it is possible
that alphabet nodes will try to appoint inner ring nodes as alphabet nodes,
which is not correct.

Fix incorrect behavior with disabling of synchronization of alphabet keys in
sidechain-only mode.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 14:50:18 +03:00
Leonard Lyubich
969af596b4 [#431] object/search: Remove no longer used unexported methods
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 13:56:55 +03:00
Leonard Lyubich
7366d62e41 [#431] object/range: Fix payload slice allocation in re-sign callback
Allocate capacity instead of length of the slice to write the object payload
range since each chunk is written through `append`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 13:56:55 +03:00