Commit graph

903 commits

Author SHA1 Message Date
Leonard Lyubich
80ef4492c1 [] node: Get rid of outdated object GC worker
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
8d17dab86e [] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
db2e43387b [] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
2a970b0125 [] logger: Do not set global fields in constructor
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
d17526f8ac [] logger: Abolish non-usable options
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.

Remove default values of no longer existing options in node/ir config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
d49bd4b94a Update neofs-api-go to v1.26.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-10 09:45:11 +03:00
Pavel Karpy
7319ca5a00 [] morph/client: Add more global config value getters
Including:
- typo fix for `StringFromStackItem` error msg
- EigenTrust alpha getter.
- renaming local var in reading uin64 values
from global cfg function

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-06 13:52:43 +03:00
Alex Vanin
dddbf0368c [] innerring: Add option to disable only main chain notary support
For N3 Testnet RC2 release inner ring app supports three modes:
- notary enabled in all chains (default),
- notary disabled in all chains,
- notary enabled only in side chain.

All notary related functions are moved to notary.go

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
94a1947482 Fix typo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
5a167f3991 [] innerring: Adopt disabled notary work flow
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
f2562e8c47 [] innerring: Use fee provider and notary disabled flag in processors
Processors that use `invoke` package to make chain invocation should provide
fee config and client with enabled or disabled notary support. If notary
support is disabled, then functions from `invoke` package will perform
ordinary method invocation with extra fee.

Processors that use `morph/client` wrappers should check `notaryDisabled`
flag to call corresponding wrapper function.

Netmap processor omits some actions during validator syncronization
if notary is disabled.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
91a1896b8b [] innerring: Use fee provider interface in invoke package
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
1f3bb33db8 [] innerring: Add fee configuration
When notary disabled, inner ring should be able to
configure extra fee for vote collections inside the
contracts. Previously these values were hardcoded,
however we might want to change them depending on
a environment.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
c4f4381b13 [] morph/client: Separate container estimation functions
Inner ring should be able to invoke contract methods both notary
and non notary way.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
7c559aaeaa [] morph/client: Add notary enabled check function
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
fd16b5701f [] morph/client: Add more global config getters
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
bd65e41257 [] innerring: Add notary support in main chain client
With `mainnet.notary_deposit=false` inner ring will ignore
notary deposit \ awaiting routines in the application start,
so it can run on the environments without notary support.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
dd1ace12f7 [] morph/client: Add option to setup custom alphabet source in notary
To enable notary support in main chain, notary subsystem should not get
alphabet keys from (main chain) committee. This key fetcher is now
separated and may be overwritten by `WithAlphabetSource` option.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
c84fe1360e [] morph/client: Remove unused fields in notary
With neo-go v0.94.1 verification fee can be calculated
precisely and alphabet keys are fetched from committee
instead of network map contract.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Pavel Karpy
20b7295087 [] reputation/router: Add commentaries and TODO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d1db54acf8 [] reputation: Change Writer interface
Includes:
- Delete first `ctx` argument in `Write` method.
- Move intermediate Initial trust struct and method
to `calculator` file.
- Change Alpha to 0.1.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
eb74a9cafc [] reputation: Add commentaries
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
2c8c9f69c8 [] storage/calculator: Make alpha dynamic
Delete reading `alpha` from env var. Cover
retrieving `alpha` behind interface in
intermediate calculator. Add TODO to decide
if it is necessary to receive that param from
global config or not.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ea781664cf [] reputation/eigentrust/calculator: Implement calc wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d3c1fc7dda [] reputation/local/storage: Add trusting peer
Fix iteration logic after adding trusting peer
in `Trust` struct.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e69917b27a [] reputation/common: Move ServerInfo to common pkg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
49d477f466 [] reputation/managers: Implement route builders
Add implementation of Builder interface for
intermediate trusts. Move all code associated
with managers to `common` directory in `cmd`
and `pkg/services/reputation`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e2a1b0e0ee [] reputation/eigentrust/storages: Fix args
Change anonymous func arg for `Iterate`
methods of Storages to `PeerTrustsHandler`
type for implementing corresponding
interface.
Implement missing `Iterate` method for
daughter Storage.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f6783f4f81 [] cmd/reputation: Add DaughterStorage
Add `DaughterStorage` init in main pkg
and start write all received daughters'
trusts to it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Leonard Lyubich
56b3e35779 [] ir: Do not sync alphabet keys in sidechain-only configuration
Alphabet nodes synchronize list of alphabet keys from main chain
`NeoFSAlphabet` role once per epoch. This can lead to a wrong behavior in
single chain deployment (`without_mainnet` config parameter). Alphabet node
in single chain environment will try to get NeoFSAlphabet role from main
chain client, but it'll get result from side chain instead. Side chain
stores list of all inner ring nodes in this role. Therefore it is possible
that alphabet nodes will try to appoint inner ring nodes as alphabet nodes,
which is not correct.

Fix incorrect behavior with disabling of synchronization of alphabet keys in
sidechain-only mode.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 14:50:18 +03:00
Leonard Lyubich
969af596b4 [] object/search: Remove no longer used unexported methods
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 13:56:55 +03:00
Leonard Lyubich
7366d62e41 [] object/range: Fix payload slice allocation in re-sign callback
Allocate capacity instead of length of the slice to write the object payload
range since each chunk is written through `append`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 13:56:55 +03:00
Leonard Lyubich
36088949fc [] services/object: Re-sign original read requests during forwarding
In previous implementation node's Object Get/Head/GetRange V2 service
handlers created a new request for each RPC. Now original requests are
re-signed according to API specification. Logical handler abstracts from
this version-dependent logic through `RequestForwarder` callback.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 13:56:55 +03:00
Leonard Lyubich
e6b30aed36 [] object/search: Re-sign original requests during forwarding
In previous implementation node's Object Search V2 service handler created a
new request for each RPC. Now original requests are re-signed according to
API specification. Logical handler abstracts from this version-dependent
logic through `RequestForwarder` callback.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-04 13:56:55 +03:00
ZhangTao1596
2877a343c3 [] audit: skip unnecessary statement
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2021-05-04 12:55:01 +03:00
Evgenii Stratonikov
59de521fd1 [] blobstor: implement write-cache
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
96a8ee7c83 [] blobstor: allow to put raw objects
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
4a1ca4ecc1 [] metabase: speedup encodelist
GOB appears to be almost twice as slow as this implementation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
62b8958177 [] blobstor: check for blobovnicza without mutex first
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
828c8695c1 [] blobstor: use DB.Batch() for metabase and blobstor
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
934e394e28 [] blobstor: move fsTree to a separate package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00
Evgenii Stratonikov
5b85519f20 [] innerring: wait until notary tx persists on chain
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-27 16:57:29 +03:00
Evgenii Stratonikov
de74fcc38f [] blobstor: use simplelru.LRU cache
Evicting from cache requires closing blobovnicza which
in turn needs to lock `activeMtx`. This lock is not needed on
every addition, but our LRU library doesn't return evicted keys.
In future we may consider switching to other implementation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-21 14:41:34 +03:00
Alex Vanin
7acfc85f8a [] morph/client: Throw panic if notary methods are not initialized
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-20 09:53:44 +03:00
Pavel Karpy
477682adb7 [] reputation/router: Move router to ./common
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
ac8441b718 [] reputation/router: Make route pkg independent
Make route package independent from
controller package. Add common interfaces
to `./common` directory.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
0a16aaacb1 [] reputation/router: Use trusting peer
In route building use `trusting` peer
field of `reputation.Trust` struct
instead of `reputation.Peer`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Pavel Karpy
1c92dc2414 [] reputation: Fix typos in comments
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-04-19 11:47:04 +03:00
Alex Vanin
376bb293b4 [] morph/timer: Move block timer to morph package
Block timer is going to be reused in storage node to tick
EigenTrust calculation rounds.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-19 09:35:54 +03:00
Alex Vanin
434ecb41da [] morph/client: Try to parse integer as a byte type
Neo node can return integers values where []byte is expected.
To cover such cases, try to parse integers in `BytesFromStackItem`.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-15 17:53:08 +03:00