ale64bit/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
988 commits 59 branches 26 tags 20 MiB
Commit graph

144 commits

Author SHA1 Message Date
Leonard Lyubich
998a097767 [#324] ir: Listen new blocks from main chain 
Listen to new blocks from mainnet until the required fix in neo-go lib is
released (https://github.com/nspcc-dev/neo-go/pull/1687).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
abc7885056 [#324] ir: Register block handler before starting the listening routine 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
96f750d9c8 [#324] ir/alphabet: Write error of TransferGas call to log 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
a6517bc877 [#324] ir/netmap: Do not return on block timer restart error 
Continue execution of the new epoch handler in case of a failed timer
restart.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
ca41810859 [#324] ir: Change level of "new block" log message to debug 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
d01b4e1a2d [#324] ir: Measure GAS emission intervals in sidechain blocks 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
19bb94cc04 [#324] ir: Measure epochs in sidechain blocks 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
b5dc28f79c [#324] ir/timers: Complicate the logic of fractional block intervals 
Call handler of the fractional block interval once between base interval
ticks by default. Add option to call handler of fractional block interval
multiple times (N times if fractional interval == BASE_INTERVAL / N).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Leonard Lyubich
476349dd1c [#324] ir/timers: Implement chain block timer 
Implement a timer that can tick chain blocks and perform actions at time
intervals in the blocks.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-01-29 11:04:30 +03:00
Alex Vanin
5d7451d388 [#349] innerring/audit: Use pivot in container placement 
Pivot used to shuffle nodes in the CRUSH tree. This is
required argument. We use container ID value to select
container nodes, so `nil` value produces incorrect placements.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-01-27 11:18:36 +03:00
Alex Vanin
0745267411 [#47] innerring: Increase minimal extra fee on contract invocations 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-01-20 16:36:10 +03:00
Alex Vanin
8865252daf [#325] innerring: Fix storage node re-bootstrap approval 
Netmap snapshot table caches bootstrap `AddPeer` txs so inner ring
node does not produce redundant approval txs for bootstrapped nodes.
However if node updates states to `Offline`, then such node should
be flagged in snapshot table, so re-bootstrap will actually produce
approval tx.

`ev.PublicKey.String()` returns uncompressed representation of the
node's public key, while snapshot contains compressed values.
Therefore the node was not flagged and re-bootstrap tx was not
approved by inner ring nodes.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-01-18 18:54:07 +03:00
Alex Vanin
b62b19cb5a [#302] Fix goimports linter errors 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-01-12 18:47:02 +03:00
Alex Vanin
3bbd0bc95c [#300] innerring: Fix status value in update peer state invocation 
We can't use enum values from SDK library directly, they can be
different from API specification. Therefore we need to convert them
into protocol level format.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-01-11 16:40:25 +03:00
Leonard Lyubich
9a86fff7e0 [#291] Remove some unused code from repository 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-30 10:27:53 +03:00
Alex Vanin
7174abcc7c [#281] service/audit: Fix typos 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
77cf97c236 [#281] innerring: Create audit context once for one audit round 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
0d7832f5e9 [#281] service/audit: Add separate pool for SG checks in PoR 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
10b548275a [#276] innerring: Use Head with TTL in PoR 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
f9e81383ae [#259] innerring: Use blocking pools for audit 
Audit task manager should not discard tasks if all workers are
busy, therefore pools should not be non-blocking.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
3ec342b2c3 [#259] services/audit: Implement PDP check 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
310a4c3e4d [#275] innerring: Add storage group search timeout 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
6836975272 [#275] innerring: Use crypto rand shuffle in audit 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
f593f0b2d6 [#274] innerring: Implement GetRangeHash in audit communicator 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
a5320408a5 [#271] service/audit: Implement PoR 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
782bcadd92 [#271] innerring: Implement GetSG and Head of audit communicator 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
dd5e67ff0c [#271] service/audit: Add netmap structure to audit task 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
a8c26378b1 [#271] pkg/network: Add multiaddr to ipaddr converter 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
2ebcbe70fb [#255] ir/audit: Make task manager to return number of skipped tasks 
Add numeric return from TaskManager.Reset method that shows the number of
canceled tasks. This values will be used for assessment of the progress of
the audit.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
03e3afb0e8 [#255] services/audit: Define interface of container communicator 
Define interface of the container communicator which methods are going to be
used in audit checks. Make innerring Server to implement this interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
754cd8b579 [#255] ir: Replace Server.WriteReport method to another source file 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
54523d2949 [#255] ir: Replace audit client creation to invoke package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
76d4e53ea0 [#255] services/audit: Skip all tasks from previous epoch in audit processor 
Implement Reset method on audit task manager that cleans task queue.
Extended TaskManager interface with Reset method on IR side. Call Reset
method in audit processor before new audit start.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Leonard Lyubich
4dc09b19f3 [#255] ir: Make audit processor to push tasks to audit task manager 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
8dd7c689f2 [#265] innerring: Select storage groups to audit 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
22cffbf529 [#265] innerring: Add client cache 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
3d3d058b05 [#265] innerring: Select containers to audit 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
87e1252065 [#265] innerring: Generate new audit event 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
bb85ac24eb [#254] innerring: Increase extra fee for container registration 
Some rough estimation until there will be p2p signature collection
inside the chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
feec559e47 [#254] Update neo-go to preview4 compatible version 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-25 16:49:27 +03:00
Alex Vanin
e2547f5bf8 [#170] Invoke all alphabet contracts for voting 
Now alphabet contracts use signature collection to make
a vote. All inner ring nodes votes for some list of validators
the same way as they vote fore new epoch or new container.
As soon as list is accepted by alphabet contracts, each of them
votes for one candidate from the list.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Alex Vanin
49666f87f1 [#170] Implement --vote argument in neofs-ir 
With this argument neofs-ir will be run as an CLI
application that initialize inner ring server, invokes
`vote` method on corresponding alphabet contract and
exits.

User can provide a comma-separated list of validator
public keys.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Alex Vanin
0b65c64e98 [#170] Vote for validators on inner ring startup 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Alex Vanin
2240cde826 [#170] Implement validator voting function in inner ring 
This function calculates validator key based on inner ring index
and invokes `Vote` method of corresponding alphabet contract.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Alex Vanin
d02c24db5c [#170] Add alphabet vote invocation method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-10 17:22:37 +03:00
Leonard Lyubich
58fcb35fb0 [#174] Use Marshal(JSON)/Unmarshal(JSON) methods for encoding/decoding 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-17 11:56:00 +03:00
Alex Vanin
b59af19dff [#171] Increase extra gas on ApprovePeer method 
With latest changes in neo's execution costs, we need more
gas to add storage node to netmap.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-13 17:12:46 +03:00
Alex Vanin
65be09d3db [#155] Update neofs-api-go with refactored pkg/netmap 
Refactored pkg/netmap package provides JSON converters for
NodeInfo and PlacementPolicy structures, that has been used
by client applications.

It also updates Node structure itself so it is a part of
grpc <-> v2 <-> pkg conversion chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-06 09:55:05 +03:00
Alex Vanin
9f8eb0fd51 [#152] Use ID in SetConfig contract method invocation. 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-06 09:26:00 +03:00
Alex Vanin
97077294fc [#141] Fix double sidechain GAS emission on asset mint 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-03 16:28:01 +03:00
Alex Vanin
0c49c08609 [#139] Use storage node GAS emission in inner ring app 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-03 16:20:41 +03:00
Alex Vanin
d193f1087c [#139] Emit gas to the storage nodes at emission tick 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-03 16:20:41 +03:00
Leonard Lyubich
a8e9d15f3f [#85] inner-ring/container: Check container format in process Put 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 14:14:38 +03:00
Alex Vanin
83119c00ec [#127] Remove magic number from config 
With neo-project/neo-modules#358 nodes support RPC to
return used magic number. Therefore client doesn't need
that configuration value any more.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 13:54:31 +03:00
Alex Vanin
c7975dbe87 [#132] Use SDK defined netmap structure in cleanup table 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
976ba06300 [#132] Use netmap cleaner in inner ring server 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
94957dd38c [#132] Generate and handle netmap cleanup event 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
5c7de27546 [#132] Add netmap cleanup event processor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
6fcaa13e0b [#132] Flag all gracefully shutdown nodes in cleaner cache 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
118f7fb309 [#132] Put new storage peers in netmap cleaner cache 
There is an optimization to send ApprovePeer tx only in
case if `touch` returns false. It returns false if node
is not in the cache or it was flagged to removal.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
0c42db2c3c [#132] Add cleanup table in netmap processor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
9b97e95f82 [#132] Add internal event to process cleanup 
This event will be produced by new epoch handler. All
cleanups should be synchronous because smart contract can't
store inner ring votes forever. Therefore voting should be done
in similar time interval for all inner ring nodes.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
dfc2e81717 [#132] Add cleanup table in inner ring netmap processor 
Cleanup table is a cache for inner ring node to look for netmap
snapshot. It updates access time of bootstrapped nodes and will
be used to send `updateState` txs to clean netmap from unresponsive
nodes.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
c80bce9438 [#132] Implement netmap snapshot getter for inner ring 
Later it should be merged with morph/client getters.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-11-02 10:52:52 +03:00
Alex Vanin
330c4b42d9 [#122] Use balance precision converter in inner ring 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-28 13:36:59 +03:00
Alex Vanin
90984ee219 [#122] Add converter interface in balance and neofs processors 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-28 13:36:59 +03:00
Alex Vanin
b6012977bc [#122] Implement balance precision getter for inner ring 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-28 13:36:59 +03:00
Alex Vanin
e6f04f7785 [#104] Update neofs-api-go with new protobuf API 
Also update contains JSON converters for neofs-cli
and fixes bug in container.set-acl command of SDK.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-20 17:31:59 +03:00
Alex Vanin
dd48666357 [#72] Shutdown inner ring app if RPC node has been terminated 
Adopt error channel from Listener interface.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-14 09:37:48 +03:00
Alex Vanin
d3d8f00757 [#72] Init inner ring configuration before run 
This way we can re-initialize application in any given
moment, which is useful for reconnects.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-14 09:37:48 +03:00
Alex Vanin
ade185191b [#47] Use less extra gas on netmap contract methods 
* 1.0 extra gas for new epoch vote
* 0.5 extra gas for new peer approve

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
b7ee05088c [#86] Use alphabet processor in inner ring app 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
f1f20b49db [#86] Add alphabet contract processor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
4308a6f522 [#86] Run timer for gas emission event 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
6bc787bb19 [#86] Add alphabet contract configuration 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
dcb384d551 [#86] Specify inner ring list index in global state 
Inner ring index will be used to access alphabet contracts.
First seven inner ring nods communicating with one alphabet
contract, depending on their index.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-12 14:43:33 +03:00
Alex Vanin
204eaab5a9 [#75] Fix order of update peer state method arguments 
This method has node status first and public key second.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-05 18:58:35 +03:00
Alex Vanin
2fc2f8792e [#48] Make IR initial config errors more readable 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:36 +03:00
Alex Vanin
be353ad69f [#48] Use more extra fee at container registration 
Large inner ring requires more gas to make container registration.
Container contract makes balance transfers for each inner ring node
and it require extra gas to execute. This estimation should be enough
for seven inner ring nodes.

Later there should be heuristic evaluations for this: #47

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:36 +03:00
Alex Vanin
f23d8a5f4a [#21] ir: Remove excess todo 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
4aee3de24e [#21] ir: Add inner ring list relay processor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
f11ae1035d [#21] ir: Add config relay processor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
be9e7664f3 [#21] ir: Provide netmap contract script hash into neofs processor 
Netmap contract script hash is used to synchronize global config
and inner ring list in both chains.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
a19e9c1f00 [#21] Add SetConfig and UpdateInnerRing method invoke helpers 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
92f448f303 [#18] Add handlers for AddPeer and UpdatePeer events 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
7bce9a3d87 [#18] Add invoke helpers for AddPeer and UpdatePeer methods 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
5e5e431534 [#7] Add container delete notification handler 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
7e099c7921 [#7] Use container processor in inner ring 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
c4cdfe3ec2 [#7] Add container processor for inner ring 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
055f7be6d1 [#7] Add new container invocation wrapper for inner ring 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
58cb90966a [#16] ir: Use mint and burn methods in balance contract 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
3d0064cb4c [#14] Use lower case in first letter of contract methods 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Alex Vanin
6520d00ff0 [#14] Adopt stackitems in inner ring 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-10-02 11:25:35 +03:00
Stanislav Bogatyrev
b7b5079934 Add Inner Ring code 2020-07-24 17:07:37 +03:00