docs/epoch.md

@@ -0,0 +1,43 @@
+# Epoch
+
+The main purpose of the `epoch` in `frostfs` environment is to manipulate `netmap`.
+Each new epoch, `ir` service trigger revision content of the `netmap` by adding or removing nodes to or from it.
+`node` service trigger few internal processes each new epoch - for example, running GC.
+Epoch also used in an object lifecycle.
+
+At the startup, `ir` service initializes an epoch timer which handles new epoch tick.
+Epoch timer is a block timer - which means that this timer ticks each block or set of blocks.
+The epoch duration stores in the configurable parameter `EpochDuration` in the blockchain.
+It is possible to get it via `frostfs-adm`:
+```shell
+> frostfs-adm morph dump-config -c config.yml -r http://morph-chain.frostfs.devenv:30333
+...
+EpochDuration:               240 (int)
+...
+>
+```
+Once epoch timer ticks, `ir` service call method [NewEpoch](https://git.frostfs.info/TrueCloudLab/frostfs-contract/src/commit/a1b61d3949581f4d65b0d32a33d98ba9c193dc2a/netmap/netmap_contract.go#L238)
+of the `netmap` contract. Each `ir` instance can do this at the same time, but it is not an issue,
+because multiple call of this method with the same set of parameters will give us the same result.
+
+Utility `frostfs-adm` have a command to trigger new epoch:
+```shell
+> frostfs-adm morph force-new-epoch -c config.yml -r http://morph-chain.frostfs.devenv:30333
+```
+Command goes directly to the `netmap` contract and call method `NewEpoch`.
+Method checks alphabet witness and stores candidates nodes which are not in the `OFFLINE` state as a current netmap.
+Then executes method `NewEpoch` in `balance` and `container` contracts.
+At the end it produces notification `NewEpoch` which is handled by `node` and `ir` services.
+
+`ir` handler for `NewEpoch` updates internal state of the netmap, if it is necessary, updates state of the nodes or
+marks for exclusion from netmap in the blockchain.
+
+`node` handler for `NewEpoch` executes method `addPeer` of the `netmap` contract.
+This method do nothing, but produces notification which handled by `ir` service.
+`ir` in handler for `AddPeer` may update node state in the netmap if it is necessary.
+
+At the startup, node bootstraps with state `ONLINE`. From the online state, it is possible to move to `MAINTENANCE` or `OFFLINE`.
+Node moved to `OFFLINE` state automatically, when there is no bootstrap request from it for a number of epochs.
+This number stored in the `ir` config `netmap_cleaner.threshold`.
+From `OFFLINE` state node, once it bootstrapped, moves to `ONLINE`.
+`MAINTENANCE` state persists even if node rebooted or unavailable for a few epochs.

pkg/innerring/processors/netmap/handlers_test.go

@@ -144,6 +144,21 @@ func TestAddPeer(t *testing.T) {
 		time.Sleep(10 * time.Millisecond)
 	}
 
+	require.Nil(t, nc.notaryInvokes, "invalid notary invokes")
+
+	node.SetOnline()
+	ev = netmapEvent.AddPeer{
+		NodeBytes: node.Marshal(),
+		Request: &payload.P2PNotaryRequest{
+			MainTransaction: &transaction.Transaction{},
+		},
+	}
+	proc.handleAddPeer(ev)
+
+	for proc.pool.Running() > 0 {
+		time.Sleep(10 * time.Millisecond)
+	}
+
 	require.EqualValues(t, []notaryInvoke{
 		{
 			contract: nc.contractAddress,

pkg/innerring/processors/netmap/process_peers.go

@@ -57,7 +57,12 @@ func (np *Processor) processAddPeer(ev netmapEvent.AddPeer) bool {
 
 	updated := np.netmapSnapshot.touch(keyString, np.epochState.EpochCounter(), nodeInfoBinary)
 
-	if updated {
+	// `processAddPeer` reacts on `AddPeer` notification, `processNewEpoch` - on `NewEpoch`.
+	// This two notification produces in order - `NewEpoch` -> `AddPeer`.
+	// But there is no guarantee that code will be executed in the same order.
+	// That is why we need to perform `addPeerIR` only in case when node is online,
+	// because in scope of this method, contract set state `ONLINE` for the node.
+	if updated && nodeInfo.IsOnline() {
 		np.log.Info(logs.NetmapApprovingNetworkMapCandidate,
 			zap.String("key", keyString))