Compress tree service storage in a snapshot #122

New Issue

Open

opened 2023-03-08 23:35:05 +00:00 by snegurochka · 0 comments

snegurochka commented

2023-03-08 23:35:05 +00:00

Collaborator

Original issue: https://github.com/nspcc-dev/neofs-node/issues/1331

Operation log representation is quite costly in terms of storage. The proposal is to compress all operations which are older than 2 epochs. It is not as easy as it sounds:

Snapshot creation is a one-way process, so all nodes need to agree on what to include in it.
In case node is down, it should be able to invalidate local data and synchronize the latest snapshot + all other operations.
If less that 1/3 nodes is malicious, all other nodes should be able to agree on a snapshot. So some interaction with the chain is required.

Describe the approach in a separate document and implement it.

Original issue: https://github.com/nspcc-dev/neofs-node/issues/1331 Operation log representation is quite costly in terms of storage. The proposal is to compress all operations which are older than 2 epochs. It is not as easy as it sounds: 1. Snapshot creation is a one-way process, so all nodes need to agree on what to include in it. 2. In case node is down, it should be able to invalidate local data and synchronize the latest snapshot + all other operations. 3. If less that 1/3 nodes is malicious, all other nodes should be able to agree on a snapshot. So some interaction with the chain is required. Describe the approach in a separate document and implement it.