Remove storage groups to rework audit mechanism

alexvanin commented

2023-04-13 08:36:53 +00:00

Owner

This is an issue from the series of incompatible protocol changes. The purpose of these changes is to rework some FrostFS protocol mechanisms which were non-demanding for the users in private and public networks or they provide unnecessary complexity in the protocol and its implementation.

Storage groups are the objects that contain list of the objects and homomorphic hash of objects payload. This object is used for a part of data audit to prove if data is stored in the network correctly. Based on the audit result, storage nodes receive should receive a reward. Unfortunately this is mechanism didn't work in both public and private networks.

Private installations don't use incentive part of the FrostFS protocol. Current implement of TZHash was a bit slow for heavy data flows.

Public installations didn't adopt storage group mechanism either. Storage groups had to be created manually and didn't provide any additional guarantees for object safity. As for storage node, we had to implement basic income settlements, which basically become the only incentive mechanism.

While TZHash implementation can be improved, and overall audit scheme for separate objects is smart and effective, storage groups are not used in any case and safely can be dropped from the protocol.

Describe the solution you'd like

Remove storage group definition from the protocol, including separate object type
Remove data audit routine from inner ring implementation
Remove data audit settlements from inner ring implementation
Reconsider default basic ACL rules for inner ring requests (system group)

Additional context

Follow ups:

new FrostFS data audit process should be proposed in #20,
api-go issues #to-be-done
sdk issue #to-be-done
node issue #to-be-done

This is an issue from the series of incompatible protocol changes. The purpose of these changes is to rework some FrostFS protocol mechanisms which were non-demanding for the users in private and public networks or they provide unnecessary complexity in the protocol and its implementation. ## Is your feature request related to a problem? Please describe. Storage groups are the objects that contain list of the objects and homomorphic hash of objects payload. This object is used for a part of data audit to prove if data is stored in the network correctly. Based on the audit result, storage nodes receive should receive a reward. Unfortunately this is mechanism didn't work in both public and private networks. Private installations don't use incentive part of the FrostFS protocol. Current implement of TZHash was a bit slow for heavy data flows. Public installations didn't adopt storage group mechanism either. Storage groups had to be created manually and didn't provide any additional guarantees for object safity. As for storage node, we had to implement basic income settlements, which basically become the only incentive mechanism. While TZHash implementation can be improved, and overall audit scheme for separate objects is smart and effective, storage groups are not used in any case and safely can be dropped from the protocol. ## Describe the solution you'd like - Remove storage group definition from the protocol, including separate object type - Remove data audit routine from inner ring implementation - Remove data audit settlements from inner ring implementation - Reconsider default basic ACL rules for inner ring requests (system group) ## Additional context Follow ups: - new FrostFS data audit process should be proposed in https://git.frostfs.info/TrueCloudLab/frostfs-api/issues/20, - api-go issues #to-be-done - sdk issue #to-be-done - node issue #to-be-done

alexvanin added this to the v2.17 - Zemu Glacier milestone 2023-04-13 08:36:53 +00:00

alexvanin added the

triage

label 2023-04-13 08:36:53 +00:00

fyrchik was assigned by alexvanin

2023-04-13 08:36:53 +00:00

alexvanin commented

2023-04-13 08:38:58 +00:00

Author

Owner

Considering v2.15 release, I think it makes sense to go for v3 with those incompatible changes.

v3.0 can be compatible with v2.15 for some time, but then v3.1+ should introduce new things, e.g. new audit mechanism, which will be incompatible with v2.15.

Considering v2.15 release, I think it makes sense to go for v3 with those incompatible changes. v3.0 can be compatible with v2.15 for some time, but then v3.1+ should introduce new things, e.g. new audit mechanism, which will be incompatible with v2.15.

fyrchik referenced this issue from TrueCloudLab/frostfs-node

2023-04-14 06:16:48 +00:00

Remove storage groups and data audit #248

fyrchik referenced this issue from TrueCloudLab/frostfs-sdk-go

2023-04-14 06:20:43 +00:00

Remove storage group clients and wrappers #52

fyrchik referenced this issue from TrueCloudLab/frostfs-api-go

2023-04-14 06:29:33 +00:00

Remove storage group support #20

fyrchik referenced this issue

2023-08-17 08:05:56 +00:00

[#19] Remove storage groups #32