Store EC chunk's parent's parent's attributes #76

New issue

Closed

opened 2025-01-09 09:27:42 +00:00 by a-savchuk · 1 comment

a-savchuk commented

2025-01-09 09:27:42 +00:00

Member

In FrostFS we can:

Split a big object into parts: Object -> Parts
Split an object into chunks: Object -> Chunks
And both: Object -> Parts -> Chunks

And object's attributes are propagated in the following way:

Object (attributes) -> Parts (no attributes, but the first part has the parent's meta)
Object (attributes) -> Chunks (attributes)
Object (attributes) -> Parts (no attributes, but the first part has the parent's meta) -> Chunks (no attributes)

As a result, in a FrostFS node, there's no way to determine the expiration epoch of an expirable regular object. Based on my experience, expirable regular object are rarely used, so this issue hasn't been encountered earlier.

Describe the solution you'd like

I suggest introducing a new filed in the EC header:

message Header {
  // ...
  message EC {
    // ...
    // EC parent's parent's attributes.
    repeated Attribute parent_parent_attributes = 9 [ json_name = "parentParentAttributes" ];
  }
}

Describe alternatives you've considered

I considered keeping the current message structure and storing attributes in the following way:

If a chunk's parent has no parent, store chunk's parent's attributes
If a chunk's parent has a parent, store chunk's parent's parent's attributes

However, in this case, the restored object will differ from the original one. Please correct me if I'm wrong.

Additional context

Store parent's attributes when splitting a big object into parts:

 			s.parent = fromObject(s.current)
 			s.parent.ResetRelations()
 			s.parent.SetSignature(nil)
 			s.parent.SetAttributes(s.parAttrs...)
 			s.parent.SetVersion(&ver)

Store parent's attributes when splitting an object into chunks:

 		ecParentInfo := objectSDK.ECParentInfo{
 			ID:            parent,
 			SplitID:       obj.SplitID(),
 			SplitParentID: parentSplitParentID,
 			Attributes:    obj.Attributes(),
 		}

## Is your feature request related to a problem? Please describe.  In FrostFS we can: - Split a big object into parts: Object -> Parts - Split an object into chunks: Object -> Chunks - And both: Object -> Parts -> Chunks And object's attributes are propagated in the following way: - Object (attributes) -> Parts (no attributes, but the first part has the parent's meta) - Object (attributes) -> Chunks (attributes) - Object (attributes) -> Parts (no attributes, but the first part has the parent's meta) -> Chunks (no attributes) As a result, in a FrostFS node, there's no way to determine the expiration epoch of an expirable regular object. Based on my experience, expirable regular object are rarely used, so this issue hasn't been encountered earlier. ## Describe the solution you'd like  I suggest introducing a new filed in the [EC header](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/e33194282c0035bed8fcc652175f5a3dabd902a8/object/types.proto#L200-L223): ```proto message Header { // ... message EC { // ... // EC parent's parent's attributes. repeated Attribute parent_parent_attributes = 9 [ json_name = "parentParentAttributes" ]; } } ``` ## Describe alternatives you've considered  I considered keeping the current message structure and storing attributes in the following way: - If a chunk's parent has no parent, store chunk's parent's attributes - If a chunk's parent has a parent, store chunk's parent's parent's attributes However, in this case, the restored object will differ from the original one. Please correct me if I'm wrong. ## Additional context  Store parent's attributes when splitting a big object into parts: https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/src/commit/42a0fc8c13aefb79bcc273be2fb2a8ae9eaf17b1/object/transformer/transformer.go#L94-L98 Store parent's attributes when splitting an object into chunks: https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/src/commit/42a0fc8c13aefb79bcc273be2fb2a8ae9eaf17b1/object/erasurecode/split.go#L44-L49

a-savchuk added the

discussion

triage

labels 2025-01-09 09:27:42 +00:00

a-savchuk commented

2025-01-09 10:04:52 +00:00

Author

Member

Discussed with @dstepanov-yadro

The alternative solution doesn't change reconstructed object as I thought before. I will adopt the alternative solution, which will require changes only to the SDK.

Discussed with @dstepanov-yadro The alternative solution doesn't change reconstructed object as I thought before. I will adopt the alternative solution, which will require changes only to the SDK.

a-savchuk closed this issue

2025-01-09 10:04:53 +00:00