Configure `copies number` parameter for specific location constraints #70

New issue

Closed

opened 2023-03-30 13:53:56 +00:00 by alexvanin · 1 comment

alexvanin commented

2023-03-30 13:53:56 +00:00

Owner

S3 gateway uses location constrains to define bucket placement policy. With introduction of copies numbers, S3 gateway should be able to set copies numbers parameter for specific containers grouped by placement policy, in other words by placement policy.

Describe the solution you'd like

Add new configuration parameter that adds tuples like

placement_policy:
  copies_numbers:
    - location_constraint: two-dc
      vector:
        - 2
        - 1
        - 0
    - location_constraint: three-dc
      vector:
        - 1
        - 1
        - 2
        - 0

Gateway should put all new objects in the bucket with specified location constraint using copies number parameter from vector. If vector is invalid, ignore such copies_number parameter.

Describe alternatives you've considered

Adding placement policy hash

Location constraint may change corresponding policy and keep the same name. To avoid issues like that we can add hash of the container policy. However it is not clear how to calculate such hash and if there will be any issues with placement structure change over time.

    - location_constraint: two-dc
      placement_hash: deadbeef
      vector:
        - 2
        - 1
        - 0

## Is your feature request related to a problem? Please describe. S3 gateway uses location constrains to define bucket placement policy. With introduction of `copies numbers`, S3 gateway should be able to set `copies numbers` parameter for specific containers grouped by placement policy, in other words by placement policy. ## Describe the solution you'd like Add new configuration parameter that adds tuples like ```yaml placement_policy: copies_numbers: - location_constraint: two-dc vector: - 2 - 1 - 0 - location_constraint: three-dc vector: - 1 - 1 - 2 - 0 ``` Gateway should put all new objects in the bucket with specified location constraint using `copies number` parameter from `vector`. If vector is invalid, ignore such `copies_number` parameter. ## Describe alternatives you've considered ### Adding placement policy hash Location constraint may change corresponding policy and keep the same name. To avoid issues like that we can add hash of the container policy. However it is not clear how to calculate such hash and if there will be any issues with placement structure change over time. ```yaml - location_constraint: two-dc placement_hash: deadbeef vector: - 2 - 1 - 0 ```