TrueCloudLab/frostfs-api
27
0
Fork 18
Code Issues 13 Pull requests 1 Releases Activity Actions

[#26] netmap: Add NOT and UNIQUE keywords #27

Merged
fyrchik merged 1 commit from aarifullin/frostfs-api:feature/26-add_keywords into master 2023-05-31 10:02:48 +00:00
Conversation 1 Commits 1 Files changed 1 +6
aarifullin commented 2023-05-23 15:49:12 +00:00
Member
Copy link

Signed-off-by: Airat Arifullin a.arifullin@yadro.com

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
aarifullin added the
enhancement
 label 2023-05-23 15:49:12 +00:00
aarifullin force-pushed feature/26-add_keywords from e8c56d3f3a to 37aaa7476b 2023-05-23 15:50:21 +00:00 Compare
requested reviews from storage-core-developers, storage-core-committers 2023-05-23 15:57:36 +00:00
acid-ant approved these changes 2023-05-24 06:11:47 +00:00
dstepanov-yadro requested changes 2023-05-24 06:44:03 +00:00
netmap/types.proto Outdated
@ -106,6 +109,9 @@ message Replica {
// container's objects. The format is simple enough to transpile from different
// storage policy definition languages.
message PlacementPolicy {
// Unique flag defines consecutive application for replicas
dstepanov-yadro commented 2023-05-24 06:44:01 +00:00
Member
Copy link

I think that this field should be placed after filters, in the order of values (1,2,3,4,5).

I think that this field should be placed after `filters`, in the order of values (1,2,3,4,5).
👍 1
aarifullin commented 2023-05-24 09:18:00 +00:00
Author
Member
Copy link

Fixed

Fixed
aarifullin commented 2023-05-24 09:53:15 +00:00
Author
Member
Copy link

But actually no strict order is required.
Once I checked proto in chromium repo and new added fields that were wished to added at the top had the biggest value. So, this is fine practice

But actually no strict order is required. Once I checked `proto` in chromium repo and new added fields that were wished to added at the top had the biggest value. So, this is fine practice
😕 1
dstepanov-yadro commented 2023-05-24 13:58:10 +00:00
Member
Copy link

Hm, maybe you missed commit, but i don't see any fix...

Hm, maybe you missed commit, but i don't see any fix...
aarifullin commented 2023-05-25 09:31:40 +00:00
Author
Member
Copy link

Oh, sorry. I've pushed!

Oh, sorry. I've pushed!
dstepanov-yadro marked this conversation as resolved
aarifullin changed title from [#26] netmap: Add NOT and UNIQUE keywords to WIP: [#26] netmap: Add NOT and UNIQUE keywords 2023-05-24 10:18:06 +00:00
aarifullin commented 2023-05-24 10:18:13 +00:00
Author
Member
Copy link

TrueCloudLab/frostfs-sdk-go#78 (comment)

https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/pulls/78#issuecomment-10825
aarifullin changed title from WIP: [#26] netmap: Add NOT and UNIQUE keywords to [#26] netmap: Add NOT and UNIQUE keywords 2023-05-24 13:01:08 +00:00
requested review from dstepanov-yadro 2023-05-24 13:01:11 +00:00
requested review from realloc 2023-05-24 13:35:06 +00:00
aarifullin force-pushed feature/26-add_keywords from 37aaa7476b to 0cd75cdd7e 2023-05-25 09:31:22 +00:00 Compare
fyrchik requested changes 2023-05-25 10:46:31 +00:00
netmap/types.proto Outdated
@ -35,1 +35,4 @@
AND = 8;
// Logical negation
NOT = 9;
fyrchik commented 2023-05-25 10:46:29 +00:00
Owner
Copy link

In the task we have mentioned that we would like a new field UnaryOperation added to the filter.
This way we can have NOT (A EQ B) as one filter.

In the task we have mentioned that we would like a new field `UnaryOperation` added to the filter. This way we can have `NOT (A EQ B)` as one filter.
aarifullin commented 2023-05-26 14:04:40 +00:00
Author
Member
Copy link

There are few problems to discuss

  1. If we add new proto structrure UnaryOperation then there are two options:
    1.1. Embed it to Filter like this:
message Filter {
 string name = 1 [json_name = "name"];

 // Key to filter
 string key = 2 [json_name = "key"];

 // Filtering operation
 Operation op = 3 [json_name = "op"];

 // Value to match
 string value = 4 [json_name = "value"];

 // List of inner filters. Top level operation will be applied to the whole
 // list.
 repeated Filter filters = 5 [json_name = "filters"];
 
 UnaryOperation unary_op = 6 [json_name = "unary_op"];
}

Cons: very easy to support the backward compability
Pros: name, key, op, value fields (that are used either for filterExpr AND/OR filterExpr or for expr (like A EQ B)) are adapted for non-existent BinaryOperation structre. And also unary_op can be assigned along with op

1.2. Introduce more structrure

message Filter {
    BinaryOperation bin_op = 1 [json_name = "binary_op"];
    
    repeated Filter filters = 2 [json_name = "filters"];

    UnaryOperation unary_op = 3 [json_name = "unary_op"]; // and so on
}

Cons: the structures are used how they should be. Everything is explicit
Pros: Requires total refactoring that may break the backward compability :(

  1. We can keep proto like that and change nothing. @fyrchik suggested interesting idea to "not know" NOT in proto and use one trick in the parser: negate operation where NOT is in the prefix:

NOT (A EQ B) AND (C GT D) = (A NE B) OR (C LE D) but this exclusive case only for NOT

There are few problems to discuss 1. If we add new proto structrure `UnaryOperation` then there are two options: 1.1. Embed it to `Filter` like this: ``` message Filter { string name = 1 [json_name = "name"]; // Key to filter string key = 2 [json_name = "key"]; // Filtering operation Operation op = 3 [json_name = "op"]; // Value to match string value = 4 [json_name = "value"]; // List of inner filters. Top level operation will be applied to the whole // list. repeated Filter filters = 5 [json_name = "filters"]; UnaryOperation unary_op = 6 [json_name = "unary_op"]; } ``` Cons: very easy to support the backward compability Pros: `name`, `key`, `op`, `value` fields (that are used either for `filterExpr AND/OR filterExpr` or for `expr` (like `A EQ B`)) are adapted for non-existent `BinaryOperation` structre. And also `unary_op` can be assigned along with `op` 1.2. Introduce more structrure ``` message Filter { BinaryOperation bin_op = 1 [json_name = "binary_op"]; repeated Filter filters = 2 [json_name = "filters"]; UnaryOperation unary_op = 3 [json_name = "unary_op"]; // and so on } ``` Cons: the structures are used how they should be. Everything is explicit Pros: Requires total refactoring that may break the backward compability :( 2. We can keep `proto` like that and change nothing. @fyrchik suggested interesting idea to "not know" `NOT` in proto and use one trick in the parser: negate operation where NOT is in the prefix: `NOT (A EQ B) AND (C GT D) = (A NE B) OR (C LE D)` but this exclusive case only for NOT
fyrchik marked this conversation as resolved
dstepanov-yadro approved these changes 2023-05-25 13:05:14 +00:00
fyrchik reviewed 2023-05-31 08:34:24 +00:00
netmap/types.proto Outdated
@ -120,2 +123,4 @@
// List of named filters to reference in selectors
repeated Filter filters = 4 [json_name = "filters"];
// Unique flag defines consecutive application for replicas
fyrchik commented 2023-05-31 08:34:24 +00:00
Owner
Copy link

It is not necessarily consecutive. Can we make use the word non-overlapping somewhere in this doc?

It is not necessarily consecutive. Can we make use the word `non-overlapping` somewhere in this doc?
fyrchik marked this conversation as resolved
aarifullin force-pushed feature/26-add_keywords from 0cd75cdd7e to dbfa9c944b 2023-05-31 08:57:58 +00:00 Compare
requested reviews from fyrchik, acid-ant 2023-05-31 09:13:49 +00:00
fyrchik approved these changes 2023-05-31 10:02:44 +00:00
fyrchik merged commit dbfa9c944b into master 2023-05-31 10:02:48 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
realloc
dstepanov-yadro
acid-ant
fyrchik
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
good first issue

Good for newcomers

  
triage

Needs to be triaged

  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-api#27
No description provided.
Delete branch "aarifullin/frostfs-api:feature/26-add_keywords"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?