TrueCloudLab
/
frostfs-api
25
0
Fork 11
Code Issues 11 Pull Requests 1 Releases Activity

[#26] netmap: Add NOT and UNIQUE keywords #27

Merged
fyrchik merged 1 commits from aarifullin/frostfs-api:feature/26-add_keywords into master 2023-05-31 10:02:48 +00:00
Conversation 1 Commits 1 Files Changed 1
aarifullin commented 2023-05-23 15:49:12 +00:00
Collaborator

Signed-off-by: Airat Arifullin a.arifullin@yadro.com

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
aarifullin added the
enhancement
 label 2023-05-23 15:49:12 +00:00
aarifullin force-pushed feature/26-add_keywords from e8c56d3f3a to 37aaa7476b 2023-05-23 15:50:21 +00:00 Compare
aarifullin requested review from storage-core-developers 2023-05-23 15:57:36 +00:00
aarifullin requested review from storage-core-committers 2023-05-23 15:57:36 +00:00
acid-ant approved these changes 2023-05-24 06:11:47 +00:00
dstepanov-yadro requested changes 2023-05-24 06:44:03 +00:00
netmap/types.proto Outdated
@ -106,6 +109,9 @@ message Replica {
// container's objects. The format is simple enough to transpile from different
// storage policy definition languages.
message PlacementPolicy {
// Unique flag defines consecutive application for replicas
dstepanov-yadro commented 2023-05-24 06:44:01 +00:00
Collaborator

I think that this field should be placed after filters, in the order of values (1,2,3,4,5).

I think that this field should be placed after `filters`, in the order of values (1,2,3,4,5).
👍 1
aarifullin commented 2023-05-24 09:18:00 +00:00
Poster
Collaborator

Fixed

Fixed
aarifullin commented 2023-05-24 09:53:15 +00:00
Poster
Collaborator

But actually no strict order is required.
Once I checked proto in chromium repo and new added fields that were wished to added at the top had the biggest value. So, this is fine practice

But actually no strict order is required. Once I checked `proto` in chromium repo and new added fields that were wished to added at the top had the biggest value. So, this is fine practice
😕 1
dstepanov-yadro commented 2023-05-24 13:58:10 +00:00
Collaborator

Hm, maybe you missed commit, but i don't see any fix...

Hm, maybe you missed commit, but i don't see any fix...
aarifullin commented 2023-05-25 09:31:40 +00:00
Poster
Collaborator

Oh, sorry. I've pushed!

Oh, sorry. I've pushed!
dstepanov-yadro marked this conversation as resolved
aarifullin changed title from [#26] netmap: Add NOT and UNIQUE keywords to WIP: [#26] netmap: Add NOT and UNIQUE keywords 2023-05-24 10:18:06 +00:00
aarifullin commented 2023-05-24 10:18:13 +00:00
Poster
Collaborator

TrueCloudLab/frostfs-sdk-go#78 (comment)

https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/pulls/78#issuecomment-10825
aarifullin changed title from WIP: [#26] netmap: Add NOT and UNIQUE keywords to [#26] netmap: Add NOT and UNIQUE keywords 2023-05-24 13:01:08 +00:00
aarifullin requested review from dstepanov-yadro 2023-05-24 13:01:11 +00:00
realloc requested review from realloc 2023-05-24 13:35:06 +00:00
aarifullin force-pushed feature/26-add_keywords from 37aaa7476b to 0cd75cdd7e 2023-05-25 09:31:22 +00:00 Compare
fyrchik requested changes 2023-05-25 10:46:31 +00:00
netmap/types.proto Outdated
@ -35,1 +35,4 @@
AND = 8;
// Logical negation
NOT = 9;
fyrchik commented 2023-05-25 10:46:29 +00:00
Owner

In the task we have mentioned that we would like a new field UnaryOperation added to the filter.
This way we can have NOT (A EQ B) as one filter.

In the task we have mentioned that we would like a new field `UnaryOperation` added to the filter. This way we can have `NOT (A EQ B)` as one filter.
aarifullin commented 2023-05-26 14:04:40 +00:00
Poster
Collaborator

There are few problems to discuss

  1. If we add new proto structrure UnaryOperation then there are two options:
    1.1. Embed it to Filter like this:
message Filter {
 string name = 1 [json_name = "name"];

 // Key to filter
 string key = 2 [json_name = "key"];

 // Filtering operation
 Operation op = 3 [json_name = "op"];

 // Value to match
 string value = 4 [json_name = "value"];

 // List of inner filters. Top level operation will be applied to the whole
 // list.
 repeated Filter filters = 5 [json_name = "filters"];
 
 UnaryOperation unary_op = 6 [json_name = "unary_op"];
}

Cons: very easy to support the backward compability
Pros: name, key, op, value fields (that are used either for filterExpr AND/OR filterExpr or for expr (like A EQ B)) are adapted for non-existent BinaryOperation structre. And also unary_op can be assigned along with op

1.2. Introduce more structrure

message Filter {
    BinaryOperation bin_op = 1 [json_name = "binary_op"];
    
    repeated Filter filters = 2 [json_name = "filters"];

    UnaryOperation unary_op = 3 [json_name = "unary_op"]; // and so on
}

Cons: the structures are used how they should be. Everything is explicit
Pros: Requires total refactoring that may break the backward compability :(

  1. We can keep proto like that and change nothing. @fyrchik suggested interesting idea to "not know" NOT in proto and use one trick in the parser: negate operation where NOT is in the prefix:

NOT (A EQ B) AND (C GT D) = (A NE B) OR (C LE D) but this exclusive case only for NOT

There are few problems to discuss 1. If we add new proto structrure `UnaryOperation` then there are two options: 1.1. Embed it to `Filter` like this: ``` message Filter { string name = 1 [json_name = "name"]; // Key to filter string key = 2 [json_name = "key"]; // Filtering operation Operation op = 3 [json_name = "op"]; // Value to match string value = 4 [json_name = "value"]; // List of inner filters. Top level operation will be applied to the whole // list. repeated Filter filters = 5 [json_name = "filters"]; UnaryOperation unary_op = 6 [json_name = "unary_op"]; } ``` Cons: very easy to support the backward compability Pros: `name`, `key`, `op`, `value` fields (that are used either for `filterExpr AND/OR filterExpr` or for `expr` (like `A EQ B`)) are adapted for non-existent `BinaryOperation` structre. And also `unary_op` can be assigned along with `op` 1.2. Introduce more structrure ``` message Filter { BinaryOperation bin_op = 1 [json_name = "binary_op"]; repeated Filter filters = 2 [json_name = "filters"]; UnaryOperation unary_op = 3 [json_name = "unary_op"]; // and so on } ``` Cons: the structures are used how they should be. Everything is explicit Pros: Requires total refactoring that may break the backward compability :( 2. We can keep `proto` like that and change nothing. @fyrchik suggested interesting idea to "not know" `NOT` in proto and use one trick in the parser: negate operation where NOT is in the prefix: `NOT (A EQ B) AND (C GT D) = (A NE B) OR (C LE D)` but this exclusive case only for NOT
fyrchik marked this conversation as resolved
dstepanov-yadro approved these changes 2023-05-25 13:05:14 +00:00
fyrchik reviewed 2023-05-31 08:34:24 +00:00
netmap/types.proto Outdated
@ -120,2 +123,4 @@
// List of named filters to reference in selectors
repeated Filter filters = 4 [json_name = "filters"];
// Unique flag defines consecutive application for replicas
fyrchik commented 2023-05-31 08:34:24 +00:00
Owner

It is not necessarily consecutive. Can we make use the word non-overlapping somewhere in this doc?

It is not necessarily consecutive. Can we make use the word `non-overlapping` somewhere in this doc?
fyrchik marked this conversation as resolved
aarifullin force-pushed feature/26-add_keywords from 0cd75cdd7e to dbfa9c944b 2023-05-31 08:57:58 +00:00 Compare
aarifullin requested review from fyrchik 2023-05-31 09:13:47 +00:00
aarifullin requested review from acid-ant 2023-05-31 09:13:49 +00:00
fyrchik approved these changes 2023-05-31 10:02:44 +00:00
fyrchik merged commit dbfa9c944b into master 2023-05-31 10:02:48 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
realloc
dstepanov-yadro
acid-ant
fyrchik
Labels
Clear labels   
P0

Highest   
P1

High   
P2

Medium   
P3

Low   
good first issue

Good for newcomers   
triage

Needs to be triaged
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff   
blocked

The issue or PR is blocked by something   
bug

Something is not working   
config

Configuration format update or breaking change   
discussion

Talking about something in order to reach a decision or to exchange ideas   
documentation

Documentation related   
duplicate

This issue or pull request already exists   
enhancement

New feature   
go

Language features adoption   
help wanted

Extra attention is needed   
internal
   
invalid

Something is wrong   
kludge

This is a kludge and we know it   
observability

Related to metrics, tracing and logs   
perfomance

Perfomance related   
question

More information is needed   
refactoring

Refactoring   
wontfix

This won't be fixed
No Label
P0
P1
P2
P3
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-api#27
There is no content yet.
Delete Branch "aarifullin/frostfs-api:feature/26-add_keywords"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?