metabase: Fix EC search with slow and fast filters #1408

Merged
fyrchik merged 1 commit from dstepanov-yadro/frostfs-node:fix/ec_combined_search into master 2024-10-26 11:30:27 +00:00

Use EC chunk for search when possible.

Before fix:

 frostfs-cli --config TemporaryDir/user-0x6231a87bd413b-config.yml object put --rpc-endpoint '10.78.70.145:8080' --cid '14aFRdfSSD4X9vkEZ5WnSrMgPmFaWNqjQdDFZjY1Pz1L' --file 'TemporaryDir/object-0x6231b07d55da2' --attributes 'key1=1,key2=abc,common_key=common_value' --no-progress --timeout '100s'
RETCODE: 0
STDOUT:
[TemporaryDir/object-0x6231b07d55da2] Object successfully stored
  OID: H6KLNDBq317LbhRektoVvauGWkBgTv2MRJFTYbRMnCnY
  CID: 14aFRdfSSD4X9vkEZ5WnSrMgPmFaWNqjQdDFZjY1Pz1L
STDERR:
Start / End / Elapsed     14:54:23.327029 / 14:54:23.837041 / 0:00:00.510012 


frostfs-cli --config TemporaryDir/user-0x6231a87bd413b-config.yml object search --rpc-endpoint '10.78.70.142:8080' --cid '14aFRdfSSD4X9vkEZ5WnSrMgPmFaWNqjQdDFZjY1Pz1L' --filters 'key1 EQ 1' --filters 'key2 EQ abc' --filters 'common_key EQ common_value' --root --timeout '100s'
RETCODE: 0
STDOUT:
Found 0 objects.
STDERR:
Start / End / Elapsed     14:54:32.954502 / 14:54:33.457778 / 0:00:00.503276 

After fix:

./bin/frostfs-cli object put -r 127.0.0.1:8080 --wallet /home/dstepanov/src/frostfs-node/dev/wallet.json -c /home/dst
epanov/src/frostfs-node/dev/empty_pass.yml --cid 8vSBLrtbT9N7jHhKa2B2mUhQnxLBnwrUPkANnC7it35a --file /home/dstepanov/payload/1MB.bin  --attributes 'key1=1,key2=abc,common_key=common_value'
 1048576 / 1048576 [=============================================================================================================================] 100.00% 0s
[/home/dstepanov/payload/1MB.bin] Object successfully stored
  OID: Fq5B5YAm14oCuDMABWX1QL5CPCqx3HC6MXa8B57g7hap
  CID: 8vSBLrtbT9N7jHhKa2B2mUhQnxLBnwrUPkANnC7it35a
  
  
  ./bin/frostfs-cli object search -r 127.0.0.1:8080 --wallet /home/dstepanov/src/frostfs-node/dev/wallet.json -c /home/
dstepanov/src/frostfs-node/dev/empty_pass.yml --cid 8vSBLrtbT9N7jHhKa2B2mUhQnxLBnwrUPkANnC7it35a --timeout=1m --filters 'key1 EQ 1' --filters 'key2 EQ abc' --filters 'common_key EQ common_value'
Found 1 objects.
Fq5B5YAm14oCuDMABWX1QL5CPCqx3HC6MXa8B57g7hap
Use EC chunk for search when possible. Before fix: ``` frostfs-cli --config TemporaryDir/user-0x6231a87bd413b-config.yml object put --rpc-endpoint '10.78.70.145:8080' --cid '14aFRdfSSD4X9vkEZ5WnSrMgPmFaWNqjQdDFZjY1Pz1L' --file 'TemporaryDir/object-0x6231b07d55da2' --attributes 'key1=1,key2=abc,common_key=common_value' --no-progress --timeout '100s' RETCODE: 0 STDOUT: [TemporaryDir/object-0x6231b07d55da2] Object successfully stored OID: H6KLNDBq317LbhRektoVvauGWkBgTv2MRJFTYbRMnCnY CID: 14aFRdfSSD4X9vkEZ5WnSrMgPmFaWNqjQdDFZjY1Pz1L STDERR: Start / End / Elapsed 14:54:23.327029 / 14:54:23.837041 / 0:00:00.510012 frostfs-cli --config TemporaryDir/user-0x6231a87bd413b-config.yml object search --rpc-endpoint '10.78.70.142:8080' --cid '14aFRdfSSD4X9vkEZ5WnSrMgPmFaWNqjQdDFZjY1Pz1L' --filters 'key1 EQ 1' --filters 'key2 EQ abc' --filters 'common_key EQ common_value' --root --timeout '100s' RETCODE: 0 STDOUT: Found 0 objects. STDERR: Start / End / Elapsed 14:54:32.954502 / 14:54:33.457778 / 0:00:00.503276 ``` After fix: ``` ./bin/frostfs-cli object put -r 127.0.0.1:8080 --wallet /home/dstepanov/src/frostfs-node/dev/wallet.json -c /home/dst epanov/src/frostfs-node/dev/empty_pass.yml --cid 8vSBLrtbT9N7jHhKa2B2mUhQnxLBnwrUPkANnC7it35a --file /home/dstepanov/payload/1MB.bin --attributes 'key1=1,key2=abc,common_key=common_value' 1048576 / 1048576 [=============================================================================================================================] 100.00% 0s [/home/dstepanov/payload/1MB.bin] Object successfully stored OID: Fq5B5YAm14oCuDMABWX1QL5CPCqx3HC6MXa8B57g7hap CID: 8vSBLrtbT9N7jHhKa2B2mUhQnxLBnwrUPkANnC7it35a ./bin/frostfs-cli object search -r 127.0.0.1:8080 --wallet /home/dstepanov/src/frostfs-node/dev/wallet.json -c /home/ dstepanov/src/frostfs-node/dev/empty_pass.yml --cid 8vSBLrtbT9N7jHhKa2B2mUhQnxLBnwrUPkANnC7it35a --timeout=1m --filters 'key1 EQ 1' --filters 'key2 EQ abc' --filters 'common_key EQ common_value' Found 1 objects. Fq5B5YAm14oCuDMABWX1QL5CPCqx3HC6MXa8B57g7hap ```
dstepanov-yadro force-pushed fix/ec_combined_search from 40782935e1 to 8fb4a6ec5a 2024-10-02 08:50:31 +00:00 Compare
dstepanov-yadro force-pushed fix/ec_combined_search from 8fb4a6ec5a to 8f6aa8677c 2024-10-02 08:51:49 +00:00 Compare
dstepanov-yadro force-pushed fix/ec_combined_search from 8f6aa8677c to 704b06272a 2024-10-02 08:52:39 +00:00 Compare
dstepanov-yadro changed title from WIP: metabase: Fix EC search with slow and fast filters to metabase: Fix EC search with slow and fast filters 2024-10-02 09:01:32 +00:00
dstepanov-yadro requested review from storage-core-committers 2024-10-02 09:01:37 +00:00
dstepanov-yadro requested review from storage-core-developers 2024-10-02 09:01:38 +00:00
fyrchik approved these changes 2024-10-02 09:08:31 +00:00
Dismissed
@ -765,0 +782,4 @@
ecParentAttr = append(ecParentAttr, attr)
chunkObj := testutil.GenerateObjectWithCID(cnr)
chunkObj.SetContainerID(cnr)
Owner

GenerateObjectWithCID should already set container ID, why do we set it again?
Same below.

`GenerateObjectWithCID` should already set container ID, why do we set it again? Same below.
Author
Member

Fixed

Fixed
fyrchik dismissed fyrchik's review 2024-10-02 11:19:13 +00:00
fyrchik requested review from storage-core-committers 2024-10-02 11:19:22 +00:00
acid-ant approved these changes 2024-10-02 11:26:02 +00:00
Dismissed
dstepanov-yadro force-pushed fix/ec_combined_search from 704b06272a to 4903b9f7c4 2024-10-02 11:37:19 +00:00 Compare
dstepanov-yadro dismissed acid-ant's review 2024-10-02 11:37:19 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

acid-ant approved these changes 2024-10-02 11:39:25 +00:00
fyrchik requested changes 2024-10-02 11:39:59 +00:00
fyrchik left a comment
Owner

Fix tests

Could you write something more descriptive? e.g. Do not set container id twice

>Fix tests Could you write something more descriptive? e.g. `Do not set container id twice`
dstepanov-yadro force-pushed fix/ec_combined_search from 4903b9f7c4 to 01e3944b31 2024-10-02 11:42:22 +00:00 Compare
Author
Member

Fix tests

Could you write something more descriptive? e.g. Do not set container id twice

Done

> >Fix tests > > Could you write something more descriptive? e.g. `Do not set container id twice` Done
Owner

image

![image](/attachments/094cd67a-5c2c-434c-805e-8748ffb8a647)
Author
Member

image

Fixed in description:
image

> ![image](/attachments/094cd67a-5c2c-434c-805e-8748ffb8a647) Fixed in description: ![image](/attachments/5db169b5-5e35-42dc-93c7-cb1a514b0ee4)
Owner

I meant fix it in the message.

I meant fix it in the message.
fyrchik merged commit 01e3944b31 into master 2024-10-03 07:24:18 +00:00
fyrchik referenced this pull request from a commit 2024-10-03 07:24:19 +00:00
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-core-developers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1408
No description provided.