Fixes for split-related EC queries #1163

fyrchik · 2024-06-05T09:40:05Z

fyrchik commented

2024-06-05 09:40:05 +00:00

Allow searching by splitID
Allow to get last part in the SplitInfo.

Tested on dev-env

$ frostfs-cli object head -w ./wallets/wallet.json -r s02.frostfs.devenv:8080 --cid HTq9uqN8afseqLvNWPe5n3X8kgX6WWb9X9aPSw7sBZ7x --oid 4uimMGYdaKHq3JypXsZkVTptzUzNGE6Z86z9wczPsn79 --ttl=1 --raw
Enter password >
Object is complex, split information received.
Split ID: e085aa2f-48ab-454a-99cb-eade42e64358
Linking object: Aj7pw6sZraC84LeHhhUvBqKbNZzRKgQqCpn1PVNbvHLN
Last object: 8EbnT7y2urj3mbsB1Gn6WW1G87kRsxDHW6g5m1sB6oKU

$ frostfs-cli object search -w ./wallets/wallet.json -r s01.frostfs.devenv:8080 --cid HTq9uqN8afseqLvNWPe5n3X8kgX6WWb9X9aPSw7sBZ7x --ttl=1 --filters '$Object:split.splitID EQ e085aa2f-48ab-454a-99cb-eade42e64358'
Enter password >
Found 3 objects.
5PSMCc8FGuK7uPFJGcbQ4qkM2VNXSufP3aQHGKnsPzUk
8EbnT7y2urj3mbsB1Gn6WW1G87kRsxDHW6g5m1sB6oKU
Aj7pw6sZraC84LeHhhUvBqKbNZzRKgQqCpn1PVNbvHLN

1. Allow searching by splitID 2. Allow to get last part in the SplitInfo. Tested on dev-env ``` $ frostfs-cli object head -w ./wallets/wallet.json -r s02.frostfs.devenv:8080 --cid HTq9uqN8afseqLvNWPe5n3X8kgX6WWb9X9aPSw7sBZ7x --oid 4uimMGYdaKHq3JypXsZkVTptzUzNGE6Z86z9wczPsn79 --ttl=1 --raw Enter password > Object is complex, split information received. Split ID: e085aa2f-48ab-454a-99cb-eade42e64358 Linking object: Aj7pw6sZraC84LeHhhUvBqKbNZzRKgQqCpn1PVNbvHLN Last object: 8EbnT7y2urj3mbsB1Gn6WW1G87kRsxDHW6g5m1sB6oKU $ frostfs-cli object search -w ./wallets/wallet.json -r s01.frostfs.devenv:8080 --cid HTq9uqN8afseqLvNWPe5n3X8kgX6WWb9X9aPSw7sBZ7x --ttl=1 --filters '$Object:split.splitID EQ e085aa2f-48ab-454a-99cb-eade42e64358' Enter password > Found 3 objects. 5PSMCc8FGuK7uPFJGcbQ4qkM2VNXSufP3aQHGKnsPzUk 8EbnT7y2urj3mbsB1Gn6WW1G87kRsxDHW6g5m1sB6oKU Aj7pw6sZraC84LeHhhUvBqKbNZzRKgQqCpn1PVNbvHLN ```

fyrchik force-pushed fix-last-part from 989f1ea63a to 6a5298a89b

2024-06-05 09:40:30 +00:00

Compare

fyrchik changed title from ~~Fixes for split-related EC queries~~ to WIP: Fixes for split-related EC queries

2024-06-05 09:40:32 +00:00

fyrchik requested review from storage-core-committers 2024-06-05 09:40:53 +00:00

fyrchik requested review from storage-core-developers 2024-06-05 09:41:09 +00:00

fyrchik commented

2024-06-06 07:13:57 +00:00

For some reason, last part is not fetched from each node (which I would expect), investigating.

fyrchik force-pushed fix-last-part from 6a5298a89b to c1cdd5d408

2024-06-06 08:25:08 +00:00

Compare

fyrchik changed title from ~~WIP: Fixes for split-related EC queries~~ to Fixes for split-related EC queries

2024-06-06 08:28:52 +00:00

fyrchik commented

2024-06-06 08:29:16 +00:00

The bug was in race condition when both linking object and the last part were put on one shard.

fyrchik force-pushed fix-last-part from c1cdd5d408 to f9efa85cbd

2024-06-06 08:30:45 +00:00

Compare

fyrchik added the

bug

label 2024-06-06 08:32:35 +00:00

fyrchik added this to the v0.41.0 milestone 2024-06-06 08:32:41 +00:00

dstepanov-yadro requested changes 2024-06-06 08:40:02 +00:00

pkg/local_object_storage/metabase/put.go Outdated

					
				@ -405,3 +414,3 @@

				}

				func putUniqueIndexItem(tx *bbolt.Tx, item namedBucketItem) error {

				func updateUniqueIndexItem(tx *bbolt.Tx, item namedBucketItem, update func(oldSI, newSI []byte) ([]byte, error)) error {