metabase: optimize `Inhume` operation #1385

Merged

dstepanov-yadro merged 1 commit from dstepanov-yadro/frostfs-node:fix/metabase_inhume_hangs into master

2024-10-26 11:30:26 +00:00

dstepanov-yadro commented

2024-09-19 15:41:57 +00:00

Member

During load testing, a high latency (> 5s) was detected when performing an Inhume operation.

Fixed:

Update replaced with Batch in metabase: based on the analysis of pprof data, it can be seen that single Update operations in the metabase take a long time
optimized isTomb check that checks that inhumed object is not tombstone: now it checks only objects from the same container
added check that tombstone and tombstoned object have the same containerID: object of type TOMBSTONE stores objectIDs as members, also see Inhume call from Delete RPC:
cmd/frostfs-node/object.go
Line 527 in c34b8ac
addrs[i].SetContainer(tombstone.Container())

Inhume latency before:

Inhume latency after:

During load testing, a high latency (> 5s) was detected when performing an `Inhume` operation. Fixed: - `Update` replaced with `Batch` in metabase: based on the analysis of pprof data, it can be seen that single `Update` operations in the metabase take a long time - optimized `isTomb` check that checks that inhumed object is not tombstone: now it checks only objects from the same container - added check that tombstone and tombstoned object have the same containerID: object of type `TOMBSTONE` stores objectIDs as members, also see `Inhume` call from `Delete` RPC: https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/c34b8acedde282bbe81efccea772a923ee570a8f/cmd/frostfs-node/object.go#L527 Inhume latency before: ![image](/attachments/af1e72dc-a167-4134-8b6b-b6b353833d62) Inhume latency after: ![image](/attachments/dbd898d2-ab6d-4868-96c2-27bfec382d98)

image.png

169 KiB

image.png

176 KiB

🚀 1

dstepanov-yadro force-pushed fix/metabase_inhume_hangs from c1b4c5c0a0 to a193465421

2024-09-19 16:51:42 +00:00

Compare

dstepanov-yadro requested review from storage-core-committers 2024-09-19 16:54:48 +00:00

dstepanov-yadro requested review from storage-core-developers 2024-09-19 16:54:49 +00:00

dstepanov-yadro changed title from ~~metabase: Use Batch for all concurrent operations~~ to WIP: metabase: Use Batch for all concurrent operations

2024-09-20 09:17:28 +00:00

dstepanov-yadro force-pushed fix/metabase_inhume_hangs from 2e35eff8cb to b15437c947

2024-09-20 12:19:09 +00:00

Compare

dstepanov-yadro changed title from ~~WIP: metabase: Use Batch for all concurrent operations~~ to metabase: optimize Inhume operation

2024-09-20 12:27:24 +00:00

dstepanov-yadro force-pushed fix/metabase_inhume_hangs from b15437c947 to 3b5a9d466f

2024-09-23 08:21:17 +00:00

Compare