Metabase batching with buckets cache #1694

New issue

Closed

opened 2025-03-21 07:14:47 +00:00 by dstepanov-yadro · 2 comments

dstepanov-yadro commented

2025-03-21 07:14:47 +00:00

Member

In #1688 and #1686 search for objects in the metabase was optimized by caching buckets within a single transaction. It is proposed to make the same optimization for write batching.

Describe the solution you'd like

Add new method in metabase (CustomBatch, BatchWithCache, Batch2, BatchEx - I'm not good at naming :( ) with signature func (db *DB) BatchEx(fn func(*bucketCache, *Tx) error) error with the same logic as 914ccefafb/db.go (L966) but use one bucketCache for all db.batch.calls.
Replace db.boltDB.Batch calls with new method calls.
Check if this will lead to better write performance.

## Is your feature request related to a problem? Please describe. In https://git.frostfs.info/TrueCloudLab/frostfs-node/pulls/1688 and https://git.frostfs.info/TrueCloudLab/frostfs-node/pulls/1686 search for objects in the metabase was optimized by caching buckets within a single transaction. It is proposed to make the same optimization for write batching. ## Describe the solution you'd like Add new method in metabase (`CustomBatch`, `BatchWithCache`, `Batch2`, `BatchEx` - I'm not good at naming :( ) with signature `func (db *DB) BatchEx(fn func(*bucketCache, *Tx) error) error ` with the same logic as https://github.com/etcd-io/bbolt/blob/914ccefafb7fa4dd809971b448515717142d9041/db.go#L966 but use one bucketCache for all `db.batch.calls`. Replace `db.boltDB.Batch` calls with new method calls. Check if this will lead to better write performance.

dstepanov-yadro added the

labels 2025-03-21 07:14:47 +00:00

dstepanov-yadro self-assigned this 2025-04-02 06:38:00 +00:00

dstepanov-yadro commented

2025-04-03 06:52:50 +00:00

Author

Member

Branch: https://git.frostfs.info/dstepanov-yadro/frostfs-node/src/branch/feat/metabase_batch_ex

Tested on hardware: there is no write performance improvements.

Branch: https://git.frostfs.info/dstepanov-yadro/frostfs-node/src/branch/feat/metabase_batch_ex Tested on hardware: there is no write performance improvements.

dstepanov-yadro commented

2025-04-03 12:29:26 +00:00

Author

Member

Test result:

k6/scenarios/preset/preset_grpc.py \
--size 8 --containers 1 --out grpc_preset.json \
--endpoint ... --preload_obj 0 --wallet k6/scenarios/files/wallet.json \
--config empty_pass.yaml --policy 'REP 2 CBF 2 SELECT 2 FROM *' --ignore-errors


k6/k6 run -e DURATION=600 -e WRITERS=600 -e READERS=0 -e DELETERS=0 -e WRITE_OBJ_SIZE=8 \
-e GRPC_ENDPOINTS=... \
-e PREGEN_JSON=grpc_preset.json \
k6/scenarios/grpc.js

100 containers

no fix

     data_received..............: 0 B      0 B/s
     data_sent..................: 255 GB   71 MB/s
     frostfs_obj_put_bytes......: 255 GB   71 MB/s
     frostfs_obj_put_duration...: avg=68.79ms min=8.15ms   med=62.95ms max=943.69ms p(90)=95.39ms p(95)=109.25ms
     frostfs_obj_put_success....: 31111627 8641.926531/s
     iteration_duration.........: avg=69.4ms  min=284.45µs med=63.55ms max=944.12ms p(90)=96.05ms p(95)=109.93ms
     iterations.................: 31111627 8641.926531/s
     vus........................: 600      min=600       max=600
     vus_max....................: 600      min=600       max=600

with fix

     data_received..............: 0 B      0 B/s
     data_sent..................: 252 GB   70 MB/s
     frostfs_obj_put_bytes......: 252 GB   70 MB/s
     frostfs_obj_put_duration...: avg=69.47ms min=7.45ms   med=63.26ms max=955.15ms p(90)=96.17ms p(95)=110.13ms
     frostfs_obj_put_success....: 30810016 8557.848099/s
     iteration_duration.........: avg=70.08ms min=148.54µs med=63.86ms max=955.59ms p(90)=96.82ms p(95)=110.79ms
     iterations.................: 30810016 8557.848099/s
     vus........................: 600      min=600       max=600
     vus_max....................: 600      min=600       max=600

1 container

no fix

     data_received..............: 0 B     0 B/s
     data_sent..................: 42 GB   70 MB/s
     frostfs_obj_put_bytes......: 42 GB   70 MB/s
     frostfs_obj_put_duration...: avg=69.8ms  min=7.44ms   med=56.93ms max=997.4ms  p(90)=100.83ms p(95)=129.79ms
     frostfs_obj_put_success....: 5108765 8513.516454/s
     iteration_duration.........: avg=70.44ms min=287.26µs med=57.55ms max=998.06ms p(90)=101.57ms p(95)=130.54ms
     iterations.................: 5108765 8513.516454/s
     vus........................: 600     min=600       max=600

with fix

     data_received..............: 0 B     0 B/s
     data_sent..................: 41 GB   68 MB/s
     frostfs_obj_put_bytes......: 41 GB   68 MB/s
     frostfs_obj_put_duration...: avg=71.28ms min=7.74ms   med=58.17ms max=928.24ms p(90)=98.04ms p(95)=123.18ms
     frostfs_obj_put_success....: 5003617 8337.672815/s
     iteration_duration.........: avg=71.92ms min=147.83µs med=58.79ms max=928.93ms p(90)=98.77ms p(95)=123.93ms
     iterations.................: 5003617 8337.672815/s
     vus........................: 600     min=600       max=600

Test result: ``` k6/scenarios/preset/preset_grpc.py \ --size 8 --containers 1 --out grpc_preset.json \ --endpoint ... --preload_obj 0 --wallet k6/scenarios/files/wallet.json \ --config empty_pass.yaml --policy 'REP 2 CBF 2 SELECT 2 FROM *' --ignore-errors k6/k6 run -e DURATION=600 -e WRITERS=600 -e READERS=0 -e DELETERS=0 -e WRITE_OBJ_SIZE=8 \ -e GRPC_ENDPOINTS=... \ -e PREGEN_JSON=grpc_preset.json \ k6/scenarios/grpc.js ``` 100 containers no fix ``` data_received..............: 0 B 0 B/s data_sent..................: 255 GB 71 MB/s frostfs_obj_put_bytes......: 255 GB 71 MB/s frostfs_obj_put_duration...: avg=68.79ms min=8.15ms med=62.95ms max=943.69ms p(90)=95.39ms p(95)=109.25ms frostfs_obj_put_success....: 31111627 8641.926531/s iteration_duration.........: avg=69.4ms min=284.45µs med=63.55ms max=944.12ms p(90)=96.05ms p(95)=109.93ms iterations.................: 31111627 8641.926531/s vus........................: 600 min=600 max=600 vus_max....................: 600 min=600 max=600 ``` with fix ``` data_received..............: 0 B 0 B/s data_sent..................: 252 GB 70 MB/s frostfs_obj_put_bytes......: 252 GB 70 MB/s frostfs_obj_put_duration...: avg=69.47ms min=7.45ms med=63.26ms max=955.15ms p(90)=96.17ms p(95)=110.13ms frostfs_obj_put_success....: 30810016 8557.848099/s iteration_duration.........: avg=70.08ms min=148.54µs med=63.86ms max=955.59ms p(90)=96.82ms p(95)=110.79ms iterations.................: 30810016 8557.848099/s vus........................: 600 min=600 max=600 vus_max....................: 600 min=600 max=600 ``` 1 container no fix ``` data_received..............: 0 B 0 B/s data_sent..................: 42 GB 70 MB/s frostfs_obj_put_bytes......: 42 GB 70 MB/s frostfs_obj_put_duration...: avg=69.8ms min=7.44ms med=56.93ms max=997.4ms p(90)=100.83ms p(95)=129.79ms frostfs_obj_put_success....: 5108765 8513.516454/s iteration_duration.........: avg=70.44ms min=287.26µs med=57.55ms max=998.06ms p(90)=101.57ms p(95)=130.54ms iterations.................: 5108765 8513.516454/s vus........................: 600 min=600 max=600 ``` with fix ``` data_received..............: 0 B 0 B/s data_sent..................: 41 GB 68 MB/s frostfs_obj_put_bytes......: 41 GB 68 MB/s frostfs_obj_put_duration...: avg=71.28ms min=7.74ms med=58.17ms max=928.24ms p(90)=98.04ms p(95)=123.18ms frostfs_obj_put_success....: 5003617 8337.672815/s iteration_duration.........: avg=71.92ms min=147.83µs med=58.79ms max=928.93ms p(90)=98.77ms p(95)=123.93ms iterations.................: 5003617 8337.672815/s vus........................: 600 min=600 max=600 ```

dstepanov-yadro closed this issue

2025-04-03 12:29:27 +00:00