Blobovnizca negative size metric value #605

Closed
opened 2023-08-14 08:15:51 +00:00 by dstepanov-yadro · 2 comments

Expected Behavior

frostfs_node_blobovniza_tree_size_bytes metric must be positive.

Current Behavior

frostfs_node_blobovniza_tree_size_bytes metric can be negative.

Steps to Reproduce (for bugs)

  1. Start xk6 test:
#!/bin/bash

echo "Generating preset..."
time /home/dstepanov/src/xk6-frostfs/scenarios/preset/preset_grpc.py \
--size 8 --containers 1 --out /home/dstepanov/src/xk6-frostfs/grpc_preset.json \
--endpoint s01.frostfs.devenv:8080 --preload_obj 0 --wallet /home/dstepanov/src/xk6-frostfs/scenarios/files/wallet.json \
--config empty_pass.yml --policy 'REP 1 CBF 1 SELECT 1 FROM *' --ignore-errors
echo "Preset generated"

echo "Starting grpc write test..."
./k6 run -e DURATION=300 -e WRITERS=20 -e READERS=0 -e DELETERS=0 -e WRITE_OBJ_SIZE=8 \
-e GRPC_ENDPOINTS=s01.frostfs.devenv:8080,s02.frostfs.devenv:8080,s03.frostfs.devenv:8080,s04.frostfs.devenv:8080 \
-e PREGEN_JSON=/home/dstepanov/src/xk6-frostfs/grpc_preset.json -e STREAM_TIMEOUT=30 -e SLEEP_READ=0.2 \
-e SLEEP_WRITE=0.2 /home/dstepanov/src/xk6-frostfs/scenarios/grpc.js
echo "grpc write test completed"
  1. See Grafana Blobovnizca dashboard

image

<!-- Provide a general summary of the issue in the Title above --> ## Expected Behavior `frostfs_node_blobovniza_tree_size_bytes` metric must be positive. ## Current Behavior `frostfs_node_blobovniza_tree_size_bytes` metric can be negative. ## Steps to Reproduce (for bugs) <!-- Provide a link to a live example, or an unambiguous set of steps to reproduce this bug. --> 1. Start xk6 test: ``` #!/bin/bash echo "Generating preset..." time /home/dstepanov/src/xk6-frostfs/scenarios/preset/preset_grpc.py \ --size 8 --containers 1 --out /home/dstepanov/src/xk6-frostfs/grpc_preset.json \ --endpoint s01.frostfs.devenv:8080 --preload_obj 0 --wallet /home/dstepanov/src/xk6-frostfs/scenarios/files/wallet.json \ --config empty_pass.yml --policy 'REP 1 CBF 1 SELECT 1 FROM *' --ignore-errors echo "Preset generated" echo "Starting grpc write test..." ./k6 run -e DURATION=300 -e WRITERS=20 -e READERS=0 -e DELETERS=0 -e WRITE_OBJ_SIZE=8 \ -e GRPC_ENDPOINTS=s01.frostfs.devenv:8080,s02.frostfs.devenv:8080,s03.frostfs.devenv:8080,s04.frostfs.devenv:8080 \ -e PREGEN_JSON=/home/dstepanov/src/xk6-frostfs/grpc_preset.json -e STREAM_TIMEOUT=30 -e SLEEP_READ=0.2 \ -e SLEEP_WRITE=0.2 /home/dstepanov/src/xk6-frostfs/scenarios/grpc.js echo "grpc write test completed" ``` 2. See Grafana Blobovnizca dashboard ![image](/attachments/3f7074c0-fd65-4ba7-9787-5b2e72b465fc)
dstepanov-yadro added the
bug
frostfs-node
triage
observability
labels 2023-08-14 08:15:51 +00:00
dstepanov-yadro added the
good first issue
label 2023-08-14 11:41:58 +00:00
Author
Member

Also after reboot metric frostfs_node_blobovniza_tree_size_bytes resets. But i don't think it's correct.

image

Also after reboot metric ```frostfs_node_blobovniza_tree_size_bytes``` resets. But i don't think it's correct. ![image](/attachments/523f5d7a-ada3-4a77-9b1d-a0c15d2a64f0)
180 KiB
dstepanov-yadro was assigned by fyrchik 2023-08-16 07:03:47 +00:00
fyrchik added this to the v0.37.0 milestone 2023-08-16 07:04:38 +00:00
Author
Member

Fixed by #612

Fixed by https://git.frostfs.info/TrueCloudLab/frostfs-node/pulls/612
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#605
No description provided.