xk6 load balancing - limit number of connection to one endpoint #170

New issue

Open

opened 2024-09-24 08:52:14 +00:00 by a-baranov · 2 comments

a-baranov commented

2024-09-24 08:52:14 +00:00

When we have "slow" / "bad" node in the list of endpoints (e.g. with broaken disk) this dramatically impact performance of the whole run. In theory round-robin would evenly distribute threads among endpoints but finished thereads again evenly distributed among all nodes (including slow nodes which has "hanging" connections.

In worst case in 2-3 minutes after beginning all threads are running for bad node. other nodes has 0 thereads.

Describe the solution you'd like

Use differennt load balansing algorythm - count number of active connections to each endpoint and distribute connections evenly.
Provide statistics for each endpoint to uncover slow nodes.

## Is your feature request related to a problem? Please describe. When we have "slow" / "bad" node in the list of endpoints (e.g. with broaken disk) this dramatically impact performance of the whole run. In theory round-robin would evenly distribute threads among endpoints but finished thereads again evenly distributed among all nodes (including slow nodes which has "hanging" connections. In worst case in 2-3 minutes after beginning all threads are running for bad node. other nodes has 0 thereads. ## Describe the solution you'd like - Use differennt load balansing algorythm - count number of active connections to each endpoint and distribute connections evenly. - Provide statistics for each endpoint to uncover slow nodes.

fyrchik commented

2024-10-02 12:28:00 +00:00

Owner

This is a part of scenario and should be implemented by a perfomance team cc @mmalygina @a.bogatyrev
Currently, we assign a network address to a worker during initialization, so it should already be fixed in default scenarios. Those slow thread skew results, however.

This is a part of scenario and should be implemented by a perfomance team cc @mmalygina @a.bogatyrev Currently, we assign a network address to a worker during [initialization](https://git.frostfs.info/TrueCloudLab/xk6-frostfs/src/commit/a47bf149d824aede7303b290f756267ca0ed49c7/scenarios/grpc.js#L30), so it should already be fixed in default scenarios. Those slow thread skew results, however.

fyrchik commented

2024-10-02 12:31:22 +00:00

Owner

Regarding statistics: let's research, we can attach tags and metadata to metric samples, need to investigate how will it look in report.