Rework worker pools in object.PUT #646

New issue

Open

opened 2023-08-24 16:41:08 +00:00 by fyrchik · 1 comment

fyrchik commented

2023-08-24 16:41:08 +00:00

Owner

There are multiple problems with them:

We spawn new goroutines on signature verification, it is not costly to spawn a new goroutine here.
The error message is not informative at all. Rate-limiting message during request initiation is (the problem is that we already have done a pretty costly work of forming/validating object header). It would also be easier to have different limit for user/other storage nodes, because with pool we do not really control anything and change this dynamically.
It heavily favours containers with bigger REP counts, which is unfair (more REP -> less speed for you, not problems for the others)

Possible different approach:

object.put.ratelimit setting which tells how many PUT can be processed simultaneously.
object.put.parallelism: N which allows N simultaneous derived requests to be executed in parallel (so that REP X where X > N is restricted. This may depend on the load.

There are multiple problems with them: 1. We spawn new goroutines on signature verification, it is not costly to spawn a new goroutine here. 2. The error message is not informative at all. Rate-limiting message during request initiation is (the problem is that we already have done a pretty costly work of forming/validating object header). It would also be easier to have different limit for user/other storage nodes, because with pool we do not really control anything and change this dynamically. 3. It heavily favours containers with bigger REP counts, which is unfair (more REP -> less speed for you, not problems for the others) Possible different approach: 1. `object.put.ratelimit` setting which tells how many PUT can be processed simultaneously. 2. `object.put.parallelism: N` which allows N simultaneous derived requests to be executed in parallel (so that `REP X` where X > N is restricted. This may depend on the load.

fyrchik added the

discussion

frostfs-node

triage

labels 2023-08-24 16:41:08 +00:00

fyrchik added this to the v0.38.0 milestone 2023-08-25 09:50:34 +00:00

fyrchik added the

refactoring

label 2023-08-25 09:51:05 +00:00

fyrchik modified the milestone from v0.38.0 to v0.39.0

2023-10-02 12:48:45 +00:00

fyrchik commented

2023-10-02 13:25:32 +00:00

Author

Owner

Also, if we are going to parallelize putting of big objects with TrueCloudLab/frostfs-sdk-go#128, we need some way to restrict node memory consumption. Ideally we would like to NOT treat big and small objects equally, so that we can either process 100 of 128MB parts or 10000 of 1MB objects without friction.

Also, if we are going to parallelize putting of big objects with https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/issues/128, we need some way to restrict node memory consumption. Ideally we would like to NOT treat big and small objects equally, so that we can either process 100 of 128MB parts or 10000 of 1MB objects without friction.

fyrchik modified the milestone from v0.39.0 to v0.40.0

2024-05-14 14:18:43 +00:00

fyrchik modified the milestone from v0.40.0 to v0.41.0

2024-06-01 09:19:47 +00:00

fyrchik modified the milestone from v0.41.0 to v0.42.0

2024-06-14 07:06:58 +00:00

fyrchik modified the milestone from v0.42.0 to v0.43.0

2024-07-23 06:34:44 +00:00

fyrchik modified the milestone from v0.43.0 to v0.44.0

2024-09-30 11:51:36 +00:00