Allow to pass custom gRPC chunk size for object put #79

dstepanov-yadro · 2023-07-05T12:36:44Z

dstepanov-yadro commented

2023-07-05 12:36:44 +00:00

Also removed buf size variable. Now we use constant for get requests. For put requests default value used if not chunk size provided. If chunk size provided, then it will be used as buf size.

Relates https://git.frostfs.info/TrueCloudLab/frostfs-node/issues/77 Also removed buf size variable. Now we use constant for get requests. For put requests default value used if not chunk size provided. If chunk size provided, then it will be used as buf size.

dstepanov-yadro force-pushed feat/put-chunk-size from f8cfc0710c to 65043620b5

2023-07-05 12:37:40 +00:00

Compare

dstepanov-yadro force-pushed feat/put-chunk-size from 65043620b5 to 896749c568

2023-07-05 12:48:10 +00:00

Compare

dstepanov-yadro force-pushed feat/put-chunk-size from 896749c568 to c5e7a41bfd

2023-07-05 12:50:42 +00:00

Compare

dstepanov-yadro changed title from ~~Allow to pass custom gRPC chunk size for object put~~ to WIP: Allow to pass custom gRPC chunk size for object put

2023-07-05 12:51:39 +00:00

fyrchik approved these changes 2023-07-05 13:00:42 +00:00

scenarios/grpc.js Outdated

													
				@ -130,3 +130,3 @@

				    const { payload, hash } = generator.genPayload(registry_enabled);

				    const resp = grpc_client.put(container, headers, payload);

				    const resp = grpc_client.put(container, headers, payload, __ENV.GRPC_CHUNK_SIZE ? parseInt(__ENV.GRPC_CHUNK_SIZE) : 0);

fyrchik commented

2023-07-05 12:59:20 +00:00

Can we move it to the init stage? Otherwise we are parsing this on each PUT.

Can we move it to the init stage? Otherwise we are parsing this on each `PUT`.

dstepanov-yadro commented

2023-07-05 13:49:28 +00:00

Poster

done

dstepanov-yadro force-pushed feat/put-chunk-size from c5e7a41bfd to 4ba50f65e7

2023-07-05 13:05:33 +00:00

Compare

dstepanov-yadro force-pushed feat/put-chunk-size from 4ba50f65e7 to d1bbd11fd4

2023-07-05 13:13:27 +00:00

Compare

dstepanov-yadro force-pushed feat/put-chunk-size from d1bbd11fd4 to 3ef8cd8322

2023-07-05 13:14:26 +00:00

Compare

dstepanov-yadro requested review from storage-core-committers 2023-07-05 13:35:45 +00:00

dstepanov-yadro requested review from storage-core-developers 2023-07-05 13:35:45 +00:00

dstepanov-yadro changed title from ~~WIP: Allow to pass custom gRPC chunk size for object put~~ to Allow to pass custom gRPC chunk size for object put

2023-07-05 13:35:49 +00:00

dstepanov-yadro requested review from anikeev-yadro 2023-07-05 13:35:58 +00:00

fyrchik reviewed 2023-07-05 13:58:06 +00:00

scenarios/grpc_car.js Outdated

													
				@ -67,6 +67,7 @@ if (write_rate > 0) {

				        gracefulStop: '5s',

				    };

				}

				const write_grpc_chunk_size = 1024 * parseInt(__ENV.GRPC_CHUNK_SIZE || '0')

fyrchik commented

2023-07-05 13:58:06 +00:00

It is related to write, maybe put it close to other declarations, before the if?

It is related to write, maybe put it close to other declarations, before the `if`?

dstepanov-yadro commented

2023-07-05 14:07:21 +00:00

Poster

const pre_alloc_write_vus = parseInt(__ENV.PRE_ALLOC_WRITERS || '0');
const max_write_vus = parseInt(__ENV.MAX_WRITERS || pre_alloc_write_vus);
const write_rate = parseInt(__ENV.WRITE_RATE || '0');
if (write_rate > 0) {
    scenarios.write = {
        executor: 'constant-arrival-rate',
        duration: `${duration}s`,
        preAllocatedVUs: pre_alloc_write_vus,
        maxVUs: max_write_vus,
        rate: write_rate,
        timeUnit: time_unit,
        exec: 'obj_write', 
        gracefulStop: '5s',
    };
}
const write_grpc_chunk_size = 1024 * parseInt(__ENV.GRPC_CHUNK_SIZE || '0')

This is already next to the other write consts.

``` const pre_alloc_write_vus = parseInt(__ENV.PRE_ALLOC_WRITERS || '0'); const max_write_vus = parseInt(__ENV.MAX_WRITERS || pre_alloc_write_vus); const write_rate = parseInt(__ENV.WRITE_RATE || '0'); if (write_rate > 0) { scenarios.write = { executor: 'constant-arrival-rate', duration: `${duration}s`, preAllocatedVUs: pre_alloc_write_vus, maxVUs: max_write_vus, rate: write_rate, timeUnit: time_unit, exec: 'obj_write', gracefulStop: '5s', }; } const write_grpc_chunk_size = 1024 * parseInt(__ENV.GRPC_CHUNK_SIZE || '0') ``` This is already next to the other `write` consts.