Add local targets

fyrchik commented

2023-02-24 14:57:13 +00:00

(Migrated from github.com)

Sometimes we want to perform benchmark on a filled cluster. Filling it with gRPC is fast, but not enough: we are talking about weeks, not hours. Writing a separate utility for this is a dead-end scenario IMO: it has only 1 use-case, maintenance costs are non-negligible, users need to learn yet another CLI interface (and, of course, inevitable bugs in the implementation).

The proposal is to use implement local client:

Use engine from frostfs-node.
Execute k6 on a target hardware using new local.js scenario.
Policer/tree service sync will do the rest.

Pros:

The results can be compared with gRPC results to see the overhead we have.
Easily automated to test for regressions. Local benchmarks in the frostfs-node are not suitable for this: they are usually not long/big enough.
S3 is also easily supported: engine exports all necessary machinery.

Cons:

Harder to maintain compatibility: we need to update k6 each time engine is updated. However, it isn't impossible: the engine interface changes very rarely.
Importing frostfs-node here is not aestethically pleasing IMO.

cc @anikeev-yadro @realloc @carpawell @alexvanin

Sometimes we want to perform benchmark on a filled cluster. Filling it with gRPC is fast, but not enough: we are talking about weeks, not hours. Writing a separate utility for this is a dead-end scenario IMO: it has only 1 use-case, maintenance costs are non-negligible, users need to learn yet another CLI interface (and, of course, inevitable bugs in the implementation). The proposal is to use implement local client: 1. Use `engine` from `frostfs-node`. 2. Execute k6 on a target hardware using new `local.js` scenario. 3. Policer/tree service sync will do the rest. Pros: 1. The results can be compared with gRPC results to see the overhead we have. 2. Easily automated to test for regressions. Local benchmarks in the `frostfs-node` are not suitable for this: they are usually not long/big enough. 4. S3 is also easily supported: engine exports all necessary machinery. Cons: 1. Harder to maintain compatibility: we need to update k6 each time engine is updated. However, it isn't impossible: the engine interface changes _very_ rarely. 2. Importing `frostfs-node` here is not aestethically pleasing IMO. cc @anikeev-yadro @realloc @carpawell @alexvanin

realloc commented

2023-02-24 17:59:10 +00:00

(Migrated from github.com)

S3 is also easily supported: engine exports all necessary machinery.

Could you please clarify here a bit? Do you mean handling both tree service calls and storage engine calls in "S3-local" case?

> S3 is also easily supported: engine exports all necessary machinery. Could you please clarify here a bit? Do you mean handling both tree service calls and storage engine calls in "S3-local" case?

fyrchik commented

2023-02-28 07:07:46 +00:00

(Migrated from github.com)

Yes. Tree service is just a wrapper over the engine, we can work with pilorama directly.
The only difficulty is knowing the tree structure for a bucket, but our bench is simple enough (flat structure, single version).

Yes. Tree service is just a wrapper over the engine, we can work with pilorama directly. The only difficulty is knowing the tree structure for a bucket, but our bench is simple enough (flat structure, single version).

realloc commented

2023-02-28 07:57:18 +00:00

(Migrated from github.com)

There is one more benchmark for versioned objects case where we put a loooong chain of versions for an object.
Ok. LGTM =)

There is one more benchmark for versioned objects case where we put a loooong chain of versions for an object. Ok. LGTM =)

ale64bit was assigned by fyrchik

2023-03-09 07:40:50 +00:00

fyrchik added a new dependency 2023-03-10 07:06:09 +00:00

TrueCloudLab/frostfs-node#18 - Simplify/modify/refactor Blobovniczas

fyrchik commented

2023-03-10 08:47:59 +00:00

Owner

It would be nice to allow setting NoSync for all subcomponents too.
While we could change the configuration (and then forget to change it back), a flag can make it easier.

It would be nice to allow setting `NoSync` for all subcomponents too. While we could change the configuration (and then forget to change it back), a flag can make it easier.

alexvanin added the

P0

label 2023-03-10 09:11:37 +00:00

fyrchik referenced this issue from TrueCloudLab/frostfs-node

2023-03-10 12:22:00 +00:00

Simplify/modify/refactor Blobovniczas #18

alexvanin referenced this issue from TrueCloudLab/frostfs-s3-gw

2023-03-15 07:32:19 +00:00

Abstract network communication in TreeClient #59

ale64bit referenced this issue from a pull request that will close it,

2023-03-16 09:45:23 +00:00

[#14]: Add local target support #35

fyrchik closed this issue

2023-03-23 11:58:12 +00:00

fyrchik referenced this issue from a commit

2023-03-23 11:58:12 +00:00

[#14]: Add local target support

fyrchik commented

2023-03-27 11:30:00 +00:00

Owner

Closed by accident, S3 still left.

fyrchik reopened this issue

2023-03-27 11:30:00 +00:00

ale64bit commented

2023-03-29 10:54:58 +00:00

Member

@fyrchik Could you clarify how much of S3 would you like to actually run in the local S3 scenario? I see the following possibilities:

Use only the recently merged NewTree and emulate the corresponding tree service calls using the existing local client, by directly invoking the tree service methods. This seems error prone and needs to be kept in sync.
Use layer.NewLayer and provide a test implementation of frostfs.FrostFS along with a tree service implementation based on engine. This also seems error prone.
Same as 2. but use the production frostfs.FrostFS. However, this requires that we are able to override the pool.Pool instance somehow.

Neither seems quite optimal to me, so hoping to hear your thoughts.

/cc @alexvanin @dkirillov

@fyrchik Could you clarify how much of S3 would you like to actually run in the local S3 scenario? I see the following possibilities: 1. Use only the recently merged [NewTree](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/src/branch/master/pkg/service/tree/tree.go#L107) and emulate the corresponding tree service calls using the existing `local` client, by directly invoking the tree service methods. This seems error prone and needs to be kept in sync. 2. Use [layer.NewLayer](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/src/commit/5c62010331ad5771fd6787ec0e1993b69c439b32/api/layer/layer.go#L271) and provide a test implementation of `frostfs.FrostFS` along with a tree service implementation based on `engine`. This also seems error prone. 3. Same as 2. but use the production [frostfs.FrostFS](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/src/commit/5c62010331ad5771fd6787ec0e1993b69c439b32/internal/frostfs/frostfs.go#L32). However, this requires that we are able to override the `pool.Pool` instance somehow. Neither seems quite optimal to me, so hoping to hear your thoughts. /cc @alexvanin @dkirillov

fyrchik commented

2023-03-29 11:59:37 +00:00

Owner

To me (2) is ok: we do not use a lot of methods, just PUT/GET and the hardest part here is to copy the logic for a tree service.

👍 1

ale64bit referenced this issue

2023-04-07 12:40:24 +00:00

[#14] Add s3 local loader #48

ale64bit referenced this issue

2023-04-10 08:01:56 +00:00

[#14] Add s3 local loader #48

fyrchik referenced this issue from a commit