Abstract network communication in TreeClient #59

New Issue

Closed

opened 2023-03-15 07:32:19 +00:00 by alexvanin · 2 comments

alexvanin commented

2023-03-15 07:32:19 +00:00

Owner

In TrueCloudLab/xk6-frostfs#14 K6 generator wants to fill tree service with data as fast and as close to real S3 Gateway as possible. To do that, K6 generator may use TreeClient instance from internal/frostfs/tree.go. However, TreeClient is set to use real gRPC connection and sign gRPC requests, which is not quite optimal for original issue.

TreeClient implementation can use an interface for network communication with tree service, and K6 can implement such interface with direct access to the tree forest structures in the file system.

Besides the interface, we have to move frostfs package out of internal package, so it can be imported into K6.

Let's discuss, is it exactly what we want to do or maybe there are different proposals. /cc @fyrchik @ale64bit

Also I invite @dkirillov to give us some code examples how it can be implemented.

In https://git.frostfs.info/TrueCloudLab/xk6-frostfs/issues/14 K6 generator wants to fill tree service with data as fast and as close to real S3 Gateway as possible. To do that, K6 generator may use `TreeClient` instance from `internal/frostfs/tree.go`. However, `TreeClient` is set to use real gRPC connection and sign gRPC requests, which is not quite optimal for original issue. `TreeClient` implementation can use an interface for network communication with tree service, and K6 can implement such interface with direct access to the tree forest structures in the file system. Besides the interface, we have to move `frostfs` package out of `internal` package, so it can be imported into K6. Let's discuss, is it exactly what we want to do or maybe there are different proposals. /cc @fyrchik @ale64bit Also I invite @dkirillov to give us some code examples how it can be implemented.

dkirillov commented

2023-03-15 12:42:50 +00:00

Collaborator

Proposed interface that should be implemented

type Meta interface {
	GetKey() string
	GetValue() []byte
}

type NodeResponse interface {
	GetMeta() []Meta
	GetNodeID() uint64
	GetParentID() uint64
	GetTimestamp() uint64
}

type TreeServiceClient interface {
	GetNodes(ctx context.Context, p *GetNodesParams) ([]NodeResponse, error)
    
	GetSubTree(ctx context.Context, bkt *data.BucketInfo, treeID string, rootID uint64, depth uint32) ([]NodeResponse, error)
    
	AddNode(ctx context.Context, bkt *data.BucketInfo, treeID string, parent uint64, meta map[string]string) (uint64, error)
    
	AddNodeByPath(ctx context.Context, bkt *data.BucketInfo, treeID string, path []string, meta map[string]string) (uint64, error)
    
	MoveNode(ctx context.Context, bkt *data.BucketInfo, treeID string, nodeID, parentID uint64, meta map[string]string) error
    
	RemoveNode(ctx context.Context, bkt *data.BucketInfo, treeID string, nodeID uint64) error
}

data.BucketInfo is a struct that contains some container info. In current implementaion we use only two field from this struct CID and Owner so probably we can define a new struct with only these two fields

The tree client can be used then as follow:

	treeGRPCClient, _ := frostfs.NewTreeServiceClientGRPC(ctx, treeServiceEndpoint, a.key)
	treeService := frostfs.NewTree(treeGRPCClient)
    
    
    newVersion := &data.NodeVersion{
	BaseNodeVersion: data.BaseNodeVersion{
		OID:      objID,
		FilePath: filePath,
	},
	}
	treeService.AddVersion(ctx, bktInfo, newVersion)

where instead of treeGRPCClient we can use any other implementaion.

Proposed interface that should be implemented ``` type Meta interface { GetKey() string GetValue() []byte } type NodeResponse interface { GetMeta() []Meta GetNodeID() uint64 GetParentID() uint64 GetTimestamp() uint64 } type TreeServiceClient interface { GetNodes(ctx context.Context, p *GetNodesParams) ([]NodeResponse, error) GetSubTree(ctx context.Context, bkt *data.BucketInfo, treeID string, rootID uint64, depth uint32) ([]NodeResponse, error) AddNode(ctx context.Context, bkt *data.BucketInfo, treeID string, parent uint64, meta map[string]string) (uint64, error) AddNodeByPath(ctx context.Context, bkt *data.BucketInfo, treeID string, path []string, meta map[string]string) (uint64, error) MoveNode(ctx context.Context, bkt *data.BucketInfo, treeID string, nodeID, parentID uint64, meta map[string]string) error RemoveNode(ctx context.Context, bkt *data.BucketInfo, treeID string, nodeID uint64) error } ``` `data.BucketInfo` is a struct that contains some container info. In current implementaion we use only two field from this struct `CID` and `Owner` so probably we can define a new struct with only these two fields The tree client can be used then as follow: ``` treeGRPCClient, _ := frostfs.NewTreeServiceClientGRPC(ctx, treeServiceEndpoint, a.key) treeService := frostfs.NewTree(treeGRPCClient) newVersion := &data.NodeVersion{ BaseNodeVersion: data.BaseNodeVersion{ OID: objID, FilePath: filePath, }, } treeService.AddVersion(ctx, bktInfo, newVersion) ``` where instead of `treeGRPCClient` we can use any other implementaion.

dkirillov commented

2023-03-16 13:46:45 +00:00

Collaborator

Here you can see all methods in addition to AddVersion (from message above) that can be used 5c62010331/api/layer/tree_service.go

Here you can see all methods in addition to `AddVersion` (from message above) that can be used https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/src/commit/5c62010331ad5771fd6787ec0e1993b69c439b32/api/layer/tree_service.go