Ekaterina Lebedeva
4cd50c8c03
All checks were successful
DCO action / DCO (pull_request) Successful in 45s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m3s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m4s
Tests and linters / Tests with -race (pull_request) Successful in 2m13s
Tests and linters / Lint (pull_request) Successful in 3m29s
* Currently, objects created in preset are never deleted. k6 deletes only objects from registry, if registry file is not provided k6 delete load fails. * Added cli utility to import objects created in preset into registry so k6 can delete them normally. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com> |
||
---|---|---|
.forgejo | ||
cmd | ||
examples | ||
internal | ||
scenarios | ||
.gitignore | ||
CONTRIBUTING.md | ||
frostfs.go | ||
go.mod | ||
go.sum | ||
help.mk | ||
LICENSE | ||
Makefile | ||
README.md |
k6 extension to test and benchmark FrostFS related protocols.
xk6-frostfs
Build
To build a k6
binary with this extension, first ensure you have the prerequisites:
- Go
- Git
- Install
xk6
framework for extendingk6
:
go install go.k6.io/xk6/cmd/xk6@latest
- Clone this repository
git clone git.frostfs.info/TrueCloudLab/xk6-frostfs
cd xk6-frostfs
- Build the binary:
xk6 build --with git.frostfs.info/TrueCloudLab/xk6-frostfs=.
- Run k6:
./k6 run test-script.js
API
Native
Create native client with connect
method. Arguments:
- frostfs storage node endpoint
- hex encoded private key (empty value produces random key)
- dial timeout in seconds (0 for the default value)
- stream timeout in seconds (0 for the default value)
- generate object header on the client side (for big object - split locally too)
- max size for generated object header on the client side (for big object - the size that the object is splitted into)
import native from 'k6/x/frostfs/native';
const frostfs_cli = native.connect("s01.frostfs.devenv:8080", "", 0, 0, false, 0)
Methods
putContainer(params)
. Theparams
is a dictionary (e.g.{acl:'public-read-write',placement_policy:'REP 3',name:'container-name',name_global_scope:'false'}
). Returns dictionary withsuccess
boolean flag,container_id
string, anderror
string.setBufferSize(size)
. Sets internal buffer size for data upload and download. Default is 64 KiB.put(container_id, headers, payload)
. Returns dictionary withsuccess
boolean flag,object_id
string, anderror
string.get(container_id, object_id)
. Returns dictionary withsuccess
boolean flag, anderror
string.onsite(container_id, payload)
. Returns FrostFS object instance with prepared headers. Invokeput(headers)
method on this object to upload it into FrostFS. It returns dictionary withsuccess
boolean flag,object_id
string anderror
string.
Local
Create a local client with connect
method. Arguments:
- local path to frostfs storage node configuration file
- local path to frostfs storage node configuration directory
- hex encoded private key (empty value produces random key)
- whether to use the debug logger (warning: very verbose)
import local from 'k6/x/frostfs/local';
const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", "", false)
Methods
put(container_id, headers, payload)
. Returns dictionary withsuccess
boolean flag,object_id
string, anderror
string.get(container_id, object_id)
. Returns dictionary withsuccess
boolean flag, anderror
string.delete(container_id, object_id)
. Returns dictionary withsuccess
boolean flag, anderror
string.
S3
Create s3 client with connect
method. Arguments:
- s3 gateway endpoint
Credentials are taken from default AWS configuration files and ENVs.
import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080")
You can also provide additional options:
import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080", {'no_verify_ssl': 'true', 'timeout': '60s'})
no_verify_ss
- Bool. Iftrue
- skip verifying the s3 certificate chain and host name (useful if s3 uses self-signed certificates)timeout
- Duration. Set timeout for requests (in http client). If omitted or zero - timeout is infinite.
Methods
createBucket(bucket, params)
. Returns dictionary withsuccess
boolean flag anderror
string. Theparams
is a dictionary (e.g.{acl:'private',lock_enabled:'true',location_constraint:'ru'}
)put(bucket, key, payload)
. Returns dictionary withsuccess
boolean flag anderror
string.get(bucket, key)
. Returns dictionary withsuccess
boolean flag anderror
string.
S3 Local
Create local s3 client with connect
method. Arguments:
- local path to frostfs storage node configuration file
- local path to frostfs storage node configuration directory
- parameter map with the following options:
hex_key
: private key to use as a hexadecimal string. A random one is created if none is provided.node_position
: position of this node in the node array if loading multiple nodes independently (default: 0).node_count
: number of nodes in the node array if loading multiple nodes independently (default: 1).debug_logger
: whether to use the development logger instead of the default. Helpful for debugging (default: false).
- bucket-container mapping, which is needed to resolve the container id for a given bucket name. Any bucket used by the client must have an entry here.
import local from 'k6/x/frostfs/local';
const params = {'node_position': 1, 'node_count': 3}
const bucketMapping = {'mytestbucket': 'GBQDDUM1hdodXmiRHV57EUkFWJzuntsG8BG15wFSwam6'}
const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", params, bucketMapping)
Methods
put(bucket, key, payload)
. Returns dictionary withsuccess
boolean flag anderror
string.get(bucket, key)
. Returns dictionary withsuccess
boolean flag anderror
string.
Examples
See native protocol and s3 test suite examples in examples dir.
Command line utils
To build all command line utils just run:
$ make
All binaries will be in bin
directory.
Export registry db
You can export registry bolt db to json file, that can be used as pregen for scenarios (see docs).
To do this use frostfs-xk6-registry-exporter
, available flags can be seen in help:
$ ./bin/frostfs-xk6-registry-exporter -h
Registry exporter for xk6
Usage:
registry-exporter [flags]
Examples:
registry-exporter registry.bolt
registry-exporter --status created --out out.json registry.bolt
Flags:
--age int Object age
--format string Output format (default "json")
-h, --help help for registry-exporter
--out string Path to output file (default "dumped-registry.json")
--status string Object status (default "created")
-v, --version version for registry-exporter