k6 extension to test and benchmark FrostFS related protocols.
--- [](https://www.gnu.org/licenses/gpl-3.0) # xk6-frostfs # Build To build a `k6` binary with this extension, first ensure you have the prerequisites: - Go - Git 1. Install `xk6` framework for extending `k6`: ```shell go install go.k6.io/xk6/cmd/xk6@latest ``` 2. Clone this repository ```shell git clone git.frostfs.info/TrueCloudLab/xk6-frostfs cd xk6-frostfs ``` 3. Build the binary: ```shell xk6 build --with git.frostfs.info/TrueCloudLab/xk6-frostfs=. ``` 4. Run k6: ```shell ./k6 run test-script.js ``` # API ## Native Create native client with `connect` method. Arguments: - frostfs storage node endpoint - hex encoded private key (empty value produces random key) - dial timeout in seconds (0 for the default value) - stream timeout in seconds (0 for the default value) - generate object header on the client side (for big object - split locally too) - max size for generated object header on the client side (for big object - the size that the object is splitted into) ```js import native from 'k6/x/frostfs/native'; const frostfs_cli = native.connect("s01.frostfs.devenv:8080", "", 0, 0, false, 0) ``` ### Methods - `putContainer(params)`. The `params` is a dictionary (e.g. `{acl:'public-read-write',placement_policy:'REP 3',name:'container-name',name_global_scope:'false'}`). Returns dictionary with `success` boolean flag, `container_id` string, and `error` string. - `setBufferSize(size)`. Sets internal buffer size for data upload and download. Default is 64 KiB. - `put(container_id, headers, payload)`. Returns dictionary with `success` boolean flag, `object_id` string, and `error` string. - `get(container_id, object_id)`. Returns dictionary with `success` boolean flag, and `error` string. - `onsite(container_id, payload)`. Returns FrostFS object instance with prepared headers. Invoke `put(headers)` method on this object to upload it into FrostFS. It returns dictionary with `success` boolean flag, `object_id` string and `error` string. ## Local Create a local client with `connect` method. Arguments: - local path to frostfs storage node configuration file - local path to frostfs storage node configuration directory - hex encoded private key (empty value produces random key) - whether to use the debug logger (warning: very verbose) ```js import local from 'k6/x/frostfs/local'; const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", "", false) ``` ### Methods - `put(container_id, headers, payload)`. Returns dictionary with `success` boolean flag, `object_id` string, and `error` string. - `get(container_id, object_id)`. Returns dictionary with `success` boolean flag, and `error` string. - `delete(container_id, object_id)`. Returns dictionary with `success` boolean flag, and `error` string. ## S3 Create s3 client with `connect` method. Arguments: - s3 gateway endpoint Credentials are taken from default AWS configuration files and ENVs. ```js import s3 from 'k6/x/frostfs/s3'; const s3_cli = s3.connect("https://s3.frostfs.devenv:8080") ``` You can also provide additional options: ```js import s3 from 'k6/x/frostfs/s3'; const s3_cli = s3.connect("https://s3.frostfs.devenv:8080", {'no_verify_ssl': 'true', 'timeout': '60s'}) ``` * `no_verify_ss` - Bool. If `true` - skip verifying the s3 certificate chain and host name (useful if s3 uses self-signed certificates) * `timeout` - Duration. Set timeout for requests (in http client). If omitted or zero - timeout is infinite. ### Methods - `createBucket(bucket, params)`. Returns dictionary with `success` boolean flag and `error` string. The `params` is a dictionary (e.g. `{acl:'private',lock_enabled:'true',location_constraint:'ru'}`) - `put(bucket, key, payload)`. Returns dictionary with `success` boolean flag and `error` string. - `get(bucket, key)`. Returns dictionary with `success` boolean flag and `error` string. ## S3 Local Create local s3 client with `connect` method. Arguments: - local path to frostfs storage node configuration file - local path to frostfs storage node configuration directory - parameter map with the following options: * `hex_key`: private key to use as a hexadecimal string. A random one is created if none is provided. * `node_position`: position of this node in the node array if loading multiple nodes independently (default: 0). * `node_count`: number of nodes in the node array if loading multiple nodes independently (default: 1). * `debug_logger`: whether to use the development logger instead of the default. Helpful for debugging (default: false). - bucket-container mapping, which is needed to resolve the container id for a given bucket name. Any bucket used by the client must have an entry here. ```js import local from 'k6/x/frostfs/local'; const params = {'node_position': 1, 'node_count': 3} const bucketMapping = {'mytestbucket': 'GBQDDUM1hdodXmiRHV57EUkFWJzuntsG8BG15wFSwam6'} const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", params, bucketMapping) ``` ### Methods - `put(bucket, key, payload)`. Returns dictionary with `success` boolean flag and `error` string. - `get(bucket, key)`. Returns dictionary with `success` boolean flag and `error` string. # Examples See native protocol and s3 test suite examples in [examples](./examples) dir. # Command line utils To build all command line utils just run: ```shell $ make ``` All binaries will be in `bin` directory. ## Export registry db You can export registry bolt db to json file, that can be used as pregen for scenarios (see [docs](./scenarios/run_scenarios.md)). To do this use `frostfs-xk6-registry-exporter`, available flags can be seen in help: ```shell $ ./bin/frostfs-xk6-registry-exporter -h Registry exporter for xk6 Usage: registry-exporter [flags] Examples: registry-exporter registry.bolt registry-exporter --status created --out out.json registry.bolt Flags: --age int Object age --format string Output format (default "json") -h, --help help for registry-exporter --out string Path to output file (default "dumped-registry.json") --status string Object status (default "created") -v, --version version for registry-exporter ``` ## Import pregen into registry db You can import pregenerated json files into registry bolt db. Use `frostfs-xk6-registry import`. Usage examples are in help: ```shell $ ./bin/frostfs-xk6-registry import -h Import objects into registry from pregenerated files Usage: xk6-registry import [flags] Examples: xk6-registry import registry.bolt preset.json xk6-registry import registry.bolt preset.json another_preset.json Flags: -h, --help help for import ``` # License - [GNU General Public License v3.0](LICENSE)