k6 extension to test and benchmark FrostFS related protocols.
Find a file
Мария Малыгина 560c60513e support multipart upload
Signed-off-by: Мария Малыгина <m.malygina@yadro.com>
2023-07-18 13:06:38 +03:00
.github [#39] Add bug label; edit feature request 2023-03-24 06:22:26 +00:00
examples [#72] Add option --prepare-locally 2023-07-07 13:16:54 +03:00
internal support multipart upload 2023-07-18 13:06:38 +03:00
scenarios support multipart upload 2023-07-18 13:06:38 +03:00
.gitignore [#30] gitignore: Add dir for presets 2022-11-03 13:11:50 +03:00
CONTRIBUTING.md [#2] Deep rebranding 2022-12-30 11:05:20 +03:00
frostfs.go [#20] Add pprof extension with support for cpu and mem 2023-05-18 15:08:33 +00:00
go.mod support multipart upload 2023-07-18 13:06:38 +03:00
go.sum support multipart upload 2023-07-18 13:06:38 +03:00
LICENSE [#8] Add LICENSE 2022-06-01 17:09:34 +03:00
README.md [#81] remove schema from preset_s3 and k6 load s3 scenarios 2023-07-14 11:36:10 +00:00

FrostFS logo

k6 extension to test and benchmark FrostFS related protocols.


License: GPL v3

xk6-frostfs

Build

To build a k6 binary with this extension, first ensure you have the prerequisites:

  • Go
  • Git
  1. Install xk6 framework for extending k6:
go install go.k6.io/xk6/cmd/xk6@latest
  1. Clone this repository
git clone git.frostfs.info/TrueCloudLab/xk6-frostfs
cd xk6-frostfs
  1. Build the binary:
xk6 build --with git.frostfs.info/TrueCloudLab/xk6-frostfs=.
  1. Run k6:
./k6 run test-script.js

API

Native

Create native client with connect method. Arguments:

  • frostfs storage node endpoint
  • hex encoded private key (empty value produces random key)
  • dial timeout in seconds (0 for the default value)
  • stream timeout in seconds (0 for the default value)
  • generate object header on the client side (for big object - split locally too)
import native from 'k6/x/frostfs/native';
const frostfs_cli = native.connect("s01.frostfs.devenv:8080", "", 0, 0, false)

Methods

  • putContainer(params). The params is a dictionary (e.g. {acl:'public-read-write',placement_policy:'REP 3',name:'container-name',name_global_scope:'false'}). Returns dictionary with success boolean flag, container_id string, and error string.
  • setBufferSize(size). Sets internal buffer size for data upload and download. Default is 64 KiB.
  • put(container_id, headers, payload). Returns dictionary with success boolean flag, object_id string, and error string.
  • get(container_id, object_id). Returns dictionary with success boolean flag, and error string.
  • onsite(container_id, payload). Returns FrostFS object instance with prepared headers. Invoke put(headers) method on this object to upload it into FrostFS. It returns dictionary with success boolean flag, object_id string and error string.

Local

Create a local client with connect method. Arguments:

  • local path to frostfs storage node configuration file
  • hex encoded private key (empty value produces random key)
  • whether to use the debug logger (warning: very verbose)
import local from 'k6/x/frostfs/local';
const local_client = local.connect("/path/to/config.yaml", "", false)

Methods

  • put(container_id, headers, payload). Returns dictionary with success boolean flag, object_id string, and error string.
  • get(container_id, object_id). Returns dictionary with success boolean flag, and error string.
  • delete(container_id, object_id). Returns dictionary with success boolean flag, and error string.

S3

Create s3 client with connect method. Arguments:

  • s3 gateway endpoint

Credentials are taken from default AWS configuration files and ENVs.

import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080")

You can also provide additional options:

import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080", {'no_verify_ssl': 'true', 'timeout': '60s'})
  • no_verify_ss - Bool. If true - skip verifying the s3 certificate chain and host name (useful if s3 uses self-signed certificates)
  • timeout - Duration. Set timeout for requests (in http client). If omitted or zero - timeout is infinite.

Methods

  • createBucket(bucket, params). Returns dictionary with success boolean flag and error string. The params is a dictionary (e.g. {acl:'private',lock_enabled:'true',location_constraint:'ru'})
  • put(bucket, key, payload). Returns dictionary with success boolean flag and error string.
  • get(bucket, key). Returns dictionary with success boolean flag and error string.

S3 Local

Create local s3 client with connect method. Arguments:

  • local path to frostfs storage node configuration file
  • parameter map with the following options:
    • hex_key: private key to use as a hexadecimal string. A random one is created if none is provided.
    • node_position: position of this node in the node array if loading multiple nodes independently (default: 0).
    • node_count: number of nodes in the node array if loading multiple nodes independently (default: 1).
    • debug_logger: whether to use the development logger instead of the default. Helpful for debugging (default: false).
  • bucket-container mapping, which is needed to resolve the container id for a given bucket name. Any bucket used by the client must have an entry here.
import local from 'k6/x/frostfs/local';
const params = {'node_position': 1, 'node_count': 3}
const bucketMapping = {'mytestbucket': 'GBQDDUM1hdodXmiRHV57EUkFWJzuntsG8BG15wFSwam6'}
const local_client = local.connect("/path/to/config.yaml", params, bucketMapping)

Methods

  • put(bucket, key, payload). Returns dictionary with success boolean flag and error string.
  • get(bucket, key). Returns dictionary with success boolean flag and error string.

Examples

See native protocol and s3 test suite examples in examples dir.

License