2023-01-07 15:18:55 +00:00
|
|
|
<p align="center">
|
|
|
|
<img src="./.github/logo.svg" width="500px" alt="FrostFS logo">
|
|
|
|
</p>
|
2022-05-31 13:33:09 +00:00
|
|
|
<p align="center">
|
2022-12-29 14:52:55 +00:00
|
|
|
<a href="https://go.k6.io/k6">k6</a> extension to test and benchmark FrostFS related protocols.
|
2022-05-31 13:33:09 +00:00
|
|
|
</p>
|
|
|
|
|
|
|
|
---
|
2022-05-31 13:32:11 +00:00
|
|
|
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
|
|
|
|
|
2022-12-29 14:52:55 +00:00
|
|
|
# xk6-frostfs
|
2022-05-12 19:37:38 +00:00
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
# Build
|
2022-05-12 19:37:38 +00:00
|
|
|
|
|
|
|
To build a `k6` binary with this extension, first ensure you have the prerequisites:
|
|
|
|
|
|
|
|
- Go
|
|
|
|
- Git
|
|
|
|
|
|
|
|
1. Install `xk6` framework for extending `k6`:
|
|
|
|
```shell
|
|
|
|
go install go.k6.io/xk6/cmd/xk6@latest
|
|
|
|
```
|
|
|
|
|
|
|
|
2. Clone this repository
|
|
|
|
```shell
|
2023-03-07 14:53:17 +00:00
|
|
|
git clone git.frostfs.info/TrueCloudLab/xk6-frostfs
|
2022-12-29 14:52:55 +00:00
|
|
|
cd xk6-frostfs
|
2022-05-12 19:37:38 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
3. Build the binary:
|
|
|
|
```shell
|
2023-03-07 14:53:17 +00:00
|
|
|
xk6 build --with git.frostfs.info/TrueCloudLab/xk6-frostfs=.
|
2022-05-12 19:37:38 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
4. Run k6:
|
|
|
|
```shell
|
|
|
|
./k6 run test-script.js
|
|
|
|
```
|
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
# API
|
2022-05-12 19:37:38 +00:00
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
## Native
|
2022-05-12 19:37:38 +00:00
|
|
|
|
|
|
|
Create native client with `connect` method. Arguments:
|
2022-12-29 14:52:55 +00:00
|
|
|
- frostfs storage node endpoint
|
2022-06-17 12:50:24 +00:00
|
|
|
- hex encoded private key (empty value produces random key)
|
2023-01-10 07:29:50 +00:00
|
|
|
- dial timeout in seconds (0 for the default value)
|
|
|
|
- stream timeout in seconds (0 for the default value)
|
2022-05-12 19:37:38 +00:00
|
|
|
|
|
|
|
```js
|
2022-12-29 14:52:55 +00:00
|
|
|
import native from 'k6/x/frostfs/native';
|
2023-01-10 07:29:50 +00:00
|
|
|
const frostfs_cli = native.connect("s01.frostfs.devenv:8080", "", 0, 0)
|
2022-05-12 19:37:38 +00:00
|
|
|
```
|
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
### Methods
|
2022-06-17 12:50:24 +00:00
|
|
|
- `putContainer(params)`. The `params` is a dictionary (e.g.
|
|
|
|
`{acl:'public-read-write',placement_policy:'REP 3',name:'container-name',name_global_scope:'false'}`).
|
|
|
|
Returns dictionary with `success`
|
|
|
|
boolean flag, `container_id` string, and `error` string.
|
2022-05-25 08:22:40 +00:00
|
|
|
- `setBufferSize(size)`. Sets internal buffer size for data upload and
|
|
|
|
download. Default is 64 KiB.
|
2022-05-12 19:37:38 +00:00
|
|
|
- `put(container_id, headers, payload)`. Returns dictionary with `success`
|
2022-05-20 09:33:37 +00:00
|
|
|
boolean flag, `object_id` string, and `error` string.
|
|
|
|
- `get(container_id, object_id)`. Returns dictionary with `success` boolean
|
|
|
|
flag, and `error` string.
|
2022-12-29 14:52:55 +00:00
|
|
|
- `onsite(container_id, payload)`. Returns FrostFS object instance with prepared
|
|
|
|
headers. Invoke `put(headers)` method on this object to upload it into FrostFS.
|
2022-09-02 19:23:33 +00:00
|
|
|
It returns dictionary with `success` boolean flag, `object_id` string and
|
2022-05-23 11:42:48 +00:00
|
|
|
`error` string.
|
2022-05-12 19:37:38 +00:00
|
|
|
|
2023-03-14 12:45:25 +00:00
|
|
|
## Local
|
|
|
|
|
|
|
|
Create a local client with `connect` method. Arguments:
|
|
|
|
- local path to frostfs storage node configuration file
|
|
|
|
- hex encoded private key (empty value produces random key)
|
2023-04-19 07:44:45 +00:00
|
|
|
- whether to use the debug logger (warning: very verbose)
|
2023-03-14 12:45:25 +00:00
|
|
|
|
|
|
|
```js
|
|
|
|
import local from 'k6/x/frostfs/local';
|
2023-04-19 07:44:45 +00:00
|
|
|
const local_client = local.connect("/path/to/config.yaml", "", false)
|
2023-03-14 12:45:25 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
### Methods
|
|
|
|
- `put(container_id, headers, payload)`. Returns dictionary with `success`
|
|
|
|
boolean flag, `object_id` string, and `error` string.
|
|
|
|
- `get(container_id, object_id)`. Returns dictionary with `success` boolean
|
|
|
|
flag, and `error` string.
|
|
|
|
- `delete(container_id, object_id)`. Returns dictionary with `success` boolean
|
|
|
|
flag, and `error` string.
|
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
## S3
|
2022-05-12 19:37:38 +00:00
|
|
|
|
|
|
|
Create s3 client with `connect` method. Arguments:
|
|
|
|
- s3 gateway endpoint
|
|
|
|
|
|
|
|
Credentials are taken from default AWS configuration files and ENVs.
|
|
|
|
|
|
|
|
```js
|
2022-12-29 14:52:55 +00:00
|
|
|
import s3 from 'k6/x/frostfs/s3';
|
|
|
|
const s3_cli = s3.connect("http://s3.frostfs.devenv:8080")
|
2022-05-12 19:37:38 +00:00
|
|
|
```
|
|
|
|
|
2022-11-30 14:47:31 +00:00
|
|
|
You can also provide additional options:
|
|
|
|
```js
|
2022-12-29 14:52:55 +00:00
|
|
|
import s3 from 'k6/x/frostfs/s3';
|
|
|
|
const s3_cli = s3.connect("http://s3.frostfs.devenv:8080", {'no_verify_ssl': 'true', 'timeout': '60s'})
|
2022-11-30 14:47:31 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
* `no_verify_ss` - Bool. If `true` - skip verifying the s3 certificate chain and host name (useful if s3 uses self-signed certificates)
|
|
|
|
* `timeout` - Duration. Set timeout for requests (in http client). If omitted or zero - timeout is infinite.
|
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
### Methods
|
2022-06-17 10:37:43 +00:00
|
|
|
- `createBucket(bucket, params)`. Returns dictionary with `success` boolean flag
|
|
|
|
and `error` string. The `params` is a dictionary (e.g. `{acl:'private',lock_enabled:'true',location_constraint:'ru'}`)
|
2022-05-20 09:33:37 +00:00
|
|
|
- `put(bucket, key, payload)`. Returns dictionary with `success` boolean flag
|
|
|
|
and `error` string.
|
|
|
|
- `get(bucket, key)`. Returns dictionary with `success` boolean flag and `error`
|
|
|
|
string.
|
2022-05-12 19:37:38 +00:00
|
|
|
|
2023-03-31 10:30:33 +00:00
|
|
|
## S3 Local
|
|
|
|
|
|
|
|
Create local s3 client with `connect` method. Arguments:
|
|
|
|
- local path to frostfs storage node configuration file
|
|
|
|
- parameter map with the following options:
|
|
|
|
* `hex_key`: private key to use as a hexadecimal string. A random one is created if none is provided.
|
|
|
|
* `node_position`: position of this node in the node array if loading multiple nodes independently (default: 0).
|
|
|
|
* `node_count`: number of nodes in the node array if loading multiple nodes independently (default: 1).
|
2023-04-19 07:44:45 +00:00
|
|
|
* `debug_logger`: whether to use the development logger instead of the default. Helpful for debugging (default: false).
|
2023-03-31 10:30:33 +00:00
|
|
|
- bucket-container mapping, which is needed to resolve the container id for a given bucket name. Any bucket
|
|
|
|
used by the client must have an entry here.
|
|
|
|
|
|
|
|
```js
|
|
|
|
import local from 'k6/x/frostfs/local';
|
|
|
|
const params = {'node_position': 1, 'node_count': 3}
|
|
|
|
const bucketMapping = {'mytestbucket': 'GBQDDUM1hdodXmiRHV57EUkFWJzuntsG8BG15wFSwam6'}
|
|
|
|
const local_client = local.connect("/path/to/config.yaml", params, bucketMapping)
|
|
|
|
```
|
|
|
|
|
|
|
|
### Methods
|
|
|
|
- `put(bucket, key, payload)`. Returns dictionary with `success` boolean flag
|
|
|
|
and `error` string.
|
|
|
|
- `get(bucket, key)`. Returns dictionary with `success` boolean flag and `error`
|
|
|
|
string.
|
|
|
|
|
2022-06-01 06:45:28 +00:00
|
|
|
# Examples
|
2022-05-12 19:37:38 +00:00
|
|
|
|
2023-03-31 10:30:33 +00:00
|
|
|
See native protocol and s3 test suite examples in [examples](./examples) dir.
|
2022-05-31 13:32:11 +00:00
|
|
|
|
|
|
|
# License
|
|
|
|
|
|
|
|
- [GNU General Public License v3.0](LICENSE)
|