xk6-frostfs/README.md

<p align="center">
<img src="./.forgejo/logo.svg" width="500px" alt="FrostFS logo">
</p>
<p align="center">
  <a href="https://go.k6.io/k6">k6</a> extension to test and benchmark FrostFS related protocols.
</p>

---
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)

# xk6-frostfs

# Build

To build a `k6` binary with this extension, first ensure you have the prerequisites:

- Go
- Git

1. Install `xk6` framework for extending `k6`:
```shell
go install go.k6.io/xk6/cmd/xk6@latest
```

2. Clone this repository
```shell
git clone git.frostfs.info/TrueCloudLab/xk6-frostfs
cd xk6-frostfs
```

3. Build the binary:
```shell
xk6 build --with git.frostfs.info/TrueCloudLab/xk6-frostfs=.
```

4. Run k6:
```shell
./k6 run test-script.js
```

# API

## Native

Create native client with `connect` method. Arguments:
- frostfs storage node endpoint
- hex encoded private key (empty value produces random key)
- dial timeout in seconds (0 for the default value)
- stream timeout in seconds (0 for the default value)
- generate object header on the client side (for big object - split locally too)
- max size for generated object header on the client side (for big object - the size that the object is splitted into)

```js
import native from 'k6/x/frostfs/native';
const frostfs_cli = native.connect("s01.frostfs.devenv:8080", "", 0, 0, false, 0)
```

### Methods
- `putContainer(params)`. The `params` is a dictionary (e.g. 
  `{placement_policy:'REP 3',name:'container-name',name_global_scope:'false'}`). 
  Returns dictionary with `success`
  boolean flag, `container_id` string, and `error` string.
- `setBufferSize(size)`. Sets internal buffer size for data upload and 
  download. Default is 64 KiB.
- `put(container_id, headers, payload)`. Returns dictionary with `success` 
  boolean flag, `object_id` string, and `error` string.
- `get(container_id, object_id)`. Returns dictionary with `success` boolean
  flag, and `error` string.
- `onsite(container_id, payload)`. Returns FrostFS object instance with prepared
  headers. Invoke `put(headers)` method on this object to upload it into FrostFS.
  It returns dictionary with `success` boolean flag, `object_id` string and
  `error` string.

## Local

Create a local client with `connect` method. Arguments:
- local path to frostfs storage node configuration file
- local path to frostfs storage node configuration directory
- hex encoded private key (empty value produces random key)
- whether to use the debug logger (warning: very verbose)

```js
import local from 'k6/x/frostfs/local';
const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", "", false)
```

### Methods
- `put(container_id, headers, payload)`. Returns dictionary with `success` 
  boolean flag, `object_id` string, and `error` string.
- `get(container_id, object_id)`. Returns dictionary with `success` boolean
  flag, and `error` string.
- `delete(container_id, object_id)`. Returns dictionary with `success` boolean
  flag, and `error` string.

## S3

Create s3 client with `connect` method. Arguments:
- s3 gateway endpoint

Credentials are taken from default AWS configuration files and ENVs.

```js
import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080")
```

You can also provide additional options:
```js
import s3 from 'k6/x/frostfs/s3';
const s3_cli = s3.connect("https://s3.frostfs.devenv:8080", {'no_verify_ssl': 'true', 'timeout': '60s'})
```

* `no_verify_ss` - Bool. If `true` - skip verifying the s3 certificate chain and host name (useful if s3 uses self-signed certificates)
* `timeout` - Duration. Set timeout for requests (in http client). If omitted or zero - timeout is infinite.

### Methods
- `createBucket(bucket, params)`. Returns dictionary with `success` boolean flag
  and `error` string. The `params` is a dictionary (e.g. `{acl:'private',lock_enabled:'true',location_constraint:'ru'}`)
- `put(bucket, key, payload)`. Returns dictionary with `success` boolean flag 
  and `error` string.
- `get(bucket, key)`. Returns dictionary with `success` boolean flag and `error`
  string.

## S3 Local

Create local s3 client with `connect` method. Arguments:
- local path to frostfs storage node configuration file
- local path to frostfs storage node configuration directory
- parameter map with the following options:
  * `hex_key`: private key to use as a hexadecimal string. A random one is created if none is provided.
  * `node_position`: position of this node in the node array if loading multiple nodes independently (default: 0).
  * `node_count`: number of nodes in the node array if loading multiple nodes independently (default: 1).
  * `debug_logger`: whether to use the development logger instead of the default. Helpful for debugging (default: false).
- bucket-container mapping, which is needed to resolve the container id for a given bucket name. Any bucket
  used by the client must have an entry here.

```js
import local from 'k6/x/frostfs/local';
const params = {'node_position': 1, 'node_count': 3}
const bucketMapping = {'mytestbucket': 'GBQDDUM1hdodXmiRHV57EUkFWJzuntsG8BG15wFSwam6'}
const local_client = local.connect("/path/to/config.yaml", "/path/to/config/dir", params, bucketMapping)
```

### Methods
- `put(bucket, key, payload)`. Returns dictionary with `success` boolean flag 
  and `error` string.
- `get(bucket, key)`. Returns dictionary with `success` boolean flag and `error`
  string.

# Examples

See native protocol and s3 test suite examples in [examples](./examples) dir.

# Command line utils

To build all command line utils just run:

```shell
$ make
```

All binaries will be in `bin` directory.

## Export registry db

You can export registry bolt db to json file, that can be used as pregen for scenarios (see [docs](./scenarios/run_scenarios.md)).
To do this use `frostfs-xk6-registry-exporter`, available flags can be seen in help:

```shell
$ ./bin/frostfs-xk6-registry-exporter -h
Registry exporter for xk6

Usage:
registry-exporter [flags]

Examples:
registry-exporter registry.bolt
registry-exporter --status created --out out.json registry.bolt

Flags:
--age int         Object age
--format string   Output format (default "json")
-h, --help            help for registry-exporter
--out string      Path to output file (default "dumped-registry.json")
--status string   Object status (default "created")
-v, --version         version for registry-exporter
```

## Import pregen into registry db

You can import pregenerated json files into registry bolt db. Use `frostfs-xk6-registry import`. Usage examples are in help:

```shell
$ ./bin/frostfs-xk6-registry import -h
Import objects into registry from pregenerated files

Usage:
  xk6-registry import [flags]

Examples:
xk6-registry import registry.bolt preset.json
xk6-registry import registry.bolt preset.json another_preset.json

Flags:
  -h, --help   help for import
```

# License

- [GNU General Public License v3.0](LICENSE)