2021-08-18 19:18:49 +00:00
# Configuration
2022-04-06 14:31:32 +00:00
There are three ways to configure the S3 GW:
2022-07-18 13:06:39 +00:00
2022-04-06 14:31:32 +00:00
1. CLI parameters
2022-07-18 13:06:39 +00:00
2. YAML file
2022-04-06 14:31:32 +00:00
3. Environment variables
2021-08-18 19:18:49 +00:00
2022-07-18 13:06:39 +00:00
Everything available as a CLI parameter can also be specified via environment variables and almost everything can be
specified via `.yaml` configuration file.
2022-04-06 14:31:32 +00:00
2022-07-18 13:06:39 +00:00
But **not vice versa** , some parameters can be configured only with environment variables/configuration file.
Most of these parameters have default values, therefore, these ways to configure the gateway are optional and
2022-04-06 14:31:32 +00:00
basic configuration can be completed with CLI parameters only.
2022-04-27 07:15:46 +00:00
1. [CLI parameters ](#cli-parameters )
1. [Nodes and weights ](#nodes-and-weights )
2. [Wallet ](#wallet )
3. [Binding and TLS ](#listening-on-address-and-TLS )
4. [RPC endpoint and resolving of bucket names ](#rpc-endpoint-and-resolving-of-bucket-names )
5. [Processing of requests ](#processing-of-requests )
2022-12-15 13:37:18 +00:00
6. [Connection to FrostFS ](#connection-to-FrostFS )
2022-04-27 07:15:46 +00:00
7. [Monitoring and metrics ](#monitoring-and-metrics )
2. [YAML file and environment variables ](#yaml-file-and-environment-variables )
2022-12-15 13:37:18 +00:00
1. [Configuration file ](#frostfs-s3-gateway-configuration-file )
2022-04-06 14:31:32 +00:00
## CLI parameters
### Nodes and weights
2021-08-18 19:18:49 +00:00
2022-12-15 13:37:18 +00:00
You can specify multiple `-p` options to add more FrostFS nodes; this will make
2021-08-19 12:46:41 +00:00
a gateway spread requests equally among them (using weight 1 for every node):
2021-08-18 19:18:49 +00:00
2022-04-06 14:31:32 +00:00
```shell
2022-12-15 13:37:18 +00:00
$ frostfs-s3-gw -p 192.168.130.72:8080 -p 192.168.130.71:8080
2021-08-18 19:18:49 +00:00
```
2022-07-18 13:06:39 +00:00
2022-04-06 14:31:32 +00:00
If you want some specific load distribution proportions, use weights and priorities, they
2022-04-13 16:56:58 +00:00
can only be specified via environment variables or a configuration file.
2021-08-18 19:18:49 +00:00
2022-04-06 14:31:32 +00:00
### Wallet
2021-08-18 19:18:49 +00:00
2022-07-18 13:06:39 +00:00
Wallet (`--wallet`) is a mandatory parameter. It is a path to a wallet file. You can provide a passphrase to decrypt
2022-04-06 14:31:32 +00:00
a wallet via env variable or conf file, or you will be asked to enter a password interactively.
You can also specify an account address to use from a wallet using the `--address` parameter.
2021-08-18 19:18:49 +00:00
2022-04-06 14:31:32 +00:00
### Listening on address and TLS
2021-08-18 19:18:49 +00:00
2022-07-18 13:06:39 +00:00
You can make the gateway listen on specific address using the `--listen_address` option.
2021-08-18 19:18:49 +00:00
2021-08-19 12:46:41 +00:00
It can also provide TLS interface for its users, just specify paths to the key and
2021-08-18 19:18:49 +00:00
certificate files via `--tls.key_file` and `--tls.cert_file` parameters. Note
2022-04-13 16:56:58 +00:00
that using these options makes gateway TLS-only. If you need to serve both TLS
and plain text, you either have to run two gateway instances or use some
2021-08-18 19:18:49 +00:00
external redirecting solution.
2022-04-06 14:31:32 +00:00
Example to bind to `192.168.130.130:443` and serve TLS there (keys and nodes are
2021-08-18 19:18:49 +00:00
omitted):
2022-04-06 14:31:32 +00:00
```shell
2022-12-15 13:37:18 +00:00
$ frostfs-s3-gw --listen_address 192.168.130.130:443 \
2021-08-18 19:18:49 +00:00
--tls.key_file=key.pem --tls.cert_file=cert.pem
```
2023-03-24 12:49:23 +00:00
Using these flag you can configure only one address. To set multiple addresses use yaml config.
2022-11-09 11:53:26 +00:00
2022-04-06 14:31:32 +00:00
### RPC endpoint and resolving of bucket names
2021-08-18 19:18:49 +00:00
2022-07-18 13:06:39 +00:00
To set RPC endpoint specify a value of parameter `-r` or `--rpc_endpoint` . The parameter is **required if** another
2022-04-14 18:10:57 +00:00
parameter's `--resolve_order` value contains `nns` .
2021-08-18 19:18:49 +00:00
2022-04-06 14:31:32 +00:00
```shell
2022-12-15 13:37:18 +00:00
$ frostfs-s3-gw --rpc_endpoint http://morph-chain.frostfs.devenv:30333/ --resolve_order nns,dns
2021-08-18 19:18:49 +00:00
```
2022-04-06 14:31:32 +00:00
### Processing of requests
2022-07-18 13:06:39 +00:00
Maximum number of clients whose requests can be handled by the gateway can be specified by the value of
`--max_clients_count` parameter.
`--max_clients_deadline` defines deadline after which the gate sends error `RequestTimeout` to a client.
2022-04-06 14:31:32 +00:00
```shell
2022-12-15 13:37:18 +00:00
$ frostfs-s3-gw --max_clients_count 150 --max_clients_deadline 1m
2021-08-18 19:18:49 +00:00
```
2022-12-15 13:37:18 +00:00
### Connection to FrostFS
2021-08-18 19:18:49 +00:00
2022-12-15 13:37:18 +00:00
Timeout to connect to FrostFS nodes can be set with `--connect_timeout`
2022-07-18 13:06:39 +00:00
and timeout to check node health during rebalance`--healthcheck_timeout`.
2021-08-18 19:18:49 +00:00
2022-07-18 13:06:39 +00:00
Also, interval to check node health can be specified by `--rebalance_interval` value.
2021-08-18 19:18:49 +00:00
2022-04-06 14:31:32 +00:00
```shell
2022-12-15 13:37:18 +00:00
$ frostfs-s3-gw --healthcheck_timeout 15s --connect_timeout 1m --rebalance_interval 1h
2021-08-18 19:18:49 +00:00
```
2022-04-06 14:31:32 +00:00
### Monitoring and metrics
2021-08-18 19:18:49 +00:00
2022-07-18 13:06:39 +00:00
Pprof and Prometheus are integrated into the gateway. To enable them, use `--pprof` and `--metrics` flags or
2022-11-09 11:53:26 +00:00
`S3_GW_PPROF_ENABLED` /`S3_GW_PROMETHEUS_ENABLED` environment variables.
2021-08-23 19:06:26 +00:00
2022-04-06 14:31:32 +00:00
## YAML file and environment variables
2021-08-23 19:06:26 +00:00
2022-07-18 13:06:39 +00:00
Example of a YAML configuration file: [yaml-example ](/config/config.yaml )
Examples of environment variables: [env-example ](/config/config.env ).
2021-08-23 19:06:26 +00:00
2022-04-06 14:31:32 +00:00
A path to a configuration file can be specified with `--config` parameter:
2021-08-18 19:18:49 +00:00
2022-04-06 14:31:32 +00:00
```shell
2022-12-15 13:37:18 +00:00
$ frostfs-s3-gw --config your-config.yaml
2021-08-18 19:18:49 +00:00
```
2022-04-06 14:31:32 +00:00
2023-02-03 10:00:18 +00:00
### Multiple configs
You can use several config files when running application. It allows you to split configuration into parts.
For example, you can use separate yaml file for pprof and prometheus section in config (see [config examples ](../config )).
You can either provide several files with repeating `--config` flag or provide path to the dir that contains all configs using `--config-dir` flag.
Also, you can combine these flags:
```shell
$ frostfs-s3-gw --config ./config/config.yaml --config /your/partial/config.yaml --config-dir ./config/dir
```
**Note:** next file in `--config` flag overwrites values from the previous one.
Files from `--config-dir` directory overwrite values from `--config` files.
So the command above run `frostfs-s3-gw` to listen on `0.0.0.0:8080` address (value from `./config/config.yaml` ),
applies parameters from `/your/partial/config.yaml` ,
enable pprof (value from `./config/dir/pprof.yaml` ) and prometheus (value from `./config/dir/prometheus.yaml` ).
2022-09-30 14:37:01 +00:00
### Reload on SIGHUP
2023-03-24 12:49:23 +00:00
Some config values can be reloaded on SIGHUP signal.
2022-09-30 14:37:01 +00:00
Such parameters have special mark in tables below.
You can send SIGHUP signal to app using the following command:
```shell
$ kill -s SIGHUP < app_pid >
```
Example:
```shell
2022-12-15 13:37:18 +00:00
$ ./bin/frostfs-s3-gw --config config.yaml & > s3.log &
2022-09-30 14:37:01 +00:00
[1] 998346
$ cat s3.log
# ...
2022-12-15 13:37:18 +00:00
2022-09-30T17:38:22.338+0300 info s3-gw/app.go:371 application started {"name": "frostfs-s3-gw", "version": "v0.24.0"}
2022-09-30 14:37:01 +00:00
# ...
$ kill -s SIGHUP 998346
$ cat s3.log
# ...
2022-09-30T17:38:40.909+0300 info s3-gw/app.go:491 SIGHUP config reload completed
```
2022-12-15 13:37:18 +00:00
### FrostFS S3 Gateway configuration file
2022-07-18 13:06:39 +00:00
2022-12-15 13:37:18 +00:00
This section contains detailed FrostFS S3 Gateway configuration file description
2022-07-18 13:06:39 +00:00
including default config values and some tips to set up configurable values.
There are some custom types used for brevity:
* `duration` -- string consisting of a number and a suffix. Suffix examples include `s` (seconds), `m` (minutes), `ms` (
milliseconds).
### Structure
2023-02-10 12:21:25 +00:00
| Section | Description |
|--------------------|----------------------------------------------------------------|
| no section | [General parameters ](#general-section ) |
| `wallet` | [Wallet configuration ](#wallet-section ) |
| `peers` | [Nodes configuration ](#peers-section ) |
| `placement_policy` | [Placement policy configuration ](#placement_policy-section ) |
| `server` | [Server configuration ](#server-section ) |
| `logger` | [Logger configuration ](#logger-section ) |
2024-07-12 12:31:43 +00:00
| `http_logging` | [HTTP Request logger configuration ](#http_logging-section ) |
2023-02-10 12:21:25 +00:00
| `cache` | [Cache configuration ](#cache-section ) |
| `cors` | [CORS configuration ](#cors-section ) |
| `pprof` | [Pprof configuration ](#pprof-section ) |
| `prometheus` | [Prometheus configuration ](#prometheus-section ) |
2023-05-31 16:35:20 +00:00
| `tracing` | [Tracing configuration ](#tracing-section ) |
2023-02-10 12:21:25 +00:00
| `frostfs` | [Parameters of requests to FrostFS ](#frostfs-section ) |
| `resolve_bucket` | [Bucket name resolving configuration ](#resolve_bucket-section ) |
2023-03-02 14:54:33 +00:00
| `kludge` | [Different kludge configuration ](#kludge-section ) |
2023-08-30 18:24:22 +00:00
| `runtime` | [Runtime configuration ](#runtime-section ) |
2023-10-02 08:52:07 +00:00
| `features` | [Features configuration ](#features-section ) |
2023-10-26 13:44:40 +00:00
| `web` | [Web server configuration ](#web-section ) |
2023-10-05 13:25:25 +00:00
| `frostfsid` | [FrostfsID configuration ](#frostfsid-section ) |
2023-12-05 09:12:35 +00:00
| `policy` | [Policy contract configuration ](#policy-section ) |
2023-12-21 14:57:12 +00:00
| `proxy` | [Proxy contract configuration ](#proxy-section ) |
2023-11-21 08:51:07 +00:00
| `namespaces` | [Namespaces configuration ](#namespaces-section ) |
2024-05-30 13:02:27 +00:00
| `retry` | [Retry configuration ](#retry-section ) |
2024-07-15 15:35:54 +00:00
| `containers` | [Containers configuration ](#containers-section ) |
2024-07-31 06:45:46 +00:00
| `vhs` | [VHS configuration ](#vhs-section ) |
2022-07-18 13:06:39 +00:00
### General section
```yaml
2022-08-30 10:52:37 +00:00
listen_domains:
2022-12-15 13:37:18 +00:00
- s3dev.frostfs.devenv
2024-07-31 06:45:46 +00:00
- s3dev.< wildcard > .frostfs.devenv
2022-12-15 13:37:18 +00:00
- s3dev2.frostfs.devenv
2022-08-30 10:52:37 +00:00
2022-12-15 13:37:18 +00:00
rpc_endpoint: http://morph-chain.frostfs.devenv:30333
2022-07-18 13:06:39 +00:00
resolve_order:
- nns
- dns
2022-07-18 13:11:25 +00:00
connect_timeout: 10s
2022-11-15 14:19:21 +00:00
stream_timeout: 10s
2022-07-18 13:06:39 +00:00
healthcheck_timeout: 15s
2022-07-18 13:11:25 +00:00
rebalance_interval: 60s
2022-07-29 06:26:11 +00:00
pool_error_threshold: 100
2022-07-18 13:06:39 +00:00
max_clients_count: 100
max_clients_deadline: 30s
2023-03-24 12:49:23 +00:00
allowed_access_key_id_prefixes:
2022-08-31 23:12:02 +00:00
- Ck9BHsgKcnwfCTUSFm6pxhoNS4cBqgN2NQ8zVgPjqZDX
- 3stjWenX15YwYzczMr88gy3CQr4NYFBQ8P7keGzH5QFn
2024-07-12 12:31:43 +00:00
2024-02-11 18:00:56 +00:00
reconnect_interval: 1m
2024-04-17 14:08:55 +00:00
source_ip_header: "Source-Ip"
2022-07-18 13:06:39 +00:00
```
2023-10-05 13:25:25 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|----------------------------------|------------|---------------|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
2024-07-31 06:45:46 +00:00
| `listen_domains` | `[]string` | yes | | Domains to be able to use virtual-hosted-style access to bucket. The presence of placeholders of the < wildcard > type is supported. |
2023-10-05 13:25:25 +00:00
| `rpc_endpoint` | `string` | no | | The address of the RPC host to which the gateway connects to resolve bucket names and interact with frostfs contracts (required to use the `nns` resolver and `frostfsid` contract). |
| `resolve_order` | `[]string` | yes | `[dns]` | Order of bucket name resolvers to use. Available resolvers: `dns` , `nns` . |
| `connect_timeout` | `duration` | no | `10s` | Timeout to connect to a node. |
| `stream_timeout` | `duration` | no | `10s` | Timeout for individual operations in streaming RPC. |
| `healthcheck_timeout` | `duration` | no | `15s` | Timeout to check node health during rebalance. |
| `rebalance_interval` | `duration` | no | `60s` | Interval to check node health. |
| `pool_error_threshold` | `uint32` | no | `100` | The number of errors on connection after which node is considered as unhealthy. |
| `max_clients_count` | `int` | no | `100` | Limits for processing of clients' requests. |
| `max_clients_deadline` | `duration` | no | `30s` | Deadline after which the gate sends error `RequestTimeout` to a client. |
| `allowed_access_key_id_prefixes` | `[]string` | no | | List of allowed `AccessKeyID` prefixes which S3 GW serve. If the parameter is omitted, all `AccessKeyID` will be accepted. |
2024-02-11 18:00:56 +00:00
| `reconnect_interval` | `duration` | no | `1m` | Listeners reconnection interval. |
2024-04-17 14:08:55 +00:00
| `source_ip_header` | `string` | yes | | Custom header to retrieve Source IP. |
2022-07-18 13:06:39 +00:00
### `wallet` section
```yaml
wallet:
2022-07-28 13:26:42 +00:00
path: /path/to/wallet.json # Path to wallet
passphrase: "" # Passphrase to decrypt wallet.
address: NfgHwwTi3wHAS8aFAN243C5vGbkYDpqLHP
2022-07-18 13:06:39 +00:00
```
2022-07-28 13:26:42 +00:00
| Parameter | Type | Default value | Description |
|--------------|----------|---------------|---------------------------------------------------------------------------|
| `path` | `string` | | Path to wallet |
| `passphrase` | `string` | | Passphrase to decrypt wallet. |
| `address` | `string` | | Account address to get from wallet. If omitted default one will be used. |
2022-07-18 13:06:39 +00:00
### `peers` section
```yaml
# Nodes configuration
2022-12-15 13:37:18 +00:00
# This configuration makes the gateway use the first node (node1.frostfs:8080)
# while it's healthy. Otherwise, gateway uses the second node (node2.frostfs:8080)
# for 10% of requests and the third node (node3.frostfs:8080) for 90% of requests.
2022-07-18 13:06:39 +00:00
# Until nodes with the same priority level are healthy
# nodes with other priority are not used.
# The lower the value, the higher the priority.
peers:
0:
2022-12-15 13:37:18 +00:00
address: node1.frostfs:8080
2022-07-18 13:06:39 +00:00
priority: 1
weight: 1
1:
2022-12-15 13:37:18 +00:00
address: node2.frostfs:8080
2022-07-18 13:06:39 +00:00
priority: 2
weight: 0.1
2:
2022-12-15 13:37:18 +00:00
address: node3.frostfs:8080
2022-07-18 13:06:39 +00:00
priority: 2
weight: 0.9
```
| Parameter | Type | Default value | Description |
|------------|----------|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| `address` | `string` | | Address of storage node. |
| `priority` | `int` | `1` | It allows to group nodes and don't switch group until all nodes with the same priority will be unhealthy. The lower the value, the higher the priority. |
| `weight` | `float` | `1` | Weight of node in the group with the same priority. Distribute requests to nodes proportionally to these values. |
2022-04-06 14:31:32 +00:00
2022-11-03 06:49:06 +00:00
### `placement_policy` section
```yaml
placement_policy:
default: REP 3
region_mapping: /path/to/mapping/rules.json
2023-04-24 23:49:12 +00:00
copies_numbers:
- location_constraint: one-dc
vector:
- 1
- 2
- 3
2022-11-03 06:49:06 +00:00
```
2023-04-24 23:49:12 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|------------------|------------------------------------------------|---------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `default` | `string` | yes | `REP 3` | Default policy of placing containers in FrostFS. If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in FrostFS, the S3 Gateway will put the container with default policy. |
| `region_mapping` | `string` | yes | | Path to file that maps aws `LocationContraint` values to FrostFS placement policy. The similar to `--container-policy` flag in `frostfs-s3-authmate` util, see in [docs ](./authmate.md#containers-policy ) |
| `copies_numbers` | [[]Copies numbers](#copies_numbers-subsection) | no | | Array of configured location constraints and their copies numbers. |
2022-11-03 06:49:06 +00:00
File for `region_mapping` must contain something like this:
```json
{
"rep-3": "REP 3",
"complex": "REP 1 IN X CBF 1 SELECT 1 FROM * AS X",
"example-json-policy": "{\"replicas\":[{\"count\":3,\"selector\":\"SelASD0\"}],\"container_backup_factor\":3,\"selectors\":[{\"name\":\"SelASD0\",\"count\":3,\"filter\":\"*\"}],\"filters\":[]}"
}
```
2023-03-24 12:49:23 +00:00
**Note:** on SIGHUP reload policies will be updated only if both parameters are valid.
2022-11-14 10:01:12 +00:00
So if you change `default` to some valid value and set invalid path in `region_mapping` the `default` value won't be changed.
2023-04-24 23:49:12 +00:00
#### `copies_numbers` subsection
```yaml
- location_constraint: sample-01
vector:
- 1
- 2
- 3
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------------------|----------|---------------|---------------|----------------------------------------------------------|
| `location_constraint` | `string` | no | | Location constraint text label. |
| `vector` | `[]int` | no | | Array of copies numbers corresponding to the constraint. |
2022-11-09 11:53:26 +00:00
### `server` section
You can specify several listeners for server. For example, for `http` and `https` .
2022-04-06 14:31:32 +00:00
2022-07-18 13:06:39 +00:00
```yaml
2022-11-09 11:53:26 +00:00
server:
- address: 0.0.0.0:8080
tls:
enabled: false
cert_file: /path/to/cert
key_file: /path/to/key
- address: 0.0.0.0:8081
tls:
enabled: true
cert_file: /path/to/another/cert
key_file: /path/to/another/key
2022-07-18 13:06:39 +00:00
```
2022-11-09 11:53:26 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------------|----------|---------------|----------------|-----------------------------------------------|
| `address` | `string` | | `0.0.0.0:8080` | The address that the gateway is listening on. |
| `tls.enabled` | `bool` | | false | Enable TLS or not. |
| `tls.cert_file` | `string` | yes | | Path to the TLS certificate. |
| `tls.key_file` | `string` | yes | | Path to the key. |
2023-10-17 14:49:57 +00:00
2022-07-18 13:06:39 +00:00
### `logger` section
```yaml
logger:
level: debug
2023-11-09 06:07:11 +00:00
destination: stdout
2024-09-20 11:34:05 +00:00
sampling:
enabled: false
initial: 100
thereafter: 100
interval: 1s
2022-07-18 13:06:39 +00:00
```
2024-09-20 11:34:05 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------------------|------------|---------------|---------------|----------------------------------------------------------------------------------------------------|
| `level` | `string` | yes | `debug` | Logging level.< br /> Possible values: `debug` , `info` , `warn` , `error` , `dpanic` , `panic` , `fatal` . |
| `destination` | `string` | no | `stdout` | Destination for logger: `stdout` or `journald` |
| `sampling.enabled` | `bool` | no | false | Sampling enabling flag. |
| `sampling.initial` | `int` | no | '100' | Sampling count of first log entries. |
| `sampling.thereafter` | `int` | no | '100' | Sampling count of entries after an `interval` . |
| `sampling.interval` | `duration` | no | '1s' | Sampling interval of messaging similar entries. |
2022-07-18 13:06:39 +00:00
2024-07-12 12:31:43 +00:00
### `http_logging` section
2024-08-01 08:21:20 +00:00
Could be enabled only in builds with `loghttp` build tag. To build with `loghttp` tag, pass `GOFLAGS` var to `make` :
```bash
make GOFLAGS="-tags=loghttp" [target]
```
2024-07-12 12:31:43 +00:00
```yaml
http_logging:
enabled: false
max_body: 1024
max_log_size: 20
gzip: true
destination: stdout
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|----------------|----------|---------------|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `enabled` | `bool` | yes | `false` | Flag to enable the logger. |
| `max_body` | `int` | yes | `1024` | Max body size for log output in bytes. |
| `max_log_size` | `int` | yes | `50` | Log file size threshold (in megabytes) to be moved in backup file. After reaching threshold, initial filename is appended with timestamp. And new empty file with initial name is created. |
| `gzip` | `bool` | yes | `false` | Whether to enable Gzip compression to backup log files. |
| `destination` | `string` | yes | `stdout` | Specify path for log output. Accepts log file path, or "stdout" and "stderr" reserved words to print in output streams. File and folders are created if necessary. |
2022-07-18 13:06:39 +00:00
### `cache` section
```yaml
cache:
objects:
lifetime: 300s
size: 150
list:
lifetime: 1m
size: 100
2023-10-16 06:27:21 +00:00
list_session:
lifetime: 1m
size: 100
2022-07-18 13:06:39 +00:00
names:
lifetime: 1m
size: 1000
buckets:
lifetime: 1m
size: 500
system:
lifetime: 2m
size: 1000
accessbox:
2024-02-06 13:44:49 +00:00
removing_check_interval: 5m
lifetime: 10m
size: 100
2022-10-03 14:36:09 +00:00
accesscontrol:
lifetime: 1m
size: 100000
2023-12-05 09:12:35 +00:00
morph_policy:
lifetime: 30s
size: 10000
2024-03-12 08:28:24 +00:00
frostfsid:
lifetime: 1m
size: 10000
2024-09-05 12:36:40 +00:00
network_info:
lifetime: 1m
2022-07-18 13:06:39 +00:00
```
2024-06-25 12:24:29 +00:00
| Parameter | Type | Default value | Description |
|-----------------|-------------------------------------------------|-----------------------------------|----------------------------------------------------------------|
| `objects` | [Cache config ](#cache-subsection ) | `lifetime: 5m` < br > `size: 1000000` | Cache for objects (FrostFS headers). |
| `list` | [Cache config ](#cache-subsection ) | `lifetime: 60s` < br > `size: 100000` | Cache which keeps lists of objects in buckets. |
| `list_session` | [Cache config ](#cache-subsection ) | `lifetime: 60s` < br > `size: 100` | Cache which keeps listing session. |
| `names` | [Cache config ](#cache-subsection ) | `lifetime: 60s` < br > `size: 10000` | Cache which contains mapping of nice name to object addresses. |
| `buckets` | [Cache config ](#cache-subsection ) | `lifetime: 60s` < br > `size: 1000` | Cache which contains mapping of bucket name to bucket info. |
| `system` | [Cache config ](#cache-subsection ) | `lifetime: 5m` < br > `size: 10000` | Cache for system objects in a bucket: bucket settings etc. |
| `accessbox` | [Accessbox cache config ](#accessbox-subsection ) | `lifetime: 10m` < br > `size: 100` | Cache which stores access box with tokens by its address. |
| `accesscontrol` | [Cache config ](#cache-subsection ) | `lifetime: 1m` < br > `size: 100000` | Cache which stores owner to cache operation mapping. |
| `morph_policy` | [Cache config ](#cache-subsection ) | `lifetime: 1m` < br > `size: 10000` | Cache which stores list of policy chains. |
| `frostfsid` | [Cache config ](#cache-subsection ) | `lifetime: 1m` < br > `size: 10000` | Cache which stores FrostfsID subject info. |
2024-09-05 12:36:40 +00:00
| `network_info` | [Cache config ](#cache-subsection ) | `lifetime: 1m` | Cache which stores network info. |
2022-07-18 13:06:39 +00:00
#### `cache` subsection
```yaml
lifetime: 2m
size: 1000
```
| Parameter | Type | Default value | Description |
|------------|------------|------------------|-------------------------------|
| `lifetime` | `duration` | depends on cache | Lifetime of entries in cache. |
| `size` | `int` | depends on cache | LRU cache size. |
2024-02-06 13:44:49 +00:00
#### `accessbox` subsection
```yaml
lifetime: 10m
size: 100
```
| Parameter | Type | Default value | Description |
|---------------------------|------------|---------------|-------------------------------------------------------|
| `removing_check_interval` | `duration` | `5m' | Time after which creds should be checked for removal. |
| `lifetime` | `duration` | '10m' | Lifetime of entries in cache. |
| `size` | `int` | '100 | LRU cache size. |
2022-07-18 13:06:39 +00:00
### `cors` section
```yaml
cors:
default_max_age: 600
```
| Parameter | Type | Default value | Description |
|-------------------|-------|---------------|------------------------------------------------------|
| `default_max_age` | `int` | `600` | Value of `Access-Control-Max-Age` header in seconds. |
2022-07-26 13:29:07 +00:00
# `pprof` section
Contains configuration for the `pprof` profiler.
```yaml
pprof:
enabled: true
address: localhost:8085
```
2022-09-30 14:37:01 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------|----------|---------------|------------------|-----------------------------------------|
| `enabled` | `bool` | yes | `false` | Flag to enable the service. |
| `address` | `string` | yes | `localhost:8085` | Address that service listener binds to. |
2022-07-26 13:29:07 +00:00
# `prometheus` section
Contains configuration for the `prometheus` metrics service.
2023-02-10 06:44:44 +00:00
General metrics are available on `/metrics` url path, billing metrics on `/metrics/billing` .
2022-07-26 13:29:07 +00:00
```yaml
prometheus:
enabled: true
address: localhost:8086
```
2022-09-30 14:37:01 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------|----------|---------------|------------------|-----------------------------------------|
| `enabled` | `bool` | yes | `false` | Flag to enable the service. |
| `address` | `string` | yes | `localhost:8086` | Address that service listener binds to. |
2022-08-11 23:13:02 +00:00
2023-05-31 16:35:20 +00:00
# `tracing` section
Contains configuration for the `tracing` service.
```yaml
tracing:
enabled: false
exporter: "otlp_grpc"
endpoint: "localhost:4318"
2024-09-10 07:31:01 +00:00
trusted_ca: "/etc/ssl/telemetry-trusted-ca.pem"
2023-05-31 16:35:20 +00:00
```
2024-09-10 07:31:01 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|--------------|----------|---------------|---------------|---------------------------------------------------------------------------------------------------------------------------------|
2024-09-25 11:00:56 +00:00
| `enabled` | `bool` | no | `false` | Flag to enable the service. |
2024-09-10 07:31:01 +00:00
| `exporter` | `string` | yes | `` | Type of tracing exporter. |
| `endpoint` | `string` | yes | `` | Address that service listener binds to. |
| `trusted_ca` | `string` | yes | | Path to certificate of a certification authority in pem format, that issued the TLS certificate of the telemetry remote server. |
2023-05-31 16:35:20 +00:00
2022-12-20 08:38:58 +00:00
# `frostfs` section
2022-08-11 23:13:02 +00:00
2023-03-24 12:49:23 +00:00
Contains parameters of requests to FrostFS.
2023-07-14 12:30:47 +00:00
The `set_copies_number` value can be overridden with `X-Amz-Meta-Frostfs-Copies-Number` (value is comma separated numbers: `1,2,3` )
2023-05-24 06:40:45 +00:00
header for `PutObject` , `CopyObject` , `CreateMultipartUpload` .
2022-08-11 23:13:02 +00:00
```yaml
2022-12-15 13:37:18 +00:00
frostfs:
2023-05-24 06:40:45 +00:00
set_copies_number: [0]
2023-07-14 12:30:47 +00:00
client_cut: false
2023-08-25 10:07:59 +00:00
buffer_max_size_for_put: 1048576 # 1mb
2023-10-23 14:06:42 +00:00
tree_pool_max_attempts: 0
2024-08-26 10:04:24 +00:00
graceful_close_on_switch_timeout: 10s
2022-08-11 23:13:02 +00:00
```
2024-08-26 10:04:24 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|------------------------------------|------------|---------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `set_copies_number` | `[]uint32` | yes | `[0]` | Numbers of the object copies (for each replica) to consider PUT to FrostFS successful. < br /> Default value `[0]` or empty list means that object will be processed according to the container's placement policy |
| `client_cut` | `bool` | yes | `false` | This flag enables client side object preparing. |
| `buffer_max_size_for_put` | `uint64` | yes | `1048576` | Sets max buffer size for read payload in put operations. |
| `tree_pool_max_attempts` | `uint32` | no | `0` | Sets max attempt to make successful tree request. Value 0 means the number of attempts equals to number of nodes in pool. |
| `graceful_close_on_switch_timeout` | `duration` | no | `10s` | Specifies the timeout after which unhealthy client be closed during rebalancing if it will become healthy back. |
2023-02-10 12:21:25 +00:00
# `resolve_bucket` section
2023-11-16 12:10:51 +00:00
Bucket name resolving parameters from and to container ID.
2023-02-10 12:21:25 +00:00
```yaml
resolve_bucket:
2023-11-16 12:10:51 +00:00
namespace_header: X-Frostfs-Namespace
2023-02-17 13:22:23 +00:00
allow:
- container
deny:
2023-02-10 12:21:25 +00:00
```
2023-11-16 12:10:51 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|--------------------|------------|---------------|-----------------------|--------------------------------------------------------------------------------------------------------------------------|
| `namespace_header` | `string` | yes | `X-Frostfs-Namespace` | Header to determine zone to resolve bucket name. |
| `allow` | `[]string` | no | | List of container zones which are available to resolve. Mutual exclusive with `deny` list. Prioritized over `deny` list. |
| `deny` | `[]string` | no | | List of container zones which are restricted to resolve. Mutual exclusive with `allow` list. |
2023-03-02 14:54:33 +00:00
# `kludge` section
Workarounds for non-standard use cases.
```yaml
kludge:
2023-10-09 12:34:51 +00:00
use_default_xmlns: false
2023-07-10 09:17:44 +00:00
bypass_content_encoding_check_in_chunks: false
2023-11-16 12:10:51 +00:00
default_namespaces: [ "", "root" ]
2023-03-02 14:54:33 +00:00
```
2023-11-16 12:10:51 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|-------------------------------------------|------------|---------------|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
2024-02-14 14:04:35 +00:00
| `use_default_xmlns` | `bool` | yes | `false` | Enable using default xml namespace `http://s3.amazonaws.com/doc/2006-03-01/` when parse xml bodies. |
| `bypass_content_encoding_check_in_chunks` | `bool` | yes | `false` | Use this flag to be able to use [chunked upload approach ](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html ) without having `aws-chunked` value in `Content-Encoding` header. |
| `default_namespaces` | `[]string` | yes | `["","root"]` | Namespaces that should be handled as default. |
2023-08-30 18:24:22 +00:00
# `runtime` section
Contains runtime parameters.
```yaml
runtime:
soft_memory_limit: 1gb
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|---------------------|--------|---------------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
2023-10-09 12:34:51 +00:00
| `soft_memory_limit` | `size` | yes | maxint64 | Soft memory limit for the runtime. Zero or no value stands for no limit. If `GOMEMLIMIT` environment variable is set, the value from the configuration file will be ignored. |
2023-10-02 08:52:07 +00:00
# `features` section
Contains parameters for enabling features.
```yaml
features:
2023-12-05 12:49:13 +00:00
policy:
deny_by_default: false
2023-10-02 08:52:07 +00:00
md5:
enabled: false
```
2023-12-05 12:49:13 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|--------------------------|--------|---------------|---------------|------------------------------------------------------------------------------|
| `md5.enabled` | `bool` | yes | false | Flag to enable return MD5 checksum in ETag headers and fields. |
| `policy.deny_by_default` | `bool` | yes | false | Enable denying access for request that doesn't match any policy chain rules. |
2023-10-26 13:44:40 +00:00
# `web` section
Contains web server configuration parameters.
```yaml
web:
read_timeout: 0
read_header_timeout: 30s
write_timeout: 0
idle_timeout: 30s
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------------------|------------|---------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `read_timeout` | `duration` | no | `0` | The maximum duration for reading the entire request, including the body. A zero or negative value means there will be no timeout. |
| `read_header_timeout` | `duration` | no | `30s` | The amount of time allowed to read request headers. If `read_header_timeout` is zero, the value of `read_timeout` is used. If both are zero, there is no timeout. |
| `write_timeout` | `duration` | no | `0` | The maximum duration before timing out writes of the response. A zero or negative value means there will be no timeout. |
| `idle_timeout` | `duration` | no | `30s` | The maximum amount of time to wait for the next request when keep-alives are enabled. If `idle_timeout` is zero, the value of `read_timeout` is used. If both are zero, there is no timeout. |
2023-10-05 13:25:25 +00:00
# `frostfsid` section
FrostfsID contract configuration. To enable this functionality the `rpc_endpoint` param must be also set.
```yaml
frostfsid:
contract: frostfsid.frostfs
2024-01-25 06:37:43 +00:00
validation:
2023-12-13 14:44:18 +00:00
enabled: false
2023-10-05 13:25:25 +00:00
```
2023-12-13 14:44:18 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|----------------------|----------|---------------|---------------------|---------------------------------------------------------------------------------------|
| `contract` | `string` | no | `frostfsid.frostfs` | FrostfsID contract hash (LE) or name in NNS. |
| `validation.enabled` | `bool` | no | `false` | Enables a check to only allow requests to users registered in the FrostfsID contract. |
2023-11-21 08:51:07 +00:00
2023-12-05 09:12:35 +00:00
# `policy` section
Policy contract configuration. To enable this functionality the `rpc_endpoint` param must be also set.
```yaml
policy:
contract: policy.frostfs
```
2024-02-19 08:45:18 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|------------|----------|---------------|----------------|-------------------------------------------|
| `contract` | `string` | no | policy.frostfs | Policy contract hash (LE) or name in NNS. |
2023-12-05 09:12:35 +00:00
2023-12-21 14:57:12 +00:00
# `proxy` section
Proxy contract configuration. To enable this functionality the `rpc_endpoint` param must be also set.
```yaml
proxy:
contract: proxy.frostfs
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|------------|----------|---------------|-----------------|------------------------------------------|
| `contract` | `string` | no | `proxy.frostfs` | Proxy contract hash (LE) or name in NNS. |
2023-11-21 08:51:07 +00:00
# `namespaces` section
Namespaces configuration.
```yaml
namespaces:
config: namespace.json
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|-----------|----------|---------------|---------------|-----------------------------------------------------|
| `config` | `string` | yes | | Path to json file with config value for namespaces. |
2024-01-25 06:37:43 +00:00
## `namespaces.config` subsection
2023-11-21 08:51:07 +00:00
Example of `namespaces.json` .
2024-01-25 06:37:43 +00:00
Note that config values from `namespaces.json` can override config values for default namespaces
2023-11-21 08:51:07 +00:00
(value for which are fetched from regular config value e.g. [placement-policy ](#placement_policy-section )).
To override config values for default namespaces use namespace names that are provided in `kludge.default_namespaces` .
```json
{
"namespaces": {
"namespace1": {
"location_constraints": {
2024-01-25 06:37:43 +00:00
"default": "REP 3",
2023-11-21 08:51:07 +00:00
"test": "{\"replicas\":[{\"count\":1,\"selector\":\"\"}],\"containerBackupFactor\":0,\"selectors\":[],\"filters\":[],\"unique\":false}"
},
"copies_numbers": {
"default": [ 0 ],
"test": [ 1 ]
}
}
}
}
```
2024-05-30 13:02:27 +00:00
# `retry` section
Retry strategy configuration.
```yaml
retry:
max_attempts: 4
max_backoff: 30s
strategy: exponential
```
| Parameter | Type | SIGHUP reload | Default value | Description |
|---------------|------------|---------------|---------------|--------------------------------------------------------------------------------------|
| `max_attemps` | `int` | yes | `4` | Max amount of request attempts. Currently only for updating bucket settings request. |
| `max_backoff` | `duration` | yes | `30s` | Max delay before next attempt. |
| `strategy` | `string` | yes | `exponential` | Backoff strategy. `exponential` and `constant` are allowed. |
2024-07-15 15:35:54 +00:00
# `containers` section
Section for well-known containers to store s3-related data and settings.
```yaml
containers:
2024-08-12 11:22:56 +00:00
cors: AZjLTXfK4vs4ovxMic2xEJKSymMNLqdwq9JT64ASFCRj
lifecycle: AZjLTXfK4vs4ovxMic2xEJKSymMNLqdwq9JT64ASFCRj
2024-08-30 12:05:32 +00:00
accessbox: ExnA1gSY3kzgomi2wJxNyWo1ytWv9VAKXRE55fNXEPL2
2024-07-15 15:35:54 +00:00
```
2024-08-30 12:05:32 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
|-------------|----------|---------------|---------------|-------------------------------------------------------------------------------------------------------------------------|
| `cors` | `string` | no | | Container name for CORS configurations. If not set, container of the bucket is used. |
| `lifecycle` | `string` | no | | Container name for lifecycle configurations. If not set, container of the bucket is used. |
| `accessbox` | `string` | no | | Container name to lookup accessbox if custom aws credentials is used. If not set, custom credentials are not supported. |
2024-07-31 06:45:46 +00:00
# `vhs` section
Configuration of virtual hosted addressing style.
```yaml
vhs:
enabled: false
2024-08-01 13:24:47 +00:00
vhs_header: X-Frostfs-S3-VHS
servername_header: X-Frostfs-Servername
2024-07-31 06:45:46 +00:00
namespaces:
"ns1": false
"ns2": true
```
2024-08-01 13:24:47 +00:00
| Parameter | Type | SIGHUP reload | Default value | Description |
2024-07-12 12:31:43 +00:00
|---------------------|-------------------|---------------|------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
2024-08-01 13:24:47 +00:00
| `enabled` | `bool` | yes | `false` | Enables the use of virtual host addressing for buckets at the application level. |
| `vhs_header` | `string` | yes | `X-Frostfs-S3-VHS` | Header for determining whether VHS is enabled for the request. |
| `servername_header` | `string` | yes | `X-Frostfs-Servername` | Header for determining servername. |
| `namespaces` | `map[string]bool` | yes | | A map in which the keys are the name of the namespace, and the values are the flag responsible for enabling VHS for the specified namespace. Overrides global 'enabled' setting even when it is disabled. |