forked from TrueCloudLab/frostfs-s3-gw
parent
d47840f137
commit
d3702f86d1
4 changed files with 70 additions and 22 deletions
18
CHANGELOG.md
18
CHANGELOG.md
|
@ -7,6 +7,24 @@ This document outlines major changes between releases.
|
||||||
### Fixed
|
### Fixed
|
||||||
- Empty bucket policy (#740)
|
- Empty bucket policy (#740)
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- Placement policy configuration (#568)
|
||||||
|
|
||||||
|
### Updating from v0.25.0
|
||||||
|
New config parameters were added. And old one `defaul_policy` were changed.
|
||||||
|
```yaml
|
||||||
|
placement_policy:
|
||||||
|
default: "REP 3"
|
||||||
|
region_mapping: /path/to/container/policies.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Make sure you update the config accordingly:
|
||||||
|
If you configure application using environment variables change:
|
||||||
|
* `S3_GW_DEFAULT_POLICY` -> `S3_GW_PLACEMENT_POLICY_DEFAULT_POLICY`
|
||||||
|
|
||||||
|
If you configure application using `.yaml` file change:
|
||||||
|
* `defaul_policy` -> `placement_policy.default`
|
||||||
|
|
||||||
## [0.25.0] - 2022-10-31
|
## [0.25.0] - 2022-10-31
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
|
@ -99,7 +99,10 @@ S3_GW_NATS_ROOT_CA=/path/to/ca
|
||||||
# Default policy of placing containers in NeoFS
|
# Default policy of placing containers in NeoFS
|
||||||
# If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway
|
# If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway
|
||||||
# will put the container with default policy. It can be specified via environment variable, e.g.:
|
# will put the container with default policy. It can be specified via environment variable, e.g.:
|
||||||
S3_GW_DEFAULT_POLICY="REP 3"
|
S3_GW_PLACEMENT_POLICY_DEFAULT_POLICY="REP 3"
|
||||||
|
# Region to placement policy mapping json file.
|
||||||
|
# Path to container policy mapping. The same as '--container-policy' flag for authmate
|
||||||
|
S3_GW_PLACEMENT_POLICY_REGION_MAPPING=/path/to/container/policy.json
|
||||||
|
|
||||||
# CORS
|
# CORS
|
||||||
# value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type.
|
# value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type.
|
||||||
|
|
|
@ -110,10 +110,15 @@ nats:
|
||||||
key_file: /path/to/key
|
key_file: /path/to/key
|
||||||
root_ca: /path/to/ca
|
root_ca: /path/to/ca
|
||||||
|
|
||||||
|
# Parameters of NeoFS container placement policy
|
||||||
|
placement_policy:
|
||||||
# Default policy of placing containers in NeoFS
|
# Default policy of placing containers in NeoFS
|
||||||
# If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway
|
# If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway
|
||||||
# will put the container with default policy. It can be specified via environment variable, e.g.:
|
# will put the container with default policy.
|
||||||
default_policy: REP 3
|
default: REP 3
|
||||||
|
# Region to placement policy mapping json file.
|
||||||
|
# Path to container policy mapping. The same as '--container-policy' flag for authmate
|
||||||
|
region_mapping: /path/to/container/policy.json
|
||||||
|
|
||||||
# CORS
|
# CORS
|
||||||
# value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type.
|
# value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type.
|
||||||
|
|
|
@ -150,10 +150,11 @@ There are some custom types used for brevity:
|
||||||
### Structure
|
### Structure
|
||||||
|
|
||||||
| Section | Description |
|
| Section | Description |
|
||||||
|--------------|---------------------------------------------------|
|
|--------------------|-------------------------------------------------------------|
|
||||||
| no section | [General parameters](#general-section) |
|
| no section | [General parameters](#general-section) |
|
||||||
| `wallet` | [Wallet configuration](#wallet-section) |
|
| `wallet` | [Wallet configuration](#wallet-section) |
|
||||||
| `peers` | [Nodes configuration](#peers-section) |
|
| `peers` | [Nodes configuration](#peers-section) |
|
||||||
|
| `placement_policy` | [Placement policy configuration](#placement_policy-section) |
|
||||||
| `tls` | [TLS configuration](#tls-section) |
|
| `tls` | [TLS configuration](#tls-section) |
|
||||||
| `logger` | [Logger configuration](#logger-section) |
|
| `logger` | [Logger configuration](#logger-section) |
|
||||||
| `tree` | [Tree configuration](#tree-section) |
|
| `tree` | [Tree configuration](#tree-section) |
|
||||||
|
@ -186,8 +187,6 @@ pool_error_threshold: 100
|
||||||
max_clients_count: 100
|
max_clients_count: 100
|
||||||
max_clients_deadline: 30s
|
max_clients_deadline: 30s
|
||||||
|
|
||||||
default_policy: REP 3
|
|
||||||
|
|
||||||
allowed_access_key_id_prefixes:
|
allowed_access_key_id_prefixes:
|
||||||
- Ck9BHsgKcnwfCTUSFm6pxhoNS4cBqgN2NQ8zVgPjqZDX
|
- Ck9BHsgKcnwfCTUSFm6pxhoNS4cBqgN2NQ8zVgPjqZDX
|
||||||
- 3stjWenX15YwYzczMr88gy3CQr4NYFBQ8P7keGzH5QFn
|
- 3stjWenX15YwYzczMr88gy3CQr4NYFBQ8P7keGzH5QFn
|
||||||
|
@ -205,7 +204,6 @@ allowed_access_key_id_prefixes:
|
||||||
| `pool_error_threshold` | `uint32` | | `100` | The number of errors on connection after which node is considered as unhealthy. |
|
| `pool_error_threshold` | `uint32` | | `100` | The number of errors on connection after which node is considered as unhealthy. |
|
||||||
| `max_clients_count` | `int` | | `100` | Limits for processing of clients' requests. |
|
| `max_clients_count` | `int` | | `100` | Limits for processing of clients' requests. |
|
||||||
| `max_clients_deadline` | `duration` | | `30s` | Deadline after which the gate sends error `RequestTimeout` to a client. |
|
| `max_clients_deadline` | `duration` | | `30s` | Deadline after which the gate sends error `RequestTimeout` to a client. |
|
||||||
| `default_policy` | `string` | | `REP 3` | Default policy of placing containers in NeoFS. If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway will put the container with default policy. |
|
|
||||||
| `allowed_access_key_id_prefixes` | `[]string` | | | List of allowed `AccessKeyID` prefixes which S3 GW serve. If the parameter is omitted, all `AccessKeyID` will be accepted. |
|
| `allowed_access_key_id_prefixes` | `[]string` | | | List of allowed `AccessKeyID` prefixes which S3 GW serve. If the parameter is omitted, all `AccessKeyID` will be accepted. |
|
||||||
|
|
||||||
### `wallet` section
|
### `wallet` section
|
||||||
|
@ -254,6 +252,30 @@ peers:
|
||||||
| `priority` | `int` | `1` | It allows to group nodes and don't switch group until all nodes with the same priority will be unhealthy. The lower the value, the higher the priority. |
|
| `priority` | `int` | `1` | It allows to group nodes and don't switch group until all nodes with the same priority will be unhealthy. The lower the value, the higher the priority. |
|
||||||
| `weight` | `float` | `1` | Weight of node in the group with the same priority. Distribute requests to nodes proportionally to these values. |
|
| `weight` | `float` | `1` | Weight of node in the group with the same priority. Distribute requests to nodes proportionally to these values. |
|
||||||
|
|
||||||
|
|
||||||
|
### `placement_policy` section
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
placement_policy:
|
||||||
|
default: REP 3
|
||||||
|
region_mapping: /path/to/mapping/rules.json
|
||||||
|
```
|
||||||
|
|
||||||
|
| Parameter | Type | Default value | Description |
|
||||||
|
|------------------|----------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||||
|
| `default` | `string` | `REP 3` | Default policy of placing containers in NeoFS. If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway will put the container with default policy. |
|
||||||
|
| `region_mapping` | `string` | | Path to file that maps aws `LocationContraint` values to NeoFS placement policy. The similar to `--container-policy` flag in `neofs-s3-authmate` util. |
|
||||||
|
|
||||||
|
File for `region_mapping` must contain something like this:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"rep-3": "REP 3",
|
||||||
|
"complex": "REP 1 IN X CBF 1 SELECT 1 FROM * AS X",
|
||||||
|
"example-json-policy": "{\"replicas\":[{\"count\":3,\"selector\":\"SelASD0\"}],\"container_backup_factor\":3,\"selectors\":[{\"name\":\"SelASD0\",\"count\":3,\"filter\":\"*\"}],\"filters\":[]}"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
### `tls` section
|
### `tls` section
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
|
Loading…
Reference in a new issue