EliChin/frostfs-s3-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-s3-gw
Code Issues Pull requests Projects Releases Packages Wiki Activity

[#568] Update docs

Browse source 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
This commit is contained in:
Denis Kirillov 2022-11-03 09:49:06 +03:00 committed by Alex Vanin
parent d47840f137
commit d3702f86d1
4 changed files with 70 additions and 22 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
CHANGELOG.md
View file

@ -7,6 +7,24 @@ This document outlines major changes between releases.
### Fixed ### Fixed
- Empty bucket policy (#740) - Empty bucket policy (#740)
### Changed
- Placement policy configuration (#568)
### Updating from v0.25.0
New config parameters were added. And old one `defaul_policy` were changed.
```yaml
placement_policy:
default: "REP 3"
region_mapping: /path/to/container/policies.json
```
Make sure you update the config accordingly:
If you configure application using environment variables change:
* `S3_GW_DEFAULT_POLICY` -> `S3_GW_PLACEMENT_POLICY_DEFAULT_POLICY`
If you configure application using `.yaml` file change:
* `defaul_policy` -> `placement_policy.default`
## [0.25.0] - 2022-10-31 ## [0.25.0] - 2022-10-31
### Fixed ### Fixed

5
config/config.env
View file

@ -99,7 +99,10 @@ S3_GW_NATS_ROOT_CA=/path/to/ca
# Default policy of placing containers in NeoFS # Default policy of placing containers in NeoFS
# If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway # If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway
# will put the container with default policy. It can be specified via environment variable, e.g.: # will put the container with default policy. It can be specified via environment variable, e.g.:
S3_GW_DEFAULT_POLICY="REP 3" S3_GW_PLACEMENT_POLICY_DEFAULT_POLICY="REP 3"
# Region to placement policy mapping json file.
# Path to container policy mapping. The same as '--container-policy' flag for authmate
S3_GW_PLACEMENT_POLICY_REGION_MAPPING=/path/to/container/policy.json
# CORS # CORS
# value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type. # value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type.

13
config/config.yaml
View file

@ -110,10 +110,15 @@ nats:
key_file: /path/to/key key_file: /path/to/key
root_ca: /path/to/ca root_ca: /path/to/ca
# Default policy of placing containers in NeoFS # Parameters of NeoFS container placement policy
# If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway placement_policy:
# will put the container with default policy. It can be specified via environment variable, e.g.: # Default policy of placing containers in NeoFS
default_policy: REP 3 # If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway
# will put the container with default policy.
default: REP 3
# Region to placement policy mapping json file.
# Path to container policy mapping. The same as '--container-policy' flag for authmate
region_mapping: /path/to/container/policy.json
# CORS # CORS
# value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type. # value of Access-Control-Max-Age header if this value is not set in a rule. Has an int type.

56
docs/configuration.md
View file

@ -149,20 +149,21 @@ There are some custom types used for brevity:
### Structure ### Structure
| Section | Description | | Section | Description |
|--------------|---------------------------------------------------| |--------------------|-------------------------------------------------------------|
| no section | [General parameters](#general-section) | | no section | [General parameters](#general-section) |
| `wallet` | [Wallet configuration](#wallet-section) | | `wallet` | [Wallet configuration](#wallet-section) |
| `peers` | [Nodes configuration](#peers-section) | | `peers` | [Nodes configuration](#peers-section) |
| `tls` | [TLS configuration](#tls-section) | | `placement_policy` | [Placement policy configuration](#placement_policy-section) |
| `logger` | [Logger configuration](#logger-section) | | `tls` | [TLS configuration](#tls-section) |
| `tree` | [Tree configuration](#tree-section) | | `logger` | [Logger configuration](#logger-section) |
| `cache` | [Cache configuration](#cache-section) | | `tree` | [Tree configuration](#tree-section) |
| `nats` | [NATS configuration](#nats-section) | | `cache` | [Cache configuration](#cache-section) |
| `cors` | [CORS configuration](#cors-section) | | `nats` | [NATS configuration](#nats-section) |
| `pprof` | [Pprof configuration](#pprof-section) | | `cors` | [CORS configuration](#cors-section) |
| `prometheus` | [Prometheus configuration](#prometheus-section) | | `pprof` | [Pprof configuration](#pprof-section) |
| `neofs` | [Parameters of requests to NeoFS](#neofs-section) | | `prometheus` | [Prometheus configuration](#prometheus-section) |
| `neofs` | [Parameters of requests to NeoFS](#neofs-section) |
### General section ### General section
@ -186,8 +187,6 @@ pool_error_threshold: 100
max_clients_count: 100 max_clients_count: 100
max_clients_deadline: 30s max_clients_deadline: 30s
default_policy: REP 3
allowed_access_key_id_prefixes: allowed_access_key_id_prefixes:
- Ck9BHsgKcnwfCTUSFm6pxhoNS4cBqgN2NQ8zVgPjqZDX - Ck9BHsgKcnwfCTUSFm6pxhoNS4cBqgN2NQ8zVgPjqZDX
- 3stjWenX15YwYzczMr88gy3CQr4NYFBQ8P7keGzH5QFn - 3stjWenX15YwYzczMr88gy3CQr4NYFBQ8P7keGzH5QFn
@ -205,7 +204,6 @@ allowed_access_key_id_prefixes:
| `pool_error_threshold` | `uint32` | | `100` | The number of errors on connection after which node is considered as unhealthy. | | `pool_error_threshold` | `uint32` | | `100` | The number of errors on connection after which node is considered as unhealthy. |
| `max_clients_count` | `int` | | `100` | Limits for processing of clients' requests. | | `max_clients_count` | `int` | | `100` | Limits for processing of clients' requests. |
| `max_clients_deadline` | `duration` | | `30s` | Deadline after which the gate sends error `RequestTimeout` to a client. | | `max_clients_deadline` | `duration` | | `30s` | Deadline after which the gate sends error `RequestTimeout` to a client. |
| `default_policy` | `string` | | `REP 3` | Default policy of placing containers in NeoFS. If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway will put the container with default policy. |
| `allowed_access_key_id_prefixes` | `[]string` | | | List of allowed `AccessKeyID` prefixes which S3 GW serve. If the parameter is omitted, all `AccessKeyID` will be accepted. | | `allowed_access_key_id_prefixes` | `[]string` | | | List of allowed `AccessKeyID` prefixes which S3 GW serve. If the parameter is omitted, all `AccessKeyID` will be accepted. |
### `wallet` section ### `wallet` section
@ -254,6 +252,30 @@ peers:
| `priority` | `int` | `1` | It allows to group nodes and don't switch group until all nodes with the same priority will be unhealthy. The lower the value, the higher the priority. | | `priority` | `int` | `1` | It allows to group nodes and don't switch group until all nodes with the same priority will be unhealthy. The lower the value, the higher the priority. |
| `weight` | `float` | `1` | Weight of node in the group with the same priority. Distribute requests to nodes proportionally to these values. | | `weight` | `float` | `1` | Weight of node in the group with the same priority. Distribute requests to nodes proportionally to these values. |
### `placement_policy` section
```yaml
placement_policy:
default: REP 3
region_mapping: /path/to/mapping/rules.json
```
| Parameter | Type | Default value | Description |
|------------------|----------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `default` | `string` | `REP 3` | Default policy of placing containers in NeoFS. If a user sends a request `CreateBucket` and doesn't define policy for placing of a container in NeoFS, the S3 Gateway will put the container with default policy. |
| `region_mapping` | `string` | | Path to file that maps aws `LocationContraint` values to NeoFS placement policy. The similar to `--container-policy` flag in `neofs-s3-authmate` util. |
File for `region_mapping` must contain something like this:
```json
{
"rep-3": "REP 3",
"complex": "REP 1 IN X CBF 1 SELECT 1 FROM * AS X",
"example-json-policy": "{\"replicas\":[{\"count\":3,\"selector\":\"SelASD0\"}],\"container_backup_factor\":3,\"selectors\":[{\"name\":\"SelASD0\",\"count\":3,\"filter\":\"*\"}],\"filters\":[]}"
}
```
### `tls` section ### `tls` section
```yaml ```yaml