[#85] Add s3 lifecycler #87

Merged
alexvanin merged 1 commit from pogpp/frostfs-dev-env:feature/85_add_lifecycler into master 2024-11-08 14:18:39 +00:00
8 changed files with 119 additions and 1 deletions

6
.env
View file

@ -28,9 +28,13 @@ REST_GW_VERSION=c9c85e90
REST_GW_IMAGE=truecloudlab/frostfs-rest-gw
# S3 Gate
S3_GW_VERSION=0.30.4
S3_GW_VERSION=0.31.0-rc.4
S3_GW_IMAGE=truecloudlab/frostfs-s3-gw
# Lifecycler
S3_LIFECYCLER_VERSION=0.1.3
S3_LIFECYCLER_IMAGE=truecloudlab/frostfs-s3-lifecycler
# FrostFS LOCODE database
LOCODE_DB_URL=https://git.frostfs.info/attachments/a2e8def7-52b6-49f1-89cd-a056712e8e54
dkirillov marked this conversation as resolved Outdated
Why don't we use version `0.1.3` ? https://hub.docker.com/r/truecloudlab/frostfs-s3-lifecycler/tags
#LOCODE_DB_PATH=/path/to/locode_db

View file

@ -3,3 +3,4 @@
http_gate
s3_gate
rest_gate
s3_lifecycler

1
services/s3_lifecycler/.env Symbolic link
View file

@ -0,0 +1 @@
../../.env
dkirillov marked this conversation as resolved Outdated

This file should be link (see example in s3_gate/.env or any other dir)

This file should be link (see example in `s3_gate/.env` or any other dir)

The same for .int_test.env

The same for `.int_test.env`

View file

@ -0,0 +1 @@
IPV4_PREFIX.84 lifecycler.LOCAL_DOMAIN

View file

@ -0,0 +1 @@
../../.int_test.env

View file

@ -0,0 +1,42 @@
logger:
level: debug
prometheus:
enabled: true
dkirillov marked this conversation as resolved Outdated

Let's enable metrics.
Probably we should choose another port (but it should be checked)

Let's enable metrics. Probably we should choose another port (but it should be checked)
address: :9090
lifecycle:
job_fetcher_buffer: 1000
dkirillov marked this conversation as resolved Outdated
This param should be in `frostfs` section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md
executor_pool_size: 100
frostfs:
dkirillov marked this conversation as resolved Outdated
This param should be in `frostfs` section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md
stream_timeout: 10s
connect_timeout: 10s
healthcheck_timeout: 15s
dkirillov marked this conversation as resolved Outdated
This param should be in `frostfs` section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md
rebalance_interval: 60s
pool_error_threshold: 100
tree_pool_max_attempts: 4
credential:
use: wallets
dkirillov marked this conversation as resolved Outdated

These params (max_clients_count, max_clients_deadline, resolve_order) aren't applicable to s3-lifecycler

These params (`max_clients_count`, `max_clients_deadline`, `resolve_order`) aren't applicable to s3-lifecycler

We should add credential section into config (this section should contain wallet from frostfs-dev-env/wallets/wallet.json) https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md#credential-section

We should add `credential` section into config (this section should contain wallet from `frostfs-dev-env/wallets/wallet.json`) https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md#credential-section

credential section still doesn't contains wallet frostfs-dev-env/wallets/wallet.json

`credential` section still doesn't contains wallet `frostfs-dev-env/wallets/wallet.json`
source:
wallets:
- path: /wallet.json
address: NTt1rxvmEDxEuuogLxs2xgxA71qhVaUcN7
passphrase: "cycle"
- path: /user-wallet.json
address: NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM
passphrase: ""
morph:
reconnect_clients_interval: 30s
dial_timeout: 5s
contract:
netmap: netmap.frostfs
frostfsid: frostfsid.frostfs
container: container.frostfs
# Wallet configuration
wallet:
path: /wallet.json # Path to wallet
passphrase: "cycle" # Passphrase to decrypt wallet

View file

@ -0,0 +1,38 @@
---
version: "2.4"
services:
s3_lifecycler:
image: ${S3_LIFECYCLER_IMAGE}:${S3_LIFECYCLER_VERSION}
domainname: ${LOCAL_DOMAIN}
hostname: s3_lifecycler
container_name: s3_lifecycler
restart: on-failure
networks:
s3_lifecycler_int:
internet:
ipv4_address: ${IPV4_PREFIX}.84
volumes:
- ./wallet.json:/wallet.json
- ./../../vendor/hosts:/etc/hosts
- ./cfg:/etc/frostfs/s3-lifecycler
- ./../../wallets/wallet.json:/user-wallet.json
stop_signal: SIGKILL
dkirillov marked this conversation as resolved Outdated

Actually, I'm not sure if we need .s3_lifecycler.env. Can we drop this file at all?

Actually, I'm not sure if we need `.s3_lifecycler.env`. Can we drop this file at all?
env_file: [ ".env", ".int_test.env" ]
command: [ "frostfs-s3-lifecycler", "--config", "/etc/frostfs/s3-lifecycler/config.yml" ]
environment:
- S3_LIFECYCLER_MORPH_RPC_ENDPOINT_0_ADDRESS=ws://morph-chain:30333/ws
- S3_LIFECYCLER_FROSTFS_PEERS_0_ADDRESS=s01.${LOCAL_DOMAIN}:8080
- S3_LIFECYCLER_FROSTFS_PEERS_0_WEIGHT=0.2
- S3_LIFECYCLER_FROSTFS_PEERS_1_ADDRESS=s02.${LOCAL_DOMAIN}:8080
- S3_LIFECYCLER_FROSTFS_PEERS_1_WEIGHT=0.2
- S3_LIFECYCLER_FROSTFS_PEERS_2_ADDRESS=s03.${LOCAL_DOMAIN}:8080
dkirillov marked this conversation as resolved Outdated

Please keep here only variable that use ${LOCAL_DOMAIN}. Other params move to cfg/conig.yml and omit them if they have default values

Please keep here only variable that use `${LOCAL_DOMAIN}`. Other params move to `cfg/conig.yml` and omit them if they have default values
- S3_LIFECYCLER_FROSTFS_PEERS_2_WEIGHT=0.2
- S3_LIFECYCLER_FROSTFS_PEERS_3_ADDRESS=s04.${LOCAL_DOMAIN}:8080
dkirillov marked this conversation as resolved Outdated

We can omit this param

We can omit this param
- S3_LIFECYCLER_FROSTFS_PEERS_3_WEIGHT=0.2
networks:
s3_lifecycler_int:
internet:
external: true
name: basenet_internet

View file

@ -0,0 +1,30 @@
{
"version": "1.0",
"accounts": [
{
"address": "NTt1rxvmEDxEuuogLxs2xgxA71qhVaUcN7",
"key": "6PYR3XurAyTzVeDG5WV2Z8vnGdySw3mTLuKjr6Nwo7tae64SJ7XjZSMMPQ",
"label": "lifecycler",
"contract": {
"script": "DCED9z0M+WSGfXZGxYLj1yYwmgxJXE/kNA4+oWNi0q1uKCdBVuezJw==",
"parameters": [
{
"name": "parameter0",
"type": "Signature"
}
],
"deployed": false
},
"lock": false,
"isDefault": false
}
],
"scrypt": {
"n": 16384,
"r": 8,
"p": 8
},
"extra": {
"Tokens": null
}
}