TrueCloudLab/frostfs-dev-env
42
0
Fork 18
Code Issues 14 Pull requests Releases Activity Actions

[#85] Add s3 lifecycler #87

Merged
alexvanin merged 1 commit from pogpp/frostfs-dev-env:feature/85_add_lifecycler into master 2024-11-08 14:18:39 +00:00
Conversation 1 Commits 1 Files changed 8 +119 -1
pogpp commented 2024-10-25 13:26:06 +00:00
Member
Copy link

close #85

Signed-off-by: Pavel Pogodaev p.pogodaev@yadro.com

close #85 Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
pogpp added 1 commit 2024-10-25 13:26:06 +00:00
[#85] Add s3 lifecycler
Some checks failed
/ DCO (pull_request) Failing after 9s
Details
a0d42eb6b7 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
requested reviews from dkirillov, alexvanin, mbiryukova 2024-10-25 13:26:29 +00:00
pogpp force-pushed feature/85_add_lifecycler from a0d42eb6b7 to d3f0e86d68 2024-10-28 12:50:49 +00:00 Compare
pogpp changed title from WIP: [#85] Add s3 lifecycler to [#85] Add s3 lifecycler 2024-10-28 12:52:28 +00:00
pogpp force-pushed feature/85_add_lifecycler from d3f0e86d68 to d67680c17d 2024-10-28 12:53:33 +00:00 Compare
dkirillov reviewed 2024-10-29 13:39:26 +00:00
.env Outdated
@ -36,2 +36,4 @@
S3_GW_IMAGE=truecloudlab/frostfs-s3-gw
# Lifecycler
S3_LIFECYCLER_VERSION=755083e36f
dkirillov commented 2024-10-29 13:39:26 +00:00
Member
Copy link

Why don't we use version 0.1.3 ? https://hub.docker.com/r/truecloudlab/frostfs-s3-lifecycler/tags

Why don't we use version `0.1.3` ? https://hub.docker.com/r/truecloudlab/frostfs-s3-lifecycler/tags
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:43:32 +00:00
services/s3_lifecycler/.hosts Outdated
@ -0,0 +1,2 @@
IPV4_PREFIX.82 s3.LOCAL_DOMAIN
IPV4_PREFIX.82 *.s3.LOCAL_DOMAIN
dkirillov commented 2024-10-29 13:43:32 +00:00
Member
Copy link

It seems this file should be

IPV4_PREFIX.84 lifecycler.LOCAL_DOMAIN
It seems this file should be ``` IPV4_PREFIX.84 lifecycler.LOCAL_DOMAIN ```
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:46:23 +00:00
services/s3_lifecycler/.env Outdated
@ -0,0 +1,66 @@
# Basenet settings
dkirillov commented 2024-10-29 13:46:23 +00:00
Member
Copy link

This file should be link (see example in s3_gate/.env or any other dir)

This file should be link (see example in `s3_gate/.env` or any other dir)
👍 1
dkirillov commented 2024-10-29 13:48:08 +00:00
Member
Copy link

The same for .int_test.env

The same for `.int_test.env`
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:51:05 +00:00
services/s3_lifecycler/cfg/config.yml Outdated
@ -0,0 +6,4 @@
address: :9090
# Interval to check node health
rebalance_interval: 30s
dkirillov commented 2024-10-29 13:51:05 +00:00
Member
Copy link

This param should be in frostfs section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md

This param should be in `frostfs` section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:51:19 +00:00
services/s3_lifecycler/cfg/config.yml Outdated
@ -0,0 +9,4 @@
rebalance_interval: 30s
# Timeout to check node health during rebalance
healthcheck_timeout: 15s
dkirillov commented 2024-10-29 13:51:19 +00:00
Member
Copy link

This param should be in frostfs section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md

This param should be in `frostfs` section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:51:31 +00:00
services/s3_lifecycler/cfg/config.yml Outdated
@ -0,0 +12,4 @@
healthcheck_timeout: 15s
# Timeout to connect to a node
connect_timeout: 10s
dkirillov commented 2024-10-29 13:51:31 +00:00
Member
Copy link

This param should be in frostfs section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md

This param should be in `frostfs` section see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:52:14 +00:00
services/s3_lifecycler/cfg/config.yml Outdated
@ -0,0 +18,4 @@
max_clients_count: 100
# Deadline after which the gate sends error `RequestTimeout` to a client
max_clients_deadline: 30s
dkirillov commented 2024-10-29 13:52:14 +00:00
Member
Copy link

These params (max_clients_count, max_clients_deadline, resolve_order) aren't applicable to s3-lifecycler

These params (`max_clients_count`, `max_clients_deadline`, `resolve_order`) aren't applicable to s3-lifecycler
👍 1
dkirillov commented 2024-10-29 13:53:30 +00:00
Member
Copy link

We should add credential section into config (this section should contain wallet from frostfs-dev-env/wallets/wallet.json) https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md#credential-section

We should add `credential` section into config (this section should contain wallet from `frostfs-dev-env/wallets/wallet.json`) https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/src/branch/master/docs/configuration.md#credential-section
👍 1
dkirillov commented 2024-10-31 06:41:07 +00:00
Member
Copy link

credential section still doesn't contains wallet frostfs-dev-env/wallets/wallet.json

`credential` section still doesn't contains wallet `frostfs-dev-env/wallets/wallet.json`
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:55:13 +00:00
services/s3_lifecycler/tls.crt Outdated
@ -0,0 +1,22 @@
-----BEGIN CERTIFICATE-----
dkirillov commented 2024-10-29 13:55:13 +00:00
Member
Copy link

We don't need this file

We don't need this file
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:55:21 +00:00
services/s3_lifecycler/tls.key Outdated
@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----
dkirillov commented 2024-10-29 13:55:21 +00:00
Member
Copy link

We don't need this file

We don't need this file
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:56:43 +00:00
services/s3_lifecycler/docker-compose.yml Outdated
@ -0,0 +49,4 @@
- S3_LIFECYCLER_PEERS_3_ADDRESS=s04.${LOCAL_DOMAIN}:8080
- S3_LIFECYCLER_PEERS_3_WEIGHT=0.2
- S3_LIFECYCLER_SERVER_0_ADDRESS=s3.${LOCAL_DOMAIN}:8080
- S3_LIFECYCLER_LISTEN_DOMAINS=s3.${LOCAL_DOMAIN}
dkirillov commented 2024-10-29 13:56:43 +00:00
Member
Copy link

Lifecycler doesn't have such variable

Lifecycler doesn't have such variable
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:57:06 +00:00
services/s3_lifecycler/docker-compose.yml Outdated
@ -0,0 +39,4 @@
- S3_LIFECYCLER_MORPH_RECONNECT_DIAL_TIMEOUT=5s
- S3_LIFECYCLER_MORPH_CONTRACT_NETMAP=netmap.frostfs
- S3_LIFECYCLER_MORPH_CONTRACT_FROSTFSID=frostfsid.frostfs
- S3_LIFECYCLER_MORPH_CONTRACT_CONTAINER=container.frostfs
dkirillov commented 2024-10-29 13:57:06 +00:00
Member
Copy link

These variable can be omitted or moved to cfg/config.yml

These variable can be omitted or moved to `cfg/config.yml`
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:58:10 +00:00
services/s3_lifecycler/docker-compose.yml Outdated
@ -0,0 +28,4 @@
- S3_LIFECYCLER_CREDENTIAL_SOURCE_WALLETS_0_PASSPHRASE="cycle"
- S3_LIFECYCLER_LIFECYCLE_JOB_FETCHER_BUFFER=1000
- S3_LIFECYCLER_LIFECYCLE_EXECUTOR_POOL_SIZE=100
- S3_LIFECYCLER_LIFECYCLE_SERVICES=031a6c6fbbdf02ca351745fa86b9ba5a9452d785ac4f7fc2b7548ca2a46c4fcf4a
dkirillov commented 2024-10-29 13:58:10 +00:00
Member
Copy link

We can omit this param

We can omit this param
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-29 13:59:29 +00:00
services/s3_lifecycler/docker-compose.yml Outdated
@ -0,0 +26,4 @@
- S3_LIFECYCLER_CREDENTIAL_SOURCE_WALLETS_0_PATH=./wallet.json
- S3_LIFECYCLER_CREDENTIAL_SOURCE_WALLETS_0_ADDRESS=NTt1rxvmEDxEuuogLxs2xgxA71qhVaUcN7
- S3_LIFECYCLER_CREDENTIAL_SOURCE_WALLETS_0_PASSPHRASE="cycle"
- S3_LIFECYCLER_LIFECYCLE_JOB_FETCHER_BUFFER=1000
dkirillov commented 2024-10-29 13:59:29 +00:00
Member
Copy link

Please keep here only variable that use ${LOCAL_DOMAIN}. Other params move to cfg/conig.yml and omit them if they have default values

Please keep here only variable that use `${LOCAL_DOMAIN}`. Other params move to `cfg/conig.yml` and omit them if they have default values
👍 1
dkirillov marked this conversation as resolved
pogpp was assigned by dkirillov 2024-10-29 14:00:09 +00:00
dkirillov commented 2024-10-29 14:00:35 +00:00
Member
Copy link

Please, add close #85 to PR description

Please, add `close #85` to PR description
👍 1
pogpp force-pushed feature/85_add_lifecycler from d67680c17d to 84f884da81 2024-10-30 13:16:00 +00:00 Compare
pogpp force-pushed feature/85_add_lifecycler from 84f884da81 to 45d3793406 2024-10-30 14:16:05 +00:00 Compare
pogpp force-pushed feature/85_add_lifecycler from 45d3793406 to 4772731866 2024-10-30 19:52:41 +00:00 Compare
dkirillov reviewed 2024-10-31 06:42:09 +00:00
services/s3_lifecycler/docker-compose.yml Outdated
@ -0,0 +17,4 @@
- ./../../vendor/hosts:/etc/hosts
- ./cfg:/etc/frostfs/s3-lifecycler
stop_signal: SIGKILL
env_file: [ ".env", ".s3_lifecycler.env", ".int_test.env" ]
dkirillov commented 2024-10-31 06:42:09 +00:00
Member
Copy link

Actually, I'm not sure if we need .s3_lifecycler.env. Can we drop this file at all?

Actually, I'm not sure if we need `.s3_lifecycler.env`. Can we drop this file at all?
👍 1
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-31 06:43:25 +00:00
services/s3_lifecycler/cfg/config.yml Outdated
@ -0,0 +2,4 @@
level: debug
prometheus:
enabled: false
dkirillov commented 2024-10-31 06:43:25 +00:00
Member
Copy link

Let's enable metrics.
Probably we should choose another port (but it should be checked)

Let's enable metrics. Probably we should choose another port (but it should be checked)
👍 1
dkirillov marked this conversation as resolved
pogpp force-pushed feature/85_add_lifecycler from 4772731866 to 0abfd23211 2024-10-31 07:55:29 +00:00 Compare
pogpp force-pushed feature/85_add_lifecycler from 0abfd23211 to fd6ff3d087 2024-10-31 08:08:05 +00:00 Compare
pogpp force-pushed feature/85_add_lifecycler from fd6ff3d087 to ab1a19eb32 2024-10-31 08:09:26 +00:00 Compare
pogpp force-pushed feature/85_add_lifecycler from ab1a19eb32 to a0fdaebbf4 2024-11-06 19:55:20 +00:00 Compare
alexvanin approved these changes 2024-11-07 14:56:50 +00:00
dkirillov approved these changes 2024-11-08 10:17:50 +00:00
dkirillov left a comment
Member
Copy link

LGTM

LGTM
alexvanin merged commit a0fdaebbf4 into master 2024-11-08 14:18:39 +00:00
alexvanin deleted branch feature/85_add_lifecycler 2024-11-08 14:18:48 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
mbiryukova
alexvanin
dkirillov
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
good first issue

Good for newcomers

  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
pogpp
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-dev-env#87
No description provided.
Delete branch "pogpp/frostfs-dev-env:feature/85_add_lifecycler"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?