TrueCloudLab/frostfs-s3-gw
18
3
Fork 20
Code Issues 29 Pull requests 7 Projects Releases 23 Packages 1 Activity Actions

[#398] Support retryer #398

Merged
alexvanin merged 1 commit from dkirillov/frostfs-s3-gw:bugfix/retry_put_bucket_settings_on_creation into master 2024-06-06 13:02:23 +00:00
Conversation 0 Commits 1 Files changed 12 +288 -1
dkirillov commented 2024-05-30 13:04:53 +00:00
Member
Copy link

Add two strategy for PutBucketSettings request retryer:

  • exponential backoff (increasing up to max_backoff delays with jitter)
  • constant backoff (always the same max_backoff delay between requests)

Signed-off-by: Denis Kirillov d.kirillov@yadro.com

This PR is blocked by TrueCloudLab/frostfs-sdk-go#226

Add two strategy for PutBucketSettings request retryer: * exponential backoff (increasing up to `max_backoff` delays with jitter) * constant backoff (always the same `max_backoff` delay between requests) Signed-off-by: Denis Kirillov <d.kirillov@yadro.com> This PR is blocked by https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/pulls/226
dkirillov self-assigned this 2024-05-30 13:04:54 +00:00
dkirillov changed title from WIP: [#XX] Support retryer to WIP: [#398] Support retryer 2024-05-30 13:05:08 +00:00
dkirillov force-pushed bugfix/retry_put_bucket_settings_on_creation from fe3452e170 to 923ed520d4 2024-05-30 13:22:58 +00:00 Compare
dkirillov force-pushed bugfix/retry_put_bucket_settings_on_creation from 923ed520d4 to 707ee11d55 2024-05-31 07:45:54 +00:00 Compare
dkirillov changed title from WIP: [#398] Support retryer to [#398] Support retryer 2024-05-31 09:09:53 +00:00
dkirillov requested review from storage-services-committers 2024-05-31 09:10:01 +00:00
dkirillov requested review from storage-services-developers 2024-05-31 09:10:03 +00:00
alexvanin reviewed 2024-06-04 14:23:41 +00:00
api/handler/api.go Outdated
@ -49,1 +49,4 @@
ACLEnabled() bool
RetryMaxAttempts() int
RetryMaxBackoff() time.Duration
RetryStrategy() RetryStrategy
alexvanin commented 2024-06-04 14:21:16 +00:00
Owner
Copy link

What you think about grouping those retry variables in single retryCfg object`?

What you think about grouping those retry variables in single `retryCfg` object`?
dkirillov commented 2024-06-05 08:25:24 +00:00
Author
Member
Copy link

Good idea. But then probably we need do the same for other parameters (placement policy related etc). Do we want such changes?

Good idea. But then probably we need do the same for other parameters (placement policy related etc). Do we want such changes?
alexvanin commented 2024-06-06 13:01:46 +00:00
Owner
Copy link

I didn't notice placement policy related parameters at first. Then it's okay to keep it for a while.

I didn't notice placement policy related parameters at first. Then it's okay to keep it for a while.
alexvanin marked this conversation as resolved
api/handler/api.go Outdated
@ -66,0 +71,4 @@
const (
RetryStrategyExponential = "exponential"
RetryStrategyConstant = "constant"
)
alexvanin commented 2024-06-04 14:19:18 +00:00
Owner
Copy link

Matter of taste, but do you consider keeping this in pkg/retryer?

Matter of taste, but do you consider keeping this in `pkg/retryer`?
dkirillov commented 2024-06-05 08:31:45 +00:00
Author
Member
Copy link

Hm... Is it ok that these constant won't be used in package when they are located? If it's ok then I'll move them

Hm... Is it ok that these constant won't be used in package when they are located? If it's ok then I'll move them
alexvanin commented 2024-06-06 13:00:40 +00:00
Owner
Copy link

Agree, this is application level constants, there is no need to define it in package.

Agree, this is application level constants, there is no need to define it in package.
alexvanin marked this conversation as resolved
go.mod Outdated
@ -10,3 +10,4 @@
git.frostfs.info/TrueCloudLab/policy-engine v0.0.0-20240527065402-303a81cdc6db
git.frostfs.info/TrueCloudLab/zapjournald v0.0.0-20240124114243-cb2e66427d02
github.com/aws/aws-sdk-go v1.44.6
github.com/aws/aws-sdk-go-v2 v1.18.1
alexvanin commented 2024-06-04 14:23:39 +00:00
Owner
Copy link

Can we keep single aws-sdk-go dependency across all the code? I don't mind to ditch v1

Can we keep single `aws-sdk-go` dependency across all the code? I don't mind to ditch v1
dkirillov commented 2024-06-05 08:27:23 +00:00
Author
Member
Copy link

I would do this when branch https://git.frostfs.info/dkirillov/frostfs-s3-gw/commits/branch/feature/339-sigv4a_support be merged to master

I would do this when branch https://git.frostfs.info/dkirillov/frostfs-s3-gw/commits/branch/feature/339-sigv4a_support be merged to master
👍 1
alexvanin marked this conversation as resolved
dkirillov force-pushed bugfix/retry_put_bucket_settings_on_creation from 707ee11d55 to cbf2bbae11 2024-06-05 08:34:04 +00:00 Compare
alexvanin added this to the v0.30.0 milestone 2024-06-06 13:02:09 +00:00
alexvanin merged commit bb81afc14a into master 2024-06-06 13:02:23 +00:00
alexvanin referenced this pull request from a commit 2024-06-06 13:02:28 +00:00
[#398] Support retryer
dkirillov deleted branch bugfix/retry_put_bucket_settings_on_creation 2024-06-11 12:15:11 +00:00
alexvanin modified the milestone from v0.30.0 to v0.29.1 2024-06-18 11:04:26 +00:00
alexvanin referenced this pull request from a commit 2024-06-20 15:04:04 +00:00
[#398] Support retryer
Sign in to join this conversation.
Reviewers
No reviewers
TrueCloudLab/storage-services-committers
TrueCloudLab/storage-services-developers
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
good first issue

Good for newcomers

  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
v0.29.1
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
dkirillov
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-s3-gw#398
No description provided.
Delete branch "dkirillov/frostfs-s3-gw:bugfix/retry_put_bucket_settings_on_creation"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?