TrueCloudLab
/
frostfs-s3-gw
15
2
Fork 14
Code Issues 20 Pull Requests 5 Actions Packages Releases 6 Activity

[#398] Support retryer #398

Merged
alexvanin merged 1 commits from dkirillov/frostfs-s3-gw:bugfix/retry_put_bucket_settings_on_creation into master 2024-06-06 13:02:23 +00:00
Conversation 0 Commits 1 Files Changed 12
dkirillov commented 2024-05-30 13:04:53 +00:00
Collaborator

Add two strategy for PutBucketSettings request retryer:

  • exponential backoff (increasing up to max_backoff delays with jitter)
  • constant backoff (always the same max_backoff delay between requests)

Signed-off-by: Denis Kirillov d.kirillov@yadro.com

This PR is blocked by TrueCloudLab/frostfs-sdk-go#226

Add two strategy for PutBucketSettings request retryer: * exponential backoff (increasing up to `max_backoff` delays with jitter) * constant backoff (always the same `max_backoff` delay between requests) Signed-off-by: Denis Kirillov <d.kirillov@yadro.com> This PR is blocked by https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/pulls/226
dkirillov self-assigned this 2024-05-30 13:04:54 +00:00
dkirillov added 1 commit 2024-05-30 13:04:55 +00:00
/ Builds (1.20) (pull_request) Failing after 1m30s Details
/ Builds (1.21) (pull_request) Failing after 1m26s Details
/ DCO (pull_request) Failing after 1m26s Details
/ Vulncheck (pull_request) Failing after 1m45s Details
/ Lint (pull_request) Failing after 2m13s Details
/ Tests (1.20) (pull_request) Failing after 1m26s Details
/ Tests (1.21) (pull_request) Failing after 48s Details
fe3452e170 [#XX] Support retryer 
Add two strategy for PutBucketSettings request retryer:
* exponential backoff (increasing up to `max_backoff` delays with jitter)
* constant backoff (always the same `max_backoff` delay between requests)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
dkirillov changed title from WIP: [#XX] Support retryer to WIP: [#398] Support retryer 2024-05-30 13:05:08 +00:00
dkirillov force-pushed bugfix/retry_put_bucket_settings_on_creation from fe3452e170 to 923ed520d4 2024-05-30 13:22:58 +00:00 Compare
dkirillov force-pushed bugfix/retry_put_bucket_settings_on_creation from 923ed520d4 to 707ee11d55 2024-05-31 07:45:54 +00:00 Compare
dkirillov changed title from WIP: [#398] Support retryer to [#398] Support retryer 2024-05-31 09:09:53 +00:00
dkirillov requested review from storage-services-committers 2024-05-31 09:10:01 +00:00
dkirillov requested review from storage-services-developers 2024-05-31 09:10:03 +00:00
alexvanin reviewed 2024-06-04 14:23:41 +00:00
api/handler/api.go Outdated
@ -49,1 +49,4 @@
ACLEnabled() bool
RetryMaxAttempts() int
RetryMaxBackoff() time.Duration
RetryStrategy() RetryStrategy
alexvanin commented 2024-06-04 14:21:16 +00:00
Owner

What you think about grouping those retry variables in single retryCfg object`?

What you think about grouping those retry variables in single `retryCfg` object`?
dkirillov commented 2024-06-05 08:25:24 +00:00
Poster
Collaborator

Good idea. But then probably we need do the same for other parameters (placement policy related etc). Do we want such changes?

Good idea. But then probably we need do the same for other parameters (placement policy related etc). Do we want such changes?
alexvanin commented 2024-06-06 13:01:46 +00:00
Owner

I didn't notice placement policy related parameters at first. Then it's okay to keep it for a while.

I didn't notice placement policy related parameters at first. Then it's okay to keep it for a while.
alexvanin marked this conversation as resolved
api/handler/api.go Outdated
@ -66,0 +71,4 @@
const (
RetryStrategyExponential = "exponential"
RetryStrategyConstant = "constant"
)
alexvanin commented 2024-06-04 14:19:18 +00:00
Owner

Matter of taste, but do you consider keeping this in pkg/retryer?

Matter of taste, but do you consider keeping this in `pkg/retryer`?
dkirillov commented 2024-06-05 08:31:45 +00:00
Poster
Collaborator

Hm... Is it ok that these constant won't be used in package when they are located? If it's ok then I'll move them

Hm... Is it ok that these constant won't be used in package when they are located? If it's ok then I'll move them
alexvanin commented 2024-06-06 13:00:40 +00:00
Owner

Agree, this is application level constants, there is no need to define it in package.

Agree, this is application level constants, there is no need to define it in package.
alexvanin marked this conversation as resolved
go.mod Outdated
@ -10,3 +10,4 @@
git.frostfs.info/TrueCloudLab/policy-engine v0.0.0-20240527065402-303a81cdc6db
git.frostfs.info/TrueCloudLab/zapjournald v0.0.0-20240124114243-cb2e66427d02
github.com/aws/aws-sdk-go v1.44.6
github.com/aws/aws-sdk-go-v2 v1.18.1
alexvanin commented 2024-06-04 14:23:39 +00:00
Owner

Can we keep single aws-sdk-go dependency across all the code? I don't mind to ditch v1

Can we keep single `aws-sdk-go` dependency across all the code? I don't mind to ditch v1
dkirillov commented 2024-06-05 08:27:23 +00:00
Poster
Collaborator

I would do this when branch https://git.frostfs.info/dkirillov/frostfs-s3-gw/commits/branch/feature/339-sigv4a_support be merged to master

I would do this when branch https://git.frostfs.info/dkirillov/frostfs-s3-gw/commits/branch/feature/339-sigv4a_support be merged to master
👍 1
alexvanin marked this conversation as resolved
dkirillov force-pushed bugfix/retry_put_bucket_settings_on_creation from 707ee11d55 to cbf2bbae11 2024-06-05 08:34:04 +00:00 Compare
alexvanin added this to the v0.30.0 milestone 2024-06-06 13:02:09 +00:00
alexvanin merged commit bb81afc14a into master 2024-06-06 13:02:23 +00:00
alexvanin referenced this issue from a commit 2024-06-06 13:02:28 +00:00
[#398] Support retryer
dkirillov deleted branch bugfix/retry_put_bucket_settings_on_creation 2024-06-11 12:15:11 +00:00
alexvanin modified the milestone from v0.30.0 to v0.29.1 2024-06-18 11:04:26 +00:00
alexvanin referenced this issue from a commit 2024-06-20 15:04:04 +00:00
[#398] Support retryer
Sign in to join this conversation.
Reviewers
No reviewers
TrueCloudLab/storage-services-committers
TrueCloudLab/storage-services-developers
Labels
Clear labels   
P0

Highest   
P1

High   
P2

Medium   
P3

Low   
good first issue

Good for newcomers
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff   
blocked

The issue or PR is blocked by something   
bug

Something is not working   
config

Configuration format update or breaking change   
discussion

Talking about something in order to reach a decision or to exchange ideas   
documentation

Documentation related   
duplicate

This issue or pull request already exists   
enhancement

New feature   
go

Language features adoption   
help wanted

Extra attention is needed   
internal
   
invalid

Something is wrong   
kludge

This is a kludge and we know it   
observability

Related to metrics, tracing and logs   
perfomance

Perfomance related   
question

More information is needed   
refactoring

Refactoring   
wontfix

This won't be fixed
No Label
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No Milestone
v0.29.1
Assignees
Clear assignees
No Assignees
dkirillov
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-s3-gw#398
There is no content yet.
Delete Branch "dkirillov/frostfs-s3-gw:bugfix/retry_put_bucket_settings_on_creation"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?