feature/115-adopt_put_single_pool #140

Merged
dkirillov merged 5 commits from dkirillov/frostfs-sdk-go:feature/115-adopt_put_single_pool into master 2024-09-04 19:51:15 +00:00
Member

close #115

Tests showed that using part buffer pool doesn't save memory a lot.
Especially on big parts.
Probably we can use pool only for small parts
after adding buffer in payloadSizeLimiter

UPD: PR is going to be merge into master

close #115 Tests showed that using part buffer pool doesn't save memory a lot. Especially on big parts. Probably we can use pool only for small parts after adding buffer in `payloadSizeLimiter` UPD: PR is going to be merge into `master`
dkirillov self-assigned this 2023-08-03 07:38:21 +00:00
dkirillov changed title from WIP: feature/115-adopt_put_single_pool to WIP: feature/115-adopt_put_single_pool 2023-08-03 07:38:47 +00:00
dkirillov changed target branch from master to pool-client-cut 2023-08-03 07:38:49 +00:00
dkirillov force-pushed feature/115-adopt_put_single_pool from c2290454aa to 922b3411c5 2023-08-04 14:06:41 +00:00 Compare
dkirillov force-pushed feature/115-adopt_put_single_pool from 922b3411c5 to 2ec3f19647 2023-08-04 14:21:56 +00:00 Compare
dkirillov changed title from WIP: feature/115-adopt_put_single_pool to feature/115-adopt_put_single_pool 2023-08-04 14:22:19 +00:00
dkirillov requested review from storage-core-committers 2023-08-04 14:22:28 +00:00
dkirillov requested review from storage-core-developers 2023-08-04 14:22:31 +00:00
dkirillov requested review from storage-services-committers 2023-08-04 14:22:31 +00:00
dkirillov requested review from storage-services-developers 2023-08-04 14:22:32 +00:00
dkirillov force-pushed feature/115-adopt_put_single_pool from 2ec3f19647 to 875746a8a9 2023-08-07 08:48:16 +00:00 Compare
dkirillov changed title from feature/115-adopt_put_single_pool to feature/115-adopt_put_single_pool 2023-08-07 08:48:32 +00:00
dkirillov changed target branch from pool-client-cut to master 2023-08-07 08:48:33 +00:00
dstepanov-yadro requested changes 2023-08-07 11:20:07 +00:00
@ -0,0 +23,4 @@
withoutHomomorphicHash bool
}
const defaultMaxPutSingleAttempts = 3

If I didn't specify that I want to retry the request, then why will the SDK try to do this?

If I didn't specify that I want to retry the request, then why will the SDK try to do this?
dstepanov-yadro marked this conversation as resolved
@ -0,0 +105,4 @@
}
func (it *internalTarget) WriteObject(ctx context.Context, o *object.Object) error {
for i := 0; i < it.prm.maxAttempts; i++ {

We should not try to repeat all the failed requests, and we also need to add backoff and jitter to do it correctly:
https://aws.amazon.com/ru/blogs/architecture/exponential-backoff-and-jitter/

Therefore, perhaps this is not necessary at all now.

We should not try to repeat all the failed requests, and we also need to add backoff and jitter to do it correctly: https://aws.amazon.com/ru/blogs/architecture/exponential-backoff-and-jitter/ Therefore, perhaps this is not necessary at all now.
Author
Member

Dropped retrying. Probably we can review approach in #25

Dropped retrying. Probably we can review approach in #25
dstepanov-yadro marked this conversation as resolved
dkirillov force-pushed feature/115-adopt_put_single_pool from 875746a8a9 to 236d7e3886 2023-08-09 13:47:41 +00:00 Compare
dkirillov requested review from dstepanov-yadro 2023-08-09 13:48:49 +00:00
dstepanov-yadro approved these changes 2023-08-11 07:03:22 +00:00
dkirillov force-pushed feature/115-adopt_put_single_pool from 236d7e3886 to e3fec9c7a2 2023-08-17 13:44:33 +00:00 Compare
dkirillov force-pushed feature/115-adopt_put_single_pool from e3fec9c7a2 to 202412230a 2023-08-21 09:04:49 +00:00 Compare
alexvanin approved these changes 2023-08-21 11:41:32 +00:00
@ -713,0 +748,4 @@
}
if prm.payload != nil {
const defaultBufferSizePut = 64 * 1024 // it's buffer size in s3-gw, configure?
Owner

Should we create an issue for that?

Should we create an issue for that?
Author
Member
https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/issues/149
pogpp approved these changes 2023-08-21 12:07:19 +00:00
dkirillov merged commit 202412230a into master 2023-08-21 12:09:47 +00:00
dkirillov deleted branch feature/115-adopt_put_single_pool 2023-08-21 12:09:47 +00:00
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-core-developers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-sdk-go#140
No description provided.