Speedup metabase resync #1024

New Issue

Closed

opened 2024-03-05 07:10:02 +00:00 by fyrchik · 1 comment

fyrchik commented

2024-03-05 07:10:02 +00:00

Owner

It is currently done in one thread.
And it currently uses batch, so it is guaranteed 10ms per object, take 100M objects and we have 1000_000 seconds or 11 days.
Far from ideal.

It is currently done in one thread. And it currently uses batch, so it is guaranteed 10ms per object, take 100M objects and we have 1000_000 seconds or 11 days. Far from ideal.

fyrchik added the

labels 2024-03-05 07:10:02 +00:00

dstepanov-yadro was assigned by fyrchik

2024-03-05 07:10:13 +00:00

dstepanov-yadro referenced this issue

2024-03-05 14:00:52 +00:00

Speed up metabase refill #1027

fyrchik added this to the v0.39.0 milestone 2024-04-05 13:36:53 +00:00

fyrchik referenced a pull request that will close this issue

2024-04-05 13:37:11 +00:00

Speed up metabase refill #1027

dstepanov-yadro referenced a pull request that will close this issue

2024-04-08 08:07:00 +00:00

Speed up metabase refill #1027

fyrchik closed this issue

2024-04-10 12:09:00 +00:00

fyrchik referenced this issue from a commit

2024-04-10 12:09:01 +00:00

[#1024] shard: Add refill metabase benchmark

fyrchik referenced this issue from a commit

2024-04-10 12:09:01 +00:00

[#1024] shard: Resync metabase concurrently

fyrchik referenced this issue from a commit

2024-04-10 12:09:01 +00:00

[#1024] blobovnicza: Copy data on iterate

dstepanov-yadro commented

2024-04-10 12:10:12 +00:00

Collaborator

Now it is able to perform resync with many workers.

Now it is able to perform resync with many workers.

Sign in to join this conversation.

No Label

good first issue

No Milestone

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1024

There is no content yet.