Move shard to read-only if there were problems with blobstor #446

New Issue

Closed

opened 2023-06-14 14:52:04 +00:00 by fyrchik · 0 comments

fyrchik commented

2023-06-14 14:52:04 +00:00

Owner

Currently we move to a DEGRADED-READ-ONLY if error counter exceeds the threshold fb8fee0c8e/pkg/local_object_storage/engine/engine.go (L90)

If there we problems with the blobstor, degraded mode is an overkill, we should just move to read-only.

Possible approach:

Make metaerr wrapper similar to how we handle logical errors.
Use it in metabase, pilorama and writecache packages.

This way we should make less errors in error handling.

Consider having 2 errors counters with the same threshold (we can report cumulative counter in control service for now).

Currently we move to a DEGRADED-READ-ONLY if error counter exceeds the threshold https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/fb8fee0c8edd3cca1d9aaa40bf7334df46400964/pkg/local_object_storage/engine/engine.go#L90 If there we problems with the blobstor, degraded mode is an overkill, we should just move to read-only. Possible approach: 1. Make `metaerr` wrapper similar to how we handle logical errors. 2. Use it in `metabase`, `pilorama` and `writecache` packages. This way we should make less errors in error handling. Consider having 2 errors counters with the same threshold (we can report cumulative counter in control service for now).

fyrchik added the

labels 2023-06-14 14:52:04 +00:00

fyrchik added this to the v0.37.0 milestone 2023-06-14 14:52:27 +00:00

fyrchik self-assigned this 2023-06-15 07:43:15 +00:00

fyrchik referenced a pull request that will close this issue

2023-06-15 12:00:31 +00:00

Move to read-only on blobstor errors #450

fyrchik referenced this issue from a commit

2023-06-19 15:34:00 +00:00

[#446] engine: Simplify logs for shard mode change

fyrchik referenced this issue from a commit

2023-06-19 15:34:00 +00:00

[#446] los: Wrap SSD errors in a separate type

fyrchik referenced this issue from a commit

2023-06-19 15:34:00 +00:00

[#446] engine: Move to read-only on blobstor errors

fyrchik closed this issue

2023-06-19 15:34:01 +00:00

fyrchik referenced this issue

2024-04-01 09:49:15 +00:00

Revert "[#446] engine: Move to read-only on blobstor errors" #1066

fyrchik removed the

label 2024-04-01 09:52:28 +00:00

fyrchik referenced this issue from a commit

2024-04-01 12:37:21 +00:00

Revert "[#446] engine: Move to read-only on blobstor errors"

fyrchik referenced this issue

2024-04-02 13:54:13 +00:00

Revert "[#446] engine: Move to read-only on blobstor errors" #1071

fyrchik referenced this issue from a commit

2024-04-02 14:37:11 +00:00

[#1071] Revert "[#446] engine: Move to read-only on blobstor errors"

Sign in to join this conversation.

No Label

good first issue

No Milestone

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#446

There is no content yet.