TrueCloudLab/frostfs-node

Move shard to read-only if there were problems with blobstor #446

New issue

Closed

opened 2023-06-14 14:52:04 +00:00 by fyrchik · 0 comments

fyrchik commented

2023-06-14 14:52:04 +00:00

Owner

Currently we move to a DEGRADED-READ-ONLY if error counter exceeds the threshold

err := sh.SetMode(mode.DegradedReadOnly)

If there we problems with the blobstor, degraded mode is an overkill, we should just move to read-only.

Possible approach:

Make metaerr wrapper similar to how we handle logical errors.
Use it in metabase, pilorama and writecache packages.

This way we should make less errors in error handling.

Consider having 2 errors counters with the same threshold (we can report cumulative counter in control service for now).

Currently we move to a DEGRADED-READ-ONLY if error counter exceeds the threshold https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/fb8fee0c8edd3cca1d9aaa40bf7334df46400964/pkg/local_object_storage/engine/engine.go#L90 If there we problems with the blobstor, degraded mode is an overkill, we should just move to read-only. Possible approach: 1. Make `metaerr` wrapper similar to how we handle logical errors. 2. Use it in `metabase`, `pilorama` and `writecache` packages. This way we should make less errors in error handling. Consider having 2 errors counters with the same threshold (we can report cumulative counter in control service for now).

fyrchik added the

labels 2023-06-14 14:52:04 +00:00

fyrchik added this to the v0.37.0 milestone 2023-06-14 14:52:27 +00:00

fyrchik self-assigned this 2023-06-15 07:43:15 +00:00

fyrchik referenced this issue from a pull request that will close it,

2023-06-15 12:00:31 +00:00

Move to read-only on blobstor errors #450

fyrchik referenced this issue from a commit

2023-06-19 15:34:00 +00:00

[#446] engine: Simplify logs for shard mode change

fyrchik referenced this issue from a commit

2023-06-19 15:34:00 +00:00

[#446] los: Wrap SSD errors in a separate type

fyrchik referenced this issue from a commit

2023-06-19 15:34:00 +00:00

[#446] engine: Move to read-only on blobstor errors

fyrchik closed this issue

2023-06-19 15:34:01 +00:00

fyrchik referenced this issue

2024-04-01 09:49:15 +00:00

Revert "[#446] engine: Move to read-only on blobstor errors" #1066

fyrchik removed the

label 2024-04-01 09:52:28 +00:00

fyrchik referenced this issue from a commit

2024-04-01 12:37:21 +00:00

Revert "[#446] engine: Move to read-only on blobstor errors"

fyrchik referenced this issue

2024-04-02 13:54:13 +00:00

Revert "[#446] engine: Move to read-only on blobstor errors" #1071

fyrchik referenced this issue from a commit

2024-04-02 14:37:11 +00:00

[#1071] Revert "[#446] engine: Move to read-only on blobstor errors"

fyrchik referenced this issue

2024-06-11 14:30:57 +00:00

Reapply "[#446] engine: Move to read-only on blobstor errors" #1172

fyrchik referenced this issue from a commit

2024-06-13 07:36:01 +00:00

Reapply "[#446] engine: Move to read-only on blobstor errors"

fyrchik referenced this issue from a commit

2024-06-13 07:36:04 +00:00

Reapply "[#446] engine: Move to read-only on blobstor errors"

Sign in to join this conversation.

No labels

good first issue

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#446

No description provided.