Alexander Chuprov
8f070d8095
All checks were successful
DCO action / DCO (pull_request) Successful in 44s
Vulncheck / Vulncheck (pull_request) Successful in 53s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m29s
Build / Build Components (pull_request) Successful in 1m34s
Tests and linters / Run gofumpt (pull_request) Successful in 2m0s
Tests and linters / Tests (pull_request) Successful in 2m8s
Tests and linters / Staticcheck (pull_request) Successful in 2m9s
Tests and linters / gopls check (pull_request) Successful in 3m1s
Tests and linters / Lint (pull_request) Successful in 3m11s
Tests and linters / Tests with -race (pull_request) Successful in 3m52s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
65 lines
4.9 KiB
Markdown
65 lines
4.9 KiB
Markdown
# Shard modes description
|
|
|
|
## List of modes
|
|
|
|
Each mode is characterized by two important properties:
|
|
1. Whether modifying operations are allowed.
|
|
2. Whether metabase and write-cache is available.
|
|
The expected deployment scenario is to place both metabase and write-cache on an SSD drive thus these modes
|
|
can be approximately described as no-SSD modes.
|
|
|
|
| Mode | Description |
|
|
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
| `read-write` | Default mode, all operations are allowed. |
|
|
| `read-only` | Read-only mode, only read operations are allowed, metabase is available. |
|
|
| `degraded` | Degraded mode in which metabase and write-cache is disabled. It shouldn't be used at all, because metabase can contain important indices, such as LOCK objects info and modifying operation in this mode can lead to unexpected behaviour. The purpose of this mode is to allow PUT/DELETE operations without the metabase if really necessary. |
|
|
| `degraded-read-only` | Same as `degraded`, but with only read operations allowed. This mode is used during SSD replacement and/or when the metabase error counter exceeds threshold. |
|
|
| `disabled` | Currently used only in config file to temporarily disable a shard.
|
|
|
|
## Shard and Component Status
|
|
|
|
| Shard Mode | Metabase Mode | Blobstore Mode | Writecache Mode | Pilorama Mode | Blobovnicza Tree Mode | FSTree Mode |
|
|
|-----------------------|---------------|----------------|-----------------|---------------|-----------------------|-------------|
|
|
| `Read-Write` | READ_WRITE | READ_WRITE | READ_WRITE | READ_WRITE | READ_WRITE | READ_WRITE |
|
|
| `Read-Only` | READ_ONLY | READ_ONLY | READ_ONLY | READ_ONLY | READ_ONLY | READ_ONLY |
|
|
| `Degraded-Read-Write` | CLOSED | READ_WRITE | CLOSED | CLOSED | READ_WRITE | READ_WRITE |
|
|
| `Degraded-Read-Only` | CLOSED | READ_ONLY | CLOSED | CLOSED | READ_ONLY | READ_ONLY |
|
|
|
|
## Transition order
|
|
|
|
Because each shard consists of multiple components changing its mode is not an atomic operation.
|
|
Instead, each component changes its mode independently.
|
|
|
|
For transitions to `read-write` mode the order is:
|
|
1. `metabase`
|
|
2. `blobstor`
|
|
3. `writecache`
|
|
4. `pilorama`
|
|
|
|
For transitions to all other modes the order is:
|
|
1. `writecache`
|
|
2. `blobstor`
|
|
3. `metabase`
|
|
4. `pilorama`
|
|
|
|
The motivation is to avoid transient errors because write-cache can write to both blobstor and metabase.
|
|
Thus, when we want to _stop_ write operations, write-cache needs to change mode before them.
|
|
On the other side, when we want to _allow_ them, blobstor and metabase should be writable before write-cache is.
|
|
|
|
If anything goes wrong in the middle, the mode of some components can be different from the actual mode of a shard.
|
|
However, all mode changing operations are idempotent.
|
|
|
|
## Automatic mode changes
|
|
|
|
Shard can automatically switch to a `read-only` mode in 3 cases:
|
|
1. If the metabase was not available or couldn't be opened/initialized during shard startup.
|
|
2. If shard error counter exceeds threshold.
|
|
3. If the metabase couldn't be reopened during SIGHUP handling.
|
|
|
|
If switching to `read-only` is impossible, the shard will switch to `degraded-read-only`.
|
|
|
|
# Detach shard
|
|
|
|
To detach a shard use `frostfs-cli control shards detach` command. This command removes the shards from the storage
|
|
engine and closes all resources associated with the shards.
|
|
Limitation: `SIGHUP` or storage node restart lead to detached shard will be again online.
|