Anton Nikiforov
2429508ac5
[ #959 ] shard: Skip rebuild in `DEGRADED` mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
Anton Nikiforov
0bd030507e
[ #948 ] metrics: Set actual value for `shard_id` after restart
...
DCO action / DCO (pull_request) Successful in 2m26s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m56s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m18s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m11s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Details
Tests and linters / Lint (pull_request) Successful in 6m3s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s
Details
Tests and linters / Tests with -race (pull_request) Successful in 9m20s
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:43:21 +03:00
Anton Nikiforov
d19ade23c8
[ #959 ] node: Set mode to shard's components when open it
...
Avoid opening database for `metabase` and `cache` in `Degraded` mode.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-09 14:04:01 +00:00
Dmitrii Stepanov
d75e7e9a21
[ #864 ] engine: Drop container size metric if container deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:44:54 +03:00
Dmitrii Stepanov
f1c7905263
[ #661 ] blobovniczatree: Make Rebuild concurrent
...
Different DBs can be rebuild concurrently.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
Dmitrii Stepanov
44552a849b
[ #661 ] shard: Add blobstor rebuilder
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
Ekaterina Lebedeva
8a81af5a3b
[ #653 ] Add context parameter to Open functions
...
DCO action / DCO (pull_request) Successful in 1m38s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m22s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m25s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m56s
Details
Tests and linters / Lint (pull_request) Successful in 6m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 8m1s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m14s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m32s
Details
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
Alejandro Lopez
5b7e4a51b7
[ #481 ] Update frostfs-sdk-go and error pointer receivers
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
Dmitrii Stepanov
cac4ed93d6
[ #428 ] engine: Add low_mem config parameter
...
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
Dmitrii Stepanov
059e9e88a2
[ #373 ] metabase: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
74578052f9
[ #412 ] node: Replace tracing package
...
Use observability module.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Anton Nikiforov
802168c0c6
[ #364 ] node: Stop flushing big object when termination signal received
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-26 16:46:58 +03:00
Dmitrii Stepanov
2613351008
[ #387 ] gc: Cancel GC is change mode requested
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-25 09:38:16 +03:00
Dmitrii Stepanov
d62c6e4ce6
[ #242 ] node: Add tracing spans
...
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
Evgenii Stratonikov
0e31c12e63
[ #240 ] logs: Move log messages to constants
...
Drop duplicate entities.
Format entities.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
Dmitrii Stepanov
a7c79c773a
[ #168 ] node: Refactor node config
...
Resolve containedctx linter for cfg
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
Dmitrii Stepanov
9f0bce5c15
[ #183 ] gc: Fix drop expired locked simple objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-30 15:33:42 +03:00
Evgenii Stratonikov
fb13902db9
[ #156 ] shard: Make refillMetabase() pass linter checks
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-22 07:14:18 +00:00
Dmitrii Stepanov
5059dcc19d
[ #145 ] shard-gc: Delete expired objects after locks
...
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
Dmitrii Stepanov
97c36ed3ec
[ #148 ] linter: Add funlen linter
...
Long functions are hard to understand and source of errors
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:54:41 +03:00
Alexey Vanin
20de74a505
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
Evgenii Stratonikov
427fe276f2
[ #2238 ] shard: Try closing all components
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
9513f163aa
[ #2116 ] metrics: Track physical object capacity in the container
...
Currently we track based on `PayloadSize`, because it is already stored
in the metabase and it is easier to calculate without slowing down the
whole system.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-01-26 20:06:28 +03:00
Evgenii Stratonikov
6451f019d2
[ #2203 ] shard: Do not panic in `Close` after unsuccessful `Init`
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
923f84722a
Move to frostfs-node
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
34501685b7
[ #1969 ] local_object_storage: Move `ErrObjectIsExpired` to another package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-31 11:41:24 +03:00
f8180447a1
[ #1938 ] meta: Make version error messages more descriptive
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-24 21:46:18 +03:00
1beafea0b5
[ #1869 ] shard: Add logs for `SetMode` operations on reload
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-24 13:28:26 +03:00
87be4f1629
[ #1869 ] shard: Restore shard mode on failed reloads
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-24 13:28:26 +03:00
c785e11b20
[ #1869 ] shard: Allow to reload metabase on SIGHUP
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-24 13:28:26 +03:00
f769fc83fc
[ #1869 ] shard: Embed `gcCfg` as raw struct
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-24 13:28:26 +03:00
4eb0ed11f8
[ #1809 ] node: Do not boot up if metabase is outdated
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-04 12:32:10 +03:00
2d7166f8d0
[ #1770 ] shard: Move NewEpoch event routing on SE level
...
It will allow dynamic shard management. Closing a shard does not allow
removing event handlers.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-04 10:08:55 +03:00
d872862710
[ #1658 ] meta: Force counters resync process
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 21:32:37 +04:00
bc20851cd1
[ #1686 ] shard: Open all components in case metabase has failed
...
All components must be opened first, before any other operation is
performed.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 10:02:43 +03:00
9c7b3ce799
[ #1658 ] node: Read metrics from meta on startup
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-25 19:20:33 +03:00
73f8bb3e5f
[ #1523 ] shard: Store generic storage ID in metabase
...
Allow to extend blobstor with more storage sub-systems. Currently
objects stored in the FSTree have empty byte slice descriptor and object
from blobovnicza tree have the same id as earlier. Each such change in
the identifier formation should be accompanied with metabase version
increase.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
156ba85326
[ #1634 ] node: Do not return expired objects
...
If an object has not been marked for removal by the GC in the current epoch
yet but has already expired, respond with `ErrObjectNotFound` api status.
Also, optimize shard iteration: a node must stop any iteration if the object
is found but gonna be removed soon.
All the checks are performed by the Metabase.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:31:49 +03:00
fabe717d32
[ #1549 ] shard: Turn to read-only mode on metabase failure
...
If metabase can't be opened in the default mode, try opening shard
first in `ReadOnly` mode and then in `DegradedReadOnly`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
1e786233bf
[ #1559 ] local_object_storage: Provide readOnly flag to `Open`
...
We should be able to reopen storage in readonly in runtime.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
34d319fed2
[ #1559 ] metabase: Use `Set` prefix for parameter setting
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
f58234aa2f
[ #1559 ] metabase: Remove public functions
...
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
a4adb79db7
[ #1607 ] pilorama: Enable tree service explicitly
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
46f4ce2773
[ #1324 ] engine: Implement Forest interface for storage engine
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
7df50297cd
[ #1520 ] shard: Ignore errors on metabase refill
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:25:28 +03:00
78ea450c25
[ #1502 ] shard: Process locks on metabase refill
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 11:07:26 +03:00
e265ce2d52
[ #1418 ] meta: Do not use pointers as parameters
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
1c30414a6c
[ #1454 ] Upgrade NeoFS SDK Go module with new IDs
...
Core changes:
* avoid package-colliding variable naming
* avoid using pointers to IDs where unnecessary
* avoid using `idSDK` import alias pattern
* use `EncodeToString` for protocol string calculation and `String` for
printing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-01 17:41:45 +03:00
f15e6e888f
[ #1377 ] oid, cid: Upgrade SDK package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-16 15:33:22 +03:00
70ffdf3478
[ #1247 ] object: Return `NOT_FOUND` and `ALREADY_REMOVED` statuses
...
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Anton Nikiforov
Dmitrii Stepanov
Ekaterina Lebedeva
Alejandro Lopez
Evgenii Stratonikov
Alexey Vanin
Evgenii Stratonikov
Pavel Karpy
Pavel Karpy
Evgenii Stratonikov
Leonard Lyubich