4dc9a1b300
[ #1413 ] engine: Remove error counting methods from Shard
...
All error counting and hangling logic is present on the engine level.
Currently, we pass engine metrics with shard ID metric to shard, then
export 3 methods to manipulate these metrics.
In this commits all methods are removed and error counter is tracked on
the engine level exlusively.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-04 15:10:17 +03:00
9206ce5cd2
[ #1410 ] shard: Provide the default implementation for MetricsWriter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
6c46044c9c
[ #1410 ] shard: Move MetricsWriter interface to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
d3b209c8e1
[ #1337 ] shard: Disable background rebuild
...
Since `frostfs-cli control shards rebuild` command was added,
there is no need for background rebuild now.
For failover tests used used value 1 to rebuild only schema change.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 15:19:55 +03:00
a61201a987
[ #1337 ] config: Move rebuild_worker_count to shard section
...
This makes it simple to limit performance degradation for every shard
because of rebuild.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
d508da8397
[ #1337 ] blobovniczatree: Add rebuild by fill percent
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
108e4e07be
[ #1349 ] node: Evacuate objects without setting mode to MAINTENANCE
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-05 16:08:27 +03:00
2b3fc50681
[ #1320 ] shard: Fix TestGCDropsObjectInhumedFromWritecache flaky test
...
The `TestGCDropsObjectInhumedFromWritecache` test was flaky because a
running asynchronous rebuild operation prevented GC from deleting the
object. A test-only shard option `WithDisabledRebuild` has been added
to fix this.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-02 10:26:53 +03:00
93d63e1632
[ #1284 ] writecache: Allow to seal writecache async
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 13:23:35 +03:00
68ac490729
[ #1174 ] shard: Update metric mode_info on Init
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-13 08:32:59 +00:00
c9efaa5819
[ #966 ] node: Add path of the write_cache to metric labels
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
1b17258c04
[ #1029 ] metabase: Add refill metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 13:05:44 +03:00
57466594fb
[ #1024 ] shard: Resync metabase concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
66a26b7775
[ #973 ] node: Resolve revive: unused-parameter linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:11:49 +03:00
675eec91f3
[ #938 ] shard: Update only changed counters
...
If metric value hasn't changed, but we update metric, then
non existed metric will apear with zero value.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
a2ab373a0a
[ #895 ] metabase: Do not delete GC mark for virtual objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
4b8b4da681
[ #864 ] engine: Drop container count metric if container removed
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:45:32 +03:00
d75e7e9a21
[ #864 ] engine: Drop container size metric if container deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:44:54 +03:00
8180a0664f
[ #887 ] node: Drop badger writecache implementation
...
Badger implementation isn't tested and works not well,
but requires human resources to maintain.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-22 13:00:54 +03:00
f1c7905263
[ #661 ] blobovniczatree: Make Rebuild concurrent
...
Different DBs can be rebuild concurrently.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
44552a849b
[ #661 ] shard: Add blobstor rebuilder
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
d30ab5f29e
[ #838 ] metabase: Count user objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
f314da4af3
[ #838 ] metabase: Add user object type counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
29550fe600
[ #838 ] shard: Refactor updateMetrics method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
07390ad4e3
[ #715 ] node: Unify config parameter names
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
70ab1ebd54
[ #763 ] metrics: Add container_objects_total metric
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
1a0cb0f34a
[ #421 ] Try using badger for the write-cache
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 08:16:57 +00:00
cac4ed93d6
[ #428 ] engine: Add low_mem config parameter
...
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
059e9e88a2
[ #373 ] metabase: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
4449006862
[ #424 ] metrics: Use mode value as metric value for shard
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00
1b364d8cf4
[ #424 ] metrics: Refactor engine metrics
...
Use histogram vector to measure request duration.
Fix naming like in Prometheus best practice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
263c6fdc50
[ #372 ] node: Add metrics for the error counter in the engine
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-07 13:04:47 +00:00
3220c4df9f
[ #376 ] metrics: Add GC metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-31 10:22:12 +00:00
2613351008
[ #387 ] gc: Cancel GC is change mode requested
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-25 09:38:16 +03:00
2ce43935f9
[ #312 ] metrics: Add writecache metrcis
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
0e31c12e63
[ #240 ] logs: Move log messages to constants
...
Drop duplicate entities.
Format entities.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
9f0bce5c15
[ #183 ] gc: Fix drop expired locked simple objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-30 15:33:42 +03:00
5059dcc19d
[ #145 ] shard-gc: Delete expired objects after locks
...
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
20de74a505
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
ab21d90cfb
[ #1794 ] shard: Add increasing case for the payload size metric
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-02-09 13:30:23 +03:00
cb016d53a6
[ #1 ] Fix comments and error messages
...
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
Pavel Karpy
89a0266f5e
[ #1794 ] metrics: Track physical object capacity per shard
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-01-26 20:06:28 +03:00
Evgenii Stratonikov
9513f163aa
[ #2116 ] metrics: Track physical object capacity in the container
...
Currently we track based on `PayloadSize`, because it is already stored
in the metabase and it is easier to calculate without slowing down the
whole system.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-01-26 20:06:28 +03:00
edb1428248
[ #2022 ] Add metric readonly to get shards mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a
Move to frostfs-node
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
d65604ad30
[ #1985 ] blobstor: Allow to report multiple errors to caller
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
777fd32d4f
[ #1818 ] writecache: Increase error counter on background errors
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
f769fc83fc
[ #1869 ] shard: Embed gcCfg as raw struct
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-24 13:28:26 +03:00
Evgenii Stratonikov
d772e35aba
[ #1910 ] .golangci.yml: Add godot linker
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-18 15:08:26 +03:00
Pavel Karpy
31c623636d
[ #1863 ] node: Fix shard id in the object counter metrics
...
If shard ID is stored in metabase (it is not the first time boot), read it,
set it, use it (not a generated one) in the metrics writer.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-13 13:06:41 +03:00
