fyrchik/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
5342 commits 20 branches 26 tags 25 MiB
Commit graph

5342 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dmitrii Stepanov
eb8b9b2b3b
[#1636] blobovniczatree: Validate limiter release in rebuild unit tests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:30 +03:00
Dmitrii Stepanov
6c6e463b73
[#1636] shard: Change ops limiter on shard reload 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:29 +03:00
Dmitrii Stepanov
401d96a89e
[#1636] config: Refactor newConfig and oldConfig 
`newConfig` is actually target config to set config values from source
(which is called `oldConfig`).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:29 +03:00
Dmitrii Stepanov
8ed71a969e
[#1636] qos: Add semaphore limiter 
If no tags specified, then limiter could be optimized to use atomic semaphore.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:29 +03:00
Dmitrii Stepanov
c2d855aedd
[#1636] qos: Return Resource Exhausted error 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:28 +03:00
Dmitrii Stepanov
2162f8e189
[#1636] object: Fix IO tag adjustment for Put/Patch 
There was no tag adjustment for CloseAndRecv.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:28 +03:00
Dmitrii Stepanov
b9360be1dc
[#1636] blobovniczatree: Use RebuildLimiter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:27 +03:00
Dmitrii Stepanov
ceff5e1f6a
[#1636] storage: Refactor shard rebuild 
Drop redundant interfaces.
Rename fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:27 +03:00
Dmitrii Stepanov
e0dc3c3d0c
[#1636] shard: Add limiter usage 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:26 +03:00
Dmitrii Stepanov
92a67a6716
[#1636] qos: Add Limiter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:26 +03:00
Dmitrii Stepanov
98d6125029
[#1636] config: Add shard.limits config 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:25 +03:00
Dmitrii Stepanov
0991077cb3 [#1657] engine: Fix data race in evacuation tests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 14:14:12 +00:00
Aleksey Savchuk
c660271039
[#1658] node: Validate RPC limiter configuration 
Validate that configured limits match the methods registered earlier.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 16:31:47 +03:00
Aleksey Savchuk
92ab58984b
[#1658] node: Simplify RPC limiter initialization 
- Move all initialization logic to one place
- Initialize the limiter after all RPC services are registered to be able
  to validate that configured limits match the methods registered earlier

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 16:24:33 +03:00
Aleksey Savchuk
dae0949f6e [#1639] node: Support active RPC limiting 
- Allow configuration of active RPC limits for method groups
- Apply RPC limiting for all services except the control service

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
5590886599 [#1639] qos: Add interceptors for limiting active RPCs 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
f0b2017057 [#1639] go.mod: Update sdk-go and qos 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
dce269c62e [#1639] services/object: Remove limiting pools for Put operation 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
a97bded440 [#1639] config: Separate replicator.pool_size from other settings 
Separated `replicator.pool_size` and `object.put.remote_pool_size` settings.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Airat Arifullin
9a0507704a [#1651] apemanager: Wrap some errors with InvalidArgument status 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2025-02-27 18:25:09 +00:00
Alexander Chuprov
2ff032db90 [#1654] adm: Get 'gasInitialTotalSupply' from network 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-27 11:57:45 +00:00
Alexander Chuprov
37972a91c1 [#1654] adm: Make 'morph-init' idempotent 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-27 11:57:45 +00:00
Evgenii Stratonikov
003d568ae2
[#1628] innerring: Relax container homomorphic hashing check 
Our initial desire was to prohibit using homomorphic hashing on the
network level because of the resource consumption. However, the ability
to use it, doesn't mean that we must. So only fail validation if
container wants to have homomorphic hashing, but the network prohibits
it.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-02-27 10:20:08 +03:00
Alexander Chuprov
b2adf1109e
[#1646] cli: Use Cmp() functions for ID-like structs 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-20 12:27:07 +03:00
Aleksey Savchuk
02f3a7f65c
[#1648] writecache: Fix race condition when reporting cache size metrics 
There is a race condition when multiple cache operation try to report
the cache size metrics simultaneously. Consider the following example:
- the initial total size of objects stored in the cache size is 2
- worker X deletes an object and reads the cache size, which is 1
- worker Y deletes an object and reads the cache size, which is 0
- worker Y reports the cache size it learnt, which is 0
- worker X reports the cache size it learnt, which is 1

As a result, the observed cache size is 1 (i. e. one object remains
in the cache), which is incorrect because the actual cache size is 0.

To fix this, let's report the metrics periodically in the flush loop.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-19 17:05:40 +03:00
Dmitrii Stepanov
9b29e7392f
[#1647] go.mod: Bump frostfs-sdk-go version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-18 09:39:43 +03:00
Dmitrii Stepanov
fe0cf86dc6
[#1643] go.mod: Bump frostfs-observability version 
To add `grpc_client_msg_send_handling_seconds` metric.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-12 15:39:29 +03:00
Vitaliy Potyarkin
1bcaa1af1f [#1641] govulncheck: Fix minor toolchain updates for good 
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2025-02-11 14:02:23 +00:00
Alexander Chuprov
304bee938b
[#1614] adm/nns: Add 'set-admin' 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-11 12:00:51 +03:00
Alexander Chuprov
b2163ff44c
[#1614] adm: Allow use any wallets 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-11 12:00:25 +03:00
Alexander Chuprov
076952f4c7
[#1614] adm/frostfsid: Add 'delete-kv' 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-10 15:28:29 +03:00
Alexander Chuprov
a7145ca9bf
[#1614] adm/frostfsid: Add 'set-kv' 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-10 15:26:12 +03:00
Dmitrii Stepanov
5d79abe523
[#1608] shard: Add IO tag for rebuild 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:07 +03:00
Dmitrii Stepanov
0671c277db
[#1608] tree: Add IO tag for tree sync requests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:06 +03:00
Dmitrii Stepanov
92450a76ba
[#1608] shard: Add IO tag to GC 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:06 +03:00
Dmitrii Stepanov
abba5b2089
[#1608] writecache: Add IO tag to flush worker 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:06 +03:00
Dmitrii Stepanov
fd0c6c461d
[#1608] cli: Drop redundant interceptors 
There is no metrics or IO tags in cli.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:05 +03:00
Dmitrii Stepanov
bfe325e61d
[#1608] policer: Add IO tag to context 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:05 +03:00
Dmitrii Stepanov
acec938b2d
[#1608] qos: Add client grpc interceptors 
`qos` client interceptors replace internal IO tags `writecache`,
`policer` and `background` with `internal` IO tag for outcomming RPC.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:04 +03:00
Dmitrii Stepanov
dc6aea7b79
[#1608] control: Use IO tag critical 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:04 +03:00
Dmitrii Stepanov
170860c14a
[#1608] logger: Add IO tag logging 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:03 +03:00
Dmitrii Stepanov
12da2f8262
[#1608] object: Add IO tag adjustment layer 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:03 +03:00
Dmitrii Stepanov
f6b3f79e89
[#1608] qos: Add qos service to adjust incoming IO tags 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:03 +03:00
Dmitrii Stepanov
9729f31e5c
[#1608] grpc: Add QoS interceptors for server and clients 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:41:06 +03:00
Dmitrii Stepanov
155f9eecb0
[#1608] config: Add QoS section and config 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:41:06 +03:00
Alexander Chuprov
69c35b1d61
[#1637] govulncheck: Use patch release with security fixes 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-06 21:19:24 +03:00
Alexander Chuprov
9b113c3156
[#1613] morph: Add tracing for morph queries to neo-go 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-05 16:38:20 +03:00
Dmitrii Stepanov
4de5fca547
[#1624] blobovniczatree: Return source error from Init() 
As it was before: when the database was opened, an error returned,
but along with the original error, the `context cancelled`` error returned,
because `iterateIncompletedRebuildDBPaths` method has `ctx.Done()` check
and egCtx passed to `iterateIncompletedRebuildDBPaths` method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-03 17:36:21 +03:00
Aleksey Savchuk
9c4c5a5262
[#1602] go.mod: Update sdk-go 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-03 14:28:23 +03:00
Evgenii Stratonikov
6fcae9f75a [#1621] treesvc: Cancel background sync on failure 
If applyOperationStream() exits prematurely, other goroutines will block
on send and errgroup will never finish waiting. In this commit we also
check whether context is cancelled.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-02-03 09:37:55 +00:00