achuprov/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
5330 commits 57 branches 27 tags 29 MiB
Commit graph

5330 commits

This branch
This branch
All branches
Author SHA1 Message Date
Aleksey Savchuk
c660271039
[#1658] node: Validate RPC limiter configuration 
Validate that configured limits match the methods registered earlier.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 16:31:47 +03:00
Aleksey Savchuk
92ab58984b
[#1658] node: Simplify RPC limiter initialization 
- Move all initialization logic to one place
- Initialize the limiter after all RPC services are registered to be able
  to validate that configured limits match the methods registered earlier

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 16:24:33 +03:00
Aleksey Savchuk
dae0949f6e [#1639] node: Support active RPC limiting 
- Allow configuration of active RPC limits for method groups
- Apply RPC limiting for all services except the control service

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
5590886599 [#1639] qos: Add interceptors for limiting active RPCs 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
f0b2017057 [#1639] go.mod: Update sdk-go and qos 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
dce269c62e [#1639] services/object: Remove limiting pools for Put operation 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Aleksey Savchuk
a97bded440 [#1639] config: Separate replicator.pool_size from other settings 
Separated `replicator.pool_size` and `object.put.remote_pool_size` settings.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-28 11:08:09 +00:00
Airat Arifullin
9a0507704a [#1651] apemanager: Wrap some errors with InvalidArgument status 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2025-02-27 18:25:09 +00:00
Alexander Chuprov
2ff032db90 [#1654] adm: Get 'gasInitialTotalSupply' from network 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-27 11:57:45 +00:00
Alexander Chuprov
37972a91c1 [#1654] adm: Make 'morph-init' idempotent 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-27 11:57:45 +00:00
Evgenii Stratonikov
003d568ae2
[#1628] innerring: Relax container homomorphic hashing check 
Our initial desire was to prohibit using homomorphic hashing on the
network level because of the resource consumption. However, the ability
to use it, doesn't mean that we must. So only fail validation if
container wants to have homomorphic hashing, but the network prohibits
it.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-02-27 10:20:08 +03:00
Alexander Chuprov
b2adf1109e
[#1646] cli: Use Cmp() functions for ID-like structs 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-20 12:27:07 +03:00
Aleksey Savchuk
02f3a7f65c
[#1648] writecache: Fix race condition when reporting cache size metrics 
There is a race condition when multiple cache operation try to report
the cache size metrics simultaneously. Consider the following example:
- the initial total size of objects stored in the cache size is 2
- worker X deletes an object and reads the cache size, which is 1
- worker Y deletes an object and reads the cache size, which is 0
- worker Y reports the cache size it learnt, which is 0
- worker X reports the cache size it learnt, which is 1

As a result, the observed cache size is 1 (i. e. one object remains
in the cache), which is incorrect because the actual cache size is 0.

To fix this, let's report the metrics periodically in the flush loop.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-19 17:05:40 +03:00
Dmitrii Stepanov
9b29e7392f
[#1647] go.mod: Bump frostfs-sdk-go version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-18 09:39:43 +03:00
Dmitrii Stepanov
fe0cf86dc6
[#1643] go.mod: Bump frostfs-observability version 
To add `grpc_client_msg_send_handling_seconds` metric.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-12 15:39:29 +03:00
Vitaliy Potyarkin
1bcaa1af1f [#1641] govulncheck: Fix minor toolchain updates for good 
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2025-02-11 14:02:23 +00:00
Alexander Chuprov
304bee938b
[#1614] adm/nns: Add 'set-admin' 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-11 12:00:51 +03:00
Alexander Chuprov
b2163ff44c
[#1614] adm: Allow use any wallets 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-11 12:00:25 +03:00
Alexander Chuprov
076952f4c7
[#1614] adm/frostfsid: Add 'delete-kv' 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-10 15:28:29 +03:00
Alexander Chuprov
a7145ca9bf
[#1614] adm/frostfsid: Add 'set-kv' 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-10 15:26:12 +03:00
Dmitrii Stepanov
5d79abe523
[#1608] shard: Add IO tag for rebuild 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:07 +03:00
Dmitrii Stepanov
0671c277db
[#1608] tree: Add IO tag for tree sync requests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:06 +03:00
Dmitrii Stepanov
92450a76ba
[#1608] shard: Add IO tag to GC 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:06 +03:00
Dmitrii Stepanov
abba5b2089
[#1608] writecache: Add IO tag to flush worker 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:06 +03:00
Dmitrii Stepanov
fd0c6c461d
[#1608] cli: Drop redundant interceptors 
There is no metrics or IO tags in cli.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:05 +03:00
Dmitrii Stepanov
bfe325e61d
[#1608] policer: Add IO tag to context 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:05 +03:00
Dmitrii Stepanov
acec938b2d
[#1608] qos: Add client grpc interceptors 
`qos` client interceptors replace internal IO tags `writecache`,
`policer` and `background` with `internal` IO tag for outcomming RPC.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:04 +03:00
Dmitrii Stepanov
dc6aea7b79
[#1608] control: Use IO tag critical 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:04 +03:00
Dmitrii Stepanov
170860c14a
[#1608] logger: Add IO tag logging 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:03 +03:00
Dmitrii Stepanov
12da2f8262
[#1608] object: Add IO tag adjustment layer 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:03 +03:00
Dmitrii Stepanov
f6b3f79e89
[#1608] qos: Add qos service to adjust incoming IO tags 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:43:03 +03:00
Dmitrii Stepanov
9729f31e5c
[#1608] grpc: Add QoS interceptors for server and clients 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:41:06 +03:00
Dmitrii Stepanov
155f9eecb0
[#1608] config: Add QoS section and config 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-07 15:41:06 +03:00
Alexander Chuprov
69c35b1d61
[#1637] govulncheck: Use patch release with security fixes 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-06 21:19:24 +03:00
Alexander Chuprov
9b113c3156
[#1613] morph: Add tracing for morph queries to neo-go 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-05 16:38:20 +03:00
Dmitrii Stepanov
4de5fca547
[#1624] blobovniczatree: Return source error from Init() 
As it was before: when the database was opened, an error returned,
but along with the original error, the `context cancelled`` error returned,
because `iterateIncompletedRebuildDBPaths` method has `ctx.Done()` check
and egCtx passed to `iterateIncompletedRebuildDBPaths` method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-03 17:36:21 +03:00
Aleksey Savchuk
9c4c5a5262
[#1602] go.mod: Update sdk-go 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2025-02-03 14:28:23 +03:00
Evgenii Stratonikov
6fcae9f75a [#1621] treesvc: Cancel background sync on failure 
If applyOperationStream() exits prematurely, other goroutines will block
on send and errgroup will never finish waiting. In this commit we also
check whether context is cancelled.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-02-03 09:37:55 +00:00
Evgenii Stratonikov
1df64c5cab [#1621] treesvc: Move variable initialization to top in mergeOperationStreams() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-02-03 09:37:55 +00:00
Dmitrii Stepanov
6a580db55e
[#1626] go.mod: Bump frostfs-sdk-go 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-30 16:23:00 +03:00
Ekaterina Lebedeva
24054cf6f4 [#1618] pkg: Refactor user.ID.ScriptHash() usage 
`user.ID.ScriptHash()` does not return an error anymore.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2025-01-30 13:18:37 +00:00
Ekaterina Lebedeva
9ee3dd4e91 [#1618] ape: Fix object service request tests 
Constant string `testOwnerID` for these tests has an invalid
format. It has 11 bytes instead of required 25 for `user.ID`.
It worked because:
  1. `user.ID` was a byte slice and didn't check length
     and format of byte slices decoded from strings.
  2. in these tests `testOwnerID` was used only to decode
     container owner id and to compare it with owner id encoded
     back to string.
Since `user.ID implementation has changed`, the problem arised.
Now `testOwnerID` is valid.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2025-01-30 13:18:37 +00:00
Dmitrii Stepanov
78bfd12229
[#1622] adm: Return non zero return code if metabase upgrade failed 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-30 15:31:25 +03:00
Evgenii Stratonikov
57dc0a8e9e [#1616] getsvc: Move break condition from body to the loop condition 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-01-30 06:50:37 +00:00
Evgenii Stratonikov
b309b34bfc [#1616] getsvc: Simplify buildChainRange() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-01-30 06:50:37 +00:00
Evgenii Stratonikov
c8acdf40bb [#1616] getsvc: Use slices.Reverse() where possible 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-01-30 06:50:37 +00:00
Evgenii Stratonikov
6410542d19 [#1616] getsvc: Move range assembling to a separate file 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-01-30 06:50:37 +00:00
Evgenii Stratonikov
c0a341a7f6 [#1616] getsvc: Move head assembling to a separate file 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-01-30 06:50:37 +00:00
Dmitrii Stepanov
e1a984e9d8
[#1620] logs: Drop redundant trace_id fields 
`trace_id` is taken from context.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-29 16:13:51 +03:00
Vitaliy Potyarkin
abfd9657f9 [#1617] govulncheck: Use patch release with security fixes 
https://go.dev/doc/devel/release#go1.23.minor

Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2025-01-28 18:02:57 +03:00