5355 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Dmitrii Stepanov	d36afa31c7	[#1653] qos: Fix logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-11 10:57:47 +00:00
Dmitrii Stepanov	8643e0abc5	[#1668] writecache: Use object size to check free space ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-10 17:52:57 +03:00
Ekaterina Lebedeva	bd61f7bf0a	[#1666] audit: Fix duplicated log in Patch method ... When we do `object patch` with audit enabled we get several duplicated entries in logs. `object patch` request is logged in 2 places: 1. `(*auditPatchStream) CloseAndRecv()` - when the client closes the request stream or when stream gets aborted. 2. `(*auditPatchStream) Send()` - when stream was NOT aborted. `Send()` doesn't check if `err != nil` before logging. It led to to logging on every `Send()` call. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 13:27:07 +00:00
Ekaterina Lebedeva	df6d9da82a	[#1635] cli: Add command to get object's shard info ... Added `frostfs-cli object locate` subcommand. It lists info about shards storing an object. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 14:32:01 +03:00
Ekaterina Lebedeva	aab8addae0	[#1635] cli: Make object.readObjectAddress() public ... This method will be useful for upcoming control command. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 14:32:01 +03:00
Ekaterina Lebedeva	9e31cb249f	[#1635] control: Add method to search shards by object ... Added method `ListShardsForObject` to ControlService and to StorageEngine. It returns information about shards storing object on the node. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 14:32:01 +03:00
Evgenii Stratonikov	6260d703ce	[#1665] treesvc: Disable service config query ... By default, gRPC fetches TXT report while resolving a domain. 0914bba6c5/internal/resolver/dns/dns_resolver.go (L336) This leads to a hanging dial if DNS is unavailable, even though the host may be specified in `/etc/hosts` (hello, localhost!). SDK client for the main API uses these options by default. Refs TrueCloudLab/frostfs-sdk-go#342 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-06 15:15:31 +03:00
Evgenii Stratonikov	a17c3356fa	[#1665] go.mod: Update sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-06 15:15:31 +03:00
Ekaterina Lebedeva	471aeeaff3	[#1659] audit: Fix duplicated request logs ... When we do `object put` with audit enabled we get several entries in logs: with and without object id. `object put` request is logged in 2 places: 1. `(*auditPutStream) CloseAndRecv()` - when the client closes the request stream or when stream gets aborted. 2. `(*auditPutStream) Send()` - when stream was NOT aborted. `Send()` does error check for `ErrAbortStream` because if there is any other error - CloseAndRecv will not be called and there won't be any audit log about failed request. It led to logging on every object chunck put, even if `err == nil`. Added check for `err != nil` in `Send()` to fix it. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-06 12:36:16 +03:00
Airat Arifullin	4c8f9580a1	[#1662] object: Fix CloseAndRecv for patch streamer ... * A client may open stream to server, not send anything and close the open stream immediatly. This shouldn't cause a panic; * Return the error if `s.patcher` is uninitialized. Uninitialized patcher cannot be closed, this causes a panic. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-03-05 17:16:10 +03:00
Aleksey Savchuk	bf8914fedc	[#1416] lens/explorer: Support metabase schema v3 ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-03-05 07:20:05 +00:00
Evgenii Stratonikov	5ba0e2918e	[#1661] cli: Clarify --rpc-endpoint values ... `multiaddr` is not something an average user knows. Personally, I have never used it in CLI. On the other hand, we need to connect with TLS quite often, so it needs to be mentioned in help. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-04 10:19:49 +03:00
Dmitrii Stepanov	4685afb1dc	[#1636] engine: Validate limiter release in unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:30 +03:00
Dmitrii Stepanov	eb8b9b2b3b	[#1636] blobovniczatree: Validate limiter release in rebuild unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:30 +03:00
Dmitrii Stepanov	6c6e463b73	[#1636] shard: Change ops limiter on shard reload ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:29 +03:00
Dmitrii Stepanov	401d96a89e	[#1636] config: Refactor newConfig and oldConfig ... `newConfig` is actually target config to set config values from source (which is called `oldConfig`). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:29 +03:00
Dmitrii Stepanov	8ed71a969e	[#1636] qos: Add semaphore limiter ... If no tags specified, then limiter could be optimized to use atomic semaphore. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:29 +03:00
Dmitrii Stepanov	c2d855aedd	[#1636] qos: Return Resource Exhausted error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:28 +03:00
Dmitrii Stepanov	2162f8e189	[#1636] object: Fix IO tag adjustment for Put/Patch ... There was no tag adjustment for CloseAndRecv. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:28 +03:00
Dmitrii Stepanov	b9360be1dc	[#1636] blobovniczatree: Use RebuildLimiter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:27 +03:00
Dmitrii Stepanov	ceff5e1f6a	[#1636] storage: Refactor shard rebuild ... Drop redundant interfaces. Rename fields. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:27 +03:00
Dmitrii Stepanov	e0dc3c3d0c	[#1636] shard: Add limiter usage ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:26 +03:00
Dmitrii Stepanov	92a67a6716	[#1636] qos: Add Limiter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:26 +03:00
Dmitrii Stepanov	98d6125029	[#1636] config: Add shard.limits config ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:25 +03:00
Dmitrii Stepanov	0991077cb3	[#1657] engine: Fix data race in evacuation tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 14:14:12 +00:00
Aleksey Savchuk	c660271039	[#1658] node: Validate RPC limiter configuration ... Validate that configured limits match the methods registered earlier. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 16:31:47 +03:00
Aleksey Savchuk	92ab58984b	[#1658] node: Simplify RPC limiter initialization ... - Move all initialization logic to one place - Initialize the limiter after all RPC services are registered to be able to validate that configured limits match the methods registered earlier Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 16:24:33 +03:00
Aleksey Savchuk	dae0949f6e	[#1639] node: Support active RPC limiting ... - Allow configuration of active RPC limits for method groups - Apply RPC limiting for all services except the control service Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	5590886599	[#1639] qos: Add interceptors for limiting active RPCs ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	f0b2017057	[#1639] go.mod: Update sdk-go and qos ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	dce269c62e	[#1639] services/object: Remove limiting pools for Put operation ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	a97bded440	[#1639] config: Separate replicator.pool_size from other settings ... Separated `replicator.pool_size` and `object.put.remote_pool_size` settings. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Airat Arifullin	9a0507704a	[#1651] apemanager: Wrap some errors with InvalidArgument status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-02-27 18:25:09 +00:00
Alexander Chuprov	2ff032db90	[#1654] adm: Get 'gasInitialTotalSupply' from network ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-27 11:57:45 +00:00
Alexander Chuprov	37972a91c1	[#1654] adm: Make 'morph-init' idempotent ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-27 11:57:45 +00:00
Evgenii Stratonikov	003d568ae2	[#1628] innerring: Relax container homomorphic hashing check ... Our initial desire was to prohibit using homomorphic hashing on the network level because of the resource consumption. However, the ability to use it, doesn't mean that we must. So only fail validation if container wants to have homomorphic hashing, but the network prohibits it. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-02-27 10:20:08 +03:00
Alexander Chuprov	b2adf1109e	[#1646] cli: Use Cmp() functions for ID-like structs ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-20 12:27:07 +03:00
Aleksey Savchuk	02f3a7f65c	[#1648] writecache: Fix race condition when reporting cache size metrics ... There is a race condition when multiple cache operation try to report the cache size metrics simultaneously. Consider the following example: - the initial total size of objects stored in the cache size is 2 - worker X deletes an object and reads the cache size, which is 1 - worker Y deletes an object and reads the cache size, which is 0 - worker Y reports the cache size it learnt, which is 0 - worker X reports the cache size it learnt, which is 1 As a result, the observed cache size is 1 (i. e. one object remains in the cache), which is incorrect because the actual cache size is 0. To fix this, let's report the metrics periodically in the flush loop. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-19 17:05:40 +03:00
Dmitrii Stepanov	9b29e7392f	[#1647] go.mod: Bump frostfs-sdk-go version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-18 09:39:43 +03:00
Dmitrii Stepanov	fe0cf86dc6	[#1643] go.mod: Bump frostfs-observability version ... To add `grpc_client_msg_send_handling_seconds` metric. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-12 15:39:29 +03:00
Vitaliy Potyarkin	1bcaa1af1f	[#1641] govulncheck: Fix minor toolchain updates for good ... Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>	2025-02-11 14:02:23 +00:00
Alexander Chuprov	304bee938b	[#1614] adm/nns: Add 'set-admin' ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-11 12:00:51 +03:00
Alexander Chuprov	b2163ff44c	[#1614] adm: Allow use any wallets ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-11 12:00:25 +03:00
Alexander Chuprov	076952f4c7	[#1614] adm/frostfsid: Add 'delete-kv' ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-10 15:28:29 +03:00
Alexander Chuprov	a7145ca9bf	[#1614] adm/frostfsid: Add 'set-kv' ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-10 15:26:12 +03:00
Dmitrii Stepanov	5d79abe523	[#1608] shard: Add IO tag for rebuild ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-07 15:43:07 +03:00
Dmitrii Stepanov	0671c277db	[#1608] tree: Add IO tag for tree sync requests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-07 15:43:06 +03:00
Dmitrii Stepanov	92450a76ba	[#1608] shard: Add IO tag to GC ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-07 15:43:06 +03:00
Dmitrii Stepanov	abba5b2089	[#1608] writecache: Add IO tag to flush worker ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-07 15:43:06 +03:00
Dmitrii Stepanov	fd0c6c461d	[#1608] cli: Drop redundant interceptors ... There is no metrics or IO tags in cli. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-07 15:43:05 +03:00