5370 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Dmitrii Stepanov	07a660fbc4	[#1677] writecache: Add QoS limiter usage ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-14 16:23:33 +03:00
Dmitrii Stepanov	7893d763d1	[#1673] logger: Add sampling for journald logger ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-13 14:52:03 +03:00
Dmitrii Stepanov	ff4e9b6ae1	[#1673] logger: Drop unused fields ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-13 14:52:02 +03:00
Alexander Chuprov	997759994a	[#1676] golangci: Enable gci linter ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-03-13 12:04:01 +03:00
Evgenii Stratonikov	ecb6b0793c	[#1671] Use slices.ContainsFunc() where possible ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-13 08:12:20 +00:00
Evgenii Stratonikov	460e5cbccf	[#1671] Use slices.Delete() where possible ... gopatch is missing for this one, because https://github.com/uber-go/gopatch/issues/179 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-13 08:12:20 +00:00
Evgenii Stratonikov	155d3ddb6e	[#1671] Use min builtin where possible ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-13 08:12:20 +00:00
Evgenii Stratonikov	40536d8a06	[#1671] Use fmt.Appendf where warranted ... Fix gopls warnings: ``` cmd/frostfs-adm/internal/modules/morph/config/config.go:68:20-64: Replace []byte(fmt.Sprintf...) with fmt.Appendf ```` gopatch: ``` @@ var f expression @@ -[]byte(fmt.Sprintf(f, ...)) +fmt.Appendf(nil, f, ...) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-13 08:12:20 +00:00
Evgenii Stratonikov	d66bffb191	[#1671] Use max builtin where possible ... gopatcH: ``` @@ var d, a expression @@ -if d < a { - d = a -} -return d +return max(d, a) @@ var d, a expression @@ -if d <= a { - d = a -} -return d +return max(d, a) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-13 08:12:20 +00:00
Evgenii Stratonikov	bcc84c85a0	[#1671] Replace interface{} with any ... gopatch: ``` @@ @@ -interface{} +any ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-13 08:12:20 +00:00
Dmitrii Stepanov	737788b35f	[#1669] go.mod: Bump frostfs-qos version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-12 09:23:53 +03:00
Dmitrii Stepanov	2005fdda09	[#1667] shard: Drop shard pool ... After adding an ops limiter, shard's `put` pool is redundant. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-11 13:59:51 +03:00
Dmitrii Stepanov	597bce7a87	[#1653] treeSvc: Add operations by IO tag metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-11 10:57:47 +00:00
Dmitrii Stepanov	4ed2bbdb0f	[#1653] objectSvc: Add operations by IO tag metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-11 10:57:47 +00:00
Dmitrii Stepanov	3727d60331	[#1653] qos: Add metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-11 10:57:47 +00:00
Dmitrii Stepanov	d36afa31c7	[#1653] qos: Fix logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-11 10:57:47 +00:00
Dmitrii Stepanov	8643e0abc5	[#1668] writecache: Use object size to check free space ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-10 17:52:57 +03:00
Ekaterina Lebedeva	bd61f7bf0a	[#1666] audit: Fix duplicated log in Patch method ... When we do `object patch` with audit enabled we get several duplicated entries in logs. `object patch` request is logged in 2 places: 1. `(*auditPatchStream) CloseAndRecv()` - when the client closes the request stream or when stream gets aborted. 2. `(*auditPatchStream) Send()` - when stream was NOT aborted. `Send()` doesn't check if `err != nil` before logging. It led to to logging on every `Send()` call. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 13:27:07 +00:00
Ekaterina Lebedeva	df6d9da82a	[#1635] cli: Add command to get object's shard info ... Added `frostfs-cli object locate` subcommand. It lists info about shards storing an object. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 14:32:01 +03:00
Ekaterina Lebedeva	aab8addae0	[#1635] cli: Make object.readObjectAddress() public ... This method will be useful for upcoming control command. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 14:32:01 +03:00
Ekaterina Lebedeva	9e31cb249f	[#1635] control: Add method to search shards by object ... Added method `ListShardsForObject` to ControlService and to StorageEngine. It returns information about shards storing object on the node. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-07 14:32:01 +03:00
Evgenii Stratonikov	6260d703ce	[#1665] treesvc: Disable service config query ... By default, gRPC fetches TXT report while resolving a domain. 0914bba6c5/internal/resolver/dns/dns_resolver.go (L336) This leads to a hanging dial if DNS is unavailable, even though the host may be specified in `/etc/hosts` (hello, localhost!). SDK client for the main API uses these options by default. Refs TrueCloudLab/frostfs-sdk-go#342 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-06 15:15:31 +03:00
Evgenii Stratonikov	a17c3356fa	[#1665] go.mod: Update sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-06 15:15:31 +03:00
Ekaterina Lebedeva	471aeeaff3	[#1659] audit: Fix duplicated request logs ... When we do `object put` with audit enabled we get several entries in logs: with and without object id. `object put` request is logged in 2 places: 1. `(*auditPutStream) CloseAndRecv()` - when the client closes the request stream or when stream gets aborted. 2. `(*auditPutStream) Send()` - when stream was NOT aborted. `Send()` does error check for `ErrAbortStream` because if there is any other error - CloseAndRecv will not be called and there won't be any audit log about failed request. It led to logging on every object chunck put, even if `err == nil`. Added check for `err != nil` in `Send()` to fix it. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-06 12:36:16 +03:00
Airat Arifullin	4c8f9580a1	[#1662] object: Fix CloseAndRecv for patch streamer ... * A client may open stream to server, not send anything and close the open stream immediatly. This shouldn't cause a panic; * Return the error if `s.patcher` is uninitialized. Uninitialized patcher cannot be closed, this causes a panic. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-03-05 17:16:10 +03:00
Aleksey Savchuk	bf8914fedc	[#1416] lens/explorer: Support metabase schema v3 ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-03-05 07:20:05 +00:00
Evgenii Stratonikov	5ba0e2918e	[#1661] cli: Clarify --rpc-endpoint values ... `multiaddr` is not something an average user knows. Personally, I have never used it in CLI. On the other hand, we need to connect with TLS quite often, so it needs to be mentioned in help. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-04 10:19:49 +03:00
Dmitrii Stepanov	4685afb1dc	[#1636] engine: Validate limiter release in unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:30 +03:00
Dmitrii Stepanov	eb8b9b2b3b	[#1636] blobovniczatree: Validate limiter release in rebuild unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:30 +03:00
Dmitrii Stepanov	6c6e463b73	[#1636] shard: Change ops limiter on shard reload ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:29 +03:00
Dmitrii Stepanov	401d96a89e	[#1636] config: Refactor newConfig and oldConfig ... `newConfig` is actually target config to set config values from source (which is called `oldConfig`). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:29 +03:00
Dmitrii Stepanov	8ed71a969e	[#1636] qos: Add semaphore limiter ... If no tags specified, then limiter could be optimized to use atomic semaphore. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:29 +03:00
Dmitrii Stepanov	c2d855aedd	[#1636] qos: Return Resource Exhausted error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:28 +03:00
Dmitrii Stepanov	2162f8e189	[#1636] object: Fix IO tag adjustment for Put/Patch ... There was no tag adjustment for CloseAndRecv. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:28 +03:00
Dmitrii Stepanov	b9360be1dc	[#1636] blobovniczatree: Use RebuildLimiter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:27 +03:00
Dmitrii Stepanov	ceff5e1f6a	[#1636] storage: Refactor shard rebuild ... Drop redundant interfaces. Rename fields. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:27 +03:00
Dmitrii Stepanov	e0dc3c3d0c	[#1636] shard: Add limiter usage ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:26 +03:00
Dmitrii Stepanov	92a67a6716	[#1636] qos: Add Limiter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:26 +03:00
Dmitrii Stepanov	98d6125029	[#1636] config: Add shard.limits config ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 17:25:25 +03:00
Dmitrii Stepanov	0991077cb3	[#1657] engine: Fix data race in evacuation tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-02-28 14:14:12 +00:00
Aleksey Savchuk	c660271039	[#1658] node: Validate RPC limiter configuration ... Validate that configured limits match the methods registered earlier. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 16:31:47 +03:00
Aleksey Savchuk	92ab58984b	[#1658] node: Simplify RPC limiter initialization ... - Move all initialization logic to one place - Initialize the limiter after all RPC services are registered to be able to validate that configured limits match the methods registered earlier Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 16:24:33 +03:00
Aleksey Savchuk	dae0949f6e	[#1639] node: Support active RPC limiting ... - Allow configuration of active RPC limits for method groups - Apply RPC limiting for all services except the control service Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	5590886599	[#1639] qos: Add interceptors for limiting active RPCs ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	f0b2017057	[#1639] go.mod: Update sdk-go and qos ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	dce269c62e	[#1639] services/object: Remove limiting pools for Put operation ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Aleksey Savchuk	a97bded440	[#1639] config: Separate replicator.pool_size from other settings ... Separated `replicator.pool_size` and `object.put.remote_pool_size` settings. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-02-28 11:08:09 +00:00
Airat Arifullin	9a0507704a	[#1651] apemanager: Wrap some errors with InvalidArgument status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-02-27 18:25:09 +00:00
Alexander Chuprov	2ff032db90	[#1654] adm: Get 'gasInitialTotalSupply' from network ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-27 11:57:45 +00:00
Alexander Chuprov	37972a91c1	[#1654] adm: Make 'morph-init' idempotent ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-02-27 11:57:45 +00:00