130 commits

Author	SHA1	Message	Date
Evgenii Stratonikov	3d953d7185	[#1605] policer: Do not mutate slice under iteration ... Nothing wrong with it, besides being difficult to read. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-20 16:30:11 +03:00
Evgenii Stratonikov	91c8a1a1fc	[#1605] policer: Simplify processRepNodes() checks ... Current flow is hard to reason about, #1601 is a notorious example of accidental complexity. 1. Remove multiple nested ifs, use depth=1. 2. Process each status exactly once, hopefully preventing bugs like #1601. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-20 16:30:11 +03:00
Evgenii Stratonikov	57efa0bc8e	[#1604] policer: Properly handle maintenance nodes ... Consider `REP 1 REP 1` placement (selects/filters are omitted). The placement is `[1, 2], [1, 0]`. We are the 0-th node. Node 1 is under maintenance, so we do not replicate object on the node 2. In the second replication group node 1 is under maintenance, but current caching logic considers it as "replica holder" and removes local copy. Voilà, we have DL if the object is missing from the node 1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:52 +03:00
Evgenii Stratonikov	26e0c82fb8	[#1604] policer/test: Add test for MAINTENANCE runtime status ... The node can have MAINTENANCE status in the network map, but can also be ONLINE while responding with MAINTENANCE. These are 2 different code paths, let's test them separately. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Evgenii Stratonikov	4538ccb12a	[#1604] policer: Do not process the same node twice ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Evgenii Stratonikov	84e1599997	[#1604] policer: Remove one-line helpers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Evgenii Stratonikov	5a270e2e61	[#1604] policer: Use status instead of bool value in node cache ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Aleksey Savchuk	f0c43c8d80	[#1502] Use zap.Error for logging errors ... Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like `zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`. Leave similar expressions with other messages unchanged, for example, `zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`. This change was made by applying the following patch: ```diff @@ var err expression @@ -zap.String("error", err.Error()) +zap.Error(err) @@ var err expression @@ -zap.String("err", err.Error()) +zap.Error(err) ``` Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 11:13:42 +03:00
Dmitrii Stepanov	15139d80c9	[#1548] policer: Do not replicate EC chunk if object already removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-11 07:26:33 +00:00
Dmitrii Stepanov	7429553266	[#1437] node: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:10 +03:00
Dmitrii Stepanov	6db46257c0	[#1437] node: Use ctx for logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:07 +03:00
Dmitrii Stepanov	33ad753302	[#1473] policer: Add tracing span ... To filter HEAD requests from policer. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:15:11 +00:00
Alexander Chuprov	d83879d4b8	[#1431] node: Fix comment format ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-10-15 08:53:59 +00:00
Dmitrii Stepanov	1b520f7973	[#1412] engine: Add IsIndexedContainer flag ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:49 +03:00
Dmitrii Stepanov	1e7f9909da	[#1382] policer: Replace deprecated methods ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-18 14:29:21 +03:00
Evgenii Stratonikov	03976c6ed5	[#1341] .golangci.yml: Replace exportloopref with copyloopvar ... exportloopref is deprecated. gopatch: ``` @@ var index, value identifier var slice expression @@ for index, value := range slice { ... -value := value ... } @@ var index, value identifier var slice expression @@ for index, value := range slice { ... -index := index ... } @@ var value identifier var channel expression @@ for value := range channel { ... -value := value ... } ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 15:44:41 +00:00
Dmitrii Stepanov	5e5ee545b6	[#1254] policer: Fix svacer warning ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-18 11:30:39 +00:00
Dmitrii Stepanov	8eb591d668	[#1231] policer: Add EC node-off unit test ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-05 12:11:36 +03:00
Ekaterina Lebedeva	5b100699d7	[#566] policer: Move isClientErrMaintenance to frostfs-sdk-go ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-06-18 10:20:45 +03:00
Airat Arifullin	5aacb8fc86	[#1144] metabase: Save parent attributes for ec-chunks ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:55:32 +03:00
Airat Arifullin	8fd678e269	[#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2 ... * Also fix unit-test. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:22:48 +03:00
Dmitrii Stepanov	436c9f5558	[#1129] policer: Restore EC object ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-17 14:36:18 +03:00
Dmitrii Stepanov	1cd8562db8	[#1129] policer: Refactor shortage ... Drop override inside method. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	4ab6c404f7	[#1129] policer: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	cbe9757490	[#1129] policer: Pull required EC chunks ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	d45d086acd	[#1129] policer: Add EC chunk replication ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:48 +03:00
Ekaterina Lebedeva	e07869a8cf	[#1100] Remove unused fields ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-05-06 10:14:36 +03:00
Evgenii Stratonikov	3dc81cb4fc	Reapply "[#972] Use min/max builtins" ... This reverts commit dad56d2e98. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Dmitrii Stepanov	f5b67c6735	[#1064] policer: Disable EC processing ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
aarifullin	4738508ce2	[#1063] go.mod: Update SDK version ... * Update frostfs-sdk and frostfs-api-go versions. * Refactor depreacted method ReplicaNumberByIndex. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-03 10:33:51 +00:00
Evgenii Stratonikov	dad56d2e98	Revert "[#972] Use min/max builtins" ... This reverts commit 89784b2e0a. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	89784b2e0a	[#972] Use min/max builtins ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	edbe06e07e	[#956] policer/test: Reuse testPool helper ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Evgenii Stratonikov	cbfeb72466	[#956] policer: Remove WithMaxCapacity option ... We already provide the pool and this argument is used only for preallocation. No functional changes. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Evgenii Stratonikov	c3fa902780	[#969] policer: Restrict the number of remembered errors ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	6010dfdf3d	[#969] policer: Make error skip thread-safe ... Introduces in afd2ba9a66. Refs #914 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Ekaterina Lebedeva	afd2ba9a66	[#110] Add check for repeated error log in policer ... processObject() returns 3 types of errors: container not found errors, could not get container error and placement vector building error. Every error will occur for all objects in container simultaneously, so we can log each error once and safely ignore the rest. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-02-06 00:56:41 +03:00
Anton Nikiforov	6534252c22	[#799] policer: Refactor method processNodes ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Anton Nikiforov	c441296592	[#930] policer: Release task pool when context cancelled ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Dmitrii Stepanov	681b2c5fd4	[#825] policer: Do not drop required linking objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 11:04:03 +00:00
Dmitrii Stepanov	0f75e48138	[#796] policer: Fix tombstone objects replication ... Tombstone objects must be replicated to all container nodes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 13:39:33 +03:00
Dmitrii Stepanov	79088baa06	[#772] node: Apply gofumpt ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:03 +03:00
Evgenii Stratonikov	aca11d7474	[#735] policer: Allow to provide metrics from the outside ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-11 15:14:13 +03:00
Ekaterina Lebedeva	5e229dc248	[#701] metrics: add metric to evaluate policer performance ... Add processed objects counter in policerMetrics, add policer field to NodeMetrics Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-10-09 19:02:08 +00:00
Anton Nikiforov	88d50e4c77	[#656] policer: Add "bad" testcase ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-09-06 08:04:59 +00:00
Airat Arifullin	4ea0df77d0	[#574] policer: Check if the container was really removed ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-08-28 14:21:38 +00:00
Airat Arifullin	554ff2c06b	[#574] core: Extend Source interface with DeletionInfo method ... * Introduce common method EverExisted * Define DeletionInfo for struct that must implement Source * Refactor tree srv Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-08-28 14:21:38 +00:00
Alejandro Lopez	5b7e4a51b7	[#481] Update frostfs-sdk-go and error pointer receivers ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-09 10:26:53 +00:00
Dmitrii Stepanov	f0355a453e	[#463] policer: Remove capacity rebalance logic ... Current implementation has some quirks. For example, using only half of object.put.pool_size_remote threads tells replicator that is node is 50% loaded, but in reality we could be putting lot's of big objects. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-18 10:52:12 +00:00
Alejandro Lopez	3b66f98f27	[#519] Use Address.Equals in policer tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-17 10:45:07 +00:00