1302 commits

Author	SHA1	Message	Date
Ekaterina Lebedeva	9ee3dd4e91	[#1618] ape: Fix object service request tests ... Constant string `testOwnerID` for these tests has an invalid format. It has 11 bytes instead of required 25 for `user.ID`. It worked because: 1. `user.ID` was a byte slice and didn't check length and format of byte slices decoded from strings. 2. in these tests `testOwnerID` was used only to decode container owner id and to compare it with owner id encoded back to string. Since `user.ID implementation has changed`, the problem arised. Now `testOwnerID` is valid. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-01-30 13:18:37 +00:00
Evgenii Stratonikov	57dc0a8e9e	[#1616] getsvc: Move break condition from body to the loop condition ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-30 06:50:37 +00:00
Evgenii Stratonikov	b309b34bfc	[#1616] getsvc: Simplify buildChainRange() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-30 06:50:37 +00:00
Evgenii Stratonikov	c8acdf40bb	[#1616] getsvc: Use slices.Reverse() where possible ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-30 06:50:37 +00:00
Evgenii Stratonikov	6410542d19	[#1616] getsvc: Move range assembling to a separate file ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-30 06:50:37 +00:00
Evgenii Stratonikov	c0a341a7f6	[#1616] getsvc: Move head assembling to a separate file ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-30 06:50:37 +00:00
Dmitrii Stepanov	e1a984e9d8	[#1620] logs: Drop redundant trace_id fields ... `trace_id` is taken from context. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-01-29 16:13:51 +03:00
Evgenii Stratonikov	951a7ee1c7	[#1605] policer: Do not mutate slice under iteration ... Nothing wrong with it, besides being difficult to read. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-21 05:34:54 +00:00
Evgenii Stratonikov	0bcbeb26b2	[#1605] policer: Simplify processRepNodes() checks ... Current flow is hard to reason about, #1601 is a notorious example of accidental complexity. 1. Remove multiple nested ifs, use depth=1. 2. Process each status exactly once, hopefully preventing bugs like #1601. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-21 05:34:54 +00:00
Evgenii Stratonikov	c98357606b	[#1606] Use slices.Clone()/bytes.Clone() where possible ... gopatch: ``` @@ var from, to expression @@ +import "bytes" -to := make([]byte, len(from)) -copy(to, from) +to := bytes.Clone(from) @@ var from, to expression @@ +import "bytes" -to = make([]byte, len(from)) -copy(to, from) +to = bytes.Clone(from) @@ var from, to, typ expression @@ +import "slices" -to := make([]typ, len(from)) -copy(to, from) +to := slices.Clone(from) @@ var from, to, typ expression @@ +import "slices" -to = make([]typ, len(from)) -copy(to, from) +to = slices.Clone(from) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-17 14:50:14 +03:00
Evgenii Stratonikov	57efa0bc8e	[#1604] policer: Properly handle maintenance nodes ... Consider `REP 1 REP 1` placement (selects/filters are omitted). The placement is `[1, 2], [1, 0]`. We are the 0-th node. Node 1 is under maintenance, so we do not replicate object on the node 2. In the second replication group node 1 is under maintenance, but current caching logic considers it as "replica holder" and removes local copy. Voilà, we have DL if the object is missing from the node 1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:52 +03:00
Evgenii Stratonikov	26e0c82fb8	[#1604] policer/test: Add test for MAINTENANCE runtime status ... The node can have MAINTENANCE status in the network map, but can also be ONLINE while responding with MAINTENANCE. These are 2 different code paths, let's test them separately. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Evgenii Stratonikov	4538ccb12a	[#1604] policer: Do not process the same node twice ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Evgenii Stratonikov	84e1599997	[#1604] policer: Remove one-line helpers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Evgenii Stratonikov	5a270e2e61	[#1604] policer: Use status instead of bool value in node cache ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-16 16:37:16 +03:00
Dmitrii Stepanov	eff95bd632	[#1598] engine: Drop unnecessary result structs ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-01-14 11:15:21 +03:00
Dmitrii Stepanov	fb928616cc	[#1598] golangci: Enable unparam linter ... To drop unnecessary parameters and return values. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-01-14 09:06:47 +03:00
Dmitrii Stepanov	4d5ae59a52	[#1598] golangci: Enable unconvert linters ... To drop unnecessary conversions. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-01-14 09:06:40 +03:00
Airat Arifullin	a9f27e074b	[#1243] object: Look for X-Headers within origin before APE check ... * X-Headers can be found in `origin` field of `MetaHeader` if the request has been forwarded from non-container node. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-01-13 12:07:27 +00:00
Evgenii Stratonikov	f7e75b13b0	[#1506] ape_manager: Await tx persist before returning response ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-01-09 12:04:21 +00:00
Evgenii Stratonikov	d7fcc5ce30	[#1586] objsvc: Allow to send search response in multiple messages ... Previously, `ln` was only set once, so search has really worked for small number of objects. Fix panic: ``` panic: runtime error: slice bounds out of range [:43690] with capacity 21238 goroutine 6859775 [running]: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object.(*searchStreamMsgSizeCtrl).Send(0xc001eec8d0, 0xc005734000) git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/transport_splitter.go:173 +0x1f0 git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2.(*streamWriter).WriteIDs(0xc000520320, {0xc00eb1a000, 0x4fd9c, 0x7fd6475a9a68?}) git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2/streamer.go:28 +0x155 git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*uniqueIDWriter).WriteIDs(0xc001386420, {0xc00eb1a000?, 0xc0013ea9c0?, 0x113eef3?}) git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/util.go:62 +0x202 git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).writeIDList(0xc00011aa38?, {0xc00eb1a000?, 0xc001eec9f0?, 0xc0008f4380?}) git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/exec.go:68 +0x91 git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).executeLocal(0xc0008f4380, {0x176c538, 0xc001eec9f0}) git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/local.go:18 +0x16b ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-28 12:29:22 +00:00
Ekaterina Lebedeva	242f0095d0	[#1577] container: Reduce iterations through container list ... * Separated iteration through container ids from `ContainersOf()` so that it could be reused. * When listing containers we used to iterate through the the whole list of containers twice: first when reading from a contract, then when sending them. Now we can send batches of containers when reading from the contract. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-12-27 15:30:26 +03:00
Airat Arifullin	e44782473a	[#1512] object: Fix writePart for EC-container ... * Immediatly return after `ObjectAlreadyRemoved` error. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-26 11:27:55 +00:00
Airat Arifullin	9cd1bcef06	[#1512] object: Make raw PutSingle check status within response ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-26 11:27:55 +00:00
Evgenii Stratonikov	ca0a33ea0f	[#465] objsvc: Set NETMAP_EPOCH xheader for auxiliary requests ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-26 09:17:58 +00:00
Evgenii Stratonikov	f6c5222952	[#1581] services/session: Use user.ID.EncodeToString() where possible ... gopatch: ``` @@ var id expression @@ -base58.Encode(id.WalletBytes()) +id.EncodeToString() ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-25 18:09:36 +00:00
Ekaterina Lebedeva	df05057ed4	[#1452] container: Add ListStream method ... * Added new method for listing containers to container service. It opens stream and sends containers in batches. * Added TransportSplitter wrapper around ExecutionService to split container ID list read from contract in parts that are smaller than grpc max message size. Batch size can be changed in node configuration file (as in example config file). * Changed `container list` implementaion in cli: now ListStream is called by default. Old List is called only if ListStream is not implemented. * Changed `internalclient.ListContainersPrm`.`Account` to `OwnerID` since `client.PrmContainerList`.`Account` was renamed to `OwnerID` in sdk. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-12-17 16:22:43 +03:00
Airat Arifullin	6e82661c35	[#1563] tree: Wrap only ChainRouterError erros with ObjectAccessDenied ... * Such wrapping helps to differentiate logical check errors and server internal errors. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-16 15:16:07 +03:00
Airat Arifullin	1a091ea7bb	[#1563] object: Wrap only ChainRouterError erros with ObjectAccessDenied ... * Such wrapping helps to differentiate logical check errors and server internal errors. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-16 15:15:25 +03:00
Airat Arifullin	7ac3542714	[#1563] ape: Introduce ChainRouterError error type ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-16 15:12:30 +03:00
Aleksey Savchuk	f0c43c8d80	[#1502] Use zap.Error for logging errors ... Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like `zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`. Leave similar expressions with other messages unchanged, for example, `zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`. This change was made by applying the following patch: ```diff @@ var err expression @@ -zap.String("error", err.Error()) +zap.Error(err) @@ var err expression @@ -zap.String("err", err.Error()) +zap.Error(err) ``` Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 11:13:42 +03:00
Dmitrii Stepanov	15139d80c9	[#1548] policer: Do not replicate EC chunk if object already removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-11 07:26:33 +00:00
Dmitrii Stepanov	41da27dad5	[#1549] engine: Drop Async flag from evacuation parameters ... Now it is only async evacuation. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-10 17:00:00 +03:00
Dmitrii Stepanov	ac0511d214	[#1549] controlSvc: Drop deprecated EvacuateShard rpc ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-10 16:59:52 +03:00
Airat Arifullin	7e542906ef	[#1539] go.mod: Bump frostfs-sdk-go version ... * Also fix placement unit-test in object manager Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-06 15:29:37 +03:00
Dmitrii Stepanov	7df3520d48	[#1540] getSvc: Drop redundant returns ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-05 12:39:49 +00:00
Dmitrii Stepanov	5fe78e51d1	[#1540] getSvc: Do not log context canceled errors during EC assemble ... Those errors are fired when it is enough chunks retrieved and error group cancels other requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-05 12:39:49 +00:00
Anton Nikiforov	edfa3f4825	[#1528] node: Keep order for equal elements when sort priority metrics ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-12-02 13:58:19 +03:00
Airat Arifullin	00c608c05e	[#1524] tree: Make check APE error get wrapped to api status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-29 10:48:16 +00:00
Airat Arifullin	bba1892fa1	[#1524] ape: Make APE checker return error without status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-29 10:48:16 +00:00
Anton Nikiforov	f12f04199e	[#1516] traverser: Check for placement vector out of range ... Placement vector may contain fewer nodes count than it required by policy due to the outage of the one of the node. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-11-21 14:18:55 +03:00
Dmitrii Stepanov	7429553266	[#1437] node: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:10 +03:00
Dmitrii Stepanov	16598553d9	[#1437] shard: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:09 +03:00
Dmitrii Stepanov	6db46257c0	[#1437] node: Use ctx for logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:07 +03:00
Dmitrii Stepanov	46fef276b4	[#1449] tree: Log tree sync with Info level ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Dmitrii Stepanov	9bd05e94c8	[#1449] tree: Add ApplyBatch method ... Concurrent Apply can lead to child node applies before parent, so undo/redo operations will perform. This leads to performance degradation in case of tree with many sublevels. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Evgenii Stratonikov	a692298533	[#1483] node: Remove eACL cache ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-12 12:10:51 +00:00
Evgenii Stratonikov	c82c753e9f	[#1480] objsvc: Remove useless stream wrappers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Evgenii Stratonikov	f666898e5d	[#1480] objsvc: Remove EACL checks ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Evgenii Stratonikov	b1a31281e4	[#1480] ape: Remove SoftAPECheck flag ... Previous release was EACL-compatible. Starting from now all EACL should've been migrated to APE chains. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00