1276 commits

Author	SHA1	Message	Date
Ekaterina Lebedeva	df05057ed4	[#1452] container: Add ListStream method ... * Added new method for listing containers to container service. It opens stream and sends containers in batches. * Added TransportSplitter wrapper around ExecutionService to split container ID list read from contract in parts that are smaller than grpc max message size. Batch size can be changed in node configuration file (as in example config file). * Changed `container list` implementaion in cli: now ListStream is called by default. Old List is called only if ListStream is not implemented. * Changed `internalclient.ListContainersPrm`.`Account` to `OwnerID` since `client.PrmContainerList`.`Account` was renamed to `OwnerID` in sdk. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-12-17 16:22:43 +03:00
Airat Arifullin	6e82661c35	[#1563] tree: Wrap only ChainRouterError erros with ObjectAccessDenied ... * Such wrapping helps to differentiate logical check errors and server internal errors. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-16 15:16:07 +03:00
Airat Arifullin	1a091ea7bb	[#1563] object: Wrap only ChainRouterError erros with ObjectAccessDenied ... * Such wrapping helps to differentiate logical check errors and server internal errors. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-16 15:15:25 +03:00
Airat Arifullin	7ac3542714	[#1563] ape: Introduce ChainRouterError error type ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-16 15:12:30 +03:00
Aleksey Savchuk	f0c43c8d80	[#1502] Use zap.Error for logging errors ... Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like `zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`. Leave similar expressions with other messages unchanged, for example, `zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`. This change was made by applying the following patch: ```diff @@ var err expression @@ -zap.String("error", err.Error()) +zap.Error(err) @@ var err expression @@ -zap.String("err", err.Error()) +zap.Error(err) ``` Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 11:13:42 +03:00
Dmitrii Stepanov	15139d80c9	[#1548] policer: Do not replicate EC chunk if object already removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-11 07:26:33 +00:00
Dmitrii Stepanov	41da27dad5	[#1549] engine: Drop Async flag from evacuation parameters ... Now it is only async evacuation. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-10 17:00:00 +03:00
Dmitrii Stepanov	ac0511d214	[#1549] controlSvc: Drop deprecated EvacuateShard rpc ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-10 16:59:52 +03:00
Airat Arifullin	7e542906ef	[#1539] go.mod: Bump frostfs-sdk-go version ... * Also fix placement unit-test in object manager Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-06 15:29:37 +03:00
Dmitrii Stepanov	7df3520d48	[#1540] getSvc: Drop redundant returns ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-05 12:39:49 +00:00
Dmitrii Stepanov	5fe78e51d1	[#1540] getSvc: Do not log context canceled errors during EC assemble ... Those errors are fired when it is enough chunks retrieved and error group cancels other requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-05 12:39:49 +00:00
Anton Nikiforov	edfa3f4825	[#1528] node: Keep order for equal elements when sort priority metrics ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-12-02 13:58:19 +03:00
Airat Arifullin	00c608c05e	[#1524] tree: Make check APE error get wrapped to api status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-29 10:48:16 +00:00
Airat Arifullin	bba1892fa1	[#1524] ape: Make APE checker return error without status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-29 10:48:16 +00:00
Anton Nikiforov	f12f04199e	[#1516] traverser: Check for placement vector out of range ... Placement vector may contain fewer nodes count than it required by policy due to the outage of the one of the node. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-11-21 14:18:55 +03:00
Dmitrii Stepanov	7429553266	[#1437] node: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:10 +03:00
Dmitrii Stepanov	16598553d9	[#1437] shard: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:09 +03:00
Dmitrii Stepanov	6db46257c0	[#1437] node: Use ctx for logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:07 +03:00
Dmitrii Stepanov	46fef276b4	[#1449] tree: Log tree sync with Info level ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Dmitrii Stepanov	9bd05e94c8	[#1449] tree: Add ApplyBatch method ... Concurrent Apply can lead to child node applies before parent, so undo/redo operations will perform. This leads to performance degradation in case of tree with many sublevels. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Evgenii Stratonikov	a692298533	[#1483] node: Remove eACL cache ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-12 12:10:51 +00:00
Evgenii Stratonikov	c82c753e9f	[#1480] objsvc: Remove useless stream wrappers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Evgenii Stratonikov	f666898e5d	[#1480] objsvc: Remove EACL checks ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Evgenii Stratonikov	b1a31281e4	[#1480] ape: Remove SoftAPECheck flag ... Previous release was EACL-compatible. Starting from now all EACL should've been migrated to APE chains. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Airat Arifullin	764450d04a	[#1479] tree: Regenerate service protobufs ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:44 +03:00
Airat Arifullin	755cae3f19	[#1479] control: Regenerate protobufs for service ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:44 +03:00
Airat Arifullin	9b13a18aac	[#1479] go.mod: Bump frostfs-sdk-go version ... * Update version within go.mod; * Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api instead. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:19 +03:00
Dmitrii Stepanov	ef64930fef	[#1477] ape: Fix EC chunk test ... Initially, this test was a check that only the container node can assemble an EC object. But the implementation of this test was wrong. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-07 16:00:20 +03:00
Dmitrii Stepanov	7edec9193c	[#1451] placement: Return copy of slice from container nodes cache ... Nodes from cache could be changed by traverser, if no objectID specified. So it is required to return copy of cache's slice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	3cf6ea745d	[#1451] ec: Check all parts are saved ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	9a77527f46	[#1451] ape: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	5b1ba8e23d	[#1451] ape: Perform strict APE checks for EC parts ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	9902965ff4	[#1451] writer: Sign EC parts with node's private key ... As EC put request may be processed only by container node, so sign requests with current node private to not to perform APE checks. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	33ad753302	[#1473] policer: Add tracing span ... To filter HEAD requests from policer. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:15:11 +00:00
Evgenii Stratonikov	15102e6dfd	[#1471] Replace sort.Slice in some places ... `slices.SortFunc` doesn't use reflection and is a bit faster. I have done some micro-benchmarks for `[]NodeInfo`: ``` $ benchstat -col "/func" out goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ sort.Slice │ slices.SortFunc │ │ sec/op │ sec/op vs base │ Sort-8 2.130µ ± 2% 1.253µ ± 2% -41.20% (p=0.000 n=10) ``` Haven't included them, though, as they I don't see them being used a lot. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-06 08:07:32 +00:00
Evgenii Stratonikov	c2effcc61c	[#1465] Makefile: Update golangci-lint, fix warnings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-31 06:39:59 +00:00
Evgenii Stratonikov	7ac0852364	[#1459] .golangci.yml: Add intrange linter, fix issues ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-30 15:18:22 +00:00
Aleksey Savchuk	d28a5d2d7a	[#1448] container/ape: Ignore an error when getting a role ... When getting a role in the APE checker for the container services, an error may be returned if network maps of the previous two epochs don't have enough nodes to fulfil a container placement policy. It's a logical error, so we should ignore it. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-10-30 12:51:57 +03:00
Airat Arifullin	87ac3c5279	[#1458] object: Make patch not set key before target construction ... * `SignRequestPrivateKey` field should be initialized either within `newUntrustedTarget` or within `newTrustedTarget`. Otherwise, all requests are signed by local node key that makes impossible to perform patch on non-container node. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-10-29 15:20:28 +00:00
Anton Nikiforov	81f4cdbb91	[#1439] object: Sort nodes by priority metrics to compute GET request ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-10-29 08:05:09 +00:00
Anton Nikiforov	3cd7d23f10	[#1439] node: Reduce usage of netmapAPI.NodeInfo ... Remove outdated code from `netmap` service. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-10-29 08:05:09 +00:00
Airat Arifullin	012af5cc38	[#1406] tree: Add unit-tests for ape check ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-10-29 08:04:23 +00:00
Airat Arifullin	eb5336d5ff	[#1406] tree: Use delete verb instead put for Remove ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-10-29 08:04:23 +00:00
Evgenii Stratonikov	bc8d79ddf9	[#1447] services/tree: Move relaying code to a separate function ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-24 10:01:03 +03:00
Aleksey Savchuk	65a4320c75	[#1441] services/tree: Use grpc.WaitForReady option when creating client ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-10-23 11:45:44 +00:00
Dmitrii Stepanov	e314f328c4	[#1422] tree: Use dialer source for tree service connections ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-17 13:15:08 +00:00
Evgenii Stratonikov	b42bcdc6fa	[#1433] services/object: Put object before auxiliary info ... Consider the following operations ordering: 1. Inhume(with tombstone A) --> add tombstone mark for an object 2. --> new epoch arives 3. --> GCMark is added for a tombstone A, because it is unavailable 4. Put(A) --> return error, because the object already has a GCMark It is possible, and I have successfully reproduced it with a test on the shard level. However, the error is related to the specific _ordering_ of operations with engine. And triggering race-conditions like this is only possible on a shard level currently, so no tests are written. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-17 14:43:13 +03:00
Alexander Chuprov	d83879d4b8	[#1431] node: Fix comment format ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-10-15 08:53:59 +00:00
Alexander Chuprov	f6582081a4	[#1431] obj_storage/metabase: Delete unused variable ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-10-15 08:53:59 +00:00
Vitaliy Potyarkin	714ff784fa	[#1431] objsvc: Use specific values in message about address mismatch ... This makes troubleshooting failed operations much easier Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>	2024-10-15 07:34:03 +00:00