3388 commits

Author	SHA1	Message	Date
Airat Arifullin	e2cb0640f1	[#1501] util: Move eACL-to-APE converter to pkg/util ... * `ConvertEACLToAPE` is useful method which couldn't be imported out of frostfs-node so far as it has been in `internal` * Since `ConvertEACLToAPE` and related structures and unit-tests are placed in `pkg/util` Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-20 07:58:32 +00:00
Evgenii Stratonikov	d82f0d1926	[#1496] node/control: Await until SetNetmapStatus() persists ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-15 16:36:07 +03:00
Evgenii Stratonikov	acd5babd86	[#1496] morph: Merge InvokeRes and WaitParams ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-15 16:36:07 +03:00
Evgenii Stratonikov	b65874d1c3	[#1496] morph: Return InvokeRes from all invoke*() methods ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-15 16:36:07 +03:00
Evgenii Stratonikov	69c63006da	[#1496] morph: Move tx waiter to morph package ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-15 16:36:07 +03:00
Evgenii Stratonikov	d77a218f7c	[#1493] metabase: Merge Inhume() and DropGraves() for tombstones ... DropGraves() is only used to drop gravemarks after a tombstone removal. Thus, it makes sense to do Inhume() and DropGraves() in one transaction. It has less overhead and no unexpected problems in case of sudden power failure. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-14 06:47:04 +00:00
Evgenii Stratonikov	44df67492f	[#1493] metabase: Split inhumeTx() into 2 functions ... No functional changes. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-14 06:47:04 +00:00
Evgenii Stratonikov	1e6f132b4e	[#1493] metabase: Pass InhumePrm by value ... Unify with the other code, no functional changes. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-14 06:47:04 +00:00
Evgenii Stratonikov	6dc0dc6691	[#1493] shard: Take mode mutex in HandleExpiredTombstones() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-14 06:47:04 +00:00
Aleksey Savchuk	7fc6101bec	[#1491] engine/test: Rework engine test utils ... - Remove `testNewShard` and `setInitializedShards` because they violated the default engine workflow. The correct workflow is: first use `New()`, followed by `Open()`, and then `Init()`. As a result, adding new logic to `(*StorageEngine).Init` caused several tests to fail with a panic when attempting to access uninitialized resources. Now, all engines created with the test utils must be initialized manually. The new helper method `prepare` can be used for that purpose. - Additionally, `setInitializedShards` hardcoded the shard worker pool size, which prevented it from being configured in tests and benchmarks. This has been fixed as well. - Ensure engine initialization is done wherever it was missing. - Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and `setShardsNum`. Make them all depend on `setShardsNumOpts`. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-11-13 14:42:53 +03:00
Aleksey Savchuk	7ef36749d0	[#1491] engine/test: Move BenchmarkExists to exists_test.go ... Move `BenchmarkExists` from `engine_test.go` to `exists_test.go` for better organization and clarity. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-11-13 14:09:29 +03:00
Aleksey Savchuk	c6066d6ee4	[#1491] engine/test: Use more suitable testing utils here and there ... Use `setShardsNum` instead of `setInitializedShards` wherever possible. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-11-13 14:09:29 +03:00
Dmitrii Stepanov	612b34d570	[#1437] logger: Add caller skip to log original caller position ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:12 +03:00
Dmitrii Stepanov	7429553266	[#1437] node: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:10 +03:00
Dmitrii Stepanov	6921a89061	[#1437] ir: Fix contextcheck linters ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:10 +03:00
Dmitrii Stepanov	16598553d9	[#1437] shard: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:09 +03:00
Dmitrii Stepanov	c139892117	[#1437] ir: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:09 +03:00
Dmitrii Stepanov	62b5181618	[#1437] blobovnicza: Fix contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:08 +03:00
Dmitrii Stepanov	6db46257c0	[#1437] node: Use ctx for logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-13 10:36:07 +03:00
Dmitrii Stepanov	c16dae8b4d	[#1437] logger: Use context to log trace id ... Signed-off-by: Dmitrii Stepanov	2024-11-13 10:36:07 +03:00
Evgenii Stratonikov	fd004add00	[#1492] metabase: Fix import formatting ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-13 07:30:25 +00:00
Evgenii Stratonikov	8ed7a676d5	[#1492] metabase: Ensure Unmarshal() is called on a cloned slice ... The slice returned from bucket.Get() is only valid during the tx lifetime. Cloning it is not necessary everywhere, but better safe than sorry. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-13 07:30:25 +00:00
Evgenii Stratonikov	b451de94c8	[#1492] metabase: Fix typo in objData ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-13 07:30:25 +00:00
Dmitrii Stepanov	46fef276b4	[#1449] tree: Log tree sync with Info level ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Dmitrii Stepanov	9bd05e94c8	[#1449] tree: Add ApplyBatch method ... Concurrent Apply can lead to child node applies before parent, so undo/redo operations will perform. This leads to performance degradation in case of tree with many sublevels. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Evgenii Stratonikov	3324c26fd8	[#1483] morph: Remove container.GetEACL() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-12 12:10:51 +00:00
Evgenii Stratonikov	a692298533	[#1483] node: Remove eACL cache ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-12 12:10:51 +00:00
Dmitrii Stepanov	8a57c78f5f	[#1484] engine: Fix engine metrics ... 1. Add forgotten metrics for client requests 2. Include execIfNotBlocked into metrics Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-11 12:59:20 +03:00
Evgenii Stratonikov	c82c753e9f	[#1480] objsvc: Remove useless stream wrappers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Evgenii Stratonikov	f666898e5d	[#1480] objsvc: Remove EACL checks ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Evgenii Stratonikov	b1a31281e4	[#1480] ape: Remove SoftAPECheck flag ... Previous release was EACL-compatible. Starting from now all EACL should've been migrated to APE chains. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-08 12:01:14 +00:00
Airat Arifullin	764450d04a	[#1479] tree: Regenerate service protobufs ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:44 +03:00
Airat Arifullin	755cae3f19	[#1479] control: Regenerate protobufs for service ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:44 +03:00
Airat Arifullin	9b13a18aac	[#1479] go.mod: Bump frostfs-sdk-go version ... * Update version within go.mod; * Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api instead. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:19 +03:00
Dmitrii Stepanov	ef64930fef	[#1477] ape: Fix EC chunk test ... Initially, this test was a check that only the container node can assemble an EC object. But the implementation of this test was wrong. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-07 16:00:20 +03:00
Evgenii Stratonikov	c8fb154151	[#1475] Remove container estimation code ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-06 15:32:33 +03:00
Dmitrii Stepanov	7edec9193c	[#1451] placement: Return copy of slice from container nodes cache ... Nodes from cache could be changed by traverser, if no objectID specified. So it is required to return copy of cache's slice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	3cf6ea745d	[#1451] ec: Check all parts are saved ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	9a77527f46	[#1451] ape: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	5b1ba8e23d	[#1451] ape: Perform strict APE checks for EC parts ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	9902965ff4	[#1451] writer: Sign EC parts with node's private key ... As EC put request may be processed only by container node, so sign requests with current node private to not to perform APE checks. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:18:10 +00:00
Dmitrii Stepanov	33ad753302	[#1473] policer: Add tracing span ... To filter HEAD requests from policer. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-06 08:15:11 +00:00
Evgenii Stratonikov	15102e6dfd	[#1471] Replace sort.Slice in some places ... `slices.SortFunc` doesn't use reflection and is a bit faster. I have done some micro-benchmarks for `[]NodeInfo`: ``` $ benchstat -col "/func" out goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ sort.Slice │ slices.SortFunc │ │ sec/op │ sec/op vs base │ Sort-8 2.130µ ± 2% 1.253µ ± 2% -41.20% (p=0.000 n=10) ``` Haven't included them, though, as they I don't see them being used a lot. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-11-06 08:07:32 +00:00
Dmitrii Stepanov	6c45a17af6	[#1467] node: Break notary deposit wait after VUB ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-01 16:42:02 +03:00
Dmitrii Stepanov	d19ab43500	[#1462] node: Add off-cpu profiler ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-31 11:32:13 +03:00
Evgenii Stratonikov	c2effcc61c	[#1465] Makefile: Update golangci-lint, fix warnings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-31 06:39:59 +00:00
Evgenii Stratonikov	7ac0852364	[#1459] .golangci.yml: Add intrange linter, fix issues ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-30 15:18:22 +00:00
Aleksey Savchuk	d28a5d2d7a	[#1448] container/ape: Ignore an error when getting a role ... When getting a role in the APE checker for the container services, an error may be returned if network maps of the previous two epochs don't have enough nodes to fulfil a container placement policy. It's a logical error, so we should ignore it. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-10-30 12:51:57 +03:00
Airat Arifullin	87ac3c5279	[#1458] object: Make patch not set key before target construction ... * `SignRequestPrivateKey` field should be initialized either within `newUntrustedTarget` or within `newTrustedTarget`. Otherwise, all requests are signed by local node key that makes impossible to perform patch on non-container node. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-10-29 15:20:28 +00:00
Dmitrii Stepanov	d5ee6d3039	[#1456] morph: Use DialerSource interface instead of internal struct ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-29 17:34:14 +03:00