3445 commits

Author	SHA1	Message	Date
Aleksey Savchuk	e2a6663ef6	[#1445] shard/gc: Make handler and callback names more descriptive ... Consider the following methods: - `(*Shard).collectExpiredTombstones` - `(*StorageEngines).processExpiredTombstones` - `(*Shard).HandleExpiredTombstones` All of them handle not tombstones but graves. `HandleExpiredTombstones` in fact deletes tombstones but it does it based on graves it received. So, rename all `...Tombstones` methods to `...Graves` method. It'll make future changes in the garbage collector behavior simpler. Also, rename all `...Locks` methods to `...LockObjects` because they handle not locks but lock objects. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-17 13:40:37 +03:00
Aleksey Savchuk	8ad8e2783a	[#1445] metabase: Add IterateLocked method ... Added a new method for iterating locked objects. It's indended to be used in the garbage collector for handling expired locks. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-17 10:40:30 +03:00
Aleksey Savchuk	a0acef639e	[#1445] metabase/test: Add test for tombstones of the new format ... Test iterating over graveyard populated with tombstones of both old and new formats. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:30 +03:00
Aleksey Savchuk	4d8550945c	[#1445] metabase/test: Add tests for locks of the new format ... Test decoding and encoding functions for locks and lists of locks of the new format. Also, test that when the object is being locked with the same lock, its expiration epoch is set if it wasn't set before. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:30 +03:00
Aleksey Savchuk	428cfed8af	[#1445] metabase/test: Add test for GetLocks method ... Since it'll be working with locks of the new format, i. e. locks with expiration epoch, it's better to have a test for this method to ensure it can handle locks of both old and new formats. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:29 +03:00
Aleksey Savchuk	d70e1d29da	[#1445] local_object_storage: Append expiration epoch to locks ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:29 +03:00
Aleksey Savchuk	17d2dc5341	[#1445] local_object_storage: Append expiration epoch to tombstones ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:28 +03:00
Aleksey Savchuk	c9c51bdb23	[#1445] core/object: Add GetExpirationEpoch function ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:28 +03:00
Aleksey Savchuk	d382357ebf	[#1445] shard/test: Add missing SysAttributeExpEpoch attribute ... Tombstone and lock objects didn't have `SysAttributeExpEpoch` attribute. This attribute is required for objects of these types. Tests didn't fail because of that, but it might be a problem in the future. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 16:31:23 +03:00
Aleksey Savchuk	f0c43c8d80	[#1502] Use zap.Error for logging errors ... Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like `zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`. Leave similar expressions with other messages unchanged, for example, `zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`. This change was made by applying the following patch: ```diff @@ var err expression @@ -zap.String("error", err.Error()) +zap.Error(err) @@ var err expression @@ -zap.String("err", err.Error()) +zap.Error(err) ``` Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-16 11:13:42 +03:00
Aleksey Savchuk	8ba9f31fca	[#1510] metabase/test: Fix BenchmarkListWithCursor ... - Fix misplaced `(*DB).Close` (broken after 47dcfa20f3) - Use `errors.Is` for error checking (broken after fcdbf5e509) Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-13 13:19:15 +03:00
Aleksey Savchuk	2af3409d39	[#1510] metabase/test: Fix BenchmarkGet ... Fix misplaced `(*DB).Close` (broken after 47dcfa20f3) Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-13 13:18:43 +03:00
Evgenii Stratonikov	d165ac042c	[#1558] morph/client: Reuse notary rpcclient wrapper ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-12 15:30:12 +03:00
Evgenii Stratonikov	7151c71d51	[#1558] morph/client: Remove "could not"/"can't"/"failed to" from error messages ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-12 15:30:12 +03:00
Evgenii Stratonikov	91d9dc2676	[#1558] morph/event: Remove "could not" from error messages ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-12 15:30:12 +03:00
Evgenii Stratonikov	7853dbc315	[#1557] morph/event: Remove embedded structs from scriptHashWithValue ... Also, make them public, because otherwise `unused` linter complains. ``` pkg/morph/event/utils.go:25:2 unused field `typ` is unused ``` This complain is wrong, though: we _use_ `typ` field because the whole struct is used as a map key. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-12 11:55:09 +00:00
Aleksey Savchuk	3821645085	[#1555] engine: Refactor (*StorageEngine).GetLocks ... Refactored after renaming the method to replace the confusing `locked` variable with `locks`. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-11 15:06:38 +03:00
Aleksey Savchuk	72470d6b48	[#1555] local_object_storage: Rename method GetLocked -> GetLocks ... Renamed to better reflect the method's purpose of returning locks for the specified object. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-11 15:06:37 +03:00
Evgenii Stratonikov	e9837bbcf9	[#1554] morph/event: Remove unused AlphabetUpdate event ... Refs TrueCloudLab/frostfs-contract#138. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-11 12:01:17 +00:00
Evgenii Stratonikov	b1614a284d	[#1546] morph/event: Export NotificationHandlerInfo fields ... Hiding them achieves nothing, as the struct has no methods and is not used concurrently. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-11 07:39:49 +00:00
Evgenii Stratonikov	d0ce835fbf	[#1546] morph/event: Merge notification parser and handlers ... They are decoupled, but it is an error to have a handler without a corresponding parser. Register them together on the code level and get rid of unreachable code. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-11 07:39:49 +00:00
Evgenii Stratonikov	dfa51048a8	[#1546] morph/event: Remove "is started" checks from event handler registrar ... This codepath hides possible bugs in code. All initialization function should run before init stage. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-11 07:39:49 +00:00
Evgenii Stratonikov	670305a721	[#1546] morph/event: Remove nil checks from event handler registrar ... This codepath hides possible bugs in code. We would rather panic then silently fail. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-11 07:39:49 +00:00
Dmitrii Stepanov	1f6cf57e30	[#1548] metabase: Check if EC parent is removed or expired ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-11 07:26:33 +00:00
Dmitrii Stepanov	386a12eea4	[#1548] engine: Rename parent -> ecParent ... Parent could mean split parent or EC parent. In this case it is EC parent only. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-11 07:26:33 +00:00
Dmitrii Stepanov	15139d80c9	[#1548] policer: Do not replicate EC chunk if object already removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-11 07:26:33 +00:00
Dmitrii Stepanov	41da27dad5	[#1549] engine: Drop Async flag from evacuation parameters ... Now it is only async evacuation. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-10 17:00:00 +03:00
Dmitrii Stepanov	ac0511d214	[#1549] controlSvc: Drop deprecated EvacuateShard rpc ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-10 16:59:52 +03:00
Airat Arifullin	7e542906ef	[#1539] go.mod: Bump frostfs-sdk-go version ... * Also fix placement unit-test in object manager Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-12-06 15:29:37 +03:00
Evgenii Stratonikov	d1bc4351c3	[#1545] morph/event: Simplify frostfs contract event parsing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-06 14:00:23 +03:00
Evgenii Stratonikov	1c12f23b84	[#1541] morph/event: Simplify netmap contract event parsing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-06 10:26:39 +00:00
Evgenii Stratonikov	a353d45742	[#1541] morph/event: Simplify container contract event parsing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-06 10:26:39 +00:00
Evgenii Stratonikov	d5d5ce2074	[#1541] morph/event: Simplify balance contract event parsing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-06 10:26:39 +00:00
Dmitrii Stepanov	7df3520d48	[#1540] getSvc: Drop redundant returns ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-05 12:39:49 +00:00
Dmitrii Stepanov	5fe78e51d1	[#1540] getSvc: Do not log context canceled errors during EC assemble ... Those errors are fired when it is enough chunks retrieved and error group cancels other requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-12-05 12:39:49 +00:00
Evgenii Stratonikov	84b4051b4d	[#1538] morph/container: Make opts struct similar to that of other contracts ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-04 15:30:58 +03:00
Evgenii Stratonikov	6a51086030	[#1538] morph/client: Remove TryNotary() option from side-chain contracts ... The notary is always enabled and this option does always work. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-04 15:30:58 +03:00
Evgenii Stratonikov	2d5d4093be	[#1537] morph: Use (user.ID).ScriptHash() where possible ... Pick up changes from TrueCloudLab/frostfs-sdk-go#198. gopatch: ``` @@ var user expression @@ -address.StringToUint160(user.EncodeToString()) +user.ScriptHash() ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-04 13:25:44 +03:00
Evgenii Stratonikov	e3487d5af5	[#1535] morph: Unify test invoke error messages ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-04 09:50:20 +00:00
Evgenii Stratonikov	e37dcdf88b	[#1535] morph/netmap: Unify error messages for config retrieval ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-04 09:50:20 +00:00
Evgenii Stratonikov	6c679d1535	[#1535] morph: Unify client creation error messages ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-12-04 09:50:20 +00:00
Aleksey Savchuk	281d65435e	[#1450] engine: Group object by shard before Inhume ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/engine cpu: 12th Gen Intel(R) Core(TM) i5-1235U │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ InhumeMultipart/objects=1-12 11.42m ± 1% 10.71m ± 0% -6.27% (p=0.000 n=10) InhumeMultipart/objects=10-12 113.5m ± 0% 100.9m ± 3% -11.08% (p=0.000 n=10) InhumeMultipart/objects=100-12 1135.4m ± 1% 681.3m ± 2% -40.00% (p=0.000 n=10) InhumeMultipart/objects=1000-12 11.358 ± 0% 1.089 ± 1% -90.41% (p=0.000 n=10) InhumeMultipart/objects=10000-12 113.251 ± 0% 1.645 ± 1% -98.55% (p=0.000 n=10) geomean 1.136 265.5m -76.63% ``` Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-04 10:09:00 +03:00
Aleksey Savchuk	b348b20289	[#1450] engine: Add benchmark for Inhume operation ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-04 10:08:34 +03:00
Aleksey Savchuk	748edd1999	[#1450] engine: Return shard-level error if object is expired on inhume ... Since we have errors defined on the shard-level, it looks strage that we check an error againt the shard-level error `ErrLockObjectRemoval`, but then return the metabase-level error. Let's return the same shard-level error instead. Since we have errors defined on the shard-level Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-04 10:06:57 +03:00
Aleksey Savchuk	432042c534	[#1527] engine: Add tests for handling expired objects on inhume and lock ... Currently, it's allowed to inhume or lock an expired object. Consider the following scenario: 1) An user inhumes or locks an object 2) The object expires 3) GC hasn't yet deleted the object 4) The node loses the associated tombstone or lock 5) Another node replicates tombstone or lock to the first node In this case, the second node succeeds, which is the desired behavior. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-03 12:29:45 +03:00
Aleksey Savchuk	9cabca9dfe	[#1527] engine/test: Move default metabase options to separate function ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-02 16:13:37 +03:00
Aleksey Savchuk	60feed3b5f	[#1527] engine/test: Allow to specify current epoch in epochState ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-12-02 15:37:25 +03:00
Anton Nikiforov	edfa3f4825	[#1528] node: Keep order for equal elements when sort priority metrics ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-12-02 13:58:19 +03:00
Aleksey Savchuk	e0ac3a583f	[#1523] metabase: Remove (*DB).IterateCoveredByTombstones ... Remove this method because it isn't used anywhere since 7799f8e4c. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-11-29 10:49:24 +00:00
Airat Arifullin	00c608c05e	[#1524] tree: Make check APE error get wrapped to api status ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-29 10:48:16 +00:00