343 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	8093e145b3	[#1412] adm: Resolve container type by metabase upgrade ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:51 +03:00
Dmitrii Stepanov	1efa64ee72	[#1412] metabase: Add search by indexed attributes ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:50 +03:00
Dmitrii Stepanov	be744ae3e6	[#1412] metabase: Index attributes for indexed containers ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:49 +03:00
Anton Nikiforov	2f710d8f94	[#1414] metabase: Check parameter for CountAliveObjectsInBucket ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-10-04 17:11:15 +03:00
Dmitrii Stepanov	01e3944b31	[#1408] metabase: Fix tests ... No need to specify container ID for objects created with `testutil.GenerateObjectWithCID`. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-02 14:41:40 +03:00
Dmitrii Stepanov	434048e8d9	[#1408] metabase: Fix EC search with slow and fast filters ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-02 14:36:59 +03:00
Anton Nikiforov	d0ed29b3c7	[#1350] node: Add ability to evacuate objects from REP 1 only ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-27 15:41:17 +03:00
Dmitrii Stepanov	401c398704	[#1388] metabase: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-25 08:55:38 +00:00
Dmitrii Stepanov	fd18aa363b	[#1385] metabase: Optimize isTomb check ... As tombstone and target must have the same containerID, do not iterate other containers. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-24 17:09:06 +03:00
Dmitrii Stepanov	76268e3ea2	[#1385] metabase: Validate that tombstone and target have the same container ID ... Target container ID is taken from tombstone: cmd/frostfs-node/object.go:507 Also object of type `TOMBSTONE` contains objectID, so tombstone and tombstoned object must have the same containerID. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-24 17:09:06 +03:00
Dmitrii Stepanov	8434f3dbfc	[#1385] metabase: Use Batch for delete-related operations ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-24 17:09:06 +03:00
Anton Nikiforov	34e6a309c6	[#1356] engine: Evacuate object from shards concurrently ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-24 12:50:19 +03:00
Aleksey Savchuk	c34b8acedd	[#1312] Drop handling of system attributes with NeoFS prefix ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-20 11:37:18 +00:00
Dmitrii Stepanov	e3764c51df	[#1347] metabase: Fix EC search ... For EC chunks need to return EC parent object ID as EC chunks don't have own attributes but inherit parent's. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-12 13:23:29 +00:00
Dmitrii Stepanov	4668efc0bf	[#1355] metabase: Upgrade improvements ... Do not fail on same latest version to run compact on upgraded metabase. Use NoSync on compact. Log every batch on bucket delete stage. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-09 11:37:36 +03:00
Ekaterina Lebedeva	a685fcdc96	[#1317] go.mod: Use range over int ... Since Go 1.22 a "for" statement with a "range" clause is able to iterate through integer values from zero to an upper limit. gopatch script: @@ var i, e expression @@ -for i := 0; i <= e - 1; i++ { +for i := range e { ... } @@ var i, e expression @@ -for i := 0; i <= e; i++ { +for i := range e + 1 { ... } @@ var i, e expression @@ -for i := 0; i < e; i++ { +for i := range e { ... } Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-09-03 13:00:54 +03:00
Dmitrii Stepanov	882c068410	[#1334] metabase: Store upgrade flag ... This allows to check if metabase upgrade was not completed. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Dmitrii Stepanov	6c2146bbc1	[#1334] metabase: Add upgrade from v2 to v3 ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Dmitrii Stepanov	76f67ea34e	[#1323] metabase: Bump version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	7d0d781db1	[#1323] metabase: Drop user attribute index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	0f08a2efba	[#1323] metabase: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	7bf20c9f1f	[#1323] metabase: Add expiration epoch buckets ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	2542d4f5df	[#1323] metabase: Drop payload checksum index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	15dae8685e	[#1323] metabase: Drop ownerID index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Evgenii Stratonikov	7bca428db0	[#1322] Use new protobuf marshaler ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-22 07:17:41 +00:00
Anton Nikiforov	cfda9003a7	[#1318] meta: Add test TestInhumeECObject ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 16:21:27 +03:00
Anton Nikiforov	6ff0b0996b	[#1318] metrics: Fix container_size_bytes for EC ... When node put chunk into EC container, `policer` may remove it as redundant. This chunk marked as removed. When parent object removed and `gc` start iterating over chunk, node count removing chunk twice. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 15:33:28 +03:00
Dmitrii Stepanov	8e51d7849a	[#1295] getSvc: Assemble complex EC object headers without linking object ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 16:48:12 +03:00
Dmitrii Stepanov	3119f2fbc3	[#1257] metabase: Delete EC gc marks and split info ... EC parent and split gc marks should be deleted after last EC chunk delete. Also should delete split info for EC parent and split parent. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-19 17:14:14 +03:00
Dmitrii Stepanov	27caa8a72f	[#1256] metabase: Put split parent object ID for EC chunks ... It is required to save split parent ID too, not only split ID. Otherwise inhume operation works incorrect: shard with last part may be skipped and parent object will be available. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-17 18:08:07 +03:00
Ekaterina Lebedeva	56eeb630b6	[#1217] Fix grammar mistakes and misspelling ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-07-01 19:14:25 +03:00
Dmitrii Stepanov	76cf7a051b	[#1178] shard: Check metabase existence before read shard ID ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-17 09:59:15 +03:00
Anton Nikiforov	b9fcaad21f	[#1168] shard: Set Disabled as default mode for components ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-11 15:13:38 +00:00
Evgenii Stratonikov	3f1961157e	[#1163] metabase: Handle multiple splitInfos for EC ... For REP updating split info is handled explicitly by a high-level PUT logic. For EC it is trickier, because the address of an object we put is only distantly related to a split info. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-06 16:26:29 +03:00
Evgenii Stratonikov	2e074d3846	[#1163] metabase: Properly save EC parent split ID ... Search by SplitID should return all parts of a complex object. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-05 12:40:16 +03:00
Alexander Chuprov	806236da78	[#1121] node: Change mode of shard components ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Dmitrii Stepanov	cc2449beaf	[#1158] metabase: Fix EC storage schema ... Do not store EC info twice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-04 17:24:40 +03:00
Airat Arifullin	5aacb8fc86	[#1144] metabase: Save parent attributes for ec-chunks ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:55:32 +03:00
Airat Arifullin	f8e33f8e3a	[#1144] metabase: Proprely choose root OID for EC-splitted objects ... * If EC-parent is a part of Split itself, then save to root bucket its parent; * If EC-parent is not a part of Split itself, then save to root bucket OID of this EC-parent. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Airat Arifullin	f0edebea18	[#1144] metabase: Support ec parent filter for Search ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Anton Nikiforov	6130650bb6	[#1147] node: Implement Lock\Delete requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Airat Arifullin	40b04c00ef	[#1141] metabase: Fix IsUserObject method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:15 +03:00
Airat Arifullin	89a80e9a0f	[#1141] metabase: Fix putUniqueIndexItem ... * `GetECHeader` is not correct way to determine if an object's got EC-header: `ECHeader` must be used for that. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:08 +03:00
Dmitrii Stepanov	d45d086acd	[#1129] policer: Add EC chunk replication ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:48 +03:00
Anton Nikiforov	5c582e96fd	[#1136] metabase: Fix creation of ECInfoError ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-15 11:04:27 +00:00
Anton Nikiforov	112a7c690f	[#1103] node: Implement Get\Head requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-24 18:15:53 +03:00
Evgenii Stratonikov	2c4b50a71e	Reapply "[#972] Use require.ElementsMatch() where possible" ... This reverts commit 7627d08914. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Dmitrii Stepanov	76398c06b0	[#1080] metabase: Add StorageID metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:00:08 +03:00
Dmitrii Stepanov	7b1adfed3e	[#1080] metabase: Open bucket for container counter once ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:00:08 +03:00
Ekaterina Lebedeva	d5194ab2a6	[#949] metabase: fix shard.UpdateID() ... metabase.Open() now reports metabase mode metric. shard.UpdateID() needs to read shard ID from metabase => needs to open metabase. It caused reporting 'shard undefined' metrics. To avoid reporting wrong metrics metabase.GetShardID() was added which also opens metabase and does not report metrics. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00