1125 commits

Author	SHA1	Message	Date
Evgenii Stratonikov	01e18eda43	[#1328] pilorama: Do not skip items in SortedByFilename ... Benchmark results: ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ ForestSortedIteration/bbolt,root-8 207.2µ ± 6% 173.6µ ± 6% -16.23% (p=0.000 n=10) ForestSortedIteration/bbolt,leaf-8 3.910µ ± 5% 3.928µ ± 7% ~ (p=0.529 n=10) geomean 28.46µ 26.11µ -8.27% ``` They are not representative, as the worst case is when we have multiple items of different lengths. However, `FileName` is usually less than 100 in practice, so the asymptotics is the same. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-23 13:41:01 +03:00
Evgenii Stratonikov	3e89e744aa	[#1328] pilorama: Add tricky test for SortedByFilename ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-23 13:23:15 +03:00
Anton Nikiforov	5aef303259	[#1318] meta: Add test TestInhumeECObject ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 16:34:42 +03:00
Anton Nikiforov	dd570c9344	[#1318] metrics: Fix container_size_bytes for EC ... When node put chunk into EC container, `policer` may remove it as redundant. This chunk marked as removed. When parent object removed and `gc` start iterating over chunk, node count removing chunk twice. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 16:31:59 +03:00
Evgenii Stratonikov	4c520be9f1	[#1313] blobovnicza: Prevent concurrent Put/Close ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-15 15:49:07 +03:00
Dmitrii Stepanov	078b5fdd6e	[#1302] writecache: Allow to specify custom page size ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 12:39:11 +03:00
Dmitrii Stepanov	b3ab9589a5	[#1302] writecache: Add put->flush->put benchmark ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 12:35:05 +03:00
Dmitrii Stepanov	54862250b2	[#1295] getSvc: Assemble complex EC object headers without linking object ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 17:51:23 +03:00
Dmitrii Stepanov	89d670c2f0	[#1295] engine: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 17:51:06 +03:00
Dmitrii Stepanov	87c5954f4e	[#1295] engine: Log object address in case of error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 17:50:48 +03:00
Dmitrii Stepanov	18182e578e	[#1261] shard: Fix delete objects from FSTree ... Replace nil storageID with empty like by shard.Get. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-22 14:38:09 +03:00
Dmitrii Stepanov	3119f2fbc3	[#1257] metabase: Delete EC gc marks and split info ... EC parent and split gc marks should be deleted after last EC chunk delete. Also should delete split info for EC parent and split parent. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-19 17:14:14 +03:00
Dmitrii Stepanov	27caa8a72f	[#1256] metabase: Put split parent object ID for EC chunks ... It is required to save split parent ID too, not only split ID. Otherwise inhume operation works incorrect: shard with last part may be skipped and parent object will be available. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-17 18:08:07 +03:00
Evgenii Stratonikov	3940bc17c1	[#1251] pilorama: Allow traversing multiple branches in parallel ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-17 11:25:07 +03:00
Anton Nikiforov	3bf6e6dde6	[#1238] engine: Add non-blocking send in the shard's notification channel ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-07-10 11:37:11 +03:00
Evgenii Stratonikov	b027a7f91e	[#1234] pilorama: Fix GetByPath() on duplicate directories ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-10 06:11:38 +00:00
Evgenii Stratonikov	6ace2f597e	[#1234] pilorama: Add test for duplicate directory behaviour ... When AddByPath() is called concurrently on 2 different nodes, internal path components may be created twice. This violates some of our assumptions in GetByPath() and, indirectly, in S3 handling of GetSubTree() results. Add a test for the correct behaviour, fixes will follow. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-10 06:11:38 +00:00
Dmitrii Stepanov	62cbb72a5e	[#1226] blobovniczatree: Delete fix db extensions in Init() ... Since several releases have been released, this code is not relevant. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 12:22:06 +03:00
Dmitrii Stepanov	78b1d9b18d	[#1226] blobovniczatree: Drop leaf width limitation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 12:22:06 +03:00
Dmitrii Stepanov	40c9ddb6ba	[#1226] blobovniczatree: Drop init in advance option ... To make blobovniczatree unlimited. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 12:22:06 +03:00
Dmitrii Stepanov	3a797e4682	[#1222] engine: Fix tree evacuation ... Do not fail if it is unable to evacuate tree to other node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 10:38:10 +03:00
Dmitrii Stepanov	2bac82cd6f	[#1222] engine: Fix object evacuation ... Do not fail evacuation if it unable to evacuate object to other node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 10:38:10 +03:00
Anton Nikiforov	bbe95dac8b	[#1225] engine: Log the error when check object existence ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-07-04 07:22:54 +00:00
Ekaterina Lebedeva	56eeb630b6	[#1217] Fix grammar mistakes and misspelling ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-07-01 19:14:25 +03:00
Dmitrii Stepanov	7a8ac4907a	[#1213] engine: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-01 06:49:35 +00:00
Evgenii Stratonikov	7085723c6b	[#1074] pilorama: Allow empty filenames in SortedByFilename() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-28 17:46:24 +03:00
Dmitrii Stepanov	4c7ff159ec	[#1201] writecache: Cancel background flush without lock ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-28 09:02:36 +03:00
Dmitrii Stepanov	4951babd5f	[#1208] blobstor: Fix delete without storage id ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-27 11:59:08 +03:00
Dmitrii Stepanov	81ea91de52	[#451] metrics: Move to internal ... `metrics` don't look like something others want to import. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-25 08:52:37 +00:00
Anton Nikiforov	9ac74efc41	[#1173] shard: Use mode from config on reload ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-20 11:29:10 +00:00
Evgenii Stratonikov	75eedf71f3	[#1187] pilorama/test: Remove debug print ... Introduced in e12fcc041d. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-18 15:09:44 +03:00
Dmitrii Stepanov	76cf7a051b	[#1178] shard: Check metabase existence before read shard ID ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-17 09:59:15 +03:00
Anton Nikiforov	68ac490729	[#1174] shard: Update metric mode_info on Init ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-13 08:32:59 +00:00
Dmitrii Stepanov	6a39c3d15e	[#1086] engine: Do not use metabase if shard looks bad ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-13 07:35:22 +00:00
Evgenii Stratonikov	9d73f9c2c6	Reapply "[#446] engine: Move to read-only on blobstor errors" ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-13 07:35:22 +00:00
Anton Nikiforov	b9fcaad21f	[#1168] shard: Set Disabled as default mode for components ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-11 15:13:38 +00:00
Dmitrii Stepanov	6cf512e574	[#1166] blobovniczatree: Handle blobovnicza's NoSpaceLeft error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-07 17:15:43 +03:00
Dmitrii Stepanov	e7d479f4c2	[#1166] blobovnicza: Return NoSpaceLeft error instead of syscall.ENOSPC ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-07 17:15:43 +03:00
Evgenii Stratonikov	3f1961157e	[#1163] metabase: Handle multiple splitInfos for EC ... For REP updating split info is handled explicitly by a high-level PUT logic. For EC it is trickier, because the address of an object we put is only distantly related to a split info. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-06 16:26:29 +03:00
Evgenii Stratonikov	2e074d3846	[#1163] metabase: Properly save EC parent split ID ... Search by SplitID should return all parts of a complex object. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-05 12:40:16 +03:00
Alexander Chuprov	806236da78	[#1121] node: Change mode of shard components ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Alexander Chuprov	6f2187a420	[#1121] node: Refactor mods of shard ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Dmitrii Stepanov	cc2449beaf	[#1158] metabase: Fix EC storage schema ... Do not store EC info twice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-04 17:24:40 +03:00
Airat Arifullin	5aacb8fc86	[#1144] metabase: Save parent attributes for ec-chunks ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:55:32 +03:00
Airat Arifullin	f8e33f8e3a	[#1144] metabase: Proprely choose root OID for EC-splitted objects ... * If EC-parent is a part of Split itself, then save to root bucket its parent; * If EC-parent is not a part of Split itself, then save to root bucket OID of this EC-parent. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Airat Arifullin	f0edebea18	[#1144] metabase: Support ec parent filter for Search ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Anton Nikiforov	92e19feb57	[#1147] node: Use public fields for shard.ExistsPrm ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	6130650bb6	[#1147] node: Implement Lock\Delete requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	a82c8cc5b8	[#1147] gc: Execute callback for expired tombstones when they exists ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Airat Arifullin	40b04c00ef	[#1141] metabase: Fix IsUserObject method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:15 +03:00