4207 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Dmitrii Stepanov	02450a9a16	[#661] blobovniczatree: Do not create DB's on init ... Blobovniczas will be created on write requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	9de91542fe	[#661] blobovnicza: Compute size with record size ... To get more accurate size of blobovnicza use record size (lenght of key + lenght of data). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	e0cb29cb6e	[#698] blobovnicza: Store counter values ... Blobovnicza initialization take a long time because of bucket Stat() call. So now blobovnicza stores counters in META bucket. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	736ec847b7	[#661] metabase: Update storage ID in case of logical errors ... If object was GC marked or deleted or expired, it is still required to update storageID to physically delete object. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	4b20f846af	[#661] metrcis: Add rebuild percent metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	04b530197e	[#661] blobovniczatree: Pass object size limit from config ... If actual small object size value lower than default object size limit, then unnecessary buckets created. If actual small object size value greated than default object size limit, then error happens. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	f8606db7f2	[#661] blobovniczatree: Do not sort DB's and indicies ... Put stores object to next active DB, so there is no need to sort DBs. In addition, it adds unnecessary DB openings. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	52c625df0f	[#661] blobovniczatree: Make Rebuild concurrent for objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:05 +03:00
Dmitrii Stepanov	888f966eb4	[#661] blobovniczatree: Make Rebuild concurrent ... Different DBs can be rebuild concurrently. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	f325b9557c	[#661] metrics: Add blobovniczatree rebuild metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	4f7f67d136	[#661] blobovniczatree: Make Rebuild failover safe ... Now move info stores in blobovnicza, so in case of failover rebuild completes previous operation first. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	8afb42aae8	[#698] blobovniczatree: Init blobovniczas concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	71fac2b9c3	[#661] shard: Fix Delete method ... Due to the flushing data from the writecache to the storage and simultaneous deletion, a partial deletion situation is possible. So as a solution, deletion is allowed only when the object is in storage, because object will be deleted from writecache by flush goroutine. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	9ef4a885de	[#661] blobovniczatree: Add Rebuild implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	b22b703325	[#661] blobstor: Add Rebuild implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	784efb6155	[#661] blobovniczatree: Allow to change depth or width ... Now it is possible to change depth or with of blobovniczatree. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	df11b40dfd	[#661] blobovniczatree: Use .db extension for db files ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Dmitrii Stepanov	4e05ce1c3a	[#661] shard: Add blobstor rebuilder ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:36:04 +03:00
Evgenii Stratonikov	1f07e8b375	Release v0.37.0 ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-07 13:36:34 +03:00
Evgenii Stratonikov	2be938f3cd	[#823] go.mod: Update frostfs-api-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-21 14:46:50 +03:00
Dmitrii Stepanov	eac61fe9b2	[#733] frostfs-cli: Add control ir remove-container ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-03 15:42:58 +03:00
Dmitrii Stepanov	dc81b4b50c	[#725] writecache: Fix metric values ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-27 16:25:04 +03:00
Anton Nikiforov	98da032324	[#758] ir: Do not exclude node in maintenance mode from netmap ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-26 16:05:02 +03:00
Evgenii Stratonikov	5acc13fa94	[#732] containersvc: Remove load announcement ... IR code was removed in 8879c6ea. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-09 15:17:10 +03:00
Dmitrii Stepanov	e0f0b93b5e	[#723] netmap: Drop already bootstraped check ... Because of this check, under certain conditions, the node could be removed from the network map, although the node was functioning normally. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-05 15:31:18 +03:00
Dmitrii Stepanov	eb5248621a	[#723] netmap: Send bootstrap at each epoch tick ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-05 15:31:10 +03:00
Dmitrii Stepanov	02be6a4341	[#702] node: Update SDK version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-09-29 18:46:06 +03:00
Evgenii Stratonikov	368774be95	[#691] node: Compare node info during initial bootstrap properly ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-18 07:30:15 +00:00
Evgenii Stratonikov	b9ef294b99	[#692] go.mod: Update sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-15 16:54:15 +03:00
Evgenii Stratonikov	10570fc035	[#690] go.mod: Update contract and api-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-14 14:50:30 +03:00
Evgenii Stratonikov	c6af4a3ec8	[#679] engine: Do not increase error counter on meta mismatch ... It was introduced in 69e1e6ca to help node determine faulty shards. However, the situation is possible in a real-life scenario: 1. Object O is evacuated from shard A to B. 2. Shard A is unmounted because of lower-level errors. 3. We now have object in meta on A and in blobstor on B. Technically we have it in meta on shard B too, but we still got the error if B goes to a degraded mode. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-14 10:39:18 +03:00
Evgenii Stratonikov	58239d1b2c	[#683] cli: Add context to policy parsing errors ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-12 09:47:21 +00:00
Evgenii Stratonikov	3c76884182	[#682] cli: Unify array of ranges type ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-12 10:42:34 +03:00
Evgenii Stratonikov	f435ab1b26	[#682] go.mod: Update sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-12 10:42:34 +03:00
Airat Arifullin	aa9f8dce3d	[#677] client: Refactor PrmAnnounceSpace/EndpointInfo/NetworkInfo usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-09-08 09:42:28 +03:00
Ekaterina Lebedeva	8a81af5a3b	[#653] Add context parameter to Open functions ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-09-07 18:03:29 +03:00
Evgenii Stratonikov	a716db99db	[#668] shard/test: Do not alter rootPath option ... Supposedly, this was added to allow creating 2 different shards without subtest. Now we use t.TempDir() everywhere, so this should not be a problem. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	36759f8434	[#668] shard/test: Properly check event processing ... See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	39879fa868	[#668] shard/test: Add dontRelease options ... Most of the time we would like to close shard with minor exceptions. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	c661ba1312	[#668] shard/test: Use sane defaults in the test constructor ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	268adb79cb	[#668] shard/test: Simplify shard construction ... newCustomShard() has many parameters but only the first is obligatory. `enableWriteCache` is left as-is, because it directly affects the functionality. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	429f941cda	[#668] shard/test: Release shard in t.Cleanup() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	382eb8a485	[#668] shard/test: Disable GC where it is not needed ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	42696016de	[#668] shard: Close stopChannel in GC ... It is done once, but now we could read it from multiple places. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	bdecfbc1be	[#668] shard/test: Move tests to the main package ... Semantic patch (also, duplicate definitions are removed): ``` @@ var e identifier @@ -import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/shard" -shard.e +e ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	aa23c6a83a	[#668] shard/test: Remove subtest from TestCounters ... Otherwise, individual tests cannot be run. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	da8f384324	[#668] shard/test: Fix typo in existence ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Anton Nikiforov	aeeb8193d2	[#676] node: Fix header source creation when checking eacl ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-09-06 17:06:54 +03:00
Airat Arifullin	d9de9e2bbb	[#675] client: Refactor PrmObjectDelete usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-09-06 08:05:47 +00:00
Anton Nikiforov	88d50e4c77	[#656] policer: Add "bad" testcase ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-09-06 08:04:59 +00:00