351 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	f526f49995	[#874] engine: Check object existance concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 09:28:29 +03:00
Dmitrii Stepanov	f5160b27fc	[#920] tests: Fix data races ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-19 14:06:05 +03:00
Dmitrii Stepanov	63d3ed1ad8	[#904] tests: Close test engine after test ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	a2ab373a0a	[#895] metabase: Do not delete GC mark for virtual objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	7166e77c2b	[#895] test: Add logger to test shard ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	47dcfa20f3	[#895] test: Use t.Cleanup only for external resources ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	f1b2b8bffa	[#895] test: Fix NewLogger arguments list ... `debug` is always true. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	4b8b4da681	[#864] engine: Drop container count metric if container removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:45:32 +03:00
Dmitrii Stepanov	d75e7e9a21	[#864] engine: Drop container size metric if container deleted ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:44:54 +03:00
Dmitrii Stepanov	581887148a	[#569] cli: Add control shards writecache seal command ... It does the same as `control shards flush-writecache --seal`, but has better name. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	0cb0fc1735	[#569] writecache: Allow to seal writecache after flush ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	8180a0664f	[#887] node: Drop badger writecache implementation ... Badger implementation isn't tested and works not well, but requires human resources to maintain. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-22 13:00:54 +03:00
Dmitrii Stepanov	f1c7905263	[#661] blobovniczatree: Make Rebuild concurrent ... Different DBs can be rebuild concurrently. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	d30ab5f29e	[#838] metabase: Count user objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Alexander Chuprov	b2c63e57ba	[#651] engine/test: Speedup StorageEngine_Inhume ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Dmitrii Stepanov	1b22801eed	[#788] engine: Fix flaky tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:25:46 +03:00
Dmitrii Stepanov	70ab1ebd54	[#763] metrics: Add container_objects_total metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	226e84d782	[#684] node: Add skipped objects count to evacuation result ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
Dmitrii Stepanov	79088baa06	[#772] node: Apply gofumpt ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:03 +03:00
Dmitrii Stepanov	d4b6ebe7e7	[#725] writecache: Fix metric values ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-27 12:22:29 +03:00
Alexander Chuprov	c1e4130020	[#146] node: Add trace_id to logs ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-09-27 11:05:27 +03:00
Evgenii Stratonikov	c6af4a3ec8	[#679] engine: Do not increase error counter on meta mismatch ... It was introduced in 69e1e6ca to help node determine faulty shards. However, the situation is possible in a real-life scenario: 1. Object O is evacuated from shard A to B. 2. Shard A is unmounted because of lower-level errors. 3. We now have object in meta on A and in blobstor on B. Technically we have it in meta on shard B too, but we still got the error if B goes to a degraded mode. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-14 10:39:18 +03:00
Ekaterina Lebedeva	8a81af5a3b	[#653] Add context parameter to Open functions ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-09-07 18:03:29 +03:00
Evgenii Stratonikov	96e690883f	[#638] Unify test loggers ... In some places we have debug=false, in others debug=true. Let's be consistent. Semantic patch: ``` @@ @@ -test.NewLogger(..., false) +test.NewLogger(..., true) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Evgenii Stratonikov	322c1dc273	[#638] Use test.NewLogger() in tests ... Semantic patch (restricted to **/*_test.go): ``` @@ @@ +import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test" -import "go.uber.org/zap" -import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger" -&logger.Logger{Logger: zap.L()} +test.NewLogger(t, false) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Alexander Chuprov	4d9a6c07fb	[#618] core: Replace fmt.Sprintf with strconv.FormatUint ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-08-21 07:14:50 +00:00
Evgenii Stratonikov	f7042c5a6f	[#609] Replace zaptest.NewLogger() with zap.L() ... Semantic patch: ``` @@ @@ -import "go.uber.org/zap/zaptest" +import "go.uber.org/zap" -zaptest.NewLogger(t) +zap.L() ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:51:10 +00:00
Evgenii Stratonikov	e604a3d749	[#607] *: Use zap.Stringer() where possible ... Semantic patch: ``` @@ var f expression var t expression var a expression @@ f( ..., - zap.String(t, a.String()), + zap.Stringer(t, a), ..., ) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:50:02 +00:00
Evgenii Stratonikov	d15199c5d8	[#596] engine: Consider context errors as logical ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-16 10:39:41 +03:00
Alejandro Lopez	ae8be495c8	[#xx] Avoid manual management of files in tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-14 14:01:39 +03:00
Alejandro Lopez	5b7e4a51b7	[#481] Update frostfs-sdk-go and error pointer receivers ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-09 10:26:53 +00:00
Alejandro Lopez	1a0cb0f34a	[#421] Try using badger for the write-cache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-07 08:16:57 +00:00
Dmitrii Stepanov	b4e72a2dfd	[#335] treesvc: Sort nodes by Filename in GetSubTree ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-20 10:14:10 +03:00
Dmitrii Stepanov	3e8de14e7d	[#382] evacuate: Fix unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-17 10:24:05 +00:00
Alexander Chuprov	033eaf77e1	[#496] node: Fix linter importas ... Standardize the alias of the import frostfs-sdk-go/object as objectSDK. Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-07-06 15:36:41 +03:00
Dmitrii Stepanov	43d263c3d5	[#428] linter: Fix unkeyed assignment ... Thanks to gopls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-26 13:29:39 +00:00
Dmitrii Stepanov	cac4ed93d6	[#428] engine: Add low_mem config parameter ... Concurrent initialization in case of the metabase resync leads to high memory consumption and potential OOM. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-26 13:29:39 +00:00
Dmitrii Stepanov	059e9e88a2	[#373] metabase: Add metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	028d4a8058	[#373] blobovnicza: Add metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	01a0c97760	[#453] engine: Set Disabled mode to deleted shard ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-20 12:04:07 +03:00
Evgenii Stratonikov	69df0d21c2	[#446] engine: Move to read-only on blobstor errors ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-16 14:53:32 +03:00
Evgenii Stratonikov	20b84f183a	[#446] engine: Simplify logs for shard mode change ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-16 14:51:29 +03:00
Dmitrii Stepanov	4449006862	[#424] metrics: Use mode value as metric value for shard ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-14 18:26:19 +03:00
Dmitrii Stepanov	1b364d8cf4	[#424] metrics: Refactor engine metrics ... Use histogram vector to measure request duration. Fix naming like in Prometheus best practice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-14 14:53:32 +03:00
Dmitrii Stepanov	85deb12f4d	[#424] writecache: Drop metrics when close ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-14 14:52:40 +03:00
Dmitrii Stepanov	07f155ac77	[#424] metrics: Use labels for writecache methods and operations ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-14 14:52:40 +03:00
Dmitrii Stepanov	2541d319de	[#266] pilorama: Allow to get current tree height ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-13 10:00:45 +00:00
Evgenii Stratonikov	41ab4d070e	[#423] *: Use hrw.StringHash() where possible ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-13 07:18:25 +00:00
Anton Nikiforov	263c6fdc50	[#372] node: Add metrics for the error counter in the engine ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-06-07 13:04:47 +00:00
Dmitrii Stepanov	74578052f9	[#412] node: Replace tracing package ... Use observability module. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-01 13:23:11 +00:00
Dmitrii Stepanov	dbf41391b5	[#401] engine: Extend evacuation logs ... Add operation-tag to logger. Log evacuation results. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-05-31 13:24:30 +03:00
Dmitrii Stepanov	3220c4df9f	[#376] metrics: Add GC metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-05-31 10:22:12 +00:00
Alejandro Lopez	faca861451	[#411] Remove unnecessary pointers for sync objects ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-05-31 10:19:14 +00:00
Dmitrii Stepanov	271a56c2ab	[#395] metrics: Drop redundant metrics ... HistogramVec already has labeled counter. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-05-26 13:39:37 +00:00
Dmitrii Stepanov	2ce43935f9	[#312] metrics: Add writecache metrcis ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-05-24 10:18:39 +00:00
Evgenii Stratonikov	4b768fd115	[#381] *: Move to sync/atomic ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-05-23 08:18:01 +03:00
Dmitrii Stepanov	e4889e06ba	[#329] node: Make evacuate async ... Now it's possible to run evacuate shard in async. Also only one evacuate process can be in progress. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-05-19 08:43:52 +00:00
Evgenii Stratonikov	945454f60c	[#321] engine/test: Execute tests in parallel ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-05-12 09:45:03 +00:00
Alejandro Lopez	973af12854	[#327] tests: replace os.MkdirTemp with t.TempDir ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-05-05 11:24:51 +00:00
Dmitrii Stepanov	eca5c210dd	[#299] evacuate: Add context cancel checks ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-05-02 15:55:30 +03:00
Airat Arifullin	ada081dfd5	[#19] node: Make policier read shards concurrently ... * Introduce ListWithMultiCursor that simultaneously reads objects from different shards Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-04-27 11:11:47 +03:00
Evgenii Stratonikov	8466894fdf	[#250] control: remove DumpShard and RestoreShard RPC ... We have `Evacuate` with a cleaner interface. Also, remove them from CLI and engine. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-04-14 12:28:49 +00:00
Dmitrii Stepanov	6121b541b5	[#242] treesvc: Add tracing spans ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-14 10:25:53 +00:00
Dmitrii Stepanov	d62c6e4ce6	[#242] node: Add tracing spans ... Add tracing spans for PUT requests. Add tracing spans for DELETE requests. Add tracing spans for SELECT requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-14 10:25:53 +00:00
Evgenii Stratonikov	0e31c12e63	[#240] logs: Move log messages to constants ... Drop duplicate entities. Format entities. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com> Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-04-14 05:06:09 +00:00
Dmitrii Stepanov	0920d848d0	[#135] get-object: Add tracing spans ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-12 06:52:00 +00:00
Evgenii Stratonikov	dbc3811ff4	[#191] engine: Allow to remove redundant object copies ... RemoveDuplicates() removes all duplicate object copies stored on multiple shards. All shards are processed and the command tries to leave a copy on the best shard according to HRW. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-04-07 17:25:50 +00:00
Airat Arifullin	ab891517de	[#116] node: Fix bug with extra generated files in TestReload ... * Create testNewEngine in engineWithShards without default opts Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-04-05 18:49:34 +03:00
Airat Arifullin	6f7b6a8813	[#116] node: Improve shard/engine construction in tests ... * Introduce testEngineWrapper that can be constructed with different options Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-04-05 14:36:40 +00:00
Dmitrii Stepanov	23575e1ac0	[#210] policier: Resolve contextcheck linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-05 14:55:52 +03:00
Dmitrii Stepanov	5a66db80c5	[#188] engine: Refactor shard evacuation ... Resolve funlen and gocognit linter for StorageEngine.Evacuate method Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-04 14:50:43 +03:00
Dmitrii Stepanov	456bc097f7	[#188] engine: Refactor get range from engine ... Resolve funlen linter for StorageEngine.getRange method Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-04 14:50:43 +03:00
Dmitrii Stepanov	3010ca2649	[#188] engine: Refactor get object from engine ... Resolve funlen linter for StorageEngine.get method Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-04-04 14:50:43 +03:00
Dmitrii Stepanov	a7c79c773a	[#168] node: Refactor node config ... Resolve containedctx linter for cfg Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-31 09:32:59 +03:00
Airat Arifullin	7be5a0fd79	[#189] node: Fix bug with sync/atomic package ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-03-30 14:50:43 +00:00
Dmitrii Stepanov	9f0bce5c15	[#183] gc: Fix drop expired locked simple objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-30 15:33:42 +03:00
Alejandro Lopez	341fe1688f	[#139] test: Add test storage implementation ... This aims to reduce the usage of chmod hackery to induce or simulate OS-related failures. Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-03-29 14:28:49 +00:00
Pavel Karpy	97e201993b	[#184] *: Unify done contexts handling ... If `ctx.Done()`, return `ctx.Err()` in every function that returns an error. Signed-off-by: Pavel Karpy <p.karpy@yadro.com>	2023-03-29 10:34:54 +00:00
Airat Arifullin	221203beeb	[#180] node: Refactor panics in unit test ... * Replace panics in unit tests by require.NoError and t.Fatalf Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-03-29 12:39:07 +03:00
Ayrat Arifullin	9808dec591	[#86] node: Move testing utils to one package ... Move testing utils from tests in local_object_storage package to unified testutil package Signed-off-by: Airat Arifullin <aarifullin@yadro.com>	2023-03-23 08:19:15 +00:00
Evgenii Stratonikov	47e8c5bf23	[#156] pilorama: Remove CIDDescriptor from TreeApply() ... Initially it was there to check whether an update is being initiated by a proper node. It is now obsolete for 2 reasons: 1. Background synchronization fetches all operations from a single node. 2. There are a lot more problems with trust in the tree service, it is only used in controlled environments. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-03-22 07:14:18 +00:00
Dmitrii Stepanov	481a1ca6f3	[#148] linter: Add gocognit linter ... Code with high cognitive complexity is hard intuitively to understand Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-21 09:54:41 +03:00
Dmitrii Stepanov	97c36ed3ec	[#148] linter: Add funlen linter ... Long functions are hard to understand and source of errors Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-21 09:54:41 +03:00
Pavel Karpy	f006f3b342	[#67] node: Make engine's IsLocked public ... It will allow reusing that method in expiration checks. Signed-off-by: Pavel Karpy <p.karpy@yadro.com>	2023-03-16 16:20:45 +03:00
Evgenii Stratonikov	3e6fd4c611	[#82] pilorama: Allow to store last sync height ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-03-13 11:25:44 +00:00
Alex Vanin	20de74a505	Rename package name ... Due to source code relocation from GitHub. Signed-off-by: Alex Vanin <a.vanin@yadro.com>	2023-03-07 16:38:26 +03:00
Dmitrii Stepanov	6925fb4c59	[TrueCloudLab/hrw#2] node: Use typed HRW methods ... Update HRW lib and use typed HRW methods to sort shards and nodes Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-02-28 13:36:25 +03:00
Dmitrii Stepanov	c3a7039801	[TrueCloudLab/hrw#2] node: Optimize shard hash ... Compute shard hash only once Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-02-28 13:36:25 +03:00
Alejandro Lopez	cb5468abb8	[#66] node: Replace interface{} with any ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-02-21 16:47:07 +03:00
Evgenii Stratonikov	5cb2c5ae62	[#2238] engine: Add test for component initialization failures ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-02-20 13:53:27 +03:00
Evgenii Stratonikov	c53903ccd0	[#2238] engine: Make Open and Init similar ... 1. Both could initialize shards in parallel. 2. Both should close shards after an error. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-02-20 13:53:27 +03:00
Pavel Karpy	40822adb51	[#2213] node: Do not return object expired object ... "Object is expired" means that object is presented in `meta` but it is not `ObjectNotFound` error. Previous implementation made `shard` search for an object without `meta` which was an error. Signed-off-by: Pavel Karpy <p.karpy@yadro.com>	2023-02-20 13:53:27 +03:00
Stanislav Bogatyrev	cb016d53a6	[#1] Fix comments and error messages ... Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>	2023-02-06 17:41:14 +03:00
Pavel Karpy	89a0266f5e	[#1794] metrics: Track physical object capacity per shard ... Signed-off-by: Pavel Karpy <p.karpy@yadro.com>	2023-01-26 20:06:28 +03:00
Evgenii Stratonikov	9513f163aa	[#2116] metrics: Track physical object capacity in the container ... Currently we track based on `PayloadSize`, because it is already stored in the metabase and it is easier to calculate without slowing down the whole system. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com> Signed-off-by: Pavel Karpy <p.karpy@yadro.com>	2023-01-26 20:06:28 +03:00
Evgenii Stratonikov	d65a95a2c6	[#28] pilorama: Remove LogMove struct ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-01-25 15:31:47 +03:00
Evgenii Stratonikov	c72576e72f	[#2208] engine: Log time-consuming shard operations ... Currently the only way to tell whether `evacuate/set-mode` is finished is to set a very big timeout and _hope_ that the operation will finish. In this commit we add INFO logs for such operations which should simplify the life of an administrator. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-01-25 15:31:47 +03:00
Evgenii Stratonikov	ba393e3e91	[#2188] engine: Fix panic during setting shard mode ... Under load changing shard mode can lead to it being removed from the list during some other PUT. ``` Dec 28 07:01:26 az neofs-node[364505]: panic: runtime error: invalid memory address or nil pointer dereference Dec 28 07:01:26 az neofs-node[364505]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xc9fbb1] Dec 28 07:01:26 az neofs-node[364505]: goroutine 11791912 [running]: Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).putToShard(0xc000435490, {0xc0003f7a28?, 0xc0001192c0?}, 0x2, {0x0, 0x> Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:91 +0x1b1 Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).put.func1(0xc000435490?, {0xc0003f7a28?, 0xc0001192c0?}) Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:71 +0x19c Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).iterateOverSortedShards(0x1?, {{0x62, 0x23, 0xfe, 0x60, 0x67, 0xd5, 0x> Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/shards.go:225 +0xc8 Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).put(0xc000435490, {0x1?}) Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:66 +0x2a9 Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).Put.func1() Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:43 +0x2a Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).execIfNotBlocked(0x8?, 0x38?) Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/control.go:147 +0xcf Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).Put(0xc4df775a80?, {0x0?}) Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:42 +0x65 Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.Put(0xc06d928b80?, 0xc06b1b8dc8?) Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:158 +0x19 Dec 28 07:01:26 az neofs-node[364505]: main.engineWithoutNotifications.Put({0x20301b?}, 0x20301b?) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-01-25 15:31:47 +03:00
Anton Nikiforov	3d57f4c961	[#2179] test: Fix test TestEvacuateNetwork/multiple_shards,_evacuate_many ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-01-24 13:37:49 +03:00
Evgenii Stratonikov	b4e90cdf51	[#2165] pilorama: Optimize TreeApply when used for synchronization ... Because synchronization _most likely_ will have apply already existing operations, it is much faster to check their presence in a read transaction. However, always doing this will degrade the perfomance for normal `Apply`. And, let's be honest, it is already not good. Thus we add a separate parameter which specifies whether this logic is enabled. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2022-12-30 11:07:35 +03:00