417 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	9bd05e94c8	[#1449] tree: Add ApplyBatch method ... Concurrent Apply can lead to child node applies before parent, so undo/redo operations will perform. This leads to performance degradation in case of tree with many sublevels. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-11-12 12:11:07 +00:00
Airat Arifullin	9b13a18aac	[#1479] go.mod: Bump frostfs-sdk-go version ... * Update version within go.mod; * Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api instead. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-11-08 10:43:19 +03:00
Evgenii Stratonikov	7ac0852364	[#1459] .golangci.yml: Add intrange linter, fix issues ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-30 15:18:22 +00:00
Evgenii Stratonikov	b0c5def2d9	[#1433] shard/test: Use WithDisabledGC() option where possible ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-17 14:38:07 +03:00
Alexander Chuprov	41038b2ec0	[#1431] node: Fix 'empty slice declaration using a literal' ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-10-15 08:53:59 +00:00
Alexander Chuprov	d83879d4b8	[#1431] node: Fix comment format ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-10-15 08:53:59 +00:00
Alexander Chuprov	00b1cecfb7	[#1431] obj_storage/shard: Fix visibility of 'newMetricStore' ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-10-15 08:53:59 +00:00
Anton Nikiforov	936ebbb8e5	[#1423] metabase: Hide BucketName form upper levels ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-10-08 18:41:16 +03:00
Dmitrii Stepanov	3da168f8cf	[#1412] shard: Resolve container is indexed on metabase resync ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:51 +03:00
Dmitrii Stepanov	4572fa4874	[#1412] searchSvc: Check container is indexed ... For non S3 containers it is expected to use attributes index for some attributes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:50 +03:00
Dmitrii Stepanov	be744ae3e6	[#1412] metabase: Index attributes for indexed containers ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-10-08 11:41:49 +03:00
Evgenii Stratonikov	4dc9a1b300	[#1413] engine: Remove error counting methods from Shard ... All error counting and hangling logic is present on the engine level. Currently, we pass engine metrics with shard ID metric to shard, then export 3 methods to manipulate these metrics. In this commits all methods are removed and error counter is tracked on the engine level exlusively. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-04 15:10:17 +03:00
Evgenii Stratonikov	9206ce5cd2	[#1410] shard: Provide the default implementation for MetricsWriter ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-03 08:23:06 +00:00
Evgenii Stratonikov	6c46044c9c	[#1410] shard: Move MetricsWriter interface to a separate file ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-03 08:23:06 +00:00
Anton Nikiforov	d0ed29b3c7	[#1350] node: Add ability to evacuate objects from REP 1 only ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-27 15:41:17 +03:00
Dmitrii Stepanov	76268e3ea2	[#1385] metabase: Validate that tombstone and target have the same container ID ... Target container ID is taken from tombstone: cmd/frostfs-node/object.go:507 Also object of type `TOMBSTONE` contains objectID, so tombstone and tombstoned object must have the same containerID. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-24 17:09:06 +03:00
Anton Nikiforov	34e6a309c6	[#1356] engine: Evacuate object from shards concurrently ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-24 12:50:19 +03:00
Dmitrii Stepanov	0b87be804a	[#1381] engine: Fix tests ... Drop not required `Eventually` calls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-23 11:10:29 +03:00
Dmitrii Stepanov	d3b209c8e1	[#1337] shard: Disable background rebuild ... Since `frostfs-cli control shards rebuild` command was added, there is no need for background rebuild now. For failover tests used used value 1 to rebuild only schema change. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 15:19:55 +03:00
Dmitrii Stepanov	a61201a987	[#1337] config: Move rebuild_worker_count to shard section ... This makes it simple to limit performance degradation for every shard because of rebuild. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	6b6eabe41c	[#1337] cli: Add control shards rebuild command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	d508da8397	[#1337] blobovniczatree: Add rebuild by fill percent ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Anton Nikiforov	108e4e07be	[#1349] node: Evacuate objects without setting mode to MAINTENANCE ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-05 16:08:27 +03:00
Ekaterina Lebedeva	a685fcdc96	[#1317] go.mod: Use range over int ... Since Go 1.22 a "for" statement with a "range" clause is able to iterate through integer values from zero to an upper limit. gopatch script: @@ var i, e expression @@ -for i := 0; i <= e - 1; i++ { +for i := range e { ... } @@ var i, e expression @@ -for i := 0; i <= e; i++ { +for i := range e + 1 { ... } @@ var i, e expression @@ -for i := 0; i < e; i++ { +for i := range e { ... } Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-09-03 13:00:54 +03:00
Aleksey Savchuk	2b3fc50681	[#1320] shard: Fix TestGCDropsObjectInhumedFromWritecache flaky test ... The `TestGCDropsObjectInhumedFromWritecache` test was flaky because a running asynchronous rebuild operation prevented GC from deleting the object. A test-only shard option `WithDisabledRebuild` has been added to fix this. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-02 10:26:53 +03:00
Dmitrii Stepanov	882c068410	[#1334] metabase: Store upgrade flag ... This allows to check if metabase upgrade was not completed. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Dmitrii Stepanov	7bf20c9f1f	[#1323] metabase: Add expiration epoch buckets ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	80ce7c3a00	[#1284] shard: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 13:23:35 +03:00
Dmitrii Stepanov	93d63e1632	[#1284] writecache: Allow to seal writecache async ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 13:23:35 +03:00
Dmitrii Stepanov	36efccd862	[#1298] writecache: Add shrink flag for Seal command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-08 16:32:29 +03:00
Dmitrii Stepanov	5c01bd5be8	[#1298] writecache: Add restore-mode flag for Seal command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-07 11:20:14 +03:00
Dmitrii Stepanov	18182e578e	[#1261] shard: Fix delete objects from FSTree ... Replace nil storageID with empty like by shard.Get. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-22 14:38:09 +03:00
Evgenii Stratonikov	3940bc17c1	[#1251] pilorama: Allow traversing multiple branches in parallel ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-17 11:25:07 +03:00
Evgenii Stratonikov	7085723c6b	[#1074] pilorama: Allow empty filenames in SortedByFilename() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-28 17:46:24 +03:00
Anton Nikiforov	9ac74efc41	[#1173] shard: Use mode from config on reload ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-20 11:29:10 +00:00
Anton Nikiforov	68ac490729	[#1174] shard: Update metric mode_info on Init ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-13 08:32:59 +00:00
Dmitrii Stepanov	6a39c3d15e	[#1086] engine: Do not use metabase if shard looks bad ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-13 07:35:22 +00:00
Alexander Chuprov	806236da78	[#1121] node: Change mode of shard components ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Alexander Chuprov	6f2187a420	[#1121] node: Refactor mods of shard ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Anton Nikiforov	92e19feb57	[#1147] node: Use public fields for shard.ExistsPrm ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	6130650bb6	[#1147] node: Implement Lock\Delete requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	a82c8cc5b8	[#1147] gc: Execute callback for expired tombstones when they exists ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Dmitrii Stepanov	d45d086acd	[#1129] policer: Add EC chunk replication ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:48 +03:00
Alexander Chuprov	c9efaa5819	[#966] node: Add path of the write_cache to metric labels ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 06:46:46 +00:00
Anton Nikiforov	411a8d0245	[#1004] blobovnicza: Use TTL for blobovnicza tree cache ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-26 19:54:29 +03:00
Evgenii Stratonikov	3dc81cb4fc	Reapply "[#972] Use min/max builtins" ... This reverts commit dad56d2e98. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Dmitrii Stepanov	1b17258c04	[#1029] metabase: Add refill metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 13:05:44 +03:00
Dmitrii Stepanov	57466594fb	[#1024] shard: Resync metabase concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:10 +03:00
Dmitrii Stepanov	1005bf4f56	[#1024] shard: Add refill metabase benchmark ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:10 +03:00
Ekaterina Lebedeva	d5194ab2a6	[#949] metabase: fix shard.UpdateID() ... metabase.Open() now reports metabase mode metric. shard.UpdateID() needs to read shard ID from metabase => needs to open metabase. It caused reporting 'shard undefined' metrics. To avoid reporting wrong metrics metabase.GetShardID() was added which also opens metabase and does not report metrics. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00