WIP: Simplify write-cache #314

carpawell · 2023-05-04T07:10:46Z

carpawell commented

2023-05-04 07:10:46 +00:00

Tests are ok. All the previous scenarios works good. Some Get requests will be slowed down of course.

Tests are ok. All the previous scenarios works good. Some `Get` requests will be slowed down of course.

carpawell self-assigned this 2023-05-04 07:10:46 +00:00

carpawell force-pushed refactor/simplify-WC from bc678bb28f to 12f3458f2a

2023-05-05 17:56:20 +00:00

Compare

carpawell force-pushed refactor/simplify-WC from 12f3458f2a to 9672484abd

2023-05-05 17:57:19 +00:00

Compare

carpawell reviewed 2023-05-05 18:03:12 +00:00

pkg/local_object_storage/writecache/delete.go Outdated

					
				@ -60,0 +65,4 @@

					// Write-cache often, and for non-existing objects (persisted

					// to the main storage and dropped from the WC's storage) it

					// is `os.Stat` vs `os.Remove` calls after all.

					res, err := c.fsTree.Get(ctx, common.GetPrm{Address: addr})

carpawell commented

2023-05-05 18:03:12 +00:00

@TrueCloudLab/storage-core-committers, what do you think about it?

some thoughts:

we can add some LRU cache again. it will store object's size;
extend FSTree with some calls like ObjectSize (os.Stat i guess) and IterateObjectSizes (to init metrics on every restart correctly). best idea, i think but it is an FSTree change for write-cache;
store metrics in a separate file. will speed up initializing but becomes more like a metabase with more disk operations per Put/Delete/Get operation;

@TrueCloudLab/storage-core-committers, what do you think about it? some thoughts: 1. we can add some LRU cache again. it will store object's size; 2. extend `FSTree` with some calls like `ObjectSize` (`os.Stat` i guess) and `IterateObjectSizes` (to init metrics on every restart correctly). best idea, i think but it is an `FSTree` change for `write-cache`; 3. store metrics in a separate file. will speed up initializing but becomes more like a `metabase` with more disk operations per `Put`/`Delete`/`Get` operation;

carpawell requested review from storage-core-developers 2023-05-05 18:04:45 +00:00

carpawell requested review from storage-core-committers 2023-05-05 18:04:45 +00:00

carpawell requested review from fyrchik 2023-05-05 18:04:48 +00:00

carpawell changed title from ~~WIP: refactor/simplify-WC~~ to refactor/simplify-WC

2023-05-05 18:05:17 +00:00

carpawell force-pushed refactor/simplify-WC from 9672484abd to 34544502dc

2023-05-05 18:05:40 +00:00

Compare

fyrchik self-assigned this 2023-05-10 12:40:34 +00:00

carpawell was unassigned by fyrchik

2023-05-10 12:40:35 +00:00

dstepanov-yadro reviewed 2023-05-10 12:52:09 +00:00

pkg/local_object_storage/writecache/mode.go

					
				@ -57,0 +43,4 @@

						workersActive = true

					}

					stopWorkers := m.NoMetabase() && !c.mode.NoMetabase() || c.mode.ReadWrite() && !m.ReadWrite()