forked from TrueCloudLab/neoneo-go
b9be892bf9
Persist by its definition doesn't change MemCachedStore visible state, all KV
pairs that were acessible via it before Persist remain accessible after
Persist. The only thing it does is flushing of the current set of KV pairs
from memory to peristent store. To do that it needs read-only access to the
current KV pair set, but technically it then replaces maps, so we have to use
full write lock which makes MemCachedStore inaccessible for the duration of
Persist. And Persist can take a lot of time, it's about disk access for
regular DBs.
What we do here is we create new in-memory maps for MemCachedStore before
flushing old ones to the persistent store. Then a fake persistent store is
created which actually is a MemCachedStore with old maps, so it has exactly
the same visible state. This Store is never accessed for writes, so we can
read it without taking any internal locks and at the same time we no longer
need write locks for original MemCachedStore, we're not using it. All of this
makes it possible to use MemCachedStore as normally reads are handled going
down to whatever level is needed and writes are handled by new maps. So while
Persist for (*Blockchain).dao does its most time-consuming work we can process
other blocks (reading data for transactions and persisting storeBlock caches
to (*Blockchain).dao).
The change was tested for performance with neo-bench (single node, 10 workers,
LevelDB) on two machines and block dump processing (RC4 testnet up to 62800
with VerifyBlocks set to false) on i7-8565U.
Reference results (
|
||
|---|---|---|
| .. | ||
| badgerdb_store.go | ||
| badgerdb_store_test.go | ||
| boltdb_store.go | ||
| boltdb_store_test.go | ||
| leveldb_store.go | ||
| leveldb_store_test.go | ||
| memcached_store.go | ||
| memcached_store_test.go | ||
| memory_store.go | ||
| memory_store_test.go | ||
| redis_store.go | ||
| redis_store_test.go | ||
| store.go | ||
| store_config.go | ||
| store_test.go | ||
| storeandbatch_test.go | ||