TrueCloudLab/neoneo-go
7
0
Fork 2
Code Issues Pull requests Projects Releases Packages Wiki Activity

core: implement basic GC for value-based storage scheme

Browse source 
The key idea here is that even though we can't ensure MPT code won't make the
node active again we can order the changes made to the persistent store in
such a way that it practically doesn't matter. What happens is:
 * after persist if it's time to collect our garbage we do it synchronously
   right in the same thread working the underlying persistent store directly
 * all the other node code doesn't see much of it, it works with bc.dao or
   layers above it
 * if MPT doesn't find some stale deactivated node in the storage it's OK,
   it'll recreate it in bc.dao
 * if MPT finds it and activates it, it's OK too, bc.dao will store it
 * while GC is being performed nothing else changes the persistent store
 * all subsequent bc.dao persists only happen after the GC is completed which
   means that any changes to the (potentially) deleted nodes have a priority,
   it's OK for GC to delete something that'll be recreated with the next
   persist cycle

Otherwise it's a simple scheme with node status/last active height stored in
the value. Preliminary tests show that it works ~18% worse than the simple
KeepOnlyLatest scheme, but this seems to be the best result so far.

Fixes #2095.
This commit is contained in:
Roman Khimov 2022-01-29 11:28:29 +03:00
parent c4ee310e85
commit 423c7883b8
5 changed files with 131 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

11
pkg/config/protocol_config.go
View file

@ -15,8 +15,13 @@ type (
ProtocolConfiguration struct {
// CommitteeHistory stores committee size change history (height: size).
CommitteeHistory map[uint32]int `yaml:"CommitteeHistory"`
Magic netmode.Magic `yaml:"Magic"`
MemPoolSize int `yaml:"MemPoolSize"`
// GarbageCollectionPeriod sets the number of blocks to wait before
// starting the next MPT garbage collection cycle when RemoveUntraceableBlocks
// option is used.
GarbageCollectionPeriod uint32 `yaml:"GarbageCollectionPeriod"`
Magic netmode.Magic `yaml:"Magic"`
MemPoolSize int `yaml:"MemPoolSize"`
// InitialGASSupply is the amount of GAS generated in the genesis block.
InitialGASSupply fixedn.Fixed8 `yaml:"InitialGASSupply"`
@ -27,7 +32,7 @@ type (
// If true, DB size will be smaller, but older roots won't be accessible.
// This value should remain the same for the same database.
KeepOnlyLatestState bool `yaml:"KeepOnlyLatestState"`
// RemoveUntraceableBlocks specifies if old blocks should be removed.
// RemoveUntraceableBlocks specifies if old data should be removed.
RemoveUntraceableBlocks bool `yaml:"RemoveUntraceableBlocks"`
// MaxBlockSize is the maximum block size in bytes.
MaxBlockSize uint32 `yaml:"MaxBlockSize"`