Blobovnicza tree optimizations #1337
No reviewers
Labels
No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: TrueCloudLab/frostfs-node#1337
Loading…
Reference in a new issue
No description provided.
Delete branch "dstepanov-yadro/frostfs-node:feat/blobovnicza_shrink"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Rebuild process adds temp file with
.rebuild
suffix. This allow to reduce DB count to open on blobovnicza tree initialization.Add
frostfs-cli control shards rebuild
command to rebuild blobovniczas to reduce disk usage.Drop completely background rebuild as now
frostfs-cli control shards rebuild
command exists and blobovnicza tree works after config change.73b6bf5b96
to79c8c89638
79c8c89638
to27a4a6a882
27a4a6a882
tof48a68c274
6fc79de12f
to10a11315cc
10a11315cc
to032ef546b8
032ef546b8
to74bc4eb1e0
74bc4eb1e0
to49e4c246dd
WIP: blobovnicza optimizationsto Blobovnicza tree optimizations49e4c246dd
to07501bfda2
07501bfda2
toe94c30a98f
e94c30a98f
to54a39dd2db
@ -110,0 +135,4 @@
return nil, fmt.Errorf("invalid fill percent value %d: must be (0; 100]", target)
}
result := make(map[string]struct{})
if err := b.iterateDeepest(ctx, oid.Address{}, func(lvlPath string) (bool, error) {
iterateDeepest
iterates blobovnicza tree leaf directories only. It is ok as blobovniczas with schema change collected before.@ -110,0 +157,4 @@
if !hasDBs {
return false, nil
}
for _, e := range entries {
I know that it could be done in single iteration, but code looks more complicated.
@ -414,4 +401,1 @@
// WithDisabledRebuild returns an option to disable a shard rebuild.
// For testing purposes only.
func WithDisabledRebuild() Option {
Background rebuild disabled now.
.rebuild
file what is the worse thing that could happen?@ -0,0 +18,4 @@
var shardsRebuildCmd = &cobra.Command{
Use: "rebuild",
Short: "Rebuild shards",
Long: "Rebuild reclaims storage occupied by dead objects",
It also, well, rebuilds the tree according to the width/depth from the configuration.
Fixed
@ -193,3 +193,3 @@
}
func (b *Blobovniczas) iterateExistingDBPathsDFS(ctx context.Context, path string, f func(string) (bool, error)) (bool, error) {
func (b *Blobovniczas) iterateExistingDBPathsDFS(ctx context.Context, path string, f func(string) (bool, error), fileFilter func(path string) bool) (bool, error) {
We now use
iterateExistingDB
function to iterate over.rebuild
files, which is notDB
, using an additional parameter.How about renaming?
Fixed
@ -114,0 +187,4 @@
return false, err
}
defer shDB.Close()
return blz.FillPercent() < targetFillPercent || blz.IsOverflowed(), nil
Could you explain how these conditions play together (have the same return value)?
They seem opposite --
FiilPercent() < ...
is true if the blobovnicza is empty andIsOverflowed
is true if it is full.Assume blobovnicza has
size_limit = 40mb
If actual data size is 30mb,
targetFillPercent
equals 90% andmaxObjSize
equals 100kb, then blobovnicza will be rebuilded.If actual data size is 39mb,
targetFillPercent
equals 90% andmaxObjSize
equals 100kb, then blobovnicza will not be rebuilded.If actual data size is 45mb,
targetFillPercent
equals 90% andmaxObjSize
equals 100kb (this could happen ifsize_limit
was changed from 45mb to 40mb), then blobovnicza will be rebuilded.Fixed
@ -155,6 +164,7 @@ func testRebuildFailoverValidate(t *testing.T, dir string, obj *objectSDK.Object
rRes, err := b.Rebuild(context.Background(), common.RebuildPrm{
MetaStorage: metaStub,
WorkerLimiter: &rebuildLimiterStub{},
FillPercent: 1,
How is this change related to
disable rebuild
commit message?Describe it in the commit message.
Fixed
@ -0,0 +31,4 @@
ctx, span := tracing.StartSpanFromContext(ctx, "StorageEngine.Rebuild",
trace.WithAttributes(
attribute.Int("shard_id_count", len(prm.ShardIDs)),
attribute.Int64("target_fill_percent", int64(prm.TargetFillPercent)),
Could you explain what tradeoff this setting represents?
I have some random tree.
What will the result be if
target_fill_percent = 50
, what will it be iftarget_fill_percent = 100
?Assume blobovnicza has
size_limit = 40mb
and actual data size in some blobovnicza is 30mb.If
target_fill_percent
equals 50, then rebuild will skip this blobovnicza to rebuild: 100 * (30mb / 40mb) = 75%, 75% > 50%.If
target_fill_percent
equals 100, then rebuild will move objects from this blobovnicza to other blobovnicza(s).This option defines how 'aggressive' disk space will be reclaimed. If there is enough free disk space, but user wants to reclaim disk space periodically, then fill percent value could be around 50%. If there is not enough free disk space and the point is that disk space will end completely, then fill percent value of 90-100% could be used.
The worst thing is if object exists in 'old' and 'new' blobovniczas at the same time. If such object is deleted, then it will be deleted only from single blobovnicza ('old' or 'new' depends on the fact if storageID updated or not). After deleting GC mark (or tombstone) and metabase resync this object will be available again.
Nothing happens: blobovnicza tree will work as usual, but next rebuild will open corresponding blobovnicza, try to find any incomplete move, then drops this file. I assume that file added by hand has valid name, otherwise frostfs-node will panic.
54a39dd2db
to25380d8de4
this concerns me a bit, is it easy to fix?
This behavior was from the very beginning. Why fix now?
Also panic signals that something wrong happens and blobovnicza tree has invalid structure.
missed that part
there are other ways to signal that, especially given that corruption happens
25380d8de4
to6a5abcb8ac
6a5abcb8ac
to9875704415
9875704415
tof56f613982
f56f613982
tod3b209c8e1
New commits pushed, approval review dismissed automatically according to repository settings