Badgerstore #833
No reviewers
Labels
No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: TrueCloudLab/frostfs-node#833
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "dstepanov-yadro/frostfs-node:feat/badgerstore"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Another small object store implemenation.
References #555
@ -0,0 +40,4 @@
//
// Badger uses Dynamic Level Sizes like RocksDB.
// See https://github.com/facebook/rocksdb/blob/v3.11/include/rocksdb/options.h#L366 for explanation.
func defaultCfg() *cfg {
badger config
f47b48d5b2
to207820df5e
@ -0,0 +6,4 @@
)
func key(add oid.Address) []byte {
res := make([]byte, 64)
Could you move
64
and32
to constants, please?done
207820df5e
to12b80b859d
12b80b859d
to11f5af27cd
11f5af27cd
to3e1e0393d7
3e1e0393d7
tod7c723d310
d7c723d310
tod6ead4d10e
d6ead4d10e
to7ea306d86c
@ -0,0 +62,4 @@
}
batch = append(batch, kv)
last = kv.key
if len(batch) == opts.PrefetchSize-1 {
My point is absolutely optional but this check looks like it can be placed to
:)
Brilliant
7ea306d86c
to5b5dc5a50d
@ -0,0 +62,4 @@
}
batch = append(batch, kv)
last = kv.key
if len(batch) == opts.PrefetchSize-1 {
Hm. Just curious about if this check is really necessary here. The iterator
it
is initialized with the optionopts.PrefetchSize
. Isn't it invalidated on itself when it fetchesopts.PrefetchSize
items?Docs: iterating-over-keys
This is done so that there are no long transactions. So storage reads one batch within transaction, returns results to invoker, reads another batch within transaction, etc.
@ -0,0 +47,4 @@
return MemTablesCountDefault
}
// CompactorsCount returns `mem_tables_count` value or CompactorsCountDefault.
mem_tables_count -> compactors_count
fixed
@ -0,0 +96,4 @@
to := from + prm.Range.GetLength()
payload := obj.Payload()
if pLen := uint64(len(payload)); to < from || pLen < from || pLen < to {
How about to check this
to < from
beforeget
?I would leave this code as is -- it is encountered in multiple places (basically, every GetRange) and it is not straightforward to do it right (wrong boundary checks can easily lead to panic or OOM), so now we can just copy it once tested.
5b5dc5a50d
to1b6a06e603
1b6a06e603
tobd2be1f64e
b2b28b3697
to82bf0cd2c8
82bf0cd2c8
to5399a852ed
5399a852ed
toc02315fc80
c02315fc80
to1d4732c2a8
1d4732c2a8
toffc7b32197
ffc7b32197
to6ebbada07c
6ebbada07c
to824251f650
20195677a5
to9f7b114d4f
I will wait until v0.38, then merge
9caf0f9a6d
tof5e7a7cffb
f5e7a7cffb
tofba92a3d61
fba92a3d61
toec81c1bcfd
Pull request closed