Blobovnicza GET after PUT is inconsistent under high concurrency #536
Labels
No Label
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
No Milestone
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: TrueCloudLab/frostfs-node#536
Loading…
Reference in New Issue
There is no content yet.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
When issuing a synchronous
GET
call immediately afterPUT
while the storage is under heavy concurrent usage, sometimesGET
returnsobject not found
.Expected Behavior
It should either:
UNAVAILABLE
in gRPC terminology, which is canonically retryable)Current Behavior
Sporadically returns
object not found
.Possible Solution
Up for discussion.
Steps to Reproduce (for bugs)
The problems is with
opened_cache_size
-- if it is small, we can have side-effects: blobovniczas are opened and closed concurrently because of the limited cache and nothing prevents the DB from being closed while some object is being read.I suggest remove
opened_cache_size
completely and always cache everything in memory: