TrueCloudLab/frostfs-node
21
1
Fork 27
Code Issues 104 Pull requests 4 Projects Releases 2 Activity Actions

local_object_storage: Guarantee graves removal when handling expired tombstones #1481

Open
a-savchuk wants to merge 9 commits from a-savchuk/frostfs-node:remove-dangling-locks into master
pull from: a-savchuk/frostfs-node:remove-dangling-locks
merge into: TrueCloudLab:master
Conversation 0 Commits 9 Files changed 31 +580 -167
a-savchuk commented 2024-11-08 09:54:58 +00:00
Member
Copy link

#1445

To summarize,

  • Add expiration epoch to records in graveyard bucket (let's call these records graves)
    • Fix existing tests because now inhume operation requires expiration epoch
    • Add a test: iterate graveyard which is populated with graves of both old and new format, i. e. without and with expiration epoch, respectively
  • Change the GC behavior
    • Change naming to better reflect the GC behavior
    • Remove expired objects of all types in one worker, before objects of different types were handled in different go-routines
    • Handle graves with expiration epoch separately from others: remove them without searching for the related tombstone
    • Add test:
      • GC handle graves without and with expiration epoch

I've described the old and new behavior of GC in #1445. Feel free to ask any questions and argue.

#1445 To summarize, - Add expiration epoch to records in graveyard bucket (let's call these records _graves_) - Fix existing tests because now inhume operation requires expiration epoch - Add a test: iterate graveyard which is populated with graves of both old and new format, i. e. without and with expiration epoch, respectively - Change the GC behavior - Change naming to better reflect the GC behavior - Remove expired objects of all types in one worker, before objects of different types were handled in different go-routines - Handle graves with expiration epoch separately from others: remove them without searching for the related tombstone - Add test: - GC handle graves without and with expiration epoch I've described the old and new behavior of GC in #1445. **Feel free to ask any questions and argue.**
a-savchuk force-pushed remove-dangling-locks from 0013294c1b to 93c9ec3256 2024-11-08 12:23:49 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 93c9ec3256 to 45cbd497cd 2024-11-10 21:16:47 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 45cbd497cd to 6738de7df7 2024-11-11 09:35:48 +00:00 Compare
acid-ant reviewed 2024-11-12 08:05:39 +00:00
pkg/local_object_storage/engine/lock.go Outdated
@ -139,0 +165,4 @@
})
}
func (e *StorageEngine) dropOrphanLocks(ctx context.Context, cnt cid.ID, locks []oid.ID) (locksToKeep []oid.ID) {
acid-ant commented 2024-11-12 08:05:39 +00:00
Member
Copy link

Why it is named drop when it is used to find IDs?

Why it is named `drop` when it is used to find IDs?
a-savchuk commented 2024-12-20 11:46:47 +00:00
Author
Member
Copy link

I've changed the solution drastically. Please see the task and PR description for more information

I've changed the solution drastically. Please see the task and PR description for more information
a-savchuk force-pushed remove-dangling-locks from 6738de7df7 to be8b99e3bc 2024-12-15 23:50:41 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from be8b99e3bc to 163c3dba71 2024-12-16 22:19:35 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 163c3dba71 to 34df21cc4a 2024-12-17 07:38:09 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 34df21cc4a to e2a6663ef6 2024-12-17 10:52:15 +00:00 Compare
a-savchuk added 3 commits 2024-12-18 20:48:27 +00:00 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
[#1445] shard/gc: Remove graves
Some checks failed
Vulncheck / Vulncheck (pull_request) Successful in 3m6s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m35s
Details
Tests and linters / gopls check (pull_request) Successful in 4m0s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 6m26s
Details
DCO action / DCO (pull_request) Successful in 6m59s
Details
Build / Build Components (pull_request) Successful in 7m56s
Details
Tests and linters / Staticcheck (pull_request) Successful in 8m0s
Details
Tests and linters / Lint (pull_request) Successful in 9m14s
Details
Tests and linters / Tests with -race (pull_request) Failing after 12m32s
Details
Tests and linters / Tests (pull_request) Failing after 16m43s
Details
42681223d0 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
a-savchuk force-pushed remove-dangling-locks from 42681223d0 to e2d26cd213 2024-12-19 20:47:18 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from e2d26cd213 to 0092a49f8f 2024-12-20 08:23:48 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 0092a49f8f to de1ef66a21 2024-12-20 09:33:31 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from de1ef66a21 to ccdcd67ec3 2024-12-20 09:35:01 +00:00 Compare
a-savchuk changed title from WIP: policer: Handle oprhan locked bucket and graveyard bucket records to WIP: local_object_storage: Guarantee consistency when handling expired tombstones and lock objects 2024-12-20 09:38:23 +00:00
a-savchuk changed title from WIP: local_object_storage: Guarantee consistency when handling expired tombstones and lock objects to local_object_storage: Guarantee consistency when handling expired tombstones and lock objects 2024-12-20 11:45:03 +00:00
a-savchuk requested review from storage-core-committers 2024-12-20 11:45:03 +00:00
a-savchuk requested review from storage-core-developers 2024-12-20 11:45:04 +00:00
a-savchuk force-pushed remove-dangling-locks from ccdcd67ec3 to 220dc8af30 2024-12-23 12:57:22 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 220dc8af30 to 5b43f2a1f0 2024-12-24 09:24:51 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 5b43f2a1f0 to 4c73f1be9c 2024-12-24 10:01:29 +00:00 Compare
a-savchuk force-pushed remove-dangling-locks from 4c73f1be9c to e12c763de6 2024-12-24 10:10:17 +00:00 Compare
a-savchuk changed title from local_object_storage: Guarantee consistency when handling expired tombstones and lock objects to local_object_storage: Guarantee graves removal when handling expired tombstones 2024-12-24 10:17:48 +00:00
dstepanov-yadro requested changes 2024-12-27 14:40:57 +00:00
pkg/local_object_storage/metabase/graveyard.go
@ -314,0 +345,4 @@
err := db.boltDB.Batch(func(tx *bbolt.Tx) error {
graveyard := tx.Bucket(graveyardBucketName)
if graveyard == nil {
return errors.New("no graveyard bucket")
dstepanov-yadro commented 2024-12-27 14:15:39 +00:00
Member
Copy link

To var

To var
pkg/services/object/common/writer/local.go
@ -42,0 +44,4 @@
return fmt.Errorf("get expiration epoch: %w", err)
}
if !has {
return errors.New("tombstone has no expiration epoch")
dstepanov-yadro commented 2024-12-27 14:40:52 +00:00
Member
Copy link

To var

To var
All checks were successful
DCO action / DCO (pull_request) Successful in 3m57s
Required
Details
Tests and linters / Run gofumpt (pull_request) Successful in 4m6s
Required
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m10s
Required
Details
Build / Build Components (pull_request) Successful in 5m1s
Required
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m1s
Required
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m10s
Required
Details
Tests and linters / gopls check (pull_request) Successful in 5m17s
Required
Details
Tests and linters / Tests (pull_request) Successful in 5m44s
Required
Details
Tests and linters / Lint (pull_request) Successful in 6m13s
Required
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m45s
Required
Details
This pull request doesn't have enough approvals yet. 0 of 2 approvals granted.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u remove-dangling-locks:a-savchuk-remove-dangling-locks
git checkout a-savchuk-remove-dangling-locks
Sign in to join this conversation.
Reviewers
No reviewers
dstepanov-yadro
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
badger

Badger DB related issues

  
frostfs-adm

   
frostfs-cli

   
frostfs-ir

   
frostfs-lens

   
frostfs-node

   
good first issue

Good for newcomers

  
triage
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1481
No description provided.
Delete branch "a-savchuk/frostfs-node:remove-dangling-locks"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?