TrueCloudLab/frostfs-s3-gw
18
3
Fork 20
Code Issues 29 Pull requests 7 Projects Releases 23 Packages 1 Activity Actions

[#430] Fix split tree #430

Merged
alexvanin merged 6 commits from dkirillov/frostfs-s3-gw:bugfix/split_tree into master 2024-09-04 19:51:14 +00:00
Conversation 0 Commits 6 Files changed 16 +559 -271
dkirillov commented 2024-07-16 13:20:28 +00:00
Member
Copy link

Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated, so we must handle this.

Blocked by TrueCloudLab/frostfs-sdk-go#239

This is the first PR (the same as #421). Next PR will contain removing unused system and multipart nodes

Update tree service to fix split tree problem. Tree intermediate nodes can be duplicated, so we must handle this. Blocked by https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/pulls/239 This is the first PR (the same as #421). Next PR will contain removing unused system and multipart nodes
dkirillov self-assigned this 2024-07-16 13:20:28 +00:00
dkirillov added 1 commit 2024-07-16 13:20:35 +00:00
[#tmp] Fix split tree
Some checks failed
/ Builds (1.20) (pull_request) Failing after 1m39s
Details
/ Builds (1.21) (pull_request) Successful in 1m41s
Details
/ Vulncheck (pull_request) Successful in 1m37s
Details
/ DCO (pull_request) Failing after 2m6s
Details
/ Lint (pull_request) Failing after 2m44s
Details
/ Tests (1.20) (pull_request) Failing after 1m59s
Details
/ Tests (1.21) (pull_request) Failing after 16s
Details
54e4762f0a 
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated, so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
dkirillov force-pushed bugfix/split_tree from 54e4762f0a to 5bc772ec21 2024-07-16 13:46:36 +00:00 Compare
dkirillov force-pushed bugfix/split_tree from 5bc772ec21 to f4055bd0b7 2024-07-16 14:37:18 +00:00 Compare
dkirillov added 1 commit 2024-07-17 06:08:29 +00:00
[#430] ci: Update go version
All checks were successful
/ DCO (pull_request) Successful in 2m6s
Details
/ Vulncheck (pull_request) Successful in 2m23s
Details
/ Builds (1.21) (pull_request) Successful in 3m40s
Details
/ Builds (1.22) (pull_request) Successful in 3m38s
Details
/ Lint (pull_request) Successful in 8m23s
Details
/ Tests (1.21) (pull_request) Successful in 3m4s
Details
/ Tests (1.22) (pull_request) Successful in 3m36s
Details
0cc13e07c4 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
dkirillov added 2 commits 2024-07-17 14:40:40 +00:00 
It's need to fit user expectation on deleting CORs for example.
Previously after removing cors (that was uploaded in split manner)
we can still get some data (from other node)
because deletion worked only for latest node version.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
[#430] Delete all split version at once
All checks were successful
/ Builds (1.21) (pull_request) Successful in 2m20s
Details
/ Builds (1.22) (pull_request) Successful in 2m55s
Details
/ DCO (pull_request) Successful in 3m27s
Details
/ Vulncheck (pull_request) Successful in 4m40s
Details
/ Lint (pull_request) Successful in 7m6s
Details
/ Tests (1.21) (pull_request) Successful in 4m35s
Details
/ Tests (1.22) (pull_request) Successful in 4m32s
Details
6943ca2e94 
Previously after split we can get two `null` versioned object with the same key
and deleting such key removes only one node/object.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
dkirillov added 1 commit 2024-07-18 13:41:17 +00:00
[#430] multipart: Support removing duplicated parts
All checks were successful
/ DCO (pull_request) Successful in 2m23s
Details
/ Builds (1.21) (pull_request) Successful in 2m33s
Details
/ Builds (1.22) (pull_request) Successful in 2m4s
Details
/ Vulncheck (pull_request) Successful in 2m10s
Details
/ Lint (pull_request) Successful in 4m11s
Details
/ Tests (1.21) (pull_request) Successful in 2m36s
Details
/ Tests (1.22) (pull_request) Successful in 2m32s
Details
de87b3c053 
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
dkirillov force-pushed bugfix/split_tree from de87b3c053 to 93cd6afede 2024-07-18 14:01:33 +00:00 Compare
dkirillov force-pushed bugfix/split_tree from 93cd6afede to 977a20760b 2024-07-19 08:28:17 +00:00 Compare
dkirillov changed title from WIP: [#tmp] Fix split tree to WIP: [#430] Fix split tree 2024-07-19 08:28:59 +00:00
dkirillov changed title from WIP: [#430] Fix split tree to [#430] Fix split tree 2024-07-19 08:49:03 +00:00
dkirillov requested review from storage-services-committers 2024-07-19 08:49:26 +00:00
dkirillov requested review from storage-services-developers 2024-07-19 08:49:36 +00:00
alexvanin added 2 commits 2024-07-19 13:35:12 +00:00 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
[#430] Bump frostfs-api-go for latest stable marshaler
Some checks failed
/ DCO (pull_request) Successful in 1m59s
Details
/ Builds (1.21) (pull_request) Failing after 2m11s
Details
/ Builds (1.22) (pull_request) Failing after 1m59s
Details
/ Vulncheck (pull_request) Successful in 2m26s
Details
/ Lint (pull_request) Successful in 4m35s
Details
/ Tests (1.21) (pull_request) Successful in 2m38s
Details
/ Tests (1.22) (pull_request) Successful in 2m39s
Details
92be2a2cb2 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
alexvanin force-pushed bugfix/split_tree from 92be2a2cb2 to b2a5da8247 2024-07-19 13:42:50 +00:00 Compare
alexvanin approved these changes 2024-07-19 13:56:18 +00:00
alexvanin merged commit b2a5da8247 into master 2024-07-19 13:56:36 +00:00
alexvanin referenced this pull request from a commit 2024-07-19 13:56:37 +00:00
[#430] Fix split tree
alexvanin referenced this pull request from a commit 2024-07-19 13:56:37 +00:00
[#430] ci: Update go version
alexvanin deleted branch bugfix/split_tree 2024-07-19 13:56:40 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
alexvanin
TrueCloudLab/storage-services-developers
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
good first issue

Good for newcomers

  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
dkirillov
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-s3-gw#430
No description provided.
Delete branch "dkirillov/frostfs-s3-gw:bugfix/split_tree"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?