TrueCloudLab
/
frostfs-s3-gw
16
2
Fork 14
Code Issues 19 Pull Requests 5 Actions Packages Releases 7 Activity

Process 'object not found' error when object is missing in object service #120

Merged
alexvanin merged 1 commits from alexvanin/frostfs-s3-gw:fix/obj-not-found into master 2023-07-26 21:08:01 +00:00
Conversation 1 Commits 1 Files Changed 1
alexvanin commented 2023-05-26 15:36:59 +00:00
Owner

Closes #78

Closes #78
alexvanin self-assigned this 2023-05-26 15:36:59 +00:00
alexvanin force-pushed fix/obj-not-found from d09d29a259 to 1a00b843b5 2023-05-29 14:06:26 +00:00 Compare
alexvanin changed title from WIP: Process 'object not found' error when object is missing in object service to Process 'object not found' error when object is missing in object service 2023-05-29 14:22:35 +00:00
alexvanin requested review from storage-services-committers 2023-05-29 14:22:43 +00:00
alexvanin requested review from storage-services-developers 2023-05-29 14:22:43 +00:00
dkirillov reviewed 2023-05-30 08:16:15 +00:00
api/layer/layer.go Outdated
@ -647,2 +647,4 @@
}
if client.IsErrObjectNotFound(obj.Error) {
n.log.Debug("object not found removed", zap.String("bucket", bkt.Name), zap.Stringer("cid", bkt.CID),
dkirillov commented 2023-05-30 08:06:51 +00:00
Collaborator

Probably it's better omit removed word from log message

Probably it's better omit `removed` word from log message
👍 1
dkirillov marked this conversation as resolved
api/layer/layer.go Outdated
@ -649,0 +650,4 @@
n.log.Debug("object not found removed", zap.String("bucket", bkt.Name), zap.Stringer("cid", bkt.CID),
zap.String("object", obj.Name), zap.String("oid", obj.VersionID))
obj.Error = nil
dkirillov commented 2023-05-30 08:11:43 +00:00
Collaborator

Don't we want to return object not found to user to tell him that something went wrong?

Don't we want to return `object not found` to user to tell him that something went wrong?
alexvanin commented 2023-06-01 12:26:51 +00:00
Poster
Owner

Isn't the main idea of the issue is to return no error when trying delete unavailable object?

Isn't the main idea of the issue is to return no error when trying delete unavailable object?
dkirillov marked this conversation as resolved
api/layer/layer.go Outdated
@ -649,0 +652,4 @@
obj.Error = nil
n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name)
dkirillov commented 2023-05-30 08:12:34 +00:00
Collaborator

Can move this out of if block to reuse with previous if?

Can move this out of `if` block to reuse with previous `if`?
alexvanin commented 2023-06-01 12:28:16 +00:00
Poster
Owner

Not sure, because we don't to delete cache on other errors besides not found / already removed?

Not sure, because we don't to delete cache on other errors besides not found / already removed?
dkirillov commented 2023-06-01 14:20:44 +00:00
Collaborator

I mean instead of:

if client.IsErrObjectAlreadyRemoved(obj.Error) {
	// ...
	n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name)
}

if client.IsErrObjectNotFound(obj.Error) {
	// ...
	n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name)
}

return obj

write:

if client.IsErrObjectAlreadyRemoved(obj.Error) {
	// ...
} else if client.IsErrObjectNotFound(obj.Error) {
	// ...
}

n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name)

return obj
I mean instead of: ```golang if client.IsErrObjectAlreadyRemoved(obj.Error) { // ... n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name) } if client.IsErrObjectNotFound(obj.Error) { // ... n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name) } return obj ``` write: ```golang if client.IsErrObjectAlreadyRemoved(obj.Error) { // ... } else if client.IsErrObjectNotFound(obj.Error) { // ... } n.cache.DeleteObjectName(bkt.CID, bkt.Name, obj.Name) return obj ```
dkirillov commented 2023-06-01 14:23:12 +00:00
Collaborator

though it's a matter of taste

though it's a matter of taste
alexvanin commented 2023-06-01 14:44:58 +00:00
Poster
Owner

Consider parsing "access denied" error.

First snippet does not clear the cache, second snippet does. I am not sure we want to clear cache for any error beside "not found" and "already removed".

Consider parsing "access denied" error. First snippet does not clear the cache, second snippet does. I am not sure we want to clear cache for any error beside "not found" and "already removed".
👍 1
dkirillov marked this conversation as resolved
dkirillov commented 2023-05-30 08:18:21 +00:00
Collaborator

Let's add the following test to this file:

func TestDeleteBucketOnNotFoundError(t *testing.T) {
	hc := prepareHandlerContext(t)

	bktName, objName := "bucket-for-removal", "object-to-delete"
	bktInfo := createTestBucket(hc, bktName)

	putObject(t, hc, bktName, objName)

	nodeVersion, err := hc.tree.GetUnversioned(hc.context, bktInfo, objName)
	require.NoError(t, err)
	var addr oid.Address
	addr.SetContainer(bktInfo.CID)
	addr.SetObject(nodeVersion.OID)
	hc.tp.SetObjectError(addr, apistatus.ObjectNotFound{})

	deleteObjects(t, hc, bktName, [][2]string{{objName, emptyVersion}})

	deleteBucket(t, hc, bktName, http.StatusNoContent)
}
Let's add the following test to [this file](https://git.frostfs.info/alexvanin/frostfs-s3-gw/src/commit/1a00b843b57c5b922cb9bae783c5f8eef5df97f4/api/handler/delete_test.go#L38): ```golang func TestDeleteBucketOnNotFoundError(t *testing.T) { hc := prepareHandlerContext(t) bktName, objName := "bucket-for-removal", "object-to-delete" bktInfo := createTestBucket(hc, bktName) putObject(t, hc, bktName, objName) nodeVersion, err := hc.tree.GetUnversioned(hc.context, bktInfo, objName) require.NoError(t, err) var addr oid.Address addr.SetContainer(bktInfo.CID) addr.SetObject(nodeVersion.OID) hc.tp.SetObjectError(addr, apistatus.ObjectNotFound{}) deleteObjects(t, hc, bktName, [][2]string{{objName, emptyVersion}}) deleteBucket(t, hc, bktName, http.StatusNoContent) } ```
👍 1
alexvanin force-pushed fix/obj-not-found from 1a00b843b5 to 6aa6679653 2023-06-01 12:31:28 +00:00 Compare
alexvanin force-pushed fix/obj-not-found from 6aa6679653 to 868edfdb31 2023-06-01 13:13:34 +00:00 Compare
dkirillov approved these changes 2023-06-01 14:23:25 +00:00
ironbee approved these changes 2023-06-02 13:00:49 +00:00
alexvanin merged commit 868edfdb31 into master 2023-06-02 13:04:30 +00:00
alexvanin deleted branch fix/obj-not-found 2023-06-02 13:04:32 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
dkirillov
ironbee
Labels
Clear labels   
P0

Highest   
P1

High   
P2

Medium   
P3

Low   
good first issue

Good for newcomers
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff   
blocked

The issue or PR is blocked by something   
bug

Something is not working   
config

Configuration format update or breaking change   
discussion

Talking about something in order to reach a decision or to exchange ideas   
documentation

Documentation related   
duplicate

This issue or pull request already exists   
enhancement

New feature   
go

Language features adoption   
help wanted

Extra attention is needed   
internal
   
invalid

Something is wrong   
kludge

This is a kludge and we know it   
observability

Related to metrics, tracing and logs   
perfomance

Perfomance related   
question

More information is needed   
refactoring

Refactoring   
wontfix

This won't be fixed
No Label
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
alexvanin
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-s3-gw#120
There is no content yet.
Delete Branch "alexvanin/frostfs-s3-gw:fix/obj-not-found"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?