Policer drops linking object #825

Closed
opened 2023-11-22 15:29:19 +00:00 by dstepanov-yadro · 0 comments

Expected Behavior

Policer must not to drop linking objects.

Current Behavior

Policer drops linking objects.

Possible Solution

Fix policer logic as for put operation.

Steps to reproduce

  1. Create container with policy REP 2 CBF 2 SELECT 2 FROM * on dev-env

  2. Put large object to created container

  3. See logs:

2023-11-22T15:13:51.751Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db PUT"}

2023-11-22T15:13:51.908Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase PUT"}

2023-11-22T15:13:52.161Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "PUT", "type": "blobovnicza", "storage_id": "1/3/0"}

2023-11-22T15:13:52.186Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db DELETE"}

2023-11-22T15:13:52.191Z        info    policer/check.go:70     redundant local object copy detected    {"component": "Object Policer", "object": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG"}

2023-11-22T15:14:30.100Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "DELETE", "type": "blobovnicza", "storage_id": "1/3/0"}

2023-11-22T15:14:30.186Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase DELETE"}

Linking object was deleted by policer right after put.

Context

Put operation saves linking objects to every container node, but policer drops redundant copies.

Put logic here:

func needAdditionalBroadcast(obj *objectSDK.Object, localOnly bool) bool {

Policer logic here:

if typ == objectSDK.TypeLock || typ == objectSDK.TypeTombstone {

## Expected Behavior Policer must not to drop linking objects. ## Current Behavior Policer drops linking objects. ## Possible Solution Fix policer logic as for `put` operation. ## Steps to reproduce 1. Create container with policy `REP 2 CBF 2 SELECT 2 FROM *` on dev-env 2. Put large object to created container 3. See logs: ``` 2023-11-22T15:13:51.751Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db PUT"} 2023-11-22T15:13:51.908Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase PUT"} 2023-11-22T15:13:52.161Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "PUT", "type": "blobovnicza", "storage_id": "1/3/0"} 2023-11-22T15:13:52.186Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db DELETE"} 2023-11-22T15:13:52.191Z info policer/check.go:70 redundant local object copy detected {"component": "Object Policer", "object": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG"} 2023-11-22T15:14:30.100Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "DELETE", "type": "blobovnicza", "storage_id": "1/3/0"} 2023-11-22T15:14:30.186Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase DELETE"} ``` Linking object was deleted by policer right after put. ## Context `Put` operation saves linking objects to every container node, but policer drops redundant copies. `Put` logic here: https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/8d18fa159e085dde62336ec80f76fdfee358c412/pkg/services/object/put/common.go#L117 Policer logic here: https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/8d18fa159e085dde62336ec80f76fdfee358c412/pkg/services/policer/check.go#L96
dstepanov-yadro added the
bug
frostfs-node
good first issue
triage
labels 2023-11-22 15:29:19 +00:00
dstepanov-yadro self-assigned this 2023-12-11 08:42:42 +00:00
fyrchik added this to the v0.38.0 milestone 2023-12-22 07:27:25 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#825
No description provided.