Policer drops linking object #825

New issue

Closed

opened 2023-11-22 15:29:19 +00:00 by dstepanov-yadro · 0 comments

dstepanov-yadro commented

2023-11-22 15:29:19 +00:00

Member

Expected Behavior

Policer must not to drop linking objects.

Current Behavior

Policer drops linking objects.

Possible Solution

Fix policer logic as for put operation.

Steps to reproduce

Create container with policy REP 2 CBF 2 SELECT 2 FROM * on dev-env
Put large object to created container
See logs:

2023-11-22T15:13:51.751Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db PUT"}

2023-11-22T15:13:51.908Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase PUT"}

2023-11-22T15:13:52.161Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "PUT", "type": "blobovnicza", "storage_id": "1/3/0"}

2023-11-22T15:13:52.186Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db DELETE"}

2023-11-22T15:13:52.191Z        info    policer/check.go:70     redundant local object copy detected    {"component": "Object Policer", "object": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG"}

2023-11-22T15:14:30.100Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "DELETE", "type": "blobovnicza", "storage_id": "1/3/0"}

2023-11-22T15:14:30.186Z        info    log/log.go:13   local object storage operation  {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase DELETE"}

Linking object was deleted by policer right after put.

Context

Put operation saves linking objects to every container node, but policer drops redundant copies.

Put logic here:

func needAdditionalBroadcast(obj *objectSDK.Object, localOnly bool) bool {

Policer logic here:

if typ == objectSDK.TypeLock || typ == objectSDK.TypeTombstone {

## Expected Behavior Policer must not to drop linking objects. ## Current Behavior Policer drops linking objects. ## Possible Solution Fix policer logic as for `put` operation. ## Steps to reproduce 1. Create container with policy `REP 2 CBF 2 SELECT 2 FROM *` on dev-env 2. Put large object to created container 3. See logs: ``` 2023-11-22T15:13:51.751Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db PUT"} 2023-11-22T15:13:51.908Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase PUT"} 2023-11-22T15:13:52.161Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "PUT", "type": "blobovnicza", "storage_id": "1/3/0"} 2023-11-22T15:13:52.186Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "type": "write-cache", "op": "db DELETE"} 2023-11-22T15:13:52.191Z info policer/check.go:70 redundant local object copy detected {"component": "Object Policer", "object": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG"} 2023-11-22T15:14:30.100Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "DELETE", "type": "blobovnicza", "storage_id": "1/3/0"} 2023-11-22T15:14:30.186Z info log/log.go:13 local object storage operation {"shard_id": "E9J55p45uaUrQTdiu5dw2T", "address": "C3MtG7enw2ixgGj5KokWRkRs5vXVXzjzWMFuKNci8viK/AQmKuh1KpFNQKQTMWn1y77xzzBtgeUxc7MgYRXY2yWjG", "op": "metabase DELETE"} ``` Linking object was deleted by policer right after put. ## Context `Put` operation saves linking objects to every container node, but policer drops redundant copies. `Put` logic here: https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/8d18fa159e085dde62336ec80f76fdfee358c412/pkg/services/object/put/common.go#L117 Policer logic here: https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/8d18fa159e085dde62336ec80f76fdfee358c412/pkg/services/policer/check.go#L96