Big helper objects #9

New issue

Closed

opened 2023-01-09 18:44:32 +00:00 by carpawell · 4 comments

carpawell commented

2023-01-09 18:44:32 +00:00

(Migrated from github.com)

2023-01-09T17:53:09.260Z        debug   delete/delete.go:52     operation finished with error   {"component": "Object.Delete service", "request": "DELETE", "address": "GcmLhnye9oMfbQbd4cYVFDKmkvY3Kmr5rtP5a1jUrQKV/Avi7VKGpx5sH3af9oV21bDdHjA1xXWhaH5zXsAR7qvCF", "local": false, "with session": true, "with bearer": false, "error": "(*putsvc.Streamer) could not write payload chunk to target: could not release object: could not close target: could not close next target: (*putsvc.distributedTarget) could not validate payload content: (*object.FormatValidator) could not unmarshal tombstone content: proto: cannot parse invalid wire-format data"}

Set really low small object limit;
Put 96KB (rand num that >> small obj limit) obj;
Try to remove it.

I think that is because TS becomes big and now validation is broken after helper objects broadcast optimization. But that must be rechecked.

``` 2023-01-09T17:53:09.260Z debug delete/delete.go:52 operation finished with error {"component": "Object.Delete service", "request": "DELETE", "address": "GcmLhnye9oMfbQbd4cYVFDKmkvY3Kmr5rtP5a1jUrQKV/Avi7VKGpx5sH3af9oV21bDdHjA1xXWhaH5zXsAR7qvCF", "local": false, "with session": true, "with bearer": false, "error": "(*putsvc.Streamer) could not write payload chunk to target: could not release object: could not close target: could not close next target: (*putsvc.distributedTarget) could not validate payload content: (*object.FormatValidator) could not unmarshal tombstone content: proto: cannot parse invalid wire-format data"} ``` 0. Set __really__ low small object limit; 1. Put 96KB (rand num that `>>` small obj limit) obj; 2. Try to remove it. I think that is because TS becomes _big_ and now validation is broken after helper objects [broadcast optimization](https://github.com/nspcc-dev/neofs-node/pull/2050/commits/be23744402009984bdb14e8eff3dc0f2a214a779). But that must be rechecked.

fyrchik commented

2023-02-10 15:50:37 +00:00

(Migrated from github.com)

I think it is expected -- basically we have some limit on an object size.
Last time I checked it was quite big: object ID is 32-byte in size, so if MaxObjectSize is 1MiB, we still can store ~ max object size * max object count in tombstone == 1 MiB * (1 MiB / 32) == 2^40 / 2^5 = 2^35 = 32 GiB.

I think we should at least extend the documentation here.

I think it is expected -- basically we have some limit on an object size. Last time I checked it was quite big: object ID is 32-byte in size, so if `MaxObjectSize` is 1MiB, we still can store ~ `max object size * max object count in tombstone == 1 MiB * (1 MiB / 32) == 2^40 / 2^5 = 2^35 = 32 GiB`. I think we should at least extend the documentation here.