Add payload field to `Init` message in `object.PUT` RPC #9

New issue

Closed

opened 2023-02-20 13:01:52 +00:00 by fyrchik · 4 comments

fyrchik commented

2023-02-20 13:01:52 +00:00

(Migrated from github.com)

For small objects we could send payload + header in one message.

Also, think about one-shot (non-stream) RPC for vNext.

For small objects we could send payload + header in one message. Also, think about one-shot (non-stream) RPC for vNext.

realloc added this to the v2.15 - Karpinsky Ice Cap milestone 2023-03-23 13:25:38 +00:00

fyrchik commented

2023-06-25 10:42:20 +00:00

Owner

I don't think possible performance gains justify complicating the current streaming interface.

How about having a separate RPC for putting prepared objects?

The compatibility will be preserved, but in some cases (replication, splitting objects on a client) we might want to just send a single message. If we extend the current interface, feels like half-measure. With a separate one we can read and object and "just replicate". This also plays nicely with recent SDK improvements: forming and object and then splitting it again to send feels like a kludge.

cc @alexvanin @realloc

I don't think possible performance gains justify complicating the current streaming interface. How about having a separate RPC for putting prepared objects? The compatibility will be preserved, but in some cases (replication, splitting objects on a client) we might want to just send a single message. If we extend the current interface, feels like half-measure. With a separate one we can read and object and "just replicate". This also plays nicely with recent SDK improvements: forming and object and then splitting it again to send feels like a kludge. cc @alexvanin @realloc

fyrchik referenced this issue from TrueCloudLab/frostfs-node

2023-06-25 10:54:18 +00:00

Pre-allocate memory in the replicator #463

alexvanin commented

2023-06-27 10:29:04 +00:00

Owner

The compatibility will be preserved, but in some cases (replication, splitting objects on a client) we might want to just send a single message.

Single message is not a single object. If API introduces new unary RPC request, then it is going to be suitable only for objects with the size of 3-4 MiB to fit in the gRPC message.

While FrostFS has plenty of such small objects (tombstones, secret boxes, small user objects in general), client side should decide which RPC to use based on the object size. I am okay with that as soon as it is hidden in the client implementation, but separate RPC might be a bit confusing for SDK developers.

> The compatibility will be preserved, but in some cases (replication, splitting objects on a client) we might want to just send a single message. Single message is not a single object. If API introduces new unary RPC request, then it is going to be suitable only for objects with the size of 3-4 MiB to fit in the gRPC message. While FrostFS has plenty of such small objects (tombstones, secret boxes, small user objects in general), client side should decide which RPC to use based on the object size. I am okay with that as soon as it is hidden in the client implementation, but separate RPC might be a bit confusing for SDK developers.

realloc added the

discussion

labels 2023-06-28 08:46:17 +00:00