morph: Fix batch size in TraverseIterator() #1000

fyrchik · 2024-02-21T12:27:15Z

fyrchik commented

2024-02-21 12:27:15 +00:00

Initial prefetch size can be arbitrary an restricted only by VM/RPC
limits. For TraverseIterator() there is an explicit check on the
server-side, though.
Introduced in df055fead5.
Refs #931.

We can increase neo-go setting, but for public component we better use defaults.
00a1a8da56/pkg/services/rpcsrv/server.go (L2533C29-L2533C30)

Testing done:

$ frostfs-cli --rpc-endpoint 127.0.0.1:8080 -w /home/service/wallet_k6.json  container list | wc -l
Enter password >
19947

Initial prefetch size can be arbitrary an restricted only by VM/RPC limits. For TraverseIterator() there is an explicit check on the server-side, though. Introduced in df055fead56e6ac. Refs #931. We can increase neo-go setting, but for public component we better use defaults. https://github.com/nspcc-dev/neo-go/blob/00a1a8da56e1149ee9d563b96f3d8d09e112c3ac/pkg/services/rpcsrv/server.go#L2533C29-L2533C30 Testing done: ``` $ frostfs-cli --rpc-endpoint 127.0.0.1:8080 -w /home/service/wallet_k6.json container list | wc -l Enter password > 19947 ```

fyrchik requested review from storage-core-committers 2024-02-21 12:27:53 +00:00

fyrchik force-pushed fix-iterator from 41647b6ac5 to 175b625400

2024-02-21 12:27:57 +00:00

Compare

fyrchik requested review from storage-core-developers 2024-02-21 12:28:00 +00:00

fyrchik force-pushed fix-iterator from 175b625400 to 628803761a

2024-02-21 12:29:47 +00:00

Compare

fyrchik changed title from ~~WIP: morph: Fix batch size in TraverseIterator()~~ to morph: Fix batch size in TraverseIterator()

2024-02-21 13:15:22 +00:00

fyrchik added this to the v0.38.0 milestone 2024-02-21 13:24:29 +00:00

acid-ant approved these changes 2024-02-21 13:27:18 +00:00

fyrchik commented

2024-02-21 13:28:07 +00:00

Poster

After https://github.com/nspcc-dev/neo-go/issues/3276 we might do better.

elebedeva reviewed 2024-02-21 14:50:11 +00:00

pkg/morph/client/client.go Outdated

													
				@ -260,8 +260,13 @@ func (c *Client) TestInvokeIterator(cb func(stackitem.Item) error, batchSize int

						_ = c.rpcActor.TerminateSession(sid)

					}()

					// Batch size for TraverseIterator() it restricted on the neo-go size.