morph: Fix batch size in TraverseIterator() #1000

Merged
fyrchik merged 2 commits from fyrchik/frostfs-node:fix-iterator into master 2024-02-23 06:24:42 +00:00
Owner

Initial prefetch size can be arbitrary an restricted only by VM/RPC
limits. For TraverseIterator() there is an explicit check on the
server-side, though.
Introduced in df055fead5.
Refs #931.

We can increase neo-go setting, but for public component we better use defaults.
00a1a8da56/pkg/services/rpcsrv/server.go (L2533C29-L2533C30)

Testing done:

$ frostfs-cli --rpc-endpoint 127.0.0.1:8080 -w /home/service/wallet_k6.json  container list | wc -l
Enter password >
19947
Initial prefetch size can be arbitrary an restricted only by VM/RPC limits. For TraverseIterator() there is an explicit check on the server-side, though. Introduced in df055fead56e6ac. Refs #931. We can increase neo-go setting, but for public component we better use defaults. https://github.com/nspcc-dev/neo-go/blob/00a1a8da56e1149ee9d563b96f3d8d09e112c3ac/pkg/services/rpcsrv/server.go#L2533C29-L2533C30 Testing done: ``` $ frostfs-cli --rpc-endpoint 127.0.0.1:8080 -w /home/service/wallet_k6.json container list | wc -l Enter password > 19947 ```
fyrchik requested review from storage-core-committers 2024-02-21 12:27:53 +00:00
fyrchik force-pushed fix-iterator from 41647b6ac5 to 175b625400 2024-02-21 12:27:57 +00:00 Compare
fyrchik requested review from storage-core-developers 2024-02-21 12:28:00 +00:00
fyrchik force-pushed fix-iterator from 175b625400 to 628803761a 2024-02-21 12:29:47 +00:00 Compare
fyrchik changed title from WIP: morph: Fix batch size in TraverseIterator() to morph: Fix batch size in TraverseIterator() 2024-02-21 13:15:22 +00:00
fyrchik added this to the v0.38.0 milestone 2024-02-21 13:24:29 +00:00
acid-ant approved these changes 2024-02-21 13:27:18 +00:00
Author
Owner
After https://github.com/nspcc-dev/neo-go/issues/3276 we might do better.
elebedeva reviewed 2024-02-21 14:50:11 +00:00
@ -260,8 +260,13 @@ func (c *Client) TestInvokeIterator(cb func(stackitem.Item) error, batchSize int
_ = c.rpcActor.TerminateSession(sid)
}()
// Batch size for TraverseIterator() it restricted on the neo-go size.
Member

shouldn't it be is restricted by?

shouldn't it be `is restricted by`?
Author
Owner

Thanks, it is actually restricted on the neo-go siDe

Thanks, it is actually `restricted on the neo-go siDe`
fyrchik force-pushed fix-iterator from 628803761a to 54caf1bde8 2024-02-21 20:55:46 +00:00 Compare
aarifullin approved these changes 2024-02-22 09:17:26 +00:00
acid-ant approved these changes 2024-02-22 09:25:01 +00:00
fyrchik merged commit abea258b65 into master 2024-02-23 06:24:42 +00:00
fyrchik deleted branch fix-iterator 2024-02-23 06:24:42 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1000
No description provided.