commonclient: Allow to partially unwrap session iterator #64

Merged
fyrchik merged 2 commits from fyrchik/frostfs-contract:partial-iterator into master 2023-12-29 14:06:47 +00:00
Owner

Neo-go sessions is a limited resource, but we must use them because the number of items can be unbounded.
With this little helper we can avoid using sessions when the number of items in the iterator is low.

Neo-go has a script for limited iterator expansion, so another option would be to call it and fallback on using sessions.
With this method the unified interface is possible.

RPC usage is covered by the frostfsid-client tests.

Neo-go sessions is a limited resource, but we must use them because the number of items can be unbounded. With this little helper we can avoid using sessions when the number of items in the iterator is low. Neo-go has a script for limited iterator expansion, so another option would be to call it and fallback on using sessions. With this method the unified interface is possible. RPC usage is covered by the frostfsid-client tests.
fyrchik force-pushed partial-iterator from aa2e5c6462 to 32a1338529 2023-12-19 13:54:05 +00:00 Compare
fyrchik changed title from WIP: commonclient: Allow to partially unwrap the iterator to commonclient: Allow to partially unwrap the iterator 2023-12-19 13:54:35 +00:00
fyrchik requested review from storage-core-committers 2023-12-19 13:54:58 +00:00
fyrchik requested review from storage-core-developers 2023-12-19 13:54:59 +00:00
fyrchik requested review from storage-services-committers 2023-12-19 13:54:59 +00:00
fyrchik requested review from storage-services-developers 2023-12-19 13:55:00 +00:00
fyrchik force-pushed partial-iterator from 32a1338529 to 53a32e0cb4 2023-12-19 13:55:43 +00:00 Compare
Author
Owner

For upstreaming this part see https://github.com/nspcc-dev/neo-go/issues/3272

For upstreaming this part see https://github.com/nspcc-dev/neo-go/issues/3272
fyrchik force-pushed partial-iterator from 53a32e0cb4 to cd37475476 2023-12-19 14:37:36 +00:00 Compare
fyrchik changed title from commonclient: Allow to partially unwrap the iterator to commonclient: Allow to partially unwrap session iterator 2023-12-19 14:37:36 +00:00
fyrchik force-pushed partial-iterator from cd37475476 to ff6499b322 2023-12-19 14:38:27 +00:00 Compare
fyrchik reviewed 2023-12-19 15:01:14 +00:00
@ -18,0 +28,4 @@
return nil, fmt.Errorf("couldn't create unwrap script: %w", err)
}
iter, shouldStop, err := UnwrapPartialIterator(inv.Run(script))
Author
Owner

You may ask why is it PartialUnwrap above and UnwrapPartial here.
Please, don't.

You may ask why is it `PartialUnwrap` above and `UnwrapPartial` here. Please, don't.
Author
Owner
There are nice suggestions in https://github.com/nspcc-dev/neo-go/issues/3272#issuecomment-1863026141, will fix.
fyrchik marked this conversation as resolved
fyrchik requested review from vdomnich-yadro 2023-12-19 15:01:57 +00:00
dstepanov-yadro approved these changes 2023-12-20 11:37:20 +00:00
acid-ant approved these changes 2023-12-21 07:10:24 +00:00
vdomnich-yadro approved these changes 2023-12-21 14:44:40 +00:00
@ -36,0 +84,4 @@
}
arr, ok := itm.Value().([]stackitem.Item)
if !ok {
return Iterator{}, false, errors.New("the item has Array type, but not is not an array")
Member

, but not is not an array -> , but is not an array

`, but not is not an array` -> `, but is not an array`
fyrchik force-pushed partial-iterator from ff6499b322 to 3fa5691218 2023-12-29 12:03:39 +00:00 Compare
fyrchik force-pushed partial-iterator from 3fa5691218 to 8750583005 2023-12-29 12:03:51 +00:00 Compare
Author
Owner

I have updated neo-go and reused everything from https://github.com/nspcc-dev/neo-go/pull/3274

I have updated neo-go and reused everything from https://github.com/nspcc-dev/neo-go/pull/3274
fyrchik force-pushed partial-iterator from 8750583005 to 13fb6bab93 2023-12-29 12:06:18 +00:00 Compare
dstepanov-yadro approved these changes 2023-12-29 12:53:38 +00:00
fyrchik force-pushed partial-iterator from 13fb6bab93 to 5724a7663e 2023-12-29 14:04:48 +00:00 Compare
fyrchik merged commit 86e6d4d334 into master 2023-12-29 14:06:47 +00:00
fyrchik deleted branch partial-iterator 2023-12-29 14:06:48 +00:00
fyrchik referenced this pull request from a commit 2023-12-29 14:06:48 +00:00
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-core-developers
TrueCloudLab/storage-services-committers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-contract#64
No description provided.