commonclient: Allow to partially unwrap session iterator #64

fyrchik · 2023-12-18T13:16:26Z

fyrchik commented

2023-12-18 13:16:26 +00:00

Neo-go sessions is a limited resource, but we must use them because the number of items can be unbounded.
With this little helper we can avoid using sessions when the number of items in the iterator is low.

Neo-go has a script for limited iterator expansion, so another option would be to call it and fallback on using sessions.
With this method the unified interface is possible.

RPC usage is covered by the frostfsid-client tests.

Neo-go sessions is a limited resource, but we must use them because the number of items can be unbounded. With this little helper we can avoid using sessions when the number of items in the iterator is low. Neo-go has a script for limited iterator expansion, so another option would be to call it and fallback on using sessions. With this method the unified interface is possible. RPC usage is covered by the frostfsid-client tests.

fyrchik force-pushed partial-iterator from aa2e5c6462 to 32a1338529

2023-12-19 13:54:05 +00:00

Compare

fyrchik changed title from ~~WIP: commonclient: Allow to partially unwrap the iterator~~ to commonclient: Allow to partially unwrap the iterator

2023-12-19 13:54:35 +00:00

fyrchik requested review from storage-core-committers 2023-12-19 13:54:58 +00:00

fyrchik requested review from storage-core-developers 2023-12-19 13:54:59 +00:00

fyrchik requested review from storage-services-committers 2023-12-19 13:54:59 +00:00

fyrchik requested review from storage-services-developers 2023-12-19 13:55:00 +00:00

fyrchik force-pushed partial-iterator from 32a1338529 to 53a32e0cb4

2023-12-19 13:55:43 +00:00

Compare

fyrchik commented

2023-12-19 14:37:14 +00:00

Poster

For upstreaming this part see https://github.com/nspcc-dev/neo-go/issues/3272

fyrchik force-pushed partial-iterator from 53a32e0cb4 to cd37475476

2023-12-19 14:37:36 +00:00

Compare

fyrchik changed title from ~~commonclient: Allow to partially unwrap the iterator~~ to commonclient: Allow to partially unwrap session iterator

2023-12-19 14:37:36 +00:00

fyrchik force-pushed partial-iterator from cd37475476 to ff6499b322

2023-12-19 14:38:27 +00:00

Compare

fyrchik reviewed 2023-12-19 15:01:14 +00:00

commonclient/iterator.go Outdated

													
				@ -18,0 +28,4 @@

						return nil, fmt.Errorf("couldn't create unwrap script: %w", err)

					}

					iter, shouldStop, err := UnwrapPartialIterator(inv.Run(script))

fyrchik commented

2023-12-19 15:01:14 +00:00

Poster

You may ask why is it PartialUnwrap above and UnwrapPartial here.
Please, don't.

You may ask why is it `PartialUnwrap` above and `UnwrapPartial` here. Please, don't.

😆 3

fyrchik commented

2023-12-19 17:53:19 +00:00

Poster

There are nice suggestions in https://github.com/nspcc-dev/neo-go/issues/3272#issuecomment-1863026141, will fix.

fyrchik marked this conversation as resolved

fyrchik requested review from vdomnich-yadro 2023-12-19 15:01:57 +00:00

dstepanov-yadro approved these changes 2023-12-20 11:37:20 +00:00

acid-ant approved these changes 2023-12-21 07:10:24 +00:00

vdomnich-yadro approved these changes 2023-12-21 14:44:40 +00:00

commonclient/iterator.go Outdated

													
				@ -36,0 +84,4 @@

					}

					arr, ok := itm.Value().([]stackitem.Item)

					if !ok {

						return Iterator{}, false, errors.New("the item has Array type, but not is not an array")