Do not use session iterators when unnecessary #902

fyrchik · 2024-01-11T11:11:31Z

fyrchik commented

2024-01-11 11:11:31 +00:00

Testing done:
frostfs-adm morph dump-hashes on dev-env

Testing done: `frostfs-adm morph dump-hashes` on dev-env

fyrchik force-pushed morph-iter from e9acc604b8 to 2ed1f0afaf

2024-01-11 11:11:49 +00:00

Compare

fyrchik force-pushed morph-iter from 2ed1f0afaf to a0759bf1fd

2024-01-11 11:13:55 +00:00

Compare

fyrchik requested review from storage-core-committers 2024-01-11 11:50:31 +00:00

fyrchik requested review from storage-core-developers 2024-01-11 11:50:31 +00:00

fyrchik added this to the v0.38.0 milestone 2024-01-11 11:50:35 +00:00

dstepanov-yadro approved these changes 2024-01-11 14:02:44 +00:00

fyrchik reviewed 2024-01-11 14:12:40 +00:00

pkg/morph/client/client.go Outdated

					
				@ -207,0 +210,4 @@

				// defaultPrefetchBatchSize is the default number of items to prefetch.

				// It is dependent on VM limits (2048 items on stack), the default works for simple items.

				// For example, to iterate over 2 field structs, the limit should be divided by 3 = 1 (struct itself) + 2 (fields).

				const defaultPrefetchBatchSize = vm.MaxStackSize - 16

fyrchik commented

2024-01-11 14:12:40 +00:00

-16 is for bookkeeping, should be more than enough for "default" cases

`-16` is for bookkeeping, should be more than enough for "default" cases

aarifullin reviewed 2024-01-12 07:58:16 +00:00

pkg/morph/client/client.go Outdated

					
				@ -245,3 +277,3 @@

							}

						}

						items, err = c.rpcActor.TraverseIterator(sid, &r, 0)

						shouldStop = len(items) < batchSize

aarifullin commented

2024-01-12 07:58:16 +00:00

Do you need shouldStop flag at all?

for {
   items, err := c.rpcActor.TraverseIterator(sid, &r, batchSize)
   /*...*/

   if len(items) < batchSize {
      break 
   }
}

?

Do you need `shouldStop` flag at all? ```go for { items, err := c.rpcActor.TraverseIterator(sid, &r, batchSize) /*...*/ if len(items) < batchSize { break } } ``` ?

fyrchik commented

2024-01-12 08:05:03 +00:00

Fixed

aarifullin marked this conversation as resolved

aarifullin approved these changes 2024-01-12 07:58:37 +00:00

fyrchik force-pushed morph-iter from a0759bf1fd to f43bfb01a8

2024-01-12 08:05:22 +00:00

Compare

aarifullin approved these changes 2024-01-12 08:20:01 +00:00

acid-ant approved these changes 2024-01-12 10:16:20 +00:00

fyrchik commented

2024-01-12 10:25:41 +00:00