pilorama: Speedup TestForest_ApplyRandom() #615

Merged

fyrchik merged 1 commit from fyrchik/frostfs-node:fix-pilorama-tests into master

2023-08-16 19:09:10 +00:00

fyrchik commented

2023-08-16 09:47:20 +00:00

Owner

Some of our pilorama tests fail on CI.
The reasons are not obvious, but one possible improvement
is using WithNoSync option for these. It should have much effect,
because we are writing on the tmpfs, but doesn't hurt anyway.

If I replace t.TempDir() with a local directory, test execution time
goes down from 5s (sync) to 0.4s (nosync), which is the same time as
with t.TempDir(). Maybe we have some strange CI configuration.

panic: test timed out after 10m0s
running tests:
	TestForest_ApplyRandom (8m22s)
	TestForest_ApplyRandom/bbolt (8m21s)
...
goroutine 170 [syscall]:
syscall.Syscall(0xc000100000?, 0xc00047b758?, 0x6aff9a?, 0xc00041c1b0?)
	/opt/hostedtoolcache/go/1.20.7/x64/src/syscall/syscall_linux.go:69 +0x27
syscall.Fdatasync(0x9e35c0?)
	/opt/hostedtoolcache/go/1.20.7/x64/src/syscall/zsyscall_linux_amd64.go:418 +0x2a
go.etcd.io/bbolt.fdatasync(0xc000189000?)

Signed-off-by: Evgenii Stratonikov e.stratonikov@yadro.com

Some of our pilorama tests fail on CI. The reasons are not obvious, but one possible improvement is using `WithNoSync` option for these. It should have much effect, because we are writing on the tmpfs, but doesn't hurt anyway. If I replace `t.TempDir()` with a local directory, test execution time goes down from 5s (sync) to 0.4s (nosync), which is the same time as with `t.TempDir()`. Maybe we have some strange CI configuration. ``` panic: test timed out after 10m0s running tests: TestForest_ApplyRandom (8m22s) TestForest_ApplyRandom/bbolt (8m21s) ... goroutine 170 [syscall]: syscall.Syscall(0xc000100000?, 0xc00047b758?, 0x6aff9a?, 0xc00041c1b0?) /opt/hostedtoolcache/go/1.20.7/x64/src/syscall/syscall_linux.go:69 +0x27 syscall.Fdatasync(0x9e35c0?) /opt/hostedtoolcache/go/1.20.7/x64/src/syscall/zsyscall_linux_amd64.go:418 +0x2a go.etcd.io/bbolt.fdatasync(0xc000189000?) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>

fyrchik force-pushed fix-pilorama-tests from 106a735f62 to fb6c3f480f

2023-08-16 09:47:48 +00:00

Compare

fyrchik requested review from storage-core-committers 2023-08-16 09:50:41 +00:00

dstepanov-yadro approved these changes 2023-08-16 10:04:37 +00:00

fyrchik commented

2023-08-16 10:06:14 +00:00

Author

Owner

It seems to work:
Compare this run:

ok  	git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama	161.701s

With e.g.
https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/904/jobs/1

ok  	git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama	370.966s

It seems to work: Compare this run: ``` ok git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama 161.701s ``` With e.g. https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/904/jobs/1 ``` ok git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama 370.966s ```