TrueCloudLab
/
frostfs-s3-gw
16
2
Fork 14
Code Issues 19 Pull Requests 4 Actions Packages Releases 6 Activity

Flaky test TestErrorTimeoutChecking #290

New Issue
Closed
opened 2024-01-17 07:22:28 +00:00 by dkirillov · 0 comments
dkirillov commented 2024-01-17 07:22:28 +00:00
Collaborator

Test TestErrorTimeoutChecking fails sometimes

Expected Behavior

Test is stable

Current Behavior

Test fails sometimes:

2024-01-17 01:01:30  FAIL
2024-01-17 01:01:30      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/handler    coverage: 57.3% of statements
2024-01-17 01:01:30  FAIL    git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/handler    8.366s
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/layer    0.426s    coverage: 40.1% of statements
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/layer/encryption    0.008s    coverage: 43.2% of statements
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/authmate    0.010s    coverage: 8.6% of statements
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/creds/accessbox    0.019s    coverage: 48.8% of statements
2024-01-17 01:01:30  --- FAIL: TestErrorTimeoutChecking (0.16s)
2024-01-17 01:01:30      --- FAIL: TestErrorTimeoutChecking/deadline_exceeded (0.16s)
2024-01-17 01:01:30  panic: runtime error: invalid memory address or nil pointer dereference [recovered]
2024-01-17 01:01:30      panic: runtime error: invalid memory address or nil pointer dereference
2024-01-17 01:01:30  [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x9ef1a2]
2024-01-17 01:01:30  
2024-01-17 01:01:30  goroutine 70 [running]:
2024-01-17 01:01:30  testing.tRunner.func1.2({0xcbd0e0, 0x16ed0a0})
2024-01-17 01:01:30      /usr/local/go/src/testing/testing.go:1526 +0x24e
2024-01-17 01:01:30  testing.tRunner.func1()
2024-01-17 01:01:30      /usr/local/go/src/testing/testing.go:1529 +0x39f
2024-01-17 01:01:30  panic({0xcbd0e0, 0x16ed0a0})
2024-01-17 01:01:30      /usr/local/go/src/runtime/panic.go:884 +0x213
2024-01-17 01:01:30  git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs/errors.IsTimeoutError({0x0, 0x0})
2024-01-17 01:01:30      /home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw/internal/frostfs/errors/errors.go:34 +0x22
2024-01-17 01:01:30  git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs.TestErrorTimeoutChecking.func2(0xc000483380?)
2024-01-17 01:01:30      /home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw/internal/frostfs/frostfs_test.go:43 +0x7a
2024-01-17 01:01:30  testing.tRunner(0xc0004836c0, 0xed80b8)
2024-01-17 01:01:30      /usr/local/go/src/testing/testing.go:1576 +0x10b
2024-01-17 01:01:30  created by testing.(*T).Run
2024-01-17 01:01:30      /usr/local/go/src/testing/testing.go:1629 +0x3ea
2024-01-17 01:01:30  FAIL    git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs    0.189s
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs/crdt    0.011s    coverage: 61.5% of statements
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/xml    0.022s    coverage: 70.0% of statements
2024-01-17 01:01:30  ok      git.frostfs.info/TrueCloudLab/frostfs-s3-gw/pkg/service/tree    0.189s    coverage: 28.3% of statements
2024-01-17 01:01:30  FAIL
2024-01-17 01:01:30  make[2]: *** [Makefile:72: test] Error 1
2024-01-17 01:01:30  make[2]: Leaving directory '/home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw'
2024-01-17 01:01:30  dh_auto_test: error: make -j1 test returned exit code 2
2024-01-17 01:01:30  make[1]: *** [debian/rules:8: build] Error 25
2024-01-17 01:01:30  make[1]: Leaving directory '/home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw'
2024-01-17 01:01:30  dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
2024-01-17 01:01:30  make: *** [Makefile:165: debpackage] Error 2

Steps to Reproduce (for bugs)

Run several times

Context

Couldn't build binary without passing tests in CI.

Regression

Probably no

Your Environment

Test [TestErrorTimeoutChecking](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/src/commit/c32220762f7c30d8917763088d3f04e3e35c7464/internal/frostfs/frostfs_test.go#L33) fails sometimes ## Expected Behavior Test is stable ## Current Behavior Test fails sometimes: ``` 2024-01-17 01:01:30 FAIL 2024-01-17 01:01:30 git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/handler coverage: 57.3% of statements 2024-01-17 01:01:30 FAIL git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/handler 8.366s 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/layer 0.426s coverage: 40.1% of statements 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/api/layer/encryption 0.008s coverage: 43.2% of statements 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/authmate 0.010s coverage: 8.6% of statements 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/creds/accessbox 0.019s coverage: 48.8% of statements 2024-01-17 01:01:30 --- FAIL: TestErrorTimeoutChecking (0.16s) 2024-01-17 01:01:30 --- FAIL: TestErrorTimeoutChecking/deadline_exceeded (0.16s) 2024-01-17 01:01:30 panic: runtime error: invalid memory address or nil pointer dereference [recovered] 2024-01-17 01:01:30 panic: runtime error: invalid memory address or nil pointer dereference 2024-01-17 01:01:30 [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x9ef1a2] 2024-01-17 01:01:30 2024-01-17 01:01:30 goroutine 70 [running]: 2024-01-17 01:01:30 testing.tRunner.func1.2({0xcbd0e0, 0x16ed0a0}) 2024-01-17 01:01:30 /usr/local/go/src/testing/testing.go:1526 +0x24e 2024-01-17 01:01:30 testing.tRunner.func1() 2024-01-17 01:01:30 /usr/local/go/src/testing/testing.go:1529 +0x39f 2024-01-17 01:01:30 panic({0xcbd0e0, 0x16ed0a0}) 2024-01-17 01:01:30 /usr/local/go/src/runtime/panic.go:884 +0x213 2024-01-17 01:01:30 git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs/errors.IsTimeoutError({0x0, 0x0}) 2024-01-17 01:01:30 /home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw/internal/frostfs/errors/errors.go:34 +0x22 2024-01-17 01:01:30 git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs.TestErrorTimeoutChecking.func2(0xc000483380?) 2024-01-17 01:01:30 /home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw/internal/frostfs/frostfs_test.go:43 +0x7a 2024-01-17 01:01:30 testing.tRunner(0xc0004836c0, 0xed80b8) 2024-01-17 01:01:30 /usr/local/go/src/testing/testing.go:1576 +0x10b 2024-01-17 01:01:30 created by testing.(*T).Run 2024-01-17 01:01:30 /usr/local/go/src/testing/testing.go:1629 +0x3ea 2024-01-17 01:01:30 FAIL git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs 0.189s 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/frostfs/crdt 0.011s coverage: 61.5% of statements 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/internal/xml 0.022s coverage: 70.0% of statements 2024-01-17 01:01:30 ok git.frostfs.info/TrueCloudLab/frostfs-s3-gw/pkg/service/tree 0.189s coverage: 28.3% of statements 2024-01-17 01:01:30 FAIL 2024-01-17 01:01:30 make[2]: *** [Makefile:72: test] Error 1 2024-01-17 01:01:30 make[2]: Leaving directory '/home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw' 2024-01-17 01:01:30 dh_auto_test: error: make -j1 test returned exit code 2 2024-01-17 01:01:30 make[1]: *** [debian/rules:8: build] Error 25 2024-01-17 01:01:30 make[1]: Leaving directory '/home/ci/jenkins_home/workspace/release_v1.3_Bundle_frostfs_build_NB/frostfs-s3-gw' 2024-01-17 01:01:30 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 2024-01-17 01:01:30 make: *** [Makefile:165: debpackage] Error 2 ``` ## Steps to Reproduce (for bugs) Run several times ## Context Couldn't build binary without passing tests in CI. ## Regression Probably no ## Your Environment <!--- Include as many relevant details about the environment you experienced the bug in --> * Version used: [master](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/commit/c32220762f7c30d8917763088d3f04e3e35c7464)
dkirillov added the
bug
 label 2024-01-17 07:22:28 +00:00
alexvanin added this to the v0.29.0 milestone 2024-01-17 13:54:14 +00:00
dkirillov self-assigned this 2024-02-14 11:17:48 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
master
support/v0.29
support/v0.28
support/v0.27
v0.30.0-rc.4
v0.30.0-rc.3
v0.30.0-rc.2
v0.29.1
v0.30.0-rc.1
v0.29.0
v0.28.2
v0.29.0-rc.11
v0.29.0-rc.10
v0.29.0-rc.9
v0.29.0-rc.8
v0.29.0-rc.7
v0.29.0-rc.6
v0.29.0-rc.5
v0.29.0-rc.4
v0.29.0-rc.3
v0.29.0-rc.2
v0.29.0-rc.1
v0.28.1
v0.28.0
v0.28.0-rc.1
v0.27.0
v0.27.0-rc.2
v0.27.0-rc.1
Labels
Clear labels   
P0

Highest   
P1

High   
P2

Medium   
P3

Low   
good first issue

Good for newcomers
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff   
blocked

The issue or PR is blocked by something   
bug

Something is not working   
config

Configuration format update or breaking change   
discussion

Talking about something in order to reach a decision or to exchange ideas   
documentation

Documentation related   
duplicate

This issue or pull request already exists   
enhancement

New feature   
go

Language features adoption   
help wanted

Extra attention is needed   
internal
   
invalid

Something is wrong   
kludge

This is a kludge and we know it   
observability

Related to metrics, tracing and logs   
perfomance

Perfomance related   
question

More information is needed   
refactoring

Refactoring   
wontfix

This won't be fixed
No Label
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No Milestone
v0.29.0
Assignees
Clear assignees
No Assignees
dkirillov
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-s3-gw#290
There is no content yet.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?