In #2134 and dfd0f4c5a4 some testing changes got committed by accident which caused this regression. This patch reverts it to how it was before.
dfd0f4c5a4