No description
Kirill Sosnovskikh
d050a94272
- Added ability to pass any parameters to `params` keyword in `pytest.fixture`, except:
1) `params=<callable object>`, but `params=[<callable object>, ...]` is allowed
2) `params=[*<any collection>]`
- Analysis of `pytest_generate_tests` is almost equivalent to regular fixtures, except:
1) `metafunc.parametrize(*<any collection>, **<any dict>)`
2) `metafunc.parametrize("any", <callable object>)`
3) `metafunc.parametrize("any1, any2", [(<callable object>, [*<any collection>])]`
Signed-off-by: Kirill Sosnovskikh <k.sosnovskikh@yadro.com>
|
||
|---|---|---|
| src/allure_validator | ||
| tests | ||
| .gitignore | ||
| .pre-commit-config.yaml | ||
| .pre-commit-hooks.yaml | ||
| CODEOWNERS | ||
| LICENSE | ||
| Makefile | ||
| pyproject.toml | ||
| pytest.ini | ||
| README.md | ||
| requirements.txt | ||
| requirements_dev.txt | ||
allure-validator
Linter for checking uniqueness and correctness of allure.title in the entire test database.
Installation
pip install git+https://git.frostfs.info/TrueCloudLab/allure-validator.git@master
Usage
To start, simply specify the folder with the test database that needs to be checked:
allure-validator <folder> [OPTIONS]
Additionally, allure-validator can be added to pre-commit hooks:
# .pre-commit-config.yaml
repos:
- repo: https://git.frostfs.info/TrueCloudLab/allure-validator
rev: latest
hooks:
- id: allure-validator
args: ["pytest_tests/"]
pass_filenames: false
Ignore mechanism
The linter can be made to ignore functions/classes. To do this, just write a comment # noqa: allure-validator to the right of the function/class name. For example:
@pytest.mark.parametrize("test_param", [1, 2])
def test_ignored_function(test_param): # noqa: allure-validator
...
@pytest.mark.parametrize("class_param", [1, 2])
class TestIngoredClass: # noqa: allure-validator
def test_1(self, class_param):
...
def test_2(self, class_param):
...
Note: this may also miss fixtures, so use with caution.
Options
The linter supports the following flags:
--plugins- regex patterns that plugins must match (paths are compiled according to the template:plugin_name / src / plugin_name).--files- paths to files with fixtures.
Configuration options can also be provided using special command line arguments, for example:
allure-validator <tests folder> --plugins <plugin regex>, ... --files <path to file with fixtures>, ...
Work example
Let's run the linter for some sample tests:
NOT UNIQUE TITLE: Not unique test
In the following tests:
tests/test_validator.py:20:0: title is not unique by `test_not_unique_2`
tests/test_validator.py:119:4: title is not unique by `test_not_unique_3`
tests/test_validator.py:15:0: title is not unique by `test_not_unique_1`
NOT UNIQUE TITLE: Test title
In the following tests:
tests/test_validator.py:30:0: title is not unique by `test_title_not_unique_2`
tests/test_validator.py:25:0: title is not unique by `test_title_not_unique_1`
tests/test_validator.py:5:0: EMPTY TITLE: Missing title in `test_empty_title`
tests/test_validator.py:10:0: EMPTY TITLE: Missing title in `test_empty_title_2`
tests/test_validator.py:20:0: MISSING PARAMS: Parameters are missing from title: object_size
tests/test_validator.py:62:0: MISSING PARAMS: Parameters are missing from title: param_two OR param_three
tests/test_validator.py:115:4: MISSING PARAMS: Parameters are missing from title: root_fixture
tests/test_validator.py:119:4: MISSING PARAMS: Parameters are missing from title: param_class_1
Undetectable cases
Since the linter works with AST python code, any dynamic titles will not be found. For example:
import allure
def test_example():
data = 10
allure.dynamic.title(f"Dynamic title {data}") # Undetected title
...
Dynamic fixtures are only supported if they are declared in pytest_generate_tests:
def pytest_generate_tests(metafunc):
...
metafunc.parametrize("fixture1, fixture2", [(1, 2), (3, 4)]) # OK
...
if some_condition:
metafunc.parametrize("fixture3", *some_value_generator) # OK