Kirill Sosnovskikh
cd41eb5d03
The following flags have been added: - `--plugins` - patterns that plugins must match. - `--files` - paths to files with fixtures. Signed-off-by: Kirill Sosnovskikh <k.sosnovskikh@yadro.com> |
||
---|---|---|
src/allure_validator | ||
tests | ||
.gitignore | ||
.pre-commit-config.yaml | ||
.pre-commit-hooks.yaml | ||
CODEOWNERS | ||
LICENSE | ||
Makefile | ||
pyproject.toml | ||
pytest.ini | ||
README.md | ||
requirements.txt | ||
requirements_dev.txt |
allure-validator
Linter for checking uniqueness and correctness of allure.title
in the entire test database.
Installation
pip install git+https://git.frostfs.info/TrueCloudLab/allure-validator.git@master
Usage
To start, simply specify the folder with the test database that needs to be checked:
allure-validator <folder> [OPTIONS]
Additionally, allure-validator
can be added to pre-commit hooks:
# .pre-commit-config.yaml
repos:
- repo: https://git.frostfs.info/TrueCloudLab/allure-validator
rev: latest
hooks:
- id: allure-validator
args: ["pytest_tests/"]
pass_filenames: false
Ignore mechanism
The linter can be made to ignore functions/classes. To do this, just write a comment # noqa: allure-validator
to the right of the function/class name. For example:
@pytest.mark.parametrize("test_param", [1, 2])
def test_ignored_function(test_param): # noqa: allure-validator
...
@pytest.mark.parametrize("class_param", [1, 2])
class TestIngoredClass: # noqa: allure-validator
def test_1(self, class_param):
...
def test_2(self, class_param):
...
Note: this may also miss fixtures, so use with caution.
Options
The linter supports the following flags:
--plugins
- regex patterns that plugins must match (paths are compiled according to the template:plugin / src / plugin
).--files
- paths to files with fixtures.
Configuration options can also be provided using special command line arguments, for example:
allure-validator pytest_tests/ --plugins some[-_]plugin --files etc/workspace/some_plugin/src/some_plugin/fixtures.py
Work example
Let's run the linter for some sample tests:
NOT UNIQUE TITLE: Not unique test
In the following tests:
tests/test_validator.py:20:0: title is not unique by `test_not_unique_2`
tests/test_validator.py:119:4: title is not unique by `test_not_unique_3`
tests/test_validator.py:15:0: title is not unique by `test_not_unique_1`
NOT UNIQUE TITLE: Test title
In the following tests:
tests/test_validator.py:30:0: title is not unique by `test_title_not_unique_2`
tests/test_validator.py:25:0: title is not unique by `test_title_not_unique_1`
tests/test_validator.py:5:0: EMPTY TITLE: Missing title in `test_empty_title`
tests/test_validator.py:10:0: EMPTY TITLE: Missing title in `test_empty_title_2`
tests/test_validator.py:20:0: MISSING PARAMS: Parameters are missing from title: object_size
tests/test_validator.py:62:0: MISSING PARAMS: Parameters are missing from title: param_two OR param_three
tests/test_validator.py:115:4: MISSING PARAMS: Parameters are missing from title: root_fixture
tests/test_validator.py:119:4: MISSING PARAMS: Parameters are missing from title: param_class_1
Undetectable cases
Since the linter works with AST python code, any dynamic titles will not be found. For example:
import allure
def test_example():
data = 10
allure.dynamic.title(f"Dynamic title {data}") # Undetected title
...
Dynamic fixtures are only supported if they are declared in pytest_generate_tests
:
def pytest_generate_tests(metafunc):
...
metafunc.parametrize("fixture1, fixture2", [(1, 2), (3, 4)]) # OK
...
if some_condition:
metafunc.parametrize("fixture3", *some_value_generator) # OK