forked from TrueCloudLab/frostfs-testlib
[#5] Implement plugin-based system for reporter
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
This commit is contained in:
parent
655a86a5b0
commit
c5ff64b3fd
5 changed files with 148 additions and 10 deletions
61
README.md
61
README.md
|
@ -7,6 +7,67 @@ Library can be installed via pip:
|
||||||
$ pip install neofs-testlib
|
$ pip install neofs-testlib
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
Library components can be configured explicitly via code or implicitly via configuration file that supports plugin-based extensions.
|
||||||
|
|
||||||
|
By default testlib uses configuration from file `.neofs-testlib.yaml` that must be located next to the process entry point. Path to the file can be customized via environment variable `NEOFS_TESTLIB_CONFIG`. Config file should have either YAML or JSON format.
|
||||||
|
|
||||||
|
### Reporter Configuration
|
||||||
|
Currently only reporter component can be configured. Function `set_reporter` assigns current reporter that should be used in the library:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from neofs_testlib.reporter import AllureReporter, set_reporter
|
||||||
|
|
||||||
|
reporter = AllureReporter()
|
||||||
|
set_reporter(reporter)
|
||||||
|
```
|
||||||
|
|
||||||
|
Assignment of reporter must happen before any testlib modules were imported. Otherwise, testlib code will bind to default dummy reporter. It is not convenient to call utility functions at specific time, so alternative approach is to set reporter in configuration file. To do that, please, specify name of reporter plugin in configuration parameter `reporter`:
|
||||||
|
```yaml
|
||||||
|
reporter: allure
|
||||||
|
```
|
||||||
|
|
||||||
|
Testlib provides two built-in reporters: `allure` and `dummy`. However, you can use any custom reporter via [plugins](#plugins).
|
||||||
|
|
||||||
|
## Plugins
|
||||||
|
Testlib uses [entrypoint specification](https://docs.python.org/3/library/importlib.metadata.html) for plugins. Testlib supports the following entrypoint groups for plugins:
|
||||||
|
- `neofs.testlib.reporter` - group for reporter plugins. Plugin should be a class that implements interface `neofs_testlib.reporter.interfaces.Reporter`.
|
||||||
|
|
||||||
|
### Example reporter plugin
|
||||||
|
In this example we will consider two Python projects:
|
||||||
|
- Project "my_neofs_plugins" where we will build a plugin that extends testlib functionality.
|
||||||
|
- Project "my_neofs_tests" that uses "neofs_testlib" and "my_neofs_plugins" to build some tests.
|
||||||
|
|
||||||
|
Let's say we want to implement some custom reporter that can be used as a plugin for testlib. Pseudo-code of implementation can look like that:
|
||||||
|
```python
|
||||||
|
# my_neofs_plugins/src/x/y/z/custom_reporter.py
|
||||||
|
from contextlib import AbstractContextManager
|
||||||
|
from neofs_testlib.reporter.interfaces import Reporter
|
||||||
|
|
||||||
|
|
||||||
|
class CustomReporter(Reporter):
|
||||||
|
def step(self, name: str) -> AbstractContextManager:
|
||||||
|
... some implementation ...
|
||||||
|
|
||||||
|
def attach(self, content: Any, file_name: str) -> None:
|
||||||
|
... some implementation ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Then in `pyproject.toml` of "my_neofs_plugins" we should register entrypoint for this plugin. Entrypoint must belong to the group `neofs.testlib.reporter`:
|
||||||
|
```yaml
|
||||||
|
# my_neofs_plugins/pyproject.toml
|
||||||
|
[project.entry-points."neofs.testlib.reporter"]
|
||||||
|
my_custom_reporter = "x.y.z.custom_reporter:CustomReporter"
|
||||||
|
```
|
||||||
|
|
||||||
|
Finally, to use this reporter in our test project "my_neofs_tests", we should specify its entrypoint name in testlib config:
|
||||||
|
```yaml
|
||||||
|
# my_neofs_tests/pyproject.toml
|
||||||
|
reporter: my_custom_reporter
|
||||||
|
```
|
||||||
|
|
||||||
|
Detailed information on registering entrypoints can be found at [setuptools docs](https://setuptools.pypa.io/en/latest/userguide/entry_point.html).
|
||||||
|
|
||||||
## Library structure
|
## Library structure
|
||||||
The library provides the following primary components:
|
The library provides the following primary components:
|
||||||
* `cli` - wrappers on top of neoFS command-line tools. These wrappers execute on a shell and provide type-safe interface for interacting with the tools.
|
* `cli` - wrappers on top of neoFS command-line tools. These wrappers execute on a shell and provide type-safe interface for interacting with the tools.
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
[build-system]
|
[build-system]
|
||||||
requires = ["setuptools>=63.0.0", "wheel"]
|
requires = ["setuptools>=65.0.0", "wheel"]
|
||||||
build-backend = "setuptools.build_meta"
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
|
@ -17,8 +17,10 @@ classifiers = [
|
||||||
keywords = ["neofs", "test"]
|
keywords = ["neofs", "test"]
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"allure-python-commons>=2.9.45",
|
"allure-python-commons>=2.9.45",
|
||||||
|
"importlib_metadata>=5.0; python_version < '3.10'",
|
||||||
"paramiko>=2.10.3",
|
"paramiko>=2.10.3",
|
||||||
"pexpect>=4.8.0",
|
"pexpect>=4.8.0",
|
||||||
|
"pyyaml>=6.0",
|
||||||
]
|
]
|
||||||
requires-python = ">=3.9"
|
requires-python = ">=3.9"
|
||||||
|
|
||||||
|
@ -28,6 +30,10 @@ dev = ["black", "bumpver", "isort", "pre-commit"]
|
||||||
[project.urls]
|
[project.urls]
|
||||||
Homepage = "https://github.com/nspcc-dev/neofs-testlib"
|
Homepage = "https://github.com/nspcc-dev/neofs-testlib"
|
||||||
|
|
||||||
|
[project.entry-points."neofs.testlib.reporter"]
|
||||||
|
allure = "neofs_testlib.reporter.allure_reporter:AllureReporter"
|
||||||
|
dummy = "neofs_testlib.reporter.dummy_reporter:DummyReporter"
|
||||||
|
|
||||||
[tool.isort]
|
[tool.isort]
|
||||||
profile = "black"
|
profile = "black"
|
||||||
src_paths = ["src", "tests"]
|
src_paths = ["src", "tests"]
|
||||||
|
|
|
@ -1,6 +1,8 @@
|
||||||
allure-python-commons==2.9.45
|
allure-python-commons==2.9.45
|
||||||
|
importlib_metadata==5.0.0
|
||||||
paramiko==2.10.3
|
paramiko==2.10.3
|
||||||
pexpect==4.8.0
|
pexpect==4.8.0
|
||||||
|
pyyaml==6.0
|
||||||
|
|
||||||
# Dev dependencies
|
# Dev dependencies
|
||||||
black==22.8.0
|
black==22.8.0
|
||||||
|
@ -10,5 +12,5 @@ pre-commit==2.20.0
|
||||||
|
|
||||||
# Packaging dependencies
|
# Packaging dependencies
|
||||||
build==0.8.0
|
build==0.8.0
|
||||||
setuptools==63.2.0
|
setuptools==65.3.0
|
||||||
twine==4.0.1
|
twine==4.0.1
|
||||||
|
|
|
@ -1 +1,60 @@
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from neofs_testlib.reporter import set_reporter
|
||||||
|
|
||||||
|
if sys.version_info < (3, 10):
|
||||||
|
from importlib_metadata import entry_points
|
||||||
|
else:
|
||||||
|
from importlib.metadata import entry_points
|
||||||
|
|
||||||
|
|
||||||
__version__ = "0.1.0"
|
__version__ = "0.1.0"
|
||||||
|
|
||||||
|
|
||||||
|
def __read_config() -> dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Loads configuration of library from default file .neofs-testlib.yaml or from
|
||||||
|
the file configured via environment variable NEOFS_TESTLIB_CONFIG.
|
||||||
|
"""
|
||||||
|
file_path = os.getenv("NEOFS_TESTLIB_CONFIG", ".neofs-testlib.yaml")
|
||||||
|
if os.path.exists(file_path):
|
||||||
|
_, extension = os.path.splitext(file_path)
|
||||||
|
if extension == ".yaml":
|
||||||
|
with open(file_path, "r") as file:
|
||||||
|
return yaml.full_load(file)
|
||||||
|
if extension == ".json":
|
||||||
|
with open(file_path, "r") as file:
|
||||||
|
return json.load(file)
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def __load_plugin(group: str, name: Optional[str]) -> Any:
|
||||||
|
"""
|
||||||
|
Loads plugin using entry point specification.
|
||||||
|
"""
|
||||||
|
if not name:
|
||||||
|
return None
|
||||||
|
plugins = entry_points(group=group)
|
||||||
|
if name not in plugins.names:
|
||||||
|
return None
|
||||||
|
plugin = plugins[name]
|
||||||
|
return plugin.load()
|
||||||
|
|
||||||
|
|
||||||
|
def __init_lib():
|
||||||
|
"""
|
||||||
|
Initializes singleton components in the library.
|
||||||
|
"""
|
||||||
|
config = __read_config()
|
||||||
|
|
||||||
|
reporter = __load_plugin("neofs.testlib.reporter", config.get("reporter"))
|
||||||
|
if reporter:
|
||||||
|
set_reporter(reporter)
|
||||||
|
|
||||||
|
|
||||||
|
__init_lib()
|
||||||
|
|
|
@ -1,14 +1,24 @@
|
||||||
import os
|
|
||||||
|
|
||||||
from neofs_testlib.reporter.allure_reporter import AllureReporter
|
from neofs_testlib.reporter.allure_reporter import AllureReporter
|
||||||
from neofs_testlib.reporter.dummy_reporter import DummyReporter
|
from neofs_testlib.reporter.dummy_reporter import DummyReporter
|
||||||
from neofs_testlib.reporter.interfaces import Reporter
|
from neofs_testlib.reporter.interfaces import Reporter
|
||||||
|
|
||||||
|
__reporter = DummyReporter()
|
||||||
|
|
||||||
|
|
||||||
def get_reporter() -> Reporter:
|
def get_reporter() -> Reporter:
|
||||||
# TODO: in scope of reporter implementation task here we will have extendable
|
"""
|
||||||
# solution for configuring and providing reporter for the library
|
Returns reporter that library should use for storing artifacts.
|
||||||
if os.getenv("TESTLIB_REPORTER_TYPE", "DUMMY") == "DUMMY":
|
"""
|
||||||
return DummyReporter()
|
return __reporter
|
||||||
else:
|
|
||||||
return AllureReporter()
|
|
||||||
|
def set_reporter(reporter: Reporter) -> None:
|
||||||
|
"""
|
||||||
|
Assigns specified reporter for storing test artifacts produced by the library.
|
||||||
|
|
||||||
|
This function must be called before any testlib modules are imported.
|
||||||
|
Recommended way to assign reporter is via configuration file; please, refer to
|
||||||
|
testlib documentation for details.
|
||||||
|
"""
|
||||||
|
global __reporter
|
||||||
|
__reporter = reporter
|
||||||
|
|
Loading…
Reference in a new issue