forked from TrueCloudLab/frostfs-testcases
138 lines
4.9 KiB
Markdown
138 lines
4.9 KiB
Markdown
## Testcases structure
|
|
|
|
Tests written with PyTest Framework are located under `pytest_tests/testsuites` directory.
|
|
|
|
These tests rely on resources and utility modules that have been originally developed for Pytest Framework.
|
|
|
|
## Testcases execution
|
|
|
|
### Initial preparation
|
|
|
|
1. Install frostfs-cli
|
|
- `git clone git@github.com:nspcc-dev/frostfs-node.git`
|
|
- `cd frostfs-node`
|
|
- `make`
|
|
- `sudo cp bin/frostfs-cli /usr/local/bin/frostfs-cli`
|
|
|
|
2. Install frostfs-authmate
|
|
- `git clone git@github.com:nspcc-dev/frostfs-s3-gw.git`
|
|
- `cd frostfs-s3-gw`
|
|
- `make`
|
|
- `sudo cp bin/frostfs-authmate /usr/local/bin/frostfs-authmate`
|
|
|
|
3. Install neo-go
|
|
- `git clone git@github.com:nspcc-dev/neo-go.git`
|
|
- `cd neo-go`
|
|
- `git checkout v0.92.0` (or the current version in the frostfs-dev-env)
|
|
- `make`
|
|
- `sudo cp bin/neo-go /usr/local/bin/neo-go`
|
|
or download binary from releases: https://github.com/nspcc-dev/neo-go/releases
|
|
|
|
4. Clone frostfs-dev-env
|
|
`git clone git@github.com:nspcc-dev/frostfs-dev-env.git`
|
|
|
|
Note that we expect frostfs-dev-env to be located under
|
|
the `<testcases_root_dir>/../frostfs-dev-env` directory. If you put this repo in any other place,
|
|
manually set the full path to frostfs-dev-env in the environment variable `DEVENV_PATH` at this step.
|
|
|
|
5. Make sure you have installed all the following prerequisites on your machine
|
|
|
|
```
|
|
make
|
|
python3.10
|
|
python3.10-dev
|
|
libssl-dev
|
|
```
|
|
As we use frostfs-dev-env, you'll also need to install
|
|
[prerequisites](https://github.com/nspcc-dev/frostfs-dev-env#prerequisites) of this repository.
|
|
|
|
6. Prepare virtualenv
|
|
|
|
```shell
|
|
$ make venv.local-pytest
|
|
$ . venv.local-pytest/bin/activate
|
|
```
|
|
|
|
7. Setup pre-commit hooks to run code formatters on staged files before you run a `git commit` command:
|
|
|
|
```shell
|
|
$ pre-commit install
|
|
```
|
|
|
|
Optionally you might want to integrate code formatters with your code editor to apply formatters to code files as you go:
|
|
* isort is supported by [PyCharm](https://plugins.jetbrains.com/plugin/15434-isortconnect), [VS Code](https://cereblanco.medium.com/setup-black-and-isort-in-vscode-514804590bf9). Plugins exist for other IDEs/editors as well.
|
|
* black can be integrated with multiple editors, please, instructions are available [here](https://black.readthedocs.io/en/stable/integrations/editors.html).
|
|
|
|
8. Install Allure CLI
|
|
|
|
Allure CLI installation is not an easy task, so a better option might be to run allure from
|
|
docker container (please, refer to p.2 of the next section for instructions).
|
|
|
|
To install Allure CLI you may take one of the following ways:
|
|
|
|
- Follow the [instruction](https://docs.qameta.io/allure/#_linux) from the official website
|
|
- Consult [the thread](https://github.com/allure-framework/allure2/issues/989)
|
|
- Download release from the Github
|
|
```shell
|
|
$ wget https://github.com/allure-framework/allure2/releases/download/2.18.1/allure_2.18.1-1_all.deb
|
|
$ sudo apt install ./allure_2.18.1-1_all.deb
|
|
```
|
|
You also need the `default-jre` package installed.
|
|
|
|
If none of the options worked for you, please complete the instruction with your approach.
|
|
|
|
### Run and get report
|
|
|
|
1. Run tests
|
|
|
|
Make sure that the virtualenv is activated, then execute the following command to run a singular test suite or all the suites in the directory
|
|
```shell
|
|
$ pytest --alluredir my-allure-123 pytest_tests/testsuites/object/test_object_api.py
|
|
$ pytest --alluredir my-allure-123 pytest_tests/testsuites/
|
|
```
|
|
|
|
2. Generate report
|
|
|
|
If you opted to install Allure CLI, you can generate a report using the command `allure generate`. The web representation of the report will be under `allure-report` directory:
|
|
```shell
|
|
$ allure generate my-allure-123
|
|
$ ls allure-report/
|
|
app.js data export favicon.ico history index.html plugins styles.css widgets
|
|
```
|
|
|
|
To inspect the report in a browser, run
|
|
```shell
|
|
$ allure serve my-allure-123
|
|
```
|
|
|
|
If you prefer to run allure from Docker, you can use the following command:
|
|
```shell
|
|
$ mkdir -p $PWD/allure-reports
|
|
$ docker run -p 5050:5050 -e CHECK_RESULTS_EVERY_SECONDS=30 -e KEEP_HISTORY=1 \
|
|
-v $PWD/my-allure-123:/app/allure-results \
|
|
-v $PWD/allure-reports:/app/default-reports \
|
|
frankescobar/allure-docker-service
|
|
```
|
|
|
|
Then, you can check the allure report in your browser [by this link](http://localhost:5050/allure-docker-service/projects/default/reports/latest/index.html?redirect=false)
|
|
|
|
NOTE: feel free to select a different location for `allure-reports` directory, there is no requirement to have it inside `frostfs-testcases`. For example, you can place it under `/tmp` path.
|
|
|
|
# Contributing
|
|
|
|
Feel free to contribute to this project after reading the [contributing
|
|
guidelines](CONTRIBUTING.md).
|
|
|
|
Before starting to work on a certain topic, create a new issue first, describing
|
|
the feature/topic you are going to implement.
|
|
|
|
|
|
# License
|
|
|
|
- [GNU General Public License v3.0](LICENSE)
|
|
|
|
## Pytest marks
|
|
|
|
Custom pytest marks used in tests:
|
|
* `sanity` - Tests must be runs in sanity testruns.
|
|
* `smoke` - Tests must be runs in smoke testruns.
|