mirror of
https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git
synced 2024-11-05 07:34:20 +00:00
06bfd2d2ae
With Rawhide updates, we quite often run into a situation where a test runs after a *later* version of the package has already gone stable. This even happens for stable releases too, though less often. The current shell-based check just always fails on this case, but it's usually OK, and manually marking every case like this with an "it's OK!" comment gets tiring. Instead, let's use a smarter Python script to do the check. We compare the EVR of all installed update packages with the EVR of the package from the update. If it's the same, fine. If the installed package is lower-versioned, that's always an error, and we fail. If the installed package is higher-versioned, we check whether the update already went stable. If it did, then we soft fail, because probably nothing can go wrong at this point (this is the usual Rawhide case). If the update did not yet go stable, we still hard fail, because something can go wrong in this case: if the update *now* goes stable, the older version from the update may be tagged over the newer version the test got (presumably from current stable). If anything goes wrong with the Bodhi check, or the test is running on a task not an advisory, we treat both cases as fatal. The script also gives easier-to-understand output than the old approach, which should be a bonus. Signed-off-by: Adam Williamson <awilliam@redhat.com>
29 lines
727 B
INI
29 lines
727 B
INI
[tox]
|
|
skipsdist = True
|
|
envlist = py37,py38,py39,py310,py311,coverage-report
|
|
skip_missing_interpreters = true
|
|
[testenv]
|
|
deps =
|
|
pytest
|
|
jsonschema
|
|
coverage
|
|
pytest-cov
|
|
commands=
|
|
python ./fifloader.py --clean templates.fif.json templates-updates.fif.json
|
|
python ./check-needles.py
|
|
py.test unittests/
|
|
py.test --cov-report term-missing --cov-report xml --cov fifloader unittests/
|
|
setenv =
|
|
PYTHONPATH = {toxinidir}
|
|
|
|
[testenv:coverage-report]
|
|
skip_install = true
|
|
deps =
|
|
diff-cover
|
|
pylint
|
|
commands =
|
|
diff-cover coverage.xml --fail-under=90 --compare-branch=origin/main
|
|
diff-quality --violations=pylint --fail-under=90 --compare-branch=origin/main
|
|
setenv =
|
|
PYTHONPATH = {toxinidir}
|