Add fifloader tests, template schemas, update README
This adds a test suite for fifloader (renamed fifloader.py for
test sanity). It adds JSON Schema form schemas for both FIF and
upstream openQA template data, and has fifloader (optionally,
but by default) validate both input and output data against the
schemas. It also adds a tox.ini configured to run the fifloader
tests, use fifloader to validate the template files, and do diff
coverage and lint checks. It also adjusts the Zuul config to run
tox instead of the test job.
There are also some pylint cleanups, since the new tests run
pylint.
fifcheck, fifconverter and tojson.pm are removed, as they were
mainly only needed for one-time conversion of the old format
templates; now they are in the git history we can always recover
them if we need them.
Along with all this I updated the README a bit to explain some
of it (and explain FIF better), and to explicitly state that this
repo is GPLv2+ licensed, and added GPL headers to some of the
files.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-25 15:35:26 +00:00
|
|
|
[tox]
|
|
|
|
|
skipsdist = True
|
2020-04-08 18:47:06 +00:00
|
|
|
envlist = py37,py38,py39
|
Add fifloader tests, template schemas, update README
This adds a test suite for fifloader (renamed fifloader.py for
test sanity). It adds JSON Schema form schemas for both FIF and
upstream openQA template data, and has fifloader (optionally,
but by default) validate both input and output data against the
schemas. It also adds a tox.ini configured to run the fifloader
tests, use fifloader to validate the template files, and do diff
coverage and lint checks. It also adjusts the Zuul config to run
tox instead of the test job.
There are also some pylint cleanups, since the new tests run
pylint.
fifcheck, fifconverter and tojson.pm are removed, as they were
mainly only needed for one-time conversion of the old format
templates; now they are in the git history we can always recover
them if we need them.
Along with all this I updated the README a bit to explain some
of it (and explain FIF better), and to explicitly state that this
repo is GPLv2+ licensed, and added GPL headers to some of the
files.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-25 15:35:26 +00:00
|
|
|
skip_missing_interpreters = true
|
|
|
|
|
[testenv]
|
|
|
|
|
deps =
|
|
|
|
|
pytest
|
|
|
|
|
jsonschema
|
|
|
|
|
coverage
|
|
|
|
|
diff-cover
|
|
|
|
|
pylint
|
|
|
|
|
pytest-cov
|
|
|
|
|
|
|
|
|
|
commands=
|
|
|
|
|
./fifloader.py --clean templates.fif.json templates-updates.fif.json
|
Add a needle check script, remove some unused needles
I call this...The @lruzicka Catcher!
It's a script that checks for needles that aren't actually used
anywhere. It also checks for cases where we have a needle JSON
file but no image, or an image file but no JSON file (and wipes
one case of the latter). It also adds a run of the script to tox
so we get it in CI.
You could make this script a lot more elaborate if you like, by
being fancier about parsing the test code and templates, but I
don't think it's really warranted, I think it just needs to be
'good enough'. It's not the end of the world if it misses the
odd thing or the whitelisting goes stale.
Quite a lot of the removed needles are remnants of different
approaches to app start/stop testing which weren't caught in the
initial PR review. The short-name partitioning ones are odd; they
were introduced in the commit that moved needles into subdirs,
but at least some of them don't actually appear to be moves. They
may have been non-tracked files Josef had lying around that got
into the commit by mistake, or they may just be old needles we
really used at some point but aren't using any more.
reclaim_space_second_partition was introduced as part of the
shrink test (along with reclaim_space_first_partition) but was
never actually used by that test - I guess, again, the test got
re-written during review but we forgot to remove the needle. We
rejigged user creation to use tab presses not a needle match a
while back, which made user_creation_password_input unnecessary.
The various cockpit_updates_* needles are I think remnants of
rewrites of the cockpit update tests that again were missed in
PR review, the tests as merged never used them.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-08 23:55:46 +00:00
|
|
|
./check-needles.py
|
Add fifloader tests, template schemas, update README
This adds a test suite for fifloader (renamed fifloader.py for
test sanity). It adds JSON Schema form schemas for both FIF and
upstream openQA template data, and has fifloader (optionally,
but by default) validate both input and output data against the
schemas. It also adds a tox.ini configured to run the fifloader
tests, use fifloader to validate the template files, and do diff
coverage and lint checks. It also adjusts the Zuul config to run
tox instead of the test job.
There are also some pylint cleanups, since the new tests run
pylint.
fifcheck, fifconverter and tojson.pm are removed, as they were
mainly only needed for one-time conversion of the old format
templates; now they are in the git history we can always recover
them if we need them.
Along with all this I updated the README a bit to explain some
of it (and explain FIF better), and to explicitly state that this
repo is GPLv2+ licensed, and added GPL headers to some of the
files.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-25 15:35:26 +00:00
|
|
|
py.test unittests/
|
|
|
|
|
py.test --cov-report term-missing --cov-report xml --cov fifloader unittests/
|
|
|
|
|
diff-cover coverage.xml --fail-under=90
|
|
|
|
|
diff-quality --violations=pylint --fail-under=90
|
|
|
|
|
setenv =
|
|
|
|
|
PYTHONPATH = {toxinidir}
|