Commit Graph

1264 Commits

Author SHA1 Message Date
Adam Williamson 1d4557126a os_release: fix ISO parsing for 'Fedora-IoT' composes
Also comment this better. We need to index from the end of the
string here, not the start, because going from the start breaks
when the compose shortname has a dash in it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-26 12:01:06 -08:00
Adam Williamson 2c236ef16a Update GNOME Software 'Download' needle for changed kerning
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-26 11:34:21 -08:00
Adam Williamson 95e6ddc896 Update blivet vdb selected needle yet again
I do not know why this needle seems so troublesome...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 23:02:29 -08:00
Adam Williamson da28938572 Update prerelease note needle for font rendering change
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 22:59:14 -08:00
Adam Williamson cc61664e9e Drop --title from live image build command
Per https://github.com/weldr/lorax/pull/881 it wasn't doing
anything anyway, and using it causes the command to fail in F32
and later.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 22:56:48 -08:00
Adam Williamson 8e43dfa304 Drop F31 workaround that's gone stable, add F32 lmod workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 17:40:22 -08:00
Lukáš Růžička 5bfc22800c Assert that terminal is really running (#144) 2020-02-25 10:24:38 +01:00
Adam Williamson adf3f91818 Workaround #1805553 (single disk not selected as install target)
Rawhide seems to have developed a bug where a single disk is no
longer automatically selected as the install target when you
enter the INSTALLATION DESTINATION spoke. We need to work around
this (but register it as a soft failure).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 17:41:34 -08:00
Adam Williamson 73896672cd Update live image build for a change to upstream mock
https://github.com/rpm-software-management/mock/commit/cb25d3ba
changes existing mock configs for stable releases to have a
`dnf.conf` section instead of a `yum.conf` section, and this
change got pushed out to F30 and F31, which breaks us :( Our
mock config that we use for building live images assumes the
existence of a `yum.conf` section in the config it inherits
from.

This change is now stable for F30 and F31, so at least we don't
have to do any conditional shenanigans; we can just change to
'the new style' unconditionally and things should work OK.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 12:24:10 -08:00
Adam Williamson 54afe23572 Update an iSCSI needle for font changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 12:14:35 -08:00
Adam Williamson 18c818da7d Update overview app grid needle (more GNOME icon changes)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 09:57:04 -08:00
Adam Williamson 5300c35106 Update desktop update notification only needle
More GNOME icon changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 09:53:30 -08:00
Adam Williamson 90cc65b2fa Wait a bit longer from software selection back to main hub
Seems like this can sometimes take even longer than 50 seconds,
e.g. https://openqa.stg.fedoraproject.org/tests/727432 . Let's
give it 90.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:52:35 -08:00
Adam Williamson 2e6cf6453d Tweak IoT initial-setup handling (only on F31)
initial-setup doesn't appear on IoT F32+, so we shouldn't expect
it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:38:50 -08:00
Adam Williamson deab03a80d Update another partitioning needle for GTK+ widget changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:12:16 -08:00
Adam Williamson 2c33325595 Add IoT 32 identification needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:11:52 -08:00
Adam Williamson 04a2c22273 Set disk size for memory_check test to 13GB
It needs more space to install Workstation.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:10:28 -08:00
Adam Williamson 7fd23dacea Update GDM input needles for non-English languages
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:37:51 -08:00
Adam Williamson 8d14addd18 Update two more anaconda needles for smaller radio buttons
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:36:59 -08:00
Adam Williamson 9f331dbbe3 Update authentication required GNOME needle
They took away the icon, so now we can only match on text :(

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:35:47 -08:00
Adam Williamson 2466cf53bd Update Firefox launcher needle
Looks like GNOME is rendering it a bit smaller now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:35:06 -08:00
Adam Williamson dde0113fb1 Update KDE 'no notifications' needle
Probably a transparency thing.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 09:47:36 -08:00
Adam Williamson f7a9625b55 Update GDM input needle
GNOME reworked this widget a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 09:47:13 -08:00
Adam Williamson 2fda57482b Update anaconda needles for icon/widget changes
Seems some changes to symbolic icons and widgets have happened
in GTK+ or something, this updates anaconda needles for them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 09:46:35 -08:00
Adam Williamson 0f8f6082eb Enable IoT testing with install and base tests
IoT is becoming a release-blocking edition for F32, so we should
be testing it for sure. We may add specific tests, but for now
let's run the install and base tests on it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-17 08:36:55 -08:00
Adam Williamson b09936ba0a Update 'select_disk_1' needle
Some kinda font rendering change or something.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-17 08:33:57 -08:00
Adam Williamson e5728beada Add version 32 identification needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-14 16:53:31 -08:00
Adam Williamson 1eb4e3dca5 Add perl syntax check test, add it to CI
Inspired by openQA's 01-compile-check-all.t, this adds a perl
test which checks the syntax of main.pm and all lib and test
files, and hooks it up to CI. Requires os-autoinst and
perl-Test-Strict.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-13 15:28:09 -08:00
Adam Williamson 80ddb89f91 Fix firefox addon_success needle
I updated this last month without looking closely enough; one of
the matches was bogus. We now don't seem to get the 'added to
Firefox' popup any more, so we can't match on it, but I guess we
can match on the 'remove' button as an indicator that the add
worked.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-07 12:10:21 -08:00
Adam Williamson 66e53feb4d Update workarounds (drop update now stable, add F31 libdnf fix)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-05 02:51:37 -08:00
Adam Williamson 214f2cc8eb Add fifloader tests, template schemas, update README
This adds a test suite for fifloader (renamed fifloader.py for
test sanity). It adds JSON Schema form schemas for both FIF and
upstream openQA template data, and has fifloader (optionally,
but by default) validate both input and output data against the
schemas. It also adds a tox.ini configured to run the fifloader
tests, use fifloader to validate the template files, and do diff
coverage and lint checks. It also adjusts the Zuul config to run
tox instead of the test job.

There are also some pylint cleanups, since the new tests run
pylint.

fifcheck, fifconverter and tojson.pm are removed, as they were
mainly only needed for one-time conversion of the old format
templates; now they are in the git history we can always recover
them if we need them.

Along with all this I updated the README a bit to explain some
of it (and explain FIF better), and to explicitly state that this
repo is GPLv2+ licensed, and added GPL headers to some of the
files.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-29 22:45:38 -08:00
Tristan Cacqueray 7b9a4306cd Add Zuul CI configuration 2020-01-27 13:34:31 +00:00
Adam Williamson ceb147200a fifloader: Simplify generated job templates a bit
Again looking at the upstream template loader, we don't need to
pass a 'machine' hash with a 'name' key (we can just pass a
'machine_name' string), or a 'test_suite' hash with a 'name' key
(we can just pass 'test_suite_name' string), and we don't need
to pass a 'product' hash, we can just pass the product keys
directly in the job template hash. In fact the upstream loader
transforms 'machine' and 'test_suite' hash 'name' keys into the
strings and moves 'product' hash keys up into the top level hash
before POSTing to the API anyway.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-25 17:39:12 +01:00
Adam Williamson 07e57c638f Re-enable memory_check test (on Everything, with WS packages)
We were only running this test on the Workstation boot ISO...
then that was removed, and so we actually never ran this test at
all any more. We should run it on at least one image as check-
compose uses the output to check whether installer memory usage
has increased. I initially ran it on Workstation as that installs
the most packages; so let's run it on Everything but use the
Workstation package set.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-25 17:15:08 +01:00
Adam Williamson 80c54d5491 Add cockpit_updates and remote_logging tests to updates
Again, no reason not to run these on updates. Includes adding
oldcantarell versions of several needles for current cockpit,
as they're needed for the tests to pass. Also tweak a couple of
needles to avoid false matches (add more empty space).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 21:03:41 +01:00
Adam Williamson 44d1dc3607 Add base_reboot_unmount and base_system_logging to update tests
No reason not to run these on updates as well. And it's much
easier with FIF!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 20:50:39 +01:00
Adam Williamson a348198cdc Correct except line in fifloader
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 19:48:22 +01:00
Adam Williamson 2c197d520c Add a whole intermediate template format ('FIF') and tools
I and @lruzicka (and I think @jskladan and @jsedlak and
@michelmno and everyone else who's ever touched it...) are being
gradually driven nuts by manually editing the test templates.
The bigger the files get the more awkward it is to keep them
straight and be sure we're doing it right. Upstream doesn't do
things the same way we do (they mostly edit in the web UI and
dump to file for the record), but we do still think making
changes in the repo and posting to the web UI is the right way
around to do it, we just wish the format was saner.

Upstream has actually recently introduced a YAML-based approach
to storing job templates which tries to condense things a bit,
and you can dump to that format with dump-templates --json, but
@lruzicka and I agree that that format is barely better for
hand editing in a text editor than the older one our templates
currently use.

So, this commit introduces...Fedora Intermediate Format (FIF) -
an alternative format for representing job templates - and some
tools for working with it. It also contains our existing
templates in this new format, and removes the old template files.
The format is documented in the docstrings of the tools, but
briefly, it keeps Machines, Products and TestSuites but improves
their format a bit (by turning dicts-of-lists into dicts-of-
dicts), and adds Profiles, which are combinations of Machines and
Products. TestSuites can indicate which Profiles they should be
run on.

The intermediate format converter (`fifconverter`) converts
existing template data (in JSON format; use tojson.pm to convert
our perl templates to JSON) to the intermediate format and
writes it out. As this was really intended only for one-time use
(the idea is that after one-time conversion, we will edit the
templates in the intermediate format from now on), its operation
is hardcoded and relies on specific filenames.

The intermediate format loader (`fifloader`) generates
JobTemplates from the TestSuites and Profiles, reverses the
quality-of-life improvements of the intermediate format, and
produces template data compatible with the upstream loader, then
can write it to disk and/or call the upstream loader directly.

The check script (`fifcheck`) runs existing template data through
both the converter and the loader, then checks that the result is
equivalent to the input. Again this was mostly written for one-
time use so is fairly rough and hard-coded, but I'm including it
in the commit so others can check the work and so on.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:21:23 +01:00
Adam Williamson 12b0fb04dc Update a ppc64le-only blivet needle for Cantarell 0.201
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:20:54 +01:00
Adam Williamson 776d31bd5b Update various Cockpit needles for 211
Note we need two variants of most of these, one for Rawhide with
Cantarell 0.201, one for F30/F31 with Cantarell 0.111.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:18:27 +01:00
Adam Williamson f11a57f01a Drop match level on cockpit main needle
New cockpit hits 95%, let's just drop the required %age rather
than make a new needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 12:53:14 +01:00
Adam Williamson b2139bf2dd Fully fix the reboot unmount test
OK, we need to define $self. And we're also gonna need to login
a bit harder later too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 12:30:58 +01:00
Adam Williamson 9d985d8ebd Fix base_reboot_unmount on graphical installs
On graphical flavors we are not at a console when this test runs.
We need to do root_console to get there, and also bypass_1691487
for ppc64le. Copied from base_selinux.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 11:51:18 +01:00
Adam Williamson 8b28625ea2 Add a little script that converts the templates to JSON
For reasons! We are thinking about ways to improve the template
workflow, and this is useful for doing that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-22 14:54:24 +01:00
Adam Williamson 0a4a787c03 Give Firefox longer to start
It seems to take a long time sometimes for some reason. Can't
pin it down but it's causing test flakes, so let's just let it
be. It *may* happen when chrony adjusts the system clock just as
Firefox is starting, for some reason.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-22 10:40:46 +01:00
Lukáš Růžička 23bae35d82 Create a test to test unmount errors during reboot (aka Reboot Testcase) 2020-01-21 16:55:43 +01:00
Lukáš Růžička 780a88f443 Fix indentation in templates. 2020-01-21 16:09:32 +01:00
Adam Williamson c922226d7c Add a variant Cockpit join_complete needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-20 13:36:58 +01:00
Adam Williamson 49a540cc3d Update addon_success needle for new Firefox
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-20 13:36:28 +01:00
Adam Williamson f8292bfeeb More Cantarell: Russian and French
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-18 23:56:11 +00:00