It seems to be actually installing fedora-release-silverblue now
so we get correct identification here. Update the tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://bodhi.fedoraproject.org/updates/FEDORA-2020-1070052d10#comment-1284223
It seems the rootfs on the Fedora 30 installer images we build at
present has gotten very big, so big that an update which contains
some very slightly larger firmware packages causes the rootfs to
be completely full (though lorax doesn't fail) and the image
doesn't boot.
I don't yet know when or why the rootfs got that big, but it's
not really a bug in this update, so for now let's just tell
lorax to use a bigger rootfs so the tests pass for this and any
similar future updates, until I can maybe find time to pinpoint
the culprit more precisely.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Initial implementation wasn't correct, I forgot CURRREL is not
'the pre-upgrade release version' but just 'the current stable
release'. This is a dumb way to figure out the correct release
number for this context but off-hand I can't think of a better
one.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When there's a failed service we get a stupid bullet-point char
at the start of its line, and all the other lines are space-
padded to match indentation. Which is annoying! This (I hope)
ditches that crap without losing anything of value.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also comment this better. We need to index from the end of the
string here, not the start, because going from the start breaks
when the compose shortname has a dash in it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Per https://github.com/weldr/lorax/pull/881 it wasn't doing
anything anyway, and using it causes the command to fail in F32
and later.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://github.com/rpm-software-management/mock/commit/cb25d3ba
changes existing mock configs for stable releases to have a
`dnf.conf` section instead of a `yum.conf` section, and this
change got pushed out to F30 and F31, which breaks us :( Our
mock config that we use for building live images assumes the
existence of a `yum.conf` section in the config it inherits
from.
This change is now stable for F30 and F31, so at least we don't
have to do any conditional shenanigans; we can just change to
'the new style' unconditionally and things should work OK.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
IoT is becoming a release-blocking edition for F32, so we should
be testing it for sure. We may add specific tests, but for now
let's run the install and base tests on it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On graphical flavors we are not at a console when this test runs.
We need to do root_console to get there, and also bypass_1691487
for ppc64le. Copied from base_selinux.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is failing a lot lately. I've no idea why, but it's not
really part of the actual test and we don't need it for debugging
all the time, so let's drop it for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The text we used to get has been replaced with a spinner, which
is difficult and unreliable to match on. This match was only
here to make the test fail a bit faster if it was broken, so
let's just live without it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're getting issues with KDE's crazy-slow alt-f2 behaviour
here. So try and be even more conservative.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This match keeps giving us problems, and now I look at the test
again, there is actually no need for it at all. Clicking it
doesn't do anything, and we already confirm that we're on the
right page at the next step, where we look for a log entry and
click on that - that will fail if we aren't actually on the
Logs page.
I don't remember what Cockpit used to look like when we first
put this line and needle in, presumably there's a reason we had
them, but they're clearly unnecessary now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
wait_idle is finally removed upstream in recent git os-autoinst.
This replaces all remaining wait_idles with sleeps, except for
one which is removed (I'm hoping improvements to typing in the
last few years should mean it isn't necessary any more, if it
turns out to be, I'll put it back as a sleep).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This failed a couple times in a row on KDE live, let's see if
wait_still_screen is better than sleep.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We need a couple of new needles, plus the 'join domain' button
has disappeared from the front page due to the very inefficient
UI redesign, so we need to scroll down to find it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lately launching anaconda on the KDE live image seems pretty
unreliable and we're not sure why. My last attempt to fix it
doesn't seem to be working, here's another effort based on the
idea it might be caused by moving the mouse from the hidden
position to the icon and back again, let's try moving the mouse
close to the icon before we assert and click it...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We still have a 'apps_run_firefox_stop' needle tag which is for
the same thing as 'firefox_close_tabs'. That's dumb. Get rid of
it and only have the firefox_close_tabs tag and needles. Also
clean up some old firefox_close_tabs needles that haven't matched
for months and all the 'apps_run_firefox_stop' needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE tests are quite frequently failing lately because anaconda
doesn't launch. I'm hoping this will help.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
autocloud is dying soon. So, we want to run the tests in openQA
instead. This adds a test module called 'autocloud' and a test
suite called 'cloud_autocloud' which basically replicate what
autocloud does (download a tarball full of tests and run each
one), and the necessary template bits to run it on Cloud_Base
qcow2 images.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
...otherwise the VNC client tests fail on aarch64 because we try
to apply the 'console=tty0' workaround for #1661288. Fortunately
we don't really need that for the VNC install test to work, so
let's just skip it. We can make this more sophisticated later if
it turns out to be necessary.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
os-autoinst has this code when doing `script_run` on a serial
terminal that does a `wait_serial` for '# ' before running the
command. For some reason when we switch to the anaconda root
console and run a couple of commands after finishing install on
ppc64, the first of these prompt checks times out, which means
the test sits there doing nothing for 90 seconds unnecessarily.
Let's try and avoid that by hacking the prompt check regex to
be empty.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There is nothing inherently 'root'-y about these so it makes no
sense to prefix their names with 'root-'. And why change from
'console' to 'terminal' compared to the naming used in the
actual qemu command and the log files? It's just confusing.
Let's be consistent (except for using - instead of _ here...
but - is easier to type!)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds tests for vncconnect, a variation on VNC install using
reverse VNC (where the client waits for the server to connect to
it). Very similar to the regular-VNC test, but we have to use
tigervnc as Boxes doesn't do reverse VNC.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.
There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:
https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34
we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't know exactly what's going on with this bug yet, but as
a 15 second sleep seems to avoid the issue, let's do that for
now so we catch any future issues.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
See https://openqa.fedoraproject.org/tests/473215 - it failed
because we tried to click FINISH CONFIGURATION while it was
still animating downwards.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The way this works at present, with a check_screen and then an
assert_and_click, there's a window where the check has passed
so we're committed to the assert, but it takes a half second or
so for the assert to actually complete (checking for a needle
is a somewhat heavy operation). During that half second the
'new update!' notification can...and quite often does...appear.
Changing the assert_and_click to a click_lastmatch should (I
hope) tighten this window; click_lastmatch should fire faster
than assert_and_click so there'll be less of a window for the
update notification to appear and break stuff.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:
* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
text is untranslated (e.g. 'Fedora 31') it appears on a
different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
area of the banner this can trigger an os-autoinst needle
comparison bug: https://progress.opensuse.org/issues/56822
All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR fixes#124.
It uses the BUILD variable to get the build string from the
tested compose and uses it to compare correct values for Silverblue.
Originally, it used Workstation values that did not match the
Silveblue ones.
Delete bad yank.
One of the test cases we didn't yet automate is:
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg
Now we have a PXE test, it's actually a good opportunity to test
that at the same time. I don't usually like combining tests like
this but in this case it sort of makes sense as otherwise we'd
have to have a whole parallel PXE install just to test this one
other detail. So, instead of doing an interactive PXE install as
we did at first, let's tweak the test to include a kickstart in
the initramfs and run the install from that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.
Review cockpit.pm
Review autoupdate test.
Review the update test.
Fix typo in cockpit.pm
Add sleep.
Add missing command.
Delete an unused needle.
mock is changing the config files to be based on templates,
which breaks how we generate the live build mock config. This
should work with both the old and new config files.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is causing all kinds of trouble, because when the test is
run on the Server DVD - with the 'orange to blue' gradient - the
prerelease note is dark red text on a dark blue background.
os-autoinst actually reduces the color depth of images/needles
and greyscales them before performing the match...but for this
dark red text on dark blue background, the result seems to be
that the text and background come out *the same grey*, so *any*
text will match the needle (even if it's completely different
text), as will *no text at all*. I've tried finessing around
this a few times but it just keeps happening, so for now I'm
just disabling the pre-release text check at this point. We still
have the check during _boot_to_anaconda, when the text appears
on a *grey* background and so isn't a problem. I'm not removing
the needles yet, until we hear back from upstream:
https://progress.opensuse.org/issues/56822
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Building images with SELinux in permissive mode worked fine in
F29 and F30, but it seems to be broken in F31. Releng seems to
have it set up so the build environment has SELinux disabled,
so let's do the same. Will file a bug for this, as bcl says it
should be considered a bug.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is doing slightly less to exercise the launcher menus and see
whether icons appear or disappear, but it's much faster and more
reliable. We do still use menu launch for one app, just to check
the mechanism works in general.
This adds a whole wodge of stuff to support_server to make it
act as a PXE server, then adds a new test which boots from PXE
and so should hit the PXE server. We use the NFS install repo as
that can be relied on to work for a support_server install.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We do this in quite a few places and need to do it in another,
so let's just have a function for it. It takes a file glob
so we can have it run on a different one for _live_build.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Split this out of install_default, because it really is not a
part of that test and we do not want that test to fail because
the desktop background is wrong. Make it its own test module
and test suite instead. Don't do it on Rawhide, because we
really can't assert anything worthwhile about Rawhide at the
moment at least (this means the test runs but is a no-op and
will always pass on Rawhide, unfortunately). Move the needles
to a more appropriate location (this has nothing to do with
anaconda) and use 'background' not 'wallpaper' naming (that's
the name we use elsewhere in the project, e.g. package names).
Also, run the test on updates, and add an F29 needle for this
purpose.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
cockpit test often seems to fail because the click on a services
entry to open the services detail screen just gets lost. Let's
wait longer before trying it, and retry once if it fails.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The user and root spokes were moved from during-install to pre-
install hub in Rawhide. This should cope with that, while still
working for older images.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There are three places where we basically want to click away
pop-up update notifications and the buggy akonadi_migration_agent
notification if it's there, in KDE tests. Let's share this code
between them, and also let's record soft failures for the buggy
cases in the desktop_notifications test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Current KDE seems to like showing us multiple update available
notifications. So the test must dismiss all of them. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1730482
Signed-off-by: Adam Williamson <awilliam@redhat.com>
* Add test to check module defaults.
* Add whitelist download.
* Fix install test to include selected profile to be on the safe side.
* Add test into templates.
Cockpit 198 comes with a UI overhaul, so almost all needles
need an update.
The 'auditd' service is no longer on the first page. To make
this less fragile (at the cost of not testing that clicking on
a service actually opens the detail page *for that service*,
tweak the needles to just look for *any* running service, click
on it, and check we got to a 'details' page. We also redo the
existing needles for this design.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I'm pretty sure I worked around the modifier bug in os-autoinst,
so this shouldn't be necessary any more. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1727388
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can run these tests much like support_server - have a test
which boots from the support_server disk image but with the ISO
to be tested attached, then mounts it and runs the scripts.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
@lruzicka added this to the KDE app tests in 71c4e273, but there
was no need for a new needle as I'd already done the same thing
in the desktop updates tests; let's just use the same needle.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This shouldn't be needed any more (the referenced bug was fixed
in F28) and doesn't work in non-live installer.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This whole block where we do various things at the console after
install completes was becoming a real mess. I had secret hopes
of killing it entirely at some point, but...that doesn't look
like it's gonna happen this century. So let's make it better
instead. The conditionals were getting very nested and icky and
it was hard to see what was actually going on. This rationalizes
things so first we figure out all the things we might want to do
at a console, then if we don't have anything to do at a console
we go ahead and hit the reboot button; otherwise we go to the
console and do all the things we need to do, including rebooting
unless this is the memory check test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This just doesn't work when running the test on a graphical
install because we're at a desktop when we try and do it, not
a text console. So let's only try to do it if there is no
DESKTOP. See e.g.:
https://openqa.fedoraproject.org/tests/408484#step/base_selinux/2
Signed-off-by: Adam Williamson <awilliam@redhat.com>
