Clocks' aaa_setup did not ever actually check the app launched
properly. It also doesn't handle granting permissions if
necessary, which the apps_startstop test for Clocks does do.
This makes the permission check in the apps_startstop test more
efficient, and adds it to the Clocks app aaa_setup test too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was just doing something silly before, but with recent
os-autoinst having function signatures, it actually causes the
test to fail because '5' isn't a sane value for the argument
this was setting before. Fix it to set the timeout, as it was
trying to do all along.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems when we quit Firefox back to a VT on ppc64le, the system
hangs. Not sure why, but we can deal with it by rebooting the
system and logging back in as root if it happens. Also take the
opportunity to clean up the flow of quit_firefox so we always
check that we get back to a console then wait 5 seconds for the
console to settle, so all the tests that call it can stop doing
that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On Rawhide update tests we often don't seem to get to the login
prompt in 10 seconds, so tweak the code a bit to let us specify
a timeout in root_console, and use 30 seconds here.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Let's not have the reference files in one person's fedorapeople
space, in case that person leaves. Let's upload the text.txt
for checking (it's easier to be able to just read what's in it
than try and figure it out from the diff output), and let's use
diff -u because non-unified diff output is awful.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is the automation of the optional testcase https://fedoraproject.org/wiki/QA:Testcase_i18n_default_fonts.
The test implementation runs the same commands as the mentioned test
case and checks the expected output. It is designed to run in the scope
of postinstall tests when the language is set to "japanese".
I'd prefer not to do this, but I don't see a better way to deal
with the stupid 'welcome' screens. I'm not maintaining needles
to click them away forever.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Firefox 97 is now stable on all releases, so we can forget about
handling browser_download_save and just assume download will
happen automatically.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Up to F33 we could check that an error message was shown when we
tried to log in with the wrong password on GNOME, but since F34
it's transient and disappears too quick to reliably catch, so
we don't check it any more. Now F33 is EOL, drop the conditional.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The g-i-s new user mode was removed in F34, so we don't need to
do this any more (as the now-removed comment said).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Before F34 we had to launch the update tool from the systray.
This isn't the case any more, so we can throw all this code and
these needles away.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We seem to be solidly back to always getting a permanent update
notification in current F36/Rawhide, so we don't need this more
complex path any more. We also don't need these needles any more,
they haven't matched for months.
Logging of additional repo setup changed recently in F37, we
need to handle the new message format here.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2 seconds isn't enough to be sure it opened, which is causing
some weird failures in KDE Rawhide update tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
aarch64 tests are failing here because anaconda's still thinking
about the delete confirmation, it seems.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When testing Rawhide updates we set VERSION to the Rawhide
release number, not "Rawhide". This post-upgrade version check
needs adjusting to handle that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seeing a failure quite often lately where when we try and hit
ctrl-p to print, we just type a p. There's no wait between
maximizing and trying to print, and only a short wait on
launch, so let's try being a bit more defensive there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This can actually take a bit of time, it seems, especially with
debug kernels. Let's give it a minute.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lately anaconda can take up to 10 seconds to exit the root pw
spoke, which can defeat the subsequent `wait_still_screen` that's
meant to wait out the 'slide-in-from-the-top' animation of the
hub. So let's assert the hub after we click Done, then the still
screen wait will only happen *once the hub is visible* and should
really do its job.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OK, this is annoying. GNOME Software intentionally does *not*
clear the 'download' or 'reboot and update' button when you hit
the refresh button, it just leaves them sitting there while the
refresh happens. So let's specifically require the 'refreshing'
text to appear and go away before we try and click on download
or apply.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Agh, GNOME's UI is *not* helpful here at all. The Apply button
remains visible for a long time after you hit Refresh.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is yet another twiddle to that same damn bit where we try
to apply updates. This more or less reverts the last tweak to
it, where we skipped hitting 'refresh' if 'download' or 'apply'
are already visible. The trouble with that is that the app may
have already found and prepared updates before we got our
"prepared" python3-kickstart update in place - so the update
operation might work perfectly, but not update the package we
expect it to update, and the test may fail.
This time, let's try *always* refreshing, then wait a bit after
hitting the refresh button before we start looking for apply
or download to try and avoid the 'race' we were trying to solve
with the last tweak (where we hit refresh then immediately try
to hit a download or apply button which vanishes before we can
hit it). I think this should be safe as both KDE and GNOME
should always show a refresh button now (this wasn't the case
before, I think, F34).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a bug in the Save As... dialog on the flatpak version
of evince currently where the existing filename is not pre-
selected, so when the test types 'alternative', it gets
prepended to the existing filename instead of overwriting it,
and we wind up with alternativeevince.pdf, not alternative.pdf.
Let's treat this as a soft failure rather than a hard failure.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On upgraded systems, gedit might still be present; remove it so
we don't launch it by accident and try to test it instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Firefox 97+ don't ask you what to do with downloads any more,
they just...download them. For now we'll handle both workflows,
once 97+ is stable everywhere we can drop handling the old one.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
At the end of _podman_client we create a mutex that we want the
server to pick up. If we exit too soon, the mutex goes away and
the server test fails. 5 seconds turns out to be not enough of a
wait because, although the server retries the lock call every
5 seconds, it can hit `api_call_2 failed` and wait 10 seconds
before retrying. Let's wait 30 seconds just to be safe.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Otherwise, if there's an update to fXX-backgrounds pending, we'll
get it on reboot in the middle of the test, and the real
backgrounds will comes back and replace our black fake ones and
break the test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Silverblue still has an old version of g-t-e where this theming
stuff worked differently, so put that stuff back for Silverblue
but keep the new stuff for RPMs.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So just type 'text-editor'. If you have both gedit and gte
installed this will find both, but that should never be the case
so it should be OK.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
To get to the keyboard/input method settings and add an input
method when doing a Japanese install test, we type 'keyboard',
but in current GNOME 42.beta that doesn't find the right pane.
Typing 'input' does work, though, so let's use that instead.
Also the GDM login needle needed updating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The backend is now cryfs in F36/Rawhide. I don't think we need
to be policing which backend Vault decides to use, so let's just
accept either.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'software' test module in GNOME apps_startstop does a subset
of what the desktop_update_graphical test does already, but
using its own needles. Let's just have it use the same needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can't use the default similarity_level for wait_still_screen
when there's a flashing cursor - flashing cursor will always
cause the similarity level to be too low and the wait will just
time out at 30 seconds. Cut it to 42.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was originally a test of nm-connection-editor. However,
at some point that app stopped shipping a .desktop file by
default (it's in a subpackage that is not included in a default
KDE install) and the needle got updated to match on what the
same string now launched, which is a random part of the KDE
system settings. But there's no real sense in this - we don't
test launching every other pane of the system settings app from
the launcher, so it doesn't make sense to just test one random
one like this. Let's just throw the test out.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We have three different needles which all match on a stock KDE
"cancel" button. Let's just have one. Also, update it for latest
Rawhide/F36 KDE.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
script_output has no arg to make it type slowly, and I'm seeing
that the test often fails trying to do full-speed script_output
at a desktop terminal. Let's do this part from a VT.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The help test had its own needle for the user creation link,
but we already have an existing one. We should probably re-
arrange all the needles that are in 'install_process' now the
root and user creation spokes are moved to the main hub, but
that's a big change so I'll do it separately. This just removes
the duplicate needle and tweaks some match names.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When the printing_builtin test ran on an F35 respin compose it
failed; it turns out the target filename was different for the
built-in print-to-PDF on GNOME on F35. So let's just always
use the 'ls' output to find the file, but pick the directory
to check based on whether we're using cups or not.
Also rename the needles to have unique names, and add one for
F35 GNOME.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The PR introduces an improved logic to the desktop_printing.pm
that allows to use the USE_CUPS variable in templates to trigger
the installation of cups-pdf prior to the actual test.
The cups-pdf is then used as an alternative PDF printer
instead the built-in Save As PDF method.
It was removed from the default install:
https://pagure.io/fedora-comps/c/5371509
in favor of a new screenshot mechanism that's built in to GNOME
Shell.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit 182bfdaa55. The
problem was not in the test code here, it's just that the tests
were run with RAWREL set to 36 when it should be 37; this is
because fedfind's data source hadn't been updated for 36 branch
when the tests ran. It's tricky to time this exactly right -
ideally we'd probably update fedfind's data source one second
before the first Branched compose completes, but of course that's
difficult in practice.
It seems that the IoT fix was not limited to the IoT
subvariant only, which caused wierd behaviour when
the version was set to Rawhide when it should not be.
This fixes it.
The latest version of Gnome-Text-Editor bring a little
bit different UI and some new or modified features.
This commit fixes the suite to run on newest version.
Workstation has replaced gedit with gnome-text-editor in Rawhide,
so this is no longer useful. We will replace it with a test suite
for gnome-text-editor.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Workstation replaced gedit with gnome-text-editor. This updates
the needles and also changes the name used for 'tagging' the
application for the core_applications test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Both were dropped from the default KDE install set:
https://pagure.io/fedora-comps/c/edd0d74
so we need to drop them here too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
anaconda changed the repository add code (again) and so the log
messages changed (again). We're now tracking three variations
from <F35, F35, and F36. Sigh.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lately Firefox sometimes just closes immediately, it doesn't
show the 'close tabs' dialog. So let's make that optional, but
check we quit properly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In https://openqa.fedoraproject.org/tests/1051360 Firefox showed
an "Open previous tabs?" bar which put this button just off the
bottom of the screen, so we need to scroll to it if that happens.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a workaround for
https://bugzilla.redhat.com/show_bug.cgi?id=1933378 . Sometimes
when booting Server on a tap test without a working network
config (no DHCP server, static network not yet configured) we
hit a bug where the splash screen does not clear completely, and
this causes all the console needle matches to fail. To work
around this, we remove plymouth from the installed system after
running the install_default_upload test on Server; all affected
tests use the image uploaded by that test. We exclude aarch64
because there's a known problem with removing plymouth on that
arch (#1940163), plus the bug doesn't actually seem to happen on
aarch64 for some reason.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
First attempt at this often fails for some reason - while we're
typing 'keyboard' the results come up as normal, but before we
hit enter, they all disappear and are replaced with "no results
found" (in Japanese). Dunno why. This will hopefully work around
that, if it works reliably on the second try.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
kgpg had its own, which doesn't make any sense. There are grey
and blue background variants that weren't consistently named.
This should rationalize things sensibly, and adds a new needle
for the new Plasma in Rawhide, with a lighter blue background.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We sometimes type 'keyboard' at the overview and get no results.
This might help something finish caching or whatever, let's see.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
After we change the log level in Cockpit, on aarch64, it seems
to take quite a long time to reload the messages. This allows
twice as long, with a soft failure if we get into the back half.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
To work around #1999321, we'll disable dnssec validation on the
FreeIPA server when doing an upgrade to Fedora 35 or later.
This sucks but I can't find a better option.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We dropped memtest from the images. We may replace it with
something better at some point, but until that day, let's drop
this test so it's not uselessly failing all the time.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This fails sometimes just because we're too early, or something.
Also with GNOME Shell 41rc, alt-f1 no longer works to open and
close the overview. super *does* seem to work in KDE these days,
so let's switch from alt-f1 to super everywhere.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seen a few failures where it takes just longer than 30 seconds
for GDM to show up here, e.g.
https://openqa.fedoraproject.org/tests/968142
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Always do the page-down thing, having it conditional just makes
things more complicated unnecessarily. Try up to 5 times to
click the link, because Cockpit sometimes redraws itself before
us identifying it and clicking on it, e.g.:
https://openqa.fedoraproject.org/tests/968422
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Closing the last notification in the tray closes the entire tray
now, which seems odd. Cope with that by re-opening it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Really this just boils down to needing an extra click. We can
even just do the click in the old UI as well, it's not needed
but won't hurt anything and keeps the code simple.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'root spoke takes forever to load if debug enabled' bug was
fixed a while back, and we now always set root password before
the install process starts, so the race problem we had when
doing it at the same time as install was happening is no longer
an issue.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We need the same alternate check method logic for NFS addrepo on
F35 now we're testing F35 updates. This is all getting a bit
messy and could maybe stand a refactor at some point...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
See https://github.com/cockpit-project/cockpit/issues/16243 .
This is a fairly minor issue upstream knows about but will not
be fixed immediately, so we'll add a workaround for it for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
As suggested by @kparal, this adds a test that specifies an
additional repository using a metalink. The repository contains
a single package, 'testpackage', that supplements glibc (so it
should always get installed). The test runs an install then
checks that testpackage got installed.
We also deduplicate a pair of needles which were matching on the
same anaconda UI feature (an "add" button) and use that same
needle in this test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This also requires a new needle for accounts.fedoraproject.org, as the webpage
is completely different now. The old needle for browser_fas_home is obsolete via
this change and can be removed
Remove a whole chunk of needles that haven't matched for more
than 3 months. Also move a few needles to appropriate locations,
simplify some code chunks that relied on removed needles (if
we're not matching the needles, we don't need those chunks any
more), and drop some other no-longer-needed conditionals for
older releases.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
`click_lastmatch` wasn't safe there, if we caught the middlle of
the scroll animation. We have to assert again after waiting.
Also do the same in `_do_install_and_reboot` as the same can
happen.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Grr, with the screen changes and the animations...waiting for
screen to change when clicking a button doesn't work very well
as the button animation often counts as a 'screen change', so
we're still having issues with the stupid animation transition.
Let's see if this helps.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test for fingerprint-based login, as requested by
@benzea in #223. We use the fprintd dummy device to let us
simulate scanning a fingerprint, and check various scenarios
recommended by @benzea.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The open dialog on Silverblue (which is apparently not at all
the same thing as the open dialog on Workstation, though they
look the same) does not default to the Documents folder, so we
have to open it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
`rpm` doesn't work. I dunno off-hand how you'd install git on
ostree if it wasn't there, so let's just assume it will be.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR fixes issue #188. It adds a test suite to test basic
functionality of Evince and brings the following features:
* test scripts for various Evince functions.
* needles to support the Evince test scripts
* new template variables `TESTPATH` and `POSTINSTALL_LOAD_ALL` (see
below)
* new logic in `main.py` (see below)
The new variables and the new logic make it easier to create test
suites for post-installation tests. If TESTPATH is used, OpenQA
will take all tests mentioned in POSTINSTALL from that specified
TESTPATH. If both TESTPATH and POSTINSTALL_LOAD_ALL are used, then
OpenQA will run all tests it can find at the TESTPATH location.
If POSTINSTALL and POSTINSTALL_LOAD_ALL are set simultaneously,
then only POSTINSTALL will be taken into account and OpenQA will
only load tests mentioned there.
The power_off function in desktop_login was not really asserting/checking
whether the VM got turned off. However, os-autoinst supports checking whether
the VM is turned off via assert_shutdown. This is additionally much more useful
than using check_shutdown, which does nothing if the VM is still running.
Wait for a change when clicking the done button in root password
screen (anaconda can pause for a long time there).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I've seen some failures caused by a sort of race where both
'apply' and 'refresh' needles match at the first assertion, but
openQA "prefers" the 'refresh' match. So we click the 'refresh'
button and *immediately* check_screen for apply, which is still
visible...but by the time we go to click it, it's gone because
the refresh found something new and now it's showing "Download".
This tweak should help, because if we can 'see' both refresh and
apply at the start, we'll just go ahead and click apply, we
won't refresh. The logic becomes a little more obscure, but I'm
not sure I see a fix for that. At least until KDE's tool finally
settles down for two releases in a row and we might be able to
simplify this whole thing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It can take some time for first login of a user (especially in
KDE). Test has been failing lately on Rawhide because of this.
It seems until recently we never got a still screen when trying
to log in as Jim - so the effective wait for login to complete
was 60 seconds, 30 seconds for wait_still_screen to time out
then 30 seconds for the actual login needle assertion - but now
we are getting a blank screen for 5 seconds which satisfies
wait_still_screen almost immediately, so effective timeout for
the login process is only 35 seconds, which isn't long enough.
So let's bump the check_desktop timeout to 60 seconds.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE has made it so you need to double-click icons on the desktop
now. Unfortunately this means a clunky conditional at least until
the update goes stable. When F33 is EOL we can reduce it to
just "if kde".
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In #235 we accidentally included an entire disks.pm test that
wasn't meant to be there - the infocenter module test is the
right thing to exercise plasma-disks, there is no standalone
app, running 'disks' just gets you the KDE Partition Manager,
which we already test. So this removes that test and renames a
needle that looks like it's for that test but is actually for
the kinfocenter module, to make this clearer.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It has been noted that updates have broken podman in the past and
this is a major issue for some users. Let's create a new update
flavor and run the test in it. We'll use the server image as a
base, but it's not really a server test, so I'm giving it its own
flavor so it's not run on updates that we only want to run server
tests on, and we can schedule just this test to run on container-y
updates.
As part of this, we need to install podman before running the
test; for flavors we currently run it on we expect podman to be
preinstalled, but that's not true for the server base image.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a new test that automates the above mentioned test case.
It starts the installation in text mode using the `install_text` test
case, which it interrupts using the Anaconda crash trigger.
When the crash happens, it goes through the process of reporting
the bug to Bugzilla, checks that Bugzilla sends a positive
confirmation of the action, but also performs some REST API
calls to do a proper check and then it closes the bug to clean up.
GNOME dropped the g-i-s new user mode in F34, so on a Japanese
install with user created in the installer, you don't get an
input source configured out of the box or on first boot. So
we'll just have to do it manually after booting, before we test
if it works.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Looks like the latest Rawhide got a permanent update notification
for KDE again. F34 is still around, though, so we can't just
revert to the old code, I don't think.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'desktop_pacakge_tool_update-kde-detected' needles and
'desktop_update_notification_systray-kde' needles are actually
matching on exactly the same thing, so drop the redundancy. We
need to have the desktop_package_tool_update tag on the older
(F33) version of this needle because on F33 we click on it to
launch the update tool in the desktop_update_graphical test; from
F34 onwards this is *not* what we want to do so the needle should
not have that tag to avoid throwing the test off. When F33 goes
EOL we can drop that tag from the needle and simplify the
destop_update_graphical test. Also add a needle for the Discover
app's 'update' icon when no updates are found.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's some cheating/sloppiness going on here, with the same tag
and sometimes same needle being used to match "LVM2 Volume Group"
and "LVM2 Logical Volume". Today this caused us to pick the thin
pool entry instead in a test, so let's just clean this up and do
it right, with separate needles for matching each thing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
A recent cups-pdf build changed the default filename for files
output by cups-pdf. We need to have the test look for the
correct filename based on the cups-pdf version.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The needle match seems to have changed when bug #1957858 showed
up, but it's actually just a text rendering change in the window
title, it's not exactly caused by the tiny window. So not marking
as a workaround needle.
Maximizing the window makes the test work faster when we hit that
bug, as type_safely needs to be able to see the results of its
typing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seems silly to wait 120 seconds when we know what the criteria
are. GNOME installs, can't do this; others, we have to.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some fixups for anaconda_help. Two runs of it failed today around
handoff from the root password screen to the install progress
screen; add a couple of wait_still_screens there to make it
safer. Drop the added nonlive needles, because they're too
permissive, causing problems for other tests (they're matching
before they should); instead we solve the problem of spokes being
highlighted by just pressing shift-tab a few times. And fix some
tabs to be spaces.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Had some repeated failures where there's kind of a race between
Software doing some kind of auto-refresh and the test clicking
on stuff. This seems to help.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR automates the mentioned testcase to test that Help can be
displayed in Anaconda during the installation. It navigates through
the available Help screens and if it can see it, it finishes.
This test runs after `install_default_upload` to override the
installation defaults defined for all primary tests.
Delete a duplicated needle.
Reformat list extensions to make it nicer.
Get rid of wrong export and an empty line.
Delete empty line.
Use _boot_to_anaconda for booting and move subroutine accordingly.
Add variable to templates.fif.json
Delete trailing whitespace.
Fix calling the pretest.
Move help checking to another place.
Since 232, there's been a bug where we need to hit tab three
times to get into the first field in the "Join domain" dialog.
In 245, it's down to two times, for some reason. So, handle
that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Since f33b50e, anaconda doesn't log "enabled repo:" any more. To
ensure the repo actually is enabled we need to check some other
lines. Good news is, we don't need the 'anaconda'|'' dodge any
more, so we can drop that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Boxes dropped VNC functionality. It's supposed to be replaced by
Connections, but we can't use that until it has fullscreen:
https://gitlab.gnome.org/GNOME/connections/-/issues/5
so use Vinagre for now. We do also prepare some needles for
Connections in anticipation of being able to use it later (since
I already did the work and don't want to waste it...)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
alt-f4 makes sense for an app (Terminal on GNOME) but not really
for closing the system menu (KDE). It seems like it worked till
a day or two back then broke, but I think just using Esc instead
rather than filing a bug is the best plan, I'm not sure I'd
*expect* alt-f4 to work for this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
anaconda changed how this log line looks (again); update the
check to handle old and new styles for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It doesn't seem to return to the top level automatically any more,
though the message goes away after a short time, still.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Modularity tests rely on an external script to test the modular
behaviour of DNF. There is a potentional risk that the connection
is be down and the script cannot be downloaded.
This enhancement uses a regular OpenQA perl test case script to only
invoke DNF commands and parse their output to test the same behaviour
that we have been testing already.
This enhancement picks a random module for each of the operations,
and thus tries to mimick reality a little bit more.
This adds a test that just fails if any one of a given list of
unwanted packages is installed. This was a request for the
Workstation edition from @catanzaro so I've just implemented it
for Workstation so far, but it's designed to be easily extended
to cover other subvariants too if we want.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is the same thing we did for install_resize_lvm, to address
issue #201. We just didn't get around to doing it for the blivet
test yet. We also change the HDDSIZEGB for the parent test to
15GB so the resizing stuff actually works in both resize tests;
ever since we changed this the install_resize_lvm test has not
been working properly, it hasn't actually been doing any resize.
Also drop the swap partition stuff from that test as it's for
sure no longer needed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On current F34 we get no permanent update notification in the
notifications view, we only get a *transient* one plus the
systray icon. This tweaks things so on F34 we check both of
those things correctly, behaviour on <F34 should be unchanged.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Man, this thing can get into a lot of states. Apparently somehow
it can go straight from refresh to reboot?
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE update was still often failing on #1943943, so this tries a
bit harder to work around it. We add a 'refresh' needle for KDE,
and tweak the 'retry' logic to click it if we get to that point.
Note adding the needle also changes behaviour slightly - we may
click this needle if we see it on first entering the screen. So
either change may be helping. Either way, this does make the test
more reliable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Sometimes we click the button, it cycles briefly, and...just
comes back. To avoid unpredictable failures on update tests that
have nothing to do with the update, let's try and handle this by
just clicking it till it works.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In GDM 40 the message is displayed only briefly, like in SDDM,
so we can't assert it. Only do it for <F34.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OK, extending the timeout didn't work. Try this instead. The
problem is when GNOME takes a long time to log out we wrongly
decide we're in the "DM is showing a 'screensaver' state" case
and hit 'ret' to clear it. In GDM that selects the highlighted
user. Maybe if we use 'esc', it'll still work in SDDM to clear
the screensaver state, but not select the first user in the list
in GDM...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a bandaid for GNOME taking a long time to log out right
now. I would prefer to make login_user more robust, but that's a
bit more complicated as it's used for both unlock and login.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Main UI appears over the tip of the day now, so we can ignore it
and just check the UI ran then close it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For consistency, let's just return to the desktop right away. We
also need to handle closing the overview before running installer
on live image boot.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In F34+, print-to-PDF in KDE is printing to /home/test , not
/home/test/Desktop. Not sure why. But let's deal with it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We need to hit 'restart' after applying updates, and we also
need the 'done' needle *not* to match the restart message, so
change that to match on the text (unfortunately). That also
means we have to add another variant of the needle for F32 as
the background of the text is a different color there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This gives apply and download longer to show up (which is an
issue for KDE right now) while also not waiting 10 seconds if
they don't.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a test that uses the Blivet interface to create an LVM
layout with ext4 filesystem as well as a postinstall test that checks
that the LVM layout has been created correctly.
This PR introduces a test case that uses the Blivet partitioning
tool to create a standard partitioning layout with / and /boot
(and specific partitions for UEFI and ARM64) using ext4 as
the selected filesystem.
It also adds a postinstallation test to check that the partitions
have been created correctly.
We finally saw a test where there were *no* errors logged by the
time Cockpit reached the log screen, so there were no entries to
click. Let's just make the test set log level to info before
looking for entries - I prefer this to 'click entry if found,
otherwise change log level' as that's twice as many branches to
look after. Of course, it means the warning triangle entry needle
is useless now :(
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Behaviour changed a bit in Cockpit 238, we may now just hit
success during this loop, so handle that. Also use 'break' for
the other two cases, not a big run counter bump.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We get Welcome Tour now, not gnome-initial-setup new user mode,
and it doesn't respect the dotfile g-i-s respected.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In GNOME 40, the new-user mode of g-i-s is gone and we get the
welcome tour where we would previously have seen that. This
should handle that, I hope. I probably messed up somewhere.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE in F34+ is now placing sleep, restart and shutdown buttons
right on the system menu, not in a submenu. So we need to sort of
tweak this logic. The approach here is: we count the GNOME
submenu as both a "power" and "leave" menu, so the needle to
enter it has both tags. KDE still has a "leave" submenu, but the
power options are not in a submenu any more, so the new "leave"
needle only has the leave tag, not the power tag. For "leave"
actions we just unconditionally expect the "leave" tag; for
power actions we first match on *either* the submenu tag (for
GNOME and earlier KDE) *or* the action tag, click whatever we
found, and then if we matched the submenu (not the action), we
assert and click the action. After that all paths should be in
sync again and we can continue.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is how our Pungi config has been set up since F32, so we
should match it here to be accurate.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR uses the Anaconda Blivet partitioning to recreate a partition
layout while preserving the content of the /home subvolume.
It also adds the postinstall test to check that the home has been
preserved.
This PR adds the `install_btrfs_upload` to install the btrfs based
image, the `btrfs_preserve_home_extras` to prepare and test the data
on the home partition, as well as the `custom_btrfs_preserve_home` that
uses the preinstalled btrfs image and uses its current partitioning to
preserve the home partition and the data on it.
Required because ppc64le has a PReP partition
before boot partition.
PReP partition must not be changed by this script.
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
OK, looked into it some more and ultimately we had problems here
because of https://bugzilla.redhat.com/show_bug.cgi?id=1908791
in fact. The password prompt was taking far longer than usual to
appear because pam_fprintd was failing because of that bug. That
should be fixed with next Firefox build, so I think it's best to
just leave this as it was, because in the usual course of events
it works fine and it saves having another needle to maintain.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
...nope, wait_screen_change wasn't enough. Let's just assert the
needle. Not sure if the existing one will work, if not we'll add
one.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Sigh I hate this test. We seem to be typing root pw before the
terminal is ready for us:
https://openqa.fedoraproject.org/tests/745007#step/desktop_terminal/3
Let's try this. Hopefully it'll wait for the Password: prompt
before typing, without us having to actually add a needle...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Using .local is apparently Bad Form because it's reserved for
mDNS. However there doesn't appear to be any particularly Good
Form for what to call a test domain you never want to exist
outside of a closed system, apparently. Sigh. Let's try this.
Includes a bump to disk_ks version because the kickstarts on
that image also need to have this change applied.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is another known "fails due to no hardware" case:
https://bugzilla.redhat.com/show_bug.cgi?id=1894654
those are explicitly excluded from the release criterion, so a
soft failure is appropriate.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't *need* to log out from the desktop and reboot from the
DM here, that's not part of the test (we test those features
later using jim and jack). Now we don't black out the background
of test's session in KDE, the logout needle doesn't match, so
instead of redoing that needle all the time or re-adding the
solidify_wallpaper call just to make one needle match reliable,
let's just reboot from the console.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
solidify_wallpaper only does the current session, and does it in
a kind of painful way on each desktop. For apps_startstop this
is kinda okay, but for desktop_login it's slow and error-prone
to do this three times, every time. Let's replace it with a hack
that just replaces the actual wallpaper files with a solid black
PNG file. This only takes effect after a logout, but it should
affect all logins on all desktops once it's done. So long as
the base backgrounds package doesn't change layout too much.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This makes it so the `wait_still_screen` that comes at the end
of `type_very_safely` happens *after we hit enter*, not after
we type the password but before we hit enter. I'm hoping this
makes the 'set new password at login' more robust on aarch64, it
seems to be failing often.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This allows us to use assert_script_run, and be more reliable.
Same approach used in _do_install_and_reboot postinstall stuff.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Another aarch64 robustness fix...sometimes hitting enter at GDM
just doesn't seem to work, let's give it three tries if needed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This matches the wait in boot_to_login_screen. The needle can
match before the UI is really done loading, and if we don't wait
long enough we wind up hitting enter before GDM is really ready
for us. This seems to be affecting the test on aarch64 quite
badly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Typing into a desktop terminal is a lot less reliable than typing
into a VT. We're seeing failures here quite often on aarch64, so
let's try doing this stuff in a VT instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I tried to bump it before, but set it to 90, which is the default.
Sigh. So this is an actual bump. It looks like until 20201124 this
took about 80 seconds, now it's taking like 93.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The review for `btrfs_preserve_home` test case has revealed,
that the way how to reach the mountpoint textfield in the Anaconda
partitioning differs between various tests. This PR makes it the
easiest way possible, as is defined by `custom_with_swap` test
case mentioned in the review.
It seems the message got moved to anaconda.log in Rawhide. I
think it should be fine to just check both.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR add the installation test which uses Standard partitioning
with an ext4 filesystem to cover one of the new requirements as
described in issue #202.
Fixed after a review
It still seems to be broken in 233 and 233.1; I limited the
workaround to 232 at first as Cockpit are usually good at fixing
things very fast, but as this one has sat for a while, let's
leave it worked-around until we know it's fixed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When Fedora went to BTRFS as a default, we lost the LVM based image to
run LVM resize tests with.
This PR introduces the `install_lvm.pm` installation test that creates
an LVM based qcow2 image to be used by follow-up tests.
I noticed a pattern lately of VNC tests failing on Rawhide when
we have a debug kernel (but passing with a regular kernel). On
closer investigation I think there's simply a screen blank
happening if the install process takes more than five minutes,
and that's more likely with a debug kernel. This extends the
loop we use to move the mouse every so often while waiting for
the install to complete (which is meant to defeat this sort of
thing) to also click the mouse, when we're a VNC client test. In
a quick check this seemed to help.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Update a needle with slightly different text rendering, and add
a workaround to hit tab three times rather than once on entering
the "Join a domain" screen, see
https://github.com/cockpit-project/cockpit/issues/14895 .
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The weird bug turned out to be caused by an internal DNS zone
in the new infra not being signed:
https://pagure.io/fedora-infrastructure/issue/9411
This is now resolved, so we can drop the workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Revert "Regenerate grub.cfg for ppc64le Silverblue to boot (step 2)"
This reverts commit d384cfed30.
Revert "Regenerate grub.cfg for ppc64le Silverblue to boot, brc#1817004"
This reverts commit 8d7be9a227.
Not required anymore for f33 (only for f32)
And bad side effect for f33 (failure not analysed)
eg: https://openqa.stg.fedoraproject.org/tests/949783#step/_do_install_and_reboot/32
Keep correction to avoid warning in autoinst-log when ABRT var not defined.
Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
We've had this 'exception' for mcelog.service failing in here for
years. Looking into it, it seems to now be fixed:
https://bugzilla.redhat.com/show_bug.cgi?id=1526725
and hasn't happened in our official instances for years (I guess
because they're all Intel boxes). However, we have a similar case
on ppc64le with hcn-init.service failing spuriously:
https://bugzilla.redhat.com/show_bug.cgi?id=1894654
so I'm just converting it into a workaround for that instead. We
could wire this up to be more sophisticated, with some kind of
array or hash of services that are allowed to fail and more
complex checking code, but let's not bother unless/until it's
necessary.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So, there's a problem with how we figure out the NetworkManager
connection to use in setup_tap_static: it expects the first
connection in the list to be the right one, but this is only
actually true so long as it's *active*. When we're in the tap
case, it's usually not going to actually *work* out of the box
on boot (or else we wouldn't need setup_tap_static at all...),
so some time after boot, NetworkManager gives up on it and marks
it as inactive. And after that, setup_tap_static won't work any
more.
I never noticed this as a problem before because usually we do
setup_tap_static before that point. But it seems in the vnc
client tests, on aarch64, desktop boot and login is slow enough
that by the time we switch to a VT and try to setup the network,
we're very close to that cutoff, and sometimes miss it.
This, I hope, avoids the problem by doing the network setup in
that test before we deal with the desktop login, then doing the
desktop login, then doing the actual VNC bits.
The alternative here would be to figure out a better way to do
setup_tap_static, but I can't.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seems like this often fails when booting the desktop disk image
on aarch64 if we start typing right away.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems like it can be *really* slow on aarch64, since 218:
https://github.com/cockpit-project/cockpit/issues/14840
this should give it a total of 180 seconds on aarch64 (90 second
still screen timeout plus 30 second assert_screen timeout, with
1.5x scale).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This sets us up to test the release-blocking aarch64 disk images
(Minimal, Server and Workstation). It also allows for testing
armhfp disk images on aarch64 worker hosts (though my testing of
that isn't going too well so far), and fixes the initial-setup
handling for a change upstream ('use password' is now the default
so we don't need to choose it). We rewire disk image deployment
test loading to work through the generic loader code rather than
using ENTRYPOINT, as it allows us to more gracefully handle
graphical (Workstation) vs. console (Server, Minimal), moving
the code for handling console initial-setup to a helper function
just like the code for gnome-initial-setup and having _console_
wait_login call it when appropriate. We also tweak desktop_vt a
bit because now we need to switch from a console running as test
to a desktop, which breaks the assumption that the highest
numbered session of user test is the desktop...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I noticed today that if we deploy FreeIPA with dnssec validation
enabled, dnf can't resolve dl.fedoraproject.org afterwards, which
is a problem because it means we wind up falling through to
random mirrors for metadata and package download once the server
is deployed, which can be slow and give old packages. This seems
to be why the server upgrade test on F33 is sometimes failing
because we get an older FreeIPA package on upgrade, even though
the newer one has been stable for a week.
It's difficult to pin down exactly where this bug is and fix it,
I've mailed some folks to try and work it out, but until that's
figured out, let's just disable dnssec validation.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We've been getting failures lately on the first page load, I
think because Firefox is getting even more grindy on startup. So
turn the 'sleep' into a 'wait_still_screen', extend another wait,
and tweak the 'browser' needle so it only matches after the
bookmark bar has loaded rather than as soon as half the chrome
appears. Also make all the wait_still_screens use similarity 45
for consistency (flashing cursor could be there on any of them).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This check wasn't working, the test passed whatever wait_serial
found. This version suggested by defolos works, I checked.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is the best option I can come up with to deal with #195.
Update notifications seem to have become transient in KDE lately
(even in F31 and F32, if I'm looking at these screenshots right).
This actually simplifies things a lot to do more or less the
same in the KDE and GNOME paths: open the 'permanent' store of
notifications (in GNOME you get to it by clicking on the clock,
in KDE via the systray) and then look for no notifications (live
path) or only an update notification (post-install path). We
only run this test for composes so we shouldn't need to worry
about anything older than F32, and I believe this should work
for KDE in F32 and F33. I left out click_unwanted_notifications
for now as I'm hoping it should be unnecessary, but we can add
it back in if necessary.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This does some of the things suggested by cheimes in
https://bugzilla.redhat.com/show_bug.cgi?id=1880628#c24 . It
seems to make the replica tests work with resolved, still work
with pre-F33 resolving, and not break anything. Also remove the
workaround to disable resolved if it's running, as we can now
work with it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't need a separate 'welcome' needle because it just matches
on an OK button anyway. So turn that needle into an OK needle
(we don't have any existing 'blue OK button' needle) and simplify
the logic to a single loop for kde_ok and krusader_settings_close.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On ppc64le it looks like this test is often failing because it
takes a second or two to update the partition list after we
click update settings, but we're not waiting for that, so we
wind up clicking in the wrong place because we match the next
partition needle before the list is refreshed but click after
it's refreshed. Let's hope these waits solve it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Having systemd-resolved in use seems to cause problems for
FreeIPA servers:
https://bugzilla.redhat.com/show_bug.cgi?id=1880628
until the scripts are enhanced to do this or something, let's
disable it before server/replica deployment.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
ipa-replica-install already changes the DNS config to use the
local bind instance, we don't need to do this and it's actually
wrong (as it bypasses the local BIND we should use and uses
the VM host's DNS servers instead).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seems what they had before worked until systemd-resolved became
the default; now we need to make sure we do nmcli mod and then
bring the connection down and up, as we do in tapnet.pm. Writing
to resolv.conf is kinda "wrong" for resolved but I don't think
it really breaks anything so I think I'll just leave those bits
in until F32 goes EOL just in case they're still somehow needed
on F31 or F32.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://openqa.fedoraproject.org/tests/667693#step/desktop_browser/8
shows us matching on Save File when the window is in kind of a
borked state; we'd probably wind up clicking on Open with,
because by the time we click the content of the window will have
moved to where it's actually supposed to be...so let's try this
to slow it down a bit.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Getting some odd failures where the downloaded file doesn't show
up in the right place which I think might be due to over rapid
clicking here. Try and slow it down a bit.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This should work even if the ifcfg plugin is not present (hi,
CoreOS) or 'predictable' (har) network names are on.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a complex bug in current Rawhide affecting the database
server test; it boils down to "deployment fails because LANG is
set to a locale for which the corresponding langpack is not
installed". As we know broadly what's going on there now, let's
work around it with a soft failure so we catch any later bugs in
the process.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The time before the ssh key provision request goes through turns
out to be kinda unpredictable, so instead of just a hardcoded
wait then assuming it should succeed, let's do a loop-y retry
thing instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The FreeIPA UI change that the previous commit adapted to is in
4.8.9. That's stable for Rawhide and F33 already, but still in
testing for F32, and won't go to F31. So we need to make the
change conditional on release number, and we also add the update
to workarounds for F32 so we don't have to do something awkward
while we wait for it to go stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OTP field was moved into the last position in the password change dialog
to prevent issues with OTP code expiring while users enter their
passwords.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
GNOME now also splits 'Restart...' and 'Power Off...' as KDE
does, so we need to tweak the conditional and add some needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In Fedora 33, we generally no longer include a disk-based swap
partition by default (instead swap-on-ZRAM is used, see
https://fedoraproject.org/wiki/Changes/SwapOnZRAM ). This tweaks
our tests to account for that. In tests that aren't to do with
swap at all, we stop including a swap partition in order to be
closer to the default layout. We replace the old _no_swap blivet
and custom tests with _with_swap tests that, as the name implies,
*explicitly include* a swap partition, and adjust the postinstall
test to check the disk swap partition is there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
30 seconds doesn't seem to be reliable enough. Let's try 60, if
that's not enough I'll try and think of something smarter.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Just repositioning the mouse appears not to be enough to prevent
the sesssion going idle any more, since the 20200731.n.0 compose.
Not sure what causes this, probably the kernel. Adding a space
keypress seems to help in both KDE and GNOME.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a bit complex to automate, because we cannot really use
the production Zezere server (provision.fedoraproject.org) as
the test case shows, as we'd have to solve authentication and
we also don't really want to constantly keep registering new
hosts to it that are going to disappear and never be seen again.
So, instead we'll do it by setting up our *own* Zezere, and
provisioning our IoT system in that. We run two tests. The
'ignition' test is the actual IoT 'device'; all it really does
is boot up, sit around, and wait to be provisioned. The 'server'
test first sets up a Zezere server, then logs into it, adds an
ssh key, claims the IoT device, provisions it, and connects to
it to create a special file which tells the 'ignition' test
everything worked and it can close out.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is to make the infra folks happy, apparently using 10.0.x.x
and 10.1.x.x is causing conflicts since our actual infra network
uses those ranges too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It actually is supposed to be installed by default, so if it's
missing that's a bug. It's been added to comps now so it should
be there from now on.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It got split out and is not installed by default in 33-0.8. This
is intentional, not a bug, see https://pagure.io/fesco/issue/2114
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test that automates
https://fedoraproject.org/wiki/QA:Testcase_Clevis. It requires
os-autoinst-4.6-18.20200623git5038d8c or newer, and a worker
host in the 'tpm' class which is set up to have an instance of
swtpm running at /tmp/mytpmX , where X is the worker instance
number, for each worker. The Fedora infrastructure ansible
plays have been updated to handle this via an instantiated
systemd service, which other instances can also adopt.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
`cp -R foo/*` doesn't get all files in `foo/`, it misses hidden
files. This turns out to be a problem with recent anaconda, as
it expects to find a .treeinfo file here. So, let's use rsync.
We could probably do this with cp too but I can't think of the
right arguments right now...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a pair of tests, one which does almost all the work
from the test case, the other just a client test to check that
we can connect to an HTTP server running in a container on the
host. We also have to bump the _console_wait_login timeout on
this path a bit as we're booting a disk image that was installed
with DHCP working, but we change the network setup so DHCP does
not work any more, and the system spends quite some time trying
to bring the network up on boot before eventually giving up and
proceeding.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Here we are creating ~/.config for a newly-created user with root
ownership. We can't leave it that way, as commands run as the
user account won't be able to change it, as they should be able
to. So we need to change the ownership (and, just in case, fix
SELinux contexts) afterwards.
This was the real source of the problem we were seeing (the test
failing early due to the gsettings command which should turn the
screen background black failing).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're trying to launch stuff the instant we see a desktop, and
it seems to be failing quite often in GNOME. Let's give it a few
seconds.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
IoT does nightly Branched and Rawhide composes that are built
as RC candidates, for some reason. So let's except it from this
check.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I started out trying to fix os-release for the recent change to
add "Prerelease" tags to the VERSION and PRETTY_NAME fields, then
things spiralled. It got me thinking about the awkward DEVELOPMENT
variable we use, so I decided to get rid of it and refactor the
few things that use it. I refactored the anaconda prerelease tag
check, and wrote a new giant comment that gives details about
exactly how anaconda decides whether to show those tags, to give
context to our choices about when to expect them. This check now
uses a new LABEL variable the scheduler now sets. I also wound up
creating new UP1REL and UP2REL vars to define the 'source' release
for upgrade tests, separate from CURRREL and PREVREL, which are
now never lies - they really are the current stable and previous
stable release, even for update upgrade tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The update live image build test keeps running out of disk space.
We've bumped the minimal disk image from 12GB all the way up to
20GB so far but it keeps happening. So let's try a different
strategy: use a scratch disk to mount /var/lib/mock. That's where
all the space gets used. This should allow us to reduce the size
of the minimal disk image again, and giving it 25GB of empty disk
should avoid it running out of space again for a while.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is failing on every update and that's not telling us anything
useful - we already know about the bug - so let's work around it.
Not adding a softfail as it's a bit more awkward to do that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can get tripped up by the tutorial screen when launching
Boxes; borrow some code from the app start/stop test to check for
and handle it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The default timeout for check_screen is 0, so we were only giving
the enter key press a fraction of a second to take effect before
expecting to see locked_screen_switch_user. This is too tight,
see https://openqa.fedoraproject.org/tests/586257 . Let's give it
five seconds before we give up.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Since GDM shows the "system-menu-button", it could not correctly
switch users on a locked screen. I added a check to see
if we are on a locked screen and behave accordingly.
This adds a new test that implementsQA:Testcase_desktop_login
on both GNOME and KDE.
While working on this, we realized that the "desktop_clean"
needles were really "app menu" needles, and for KDE, this was
a duplication with the new "system menu" needles, because on KDE
the app menu and the system menu are the same. So I (Adam)
started to de-duplicate that, but also realized that "app menu
button" is a much more accurate name for these needles, so I was
renaming the old desktop_clean needles to app_menu_button. That
led me to the realization that "check_desktop_clean" is itself a
dumb name, because we don't (at least, any more, way back in the
mists of time we may have done) do anything to check that the
desktop is "clean" - we're really just asserting that we're at a
desktop *at all*. While thinking *that* through, I *also* realized
that the whole "open the overview and look for the app grid icon"
workaround it did is no longer necessary, because GNOME doesn't
use a translucent top bar any more. That went away in GNOME 3.32,
which is in Fedora 30, our oldest supported release.
So I threw that away, renamed the function "check_desktop",
cleaned up all the needle naming and tagging, and also added an
app menu needle for GNOME in Japanese because we were missing
one (the Japanese tests have been using the "app grid icon"
workaround the whole time).
This stuff is kinda broken in various ways and halfline thinks
he can fix the underlying bug anyway. So let's go back to just
the GNOME live test being broken for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I tested this workaround on staging before pushing it to git and
it worked, but then when I pushed it to prod it didn't work. On
stg I also had this to set GDM to debugging mode, so maybe this
is also needed for the workaround to work for some reason?
Signed-off-by: Adam Williamson <awilliam@redhat.com>
A GNOME bug seems to result in us getting to GDM, not a liveuser
desktop, after running 'systemctl isolate graphical.target' from
a live boot to runlevel 3 since the end of March. This works
around that to let the test run, as it's not really a failure of
the test per se.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Previous commit same summary had some side effect
solved by this new one.
And avoid a warning in autoinst-log when ABRT var not defined.
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
I merged the previous commit before realizing the ordering was
wrong. All other 'actions' lines have to come *before* the one
that adds 'reboot', because one of the conditions for that is
whether @actions is populated - basically, if we're taking any
actions, we also have to reboot afterwards. If we add an action
*after* that line (but no actions were added before that line),
we'll do it but then not reboot and the test will break.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit 00b756f0e2.
Unfortunately, I made a typo in the script and the fix did not
work. I do not want to rebase the master (in order not to break
things for everyone) so I am reverting again.
Sorry.
This reverts commit d784bf54ca.
It turned out that Locations are not connected to Konqueror
at all. The reason why the test is failing is that the
application has been removed to limit the number of
web browsers.
We seem to be seeing the bug this works around:
https://bugzilla.redhat.com/show_bug.cgi?id=1765685
in F30 and F31 update tests even with this wait. At least, it
looks that way. Trying this to see if a longer wait helps.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
RHBZ #1692972 was fixed long ago, so we don't need to worry
about that any more. But this test failed on the recent F31 live
respin compose because it was changed to assume the tutorial
would appear on startup, which only happens on F32+. To make the
test work on F31 respins, let's handle both paths. Once F32 is
stable we can drop this as we won't run the test on F31 any more
after that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
IoT created a branch that's basically Rawhide but is versioned
33. This causes the release_identification tests to fail. I don't
think they'll change this on their end, so let's just have the
test cope with it and expect branches versioned as the Rawhide
release number to behave as Rawhide does here.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems to be actually installing fedora-release-silverblue now
so we get correct identification here. Update the tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://bodhi.fedoraproject.org/updates/FEDORA-2020-1070052d10#comment-1284223
It seems the rootfs on the Fedora 30 installer images we build at
present has gotten very big, so big that an update which contains
some very slightly larger firmware packages causes the rootfs to
be completely full (though lorax doesn't fail) and the image
doesn't boot.
I don't yet know when or why the rootfs got that big, but it's
not really a bug in this update, so for now let's just tell
lorax to use a bigger rootfs so the tests pass for this and any
similar future updates, until I can maybe find time to pinpoint
the culprit more precisely.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Initial implementation wasn't correct, I forgot CURRREL is not
'the pre-upgrade release version' but just 'the current stable
release'. This is a dumb way to figure out the correct release
number for this context but off-hand I can't think of a better
one.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When there's a failed service we get a stupid bullet-point char
at the start of its line, and all the other lines are space-
padded to match indentation. Which is annoying! This (I hope)
ditches that crap without losing anything of value.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also comment this better. We need to index from the end of the
string here, not the start, because going from the start breaks
when the compose shortname has a dash in it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Per https://github.com/weldr/lorax/pull/881 it wasn't doing
anything anyway, and using it causes the command to fail in F32
and later.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://github.com/rpm-software-management/mock/commit/cb25d3ba
changes existing mock configs for stable releases to have a
`dnf.conf` section instead of a `yum.conf` section, and this
change got pushed out to F30 and F31, which breaks us :( Our
mock config that we use for building live images assumes the
existence of a `yum.conf` section in the config it inherits
from.
This change is now stable for F30 and F31, so at least we don't
have to do any conditional shenanigans; we can just change to
'the new style' unconditionally and things should work OK.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
IoT is becoming a release-blocking edition for F32, so we should
be testing it for sure. We may add specific tests, but for now
let's run the install and base tests on it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On graphical flavors we are not at a console when this test runs.
We need to do root_console to get there, and also bypass_1691487
for ppc64le. Copied from base_selinux.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is failing a lot lately. I've no idea why, but it's not
really part of the actual test and we don't need it for debugging
all the time, so let's drop it for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The text we used to get has been replaced with a spinner, which
is difficult and unreliable to match on. This match was only
here to make the test fail a bit faster if it was broken, so
let's just live without it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're getting issues with KDE's crazy-slow alt-f2 behaviour
here. So try and be even more conservative.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This match keeps giving us problems, and now I look at the test
again, there is actually no need for it at all. Clicking it
doesn't do anything, and we already confirm that we're on the
right page at the next step, where we look for a log entry and
click on that - that will fail if we aren't actually on the
Logs page.
I don't remember what Cockpit used to look like when we first
put this line and needle in, presumably there's a reason we had
them, but they're clearly unnecessary now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
wait_idle is finally removed upstream in recent git os-autoinst.
This replaces all remaining wait_idles with sleeps, except for
one which is removed (I'm hoping improvements to typing in the
last few years should mean it isn't necessary any more, if it
turns out to be, I'll put it back as a sleep).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This failed a couple times in a row on KDE live, let's see if
wait_still_screen is better than sleep.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We need a couple of new needles, plus the 'join domain' button
has disappeared from the front page due to the very inefficient
UI redesign, so we need to scroll down to find it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lately launching anaconda on the KDE live image seems pretty
unreliable and we're not sure why. My last attempt to fix it
doesn't seem to be working, here's another effort based on the
idea it might be caused by moving the mouse from the hidden
position to the icon and back again, let's try moving the mouse
close to the icon before we assert and click it...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We still have a 'apps_run_firefox_stop' needle tag which is for
the same thing as 'firefox_close_tabs'. That's dumb. Get rid of
it and only have the firefox_close_tabs tag and needles. Also
clean up some old firefox_close_tabs needles that haven't matched
for months and all the 'apps_run_firefox_stop' needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE tests are quite frequently failing lately because anaconda
doesn't launch. I'm hoping this will help.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
autocloud is dying soon. So, we want to run the tests in openQA
instead. This adds a test module called 'autocloud' and a test
suite called 'cloud_autocloud' which basically replicate what
autocloud does (download a tarball full of tests and run each
one), and the necessary template bits to run it on Cloud_Base
qcow2 images.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
...otherwise the VNC client tests fail on aarch64 because we try
to apply the 'console=tty0' workaround for #1661288. Fortunately
we don't really need that for the VNC install test to work, so
let's just skip it. We can make this more sophisticated later if
it turns out to be necessary.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
os-autoinst has this code when doing `script_run` on a serial
terminal that does a `wait_serial` for '# ' before running the
command. For some reason when we switch to the anaconda root
console and run a couple of commands after finishing install on
ppc64, the first of these prompt checks times out, which means
the test sits there doing nothing for 90 seconds unnecessarily.
Let's try and avoid that by hacking the prompt check regex to
be empty.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There is nothing inherently 'root'-y about these so it makes no
sense to prefix their names with 'root-'. And why change from
'console' to 'terminal' compared to the naming used in the
actual qemu command and the log files? It's just confusing.
Let's be consistent (except for using - instead of _ here...
but - is easier to type!)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds tests for vncconnect, a variation on VNC install using
reverse VNC (where the client waits for the server to connect to
it). Very similar to the regular-VNC test, but we have to use
tigervnc as Boxes doesn't do reverse VNC.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.
There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:
https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34
we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't know exactly what's going on with this bug yet, but as
a 15 second sleep seems to avoid the issue, let's do that for
now so we catch any future issues.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
See https://openqa.fedoraproject.org/tests/473215 - it failed
because we tried to click FINISH CONFIGURATION while it was
still animating downwards.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The way this works at present, with a check_screen and then an
assert_and_click, there's a window where the check has passed
so we're committed to the assert, but it takes a half second or
so for the assert to actually complete (checking for a needle
is a somewhat heavy operation). During that half second the
'new update!' notification can...and quite often does...appear.
Changing the assert_and_click to a click_lastmatch should (I
hope) tighten this window; click_lastmatch should fire faster
than assert_and_click so there'll be less of a window for the
update notification to appear and break stuff.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:
* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
text is untranslated (e.g. 'Fedora 31') it appears on a
different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
area of the banner this can trigger an os-autoinst needle
comparison bug: https://progress.opensuse.org/issues/56822
All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR fixes#124.
It uses the BUILD variable to get the build string from the
tested compose and uses it to compare correct values for Silverblue.
Originally, it used Workstation values that did not match the
Silveblue ones.
Delete bad yank.
One of the test cases we didn't yet automate is:
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg
Now we have a PXE test, it's actually a good opportunity to test
that at the same time. I don't usually like combining tests like
this but in this case it sort of makes sense as otherwise we'd
have to have a whole parallel PXE install just to test this one
other detail. So, instead of doing an interactive PXE install as
we did at first, let's tweak the test to include a kickstart in
the initramfs and run the install from that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.
Review cockpit.pm
Review autoupdate test.
Review the update test.
Fix typo in cockpit.pm
Add sleep.
Add missing command.
Delete an unused needle.
mock is changing the config files to be based on templates,
which breaks how we generate the live build mock config. This
should work with both the old and new config files.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is causing all kinds of trouble, because when the test is
run on the Server DVD - with the 'orange to blue' gradient - the
prerelease note is dark red text on a dark blue background.
os-autoinst actually reduces the color depth of images/needles
and greyscales them before performing the match...but for this
dark red text on dark blue background, the result seems to be
that the text and background come out *the same grey*, so *any*
text will match the needle (even if it's completely different
text), as will *no text at all*. I've tried finessing around
this a few times but it just keeps happening, so for now I'm
just disabling the pre-release text check at this point. We still
have the check during _boot_to_anaconda, when the text appears
on a *grey* background and so isn't a problem. I'm not removing
the needles yet, until we hear back from upstream:
https://progress.opensuse.org/issues/56822
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Building images with SELinux in permissive mode worked fine in
F29 and F30, but it seems to be broken in F31. Releng seems to
have it set up so the build environment has SELinux disabled,
so let's do the same. Will file a bug for this, as bcl says it
should be considered a bug.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is doing slightly less to exercise the launcher menus and see
whether icons appear or disappear, but it's much faster and more
reliable. We do still use menu launch for one app, just to check
the mechanism works in general.
This adds a whole wodge of stuff to support_server to make it
act as a PXE server, then adds a new test which boots from PXE
and so should hit the PXE server. We use the NFS install repo as
that can be relied on to work for a support_server install.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We do this in quite a few places and need to do it in another,
so let's just have a function for it. It takes a file glob
so we can have it run on a different one for _live_build.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Split this out of install_default, because it really is not a
part of that test and we do not want that test to fail because
the desktop background is wrong. Make it its own test module
and test suite instead. Don't do it on Rawhide, because we
really can't assert anything worthwhile about Rawhide at the
moment at least (this means the test runs but is a no-op and
will always pass on Rawhide, unfortunately). Move the needles
to a more appropriate location (this has nothing to do with
anaconda) and use 'background' not 'wallpaper' naming (that's
the name we use elsewhere in the project, e.g. package names).
Also, run the test on updates, and add an F29 needle for this
purpose.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
cockpit test often seems to fail because the click on a services
entry to open the services detail screen just gets lost. Let's
wait longer before trying it, and retry once if it fails.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The user and root spokes were moved from during-install to pre-
install hub in Rawhide. This should cope with that, while still
working for older images.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There are three places where we basically want to click away
pop-up update notifications and the buggy akonadi_migration_agent
notification if it's there, in KDE tests. Let's share this code
between them, and also let's record soft failures for the buggy
cases in the desktop_notifications test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Current KDE seems to like showing us multiple update available
notifications. So the test must dismiss all of them. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1730482
Signed-off-by: Adam Williamson <awilliam@redhat.com>
* Add test to check module defaults.
* Add whitelist download.
* Fix install test to include selected profile to be on the safe side.
* Add test into templates.
Cockpit 198 comes with a UI overhaul, so almost all needles
need an update.
The 'auditd' service is no longer on the first page. To make
this less fragile (at the cost of not testing that clicking on
a service actually opens the detail page *for that service*,
tweak the needles to just look for *any* running service, click
on it, and check we got to a 'details' page. We also redo the
existing needles for this design.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I'm pretty sure I worked around the modifier bug in os-autoinst,
so this shouldn't be necessary any more. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1727388
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can run these tests much like support_server - have a test
which boots from the support_server disk image but with the ISO
to be tested attached, then mounts it and runs the scripts.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
@lruzicka added this to the KDE app tests in 71c4e273, but there
was no need for a new needle as I'd already done the same thing
in the desktop updates tests; let's just use the same needle.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This shouldn't be needed any more (the referenced bug was fixed
in F28) and doesn't work in non-live installer.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This whole block where we do various things at the console after
install completes was becoming a real mess. I had secret hopes
of killing it entirely at some point, but...that doesn't look
like it's gonna happen this century. So let's make it better
instead. The conditionals were getting very nested and icky and
it was hard to see what was actually going on. This rationalizes
things so first we figure out all the things we might want to do
at a console, then if we don't have anything to do at a console
we go ahead and hit the reboot button; otherwise we go to the
console and do all the things we need to do, including rebooting
unless this is the memory check test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This just doesn't work when running the test on a graphical
install because we're at a desktop when we try and do it, not
a text console. So let's only try to do it if there is no
DESKTOP. See e.g.:
https://openqa.fedoraproject.org/tests/408484#step/base_selinux/2
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The way KDE does update notifications has changed - it's now a
permanent pop-up notification. This is a bit awkward for our
logic; it's hard to define a needle that proves this pop-up is
the only notification. Instead, let's dismiss it, then open the
notification tray and assert that there aren't any others. But
we also retain the old behaviour (more or less) for testing old
releases.
The popup notification also blocks the 'refresh' needle in the
systray and so breaks the desktop update test, so we deal with
that too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So, turns out new os-autoinst does *not* still accept the old
argument style for assert_and_click...and old os-autoinst
doesn't accept the new one. This adds a wrapper that handles
both, so our tests can work with old and new os-autoinst. We can
drop this once both deployments are on newer os-autoinst.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In the new os-autoinst I just sent to staging, the old style
doesn't work any more, breaks all tests. This style should also
work with the older os-autoinst on stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
'use password' now seems to be default when creating a user in
text mode, so we need to handle that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was necessary for debugging the FreeIPA 4.8 pre-release
update bug, so let's have it for all runs, just in case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems that when this problem happens now we get *two* auth
requests, so we need to handle that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
File doesn't exist for ostree installs. No point failing if this
fails, we may as well just continue and see what happens.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
No purpose is being served by all update tests failing on this
bug, so let's try and work around it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is covering the NFSISO_variation test as per:
https://fedoraproject.org/wiki/QA:Testcase_install_repository_NFSISO_variation
Change support_server to use support_5 disk image in templates
supposed to be previously created by updated createhdds tool
required for new test.
Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
Since the KDE menus have transparency set, any time the wallpaper
changes the menus will look different, and the app tests may
fail. This sets the desktop wallpaper to black at the start of
the test suite to avoid this problem.
It seems that there was a lag in button drawing, which prevented
OpenQA from fully recognizing the krusader_welcome needle because
it was evaluated too soon.
I added "wait_still_screen 3" to make sure, there is time for a
button to be drawn correctly before assertion.
Change method from assert_script_run to script_run
Change to reboot from console.
Move code in submethod, use ifs to make sure everything runs.
Fix chroot rebooting problem.
In the last few weeks Boxes is showing an auth dialog on start.
I've filed a bug on this; let's have the test handle it as a
soft failure, since this isn't a fatal problem. Do this by
making an existing needle for this dialog a bit more generic
in name and using it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The KDE and GNOME 'apps' tests for Firefox both invented their
own 'oh look Firefox is running' needle, even though we already
had one. The GNOME one was broken by the removal of the app
title bar in Firefox 66.
Instead of having three needles for the same thing, let's just
throw the 'apps' ones out and use the pre-existing one from
needles/firefox for all cases.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Add png files as a background for the needles.
Rewrite the test handling methods to make them ready for KDE
Add the ABRT startstop test.
Make changes and corrections to the abrt test.
Add test for Firewall.
Add test for DNF dragora.
Add wait time for dnfdragora test.
Correct syntax.
Add Language test.
Make some changes to the DNF dragora test.
Add Users test case.
Add needles for DBUS viewer test.
Add Dbusviewer test.
Add Mahjong test and needles for games.
Add Minesweeper tests.
Add Patience test.
Add test for Document Viewer.
add test for gwenview
Add test for koulourpaint.
Add test for Kruler
Add test for Kcolorchooser
Reneedle failing needle.
Add ktorrent tests.
Add tests for CPT editors.
Add test for Krfb
Fix names for those files.
Add test for Kget
Add Akregator test.
Add test for Konversation.
Make Konversation end really.
Add tests for Kmail
Add test for PIM exporter.
Add test for KTnef and Krdc.
Fix problems after test runs.
Make more tries.
Fix needle to be found better.
Fix more errors.
Add test for Falkon.
Add tests for browsers.
Add support for closing tabs into Firefox test.
Add tests for K3B
move needles to correct directory
Add Kaddressbook test.
Add Kontakt text.
Add test for korganizer.
Add menu office needle and correct konqueror needle.
Add test for calligra stage.
Add test for Calligra.
Add test for network connections.
Modify needle for kaddressbook to prevent failing.
Add test for system settings and fix others.
Add test for FMW.
Add test for Dolphin
Add test for Infocenter.
Add test for kparted.
Fix a wrong needle.
Test relnotes.
Fix some errors in tests.
Add test for Discover.
Add test for Ksysguard.
Add tests for Konsole.
Add tests for KDE wallet.
Add tests for several utilities.
Add Krusader test.
Finish utility tests.
Fix some errors.
Fix needle for spectacle.
Add wait time to let Dragora wait for network.
As of Fedora-30-20190316.n.1, gnome-abrt is back under the
Utilities menu and the 'Sundry' menu seems to have disappeared.
Change the test back to using utilities and drop the sundry
needle.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems 3 secs was a bit tight for recent Branched and Rawhide,
test was failing when the screen was just a bit slow to update
for some reason.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Sigh. It seems sometimes the button isn't visible because GNOME
pops up a fricking notification that blocks it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
As there is *always* a base repo error when booting a netinst
image, this check was giving us spurious failures when running
universal tests on a netinst (which we are currently doing on
ppc64 it seems). Tweak it a bit to hopefully avoid this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
wait_idle was deprecated and effectively it just sleeps, so
replace these wait_idles with explicit sleeps - I can't think
of a better choice here as we just need to wait for Firefox to
calm down after various operations.
The wait_still_screens were always hitting the 30 second time
out because of the flashing cursor; setting the similarity level
required to lower than the default 47 should fix this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
ctrl-t seems to be failing often in Firefox tests, and I can't
figure out why. Let's try clicking the 'new tab' button instead
as a workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Just like the installer image build test, only...it builds a live
image. This involves reimplementing quite a chunk of the Koji
livemedia task. Ah, well. Also involves rethinking the flavor
names a bit here, these seem...better.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're getting a spurious "Authentication required" for a
background refresh of the system repos sometimes, depending on
whether it happens while we're at a tty. This is now accepted
by upstream, so let's handle it and make it a soft failure for
now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We quite often want to run the update tests on a Koji task (not
a Bodhi update) for some reason - usually to test a potential
fix for an issue, or at a maintainer's request to test a change
before it is merged upstream and officially sent out as an
update. Up till now I've always hacked up utils.pm on the
staging server by hand to do this, which is horrible. Together
with a commit to fedora_openqa, this should allow us to do it in
a nice, sane way via the CLI. It's mostly just tweaking the
"updates" repo setup in utils.pm as you'd expect, but there's a
bit of subtlety to it because of the installer tests that use
%ADVISORY% as a variable substitution in the disk image name;
you can't do something like `%ADVISORY or KOJITASK%`, sadly, so
I had to have almost-redundant variables ADVISORY, KOJITASK and
ADVISORY_OR_TASK (we could kinda just live with ADVISORY_OR_TASK
except I didn't want to drop ADVISORY as it's an unnecessary
change from previous behavior).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
inst.addrepo was added in F29. In F28 it should just sorta not
do anything at all. It's not worth doing a big workaround to
make F28 graphically add the repo in this case; let's just allow
F28 to not actually add the repo. It's fine! This is fine.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became
obvious that there's a missing feature in the new installer test
for updates: the update is both used in the image build process
and built into the installer environment itself, but it is not
actually included in the installed package set. This can be a
problem if the update has a bug that manifests *only* at install
time if it is in the install transaction (which is exactly the
case there), because the test will not catch this, and nor will
any other test.
So this commit makes `support_server` set up the update repo and
serve it out via NFS when it's run in an update context, and
makes the actual update install test run parallel with it and
use that repository. This way the install should include the
package(s) from the update. (It also of course means the test
fails if an update breaks NFS or something like that, but hey,
we want to know that!)
A parallel commit for fedora_openqa is necessary to add the
required CURRREL setting for the updates-installer flavor.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test which builds a netinst image potentially with
the package(s) from the update, and uploads that image. It also
adds a test which runs a default install using that image. This
is intended to check whether the update breaks the creation or
use of install images; particularly this will let us test
anaconda etc. updates. We also update the minimal disk image
name, as we have to make it bigger to accommodate this test,
and making it bigger changes its name - the actual change to
the disk image itself is in createhdds. We also have to redo a
bunch of installer needles for F28 fonts, after I removed them
a month or so back...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There seems to be an issue in Rawhide ATM which can cause the
'beta nag' screen to take a very long time to appear. Bump the
timeout to avoid tests failing on this.
https://bugzilla.redhat.com/show_bug.cgi?id=1666112
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't want the tests to fail on this now we know what the
bug is, really - we want to find if there are any subsequent
fails, and allow the post-install tests to run also. So, let's
make it a soft failure.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I'm pretty sure we got all the bugs this was working around
fixed. Again, if not, we can put this back!
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The bug never got explicitly addressed, but it's very old and
anaconda has been substantially changed since. The workaround
sometimes triggers erroneously now (because the icon sometimes
goes black while the spoke is still in 'Checking storage
configuration...' state), which is awkward. I can't be 100% sure
the bug doesn't sometimes still happen, but if it does, we'll
notice fairly soon, and we can tweak this and put it back.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems Rawhide auto-signing is working fine now: openQA claims
this needle has 'never' been matched (which really means 'not
for a long time'), and I can't find any test fail in the last
year which looks like it landed on this 'authenticate' screen
but the needle failed to match, or anything like that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://openqa.stg.fedoraproject.org/tests/424393 is a failure
where the 'Download' [updates] button was already visible when
we went to the tab. We already checked whether an 'apply' button
is visible and skipped the 'refresh' click if so, but because
the 'download' button is a new thing, we weren't skipping the
'refresh' click if 'download' was already visible.
So in this case, even though we could already see 'download', we
went ahead and clicked 'refresh'...then *immediately* started
looking for 'download'. It seems that Software did not refresh
and remove the 'Download' button *immediately* when we pressed
'refresh' - it left the 'Download' button visible briefly, and
*in this brief window*, we clicked it. *Then* Software kinda
'noticed' we'd clicked 'Update', and it seems it just sort of
throws away our click on 'Download' at that point and does the
refresh.
So at that point, the test thinks it's clicked 'Download' and
expects to see 'Apply', but actually the 'Download' click got
more or less thrown away, so the test fails, sitting at the
'Download' button.
To solve this, let's just extend the existing check to skip the
'refresh' click if 'download' *or* 'apply' are already visible.
There is a sort of possibility here that we could wind up
downloading and installing some updates that existed and were
noticed *before* we did our python3-kickstart trick, but not
install the python3-kickstart update, and cause the test to fail
because of that, but that doesn't seem to have happened before
when we were seeing the 'update' button, so I think I'm not
going to borrow trouble. If it happens, we'll deal with it I
guess.
The comment talks only about KDE, but clearly it can be the case
that an automatic check makes the button visible on GNOME too,
so let's rewrite the comment too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The fix for this bug was sent to all releases now, so we should
not need the workaround any more. Let's kill it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is breaking the memory_check tests. I just reproduced it
manually and the UI *does* come back to life if you wait some
time; let's see if we can work around the bug this way.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
If an update test fails before reaching advisory_post, we don't
generate the 'what update packages were installed' and 'were
any update packages *not* installed when they should have been'
logs, but these may well be useful for diagnosing the failure -
so let's also do the same stuff there. Only let's not do it all
twice.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We hit an interesting case in update testing recently:
https://bodhi.fedoraproject.org/updates/FEDORA-2018-115068f60e
An earlier version of that update failed testing. When we dug
into it a bit, we found that the test was failing because an
earlier version of the `pki-server` package was installed than
the version that was in the update; when asked (as part of
FreeIPA deployment) to install it, dnf had noticed that there
were dependency issues with the version of the package from the
update, but it happened to be able to install the version from
the frozen 'stable' repo...so it just went ahead and did that.
In this case, the 'missed' package resulted in a test failure,
but it'd actually be possible for this to happen and the test
to complete; we really ought to notice when this happens, and
treat it as a test failure.
So what this attempts to do is: at the end of all update tests,
check for all installed packages with the same name as a package
from the update, and compare their full NEVR to the one of the
package from the update. If a package with the same name as one
of the update packages is installed, but does not appear to be
the *same NEVR*, we fail, and upload the lists of packages for
manual investigation as to what the heck's going on.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
From local experimentation, it still actually produces the
output, even though it prints the message about the order being
wrong and exits 1.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Weirdly, occasionally some update tests seem to fail because
the 'comm' util we use to produce the list of packages from the
update that were actually tested during the job doesn't think
one of the input files is in alphabetical order, even though we
sort them both when they're produced. I don't know if this is
possibly due to the definition of 'alphabetical order' changing
as part of the update, or what. But we really shouldn't *fail*
the test when this happens, as it's not part of the functional
test, we're just producing convenience data. So, let's handle
the command failing, and if it happens, upload the input files
so we can maybe figure out why it's unhappy...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The previous commit would lead to the 'workaround' getting hit
incorrectly, and might have had some other issues...tweak it a
bit.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
GNOME Software 3.30.5 split the offline update process into two
separate 'download' and 'apply' phases. So we need to handle
clicking 'download' before 'apply', if that happens.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Somehow, recently, FreeIPA tests are running into Firefox not
quitting because it's showing a warning about closing multiple
tabs. (I think we didn't *get* multiple tabs before but now we
do, for some reason). So let's work around this by clicking
"Close tabs" if the warning appears.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lately, we can't be sure the desktop will be on tty1 after we
do 'systemctl isolate graphical.target'. For recent Workstation
lives it actually shows up on tty2.
We could be 'clever' and switch to tty2 on F29+ Workstation
lives...but actually it seems like if we just don't do anything,
systemd switches us to the correct tty. So let's rely on that,
at least as long as it's working.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
At least one test (desktop_notifications_postinstall) boots from
the disk image uploaded by install_default_upload, and needs to
access the grub menu. On F29+ Workstation this is failing,
because the grub menu is now hidden by default, so when the test
boots, it never sees the bootloader screen, and fails.
I considered trying to teach it to hold down shift or hit f8 or
esc at the right time, but that seems like it might be hard. So
instead let's just try to disable the hidden menu when we're
about to upload the installed system image. This is kinda going
against the 'preserve natural system behaviour' principle we try
to use for openQA, but I think it's OK as we do have other tests
that will exercise the 'hidden boot menu' stuff to some extent.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On F27 we don't get a 'Software is up to date' screen because
there's an upgrade available. Let's work with the refresh button
instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're not seeing *exactly* #1314991 any more, but we're seeing
something that looks quite similar: the first attempt to find
updates just doesn't find any. No error message, no updates. I
have reported a bug for this and am investigating it, in the
meantime, let's restore the workaround, elaborated a bit, and
looking for the 'Software is up to date' screen instead of the
error message.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I rather suspect the *bug* is still basically present and it's
why this test often fails, but we no longer seem to see the
*error message* which lets us detect the bug happening. This
needle has not been hit by any test for six months. So let's
remove the workaround as it adds complexity.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For some reason, in recent tests, switching to a console after
live install completes is taking a long time, and tests are
failing because we 'only' allow 10 seconds for the login prompt
to appear. This seems to indicate some kind of performance bug,
but we don't really want all liveinst tests to fail on in, this
is not primarily a performance testing framework. So let's
tweak the root_console / console_login bits a bit to allow a
configurable timeout for the login prompt to appear, and use
that to wait 30 secs instead of 10 in this case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The FreeIPA upgrade test didn't actually check that FreeIPA is
actually running after the upgrade and reboot, it just kinda
assumed it is. Let's add a check to the start of the 'check'
test module that makes sure ipa.service actually comes up to
'active' state. This'll make it clearer when tests are failing
because FreeIPA didn't come up right after the upgrade. The
check will run on non-upgrade tests too, but that's fine.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a new test intended to just check boot chain things
for updates. It doesn't run any test modules besides the stock
update ones, but sets a variable, ADVISORY_BOOT_TEST, which
causes _advisory_update to do some additional stuff after
installing the updates but before rebooting: it forces regen
of the initramfs and bootloader config, and reinstalls the
bootloader on BIOS (not UEFI as it's not relevant). If the
following boot fails, we probably have a bug somewhere.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's this annoying problem where the screen sometimes goes
messed up after ipa-server-uninstall. 'clear' doesn't seem to
really work to fix it up either. Let's try flipping between
ttys. I don't like this much as it's already a pain trying to
work out / remember what tty we might possibly be on at any
given time, but I think we're always on either 1 or 3 here, so
let's do ctrl-alt-f1 ctrl-alt-f3 to ensure at least one change
and wind up on tty3...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Looking at this, it's a bit weird: the updated packages are
actually included in the upgrade process, but we still run
_advisory_update, which does basically nothing...then reboots.
That's kinda silly and makes the tests a bit flaky, let's fix
it. I don't think there's actually any problem with doing the
upload of updatepkgs.txt in _repo_setup_updates, becase that
already guards against being run more than once, it just bails
very early if it's already been run.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I'm going to figure out if it's a bug that it takes so long, but
for now let's just bump the timeout.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
RHBZ#1625572 is for gnome-initial-setup running in 'first login'
mode after it's already run in 'user creation' mode (which isn't
meant to happen). This works around that so the subsequent tests
can run. We don't soft-fail because meh.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
If USER_LOGIN is false we can just return; when we reach the
login screen. We don't need a huge conditional when we don't do
anything *after* it, in the false case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It transpires that decommissioning wipes some stuff, like the
dirsrv logs. Obviously we want these included in the logs we
upload for reference purposes, so let's upload earlier.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
