Now we have two areas, openQA wants to click in the wrong one.
Let's tell it which one to click in.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The main screen now also has identical "Japanese" (that's what it
says) text. To avoid false matching before the picker opens, add
another match area.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The open dialog on Silverblue (which is apparently not at all
the same thing as the open dialog on Workstation, though they
look the same) does not default to the Documents folder, so we
have to open it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These all failed to match first time the test was run in
production. I guess Lukas was working from an older release.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR fixes issue #188. It adds a test suite to test basic
functionality of Evince and brings the following features:
* test scripts for various Evince functions.
* needles to support the Evince test scripts
* new template variables `TESTPATH` and `POSTINSTALL_LOAD_ALL` (see
below)
* new logic in `main.py` (see below)
The new variables and the new logic make it easier to create test
suites for post-installation tests. If TESTPATH is used, OpenQA
will take all tests mentioned in POSTINSTALL from that specified
TESTPATH. If both TESTPATH and POSTINSTALL_LOAD_ALL are used, then
OpenQA will run all tests it can find at the TESTPATH location.
If POSTINSTALL and POSTINSTALL_LOAD_ALL are set simultaneously,
then only POSTINSTALL will be taken into account and OpenQA will
only load tests mentioned there.
Recent git os-autoinst no longer downsamples screenshots as far
as it did before comparison. This makes a lot of needles where
colors have changed slightly no longer match, so they all needed
updating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Again, to combat os-autoinst's new behaviour without adding yet
more needles. We're getting 96% match on stg.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also delete ones for F32 and earlier. This should hopefully help
us not need to have too many needles for the newer stricter
matching.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Latest Firefox has moved the underline down slightly for some
dumb reason. This tweaks the match area to only the text, so it
matches wherever the underline is.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In #235 we accidentally included an entire disks.pm test that
wasn't meant to be there - the infocenter module test is the
right thing to exercise plasma-disks, there is no standalone
app, running 'disks' just gets you the KDE Partition Manager,
which we already test. So this removes that test and renames a
needle that looks like it's for that test but is actually for
the kinfocenter module, to make this clearer.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a new test that automates the above mentioned test case.
It starts the installation in text mode using the `install_text` test
case, which it interrupts using the Anaconda crash trigger.
When the crash happens, it goes through the process of reporting
the bug to Bugzilla, checks that Bugzilla sends a positive
confirmation of the action, but also performs some REST API
calls to do a proper check and then it closes the bug to clean up.
GNOME dropped the g-i-s new user mode in F34, so on a Japanese
install with user created in the installer, you don't get an
input source configured out of the box or on first boot. So
we'll just have to do it manually after booting, before we test
if it works.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Looks like the latest Rawhide got a permanent update notification
for KDE again. F34 is still around, though, so we can't just
revert to the old code, I don't think.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'desktop_pacakge_tool_update-kde-detected' needles and
'desktop_update_notification_systray-kde' needles are actually
matching on exactly the same thing, so drop the redundancy. We
need to have the desktop_package_tool_update tag on the older
(F33) version of this needle because on F33 we click on it to
launch the update tool in the desktop_update_graphical test; from
F34 onwards this is *not* what we want to do so the needle should
not have that tag to avoid throwing the test off. When F33 goes
EOL we can drop that tag from the needle and simplify the
destop_update_graphical test. Also add a needle for the Discover
app's 'update' icon when no updates are found.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's some cheating/sloppiness going on here, with the same tag
and sometimes same needle being used to match "LVM2 Volume Group"
and "LVM2 Logical Volume". Today this caused us to pick the thin
pool entry instead in a test, so let's just clean this up and do
it right, with separate needles for matching each thing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The needle match seems to have changed when bug #1957858 showed
up, but it's actually just a text rendering change in the window
title, it's not exactly caused by the tiny window. So not marking
as a workaround needle.
Maximizing the window makes the test work faster when we hit that
bug, as type_safely needs to be able to see the results of its
typing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some fixups for anaconda_help. Two runs of it failed today around
handoff from the root password screen to the install progress
screen; add a couple of wait_still_screens there to make it
safer. Drop the added nonlive needles, because they're too
permissive, causing problems for other tests (they're matching
before they should); instead we solve the problem of spokes being
highlighted by just pressing shift-tab a few times. And fix some
tabs to be spaces.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR automates the mentioned testcase to test that Help can be
displayed in Anaconda during the installation. It navigates through
the available Help screens and if it can see it, it finishes.
This test runs after `install_default_upload` to override the
installation defaults defined for all primary tests.
Delete a duplicated needle.
Reformat list extensions to make it nicer.
Get rid of wrong export and an empty line.
Delete empty line.
Use _boot_to_anaconda for booting and move subroutine accordingly.
Add variable to templates.fif.json
Delete trailing whitespace.
Fix calling the pretest.
Move help checking to another place.
Not sure why these changed, but oh well. Utilities menu was
highlighted in a test run for some reason, so let's just handle
that. Other needles changed very slightly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're not using them, so they fail the unused needles check. We
can just revert this commit when we want them back.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The background color of the generic left bar needle has changed
to match the new logo base color. The top bar's background color
has similarly changed, but this also caused us to notice a bug
in fedora-logos - that topbar image file seems to be basically
empty (just a transparent rectangle) so we see no 'image' in the
top bar, just solid electric blue. This needle matches that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Boxes dropped VNC functionality. It's supposed to be replaced by
Connections, but we can't use that until it has fullscreen:
https://gitlab.gnome.org/GNOME/connections/-/issues/5
so use Vinagre for now. We do also prepare some needles for
Connections in anticipation of being able to use it later (since
I already did the work and don't want to waste it...)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OK, so sometimes we have a Reboot button, sometimes a Restart
button, sometimes both, but never *always* one or the other. So
we need both needles. Sigh.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a spurious warning on the Updates page, but we don't
want to fail tests for non-related updates on that, and I've
already reported it so it should get fixed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'reboot' button isn't always there in this case, it seems,
but the 'restart services' one is more likely to be. So let's
switch.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Stupid Firefox survey means the thing we usually check isn't
always on the screen. This one checks for PRIORITY.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE switched to using Noto fonts by default (and title bars seem
to be blue again), many needles need to be updated.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On current F34 we get no permanent update notification in the
notifications view, we only get a *transient* one plus the
systray icon. This tweaks things so on F34 we check both of
those things correctly, behaviour on <F34 should be unchanged.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This dialog gets cut off sometimes, it seems. That's a bug, but
we just want to handle it, because we want to quit cleanly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE update was still often failing on #1943943, so this tries a
bit harder to work around it. We add a 'refresh' needle for KDE,
and tweak the 'retry' logic to click it if we get to that point.
Note adding the needle also changes behaviour slightly - we may
click this needle if we see it on first entering the screen. So
either change may be helping. Either way, this does make the test
more reliable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Design team tweaked the new logo a bit in anaconda. We probably
won't need the old needle any more, but I'll take that out later.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some apps moved around, others the needles stopped matching for
some reason, some kind of slight scale change or something.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The package with the new logo is not submitted as an update yet,
but we ran the tests on the Koji build and these are the new
needles. We'll need more when we run the full set of compose
tests on the change.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When we have to restart Cockpit during the update, we don't get
the usual "successful" screen after logging back in, we'll have
to match on something indicating an update did happen. I wanted
to use the little icon next to the package count but it seems to
have some kind of problem with anti-aliasing or something, I've
created two needles for it and neither matched on the next run
of the test. So let's match on this "reboot needed" text instead,
I hope it'll always be there when we hit this case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Make the 'deactivate overview if it's active' thing a bit more
robust by asserting the inactive state after deactivating it,
and add new needles for the new RC (text got a bit brighter).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For consistency, let's just return to the desktop right away. We
also need to handle closing the overview before running installer
on live image boot.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We worked around it in a different way (installing a Plymouth
theme in the base disk image) so these shouldn't be needed any
more.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a bug in F34/Rawhide lately where the bootsplash isn't
fully cleared when displaying text consoles, this causes all the
login needles not to match. Add several workarounds to make the
most common cases at least work with the bug, failing on it is
not much use.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We need to hit 'restart' after applying updates, and we also
need the 'done' needle *not* to match the restart message, so
change that to match on the text (unfortunately). That also
means we have to add another variant of the needle for F32 as
the background of the text is a different color there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a test that uses the Blivet interface to create an LVM
layout with ext4 filesystem as well as a postinstall test that checks
that the LVM layout has been created correctly.
We finally saw a test where there were *no* errors logged by the
time Cockpit reached the log screen, so there were no entries to
click. Let's just make the test set log level to info before
looking for entries - I prefer this to 'click entry if found,
otherwise change log level' as that's twice as many branches to
look after. Of course, it means the warning triangle entry needle
is useless now :(
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE in F34+ is now placing sleep, restart and shutdown buttons
right on the system menu, not in a submenu. So we need to sort of
tweak this logic. The approach here is: we count the GNOME
submenu as both a "power" and "leave" menu, so the needle to
enter it has both tags. KDE still has a "leave" submenu, but the
power options are not in a submenu any more, so the new "leave"
needle only has the leave tag, not the power tag. For "leave"
actions we just unconditionally expect the "leave" tag; for
power actions we first match on *either* the submenu tag (for
GNOME and earlier KDE) *or* the action tag, click whatever we
found, and then if we matched the submenu (not the action), we
assert and click the action. After that all paths should be in
sync again and we can continue.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Cockpit 237 has misaligned radio buttons here. We know about the
issue and want the test to not fail on unrelated updates, so let's
make it a soft fail.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR uses the Anaconda Blivet partitioning to recreate a partition
layout while preserving the content of the /home subvolume.
It also adds the postinstall test to check that the home has been
preserved.
This PR adds the `install_btrfs_upload` to install the btrfs based
image, the `btrfs_preserve_home_extras` to prepare and test the data
on the home partition, as well as the `custom_btrfs_preserve_home` that
uses the preinstalled btrfs image and uses its current partitioning to
preserve the home partition and the data on it.
We don't always get a smartd error any more, so add one for a
different error that's more consistent ATM.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The background seems to be different in different tests, even
the reduced area I tried most recently seems to be different in
some runs. So let's just match in the text area instead. This
captures some "Fedora" text so it should still notice if branding
is broken.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Required because ppc64le has a PReP partition
before boot partition.
PReP partition must not be changed by this script.
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
The font of the identifier text got a bit smaller. I think it
looks kinda bad now, but it's not an outright bug, so just
living with it. I did file an upstream issue:
https://github.com/cockpit-project/cockpit/issues/15111
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a race issue with just treating it as a next button: it's
not in the same place as a next button. Sometimes in the g-i-s
code we actually get ahead of ourselves and click early, which
isn't really a problem when the buttons are all in the same
place, but if we click "Start Setup" in the middle of transition
to the Privacy screen - as in
https://openqa.fedoraproject.org/tests/745034#step/_graphical_wait_login/4
- the click effectively gets lost. So let's make it its own tag
and have the initial assert look for it too. That way we won't
match on it again in the main loop over "@nexts".
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There seem to be some weird shenanigans going on with the pipe
bit here (it looks different on different test runs? what?) so
let's just match on the plain grey area. Should still be enough.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Just matching the Overview entry isn't really enough, the app
hasn't really run yet. This makes the test more robust and also
helps out on aarch64 desktop tests where the app window takes a
long time to appear.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When Fedora went to BTRFS as a default, we lost the LVM based image to
run LVM resize tests with.
This PR introduces the `install_lvm.pm` installation test that creates
an LVM based qcow2 image to be used by follow-up tests.
Styling on the KDE base disk image seems to have gone wrong
again, which throws this test off. We need to figure out what's
gone wrong in the base image, but it's a Friday night and I don't
want the test failing all weekend, and there's nothing wrong in
the *test itself*, so let's create some workaround needles and
figure it out next week.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Update a needle with slightly different text rendering, and add
a workaround to hit tab three times rather than once on entering
the "Join a domain" screen, see
https://github.com/cockpit-project/cockpit/issues/14895 .
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We've been getting failures lately on the first page load, I
think because Firefox is getting even more grindy on startup. So
turn the 'sleep' into a 'wait_still_screen', extend another wait,
and tweak the 'browser' needle so it only matches after the
bookmark bar has loaded rather than as soon as half the chrome
appears. Also make all the wait_still_screens use similarity 45
for consistency (flashing cursor could be there on any of them).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Dropped the use of this in the recent notification simplification,
but forgot to remove the needle. Should've run tox...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is the best option I can come up with to deal with #195.
Update notifications seem to have become transient in KDE lately
(even in F31 and F32, if I'm looking at these screenshots right).
This actually simplifies things a lot to do more or less the
same in the KDE and GNOME paths: open the 'permanent' store of
notifications (in GNOME you get to it by clicking on the clock,
in KDE via the systray) and then look for no notifications (live
path) or only an update notification (post-install path). We
only run this test for composes so we shouldn't need to worry
about anything older than F32, and I believe this should work
for KDE in F32 and F33. I left out click_unwanted_notifications
for now as I'm hoping it should be unnecessary, but we can add
it back in if necessary.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't need a separate 'welcome' needle because it just matches
on an OK button anyway. So turn that needle into an OK needle
(we don't have any existing 'blue OK button' needle) and simplify
the logic to a single loop for kde_ok and krusader_settings_close.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It had krusader_settings_close as its tag, not kde_ok. That's
why the krusader test module was failing weirdly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'universal' tests have quite a few failures if you run them
on an image where btrfs is the default (currently they usually
run on the Server DVD, where xfs-on-LVM is still the default).
This fixes some of them, the others would need code fixes.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Even though we have subdirs, we actually usually make needle
names unique across subdirs due to limitations of openQA's UI
upstream.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OTP field was moved into the last position in the password change dialog
to prevent issues with OTP code expiring while users enter their
passwords.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
In g-i-s 3.37.91, the first screen has a 'Start Setup' button
rather than a 'Next' button. Easiest thing for us to do here is
just to add a new needle which has the 'next_button' tag even
though it's clearly not a 'Next' button, because then the code
still works :) So do that, but give the file a suggestive name
and explain the situation in a code comment.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
GNOME now also splits 'Restart...' and 'Power Off...' as KDE
does, so we need to tweak the conditional and add some needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a bit complex to automate, because we cannot really use
the production Zezere server (provision.fedoraproject.org) as
the test case shows, as we'd have to solve authentication and
we also don't really want to constantly keep registering new
hosts to it that are going to disappear and never be seen again.
So, instead we'll do it by setting up our *own* Zezere, and
provisioning our IoT system in that. We run two tests. The
'ignition' test is the actual IoT 'device'; all it really does
is boot up, sit around, and wait to be provisioned. The 'server'
test first sets up a Zezere server, then logs into it, adds an
ssh key, claims the IoT device, provisions it, and connects to
it to create a special file which tells the 'ignition' test
everything worked and it can close out.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
See https://pagure.io/background-logo-extension/issue/26 - in
current Rawhide, the search box in the overview is not active
when the overview is opened, so you can't just open the
overview and type, you have to click it first.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
See https://bugzilla.redhat.com/show_bug.cgi?id=1854225 . A new
fontawesome release broke nearly all font-based interface
elements in the FreeIPA web UI, which breaks several of our
needles. This adds workaround needles for each one (plus a couple
of non-workaround needles for cases which don't seem to be the
icon bug, but somehow changed anyway).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In some recent tests the text is rendered slightly differently
for some reason. Not sure why, but I don't see a problem with
just adding a variant needle.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Recent live respins were built with livecd-creator rather than
livemedia-creator because the maintainer had trouble with LMC.
This results in grub looking a bit different, we need a variant
needle for it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Not sure if the needles changed or just the way they're rendered
in the overview, but either way, we need to update a bunch.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The little triangle that's used on drop-down menus and stuff got
bigger. That breaks all these needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This goes back to when we called this needle desktop_clean, but
there's really no point in having it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In Cockpit 220, the Updates entry is off the bottom of the screen
so we need to scroll the left bar down before we can click it.
Also update some other needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems like sometimes the machine ID is off the bottom of the
screen now, so let's just match on the section title instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Unfortunately only the entry name is clickable now so we can't
have a 'generic' needle that'll work for any service.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Since GDM shows the "system-menu-button", it could not correctly
switch users on a locked screen. I added a check to see
if we are on a locked screen and behave accordingly.
This adds a new test that implementsQA:Testcase_desktop_login
on both GNOME and KDE.
While working on this, we realized that the "desktop_clean"
needles were really "app menu" needles, and for KDE, this was
a duplication with the new "system menu" needles, because on KDE
the app menu and the system menu are the same. So I (Adam)
started to de-duplicate that, but also realized that "app menu
button" is a much more accurate name for these needles, so I was
renaming the old desktop_clean needles to app_menu_button. That
led me to the realization that "check_desktop_clean" is itself a
dumb name, because we don't (at least, any more, way back in the
mists of time we may have done) do anything to check that the
desktop is "clean" - we're really just asserting that we're at a
desktop *at all*. While thinking *that* through, I *also* realized
that the whole "open the overview and look for the app grid icon"
workaround it did is no longer necessary, because GNOME doesn't
use a translucent top bar any more. That went away in GNOME 3.32,
which is in Fedora 30, our oldest supported release.
So I threw that away, renamed the function "check_desktop",
cleaned up all the needle naming and tagging, and also added an
app menu needle for GNOME in Japanese because we were missing
one (the Japanese tests have been using the "app grid icon"
workaround the whole time).
