These hit low-90% matches when run on the F38 respins for some
reason, just add another variant...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I don't know exactly why this sometimes shows up highlighted and
sometimes unhighlighted, but hey, it's not wrong either way, so
we just handle it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was resolved upstream and we're no longer hitting this bug
in tests on F38, Rawhide or even F37 respins, so we should no
longer need this workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think the shade of grey on the background changed. We'll
probably need more needle updates for the compose tests (the
desktop_login test and different languages), but this covers the
update tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In today's Rawhide, for some reason, after we delete the first
file, the second file we want to delete is highlighted.
Previously the other file in the directory was highlighted. No
biggie, just handle both cases.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a difference in the Info page and we get every font
twice on Silverblue because they're present in two locations.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The font rendering on this URL bar seems to be slightly unstable
for some reason. We got a 95% match on this needle in
https://openqa.fedoraproject.org/tests/1846906#step/about/10
so let's just bump the level down a bit.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
They've dropped the IRC section from this page. We don't really
need three match areas, just the two is sufficient to identify
it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Contacts now has two burger menus, which is awkward. We need
specific needles to identify each, we can't rely on the generic
needle any more as it won't always open the right menu. We also
need to still work with the old UI for the flatpak.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We still need to handle 43 only requiring one for now, and we
can't just make it release-dependent until 44 is stable for both
38 and Rawhide, so let's use a needle match temporarily. Only
44 has these eye/pencil icons on this screen.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Bike route changed again. For the website opened needle, the
URL bar text seems to keep changing slightly, so instead let's
try matching on the big app logo...hopefully that won't change
as much.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a small test suite to test the Characters applications.
It displays several different groups of characters and then tries
to copy one of the characters and place it into a text editor.
GNOME Software no longer has a welcome screen in any current
Fedora (it was dropped between 35 and 36), but in Rawhide it now
has a popup that prompts you to enable third-party repos which
we need to get rid of, so just convert the welcome screen check
to handle that, and drop all the welcome screen needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
* Scarborough provided quite a messy map that resulted
in frequent needle failure. Changing the location
for something better to make it more reliable.
* The zoom test could have failed with a low resolution
image. Adding some timeout to the needle give more
time to load the proper image.
I think these needles are pretty fragile to changes in the
underlying OSM dataset, not just in Maps itself...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Rawhide 'accept fate' text seems to have lost its Japanese
translation for some reason (I think the string might have had
a terminating period removed). And in one test, the "Extract"
menu item in Archiver was pre-highlighted so the needle didn't
match. Not sure why, but this doesn't seem like a problem, so
let's just handle it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts the last few commits which worked around a focus bug
in GTK. This bug is now (I hope) fixed, so I'm dropping the
workarounds so the tests will confirm whether it's fixed.
GTK4 enabled font hinting recently, and that breaks this huge pile
of needles.
There are probably a few more that need doing, but it's 2am and
I've had enough.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lukas left these lying around stg, they are needed.
clocks_alarm_cross_remove is a dupe of gnome_button_cross_remove
with a bad name, so removed it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We already changed how we do this since the context menu entry
was removed, just cleaning up the now-unneeded needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We had a ton of needles all covering something very similar
(press a "Credits" button in a GNOME app). There are about four
real variations: old-style regular face white-on-black (eog),
old-style regular face (nautilus and evince before recent
libadwaita ports), old-style bold face (GTE and Clocks before
new libadwaita), and new-style (everything that's been ported
to use libadwaita for its About page). Let's just rationalize
it down to those, using the same needle tag for all of them.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Silverblue has Calculator as an older-versioned flatpak, so it
still looks like it did in GNOME 42 (blue equals button, lighter
colored number buttons).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
That last commit to 'fix' the Clocks tests when Silverblue needs
location access to be granted wasn't complete, I left the needle
out. D'oh. Take the chance to give it a better name too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR changes the way to download the test data into the VM.
Although it does not use a disk image as suggested in one
of the review, it does not clone the entire repository, but
a simple tar.gz file that holds the data which will be
distributed into the directory structure.
This way, the amount of data needed to be downloaded dropped
from approximately 50MB to below 2MB.
Also, the existing test suites were adapted to this situation.
This is the automation of the optional testcase https://fedoraproject.org/wiki/QA:Testcase_i18n_default_fonts.
The test implementation runs the same commands as the mentioned test
case and checks the expected output. It is designed to run in the scope
of postinstall tests when the language is set to "japanese".
So it turns out the translated layout indicators in Arabic are
intentional:
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5257
so we can just add needles for them and we're good. Also update
a couple of other needles which need updates since we last
reached this far in the tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OK, this is annoying. GNOME Software intentionally does *not*
clear the 'download' or 'reboot and update' button when you hit
the refresh button, it just leaves them sitting there while the
refresh happens. So let's specifically require the 'refreshing'
text to appear and go away before we try and click on download
or apply.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Again, a dialog changed a bit in the flatpak version of the app.
For this needle, the shade of grey is slightly different. Change
happened between Fedora-36-20220314.n.0 and 20220315.n.0.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The open/save dialogs for Flatpak apps in current F36 (and
probably Rawhide, but can't tell due to another bug) look a bit
odd: https://github.com/fedora-silverblue/issue-tracker/issues/245
This adds variant needles to handle various differences there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For some reason various bits of gte started looking slightly
different on the F36 upgrade test. I don't really know why and
don't care enough to look it up. I'm just hoping all these
variations between gte running in slightly different contexts
calm down soon...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Both KDE and GNOME saw some changes to desktop_login needles in
recent Rawhide, this updates them.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Silverblue has an older version of gnome-text-editor, using a
Flatpak base environment built off an older GNOME. Because of
this most of the needles for the current RPM-packaged version
don't match. For some reason the old needles we delete don't
match either - some difference in font rendering configuration
or something. So I had to create a bunch of new needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It matches on the "close the app" button on older Flatpak GTE.
So widen the needle to include the "Preferences" text, and add
a click point.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
To get to the keyboard/input method settings and add an input
method when doing a Japanese install test, we type 'keyboard',
but in current GNOME 42.beta that doesn't find the right pane.
Typing 'input' does work, though, so let's use that instead.
Also the GDM login needle needed updating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These are taken with the latest gnome-shell build, with CSS
fixes for the overview applied. They don't work for current
F36/Rawhide but will work once that gnome-shell build lands.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
GNOME 42 and adwaita-icon-theme 42 changed a lot of things in
GNOME and anaconda, we need to update all these needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When the printing_builtin test ran on an F35 respin compose it
failed; it turns out the target filename was different for the
built-in print-to-PDF on GNOME on F35. So let's just always
use the 'ls' output to find the file, but pick the directory
to check based on whether we're using cups or not.
Also rename the needles to have unique names, and add one for
F35 GNOME.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We should not use the same name for two different needles even
in two different directories as it can be confusing in some parts
of the UI which don't account for the directory name. Let's use
names differentiated by desktop. Also add a needle for F35 as
the one from the PR doesn't match (different relative placement
of icon and text).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The PR introduces an improved logic to the desktop_printing.pm
that allows to use the USE_CUPS variable in templates to trigger
the installation of cups-pdf prior to the actual test.
The cups-pdf is then used as an alternative PDF printer
instead the built-in Save As PDF method.
It was removed from the default install:
https://pagure.io/fedora-comps/c/5371509
in favor of a new screenshot mechanism that's built in to GNOME
Shell.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The latest version of Gnome-Text-Editor bring a little
bit different UI and some new or modified features.
This commit fixes the suite to run on newest version.
Not sure if we lost one with the recent adjustment to the tests
or F35 actually changed somehow, but hey, we need this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These needles have been probably forgotten and left over from one
of the previous commits. The tox tests are failing on other
pull requests and this PR fixes it.
Workstation has replaced gedit with gnome-text-editor in Rawhide,
so this is no longer useful. We will replace it with a test suite
for gnome-text-editor.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Workstation replaced gedit with gnome-text-editor. This updates
the needles and also changes the name used for 'tagging' the
application for the core_applications test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Add the download and apply needles (whoops), and tighten the
match area on the update needle a bit so it matches even when
there's a little blue balloon to the left of the text.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Calculator's equals button changed a bit somehow, EOG changed a
button and the kerning on Activities button changed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We get a 96% match on this because the "A" in "Activities" moved
a bit. Instead of a new needle let's just be OK with the lower
match.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So it doesn't fail if there's a dotted line around the entry
(indicating keyboard active, I think).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The cross got bigger. This needle might be a dupe with something
but I couldn't be bothered finding it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some changes were made in the icon theme used by the installer
and GNOME, update various needles for this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Similar to the 'open' needle, adjust match area to work even
when there's a notification obscuring the top bar.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We do this slightly differently on KDE and GNOME for whatever
reason, so this needle needs updating too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
LibreOffice adjusted layout a bit and dropped a template, one
icon changed in the utilities menu compilation.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Remove a whole chunk of needles that haven't matched for more
than 3 months. Also move a few needles to appropriate locations,
simplify some code chunks that relied on removed needles (if
we're not matching the needles, we don't need those chunks any
more), and drop some other no-longer-needed conditionals for
older releases.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test for fingerprint-based login, as requested by
@benzea in #223. We use the fprintd dummy device to let us
simulate scanning a fingerprint, and check various scenarios
recommended by @benzea.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Now we have two areas, openQA wants to click in the wrong one.
Let's tell it which one to click in.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The main screen now also has identical "Japanese" (that's what it
says) text. To avoid false matching before the picker opens, add
another match area.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The open dialog on Silverblue (which is apparently not at all
the same thing as the open dialog on Workstation, though they
look the same) does not default to the Documents folder, so we
have to open it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These all failed to match first time the test was run in
production. I guess Lukas was working from an older release.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR fixes issue #188. It adds a test suite to test basic
functionality of Evince and brings the following features:
* test scripts for various Evince functions.
* needles to support the Evince test scripts
* new template variables `TESTPATH` and `POSTINSTALL_LOAD_ALL` (see
below)
* new logic in `main.py` (see below)
The new variables and the new logic make it easier to create test
suites for post-installation tests. If TESTPATH is used, OpenQA
will take all tests mentioned in POSTINSTALL from that specified
TESTPATH. If both TESTPATH and POSTINSTALL_LOAD_ALL are used, then
OpenQA will run all tests it can find at the TESTPATH location.
If POSTINSTALL and POSTINSTALL_LOAD_ALL are set simultaneously,
then only POSTINSTALL will be taken into account and OpenQA will
only load tests mentioned there.
Recent git os-autoinst no longer downsamples screenshots as far
as it did before comparison. This makes a lot of needles where
colors have changed slightly no longer match, so they all needed
updating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
GNOME dropped the g-i-s new user mode in F34, so on a Japanese
install with user created in the installer, you don't get an
input source configured out of the box or on first boot. So
we'll just have to do it manually after booting, before we test
if it works.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Not sure why these changed, but oh well. Utilities menu was
highlighted in a test run for some reason, so let's just handle
that. Other needles changed very slightly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some apps moved around, others the needles stopped matching for
some reason, some kind of slight scale change or something.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The package with the new logo is not submitted as an update yet,
but we ran the tests on the Koji build and these are the new
needles. We'll need more when we run the full set of compose
tests on the change.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Make the 'deactivate overview if it's active' thing a bit more
robust by asserting the inactive state after deactivating it,
and add new needles for the new RC (text got a bit brighter).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For consistency, let's just return to the desktop right away. We
also need to handle closing the overview before running installer
on live image boot.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE in F34+ is now placing sleep, restart and shutdown buttons
right on the system menu, not in a submenu. So we need to sort of
tweak this logic. The approach here is: we count the GNOME
submenu as both a "power" and "leave" menu, so the needle to
enter it has both tags. KDE still has a "leave" submenu, but the
power options are not in a submenu any more, so the new "leave"
needle only has the leave tag, not the power tag. For "leave"
actions we just unconditionally expect the "leave" tag; for
power actions we first match on *either* the submenu tag (for
GNOME and earlier KDE) *or* the action tag, click whatever we
found, and then if we matched the submenu (not the action), we
assert and click the action. After that all paths should be in
sync again and we can continue.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
