This branch is very fragile, because the test won't fail if we
miss the match on the security needle. So in practice, we are
never going to notice when the needle goes stale, and we'll just
wind up never triggering this branch and always going down the
other path. That's the current situation: the security_install
needle last matched more than a year ago at least. Let's just
admit the truth here and drop the branch entirely.
Also update the cockpit_updates_restart_ignore needle. This is
in a similar case - we don't really notice when it goes stale,
as the test completes, it just takes a bit longer - but since
this one is quite easy to find, let's just update it instead of
dropping it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was resolved upstream and we're no longer hitting this bug
in tests on F38, Rawhide or even F37 respins, so we should no
longer need this workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It looks like neither of these has been a problem for some time.
The notification needle has not matched for a year. The akonadi
needle doesn't exist any more - it was cleaned up in the 2021
needle cleanup, meaning it hadn't matched for weeks in 2021. I
checked the last several months of KDE app start/stop tests and
don't see any case where there was a stray notification that we
missed. So I think we can just ditch this whole mechanism for
now; if we have problems with these notifications again in future
we can put it back.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The last one of these was deleted during the last needle cleanup,
but we do actually still occasionally hit the dialog, e.g. in
https://openqa.fedoraproject.org/tests/1837435/modules/kontakt/steps/3
so let's add an updated version of the needle.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think the shade of grey on the background changed. We'll
probably need more needle updates for the compose tests (the
desktop_login test and different languages), but this covers the
update tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In today's Rawhide, for some reason, after we delete the first
file, the second file we want to delete is highlighted.
Previously the other file in the directory was highlighted. No
biggie, just handle both cases.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a difference in the Info page and we get every font
twice on Silverblue because they're present in two locations.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This actually hasn't matched for years, we've been falling back
to ctrl-t the entire time. Happened to notice it today while
debugging https://bugzilla.redhat.com/show_bug.cgi?id=2184549
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These variants only show up when a security update is in play,
so not very often; this happened today, so now we get to update
them for current KDE.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The font rendering on this URL bar seems to be slightly unstable
for some reason. We got a 95% match on this needle in
https://openqa.fedoraproject.org/tests/1846906#step/about/10
so let's just bump the level down a bit.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
...for the GNOME icon theme change. This needle isn't hit very
often so it didn't get updated with the others.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
They've dropped the IRC section from this page. We don't really
need three match areas, just the two is sufficient to identify
it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This test hasn't run for a long time due to
https://bugzilla.redhat.com/show_bug.cgi?id=2151607 , now we got
it working again, almost all the needles need updating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
If the cursor is visible in the middle of the screen, this would
not match. So move the match area a bit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Contacts now has two burger menus, which is awkward. We need
specific needles to identify each, we can't rely on the generic
needle any more as it won't always open the right menu. We also
need to still work with the old UI for the flatpak.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE has a welcome tour now, on F38 and Rawhide at least. Let's
"handle" it with extreme prejudice...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We still need to handle 43 only requiring one for now, and we
can't just make it release-dependent until 44 is stable for both
38 and Rawhide, so let's use a needle match temporarily. Only
44 has these eye/pencil icons on this screen.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These are similar to the changes in F37 and Rawhide, but these
needles are specific to F36 somehow so weren't updated in earlier
rounds.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE live installer started looking different on F37 too so we
need a new needle there, plus we need F39 needles now Rawhide is
F39.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In today's F38 and Rawhide, changes to the persistent overlay
stuff result in a boot warning you have to spam through. Let's
handle this as a soft fail so we don't have floods of failed
tests till it's fixed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Looks like what happened here is font kerning changed (got
better) in the nm-connection-editor spawned from anaconda.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
A lot of these are because anaconda is more KDE-themed on the
KDE live ISO now. The rest are just miscellaneous appearance
changes to KDE apps in recent Rawhide.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Bike route changed again. For the website opened needle, the
URL bar text seems to keep changing slightly, so instead let's
try matching on the big app logo...hopefully that won't change
as much.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Now we sometimes have to scroll the page down to add the addon,
the Remove button isn't always fully visible behind the popup.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
"Additional repositories" is now hidden behind a dropdown we
have to open first. This will make the test fail on anything
older than Fedora-Rawhide-20230121.n.0, but I don't think we
run this test anywhere that would be a problem.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a small test suite to test the Characters applications.
It displays several different groups of characters and then tries
to copy one of the characters and place it into a text editor.
GNOME Software no longer has a welcome screen in any current
Fedora (it was dropped between 35 and 36), but in Rawhide it now
has a popup that prompts you to enable third-party repos which
we need to get rid of, so just convert the welcome screen check
to handle that, and drop all the welcome screen needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This test fails on Rawhide (because sddm-on-wayland) but we just
got an F37 respin, where it passes but needs some needle updates.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
All the deleted ones haven't been matched for five months. Drop
match level to 90 on the remaining ones, we got a 96 match for
one of them in today's respin test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I don't know why we wind up with so many slightly different
matches on the login screen. It's weird.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
it's really just a dupe of the -problems needles, it turns out,
Lukas was reinventing that wheel. He had to add another one
today because I broke the JSON in this one when I was simplifying
it yesterday, but I think this one on the new -problems needle
are really just dupes.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
* Scarborough provided quite a messy map that resulted
in frequent needle failure. Changing the location
for something better to make it more reliable.
* The zoom test could have failed with a low resolution
image. Adding some timeout to the needle give more
time to load the proper image.
I think these needles are pretty fragile to changes in the
underlying OSM dataset, not just in Maps itself...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Rawhide 'accept fate' text seems to have lost its Japanese
translation for some reason (I think the string might have had
a terminating period removed). And in one test, the "Extract"
menu item in Archiver was pre-highlighted so the needle didn't
match. Not sure why, but this doesn't seem like a problem, so
let's just handle it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts the last few commits which worked around a focus bug
in GTK. This bug is now (I hope) fixed, so I'm dropping the
workarounds so the tests will confirm whether it's fixed.
I think the reason the match failed is the pixels under the text
changed when the pre-release warning disappeared. We don't really
need a new needle, we just need to make the existing one less
tall so no part of the text underneath is included.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think this is slightly different on openQA stg with current
qemu with the EDID settings, for some reason.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These look slightly different when there are security updates,
the security needle variants hadn't been updated yet for the
latest changes in background, upstream toolkit etc.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
GTK4 enabled font hinting recently, and that breaks this huge pile
of needles.
There are probably a few more that need doing, but it's 2am and
I've had enough.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
From anaconda-37.12.1, anaconda defaults to GPT for all BIOS
installs. So we need to create a BIOS boot partition when doing
a BIOS install. I think all other potential configs (x86_64
UEFI, aarch64 (UEFI), ppc64le (OFW)) are covered under the other
two paths, so just making this `else` should be OK.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lukas left these lying around stg, they are needed.
clocks_alarm_cross_remove is a dupe of gnome_button_cross_remove
with a bad name, so removed it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We already changed how we do this since the context menu entry
was removed, just cleaning up the now-unneeded needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We had a ton of needles all covering something very similar
(press a "Credits" button in a GNOME app). There are about four
real variations: old-style regular face white-on-black (eog),
old-style regular face (nautilus and evince before recent
libadwaita ports), old-style bold face (GTE and Clocks before
new libadwaita), and new-style (everything that's been ported
to use libadwaita for its About page). Let's just rationalize
it down to those, using the same needle tag for all of them.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Silverblue has Calculator as an older-versioned flatpak, so it
still looks like it did in GNOME 42 (blue equals button, lighter
colored number buttons).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It needs to match even if the 'link target hint' is showing at
bottom left and kinda obscuring it a little.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
They changed from "Software Updates" to "Software updates".
Apparently this was intentional and in line with Patternfly
guidance, so not marking as a workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think the change here is the new version of noto fonts. GNOME
uses adwaita so it's not affected.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
That last commit to 'fix' the Clocks tests when Silverblue needs
location access to be granted wasn't complete, I left the needle
out. D'oh. Take the chance to give it a better name too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The previous commit is the correct fix for the problem here.
Adding a needle that matches on the tray icon was not correct,
we need to be sure that we can access the Updates view from
within Discover itself.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR changes the way to download the test data into the VM.
Although it does not use a disk image as suggested in one
of the review, it does not clone the entire repository, but
a simple tar.gz file that holds the data which will be
distributed into the directory structure.
This way, the amount of data needed to be downloaded dropped
from approximately 50MB to below 2MB.
Also, the existing test suites were adapted to this situation.
The wider one seems to happen when text at the bottom of the
login box prevents the "Fedora Linux..." text from wrapping
across two lines. I have no idea why we see two different
cases of the "Fedora Linux text wrapped" variant with very
slightly different amounts of whitespace, but we do, it seems
we need both of these.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Not sure what's going on here, but it seems like the resolution
of the UEFI bootloader screen in some tests changed. Not sure
if this is a qemu or edk2 change or what.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is the automation of the optional testcase https://fedoraproject.org/wiki/QA:Testcase_i18n_default_fonts.
The test implementation runs the same commands as the mentioned test
case and checks the expected output. It is designed to run in the scope
of postinstall tests when the language is set to "japanese".
We've been told the bookmark toolbar no longer being shown on app
start is by design, so we can't call this a workaround needle
any more really. There's a preference we could use to turn it
back on so we can wait for the bookmark toolbar to show up to
be sure firefox is fully started, but in fact tests seem to be
passing OK with just this needle these days, so we may not need
to.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Firefox 97 is now stable on all releases, so we can forget about
handling browser_download_save and just assume download will
happen automatically.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Before F34 we had to launch the update tool from the systray.
This isn't the case any more, so we can throw all this code and
these needles away.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We seem to be solidly back to always getting a permanent update
notification in current F36/Rawhide, so we don't need this more
complex path any more. We also don't need these needles any more,
they haven't matched for months.
Same difference as for English - 4 pixels of padding was added.
Also move a stray Russian needle to the right directory.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can wind up with this a bit hidden by the bottom-left URL
location indicator, so shrink it a bit to hopefully match even
in that case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Cockpit update test relies on scrolling the Cockpit left bar
to find the Software Updates page, but in Firefox 100, the scroll
bars disappear shortly after the last time you moved the mouse.
To deal with this, instead of looking for a scroll bar to scroll,
we'll click the search box in the left bar and hit 'up' to scroll
it to the bottom.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Not sure what changed in the addon needles. For the webui needle
I think it's affected by the new scrollbar behaviour.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We get these variant needles with the keyboard icon and layout
name at slightly different relative vertical positions every
time we do a Final RC. I think the layout is affected by the
pre-release warning text no longer being present. Usually each
cycle something changes with the icon or with font rendering,
so the needles have to be updated, as in this case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So it turns out the translated layout indicators in Arabic are
intentional:
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5257
so we can just add needles for them and we're good. Also update
a couple of other needles which need updates since we last
reached this far in the tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The new version of the F36 backgrounds means all these needles
that hit translucent panel elements need updating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
SDDM looks slightly different on F36 since 20220319.n.0. The
Sleep / Restart / Shut Down / Other... buttons are visible, and
the keyboard layout indicator isn't. Previously on F36, and still
on Rawhide, those buttons aren't shown but the layout indicator
is. I'm not sure why this is, but it means the size of the name
labels is slightly different, and we need new needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We very rarely see this needle - only when we run the 'generic'
tests on an Everything image instead of a Server one, which
will usually only happen if we trigger it manually. I did this
recently to test an anaconda update, and found it needs an
update.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OK, this is annoying. GNOME Software intentionally does *not*
clear the 'download' or 'reboot and update' button when you hit
the refresh button, it just leaves them sitting there while the
refresh happens. So let's specifically require the 'refreshing'
text to appear and go away before we try and click on download
or apply.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Again, a dialog changed a bit in the flatpak version of the app.
For this needle, the shade of grey is slightly different. Change
happened between Fedora-36-20220314.n.0 and 20220315.n.0.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The open/save dialogs for Flatpak apps in current F36 (and
probably Rawhide, but can't tell due to another bug) look a bit
odd: https://github.com/fedora-silverblue/issue-tracker/issues/245
This adds variant needles to handle various differences there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For some reason various bits of gte started looking slightly
different on the F36 upgrade test. I don't really know why and
don't care enough to look it up. I'm just hoping all these
variations between gte running in slightly different contexts
calm down soon...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Both KDE and GNOME saw some changes to desktop_login needles in
recent Rawhide, this updates them.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
