When Software complains about something, it shows a grey banner
in the lower part with a button to click (Details).
The update test has not utilized this anyhow, so if it happened,
we would not know what the error was.
This fix makes sure to click on the Details button when it
appears, revealing the reason behind it, which could help
us understand what happened.
Thanks to @luap99 for the neat idea here. It's valuable to run
the upstream integration tests both rooted and rootless - some
of the tests are unique to each mode, and other tests may pass
in one but fail in the other. However, we don't want to double
the execution time of the test. We could duplicate the test and
have rooted/rootless versions, but that increases load.
This is a neat trick: instead, just run the tests with root on
x86_64 but rootless on other arches (which for now is just
aarch64). This way we should get pretty much full coverage with
the same test execution time and overall load as before.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Until https://github.com/bats-core/bats-core/pull/1114 we can't
easily filter out specific tests in a bats run. So let's hack it
up with a sed that just negates the behavior of --filter - we
don't need to use its default behavior. Then we can use --filter
to filter out specific tests, like this. Use it to filter out
the test that's known to fail with recent kernels.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Don't run only a minor subset of the tests instead run all of them. The
main advantage is that we get much better coverage and upstream don't
need to maintain an extra subset of tags. Of course tests will be much
slower now as it will now run like 11x the amount of test cases.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/373
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
rootless podman was broken on new installs because shadow-utils
was changed not to assign subuid/subgid ranges to new users by
default, because Security.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
New button were added to the Calculator's Basic view,
which allow access to constants and functions. In this test,
we check some of the constants and functions.
This has always incorrectly been a race, it looks like, but for
some reason we were winning it before but we're losing it now.
The client seems to be pinging while the server's still typing
stuff into grub. So let's have the server set a mutex, and the
client wait for it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some of the buttons disappeared from the basic view of the applications.
This, for the times being, removes the operations that used these
buttons, for the application test to pass again.
All the old patterns we accept and differing files they might be
in are obsolete now, I'm pretty sure - I checked that in F41
tests we get the most recent messages and they're all in syslog.
So drop all of those. But we need to add new handling for dnf5,
now anaconda switched to that and the messages changed again.
I can't find any messages in the dnf5 logs that confirm the
correct URL was used, unfortunately - these are the best I can
find.
Also, since 31cd5e7 , we won't get the base repository error
strings we're looking for any more, even if there *is* a problem
with the base repo. I don't think there's any single error we
can rely on getting any more, so we'll have to trust that the
other checks are sufficient, or figure out ones to add as we go
along if not.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This handles differences in webUI's appearance on the KDE live,
compared to the GNOME live which suppresses keyboard layout
selection, user creation and root password creation. By Lukas,
modified by Adam.
F42 to F43 upgrades suddenly started failing, and I think this
is why - we actually wind up installing packages from the target
release in setup_repos (called from repo_setup), because we set
up the buildroot repo then run some dnf commands. Let's tweak
that so for upgrade tests we create the buildroot repo disabled
at first, then enable it in upgrade_run.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Latest Vault doesn't have multiple encryption backends any more,
so there is no indication of the one being used here any more.
Let's just drop the check.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It is required with podman 5.5.0 for the "podman run --device-read-bps"
test. If we don't load it the test will be skipped and we loose some
coverage.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This test suite replaces the Evince test suite and
adds altered scripts and needles to go with the
Papers applications. At the same time, it provides
the same level of functionality and testability
as the original evince test.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/377
We will fix that test upstream but for now skip it to prevent false
positives on all future kernel updates.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This is how the test did it before the big menu_launch_type
commit, and we use 'kwallet' in another test. Doing this needs an
additional needle or needle tag, which is pointless, let's just
be consistent that it's 'kwallet'.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We have enhanced the menu_launch_type to allow for
start checking and maximizing applications.
This PR uses the new functions wherever it seems
logical.
If special logic was used for certain cases,
we have not touched these to preserve the
exact behaviour.
The crash workaround for the Fonts flatpak is dropped because it
no longer seems to be needed with the latest version of the
flatpak, and dropping it simplifies this migration.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/358
Fedora Rawhide (to be 43) has new applications that replace
the older ones, namely Papers replacing Evince and Showtime
replacing the Totem.
We are adding a condition to run correct applications on Rawhide
while retaining the older applications for a while until the change
has been made in whole.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/375
This seems to be sometimes *causing* problems now. In some cases,
the first launch actually worked but we don't wait long enough
for anaconda to show up, so we launch it twice, and that can
cause failures like https://bugzilla.redhat.com/2360859 .
I did a dry run for a few days on staging and just dropping the
loop entirely didn't seem to produce any failures-to-launch, so
maybe the various bugs we added this for in the first place have
all gone away? Let's try dropping it. If we run into failure-to
-launch problems again, we can add it back but bump the timeout
or something.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In two cases we don't need separate needles for identifying a
screen and then clicking something on it: we can just also use
the thing-to-click for identification purposes. Also remove the
connect_button-verify needle which has never matched (it matches
on the Verify button but has the tag for the Connect button, no
idea why), and update all the needles for the new GNOME fonts.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Oh, no particular reason or anything. *ahem*
https://bugzilla.redhat.com/2358785
This should cover a decent range of transient bootable media,
ensuring the UEFI fallback mechanism doesn't kick in if you're
just booting the installer or live environment.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's now a header bar in fullscreen mode which appears until
you move the mouse away from it, and screws up the needle match.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This got messed up in 1e6da7019c
where the KDE needles all got renamed to apps_run_abrt* and their
tags changed to apps_run_abrt, but the KDE *test* was not changed
to look for apps_run_abrt instead of abrt_runs, so we wound up
creating a whole new bunch of abrt_runs needles so we had *three*
sets of needles...
This rationalizes it down to the needles that actually match in
current tests, properly renames them all to apps_run_abrt and
updates the KDE test to look for that tag, and adds a couple of
new needles for the recent downgrade of the app.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think it should be OK to do this without any fancy conditions.
It'll fail for F41 respins, but eh. May fail for F42 nightlies
until we push the update stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Trying to fix the frequent failures of this test, still. I don't
think we need the loop if we make sure to select the *parent*
entry in the list, which the needle tweaks should ensure, but
we might need to click twice to ensure it's selected and not
delete the entire btrfs volume by mistake, which is what we keep
doing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This caused problems (particularly with the more obscure package
install paths like cockpit and realmd) before, but I don't really
like removing it, as it differs from real-world usage. Tests are
currently failing because of a bug in this repo wiping - in
upgrade_preinstall, we should wipe the file again after the
dnf -y update --refresh call, in case that reinstalls it - but
instead of fixing that, let's try just leaving the file alone.
The risk here is that we run into problems when the repo doesn't
exist, again. In theory we should not because it has
skip_if_unavailable=True , and everything *should* respect that.
But if it does turn out to still be a problem we'll have to
revert this and fix upgrade_preinstall.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Because of the 'can't go from > 41 to <= 41' issue, we need to
tweak the coreos rebase targets to get the tests to pass.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
it seems like sometimes we delete the entire btrfs volume instead
of the root device we're trying to delete; I think this is
because we click delete *immediately* after clicking the device,
and that might be too fast. Let's see if a wait_still_screen
helps. See:
https://openqa.fedoraproject.org/tests/3353598https://openqa.fedoraproject.org/tests/3299570
etc.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This came up in blocker bug meeting discussion today. We really
should check that all packages are signed after a default install.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
@lruzicka recently made this test click through the setup page,
if it sees it...but he used new needles (which he didn't commit)
even though we already have those same needles as part of the
detailed contacts app test.
This setup page always appears, we don't need to check for it,
and we don't need separate needles to identify the page and to
click on the local address book option. All we need to do is the
same as we do in aaa_setup: unconditionally look for and click
the local address book option, using the already-existing needle.
So let's do that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seems like GNOME 48 changed window size/positioning a bit, we
have to move the text editor so it will still be visible when
the file manager window is in the front.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The current check never fails - if we don't see the details after
30 seconds, we never actually assert them. We may or may not
soft fail, but we'll never fail.
This simplifies the check (there's no need to specifically look
for the 'loading' screen) and makes it actually fail if the
details don't show up in 90 seconds total.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'settings' menu is replaced by an 'info' panel, and *most* of
the things from 'settings' moved to 'preferences'. But Document
Type is in the 'info' panel. Just to make things fun. The grid
feature is gone. And of course all the needles needed updating
for the new font. The flatpak build is still 47 and so has the
old UI but the new font, and line spacing in it seems slightly
different, so we need conditional paths and more needles. Yay.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Most GNOME apps now have a standardized About screen with links
(not buttons) for credits, website and links. Lukas called these
'selectors', which I like - but inconsistently; as well as
generic gnome_selector_foo needles, we have some app-specific
needles, and some with 'button' in the name.
Let's always call these 'selectors', always use generic needle
names (since the same needles should match for almost all apps),
and have the one remaining case where we have a 'button' (the
credits button in Evince) be the variant case, handled by putting
'button' in the needle name, but using the same tag as other
needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Various tests use scripts or data that are stored within this
distri itself. To improve reliability and lessen the load on
Pagure, let's move them all to data/ and retrieve them from the
test runner using autoinst_url instead of going out to Pagure.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This doesn't affect most cases of the test as it gets replaced by
a subvariant, but it *does* affect i3.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE seems to be taking quite a while to show up after we go to
graphical.target ATM. I've reported this to the KDE team, but
let's just bump the timeout so the test can pass.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For deciding whether to show a release as available for upgrade,
Discover now also checks its date as well as its 'type'. If the
date is in the future, even if the type is stable, it won't show
it unless the allow-pre-releases flag is set.
So, we need to also patch out the date. Just blanking it also
works, but let's hardcode it to the start of 2025 to be a bit
more realistic (in case there's ever a situation where the check
passes with an empty date, but fails with a date it should pass
with).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
these don't work correctly on IoT. Let's just skip them - testing
on the main release should be sufficient. Let's not do the EOL
consistency check on Rawhide, as Rawhide EOL is a pretty notional
concept. They don't line up ATM and I'm not sure we want to spend
too much time trying to make them line up. Let's just focus on
Branched.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The enhanced method of menu_launch_type does not cover for
special corner cases, where the application starts in a
specific mode (settings dialogues etc.)
This handles exceptions for Software.
The previous version of menu_launch_type took the name of the
application as an argument and it started the application.
To maximize the application or to check that it has started indeed
we had to do it manually.
Now, the application also takes "maximize => 1" or "checkstart => 1"
to maximize the application or check that it has started as optional
arguments to avoid doing it manually, while it still accepts just
the name of the application and behaves like it did before.
Note that if you decide to use the checkstart argument, you
also need to update the check-needles.py script to whitelist
the application needle tag, see the example test scripts
attached to this PR.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/329
Sometimes, update notification would pop up during the testing
and prevent the needles from being matched correctly.
The addition to the code makes the notifications go away for updates.
Before this PR, we would have a different naming scheme
of application running needles for Gnome, a.k.a
apps_run_application, while for KDE we had application_runs.
This PR unifies all name under the Gnome scheme,
replaces the tags in the needles and test scripts.
This PR fixes https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/330
When we added fmw to apps_startstop tests it was only preinstalled
on Silverblue, but now it's on KDE and Workstation too.
Also includes a needle that matches on part of the UI, which
will work on all desktops.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Normally, the /etc/fedora-release file shows a text consisting
of "Fedora release" with the version number and a spelt out
version number in brackets.
For Fedora 42, the code name Adams should be used instead to
celebrate Douglas Adams's number of the universe.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/360
This is intended to address the problem that we don't use the
buildroot repo for Branched between branching and u-t activation,
which often causes problems when newly-created updates depend on
builds that recently went stable. So instead of having logic in
the tests that sets up the buildroot repo only if the release is
Rawhide (or ELN), we'll do the decision about whether to set it
up in the scheduler (which decides based on info from Bodhi), and
just have the 'set up the repo' steps in the tests. Let's also
use the same code and repo name for ELN and non-ELN so we don't
have pointless divergence there.
Depends on the matching fedora_openqa change, of course - without
that, no test will use the buildroot repo.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't really need two separate CANNED conditionals and two
dnf commands, let's flip it around like this, and give the
remaining command 300 seconds not 240.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This isn't a test of fwupd, and we seem to be having issues with
it in openQA lately - I think maybe the fwupd server is blocking
openqa because we're hitting it too much? Anyway, let's disable
it for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds two initial custom layout install tests for webUI, and
wires up some library functions for other tests to use. They may
need refining over time but this should be good enough for a
start.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Fixes#347.
This PR adds two tests to check the EOL written in the
/etc/os-release file. The first test checks that the
SUPPORT_END value lies at least 12 months in the future.
The second test checks the EOL values in Fedora Schedule,
Bodhi, and /etc/os-release if they are the same.
Now fwupd is showing us dbx updates, it seems we get *both* a
"Restart & Update..." button *and* a "Download" button at the
same time. If we click Restart & Update without clicking Download
first, package updates aren't applied.
So, tweak this logic yet again so that it 'breaks the tie' in
the way we want (if both buttons are present, click the download
one, take download out of the tags, and go to the next loop
iteration). Hopefully this solves the problem without breaking
any other paths.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
webui has a language selection screen as of anaconda-42.24-1, so
handle it. For now still handle it not appearing, we can drop
that path once the update is stable and in a compose.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
