Commit Graph

1212 Commits

Author SHA1 Message Date
Lukas Ruzicka d3a6beda3d Add the graphical upgrade tests.
This PR builds on some elements of the current upgrade process,
such as upgrade_boot, upgrade_preinstall, upgrade_postinstall, but
replaces the upgrade_run with graphical_upgrade_run to use graphical
methods to upgrade the system.
This would not be possible without necessary settings, that are
performed by graphical_upgrade_prerequisites.
Works for both Gnome and KDE.
2024-04-15 23:08:58 -07:00
Adam Williamson 34eba25858 KDE apps_startstop firewall: type password faster
This is obviously more prone to mistypes, but firewall-config
seems to be timing out if we take more than 25 seconds to type
the password, and we take juuust too long with type_very_safely,
even after tweaking the sleeps to shorter wait_still_screens
here. We could twiddle with those even more, but let's just go
with type_safely for now, if that turns out to be too unreliable
I'll change tack.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-15 23:08:20 -07:00
Adam Williamson 86c975e49a Whoops, fix a wait_still_screen call from last commit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-13 14:59:14 -07:00
Adam Williamson 7465985c50 Add some waits to root password creation on ARM
This is failing quite consistently lately because we're typing
too fast, we need to wait a bit after the sudo su at least. Let's
be safer.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-13 14:22:30 -07:00
Adam Williamson adedf40b6f apps_startstop: don't rollback, defend against KDE mistypes
This comes from trying to fix the annoying recurring problem with
mistypes in KDE which has been going on since at least December.
First, we add the attempt to kind of 'precache' the kicker menu
in aasetting.pm. Then, I thought, all this snapshot loading has
to be putting a lot of load on the workers. And when each subtest
passes, it shouldn't really be necessary - they all end with
quit_with_shortcut(), which verifies that the app exited and we
got back to a blank desktop, so successful subtests should not
usually interfere with each other. We probably only want to
rollback on *failed* subtests, which is in fact openQA's default
behavior. There only seems to be one case where a test changes the
system state such that later tests might be affected, so I kept
always_rollback just for that one. I've run this through three
cycles on GNOME and KDE and it looks good.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 15:53:48 -07:00
Adam Williamson 96c75c5e28 Update tour needles for upstream changes
Various changes to the Tour text needed needle updates. The
final screen doesn't say "Have a nice day!" any more, so let's
rename that needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 12:04:25 -07:00
Adam Williamson 6b6cd20956 clocks/timer: don't always expect 'start' button
Since 46, the 1 minute button is a quickstart, it doesn't just
set the timer but starts it. So we can't always expect to have
to click the start button. Let's keep it working both ways for
now for respin testing, we can drop it once we're sure we're not
doing any testing on F39 any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 11:11:58 -07:00
Lukas Ruzicka b0bc18b187 Add a 'navigation' test for GNOME.
This PR partly solves the issue #301 when it adds the navigation
testing for Gnome. It uses the keyboard combinations to cycle through
running applications and checks that applications could be switched
accordingly. It also tests that you can switch between workspaces
and that you can move an applications to another workspace.
2024-04-10 21:46:43 +00:00
Adam Williamson 7e0cd0898e Make UEFI testing the default, add Secure Boot testing
This essentially inverts the x86_64 machines so that '64bit' is
UEFI and instead of a variant 'uefi' machine we have a variant
'bios' machine that is BIOS. The point is to make UEFI testing
the default. We also enable Secure Boot in the UEFI testing,
and add a test of UEFI fallback booting on various products.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 09:24:52 -07:00
Adam Williamson f1c23be409 Fix #2268505 workaround - move the actions push earlier
Oops, all other actions pushes need to happen *before* the reboot
action push, or the logic breaks.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-26 08:29:25 -07:00
Adam Williamson 1739f38061 Add a workaround for RHBZ #2268505
My anaconda patch for ensuring we get an EFI boot manager entry
on installs using bootupd (currently only 'canned', i.e. Atomic,
installs - IoT and Atomic Desktops - are affected) was rejected
and dropped in the latest Rawhide build. It'll probably also be
lost in the next F40 build. So install tests of affected images
have started failing. This is a kinda awkward workaround: on
UEFI canned installs, we check whether it looks like there is no
"Fedora" efibootmgr entry, and if so, we delete the entry for the
optical drive, so hopefully we'll boot via fallback path from
the hard disk on reboot.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-25 17:25:10 -07:00
Adam Williamson cc832a4ad8 Fix new logic in Clocks test
Whoops, we can't just use a straight match_has_tag there as we
did another assert in the middle...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-25 08:22:53 -07:00
Adam Williamson 8ebf3bdddb Tweak Clocks alarm test for change in upstream code
Upstream now sets an alarm with no explicit recurrence inactive
after it is stopped (previously it assumed recurrence every day).
We need the test to handle both behaviours for a while (until
we are no longer testing < 46.0 anywhere).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-24 09:47:21 -07:00
Adam Williamson a0eb2827b8 aarch64 kernel arg fixup: fix for bootupd
We now have some images using bootupd, at least Fedora IoT images
from F40 onwards do. These don't have a /etc/default/grub and
don't really intend you to run grub2-mkconfig. As advised at
https://github.com/ostreedev/ostree/pull/3150#issuecomment-1998768240
let's just add our required arguments directly in the BLS snippet
files.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-15 09:30:52 -07:00
Adam Williamson dc7ff01f27 toolbox: handle OCI images (as produced by Kiwi)
We changed to building toolbox container images with Kiwi. These
are OCI archives, not docker archives. So we need to call skopeo
appropriately.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-14 14:34:38 -07:00
Adam Williamson 4ee74a0bca iot_zezere_remote: add a safety wait after adding ssh key
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-14 08:32:25 -07:00
Adam Williamson 41f2adaf46 os_release: update for osbuild change to ISO names
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-14 08:25:26 -07:00
Adam Williamson 2b51d930b9 Tweak the IoT incomplete hub workaround a bit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-08 08:49:52 -08:00
Adam Williamson 89686bceba desktop_background: drop F40 exemption
Now we have F40 backgrounds, we can drop this exemption and have
the test always fail on non-Rawhide again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-07 16:05:51 -08:00
Adam Williamson 8da5b7c010 Work around IoT incomplete hub issue
This is a workaround for
https://github.com/osbuild/images/issues/309 , the IoT installer
showing incomplete spokes in the main hub. We work around it by
visiting them all.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-06 17:52:04 -08:00
Adam Williamson a5f9c88584 Tidy a couple of things that got missed in earlier commits
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-05 14:27:10 -08:00
Adam Williamson 33ea6ef506 podman: give bats test a bit longer to run
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-03 09:04:16 -08:00
Adam Williamson aeac70814b Drop the addrepo_metalink_graphical test
Anaconda has dropped the ability to interactively configure
additional repositories, so this test cannot work any more.
It's now possible only with inst.addrepo or a kickstart.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-01 16:55:59 -08:00
Adam Williamson 246e1c02af Give FreeIPA uninstall a bit longer to run
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-29 19:23:04 -08:00
Adam Williamson 54f69463b0 Use NUMDISKS=2 (or higher) on all update flavors
We stopped doing this on Server because it caused problems with
tests that use a disk image uploaded by another test, e.g. the
cockpit tests - they use the `/etc/fstab` from the disk image
the parent test uploaded, which says to mount the second disk as
/mnt/update_repo, but since this is a new test it has a fresh,
empty second disk with no filesystems to mount. This tries to
fix that by making _console_shutdown.pm edit that line back out
of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree
flavor, which had a similar problem with the overlay and rebase
tests using a disk image uploaded by the install test).

We need to fix this because FEDORA-2024-9b9da603e1 is so big
it causes the tests that don't use a scratch disk to run out of
disk space.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-28 15:26:01 -08:00
Adam Williamson 4c83347d54 Adjust for webUI deferral to Fedora 41
Same as the deferral from 39 to 40, except one of the tests has
gone away in the mean time.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-27 19:49:40 -08:00
Adam Williamson 10b7f66a02 desktop_printing: wait a bit after switching back from console
This test is failing a lot recently on KDE because when we try
to launch konsole immediately after returning from a console to
the desktop, it is mistyped and we get a browser instead. Let's
try a little sleep after the switch back in case KDE just needs
to settle down a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-27 10:17:38 -08:00
Lukas Ruzicka 614a2f5e5d Fix the failing KDE startstop tests. 2024-02-27 12:18:45 +01:00
Lukas Ruzicka e069a25396 Create a workaround for Flatpak Fonts start up.
On Silverblue, Fonts cannot be started using the menu_launch_type
for the first time, or it starts and crashes immediately.
However, if Fonts are started with flatpak run org.gnome.font-viewer,
it seems that the application starts and holds.
Let's start it using this workaround and when it still crashes, let's restart.
2024-02-23 12:38:42 +01:00
Adam Williamson 777e378a0d toolbox: test image from the compose where possible
per https://pagure.io/fedora-qa/issue/766 , this is a hole in our
current test approach: we are testing whatever the current
'stable' toolbox image is for the release, not the image that
is produced as part of the compose. This enhances the test to
test both, when possible. If the var TOOLBOX_IMAGE is set, we
will first check that a 'normal' `toolbox create` works - i.e.
that all the toolbox logic works right and it can actually find
a default image to download - but then we throw that toolbox
away, download the image (the value of the var is expected to
be a URL for the image file), register it with skopeo, and then
recreate the container using that image. Then we proceed with
the rest of the test as usual.

If TOOLBOX_IMAGE is not set, the test should proceed as before,
using the 'default' downloaded image.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-22 17:17:05 -08:00
Adam Williamson c956e41e50 base_update_cli: handle updates-testing not existing on ELN
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-22 16:44:21 -08:00
Lukas Ruzicka 1cd5455eae Revert "Restart Fonts on Silverblue (bug)"
This reverts commit 5ae9926225.
Unfortunately, the fix did not work on Staging, and some
other precaution will have to be made.
2024-02-22 17:01:51 +01:00
Lukas Ruzicka 5ae9926225 Restart Fonts on Silverblue (bug) 2024-02-22 15:31:14 +01:00
Adam Williamson 0425be0d8e live_build: hack anaconda-webui out of kickstart on g-i-s update
We need to special-case the g-i-s update to get it stable before
we can actually change this in the real kickstarts repo...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-21 12:16:18 -08:00
Adam Williamson b3e5f7fb54 live_build: drop a hack that should no longer be needed
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-21 12:13:43 -08:00
Adam Williamson d85f199f13 deduplicate needle: grant_access / gnome_allow
these were duplicates. In GNOME 46-beta this dialog seems to be
in 'light mode' at least some of the time, so we'll keep the
'light mode' gnome_allow needle we added for Snapshot but
rename it and change its tags. We'll wipe the 'dark mode'
gnome_allow because it should be just the same as the existing
grant_access needle. The two tests that used gnome_allow are
changed to use grant_access.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-21 08:12:32 -08:00
Adam Williamson 8e929f079f Try and make the archiver test more robust
These are some changes @lruzicka left lying around uncommitted
on openqa01.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-20 17:40:55 -08:00
Lukáš Růžička 3b4c94bb55 Add some more fixes for Nautilus. 2024-02-20 15:58:25 +01:00
Lukáš Růžička d656bc45a6 Wait until the dialogue settles 2024-02-20 15:01:14 +01:00
Adam Williamson 47e9f576c0 CoreOS: tweak rpmostree_rebase test to work from OCI remote
The rpmostree_rebase test has been failing on CoreOS 40+ for a
while. Per
https://github.com/coreos/fedora-coreos-tracker/issues/1672
it turns out this is because FCOS actually deploys OCI remotes
by default now. Rebasing from an OCI remote to an ostree
remote (as we are trying to do here) requires specifying the
registry, so let's do that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-19 15:00:34 -08:00
Lukáš Růžička e2e22341d3 Fix nautilus issues. 2024-02-19 14:19:52 +01:00
Lukáš Růžička 99b4d6ec17 Run scripts with assertions.
Creating the .invisible.txt file was done using non-assertion commands.
The tests have been failing for some and it seems like the commands
did not run correctly. Running them with assertions will make sure
that they will run (or fail correctly).
2024-02-19 11:26:39 +01:00
Adam Williamson 98513a0185 archive: be safer about the 'archive' dialog appearing
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-16 17:53:42 -08:00
Adam Williamson caa4353fe0 maps: add some sleeps to the zoom test
These loops make us click extremely fast. This may cause
unreliable results, I think. At least, the test is failing a
lot lately, with results that look like it's not always getting
the expected four levels of zoom. Let's try a short sleep
between clicks.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-16 17:52:19 -08:00
Adam Williamson 106a63f101 Maps: try and be a bit safer with the export screen
It seems like the export screen takes a while to appear on 40 and
Rawhide ATM, and we might start typing before it's there. Let's
assert it's actually there before we start doing stuff.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-16 17:22:22 -08:00
Adam Williamson b3e5dd41cb desktop_update_graphical: try to workaround GGO #2442
In current F40 and Rawhide, this test is frequently failing
because gnome-software is behaving weirdly at startup - the
third-party software dialog moves around even more than before,
the app seems to get stuck in the "not responding" state
briefly sometimes, and there's a very weird state it gets into
sometimes where the window is shorter than usual and clicks
don't seem to register in the right place. While I'm trying to
bisect these bugs, these magic voodoo incantations (tested on
the staging instance) seem to mostly work around the weird
behaviour, and setting RETRY=2 should backstop it a bit further.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-16 12:50:05 -08:00
Lukáš Růžička 58544fcbce Wait until Loupe responds correctly. 2024-02-16 17:22:52 +01:00
Adam Williamson b371331436 _ostree_build: handle change to repos in workstation-ostree-config
The place where repos are defined changed on the F40+ branches
of workstation-ostree-config, this handles both possibilities.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-14 12:29:51 -08:00
Adam Williamson de4735f6f1 Soft fail on missing background for F40
...we know there isn't one yet.
https://bugzilla.redhat.com/show_bug.cgi?id=2230720

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-14 00:43:46 -08:00
Adam Williamson 375250000f Add a wait_still_screen to update_graphical for an animation
GNOME Software seems to be doing some kind of animation between
the third party dialog and the main UI, and we're clicking on
a banner instead of the update button. Try a wait_still_screen
to deal with this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-12 09:55:31 -08:00