1
0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-11-24 14:53:09 +00:00
Commit Graph

3339 Commits

Author SHA1 Message Date
Adam Williamson
9f2d44a181 drop a FIXME on enabling/disabling repos with sed
dnf5 config-manager can now do this, but the syntax is different
to dnf4, and honestly, it seems easier to just stick with this
going forward than make it conditional on dnf version until
dnf4 goes away. So let's stop marking this as a FIXME.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-22 16:14:29 -07:00
Adam Williamson
f4bd3c6f58 Add a container build test
This test, much like _live_build or _installer_build, builds a
container image in a way intended to be as similar as possible
to how official compose images are built. The purpose of the test
is to make sure updates do not break official container image
builds.

At the end of the test, we also check that the built container
is functional (at least, that we can run a 'hello world' command
in it). This can't really be rolled into podman.pm because that
test is more about testing podman itself, and it's just a one-
liner here anyway. We also run the 'if any packages from the
update are installed, are they the versions from the update?'
check inside the container, which required giving that check the
ability to 'wrap' the rpm commands to run inside a container.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-22 16:01:41 -07:00
Adam Williamson
d65aac4c83 Drop a no-longer-needed workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-19 16:30:52 -07:00
Adam Williamson
f9cda76860 Revert "Install composefs with podman/toolbox to workaround #2275820"
This reverts commit 7b29a8f1b2.
It should no longer be needed now containers-common added a
dependency.
2024-04-19 16:29:58 -07:00
Adam Williamson
5a548fd3c7 Update GNOME lock screen needle
Seems they changed it from the user's initials to a generic icon.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-19 16:18:34 -07:00
Adam Williamson
b976fb2d58 Yet another KDE system tray expander needle
Don't know why we need so many of these. There's something odd
about the panel in Plasma 6, I think.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-19 16:17:57 -07:00
Adam Williamson
4b3ee64ee8 Update gnome_reboot_confirm needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-18 12:59:41 -07:00
Adam Williamson
dd20d1f224 Add a missing needle for KDE graphical upgrade tests
This is the variant we hit when upgrading from Fedora 40 (the
button looks a bit different than on F39). Without it the test
for Rawhide (which upgrades from F40) will fail.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-18 12:55:48 -07:00
Adam Williamson
7b29a8f1b2 Install composefs with podman/toolbox to workaround #2275820
containers-common seems to have inadvertently introduced a hard
dependency on composefs, but not expressed it as a package dep.
While I'm trying to get that fixed, let's ensure the podman and
toolbox tests don't fail on it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-17 16:58:39 -07:00
Lukas Ruzicka
d3a6beda3d Add the graphical upgrade tests.
This PR builds on some elements of the current upgrade process,
such as upgrade_boot, upgrade_preinstall, upgrade_postinstall, but
replaces the upgrade_run with graphical_upgrade_run to use graphical
methods to upgrade the system.
This would not be possible without necessary settings, that are
performed by graphical_upgrade_prerequisites.
Works for both Gnome and KDE.
2024-04-15 23:08:58 -07:00
Adam Williamson
34eba25858 KDE apps_startstop firewall: type password faster
This is obviously more prone to mistypes, but firewall-config
seems to be timing out if we take more than 25 seconds to type
the password, and we take juuust too long with type_very_safely,
even after tweaking the sleeps to shorter wait_still_screens
here. We could twiddle with those even more, but let's just go
with type_safely for now, if that turns out to be too unreliable
I'll change tack.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-15 23:08:20 -07:00
Adam Williamson
a8f547f5ef Fix check_prerelease when we don't care
I inadvertently broke this in the logic change in the previous
commit to this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-14 08:36:50 -07:00
Adam Williamson
86c975e49a Whoops, fix a wait_still_screen call from last commit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-13 14:59:14 -07:00
Adam Williamson
7465985c50 Add some waits to root password creation on ARM
This is failing quite consistently lately because we're typing
too fast, we need to wait a bit after the sudo su at least. Let's
be safer.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-13 14:22:30 -07:00
Adam Williamson
a07ad20240 Add FEDORA-2024-b156c57ef1 as F40 workaround to fix upgrades
See https://bugzilla.redhat.com/show_bug.cgi?id=2274833. Upgrades
from F39 to F40 fail without this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 18:59:41 -07:00
Adam Williamson
fcbb00d0ac Drop workaround which went stable weeks ago
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 18:59:24 -07:00
Adam Williamson
adedf40b6f apps_startstop: don't rollback, defend against KDE mistypes
This comes from trying to fix the annoying recurring problem with
mistypes in KDE which has been going on since at least December.
First, we add the attempt to kind of 'precache' the kicker menu
in aasetting.pm. Then, I thought, all this snapshot loading has
to be putting a lot of load on the workers. And when each subtest
passes, it shouldn't really be necessary - they all end with
quit_with_shortcut(), which verifies that the app exited and we
got back to a blank desktop, so successful subtests should not
usually interfere with each other. We probably only want to
rollback on *failed* subtests, which is in fact openQA's default
behavior. There only seems to be one case where a test changes the
system state such that later tests might be affected, so I kept
always_rollback just for that one. I've run this through three
cycles on GNOME and KDE and it looks good.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 15:53:48 -07:00
Adam Williamson
2880edced1 More KDE needle updates
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 12:11:04 -07:00
Adam Williamson
96c75c5e28 Update tour needles for upstream changes
Various changes to the Tour text needed needle updates. The
final screen doesn't say "Have a nice day!" any more, so let's
rename that needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 12:04:25 -07:00
Adam Williamson
6b6cd20956 clocks/timer: don't always expect 'start' button
Since 46, the 1 minute button is a quickstart, it doesn't just
set the timer but starts it. So we can't always expect to have
to click the start button. Let's keep it working both ways for
now for respin testing, we can drop it once we're sure we're not
doing any testing on F39 any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 11:11:58 -07:00
Adam Williamson
394794989b Update KDE printing needle for some upstream change
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 10:06:51 -07:00
Adam Williamson
4938c0e5ae Make pre-release check a soft failure on osbuild images
osbuild is struggling with the pre-release warnings ATM:

https://pagure.io/fedora-iot/issue/57
https://github.com/osbuild/images/issues/515

Until that mess is cleaned up we can't really make sensible
assertions for osbuild images, so let's make the check a soft
failure for now (now we know about the bugs, we want to let
the rest of the tests run and not block them on this).

Note for IoT the behaviour has never really been correct (IoT
images never get pre-release warnings), but the logic in this
check matches the wrong behaviour (IoT composes always have
RC- labels even when they're clearly not RCs), so the test
didn't fail. While fixing this in osbuild we might try to get
'correct' behaviour for IoT, and then we'd need to tweak the
logic here.

While we're at it, tweak the implementation a bit; without this
tweak, implementing this 'soft fail if osbuild' behaviour is more
awkward and ugly.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-12 09:00:25 -07:00
Lukas Ruzicka
b0bc18b187 Add a 'navigation' test for GNOME.
This PR partly solves the issue #301 when it adds the navigation
testing for Gnome. It uses the keyboard combinations to cycle through
running applications and checks that applications could be switched
accordingly. It also tests that you can switch between workspaces
and that you can move an applications to another workspace.
2024-04-10 21:46:43 +00:00
Adam Williamson
5bd224687f Bump base version in ostree-parse-pungi.py
The flatpaks have been updated to the F40 runtime, so we need to
keep pace here.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-09 20:28:13 -07:00
Adam Williamson
d74fd9e6b4 Update some GNOME needles too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-08 13:21:08 -07:00
Adam Williamson
3ca5816696 Update some more needles for KDE
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-08 13:18:03 -07:00
Adam Williamson
d0912fd217 Fix blivet preserve_home test templates for UEFI switchover
Somehow these two got completely scrambled, testing the wrong
things on the wrong images. Fix them to match the non-blivet
counterparts, which are correct.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-08 12:19:51 -07:00
Deborah Brouwer
4eec75bec3 Get dns server address if host isn’t using systemd-resolved
The worker containers aren’t using systemd-resolved so
get_host_dns() can’t find the dns server causing tests that need
it to fail. Add an alternative place to look for the dns server address
that doesn’t rely on systemd-resolved.

Signed-off-by: Deborah Brouwer <deborah.brouwer@collabora.com>
2024-04-05 21:10:11 +00:00
Adam Williamson
c3baa66465 Bump retry count on some flaky tests
The silverblue tests are flaky because of
https://github.com/fedora-silverblue/issue-tracker/issues/548 .
The desktop_upgrade_encrypted test is flaky on Rawhide (so, when
booting from F40 initially) since we switched to UEFI, not sure
if it's because of UEFI somehow or just a timing coincidence.
Am going to look into it, but for now this should save lots of
manual restarting.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-05 08:01:16 -07:00
Lukáš Růžička
f48dc5a0d5 Add wait_still_screen to menu_launch_type.
This is actually a very useful to wait a little bit before
we try to operate with the started application, especially
if we want to send keys.

Having this wait time here can save us doing this every time,
we start an application.
2024-04-05 14:09:28 +02:00
Lukáš Růžička
d2238360b8 Revert "Add needles"
This reverts commit bde855b343.
2024-04-05 10:04:17 +02:00
Adam Williamson
5701d1a5a5 Try and robustify menu_launch_type a bit
The mistypes on KDE are bugging me. Let's see if this - mainly
the wait_screen_change, the other just seems like a logical
tweak - helps at all.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-04 11:19:16 -07:00
Lukáš Růžička
bde855b343 Add needles 2024-04-03 09:19:30 +02:00
Adam Williamson
64f06b6415 Don't run secure_boot_fallback on ELN for now
As we're not running on UEFI, it of course fails.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 21:16:39 -07:00
Adam Williamson
af7705f47b Run ELN tests on BIOS
ELN can't boot with Secure Boot enabled currently:
https://github.com/fedora-eln/eln/issues/183

Until that's resolved, let's go back to testing on BIOS (this is
easier than testing on non-SB UEFI for...reasons).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 15:39:42 -07:00
Adam Williamson
7e0cd0898e Make UEFI testing the default, add Secure Boot testing
This essentially inverts the x86_64 machines so that '64bit' is
UEFI and instead of a variant 'uefi' machine we have a variant
'bios' machine that is BIOS. The point is to make UEFI testing
the default. We also enable Secure Boot in the UEFI testing,
and add a test of UEFI fallback booting on various products.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 09:24:52 -07:00
Adam Williamson
85943b9aeb Update a GNOME needle for a slightly changed icon
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 09:20:07 -07:00
Adam Williamson
615be43cd5 Drop match level on KDE menu button needles
For some reason we keep getting not-quite matches on this, even
though we added two new needles. Let's drop the match level on
all existing needles as a coping mechanism.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 09:15:25 -07:00
Adam Williamson
cf51484b2e Update several KDE needles for Plasma 6 changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 09:13:54 -07:00
Adam Williamson
eaf7c48e0b Update several Arabic needles to translated versions
Someone's knocking out the Arabic translation, yay.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-02 09:12:38 -07:00
Adam Williamson
f1c23be409 Fix #2268505 workaround - move the actions push earlier
Oops, all other actions pushes need to happen *before* the reboot
action push, or the logic breaks.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-26 08:29:25 -07:00
Adam Williamson
1739f38061 Add a workaround for RHBZ #2268505
My anaconda patch for ensuring we get an EFI boot manager entry
on installs using bootupd (currently only 'canned', i.e. Atomic,
installs - IoT and Atomic Desktops - are affected) was rejected
and dropped in the latest Rawhide build. It'll probably also be
lost in the next F40 build. So install tests of affected images
have started failing. This is a kinda awkward workaround: on
UEFI canned installs, we check whether it looks like there is no
"Fedora" efibootmgr entry, and if so, we delete the entry for the
optical drive, so hopefully we'll boot via fallback path from
the hard disk on reboot.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-25 17:25:10 -07:00
Adam Williamson
66aa587755 Updated/variant Firefox main screen needle
Old one was a 96% match. Not really sure what changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-25 08:56:43 -07:00
Adam Williamson
cc832a4ad8 Fix new logic in Clocks test
Whoops, we can't just use a straight match_has_tag there as we
did another assert in the middle...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-25 08:22:53 -07:00
Adam Williamson
5332b99076 Update a bunch of GNOME needles for 46.0 changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-24 18:12:25 -07:00
Adam Williamson
8ebf3bdddb Tweak Clocks alarm test for change in upstream code
Upstream now sets an alarm with no explicit recurrence inactive
after it is stopped (previously it assumed recurrence every day).
We need the test to handle both behaviours for a while (until
we are no longer testing < 46.0 anywhere).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-24 09:47:21 -07:00
Adam Williamson
37150e430f Add yet another variant 'updates available' needle
GNOME seems to be growing a bunch of different variants of this
alert. Annoying.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-24 09:23:38 -07:00
Adam Williamson
f6273425ee Add FEDORA-2024-01fd5aa621 to workarounds
gvfs from the GNOME 46 megaupdate needs it. It's been pushed
stable but not included in a compose, yet.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-23 17:12:40 -07:00
Adam Williamson
92c822a719 Clear no-longer-needed update from workarounds
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-23 17:12:11 -07:00
Adam Williamson
222223309e Update another GNOME needle for 46
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-21 12:15:12 -07:00