Improve Fedora Media Writer tests by validating written USB media
against the original ISO and by making the workflow work reliably
across classic and Silverblue variants.
The previous post-write verification relied on mounting the USB partitions
and checking for a handful of expected directory structures. This is replaced
with a stronger end-to-end check that verifies the embedded checksum on the
target device and then compares the raw device contents byte-for-byte
with the downloaded ISO, treating the expected cmp “EOF” condition as success.
To support Silverblue, where the required isomd5sum package is not installed
by default, a generic toolbox helper is introduced in utils.pm to create a toolbox,
install required packages inside it, and run the necessary command there.
The media writer test template is updated to explicitly model the USB target
as /dev/sda by enforcing a single-disk setup and adjusting that QEMU uses
a snapshot to create the device and thus not create a file lock on it to
prevent other tests from running.
This PR adds the test suite for Fedora Media Writer, which is
an application to download and copy the installation media files
to the a USB disk which can be then used for installation.
So far, we have been testing this application with manual testing,
this is an attempt to automate it. It tests the following:
* It chooses the Workstation media, downloads it, saves it locally
and writes it to the USB disk. It also checks that the usual
content exists on the USB disk.
* It opens the previously downloaded ISO and writes it to the USB disk.
Then it check that the usual content exists on the USB disk.
* Removes the installation from the USB disk and restores it to its
original state.
* This does not test any hotplugging mechanisms yet.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/450
Fix after review
It's going to go stable fairly soon, but I need to go out and I
don't want to wait to re-trigger all the other failed tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Cockpit has worked around the Firefox bug with the Services page,
so let's add the fixed builds to workarounds and re-enable the
tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It looks like we haven't need to do this for some time. The search
needle doesn't match current Cockpit at all, so we'd be getting
test failures if we needed to scroll the sidebar to see the
Updates link, but we aren't. This needle hasn't matched for 8
months.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also explain why, in cleanup-needles and also in the test code.
This is for the "gnome-initial-setup before anaconda" flow for
Workstation live. It was temporarily implemented, which is why
we have the code and the needles, but then disabled because the
changes weren't successfully upstreamed and we didn't want to
maintain them downstream any more. It's still *supposed* to come
back at some point, so let's keep the code and the needles for a
while at least.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Let's not make this conditional, let's just add it, and add the
update that adds the screen as a workaround until it's stable and
in the next nightly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE is switching to a Workstation-like model where the user and
root password screens in the installer are suppressed, and a
KDE-specific initial setup wizard runs on first boot. This adapts
to handle that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This seems to be going wrong quite often in the remote desktop
client test - we're getting 'lo' and trying to do a static config
on the loopback interface, which clearly isn't going to work.
This should either fix it or at least give us some idea why it's
going wrong (maybe we don't have any ethernet connections?)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Lately quite a lot of tests are failing here on GNOME because we
don't type terminal properly (we wind up with 'trminal' or
'teminal' usually). At first I thought my os-autoinst typing
delay change might be causing this, but it actually seems to have
started before I deployed that, so no idea what's going on. Let's
click instead of typing, to avoid the problem.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The failure message here isn't very helpful unless you know what
is going on and where to look for the errors. Let's improve it
so packagers have a better chance of understanding failures.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It's still going through gating tests but I want to be able to
start restarting other failures with the fix.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Thanks to @jlinton , who worked out that the repeated inputs on
aarch64 seem to be caused by the relatively lengthy delay on
key down that we get when typing very slowly. Typing a bit faster
is actually more reliable in this case. I'm working on an
os-autoinst tweak that would avoid the lengthy key holds when
typing very slowly, which would allow us to revert this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
No reason these should only be run on the x86_64 live. Also
refactor the remote_desktop_client test to share the code for
opening a connection in GNOME Connections with the RDP install
test. The shared code is a combination of the two which should be
more robust than the previous remote_desktop_client version, which
was relying far too much on timings working out. This caused it to
fail often on x86_64 as well as on aarch64.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Extending the wait to 3 seconds in the utils.pm function probably
wasn't necessary; the problem is we were reimplementing it in
_do_install_and_reboot but without the wait. Let's revert the
wait to 3 seconds and use the function.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
10 seconds isn't enough, the aarch64 upgrade tests often fail
because they're still Refreshing Data after 10 seconds, then
the ignore button appears but we already quit looking for it.
Let's wait 150 seconds for *either* the ignore button *or* the
UI to appear, if the UI appears we're done instantly, if the
ignore button appears, click it and wait for the UI.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test for remote connection. It consists of a server part
in which Gnome RDP connection is set up, enabled, and the server is run,
and the client part that uses Connections as a client to connect to the
server.
The test is considered successful when the connection is established
and the user has logged into the system.
Fixes: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/333
This reverts commit 8a657904d0.
We already had a tweak in _ostree_build to put python3-rpm in the
ostree; the real problem is that had broken with changes to
workstation-ostree-config. The next commit solves it properly.
This started failing in install_default_update_ostree
_advisory_post because the ostree no longer has python3-rpm baked
in, by the looks of it. Moving the execution of updvercheck.py
into a toolbox container *should* fix it with no undesirable side
effects, I hope. It looks like /tmp should be shared between the
host and the container so the file location should be OK.
The use of `toolbox run --` as a wrapper instead of interactive
toolbox entry/exit was suggested by Gemini PR review:
https://github.com/AdamWill/os-autoinst-distri-fedora-ai-test/pull/26#discussion_r2431461670
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Assisted-by: Google Gemini Code Assist
We are missing a check that the expected keyboard layout is used
at the desktop, for non-switched layouts (e.g. French). This is
actually broken for Workstation live installs at present, so we
really ought to be testing it. Along the way, let's clean up the
conditionals a bit, and let's do the actual input testing at a
console not the overview, since we want to do it on KDE as well
as GNOME now, and KDE's kicker menu is a pain to type in.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also add keyboard layout selection handling for webUI, natively
for the KDE live, via GNOME settings for Workstation live.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The bug causes a crash at boot on Silverblue installer, we need
the fix to stop all Rawhide Silverblue tests failing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
FreeIPA team bumped the F41 NVR past the F42 one, so now upgrade
tests from F41 to F42 are broken. To workaround this we need
bumped rebuilds for F42, F43 and F44, because if only do F42 the
problem moves to F42->F43 upgrades, etc.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For some reason, the login name is filled in double time.
Let's fix it with adding more wait time and also an extra
delete key to wipe the field properly.
See https://github.com/rpm-software-management/dnf5/issues/2435 .
Lately this is sometimes failing because something else holds a
lock on the RPM database. To work around that, let's try again a
few times if it fails, with a bit of a wait.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In GNOME 49 RC, the Time Zone page has been changed so the search
box is active by default. Previously we had to press tab to
activate it. We need to keep working with both, so unfortunately
that means adding a needle :/
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Two different dep issues started breaking tests today -
a libiscsi soname bump broke deps in qemu and libvirt, and a
gnome-shell-extension-background-logo update had an invalid dep.
This adds fixes for all three as workarounds so I can get the
tests of each to pass, and other tests to pass while these are
going through gating.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
@kparal pointed out that it's a bit inconvenient to quickly check
the journal and any SELinux AVCs for a failed test. All the info
is in the /var/log tarball, but you have to download it, extract
it, and know how to run journalctl on it. For convenience, this
adds a text dump of the journal and one of any AVCs found with
ausearch.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This makes us create a generic cloud disk image and then run
tests on it, just like we do for Workstation and KDE lives,
everything netinst, Silverblue, and a container image. This is
intended to help avoid anything like
https://bugzilla.redhat.com/show_bug.cgi?id=2390898 happening
again - if we'd had these tests set up and gating enabled, the
update that caused that bug would've been gated.
This requires some changes that affect other things: bind mounting
the whole of /dev in our mock config when using simple isolation,
and forcing START_AFTER_TEST to "" for kiwi_build (this is needed
to override the setting of it in the updates-cloud flavor). bind
mounting is what koji does for 'real' kiwi build jobs so it's fine
(in fact, better); previously the mock config was shared with
livemedia-creator, but we don't use that any more so it's not a
problem.
It's a bit icky when we get to the point of having *multiple*
+START_AFTER_TEST settings, but I can't see another way to get
everything right here and still share the test suite definition
with compose tests.
Moving the definition of KIWI_PROFILE for container tests is not
strictly related, but I noticed it was weird while working on
this and couldn't leave it alone.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
