The test case was completely changed earlier this year. Per
https://bugzilla.redhat.com/show_bug.cgi?id=2093080#c17 , atagoh
wants us to use the new test case for validation and have openQA
automate it, so, this does that. I already updated the matrix
earlier in the year, I will update fedora_openqa when this is
merged.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit b3e5dd41cb.
Some testing on lab seems to indicate it's not needed any more,
at least several runs with the workarounds reverted have passed.
Will put them back if we hit failures.
As discussed in https://pagure.io/releng/failed-composes/issue/6538
we noticed a gap in openQA coverage here. We don't check the
versions of packages lorax installs to the installer environment,
and those packages do not make it to the installed system, so if
there's a dep issue that prevents a package in the update from
being included in the installer environment, but the same dep
issue isn't caught on any other path, we miss the problem. This
wires the updvercheck.py script into the _installer_build and
_ostree_build tests to catch this kind of problem, and makes it
capable of parsing pylorax.log files into its preferred format
to enable that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We also need to use the old workflow for f40 respin testing.
Also, put back a wait and some comments that were left out of
the 'old' flow.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Silverblue Rawhide contains an older version of Maps than Workstation
Rawhide. This PR allow to test both of the versions depending on the
subvariant variable. When Silverblue and Workstation will use the
same version, this should be removed.
In https://openqa.fedoraproject.org/tests/2724005 , it seems that
cockpit.service was restarted by a systemctl daemon reload during
install of the necessary packages to enrol to the domain, which
causes the web UI to unexpectedly show a "Server has closed the
connection" error and fail the test.
It seems like we can mitigate this by triggering a daemon reload
manually before launching cockpit - the reload seems to be sort
of 'queued' at this point, I'm not sure why - so let's do that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The new layout needs to scroll down to reveal some of the elements.
Use send_until_needlematch to navigate in the application and
arrive at those switch buttons to enable the requested functionality.
This makes them diverge slightly from official ones, but...we
need it to run updvercheck.py, and doing that in a container is
a pain because we have to get the input files in. So, let's just
make sure it's in our ostrees.
It used to be in them anyway, but last night it stopped being in
Rawhide ones for some reason.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Intentionally or not, a recent anaconda change made it so the
"text mode sucks, use VNC instead?" question is no longer ever
shown - https://bugzilla.redhat.com/show_bug.cgi?id=2293672 .
So, we should handle the flow where we just go straight to the
hub. If they decide this was intentional and kill the question
for good we can drop the path that handles it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These filenames changed again:
https://pagure.io/workstation-ostree-config/pull-request/523
and I'm sick of dealing with it. Let's just fire away at *.yaml,
it should be safe. If we ever need to check a file we can add
a targeted upload_log temporarily.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
using podman run leaves containers and/or images lying around; if
the thing being tested is big enough, we can wind up with enough
that podman just stops working (I saw this while testing the
python 3.13 side tag). To avoid this, use podman run --rm, which
clears up the container/image after the command is run.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Tests running in cloud instances are failing because installing
rpm-ostree packages, particularly writing the OSTree commit, is often
taking longer than the timeout maximum of five minutes.
Stop these tests from needlessly failing by doubling the allowable
time for installing each package.
Signed-off-by: Deborah Brouwer <deborah.brouwer@collabora.com>
When we test major KDE version upgrades, we get a post-upgrade
version of the welcome center again after the upgrade, which we
don't really expect. This breaks the background test because the
welcome center is in the way of the background, it doesn't seem
to break any other update tests. So let's just handle it here.
Fortunately this version has an "OK" button we can click.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a couple of places where we do menu_launch_type in KDE
without doing this workaround first, and they do run into the
bug sometimes. Let's factor it out from the few places it's
already repeated, and add it to the places it is missing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds both the Gnome and the KDE tests to test the
Desktop Keyring. After a discussion with the Brno team,
how this could be tested without the need to rely on
external servers to log into, we set up a local FTP server,
we will log into it and remember the credentials and verify
that the credentials will be stored in the keyring correctly.
When using a side repo for testing a COPR or a side tag, there
may be unsigned packages. We set gpgcheck=0 to make dnf okay
with this, but gnome-software still shows a warning. Let's
click through it so the test can complete.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We already try twice, but that seems to be not enough for the
annoying #2280840, we're often seeing failures.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The service cockpit enables is different when it detects dnf5,
since cockpit 317. Let's just make this an F40/F41 boundary
thing, and add the cockpit 317 update as a workaround for F41
until it goes stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think the lack here is sometimes causing us to click more
times than we should. Let's do this to try and make sure we
don't click once it worked.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're getting failures in the update network install tests today
which seem to be because we're using an image built with systemd
256 to install systemd 255. This is because systemd 256 has been
tagged but isn't in a compose yet, and we use the Rawhide tag
repo when building the installer image but we don't add it as an
additional repo for the install itself.
This is obviously a hole in the process, we should use the extra
repos, where appropriate, all the way through. So this makes us
use both the Rawhide tag repo (when doing a Rawhide install test)
and the workarounds repo (when there are workarounds) for network
install tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is intended to reduce the amount of traffic we generate to
flathub, particularly so we can run this test on updates as well
as composes. We have to set a proxy and trust an SSL cert.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This works more or less like testing side tags. We also fix up
some flow problems with this path (that also affect the side tag
case), and enable the package checks on this path - it's not too
hard really, we just need to write the updatepkgs file when we
set up the repo, which we can do with dnf repoquery.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is the same thing we do for the app_startstop tests in
aaa_setup, applied to a couple of other places we use
menu_launch_type in KDE and it's having trouble.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When running these tests on updates we inherit the main disk
image from server_cockpit_default, which has the repo config,
but the actual repo data is on HDD_3 which is not inherited.
We need to re-download the updates here to ensure they're
available to the tests (the automatic update test installs the
dnf-automatic package, so it should pull it from the update repo
if it is part of the update being tested).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When testing the update that implements the dnf5 switchover, we
need to patch the kiwi config on the fly for dnf5.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The bug number is wrong and I can't find the right one, d'oh.
We could *probably* safely remove this right now but I'm not
100% sure, I think it should be fine when F38 is EOL.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit b203f41f55.
The bug has been worked around for some time with a downstream
patch. Dropping the extended timeout means we'll notice if the
workaround is dropped prematurely or stops working.
This whole complicated loop looks like it's no longer needed for
current KDE. It seems like we always refresh, then we hit
"Update All", and from there we go straight to "Restart Now".
Clicking the button always seems to work, we never seem to need
to click "Refresh" again. So, let's drop it and simply expect to
see and click Restart Now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This test, much like _live_build or _installer_build, builds a
container image in a way intended to be as similar as possible
to how official compose images are built. The purpose of the test
is to make sure updates do not break official container image
builds.
At the end of the test, we also check that the built container
is functional (at least, that we can run a 'hello world' command
in it). This can't really be rolled into podman.pm because that
test is more about testing podman itself, and it's just a one-
liner here anyway. We also run the 'if any packages from the
update are installed, are they the versions from the update?'
check inside the container, which required giving that check the
ability to 'wrap' the rpm commands to run inside a container.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
