This PR partly solves the issue #301 when it adds the navigation
testing for Gnome. It uses the keyboard combinations to cycle through
running applications and checks that applications could be switched
accordingly. It also tests that you can switch between workspaces
and that you can move an applications to another workspace.
The worker containers aren’t using systemd-resolved so
get_host_dns() can’t find the dns server causing tests that need
it to fail. Add an alternative place to look for the dns server address
that doesn’t rely on systemd-resolved.
Signed-off-by: Deborah Brouwer <deborah.brouwer@collabora.com>
The silverblue tests are flaky because of
https://github.com/fedora-silverblue/issue-tracker/issues/548 .
The desktop_upgrade_encrypted test is flaky on Rawhide (so, when
booting from F40 initially) since we switched to UEFI, not sure
if it's because of UEFI somehow or just a timing coincidence.
Am going to look into it, but for now this should save lots of
manual restarting.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is actually a very useful to wait a little bit before
we try to operate with the started application, especially
if we want to send keys.
Having this wait time here can save us doing this every time,
we start an application.
The mistypes on KDE are bugging me. Let's see if this - mainly
the wait_screen_change, the other just seems like a logical
tweak - helps at all.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
ELN can't boot with Secure Boot enabled currently:
https://github.com/fedora-eln/eln/issues/183
Until that's resolved, let's go back to testing on BIOS (this is
easier than testing on non-SB UEFI for...reasons).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This essentially inverts the x86_64 machines so that '64bit' is
UEFI and instead of a variant 'uefi' machine we have a variant
'bios' machine that is BIOS. The point is to make UEFI testing
the default. We also enable Secure Boot in the UEFI testing,
and add a test of UEFI fallback booting on various products.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For some reason we keep getting not-quite matches on this, even
though we added two new needles. Let's drop the match level on
all existing needles as a coping mechanism.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Oops, all other actions pushes need to happen *before* the reboot
action push, or the logic breaks.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
My anaconda patch for ensuring we get an EFI boot manager entry
on installs using bootupd (currently only 'canned', i.e. Atomic,
installs - IoT and Atomic Desktops - are affected) was rejected
and dropped in the latest Rawhide build. It'll probably also be
lost in the next F40 build. So install tests of affected images
have started failing. This is a kinda awkward workaround: on
UEFI canned installs, we check whether it looks like there is no
"Fedora" efibootmgr entry, and if so, we delete the entry for the
optical drive, so hopefully we'll boot via fallback path from
the hard disk on reboot.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Whoops, we can't just use a straight match_has_tag there as we
did another assert in the middle...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Upstream now sets an alarm with no explicit recurrence inactive
after it is stopped (previously it assumed recurrence every day).
We need the test to handle both behaviours for a while (until
we are no longer testing < 46.0 anywhere).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
gvfs from the GNOME 46 megaupdate needs it. It's been pushed
stable but not included in a compose, yet.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We now have some images using bootupd, at least Fedora IoT images
from F40 onwards do. These don't have a /etc/default/grub and
don't really intend you to run grub2-mkconfig. As advised at
https://github.com/ostreedev/ostree/pull/3150#issuecomment-1998768240
let's just add our required arguments directly in the BLS snippet
files.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It's queued for stable but blocked by the freeze, and
FEDORA-2024-9c9174f3dd requires it. Let's just make it a
workaround to clear the other update's tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We changed to building toolbox container images with Kiwi. These
are OCI archives, not docker archives. So we need to call skopeo
appropriately.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Just checking for the package update list is wrong in a couple of
ways:
1. It goes wrong for child tests that inherit the main disk from
the parent test, but not the second disk where the actual update
repo lives. They'll have the package list, but not the repo. So
we should test for both the package list and the repo dir.
2. It goes wrong for side tag tests, because we don't write a
package list (or create an update repo) on that path. So on that
path let's check whether we already created the repo definition
we use on that path.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The existing needle matches on bits of the keyboard layout
selector under the note and the Help button next to it, for some
reason (I don't recall why). The Help button is gone because we
killed installer help, and the keyboard layout indicator is not
always shown any more, there's no reason to match on it here
that I can recall.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We got a different console font affecting this needle recently.
It *may* be due to building the ARM minimal disk image (which I
think is the only image whose tests hit this needle) with osbuild,
but anyway, it's not a bug, so we should just accept it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
https://bugzilla.redhat.com/show_bug.cgi?id=2268505 made it
clear that this is a bit of a hole. We don't test installing the
Silverblue image we build on UEFI, only on BIOS. Add this as a
separate test so we don't uselessly upload a disk image we won't
use for any follow-on tests.
This also adds an anaconda build that fixes a bug in this path
as a workaround for F40, so the test won't fail on all F40
updates.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Now we have F40 backgrounds, we can drop this exemption and have
the test always fail on non-Rawhide again.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This should mean all F40 update tests get the new background,
and we can once again make that test fail if it doesn't see it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a workaround for
https://github.com/osbuild/images/issues/309 , the IoT installer
showing incomplete spokes in the main hub. We work around it by
visiting them all.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Anaconda has dropped the ability to interactively configure
additional repositories, so this test cannot work any more.
It's now possible only with inst.addrepo or a kickstart.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Java 21 update broke FreeIPA and was waived stable. This
update rebuilds dogtag-pki and should fix it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We stopped doing this on Server because it caused problems with
tests that use a disk image uploaded by another test, e.g. the
cockpit tests - they use the `/etc/fstab` from the disk image
the parent test uploaded, which says to mount the second disk as
/mnt/update_repo, but since this is a new test it has a fresh,
empty second disk with no filesystems to mount. This tries to
fix that by making _console_shutdown.pm edit that line back out
of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree
flavor, which had a similar problem with the overlay and rebase
tests using a disk image uploaded by the install test).
We need to fix this because FEDORA-2024-9b9da603e1 is so big
it causes the tests that don't use a scratch disk to run out of
disk space.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I missed reverting the g-i-s page skip config when deferring
the webUI Change to F40. Now I changed the conditional in the
openQA test to expect F40 to behave like F39 and earlier, it's
failing because it's seeing too many pages. This should resolve
it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
jlebon and jmarrero figured out the fix for this, while we wait
for the packager to merge it their preferred way, let's use
scratch builds to work around it so every update doesn't get a
failed test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This test is failing a lot recently on KDE because when we try
to launch konsole immediately after returning from a console to
the desktop, it is mistyped and we get a browser instead. Let's
try a little sleep after the switch back in case KDE just needs
to settle down a bit.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On Silverblue, Fonts cannot be started using the menu_launch_type
for the first time, or it starts and crashes immediately.
However, if Fonts are started with flatpak run org.gnome.font-viewer,
it seems that the application starts and holds.
Let's start it using this workaround and when it still crashes, let's restart.
per https://pagure.io/fedora-qa/issue/766 , this is a hole in our
current test approach: we are testing whatever the current
'stable' toolbox image is for the release, not the image that
is produced as part of the compose. This enhances the test to
test both, when possible. If the var TOOLBOX_IMAGE is set, we
will first check that a 'normal' `toolbox create` works - i.e.
that all the toolbox logic works right and it can actually find
a default image to download - but then we throw that toolbox
away, download the image (the value of the var is expected to
be a URL for the image file), register it with skopeo, and then
recreate the container using that image. Then we proceed with
the rest of the test as usual.
If TOOLBOX_IMAGE is not set, the test should proceed as before,
using the 'default' downloaded image.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Another case where an update was submitted after its deps had
gone stable, but before they made a compose. Maybe we really need
to set up a buildroot repo for tests of updates to branched-before
-bodhi-activation too. I'll think about it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Now the fedora-kickstarts PR to remove anaconda-webui was merged
we need this for tests to pass. Remove after next compose.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We need to special-case the g-i-s update to get it stable before
we can actually change this in the real kickstarts repo...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
these were duplicates. In GNOME 46-beta this dialog seems to be
in 'light mode' at least some of the time, so we'll keep the
'light mode' gnome_allow needle we added for Snapshot but
rename it and change its tags. We'll wipe the 'dark mode'
gnome_allow because it should be just the same as the existing
grant_access needle. The two tests that used gnome_allow are
changed to use grant_access.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Seems we need more space on the scratch disk used by the ostree
build test, it's failing due to lack of space. Tested on lab, it
worked.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This also rolls in a few Arabic translations, I think, cos I
can't be bothered going through and picking them out.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The rpmostree_rebase test has been failing on CoreOS 40+ for a
while. Per
https://github.com/coreos/fedora-coreos-tracker/issues/1672
it turns out this is because FCOS actually deploys OCI remotes
by default now. Rebasing from an OCI remote to an ostree
remote (as we are trying to do here) requires specifying the
registry, so let's do that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Creating the .invisible.txt file was done using non-assertion commands.
The tests have been failing for some and it seems like the commands
did not run correctly. Running them with assertions will make sure
that they will run (or fail correctly).
apps_menu_firefox and firefox/launcher are matching on the same
thing. Also, launcher-kde actually happens to match current
GNOME, so rename it to be generic. Give the launcher needles
the apps_menu_firefox tags and wipe the apps_menu_firefox
needle.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These loops make us click extremely fast. This may cause
unreliable results, I think. At least, the test is failing a
lot lately, with results that look like it's not always getting
the expected four levels of zoom. Let's try a short sleep
between clicks.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems like the export screen takes a while to appear on 40 and
Rawhide ATM, and we might start typing before it's there. Let's
assert it's actually there before we start doing stuff.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The new gnome-shell from FEDORA-2024-f543e8595c requires this.
We don't use a buildroot repo for Branched even in the short
period where it acts like Rawhide (no karma or time requirements)
so let's just use a workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In current F40 and Rawhide, this test is frequently failing
because gnome-software is behaving weirdly at startup - the
third-party software dialog moves around even more than before,
the app seems to get stuck in the "not responding" state
briefly sometimes, and there's a very weird state it gets into
sometimes where the window is shorter than usual and clicks
don't seem to register in the right place. While I'm trying to
bisect these bugs, these magic voodoo incantations (tested on
the staging instance) seem to mostly work around the weird
behaviour, and setting RETRY=2 should backstop it a bit further.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
F38 is the oldest thing we test, and it has this flow. So let's
just drop the check here and always hit tab twice.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We should wipe it both before and after running repo_setup to
really get rid of it. Ideally we should stop doing this at all,
but for right now the repo isn't there for Rawhide post-F40
branching yet, so let's keep this until that's done.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Er, whoops, that was *not* meant to get rolled into an unrelated
commit and break every test on the public instance. Sorry.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The place where repos are defined changed on the F40+ branches
of workstation-ostree-config, this handles both possibilities.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't need this anaconda scratch build any more (the official
build has the fix now), but we *do* need the new lorax update I
just built to fix installer image builds with branched F40. See
https://github.com/weldr/lorax/pull/1379
Signed-off-by: Adam Williamson <awilliam@redhat.com>
GNOME Software seems to be doing some kind of animation between
the third party dialog and the main UI, and we're clicking on
a banner instead of the update button. Try a wait_still_screen
to deal with this.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It kinda looks like the text has been moved closer to the icon
in the latest gnome-software, such that the little '1' bubble
is now in the needle match area if it's present. Obviously that
may vary, so let's tweak the needle match area a bit to the
right so it's hopefully consistent and still matches on older
versions.
Another bunch of these timed out. Not sure why. Maybe it's when
I run a lot of them at the same time? Let's try this, again.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a scratch build with my proposed patch:
https://github.com/rhinstaller/anaconda/pull/5460
to get the tests to start passing again, so we don't have the
flood of red which makes it hard to spot other problems.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
accounts.fp.o seems to be unreliable again today, let's drop this
again so tests don't fail on it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit 56c9e80f60.
Things seem to have settled down with the mass rebuild and this
test seems to be back to consistently taking about 90 minutes.
It seems to be timing out a lot on Rawhide lately. Not sure if
it's just mass rebuild stuff, but anyhow...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit ab5b1a4367. A
new colord build has been pushed which should resolve the issue,
so I'm disabling the workaround to ensure that's the case.
There's an untranslated string here, so this has to be a
workaround needle. I didn't get to reporting the missing
translation yet.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Somehow, colord is sometimes failing to start in stable Rawhide
ATM - I don't know how this problem got through testing. It's
now blocking other updates.
Doing this only on x86_64 is lazy and wrong, but the logic gets
way more complicated if we need to allow potentially *two* things
to fail on aarch64 and ppc64le, and it's the weekend and we
don't gate updates on those arches so I'm not doing it. Hopefully
this will be resolved quite quickly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The color shell prompt feature just got applied to VTs in Rawhide.
That broke...everything. Here, new needles!
Signed-off-by: Adam Williamson <awilliam@redhat.com>