There's this awkward path for the live image install tests on
updates. We run the 'are the correct versions of all the packages
installed' check on these tests to ensure the right versions
actually made it onto the live image. So we don't run
`dnf -y update` at the end of repo_setup_updates on that path,
because if we did that, even if the packages on the live image
were old, we'd update them there and hide the problem.
However, this causes a bit of an ordering issue, because in
order to set up the advisory repo, we need to install a few
packages. What if the update under test includes one of those
packages, or a dependency that wasn't already installed? In
that case, we wind up with the older stable version of the
package (because obviously we can't install the newer version
from the advisory repo *before we've set up the advisory repo*),
don't update it later, and so the 'correct version' check at
the end of the test fails. See:
https://openqa.fedoraproject.org/tests/1778707 for a case of
this happening with a python-cryptography update.
Up till now I was trying to handle this by just updating the
specific packages we install, but that doesn't account for
*dependencies* of them. I looked down the path of trying to
generate a list of all those dependencies and update all of
them but it looks a bit mad. So instead let's try this. On that
specific path, we'll generate the "all installed packages" list
*before* we run repo_setup, so it just doesn't include anything
that gets installed during repo_setup. The implementation is a
bit icky but not too horrible.
We *could* just *always* generate the all installed packages
list earlier, but then that would mean we *wouldn't* catch dep
issues in this kind of package on the other test paths, whereas
currently we do. I don't want to lose that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
5.27.1 is going to F37, and adds it. In the short term this
will waste a minute and a half and cause soft fails on all other
F37 updates until the update that adds this goes stable, but
I don't really feel like working around this, let's just live
with it till the update goes stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
OK, neither 'input' nor 'keyboard' actually gives us the Keyboard
pane, they both give results for uninstalled apps from Software
:(. 'hotkey' (which is one of the keywords in the .desktop file)
does seem to work, for now at least, let's try that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Contacts now has two burger menus, which is awkward. We need
specific needles to identify each, we can't rely on the generic
needle any more as it won't always open the right menu. We also
need to still work with the old UI for the flatpak.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE has a welcome tour now, on F38 and Rawhide at least. Let's
"handle" it with extreme prejudice...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On GNOME 44, typing 'input' is now giving us the Software page
for PulseAudio Volume Control, for some reason. Let's try typing
'keyboard' again and hope that works again now...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Logout seems to be taking a long time in Rawhide currently. Give
it longer to run, but soft fail. I'll add a bug link once I've
investigated and filed one.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In today's F38 and Rawhide, changes to the persistent overlay
stuff result in a boot warning you have to spam through. Let's
handle this as a soft fail so we don't have floods of failed
tests till it's fixed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Tested that this is not necessary on KDE or GNOME, and on current
KDE it actually seems to break stuff. It's better to just start
typing the password.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Typing a partial binary name no longer seems to work. Typing the
full binary name works, but differently from before; seems best
to do a partial entry name search so we launch the actual entry,
not the executable directly.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I've seen a few cases of this test failing because there was
some running operation when it tries to do `rpm-ostree install`.
I think this is GNOME Software checking for updates or something,
I've seen it on my own Silverblue install too. Let's just throw
some `rpm-ostree cancel`s at it and hope that helps.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
"Additional repositories" is now hidden behind a dropdown we
have to open first. This will make the test fail on anything
older than Fedora-Rawhide-20230121.n.0, but I don't think we
run this test anywhere that would be a problem.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a small test suite to test the Characters applications.
It displays several different groups of characters and then tries
to copy one of the characters and place it into a text editor.
GNOME Software no longer has a welcome screen in any current
Fedora (it was dropped between 35 and 36), but in Rawhide it now
has a popup that prompts you to enable third-party repos which
we need to get rid of, so just convert the welcome screen check
to handle that, and drop all the welcome screen needles.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems ending the test right after we create the mutex can
cause the client not to catch it, sometimes. So let's sleep for
a few seconds after creating it to make sure it does.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These tests weren't doing it, I guess it's just an oversight;
this is probably why they often fail on slow repos.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This makes the two rpm-ostree tests written for IoT - overlaying
and rebasing - work across all rpm-ostree-based flavors we
currently test (IoT, CoreOS and Silverblue) and runs them on
all those flavors.
This requires some other changes. For the Workstation ostree
installer update tests, we have install_default_update_ostree
upload a disk image and run these tests on that image. That means
install_default_update_ostree cannot use a scratch disk (as if
we boot it with two disks but only upload one, the subsequent
tests fail to boot, looking for the missing second disk), but its
specified disk size should be large enough for all updates.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
What's supposed to happen here is the `do_bootloader` invocation
a few lines back boots to the installer, then here, we wait for
the install to complete and the system to reboot, and match the
bootloader again. However, on PXE installs, the bootloader screen
can hang around for quite a long time here, and if it does, we
can match it again before the installer starts up, and move on
too early. Hence the sleep.
It seems on current Rawhide 20 seconds isn't long enough - we're
still matching the installer bootloader after the sleep, see
e.g. https://openqa.stg.fedoraproject.org/tests/2431660#step/_boot_to_anaconda/3
This is causing the test to almost always fail (it'll only pass
if the install+reboot takes less than five minutes). Let's bump
it to 60 seconds and hope that's long enough.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Sadly, dropping this sleep caused the test to start failing
again at least on F36, so we still need it - update the note.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Thankfully this all calmed down a bit so we can simplify it a
lot. Clean things up a bit at the same time; escaping nested
single quotes is a lot clearer than concatening blocks with
different quote marks.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The bug seems to have gone away, at least I don't see that this
soft failure has been hit much for the last two months.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It's been on 1 so long now I kinda don't want to change it to 3
or 4 or anything. That might break something. As long as it's not
causing any trouble let's just leave it on 1.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
PXE install on UEFI (incl. aarch64) is failing at present, this
seems to be due to a grub bug:
https://bugzilla.redhat.com/show_bug.cgi?id=2152763
we're really intending to test the client side here, not the
server end, so let's work around this problem on the server end
by installing a grub2 scratch build that's the package from just
before the bad change, but with the release and epoch bumped,
from a side repo.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This turns out to be overcomplicated. We don't need the special
handling for updates any more, because a few months after we
implemented it, we had to make sure the affected update tests had
an empty START_AFTER_TEST anyway, or else openQA would refuse to
schedule them. So we can just rely on the START_AFTER_TEST
condition for those now. We also don't need the additional
INSTALL_NO_USER condition; the only case where it's actually used
is for install_arm_image_deployment_upload on Workstation, and
that test does not have START_AFTER_TEST set, so the other
condition catches it for welcome screen handling purposes. There
is no need to nest the IMAGE_DEPLOY conditional inside a check
for the desktop and the INSTALL_NO_USER var either; we don't
test any other desktop on ARM, and the IMAGE_DEPLOY var is only
set for that one install_arm_image_deployment_upload test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We recently started using the buildroot repo for Rawhide update
tests, but weren't including it in the image build tests. This
should include it in all the image build tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
