This adds a new script - cleanup-needles.py - to use for cleaning
up old needles. It has to be used in conjunction with a database
query; the comment at the top explains how to do that query and
export the needed information. It produces a git commit with
needles that haven't matched since a certain date (specified in
the sql query) removed, subject to a 'keeplist' of needles we
keep even if they seem to be old.
I also enhanced check-needles.py to check for cases where tests
seem to be trying to match a tag we have no needles for. This
was necessary to find cases where the cleanup script was too
aggressive (i.e. the things that wound up in the 'keeplist'),
but it also turned out to find quite a lot of cases where the
code really *was* looking for a needle that had gone in a
previous cleanup or that never existed; the commits before this
one clean up a lot of those cases.
The code to decide which string literals are needle tags is
pretty dumb and hacky and needs some manual cueing sometimes -
that's what the `# testtag` changes in this commit are for.
Making it smarter would probably require this script to get a
lot more complicated and either incorporate or become a
tokenizer, which I don't really want to do.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It looks like neither of these has been a problem for some time.
The notification needle has not matched for a year. The akonadi
needle doesn't exist any more - it was cleaned up in the 2021
needle cleanup, meaning it hadn't matched for weeks in 2021. I
checked the last several months of KDE app start/stop tests and
don't see any case where there was a stray notification that we
missed. So I think we can just ditch this whole mechanism for
now; if we have problems with these notifications again in future
we can put it back.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For several releases now, the 'new user mode' of g-i-s is just
gone: https://gitlab.gnome.org/GNOME/gnome-initial-setup/-/issues/12
so this whole path where we used to be able to set up Japanese
input methods on first boot after install doesn't work any more.
We had to set up a whole different route to set the input method
via control center instead (which lives in _graphical_input).
This block is never reached any more, and the needles for it were
cleaned up in 2021.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Since 022865ab we do not use start_with_launcher on KDE any more.
The needle for it has since got lost in an unused needle
cleanup. Let's just drop this branch for now; we can add it back
if we ever need it again.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The needle that backs this workaround was dropped in the 2021
needle cleanup, so it's never worked since then. I checked, and
no aarch64 tests seem to be failing on this any more.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Upstream https://github.com/os-autoinst/openQA/pull/4973 requires
us to poke things here a bit. This only works with the newer
os-autoinst and openQA (there may be a way to conditionalize it
to work with both, but I can't be bothered figuring it out, let's
just update).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a fairly longstanding issue in GDM where switching layout
just doesn't work sometimes:
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/6066#note_1707051
there doesn't seem to be any progress on getting this fixed, and
it's annoying constantly restarting tests that fail on it. So
this just makes us try three times to switch before giving up.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In install_repository_hd_variation , we have two disks attached,
so when we reach the INSTALLATION DESTINATION screen we expect
we have to select the correct target disk. However, as of the
most recent anaconda build in Rawhide, anaconda realizes we're
using a file from the other disk as our install repo, and
'protects' it - i.e. it does not show it on the INSTALLATION
DESTINATION screen, meaning only one disk is shown there, and
when only one disk is shown, it's pre-selected. So when we click,
we actually *un*select it, and the test fails.
Fixing this is a bit awkward; I wanted to add a new variable,
ANACONDA_PROTECTED_DISKS or something, and subtract that from
the NUMDISKS count; but we can't really do that as the enhanced
protection isn't in F38, and we can't easily set variables
differently on different releases (we'd have to set them in the
scheduler code, not just put them in the templates). So we just
code in a doofy condition for this instead. Maybe when F38 is
EOL we can change to the variable approach.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
An lvm2 issue which breaks the installer (#2180557) and anaconda
renaming the .desktop file for the Workstation live welcome
screen, which caused it not to appear -
https://pagure.io/livesys-scripts/pull-request/12 .
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Reasoning:
1. pandoc is not in critpath so will not itself be tested
2. pandoc is widely used and actively maintained
3. package is noarch
4. package has minimal deps
Hopefully this will work for everything. For some reason, the
"use python3-blivet for pykickstart tests" fails mysteriously
sometimes, see e.g.
https://openqa.stg.fedoraproject.org/tests/2672282
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Somehow, the dummy package being python3-kickstart causes the
graphical update tests (only) to fail for pykickstart updates
(that's the source package of python3-kickstart). The CLI and
Cockpit update tests are fine with this and pass.
To workaround this, use python3-blivet as the dummy package for
the graphical update tests when testing an update that contains
python3-kickstart. I've updated the test repo to contain both
dummy packages.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So...there's an ffmpeg update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-a5e10b188a
which went stable. It includes new sonames of all the ffmpeg
libs. It also pulls in a thing called oneVPL, which has a bug
that breaks ostree composes.
There's a big kf5 update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-b086a98f78
which contains kf5-kfilemetadata, which is built against ffmpeg.
Neal made sure that update's build of it was built against the
new ffmpeg and submitted both for stable at once - but the tests
on the kf5 update failed because they weren't run against the
new ffmpeg as it wasn't yet stable, and the kf5 update was
ejected from the push because of the failed tests.
So now we have the ffmpeg update stable but not the
kf5-kfilemetadata rebuild for it, which will break KDE stuff,
and the oneVPL issue means ostree composes will all fail.
This adds the ffmpeg update as a workaround so we can re-run the
tests for the kf5 update and get them to pass so we can push it
stable. It also adds the oneVPL update as a workaround so ostree
compose tests don't start failing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Move the xauth disablement and the disabling of studies into
_setup_browser, instead of repeating it in a couple of other
places (but *not* doing it in the zezere test, where we should
be doing it). Drop some explicit package installs that should
no longer be needed as Firefox and/or X.org now depend on those
things. Install the current default fonts (Noto), not the old
ones (DejaVu).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Just using a scratch build for now as my fix hasn't been reviewed
and may have dumb mistakes in it, but it does seem to fix the
openQA test at least.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
A different way to address the same problem as 56936df7 . Let's
just *remove* the repo management packages after we're done
creating the repos. dnf will automatically remove the unused
dependencies too. This fixes the python-cryptography case at
least - I tested.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's this awkward path for the live image install tests on
updates. We run the 'are the correct versions of all the packages
installed' check on these tests to ensure the right versions
actually made it onto the live image. So we don't run
`dnf -y update` at the end of repo_setup_updates on that path,
because if we did that, even if the packages on the live image
were old, we'd update them there and hide the problem.
However, this causes a bit of an ordering issue, because in
order to set up the advisory repo, we need to install a few
packages. What if the update under test includes one of those
packages, or a dependency that wasn't already installed? In
that case, we wind up with the older stable version of the
package (because obviously we can't install the newer version
from the advisory repo *before we've set up the advisory repo*),
don't update it later, and so the 'correct version' check at
the end of the test fails. See:
https://openqa.fedoraproject.org/tests/1778707 for a case of
this happening with a python-cryptography update.
Up till now I was trying to handle this by just updating the
specific packages we install, but that doesn't account for
*dependencies* of them. I looked down the path of trying to
generate a list of all those dependencies and update all of
them but it looks a bit mad. So instead let's try this. On that
specific path, we'll generate the "all installed packages" list
*before* we run repo_setup, so it just doesn't include anything
that gets installed during repo_setup. The implementation is a
bit icky but not too horrible.
We *could* just *always* generate the all installed packages
list earlier, but then that would mean we *wouldn't* catch dep
issues in this kind of package on the other test paths, whereas
currently we do. I don't want to lose that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is to handle a temporary condition where the screen isn't
present on the KDE base disk images for F38 or F39 yet, so they
only see it on the second boot on update tests, but don't handle
it because we marked it as already 'done'.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE has a welcome tour now, on F38 and Rawhide at least. Let's
"handle" it with extreme prejudice...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The F38 update that breaks this hasn't gone stable due to gating
and the F39 update will be pulled in once the tests are done
and it goes stable, but doing this anyway so I can re-run the
tests on the F38 plasma-workspace update and push it stable,
and rerun all the failed Rawhide tests without waiting for all
the tests on this update to finish first.
We still need to handle 43 only requiring one for now, and we
can't just make it release-dependent until 44 is stable for both
38 and Rawhide, so let's use a needle match temporarily. Only
44 has these eye/pencil icons on this screen.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
As we're not getting composes ATM this isn't being pulled into
tests of subsequent updates, but we need it to be or else there
are issues.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
With Rawhide updates, we quite often run into a situation where
a test runs after a *later* version of the package has already
gone stable. This even happens for stable releases too, though
less often. The current shell-based check just always fails on
this case, but it's usually OK, and manually marking every case
like this with an "it's OK!" comment gets tiring. Instead, let's
use a smarter Python script to do the check. We compare the EVR
of all installed update packages with the EVR of the package
from the update. If it's the same, fine. If the installed package
is lower-versioned, that's always an error, and we fail. If the
installed package is higher-versioned, we check whether the
update already went stable. If it did, then we soft fail, because
probably nothing can go wrong at this point (this is the usual
Rawhide case). If the update did not yet go stable, we still
hard fail, because something can go wrong in this case: if the
update *now* goes stable, the older version from the update may
be tagged over the newer version the test got (presumably from
current stable).
If anything goes wrong with the Bodhi check, or the test is
running on a task not an advisory, we treat both cases as fatal.
The script also gives easier-to-understand output than the old
approach, which should be a bonus.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Same conditions as used in main.pm to load the tests in the
normal flow. It makes no sense to do this on non-update tests,
or on the non-matching support server case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is safer if the advisory stuff was done on a previous test
run. Hilariously, this exposed a dumb mistake I made years ago
in installedtest.pm and never noticed before: the calls to
advisory_* at the bottom of that file are meant to be in the
post_fail_hook, but they weren't, which meant they got called
by the scheduler. This didn't cause any failures because the
first line caused them to return immediately based on a get_var
call (which it's OK to do in the scheduler), but changing it
to a script_run call (which it's *not* OK to do in scheduling)
caused all the tests to blow up immediately and confused me
*a lot* until I spotted this!
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When I enabled _advisory_post for live and ostree install tests,
the point was to check that updated packages were included in
the install media and used during installation. We shouldn't run
a system update in _repo_setup_updates on this path because it
will hide the problem if the updated packages weren't included.
The INSTALL variable is for this purpose - it was previously
used to skip _advisory_post on the same path. At the same time
let's remove some stray settings of this var on non-update tests
as it serves no purpose there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Per discussion at https://pagure.io/fedora-ci/general/issue/376
it really feels like this is the right thing to do. There are no
buildroot overrides for Rawhide, so we don't have to worry about
cross-pollution. The buildroot repo only contains builds that
have been tagged stable since the most recent Rawhide compose,
and thus will go into the next one. It makes sense to test later
updates against these. This avoids issues like:
https://openqa.fedoraproject.org/tests/overview?distri=fedora&version=38&build=Update-FEDORA-2022-30a952e331&groupid=2
where the tests of an update failed because it requires another
update which had been submitted and tagged stable previously, but
after the last compose.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also use get_var("TEST") for installer_build - no point trying
to upload these logs for the other tests in the same flavor,
they won't be there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is like the existing tests that build network install and
live images then install them, only for Silverblue. First we
build an ostree, using the standard configuration for the release
and subvariant but with the 'advisory' and 'workarounds' repos
included, so it will contain current stable packages plus the
packages from the update and any workarounds. Then we build an
ostree installer image with the ostree embedded, again including
advisory and workarounds repos in the installer build config so
packages from them will be included in the installer environment.
The image is uploaded, which completes the _ostree_build test.
Then an install_default_update_ostree test runs, which does a
standard install and boot from the installer image.
We do make a change that affects other tests, too. We now run
_advisory_post on live image install tests, as well as this new
ostree install image install test. It was skipped before because
of an exception that's really only needed for the netinst image
install test. In that test, packages from the update won't be
included in the installed system, so we can't run _advisory_post
on it. But for ostree and live image build/install tests, the
installed system *should* include packages from the update, so
we should check and make sure that it does.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's no need to run this twice (which can happen on some
paths), so if the first file already exists, just bail. Also,
don't bother uploading the config files any more - that was just
for debug while I was making this stuff work, now it works, and
this saves some time.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's one point in the tests where we may log into cockpit for
the second time in one run (it depends how a package update
process goes). When this happens, we don't get prompted again
for admin access, so we need to *not* expect that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems to time out a lot on lab but not on prod, for some
reason. Let's just give it a little longer.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
From 280, the cockpit package disabled logging in as root by
default. We could wipe that config file, but it seems better to
respect the default config and log in as the admin user 'test'
instead of as root.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is simpler if we just always lowercase $iso, plus it saves
us when somebody (*cough*) messes up the casing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
mock 3.2 does not allow running as root. This breaks the live
image build tests. It's being reverted upstream, this scratch
build has the reversion backported.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
desktop-backgrounds update went stable, but we also need an
f37-backgrounds update which was only just submitted to fix the
problem.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
podman tests are currently failing for all Rawhide updates. Not
sure what triggered this, but this update seems to fix it, though
it's failing its own test suite so not yet being pushed stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We use variables to track test state across modules, sometimes.
As this is all internal to the test logic I didn't bother always
making these variables upper-case, but os-autoinst now treats
lower-case variables as a fatal error, so we have to change.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This issue appeared when we started testing Rawhide updates, but
I only noticed it today. When testing Rawhide updates after
Branch point, the upgrade tests upgrade from Branched to Rawhide.
On Branched, updates-testing is enabled by default. We only
disable it when we reach `upgrade_run`, but by that point we've
already done a `dnf -y update` in `upgrade_preinstall` and
potentially installed other packages in steps between
`upgrade_preinstall` and `upgrade_run`. That can cause problems,
like today all FreeIPA upgrade tests on Rawhide are failing
because there's a newer freeipa in updates-testing for F37 than
is in the current Rawhide compose.
Solve this by disabling updates-testing before we do the update
in `upgrade_preinstall`. To avoid excessive code duplication,
factor out the repo disabling code.
We'll do this twice on upgrade tests now, but it shouldn't be a
problem.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When we run Firefox directly on X lately, we often hit a bug
where X just suddenly exits in the middle of doing stuff in
Firefox. I'm not sure if this is a bug in X or in Firefox (if
Firefox crashed, X would immediately exit). Let's see if this
helps get any info on what's going on with Firefox.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
samba-client-libs with an soname bump made today's Rawhide, a
freeipa rebuilt against it did not, so we need this in overrides
or all FreeIPA tests fail on all Rawhide updates.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Editing the files under /ostree/deploy doesn't work on Rawhide
any more because it's been made read-only. Thanks to Timothée
Ravier for pointing out this way of doing it, which is better
anyway and avoids awful quotation issues in the code.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
A samba update was pushed for F37 without rebuilds of sssd and
freeipa (which needed to be rebuilt against it). Tests of any
of those updates on its own will fail, so including them all as
workarounds so the tests for each should work. I'll make sure
they all get pushed stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The file conflict between older webkit2gtk4.1-jsc and new
javascriptcoregtk4.1 is causing all Workstation update tests to
fail on all Rawhide updates. This new version of webkitgtk adds
obsoletes to javascriptcoregtk4.1 which we hope should fix it.
Adding the x86_64 build directly as other arches aren't done
yet, and we only run the Workstation update tests on x86_64
anyway.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
openQA choked badly on
https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71
because it's, well, huge - 87 builds including texlive, which
has hundreds (thousands?) of subpackages. This exposed several
frailties against such updates.
First of all, we set NUMDISKS to at least 2 for *all* update
tests, which should mean they all stash the RPMs from the update
on a non-system disk and avoid problems with space exhaustion.
After that, just extend a few timeouts in particularly fragile
places, including one which is specific to texlive (as I don't
know of any other source package with so many subpackages).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
F35 to F36 FreeIPA server upgrade test is failing because the
latest F36 package is lower-versioned than the latest F35
package, and FreeIPA's upgrade scripts are written to fail if
this is the case. The update bumps the F36 NVR to be higher
than the F35 one.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
bodhi-client should depend on it, but technically since we have
code that calls `koji` directly here, we should probably also
include it in our install anyway, so not marking this as a
workaround.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR changes the way to download the test data into the VM.
Although it does not use a disk image as suggested in one
of the review, it does not clone the entire repository, but
a simple tar.gz file that holds the data which will be
distributed into the directory structure.
This way, the amount of data needed to be downloaded dropped
from approximately 50MB to below 2MB.
Also, the existing test suites were adapted to this situation.
Now we've ditched syslinux in Rawhide, we should just always
expect to see grub if release number is > 36.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It seems when we quit Firefox back to a VT on ppc64le, the system
hangs. Not sure why, but we can deal with it by rebooting the
system and logging back in as root if it happens. Also take the
opportunity to clean up the flow of quit_firefox so we always
check that we get back to a console then wait 5 seconds for the
console to settle, so all the tests that call it can stop doing
that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
On Rawhide update tests we often don't seem to get to the login
prompt in 10 seconds, so tweak the code a bit to let us specify
a timeout in root_console, and use 30 seconds here.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
After typing the username, on some Rawhide tests, it's taking
over 30 seconds for the password prompt to appear. This isn't
ideal but we don't want the test to fail on this. Give it up
to 45 seconds.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Wow, so the real way to do config overrides is, uh...obscure.
One file pointing to another file, both with mandatory comment
lines and one with a weird required value. Wat. Anyway, this
works in a VM. I still don't know why the policy for the first
run page isn't working as advertised.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The new way isn't working, so put the old way back but change it
to use user_pref instead of pref to see if that helps, and upload
the files for checking.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is required for script_run invocations in current
os-autoinst to tell it what to do if script_run times out, rather
than failing or succeeding (this behaviour was always undefined).
Right now you just get a deprecation warning but at some point
not setting it will be fatal. This tells it to die if script_run
times out; 0 would mean "just keep going, even though the
terminal is probably still blocked waiting for the command to
return".
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Thanks to Mike Kaply in
https://bugzilla.mozilla.org/show_bug.cgi?id=1703903#c18 , this
should avoid Quick Suggest onboarding, "Total Cookie Protection"
onboarding, and future annoying things using the same mechanisms
without using the prefs mechanism or going after each one case
by case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Fedora infra workers got changed so their resolv.conf doesn't
have the correct DNS server addresses in it any more, it just
has 127.0.0.53 (the systemd-resolved resolver address). We need
to query resolved to get the correct addresses.
This is quick and hacky; it doesn't accommodate anyone running
a worker host that *isn't* using resolved, and it doesn't handle
IPv6 server addresses correctly if any are present (in infra we
currently don't use any on the worker hosts). I'll try and find
time to refine it but need to deploy this for now to make the
tests pass again.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When we're logging via the serial console when a test fails and
no network is available, we only log the journal from the current
boot. But we might well need to see messages from previous boots.
So let's just log the whole journal.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Otherwise we race between the needle match and the screen scroll
update, and can click the wrong thing.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The Cockpit update test relies on scrolling the Cockpit left bar
to find the Software Updates page, but in Firefox 100, the scroll
bars disappear shortly after the last time you moved the mouse.
To deal with this, instead of looking for a scroll bar to scroll,
we'll click the search box in the left bar and hit 'up' to scroll
it to the bottom.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We get these variant needles with the keyboard icon and layout
name at slightly different relative vertical positions every
time we do a Final RC. I think the layout is affected by the
pre-release warning text no longer being present. Usually each
cycle something changes with the icon or with font rendering,
so the needles have to be updated, as in this case.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I meant to include this in the earlier commit that does the same
for type_very_safely, but forgot.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I noticed today that we're timing out all the time on these
wait_still_screens in gnome-terminal, because it uses a big
flashing cursor and the similarity between "cursor there" and
"cursor not there" is less than 45 or 42 (it seems to be 38.x).
So let's drop these levels to 38, hopefully that's not too low.
There are probably more places where this is an issue, I'll
change them as I notice them.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can't use the default similarity_level for wait_still_screen
when there's a flashing cursor - flashing cursor will always
cause the similarity level to be too low and the wait will just
time out at 30 seconds. Cut it to 42.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
check_desktop tries to catch when the overview is open and close
it. But with GNOME 42, it seems the inactive "Activities" button
is shown briefly on login before GNOME opens the overview. If
check_desktop catches that, it will think the overview isn't
open and it doesn't need to do anything. So if we match on first
cycle through the loop, let's wait_still_screen then match again.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
So these kind of things go through perl *and* bash string
interpretation and the escaping can get pretty wacky. Turns out
we need *eight* slashes here to get four through to bash (which
we need to deal with *sed*'s escaping rules), and it only works
in single quotes for some reason, not double quotes.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a bug in rpm where doing `rpm -q [NVRA]` doesn't work
if the NVRA contains a caret. To make it work you have to add
a literal slash character before the caret character, so we add
a sed command to do that, when we're checking whether packages
from the update actually got installed.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Scratch build because we don't have a single-build update with
the fix, it's likely gonna be included in the 5.24.2 megaupdate.
I don't want to use that whole thing as a workaround, so I did
a 5.24.1 scratch build with the fix instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The intent is that if the ps check finds nothing we'll use the
loginctl output, but that doesn't work because script_output
doesn't return the output if the script fails. There's an arg
you can pass to make it do so, but let's just make it always
succeed instead, by adding a ||: to the second grep like we have
for the first.
Also, I noticed this problem because the ps check started not
working on F36 KDE because none of the processes we check for
are shown as running on a tty, so let's add one more that *is*
shown as running on a tty...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Tests that use Firefox started failing recently because Firefox
grew yet another stupid pop-up thing that *might* show on start
up, this appears to be trying to get you to sign up for a
feature called "quick suggest". After half an hour trawling the
relevant code, this is my best guess as to how to turn it off.
Don't know for sure if it works because the thing doesn't pop
up every time, but it at least doesn't make things worse.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
KDE tests are failing on all F34 updates ATM due to
https://bugzilla.redhat.com/show_bug.cgi?id=2049560 , add the
update to fix it. Also drop 33 from the workarounds hash as it's
EOL.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds implementations of distribution methods to make them
usable on Fedora. It adds the following methods
* ensure_installed (to install packages)
* become_root (to switch to root account)
* script_sudo (run script with sudo)
* assert_script_sudo (run and assert a sudo script)
It also adds a helper script to the utils.pm
* make_serial_writable
that makes the serial console writable for normal users
and so enables to run commands that check their progress
by sending messages to the serial console. Normally, they
fail, because the messages will not be written their, so
the checking mechanism will never see them.
As of today's Rawhide compose, user accounts added in Rawhide
have admin privs by default. For now we need to handle both
possibilities here (click the box if it's not clicked already);
after F35 EOL we can just drop all handling of that box.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Rawhide had a failure today where the dropdowns moved between
us matching and clicking, so we clicked the wrong one:
https://openqa.fedoraproject.org/tests/1005662
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This fails sometimes just because we're too early, or something.
Also with GNOME Shell 41rc, alt-f1 no longer works to open and
close the overview. super *does* seem to work in KDE these days,
so let's switch from alt-f1 to super everywhere.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a bug in current F35/Rawhide which causes the GOA screen
in g-i-s not to show up. Since failing on that will block a lot
of testing, let's handle it as a soft failure.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
As suggested by @kparal, this adds a test that specifies an
additional repository using a metalink. The repository contains
a single package, 'testpackage', that supplements glibc (so it
should always get installed). The test runs an install then
checks that testpackage got installed.
We also deduplicate a pair of needles which were matching on the
same anaconda UI feature (an "add" button) and use that same
needle in this test.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Remove a whole chunk of needles that haven't matched for more
than 3 months. Also move a few needles to appropriate locations,
simplify some code chunks that relied on removed needles (if
we're not matching the needles, we don't need those chunks any
more), and drop some other no-longer-needed conditionals for
older releases.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We can match on apps_menu_button_active before the overview has
totally loaded in, and it seems the alt-f1 press can be dropped
in that case. Add a wait_still_screen to try and avoid that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a test for fingerprint-based login, as requested by
@benzea in #223. We use the fprintd dummy device to let us
simulate scanning a fingerprint, and check various scenarios
recommended by @benzea.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a new test that automates the above mentioned test case.
It starts the installation in text mode using the `install_text` test
case, which it interrupts using the Anaconda crash trigger.
When the crash happens, it goes through the process of reporting
the bug to Bugzilla, checks that Bugzilla sends a positive
confirmation of the action, but also performs some REST API
calls to do a proper check and then it closes the bug to clean up.
This systemd update went stable ages ago. But now we need to add
a jpegxl update as a workaround to avoid KDE live build tests
failing on the problematic aom->jpegxl-libs->gimp dependency
chain I identified yesterday. It makes KDE live builds pull in
too many packages and fail because they run out of space.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Still hitting a fail sometimes on the spoke after Installation
Destination, when anaconda is still sorting things out and the
test tries to do stuff too fast. e.g.
https://openqa.stg.fedoraproject.org/tests/1206252 . See if this
helps.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Otherwise if it lags a bit we might try and click the Help!
button on the hub, and if that happens before anaconda has caught
up, we won't open Help at all.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Some fixups for anaconda_help. Two runs of it failed today around
handoff from the root password screen to the install progress
screen; add a couple of wait_still_screens there to make it
safer. Drop the added nonlive needles, because they're too
permissive, causing problems for other tests (they're matching
before they should); instead we solve the problem of spokes being
highlighted by just pressing shift-tab a few times. And fix some
tabs to be spaces.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR automates the mentioned testcase to test that Help can be
displayed in Anaconda during the installation. It navigates through
the available Help screens and if it can see it, it finishes.
This test runs after `install_default_upload` to override the
installation defaults defined for all primary tests.
Delete a duplicated needle.
Reformat list extensions to make it nicer.
Get rid of wrong export and an empty line.
Delete empty line.
Use _boot_to_anaconda for booting and move subroutine accordingly.
Add variable to templates.fif.json
Delete trailing whitespace.
Fix calling the pretest.
Move help checking to another place.
The Modularity tests rely on an external script to test the modular
behaviour of DNF. There is a potentional risk that the connection
is be down and the script cannot be downloaded.
This enhancement uses a regular OpenQA perl test case script to only
invoke DNF commands and parse their output to test the same behaviour
that we have been testing already.
This enhancement picks a random module for each of the operations,
and thus tries to mimick reality a little bit more.
Make the 'deactivate overview if it's active' thing a bit more
robust by asserting the inactive state after deactivating it,
and add new needles for the new RC (text got a bit brighter).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Drop Firefox and NSS updates which have been stable for a while,
update the systemd one to the latest which should hopefully
finally workaround the DNS issues.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit e23df39ee1,
putting the systemd workaround back in place, now we know how to
avoid the bug it causes. It's going stable tomorrow anyway, but
I want to re-run failed tests with the fix right away. Used the
update ID this time, not the build number.
This will avoid us hitting a crash in systemd during update when
systemd is being updated. That's a real bug and it's good that
it's been caught, but we don't want unrelated updates to fail on
this just because systemd is in the update set.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It's causing crashes on update. I tried making us do an offline
update in repo_setup, not online, but that's actually quite hard
to implement so I'm not gonna do it on a Friday night. We'll
just live with unreliable _live_build for a bit longer.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
For consistency, let's just return to the desktop right away. We
also need to handle closing the overview before running installer
on live image boot.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When we right-click then left-click, we should move away from
the menu to avoid actually clicking anything in it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It's gone stable, but we haven't had a compose yet. Adding it as
a workaround so I can just revert the workaround for the bug
(see previous commit).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
desktop_browser fails frequently on F34 KDE due to #1927972. The
new version should avoid that problem, adding it as a workaround
to hopefully make the test more reliable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Add FEDORA-2021-263244c071 as a workaround to fix FreeIPA tests
on F34. Drop current 32 and 33 workarounds as they're all stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
In GNOME 40, the new-user mode of g-i-s is gone and we get the
welcome tour where we would previously have seen that. This
should handle that, I hope. I probably messed up somewhere.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These two are for a couple of FreeIPA bugs that showed up today
and were worked out with pemensik and mreynolds.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There's a race issue with just treating it as a next button: it's
not in the same place as a next button. Sometimes in the g-i-s
code we actually get ahead of ourselves and click early, which
isn't really a problem when the buttons are all in the same
place, but if we click "Start Setup" in the middle of transition
to the Privacy screen - as in
https://openqa.fedoraproject.org/tests/745034#step/_graphical_wait_login/4
- the click effectively gets lost. So let's make it its own tag
and have the initial assert look for it too. That way we won't
match on it again in the main loop over "@nexts".
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Using .local is apparently Bad Form because it's reserved for
mDNS. However there doesn't appear to be any particularly Good
Form for what to call a test domain you never want to exist
outside of a closed system, apparently. Sigh. Let's try this.
Includes a bump to disk_ks version because the kickstarts on
that image also need to have this change applied.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Otherwise we can immediately match 'fs is already selected'
for the *previously selected* fs before the UI updates and exit
without actually changing the fs of the intended partition.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This sets us up to test the release-blocking aarch64 disk images
(Minimal, Server and Workstation). It also allows for testing
armhfp disk images on aarch64 worker hosts (though my testing of
that isn't going too well so far), and fixes the initial-setup
handling for a change upstream ('use password' is now the default
so we don't need to choose it). We rewire disk image deployment
test loading to work through the generic loader code rather than
using ENTRYPOINT, as it allows us to more gracefully handle
graphical (Workstation) vs. console (Server, Minimal), moving
the code for handling console initial-setup to a helper function
just like the code for gnome-initial-setup and having _console_
wait_login call it when appropriate. We also tweak desktop_vt a
bit because now we need to switch from a console running as test
to a desktop, which breaks the assumption that the highest
numbered session of user test is the desktop...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
