1
0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-12-19 08:53:08 +00:00
Commit Graph

509 Commits

Author SHA1 Message Date
Adam Williamson
ac6ecb1058 Drop workarounds that have gone stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-03-17 21:26:16 -07:00
Adam Williamson
cf5532fc3a Add workarounds for ffmpeg/kf5/oneVPL mess
So...there's an ffmpeg update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-a5e10b188a
which went stable. It includes new sonames of all the ffmpeg
libs. It also pulls in a thing called oneVPL, which has a bug
that breaks ostree composes.

There's a big kf5 update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-b086a98f78
which contains kf5-kfilemetadata, which is built against ffmpeg.
Neal made sure that update's build of it was built against the
new ffmpeg and submitted both for stable at once - but the tests
on the kf5 update failed because they weren't run against the
new ffmpeg as it wasn't yet stable, and the kf5 update was
ejected from the push because of the failed tests.

So now we have the ffmpeg update stable but not the
kf5-kfilemetadata rebuild for it, which will break KDE stuff,
and the oneVPL issue means ostree composes will all fail.

This adds the ffmpeg update as a workaround so we can re-run the
tests for the kf5 update and get them to pass so we can push it
stable. It also adds the oneVPL update as a workaround so ostree
compose tests don't start failing.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-03-13 18:20:14 -07:00
Adam Williamson
a9a3cea174 Don't use koji-rawhide repo on support_server test
We definitely don't want the support server pulling in random
packages from Rawhide.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-25 10:22:50 -08:00
Adam Williamson
1d3fe8dbb6 Drop workaround that's now stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-24 12:57:37 -08:00
Adam Williamson
471ea9d27b workarounds: add plasma-discover with fix for #2173022
Just using a scratch build for now as my fix hasn't been reviewed
and may have dumb mistakes in it, but it does seem to fix the
openQA test at least.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-23 12:15:15 -08:00
Adam Williamson
f42d921b0b Drop workarounds that have gone stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-23 12:14:48 -08:00
Adam Williamson
bb3492ac9d Remove repo management packages on update live install path
A different way to address the same problem as 56936df7 . Let's
just *remove* the repo management packages after we're done
creating the repos. dnf will automatically remove the unused
dependencies too. This fixes the python-cryptography case at
least - I tested.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-22 15:57:02 -08:00
Adam Williamson
9a0ef37a25 Revert "Update live install tests: handle awkward install ordering"
This reverts commit 56936df7a5. It
was a lovely idea, but forgot that the 'matching update version'
check doesn't actually use the allpkgs.txt list...
2023-02-22 15:55:24 -08:00
Adam Williamson
56936df7a5 Update live install tests: handle awkward install ordering
There's this awkward path for the live image install tests on
updates. We run the 'are the correct versions of all the packages
installed' check on these tests to ensure the right versions
actually made it onto the live image. So we don't run
`dnf -y update` at the end of repo_setup_updates on that path,
because if we did that, even if the packages on the live image
were old, we'd update them there and hide the problem.

However, this causes a bit of an ordering issue, because in
order to set up the advisory repo, we need to install a few
packages. What if the update under test includes one of those
packages, or a dependency that wasn't already installed? In
that case, we wind up with the older stable version of the
package (because obviously we can't install the newer version
from the advisory repo *before we've set up the advisory repo*),
don't update it later, and so the 'correct version' check at
the end of the test fails. See:
https://openqa.fedoraproject.org/tests/1778707 for a case of
this happening with a python-cryptography update.

Up till now I was trying to handle this by just updating the
specific packages we install, but that doesn't account for
*dependencies* of them. I looked down the path of trying to
generate a list of all those dependencies and update all of
them but it looks a bit mad. So instead let's try this. On that
specific path, we'll generate the "all installed packages" list
*before* we run repo_setup, so it just doesn't include anything
that gets installed during repo_setup. The implementation is a
bit icky but not too horrible.

We *could* just *always* generate the all installed packages
list earlier, but then that would mean we *wouldn't* catch dep
issues in this kind of package on the other test paths, whereas
currently we do. I don't want to lose that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-22 14:54:18 -08:00
Adam Williamson
a4ff85695b handle_welcome_screen: only set _WELCOME_DONE if we saw it
This is to handle a temporary condition where the screen isn't
present on the KDE base disk images for F38 or F39 yet, so they
only see it on the second boot on update tests, but don't handle
it because we marked it as already 'done'.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-22 08:04:29 -08:00
Adam Williamson
4254906308 Try clicking Skip on the KDE welcome tour thing
It seems like just closing it results in it showing up again on
the next boot...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-21 19:57:19 -08:00
Adam Williamson
ddb3f44c57 Extend handle_welcome_screen to cover new KDE welcome tour
KDE has a welcome tour now, on F38 and Rawhide at least. Let's
"handle" it with extreme prejudice...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-21 18:16:13 -08:00
Adam Williamson
ac1a59ef77 workarounds: kwin build that fixes VT switch for F38 and F39
The F38 update that breaks this hasn't gone stable due to gating
and the F39 update will be pulled in once the tests are done
and it goes stable, but doing this anyway so I can re-run the
tests on the F38 plasma-workspace update and push it stable,
and rerun all the failed Rawhide tests without waiting for all
the tests on this update to finish first.
2023-02-20 22:11:31 -08:00
Adam Williamson
19500a4018 drop workarounds that have gone stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-20 22:10:40 -08:00
Adam Williamson
7ea4ed733c Sigh, do the are-we-44 check in the right place
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-20 17:29:13 -08:00
Adam Williamson
d0699217a4 Handle g-i-s 44 requiring two tabs at 'set a password' screen
We still need to handle 43 only requiring one for now, and we
can't just make it release-dependent until 44 is stable for both
38 and Rawhide, so let's use a needle match temporarily. Only
44 has these eye/pencil icons on this screen.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-20 17:13:37 -08:00
Adam Williamson
813c329c0c workarounds: add lorax with disabled persistence for 38/39 (#2170544)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-16 10:44:37 -08:00
Adam Williamson
35eec4062e Drop workarounds that went stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-16 10:43:22 -08:00
Adam Williamson
7156881b64 workarounds: add kde-settings for F38 too
This gets us the F38 background in KDE.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-14 11:27:18 -08:00
Adam Williamson
42316bdfb1 workarounds: add desktop-backgrounds too for F38
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-14 09:58:49 -08:00
Adam Williamson
8ce8f2aba4 f38 workarounds: drop updates now stable, add f38-backgrounds
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-14 09:17:49 -08:00
Adam Williamson
6baf67aefd Work around the f39 toolbox container not existing
On Rawhide, just use an f38 toolbox container for now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-13 19:16:09 -08:00
Adam Williamson
619f5a70fe Add glib2 2.75.3 to f38 workarounds (FEDORA-2023-965f517da3)
GNOME blows up without it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-13 15:09:14 -08:00
Adam Williamson
860c8e0a09 Add bug F38 GNOME update to workarounds
As we're not getting composes ATM this isn't being pulled into
tests of subsequent updates, but we need it to be or else there
are issues.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-13 12:23:08 -08:00
Adam Williamson
016c78d80b Add FEDORA-2023-ad52b2e4b9 as a workaround for F38
tracker-miners had a bad dep which broke live image builds.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-10 00:05:40 -08:00
Adam Williamson
95de85b33a Adjust a wait_still_screen not to time out on a flashing cursor
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-07 10:02:38 -08:00
Adam Williamson
8bcabe25a8 Give some long-running package install operations a bit longer
These have timed out quite often recently.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-03 09:04:48 -08:00
Adam Williamson
6a7c11466c Get updvercheck.py from main now it's merged
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-02-03 08:58:38 -08:00
Adam Williamson
06bfd2d2ae Make the update non-matching package check smarter
With Rawhide updates, we quite often run into a situation where
a test runs after a *later* version of the package has already
gone stable. This even happens for stable releases too, though
less often. The current shell-based check just always fails on
this case, but it's usually OK, and manually marking every case
like this with an "it's OK!" comment gets tiring. Instead, let's
use a smarter Python script to do the check. We compare the EVR
of all installed update packages with the EVR of the package
from the update. If it's the same, fine. If the installed package
is lower-versioned, that's always an error, and we fail. If the
installed package is higher-versioned, we check whether the
update already went stable. If it did, then we soft fail, because
probably nothing can go wrong at this point (this is the usual
Rawhide case). If the update did not yet go stable, we still
hard fail, because something can go wrong in this case: if the
update *now* goes stable, the older version from the update may
be tagged over the newer version the test got (presumably from
current stable).

If anything goes wrong with the Bodhi check, or the test is
running on a task not an advisory, we treat both cases as fatal.

The script also gives easier-to-understand output than the old
approach, which should be a bonus.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-01-31 10:14:45 -08:00
Adam Williamson
6a36fe28d1 Handle a false failure when testing koji updates
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-01-14 02:57:38 -08:00
Adam Williamson
02136a80e1 utils.pm: drop some stuff unneeded after F35 EOL
and associated needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-13 14:34:34 -08:00
Adam Williamson
6f0478d4b4 gnome_initial_setup: drop a <35 conditional that's not needed
F34 is EOL for a while.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-13 14:34:34 -08:00
Adam Williamson
6e9213f4b7 Drop use of _ADVISORY_REPO_DONE, always use file tests
This is safer if the advisory stuff was done on a previous test
run. Hilariously, this exposed a dumb mistake I made years ago
in installedtest.pm and never noticed before: the calls to
advisory_* at the bottom of that file are meant to be in the
post_fail_hook, but they weren't, which meant they got called
by the scheduler. This didn't cause any failures because the
first line caused them to return immediately based on a get_var
call (which it's OK to do in the scheduler), but changing it
to a script_run call (which it's *not* OK to do in scheduling)
caused all the tests to blow up immediately and confused me
*a lot* until I spotted this!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-12 08:53:02 -08:00
Adam Williamson
7429b438fd Don't update system in repo_setup on update install tests
When I enabled _advisory_post for live and ostree install tests,
the point was to check that updated packages were included in
the install media and used during installation. We shouldn't run
a system update in _repo_setup_updates on this path because it
will hide the problem if the updated packages weren't included.
The INSTALL variable is for this purpose - it was previously
used to skip _advisory_post on the same path. At the same time
let's remove some stray settings of this var on non-update tests
as it serves no purpose there.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-09 13:07:15 -08:00
Adam Williamson
5698525cc7 Enable the buildroot repository for Rawhide update tests
Per discussion at https://pagure.io/fedora-ci/general/issue/376
it really feels like this is the right thing to do. There are no
buildroot overrides for Rawhide, so we don't have to worry about
cross-pollution. The buildroot repo only contains builds that
have been tagged stable since the most recent Rawhide compose,
and thus will go into the next one. It makes sense to test later
updates against these. This avoids issues like:
https://openqa.fedoraproject.org/tests/overview?distri=fedora&version=38&build=Update-FEDORA-2022-30a952e331&groupid=2
where the tests of an update failed because it requires another
update which had been submitted and tagged stable previously, but
after the last compose.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-08 14:28:09 -08:00
Adam Williamson
ba88dac325 Handle chunked ADVISORY_NVRS_N settings
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-01 15:52:20 -08:00
Adam Williamson
03b6663339 Add tests to build a Silverblue installer image and install it
This is like the existing tests that build network install and
live images then install them, only for Silverblue. First we
build an ostree, using the standard configuration for the release
and subvariant but with the 'advisory' and 'workarounds' repos
included, so it will contain current stable packages plus the
packages from the update and any workarounds. Then we build an
ostree installer image with the ostree embedded, again including
advisory and workarounds repos in the installer build config so
packages from them will be included in the installer environment.
The image is uploaded, which completes the _ostree_build test.
Then an install_default_update_ostree test runs, which does a
standard install and boot from the installer image.

We do make a change that affects other tests, too. We now run
_advisory_post on live image install tests, as well as this new
ostree install image install test. It was skipped before because
of an exception that's really only needed for the netinst image
install test. In that test, packages from the update won't be
included in the installed system, so we can't run _advisory_post
on it. But for ostree and live image build/install tests, the
installed system *should* include packages from the update, so
we should check and make sure that it does.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-30 13:17:28 -08:00
Adam Williamson
c9191b0d4a disable_firefox_studies: don't run twice, skip uploads
There's no need to run this twice (which can happen on some
paths), so if the first file already exists, just bail. Also,
don't bother uploading the config files any more - that was just
for debug while I was making this stuff work, now it works, and
this saves some time.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-25 11:39:39 -08:00
Adam Williamson
c9924ecdeb Drop workaround which went stable two weeks ago
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-18 13:19:48 -08:00
Adam Williamson
946411cba4 Add layer-shell-qt workaround to make KDE work on Rawhide again
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-02 11:04:37 -07:00
Adam Williamson
c1d676e6ae Drop a workaround that went stable weeks ago
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-02 11:04:19 -07:00
Adam Williamson
b5c8a8d5f7 Fix tell_source
OK, that last commit was just, uh, wrong.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 12:06:57 -07:00
Adam Williamson
31826f7938 utils: simply and robustify tell_source
This is simpler if we just always lowercase $iso, plus it saves
us when somebody (*cough*) messes up the casing.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 11:57:33 -07:00
Adam Williamson
6b341097f2 Add some missing underscores to a variable check
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-22 14:10:13 -07:00
Adam Williamson
b84125d4f5 Replace scratch build override for mock with official update
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-18 09:21:54 -07:00
Adam Williamson
b6fdb4c3fa Add a mock build that works as root to Rawhide workarounds
mock 3.2 does not allow running as root. This breaks the live
image build tests. It's being reverted upstream, this scratch
build has the reversion backported.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-17 07:18:40 -07:00
Adam Williamson
9f1c136c37 Clear pushed workarounds for Rawhide
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-17 16:17:43 +02:00
Adam Williamson
1c819304d2 F38 workarounds: replace desktop-backgrounds with f37-backgrounds
desktop-backgrounds update went stable, but we also need an
f37-backgrounds update which was only just submitted to fix the
problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-08 23:13:33 +02:00
Adam Williamson
6b31f622d8 Add podman update as Rawhide workaround
podman tests are currently failing for all Rawhide updates. Not
sure what triggered this, but this update seems to fix it, though
it's failing its own test suite so not yet being pushed stable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-08 10:25:47 +02:00
Adam Williamson
fb774a7326 Rawhide workaround: desktop_backgrounds with symlink fix
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-07 18:55:14 +02:00
Adam Williamson
59c52453fe Fix an erroneous result of over-enthusiastic sed'ing
Whoops.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-09-27 23:46:55 -07:00
Adam Williamson
7675e99fcc Make internal state marker variables upper-case
We use variables to track test state across modules, sometimes.
As this is all internal to the test logic I didn't bother always
making these variables upper-case, but os-autoinst now treats
lower-case variables as a fatal error, so we have to change.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-09-27 23:18:42 -07:00
Adam Williamson
4b516b6b31 Drop Rawhide workarounds that are now stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-09-15 18:14:11 -07:00
Adam Williamson
2b79d59379 Disable updates-testing during upgrade_preinstall
This issue appeared when we started testing Rawhide updates, but
I only noticed it today. When testing Rawhide updates after
Branch point, the upgrade tests upgrade from Branched to Rawhide.
On Branched, updates-testing is enabled by default. We only
disable it when we reach `upgrade_run`, but by that point we've
already done a `dnf -y update` in `upgrade_preinstall` and
potentially installed other packages in steps between
`upgrade_preinstall` and `upgrade_run`. That can cause problems,
like today all FreeIPA upgrade tests on Rawhide are failing
because there's a newer freeipa in updates-testing for F37 than
is in the current Rawhide compose.

Solve this by disabling updates-testing before we do the update
in `upgrade_preinstall`. To avoid excessive code duplication,
factor out the repo disabling code.

We'll do this twice on upgrade tests now, but it shouldn't be a
problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-09-15 10:18:26 -07:00
Adam Williamson
0a411bd8e1 Add python-jwcrypto as workaround for F38
FreeIPA replica tests broke in today's compose, this solves it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-09-14 15:32:55 -07:00
Adam Williamson
aa73342380 Remove FEDORA-2022-2beaa02bb4 workaround
It's stable now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-09-04 08:38:33 -07:00
Adam Williamson
f81f3fea14 Add FEDORA-2022-2beaa02bb4 as Rawhide workaround (#2121952)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-27 12:36:15 -07:00
Adam Williamson
7bbec4c6bb Drop a workaround for 38 that went stable 3 days ago
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-27 12:29:58 -07:00
Adam Williamson
6d5c0c723e Add latest FreeIPA as a workaround for Rawhide
samba-client-libs with an soname bump made today's Rawhide, a
freeipa rebuilt against it did not, so we need this in overrides
or all FreeIPA tests fail on all Rawhide updates.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-24 12:09:32 -07:00
Adam Williamson
2c7d6b53db Drop a no-longer-needed workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-24 12:08:49 -07:00
Adam Williamson
87a0edf0fc Use rpm-ostree usroverlay for Firefox config on rpm-ostree
Editing the files under /ostree/deploy doesn't work on Rawhide
any more because it's been made read-only. Thanks to Timothée
Ravier for pointing out this way of doing it, which is better
anyway and avoids awful quotation issues in the code.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-19 00:20:36 -04:00
Adam Williamson
e9f5378df5 Remove some F37 workarounds that are in current compose
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-13 10:13:51 -07:00
Adam Williamson
51eed16128 Add g-i-s with fixed page skipping as f37 workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-13 00:20:39 -07:00
Adam Williamson
71a7b8f60d Drop several F37 workarounds that went into Branched compose
This should ease space pressure on some tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-12 01:14:28 -07:00
Adam Williamson
fc11917a01 Add f37-backgrounds and desktop-backgrounds as workarounds
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-11 10:26:30 -07:00
Adam Williamson
2807410dd8 Add interdependent samba, sssd and freeipa updates as workarounds
A samba update was pushed for F37 without rebuilds of sssd and
freeipa (which needed to be rebuilt against it). Tests of any
of those updates on its own will fail, so including them all as
workarounds so the tests for each should work. I'll make sure
they all get pushed stable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-09 12:12:56 -07:00
Adam Williamson
8ede32a76c Give workaround download longer
the webkitgtk workaround is large. 3 minutes isn't long enough
apparently.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-08 15:34:30 -07:00
Adam Williamson
8fc2f71df9 Add new webkitgtk build as workaround for file conflict issue
The file conflict between older webkit2gtk4.1-jsc and new
javascriptcoregtk4.1 is causing all Workstation update tests to
fail on all Rawhide updates. This new version of webkitgtk adds
obsoletes to javascriptcoregtk4.1 which we hope should fix it.
Adding the x86_64 build directly as other arches aren't done
yet, and we only run the Workstation update tests on x86_64
anyway.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-08 12:05:09 -07:00
Adam Williamson
45c25b2e6c Drop workaround that's now stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-08 12:04:37 -07:00
Adam Williamson
f5946e678c Make update testing more robust for very large updates
openQA choked badly on
https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71
because it's, well, huge - 87 builds including texlive, which
has hundreds (thousands?) of subpackages. This exposed several
frailties against such updates.

First of all, we set NUMDISKS to at least 2 for *all* update
tests, which should mean they all stash the RPMs from the update
on a non-system disk and avoid problems with space exhaustion.
After that, just extend a few timeouts in particularly fragile
places, including one which is specific to texlive (as I don't
know of any other source package with so many subpackages).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 11:29:21 -07:00
Adam Williamson
e195f9d79a Add dogtag-pki that works with new python-ldap as workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-29 09:47:05 -07:00
Adam Williamson
dbe7f567d8 Drop FreeIPA workaround for Rawhide (it went stable)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-29 09:46:38 -07:00
Adam Williamson
1a65993d36 Add a perltidy check and apply it to the entire codebase
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-28 14:38:38 -07:00
Adam Williamson
ad5eb54715 Add FreeIPA update as workaround for F37
We need this FreeIPA rebuild for Rawhide update tests to pass.
Can drop once it's in a compose.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-26 14:39:33 -07:00
Adam Williamson
73a07f38aa Drop now-stable FreeIPA workaround for F36
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-26 14:38:27 -07:00
Adam Williamson
e8543a1764 Add freeipa-4.9.10-2.fc36 as a workaround for F36
F35 to F36 FreeIPA server upgrade test is failing because the
latest F36 package is lower-versioned than the latest F35
package, and FreeIPA's upgrade scripts are written to fail if
this is the case. The update bumps the F36 NVR to be higher
than the F35 one.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-18 11:12:57 -07:00
Adam Williamson
f80e42fd32 Drop mesa workaround for F36, it's stable now
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-18 11:12:40 -07:00
Adam Williamson
cf352a4e3e setup_workaround_repo: explicitly install koji
bodhi-client should depend on it, but technically since we have
code that calls `koji` directly here, we should probably also
include it in our install anyway, so not marking this as a
workaround.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-14 11:46:12 -07:00
Adam Williamson
4d9831efcc Add mesa update that fixes kwin crashes as workaround for F36
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-14 11:14:43 -07:00
Lukáš Růžička
1f8b56e068 Change the way to download test data.
This PR changes the way to download the test data into the VM.
Although it does not use a disk image as suggested in one
of the review, it does not clone the entire repository, but
a simple tar.gz file that holds the data which will be
distributed into the directory structure.

This way, the amount of data needed to be downloaded dropped
from approximately 50MB to below 2MB.

Also, the existing test suites were adapted to this situation.
2022-07-04 09:34:04 +02:00
Adam Williamson
a6eb9024cf do_bootloader: bootloader is always grub on F37+
Now we've ditched syslinux in Rawhide, we should just always
expect to see grub if release number is > 36.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-17 16:58:34 -07:00
Lukáš Růžička
7637238951 Add some extra steps for KDE to overcome BZ2097208. 2022-06-15 10:35:57 +02:00
Adam Williamson
4971de39e9 Tweak quit_firefox to handle hang on exit on ppc64le (#2094137)
It seems when we quit Firefox back to a VT on ppc64le, the system
hangs. Not sure why, but we can deal with it by rebooting the
system and logging back in as root if it happens. Also take the
opportunity to clean up the flow of quit_firefox so we always
check that we get back to a console then wait 5 seconds for the
console to settle, so all the tests that call it can stop doing
that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-06 18:35:52 -07:00
Adam Williamson
4ee583ad43 Bump a console login timeout a little
After typing the username, on some Rawhide tests, it's taking
over 30 seconds for the password prompt to appear. This isn't
ideal but we don't want the test to fail on this. Give it up
to 45 seconds.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-01 09:04:34 -07:00
Adam Williamson
21e4ded350 Fix disable_firefox_studies for ostree installs
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-26 09:25:07 -07:00
Adam Williamson
919ade3152 Tweak the Firefox config override stuff again based on research
Wow, so the real way to do config overrides is, uh...obscure.
One file pointing to another file, both with mandatory comment
lines and one with a weird required value. Wat. Anyway, this
works in a VM. I still don't know why the policy for the first
run page isn't working as advertised.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-24 23:49:31 -07:00
Adam Williamson
d236ea2cfb Fiddle with firefox welcome screen disabling again
The new way isn't working, so put the old way back but change it
to use user_pref instead of pref to see if that helps, and upload
the files for checking.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-24 23:20:23 -07:00
Adam Williamson
3f74203003 Fix missing quote in Firefox policy JSON
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-24 16:54:00 -07:00
Adam Williamson
1e8a82f0e1 Try different way to disable Firefox 'welcome pages' (mkaply)
Thanks to Mike Kaply in
https://bugzilla.mozilla.org/show_bug.cgi?id=1703903#c18 , this
should avoid Quick Suggest onboarding, "Total Cookie Protection"
onboarding, and future annoying things using the same mechanisms
without using the prefs mechanism or going after each one case
by case.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-24 08:28:31 -07:00
Adam Williamson
f3e4750f6e Fix line wrapping in firefox config override
Missed a line break...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-20 23:41:26 -07:00
Adam Williamson
7ee3a91ffc Try and disable yet another Firefox popup that breaks tests
See https://bugzilla.mozilla.org/show_bug.cgi?id=1703903#c9 and
comment #13 for this one.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-19 14:54:54 -07:00
Adam Williamson
01c2962f41 Note when #1933118 workaround can be removed
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-05-13 17:07:14 -07:00
Adam Williamson
94520bb7a8 Drop similarity_level on type_safely
I meant to include this in the earlier commit that does the same
for type_very_safely, but forgot.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-03-16 11:14:50 -07:00
Adam Williamson
7bf3bd8689 Drop similarity levels on type(_very)_safely, solidify_wallpaper
I noticed today that we're timing out all the time on these
wait_still_screens in gnome-terminal, because it uses a big
flashing cursor and the similarity between "cursor there" and
"cursor not there" is less than 45 or 42 (it seems to be 38.x).
So let's drop these levels to 38, hopefully that's not too low.
There are probably more places where this is an issue, I'll
change them as I notice them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-03-14 16:32:29 -07:00
Adam Williamson
ad1d656124 Drop F36 kwin workaround, should not be needed any more
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-03-01 18:31:37 -08:00
Adam Williamson
8fab101745 Use lower similarity_level for some terminal wait_still_screens
We can't use the default similarity_level for wait_still_screen
when there's a flashing cursor - flashing cursor will always
cause the similarity level to be too low and the wait will just
time out at 30 seconds. Cut it to 42.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-03-01 10:31:06 -08:00
Adam Williamson
abd1d1a698 check_desktop: handle GNOME showing inactive button briefly
check_desktop tries to catch when the overview is open and close
it. But with GNOME 42, it seems the inactive "Activities" button
is shown briefly on login before GNOME opens the overview. If
check_desktop catches that, it will think the overview isn't
open and it doesn't need to do anything. So if we match on first
cycle through the loop, let's wait_still_screen then match again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-03-01 10:24:48 -08:00
Adam Williamson
c5e6d80684 Fix escaping in the caret workaround
So these kind of things go through perl *and* bash string
interpretation and the escaping can get pretty wacky. Turns out
we need *eight* slashes here to get four through to bash (which
we need to deal with *sed*'s escaping rules), and it only works
in single quotes for some reason, not double quotes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-25 14:05:10 -08:00
Adam Williamson
371cb1c6eb Work around RPM caret bug when checking advisory packages
There's a bug in rpm where doing `rpm -q [NVRA]` doesn't work
if the NVRA contains a caret. To make it work you have to add
a literal slash character before the caret character, so we add
a sed command to do that, when we're checking whether packages
from the update actually got installed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-25 13:07:46 -08:00
Adam Williamson
1417f1023e Add kwin scratch build to fix RHBZ #2047503 as an F36 workaround
Scratch build because we don't have a single-build update with
the fix, it's likely gonna be included in the 5.24.2 megaupdate.
I don't want to use that whole thing as a workaround, so I did
a 5.24.1 scratch build with the fix instead.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-23 13:09:22 -08:00
Adam Williamson
815fc3a758 Drop workarounds that already went stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-23 13:08:13 -08:00
Adam Williamson
c33b137e88 Tweak desktop_vt tty detector a bit
The intent is that if the ps check finds nothing we'll use the
loginctl output, but that doesn't work because script_output
doesn't return the output if the script fails. There's an arg
you can pass to make it do so, but let's just make it always
succeed instead, by adding a ||: to the second grep like we have
for the first.

Also, I noticed this problem because the ps check started not
working on F36 KDE because none of the processes we check for
are shown as running on a tty, so let's add one more that *is*
shown as running on a tty...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-22 16:54:03 -08:00
Adam Williamson
628c4a5d8f Try and disable Firefox "quick suggest onboarding" thing
Tests that use Firefox started failing recently because Firefox
grew yet another stupid pop-up thing that *might* show on start
up, this appears to be trying to get you to sign up for a
feature called "quick suggest". After half an hour trawling the
relevant code, this is my best guess as to how to turn it off.
Don't know for sure if it works because the thing doesn't pop
up every time, but it at least doesn't make things worse.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-07 12:06:23 -08:00
Adam Williamson
afbb62d514 Add F34 qt5-qtwayland update to workarounds to fix crashes
KDE tests are failing on all F34 updates ATM due to
https://bugzilla.redhat.com/show_bug.cgi?id=2049560 , add the
update to fix it. Also drop 33 from the workarounds hash as it's
EOL.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-02-03 16:31:34 -08:00
Adam Williamson
ec83223ed1 Replace 389-ds-base workaround scratch build with official update
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-01-27 14:07:05 -08:00
Lukáš Růžička
fe8306d2cd Implement distribution methods to replace the testapi defaults.
This PR adds implementations of distribution methods to make them
usable on Fedora. It adds the following methods

* ensure_installed (to install packages)
* become_root (to switch to root account)
* script_sudo (run script with sudo)
* assert_script_sudo (run and assert a sudo script)

It also adds a helper script to the utils.pm

* make_serial_writable

that makes the serial console writable for normal users
and so enables to run commands that check their progress
by sending messages to the serial console. Normally, they
fail, because the messages will not be written their, so
the checking mechanism will never see them.
2022-01-27 21:44:35 +00:00
Adam Williamson
9c225c5b40 Add 389-ds-base scratch build to fix RHBZ#2047323 to workarounds
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-01-27 12:49:30 -08:00
Adam Williamson
a1e20088b6 Drop now-stable update from F35 workarounds list
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-01-27 12:48:54 -08:00
Adam Williamson
5296ff8114 Add selinux-policy-35.11-1.fc35 as workaround for bad .10 update
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-01-19 09:48:28 -08:00
Adam Williamson
8e414cc469 Handle anaconda giving users admin by default in Rawhide
As of today's Rawhide compose, user accounts added in Rawhide
have admin privs by default. For now we need to handle both
possibilities here (click the box if it's not clicked already);
after F35 EOL we can just drop all handling of that box.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-01-18 11:36:06 -08:00
Adam Williamson
3a9dfb6a58 Drop remaining workaround update (it went stable two months ago)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-11-11 08:52:45 -08:00
Adam Williamson
ff8603b96e Give KDE Desktop Settings window a few seconds to stabilize
Rawhide had a failure today where the dropdowns moved between
us matching and clicking, so we clicked the wrong one:
https://openqa.fedoraproject.org/tests/1005662

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-29 09:52:59 -07:00
Adam Williamson
9f15350a99 Allow g-i-s longer to start up (with a soft fail) (#1997310)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-16 15:18:47 -07:00
Adam Williamson
9882ddcaa5 Robustify and tweak switch out of overview on GNOME boot
This fails sometimes just because we're too early, or something.
Also with GNOME Shell 41rc, alt-f1 no longer works to open and
close the overview. super *does* seem to work in KDE these days,
so let's switch from alt-f1 to super everywhere.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-07 11:05:55 -07:00
Adam Williamson
81c0a861d6 Add FEDORA-2021-9677463ae9 as a workaround for F35 update issues
This will hopefully resolve desktop_update_graphical failures on
Workstation.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-01 14:49:37 -07:00
Adam Williamson
bc7deabd24 Drop workaround update that's stable now
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-27 17:19:13 -07:00
Adam Williamson
f56b6c6c2a Add FEDORA-2021-41d8b36cd2 as workaround for RHBZ #1996901
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-24 15:53:21 -07:00
Adam Williamson
0b5746248f Drop some update workarounds that went stable already
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-24 15:52:27 -07:00
Adam Williamson
453053f4ed gnome-initial-setup: work around #1997310 (GOA screen missing)
There's a bug in current F35/Rawhide which causes the GOA screen
in g-i-s not to show up. Since failing on that will block a lot
of testing, let's handle it as a soft failure.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-24 15:14:22 -07:00
Adam Williamson
5661d3c1a7 Handle Software screen making a comeback in gnome-initial-setup
It's back from the dead! In F35+, anyway.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-23 12:45:35 -07:00
Adam Williamson
4c19ed1723 Add FEDORA-2021-0192369fd9 as a workaround for F35 update tests
This should fix FreeIPA server deployment failure.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-20 11:41:07 -07:00
Lukáš Růžička
b65f470eee Create basic functions tests for Gedit. 2021-08-12 15:37:37 +02:00
Adam Williamson
492fcf62e4 Great Needle Cleanup 2021
Remove a whole chunk of needles that haven't matched for more
than 3 months. Also move a few needles to appropriate locations,
simplify some code chunks that relied on removed needles (if
we're not matching the needles, we don't need those chunks any
more), and drop some other no-longer-needed conditionals for
older releases.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-10 11:57:48 -07:00
Adam Williamson
a9f54e9a50 Try to avoid a race in GNOME check_desktop
We can match on apps_menu_button_active before the overview has
totally loaded in, and it seems the alt-f1 press can be dropped
in that case. Add a wait_still_screen to try and avoid that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-05 11:22:25 -07:00
Adam Williamson
05c793179a Add a fingerprint login test (GNOME only) (#223)
This adds a test for fingerprint-based login, as requested by
@benzea in #223. We use the fprintd dummy device to let us
simulate scanning a fingerprint, and check various scenarios
recommended by @benzea.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-08-03 15:21:33 -07:00
Adam Williamson
20c1f20880 F34 update workarounds: drop systemd, add jpegxl
This systemd update went stable ages ago. But now we need to add
a jpegxl update as a workaround to avoid KDE live build tests
failing on the problematic aom->jpegxl-libs->gimp dependency
chain I identified yesterday. It makes KDE live builds pull in
too many packages and fail because they run out of space.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-16 09:20:34 -07:00
Adam Williamson
c9609cf26d Drop bypass_1691487 function and usage (bug seems fixed)
Per https://bugzilla.redhat.com/show_bug.cgi?id=1691487#c14 it
seems like the bug here got fixed along the way and we probably
don't need this any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-05-06 09:02:52 -07:00
Adam Williamson
05dfbef1a8 Update Activities button handling for latest GNOME
Make the 'deactivate overview if it's active' thing a bit more
robust by asserting the inactive state after deactivating it,
and add new needles for the new RC (text got a bit brighter).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-16 17:24:08 -07:00
Adam Williamson
c44e7fb869 Update F34 workarounds: drop stable ones, update systemd
Drop Firefox and NSS updates which have been stable for a while,
update the systemd one to the latest which should hopefully
finally workaround the DNS issues.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-12 18:34:08 -08:00
Adam Williamson
12de9331c5 Revert "Ditch the systemd workaround build for now"
This reverts commit e23df39ee1,
putting the systemd workaround back in place, now we know how to
avoid the bug it causes. It's going stable tomorrow anyway, but
I want to re-run failed tests with the fix right away. Used the
update ID this time, not the build number.
2021-03-08 14:24:24 -08:00
Adam Williamson
21385361ed F34+: disable systemd-oomd during update install (#1931034)
This will avoid us hitting a crash in systemd during update when
systemd is being updated. That's a real bug and it's good that
it's been caught, but we don't want unrelated updates to fail on
this just because systemd is in the update set.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-08 14:01:46 -08:00
Adam Williamson
e23df39ee1 Ditch the systemd workaround build for now
It's causing crashes on update. I tried making us do an offline
update in repo_setup, not online, but that's actually quite hard
to implement so I'm not gonna do it on a Friday night. We'll
just live with unreliable _live_build for a bit longer.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-05 18:22:13 -08:00
Adam Williamson
52a862aae9 Add scratch build for #1933433 as a workaround
I'm hoping this should fix the intermittent _live_build failures.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-05 13:24:30 -08:00
Adam Williamson
540f667c1d Drop F34 workaround updates that have gone stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-05 13:23:59 -08:00
Adam Williamson
1e5d767fa3 Handle GNOME 40 now starting at overview, not desktop
For consistency, let's just return to the desktop right away. We
also need to handle closing the overview before running installer
on live image boot.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-04 16:03:21 -08:00
Adam Williamson
42d41a9419 Correct last commit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-03 23:43:53 -08:00
Adam Williamson
c91e682d09 solidify_wallpaper: try to avoid clicking anything in the menu
When we right-click then left-click, we should move away from
the menu to avoid actually clicking anything in it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-03 20:03:59 -08:00
Adam Williamson
25214b7ff7 Add the kde-settings update that fixes XDG dirs as a workaround
It's gone stable, but we haven't had a compose yet. Adding it as
a workaround so I can just revert the workaround for the bug
(see previous commit).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-03 15:11:02 -08:00
Adam Williamson
b71693e4d1 Add Firefox 86 to F34 workarounds (#1927972)
desktop_browser fails frequently on F34 KDE due to #1927972. The
new version should avoid that problem, adding it as a workaround
to hopefully make the test more reliable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-02 17:42:18 -08:00
Adam Williamson
6b4b452521 Drop 32 and 33 workarounds, add 34 FreeIPA fix as workaround
Add FEDORA-2021-263244c071 as a workaround to fix FreeIPA tests
on F34. Drop current 32 and 33 workarounds as they're all stable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-03-01 17:03:50 -08:00
Adam Williamson
7c6188c087 Workaround KDE bug accessing desktop settings (#1933118)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-02-25 10:53:33 -08:00
Adam Williamson
09a0f19e36 Try and split 'g-i-s done' and 'welcome tour done'
They're not always the same, and it's breaking things on aarch64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-02-19 17:58:58 -08:00
Adam Williamson
81d6db802f ...yup, I durn goofed. Share properly, kids!
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-02-18 09:56:34 -08:00
Adam Williamson
4489a6cb3f Rejig GNOME welcome screen/g-i-s handling for GNOME 40
In GNOME 40, the new-user mode of g-i-s is gone and we get the
welcome tour where we would previously have seen that. This
should handle that, I hope. I probably messed up somewhere.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-02-18 09:54:06 -08:00
Adam Williamson
436e9cfdf2 Update GNOME 'getting started' needle and dismissal key
The new one isn't an app, so alt-f4 doesn't work. Esc does.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-02-18 09:38:52 -08:00
Adam Williamson
1308d104d4 Update F33 workarounds: drop one that went stable, add two
These two are for a couple of FreeIPA bugs that showed up today
and were worked out with pemensik and mreynolds.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-01-25 23:48:48 -08:00
Adam Williamson
15503b76b9 Add fwupd updates as workarounds to fix GNOME Software crash
This periodically causes the desktop_update_graphical test to
fail.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-01-11 10:53:05 -08:00
Adam Williamson
0e534a3150 Allow tasks as well as advisories as workarounds
This will make my life a bit easier in the next hour or two.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-01-05 09:38:29 -08:00
Adam Williamson
b9cbd6f28f Complete separation of g-i-s next_button and start_setup
Previous attempt was incomplete, didn't actually handle the first
click.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-12-21 12:51:50 -08:00
Adam Williamson
fba4a82587 Make g-i-s Start Setup button needle its own thing
There's a race issue with just treating it as a next button: it's
not in the same place as a next button. Sometimes in the g-i-s
code we actually get ahead of ourselves and click early, which
isn't really a problem when the buttons are all in the same
place, but if we click "Start Setup" in the middle of transition
to the Privacy screen - as in
https://openqa.fedoraproject.org/tests/745034#step/_graphical_wait_login/4
 - the click effectively gets lost. So let's make it its own tag
and have the initial assert look for it too. That way we won't
match on it again in the main loop over "@nexts".

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-12-21 09:16:40 -08:00
Michel Normand
05cf6f03bc Increase from 120 to 180s delay for rescue_select
required at least for install_repository_hd_variation
test that failed systematically for ppc64le since a while
https://openqa.stg.fedoraproject.org/tests/980357#step/preinstall_iso_in_hd/17
https://openqa.stg.fedoraproject.org/tests/980357#next_previous

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2020-12-16 04:18:51 +01:00
Adam Williamson
c7a1b94c84 Enable aarch64 disk image testing, related fixes
This sets us up to test the release-blocking aarch64 disk images
(Minimal, Server and Workstation). It also allows for testing
armhfp disk images on aarch64 worker hosts (though my testing of
that isn't going too well so far), and fixes the initial-setup
handling for a change upstream ('use password' is now the default
so we don't need to choose it). We rewire disk image deployment
test loading to work through the generic loader code rather than
using ENTRYPOINT, as it allows us to more gracefully handle
graphical (Workstation) vs. console (Server, Minimal), moving
the code for handling console initial-setup to a helper function
just like the code for gnome-initial-setup and having _console_
wait_login call it when appropriate. We also tweak desktop_vt a
bit because now we need to switch from a console running as test
to a desktop, which breaks the assumption that the highest
numbered session of user test is the desktop...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 12:39:49 -07:00
Adam Williamson
e5f396a785 Drop workaround for fixed GNOME submenu bug
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/2878 was
fixed (or just rendered obsolete by a change to how submenus
work), so we don't need this any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-15 15:36:09 -07:00
Adam Williamson
1c33d07d38 Drop workaround_ble26, bug was fixed months ago
https://pagure.io/background-logo-extension/issue/26 was fixed
months back, we don't need this any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-15 15:36:04 -07:00
Adam Williamson
db99e5462c Drop F33 FreeIPA update from workarounds, it went stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-10 17:27:33 -07:00
Adam Williamson
9de75af60b Drop f33-backgrounds from workarounds (it went stable)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-09 16:57:21 -07:00
Adam Williamson
401ebecc10 Add F33 FreeIPA 4.8.10-5 update as workaround to fix upgrades
FreeIPA upgrade test is failing because of
https://bugzilla.redhat.com/show_bug.cgi?id=1886205. The test
failing every time is not useful as we know what the issue is,
so add the update as a workaround to avoid it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-08 11:28:10 -07:00
Adam Williamson
4fcea15610 Updates testing: add f33-backgrounds workaround
Add FEDORA-2020-27f80050a2 as a workaround because without it the
KDE desktop background test fails due to the bug in -6.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 12:40:27 -07:00
Adam Williamson
8d9684dd80 Update tests: don't install git any more
We were using it to checkout a git version of python-fedora to
work around a bug, long ago, but we don't do that any more so
we don't need it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 10:19:33 -07:00
Adam Williamson
aafe460f85 Update testing: drop all workarounds
They're all stable now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 10:09:59 -07:00
Adam Williamson
1fdf7cd567 Handle update build not having any packages for target arch
openQA sometimes winds up testing an update that doesn't have
any packages for x86_64 (or aarch64). The most common case is
s390utils, which is on the critpath but only has packages for
s390x. I would ideally like to skip scheduling entirely if the
update has no packages for the arch we're scheduling on, but
sadly that involves using the Koji API which is XML-RPC and I
don't really want to deal with that again. This deals with it
at the test level instead, by checking the error message if
`koji download-build` fails and carrying on if it's the "no
packages for this arch" error. That means if the update has no
packages at all for our arch we're not really testing anything,
but that's better than a bunch of false failures, I guess.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 09:49:03 -07:00
Adam Williamson
fb1bc1dad8 Add mock 2.6 to workarounds, drop kernel for F31 and F32
Kernel updates for F31 and F32 went stable so they can come out.
mock 2.6 fixes the bug that occurs when /etc/resolv.conf is a
dangling symlink - this breaks the live_build test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-17 16:40:53 -07:00
Adam Williamson
ee666d5f13 Add kernel 5.8.9 updates to workarounds (fix parted problems)
Kernel 5.8.8 broke the installer by changing return codes for
partition operations, these updates are listed as fixing it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-14 18:48:08 -07:00
Adam Williamson
d86960166d drop FEDORA-2020-2bef864cab as a workaround (it went stable)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-14 18:47:23 -07:00
Adam Williamson
0350977ba3 Update workarounds: drop stable one, add F33 FreeIPA upgrade one
The F32 FreeIPA update that changed the web UI has gone stable
now, so remove it. A pki-core update has just come out that fixes
F32 -> F33 FreeIPA server upgrade test: add this as a workaround
for F33 so that test stops failing on every update.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-08 14:13:29 -07:00
Adam Williamson
0999921fe3 Tweak detection of installed update packages
Previously we were relying on `rpm -q` always outputting the
right package last. We saw some test failures on recent kernel
updates, e.g. https://openqa.fedoraproject.org/tests/658768 ,
which indicate this isn't always the case; there the 'right'
package was second of three for kernel, third of three for
kernel-core and first of three for kernel-modules. So we need to
make it more robust. This uses an additional call:
`rpm -q $pkg --last | head -1` to find the most recent package,
if there are more than one; this should always be the right one,
I hope. Note we cannot just add `--last` to the `rpm -q --qf...`
call because the output when you do that is weird; you get the
output you'd get if you just called `rpm -q --last` first, and
*then* the query-formatted output afterwards (though with the
modified order as expected). There doesn't seem to be any way to
get only the latter.

I also tweaked the log uploading so we always upload the working
logs even when the test passes; it can't hurt anything and it is
sometimes useful to have them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-08 14:03:23 -07:00
Adam Williamson
478b7eff9e Add initial CoreOS product and test templates
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-02 14:49:29 -07:00
Adam Williamson
b9f6ecd72d Conditionalize FreeIPA UI change, add 4.8.9 update to workarounds
The FreeIPA UI change that the previous commit adapted to is in
4.8.9. That's stable for Rawhide and F33 already, but still in
testing for F32, and won't go to F31. So we need to make the
change conditional on release number, and we also add the update
to workarounds for F32 so we don't have to do something awkward
while we wait for it to go stable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-21 14:00:42 -07:00
Adam Williamson
2713ce60dd Add a new not-really-a-Next-button needle for gnome-initial-setup
In g-i-s 3.37.91, the first screen has a 'Start Setup' button
rather than a 'Next' button. Easiest thing for us to do here is
just to add a new needle which has the 'next_button' tag even
though it's clearly not a 'Next' button, because then the code
still works :) So do that, but give the file a suggestive name
and explain the situation in a code comment.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-18 12:23:09 -07:00
Adam Williamson
8ee330b5d3 Add krb5 reversions for #1868482 to update workarounds
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-13 15:11:35 -07:00
Adam Williamson
0557a774ac Retry bodhi updates download a few times if it fails
It's failing about one in six tries currently, with Bodhi 5.5 on
the server end: https://github.com/fedora-infra/bodhi/issues/4105
this should work around that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-13 15:03:36 -07:00
Adam Williamson
41ddac0951 Drop all update workarounds
The ones that were in there are stable now, plus downloading them
is hitting a bug in Bodhi and breaking tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-13 07:55:05 -07:00
Adam Williamson
aa41fe4e4e Automate QA:Testcase_Zezere_Ignition
This is a bit complex to automate, because we cannot really use
the production Zezere server (provision.fedoraproject.org) as
the test case shows, as we'd have to solve authentication and
we also don't really want to constantly keep registering new
hosts to it that are going to disappear and never be seen again.

So, instead we'll do it by setting up our *own* Zezere, and
provisioning our IoT system in that. We run two tests. The
'ignition' test is the actual IoT 'device'; all it really does
is boot up, sit around, and wait to be provisioned. The 'server'
test first sets up a Zezere server, then logs into it, adds an
ssh key, claims the IoT device, provisions it, and connects to
it to create a special file which tells the 'ignition' test
everything worked and it can close out.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-23 18:01:06 -07:00
Adam Williamson
15eae21722 start_with_launcher: adjust for new overview behaviour
The annoying submenus in the overview app list now scroll right
not down :/ have to adapt this function for that. Had to move
get_release_number earlier because perl ordering.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-09 15:54:33 -07:00
Adam Williamson
a3943dc307 Sigh, fix missing brace
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-09 14:31:44 -07:00
Adam Williamson
4a6cd8bcd5 Abstract out overview type-to-search bug workaround
And also use it in GNOME apps settings.pm test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-09 14:25:22 -07:00
Adam Williamson
6232be32cf Sigh, move menu_launch_type after get_release_number
Stupid perl.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-08 17:09:45 -07:00
Adam Williamson
2f93645a55 Workaround GNOME overview search box not active bug
See https://pagure.io/background-logo-extension/issue/26 - in
current Rawhide, the search box in the overview is not active
when the overview is opened, so you can't just open the
overview and type, you have to click it first.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-08 17:07:48 -07:00
Adam Williamson
7ee071ca08 Add g-i-s timezone bug fix updates as workarounds
This should avoid the bug happening in upgrade tests (I already
built the fix into the base disk image, which should avoid it
happening in other tests).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-24 09:31:40 -07:00
Adam Williamson
f96c14adc0 gnome-initial-setup: add one more check for that auth_required
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-17 08:17:49 -07:00
Adam Williamson
fd26d347fe Workaround unexpected authentication required dialog in g-i-s
https://gitlab.gnome.org/GNOME/gnome-initial-setup/-/issues/106
This started happening after the data center move. Really not
sure why.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-12 12:28:18 -07:00
Adam Williamson
1b37200446 Work around GNOME Shell app submenu bug
See https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/2878 .
GNOME 3.37.2 seems to have a bug with submenus in the app menu;
the first time you open one you can't scroll through it using
the keyboard. On every open after the first it works fine. This
is a quick and dirty workaround - when we're dealing with a
submenu, open it then close it then open it again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-11 16:00:56 -07:00
Adam Williamson
b59f875af1 Drop F32 workaround updates (both now stable)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-27 13:15:37 -07:00
Adam Williamson
acc46464f8 Revise release variable handling, prerelease checks, os-release
I started out trying to fix os-release for the recent change to
add "Prerelease" tags to the VERSION and PRETTY_NAME fields, then
things spiralled. It got me thinking about the awkward DEVELOPMENT
variable we use, so I decided to get rid of it and refactor the
few things that use it. I refactored the anaconda prerelease tag
check, and wrote a new giant comment that gives details about
exactly how anaconda decides whether to show those tags, to give
context to our choices about when to expect them. This check now
uses a new LABEL variable the scheduler now sets. I also wound up
creating new UP1REL and UP2REL vars to define the 'source' release
for upgrade tests, separate from CURRREL and PREVREL, which are
now never lies - they really are the current stable and previous
stable release, even for update upgrade tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-07 15:42:08 -07:00
Adam Williamson
8eaddd8c3d Set OfferToSaveLogins policy to false in Firefox config
This seems to work around the Firefox 76 bug with password entry
sometimes breaking:
https://bugzilla.mozilla.org/show_bug.cgi?id=1635833

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-06 13:25:46 -07:00
Adam Williamson
d66b185471 solidify_wallpaper: restore a wait_still_screen in GNOME path
The old code waited after launching the terminal, the new code
doesn't, which led to a 'g' being swallowed in the first command
in https://openqa.fedoraproject.org/tests/592759 .

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-05 10:19:31 -07:00
Adam Williamson
8866b5289e Fix solidify_wallpaper for KDE
The desktop check used the wrong case.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-05 10:08:47 -07:00
Lukáš Růžička
61f845dbbf Make solidify_wallpaper more universal to handle different desktops. 2020-05-05 00:14:35 +00:00
Adam Williamson
ea9ac508ac Fix check_desktop variable timeouts
I forgot that tries was configurable. Sigh. Convert it to a
timeout argument.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-18 14:54:48 -07:00
Lukáš Růžička
f3d6a9574c Add desktop login test, revise and rename check_desktop
This adds a new test that implementsQA:Testcase_desktop_login
on both GNOME and KDE.

While working on this, we realized that the "desktop_clean"
needles were really "app menu" needles, and for KDE, this was
a duplication with the new "system menu" needles, because on KDE
the app menu and the system menu are the same. So I (Adam)
started to de-duplicate that, but also realized that "app menu
button" is a much more accurate name for these needles, so I was
renaming the old desktop_clean needles to app_menu_button. That
led me to the realization that "check_desktop_clean" is itself a
dumb name, because we don't (at least, any more, way back in the
mists of time we may have done) do anything to check that the
desktop is "clean" - we're really just asserting that we're at a
desktop *at all*. While thinking *that* through, I *also* realized
that the whole "open the overview and look for the app grid icon"
workaround it did is no longer necessary, because GNOME doesn't
use a translucent top bar any more. That went away in GNOME 3.32,
which is in Fedora 30, our oldest supported release.

So I threw that away, renamed the function "check_desktop",
cleaned up all the needle naming and tagging, and also added an
app menu needle for GNOME in Japanese because we were missing
one (the Japanese tests have been using the "app grid icon"
workaround the whole time).
2020-04-17 17:27:04 -07:00
Michel Normand
cc5c85a3e4 Increase timeout for getting_started ppc64le f32
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2020-04-03 19:38:04 +02:00
Lukáš Růžička
d19cf9553e Reneedle the Gnome Application StartStop tests. 2020-03-18 18:06:34 +01:00
Adam Williamson
e8ea8a7fe5 OK adam remember how we make an empty array? you got this buddy
I believe in you! We ALL believe in you!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-17 08:33:05 -07:00
Adam Williamson
8832c5b81e Add FEDORA-2020-5f05c3ec46 as a workaround to fix FreeIPA upgrade
FreeIPA F31 -> F32 upgrade test is currently failing because
a new pki-core hit F31 stable but not F32 stable yet. It can't go
backwards on upgrade, that breaks stuff. The F32 update has been
pushed stable but just hasn't made mirrors yet as the last F32
nightly compose failed, so let's add it to the workarounds for
now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-17 08:18:11 -07:00
Adam Williamson
d9438e8042 Drop workaround updates that have been pushed stable
Except one that's pushed stable but hasn't made repos yet (as the
last F32 nightly compose failed).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-17 08:17:00 -07:00
Adam Williamson
5c6ee14ed3 Add F32 backgrounds update as an update test workaround
So at least GNOME background test passes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 21:51:32 -08:00
Adam Williamson
e7c6501737 Extend update test workaround mechanism to upgrade tests
To cover cases like #1767351 where we need to apply a workaround
to the pre-upgrade environment.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 19:21:44 -08:00
Adam Williamson
9df8fdb00e Drop workaround package that's gone stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 12:54:16 -08:00
Adam Williamson
3d29d7b746 Add efivar update for F32 as workaround to fix UEFI update tests
This is pending stable, but looks like the update push won't
happen for a few hours, so I'm adding it as a workaround so we
can re-run the tests and get them to pass.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-26 12:48:17 -08:00
Adam Williamson
8e43dfa304 Drop F31 workaround that's gone stable, add F32 lmod workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 17:40:22 -08:00