1
0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-11-15 19:23:10 +00:00
Commit Graph

30 Commits

Author SHA1 Message Date
Adam Williamson
460eee87bc updates: test Silverblue install on UEFI as well as BIOS
https://bugzilla.redhat.com/show_bug.cgi?id=2268505 made it
clear that this is a bit of a hole. We don't test installing the
Silverblue image we build on UEFI, only on BIOS. Add this as a
separate test so we don't uselessly upload a disk image we won't
use for any follow-on tests.

This also adds an anaconda build that fixes a bug in this path
as a workaround for F40, so the test won't fail on all F40
updates.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-07 16:13:02 -08:00
Adam Williamson
54f69463b0 Use NUMDISKS=2 (or higher) on all update flavors
We stopped doing this on Server because it caused problems with
tests that use a disk image uploaded by another test, e.g. the
cockpit tests - they use the `/etc/fstab` from the disk image
the parent test uploaded, which says to mount the second disk as
/mnt/update_repo, but since this is a new test it has a fresh,
empty second disk with no filesystems to mount. This tries to
fix that by making _console_shutdown.pm edit that line back out
of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree
flavor, which had a similar problem with the overlay and rebase
tests using a disk image uploaded by the install test).

We need to fix this because FEDORA-2024-9b9da603e1 is so big
it causes the tests that don't use a scratch disk to run out of
disk space.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-28 15:26:01 -08:00
Adam Williamson
0aba13ec8f Bump disk size on ostree build test to 30 GB
Seems we need more space on the scratch disk used by the ostree
build test, it's failing due to lack of space. Tested on lab, it
worked.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-20 20:14:34 -08:00
Adam Williamson
965dc99ef5 Give ostree_build another half hour again
Another bunch of these timed out. Not sure why. Maybe it's when
I run a lot of them at the same time? Let's try this, again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-06 23:07:25 -08:00
Adam Williamson
09e4f526a8 Revert "Give ostree_build another half hour"
This reverts commit 56c9e80f60.
Things seem to have settled down with the mass rebuild and this
test seems to be back to consistently taking about 90 minutes.
2024-01-30 09:47:44 -08:00
Adam Williamson
56c9e80f60 Give ostree_build another half hour
It seems to be timing out a lot on Rawhide lately. Not sure if
it's just mass rebuild stuff, but anyhow...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-01-29 17:03:30 -08:00
Adam Williamson
309ccc45ee Revert "Bump RETRY for updates-kde to 2 while we have oom kill issues"
This reverts commit 1bea573fec. We
don't have OOM kill issues any more.
2023-12-04 10:07:24 -08:00
Adam Williamson
74b468b949 Add Samba AD tests
This adds a Samba AD server test, and client enrolment tests via
sssd, Cockpit and kickstart. Requires the matching createhdds
commit to add the kickstart to the disk_ks image.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-27 16:14:30 -07:00
Adam Williamson
1f24f84bb1 Support testing a side tag instead of an update or task
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-04 19:10:03 -07:00
Adam Williamson
97618193c6 Adjust tests for update and workaround repos provided as ISO
I'm attempting a new approach to the update and workaround repos.
Instead of having each update test recreate them for itself -
which is slow and wastes bandwidth - the dispatcher will create
an ISO at test schedule time and pass it as ISO_2. Then the test
just mounts the ISO. This makes the necessary adjustments on the
test side.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-06-19 20:21:07 +02:00
Adam Williamson
6daf9c36a0 Make rpm-ostree tests generic, run on Silverblue and CoreOS
This makes the two rpm-ostree tests written for IoT - overlaying
and rebasing - work across all rpm-ostree-based flavors we
currently test (IoT, CoreOS and Silverblue) and runs them on
all those flavors.

This requires some other changes. For the Workstation ostree
installer update tests, we have install_default_update_ostree
upload a disk image and run these tests on that image. That means
install_default_update_ostree cannot use a scratch disk (as if
we boot it with two disks but only upload one, the subsequent
tests fail to boot, looking for the missing second disk), but its
specified disk size should be large enough for all updates.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-16 08:44:43 -08:00
Adam Williamson
03b6663339 Add tests to build a Silverblue installer image and install it
This is like the existing tests that build network install and
live images then install them, only for Silverblue. First we
build an ostree, using the standard configuration for the release
and subvariant but with the 'advisory' and 'workarounds' repos
included, so it will contain current stable packages plus the
packages from the update and any workarounds. Then we build an
ostree installer image with the ostree embedded, again including
advisory and workarounds repos in the installer build config so
packages from them will be included in the installer environment.
The image is uploaded, which completes the _ostree_build test.
Then an install_default_update_ostree test runs, which does a
standard install and boot from the installer image.

We do make a change that affects other tests, too. We now run
_advisory_post on live image install tests, as well as this new
ostree install image install test. It was skipped before because
of an exception that's really only needed for the netinst image
install test. In that test, packages from the update won't be
included in the installed system, so we can't run _advisory_post
on it. But for ostree and live image build/install tests, the
installed system *should* include packages from the update, so
we should check and make sure that it does.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-30 13:17:28 -08:00
Adam Williamson
1429e5b1c8 Give all update tests the same priority, ahead of compose tests
It doesn't make sense for update tests to have different
priorities. We gate on all update tests, so they are all equally
important. With varying priorities we get a problem: if 10
updates are created quite close together, first we do all the
prio 30 and prio 40 tests for all of them, and only after all of
those are done do we get to the prio 41 and 42 tests for the
earliest of the updates, which means they can be waiting longer
than they should to have all the tests done. If all the tests
have the same priority, openQA should always use creation time
to order them, so it should prioritize finishing tests for
earlier updates, which is what we want.

Update tests are also more time-critical than compose tests,
because we gate on update tests. So we should prioritize update
tests over compose tests in all cases.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 08:43:30 -08:00
Adam Williamson
b7a3d9ee1b Correct casing of SUBVARIANT variables
In moving these from fedora_openqa to here, I got the casing
wrong, which breaks `tell_source` and causes a test to fail.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 11:55:47 -07:00
Adam Williamson
73102e877a Move some update params here from fedora_openqa schedule.py
We had code in fedora_openqa schedule.py to set some params for
update tests depending on the flavor. I don't remember why I
ever wrote it that way, but that's an odd way to do it, the
obvious way to do it is just to have them here, like we already
have several others and like we do it for composes. This will
help with https://progress.opensuse.org/issues/119200 because
the params set by fedora_openqa schedule.py go in the request
URI but ones set here don't.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 09:51:38 -07:00
Adam Williamson
1bea573fec Bump RETRY for updates-kde to 2 while we have oom kill issues
https://bugzilla.redhat.com/show_bug.cgi?id=2133829 is a real
problem for Rawhide KDE update tests right now, despite all
I've tried to do to work around it. I'm going to regenerate the
base image again tomorrow, but for now, let's bump RETRY to 2
as a mitigation.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-23 09:14:35 -07:00
Adam Williamson
a98671a670 Give install_default_update_live 4G of RAM too
This test seems to be struggling with 3G of RAM also.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-17 14:19:50 -07:00
Adam Williamson
aadcc428d6 Give live image build even longer and bump max job time
Rawhide live image builds are still taking an awful long time
and often failing. I will look more into why later, but for now,
let's bump the timeouts even more just to try and get through
the job backlog.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-19 15:35:24 -04:00
Adam Williamson
66e5276544 Drop NUMDISKS=2 for update server flavor
It causes a bit of an awkward problem for tests which use disk
images from another test, specifically the cockpit tests. We
put the update repo on this second disk and update /etc/fstab
but we aren't actually uploading the second disk image and using
it as the second disk on the child tests, so they get messed
up.

I'm having trouble coming up with an elegant solution so for now
let's kick the affected flavor (server) back to one disk. I'll
try and figure a more permanent fix tomorrow.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:28:15 -07:00
Adam Williamson
f5946e678c Make update testing more robust for very large updates
openQA choked badly on
https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71
because it's, well, huge - 87 builds including texlive, which
has hundreds (thousands?) of subpackages. This exposed several
frailties against such updates.

First of all, we set NUMDISKS to at least 2 for *all* update
tests, which should mean they all stash the RPMs from the update
on a non-system disk and avoid problems with space exhaustion.
After that, just extend a few timeouts in particularly fragile
places, including one which is specific to texlive (as I don't
know of any other source package with so many subpackages).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 11:29:21 -07:00
Adam Williamson
343ef8226b Use new RETRY variable handling to restart update tests on fail
Upstream recently implemented support for using the variable
RETRY to specify how many times a test should be restarted on
failure. This is something we currently handle with a downstream
openQA plugin; if we switch to using this upstream feature
instead, we can drop the plugin.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 10:28:57 -07:00
Adam Williamson
25f59d86da Bump HDDSIZEGB for more GNOME and KDE products
We have it at 20G for Workstation live, but not for KDE live or
for the update Workstation or KDE products. We just hit an issue
today where anaconda thinks 10G isn't enough space for a KDE
live install after a grub2 update (which I think only bumped
the required space veeery slightly, but enough to throw anaconda
over the limit). Let's just go up to 15G for all GNOME and KDE
cases where we're not at 20G.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 16:53:47 -07:00
Adam Williamson
23c9adac93 Update names of qcow2 base disk images
We're changing these to be named `foo.qcow2` not `foo.img` due
to a change in qemu and os-autoinst to do with backing file
format detection.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-10 10:58:08 -07:00
Adam Williamson
9174472637 Run podman tests on updates
It has been noted that updates have broken podman in the past and
this is a major issue for some users. Let's create a new update
flavor and run the test in it. We'll use the server image as a
base, but it's not really a server test, so I'm giving it its own
flavor so it's not run on updates that we only want to run server
tests on, and we can schedule just this test to run on container-y
updates.

As part of this, we need to install podman before running the
test; for flavors we currently run it on we expect podman to be
preinstalled, but that's not true for the server base image.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 12:20:09 -07:00
Adam Williamson
bc1e9681f9 Add KDE live image build and test for updates
I hacked this up quickly on staging to test a specific update,
but there's really no reason we shouldn't just do it generally.
We have the capacity.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 16:00:16 -07:00
Adam Williamson
72edbfe991 Use qemu host IP 172.16.2.2 not 10.0.2.2
This is to make the infra folks happy, apparently using 10.0.x.x
and 10.1.x.x is causing conflicts since our actual infra network
uses those ranges too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-23 16:40:45 -07:00
Adam Williamson
3bc1e8335a Put /var/lib/mock on separate disk for live image build test
The update live image build test keeps running out of disk space.
We've bumped the minimal disk image from 12GB all the way up to
20GB so far but it keeps happening. So let's try a different
strategy: use a scratch disk to mount /var/lib/mock. That's where
all the space gets used. This should allow us to reduce the size
of the minimal disk image again, and giving it 25GB of empty disk
should avoid it running out of space again for a while.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-05 21:12:12 -07:00
Adam Williamson
80c54d5491 Add cockpit_updates and remote_logging tests to updates
Again, no reason not to run these on updates. Includes adding
oldcantarell versions of several needles for current cockpit,
as they're needed for the tests to pass. Also tweak a couple of
needles to avoid false matches (add more empty space).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 21:03:41 +01:00
Adam Williamson
44d1dc3607 Add base_reboot_unmount and base_system_logging to update tests
No reason not to run these on updates as well. And it's much
easier with FIF!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 20:50:39 +01:00
Adam Williamson
2c197d520c Add a whole intermediate template format ('FIF') and tools
I and @lruzicka (and I think @jskladan and @jsedlak and
@michelmno and everyone else who's ever touched it...) are being
gradually driven nuts by manually editing the test templates.
The bigger the files get the more awkward it is to keep them
straight and be sure we're doing it right. Upstream doesn't do
things the same way we do (they mostly edit in the web UI and
dump to file for the record), but we do still think making
changes in the repo and posting to the web UI is the right way
around to do it, we just wish the format was saner.

Upstream has actually recently introduced a YAML-based approach
to storing job templates which tries to condense things a bit,
and you can dump to that format with dump-templates --json, but
@lruzicka and I agree that that format is barely better for
hand editing in a text editor than the older one our templates
currently use.

So, this commit introduces...Fedora Intermediate Format (FIF) -
an alternative format for representing job templates - and some
tools for working with it. It also contains our existing
templates in this new format, and removes the old template files.
The format is documented in the docstrings of the tools, but
briefly, it keeps Machines, Products and TestSuites but improves
their format a bit (by turning dicts-of-lists into dicts-of-
dicts), and adds Profiles, which are combinations of Machines and
Products. TestSuites can indicate which Profiles they should be
run on.

The intermediate format converter (`fifconverter`) converts
existing template data (in JSON format; use tojson.pm to convert
our perl templates to JSON) to the intermediate format and
writes it out. As this was really intended only for one-time use
(the idea is that after one-time conversion, we will edit the
templates in the intermediate format from now on), its operation
is hardcoded and relies on specific filenames.

The intermediate format loader (`fifloader`) generates
JobTemplates from the TestSuites and Profiles, reverses the
quality-of-life improvements of the intermediate format, and
produces template data compatible with the upstream loader, then
can write it to disk and/or call the upstream loader directly.

The check script (`fifcheck`) runs existing template data through
both the converter and the loader, then checks that the result is
equivalent to the input. Again this was mostly written for one-
time use so is fairly rough and hard-coded, but I'm including it
in the commit so others can check the work and so on.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:21:23 +01:00