This PR tries to respond to issue #294.
On Silverblue, this will try:
* flatpak install
* flatpak remote-add
* flatpak list
* flatpak remotes
* flatpak remove
* flatpak update
and also it tests that a flatpak can be built.
It's super flaky ATM due to some GNOME/mesa bug(?) This will make
life easier till I have time to actually debug the issue.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Silverblue still has eog. It seems a bit excessive to bring back
the eog test just for Silverblue, so let's just disable loupe
until Silverblue has it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a test for a new Image Viewer called Loupe.
It is based on the old Image Viewer test, newly reneedled
with some of the tests shortened, deleted or edited
as the new Image Viewer has a little bit less functions
compared to the previous one.
Add needles.
There's no obvious reason it'd ever give a different result on
KDE vs. Workstation, but since we added a matrix box for it on
both, we'd better run it on both.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The affected tests start after install_lvm_ext4, so we need to
bump HDDSIZEGB there, not in the downstream tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
With the 2G ESP change, we need bigger disks for these tests to
work correctly. On BIOS the test still just about works because
anaconda rounds the remaining space after BIOS boot partition
and /boot to 14G, but it seems fragile to rely on this. On UEFI
(aarch64), anaconda sees a max of 12G remaining after 2G ESP
and 1G /boot, so the test fails because we cannot size to 13G.
Give the test an extra 2G, this should be sufficient to make
sure it's valid on all platforms.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a Samba AD server test, and client enrolment tests via
sssd, Cockpit and kickstart. Requires the matching createhdds
commit to add the kickstart to the disk_ks image.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Modularity will be retired in Fedora 39 and the modular repository
removed. Also, the upcoming version of DNF that is default in
Rawhide, does not support modularity, so the tests are currently
failing anyway.
This requires a change in the package we use for base_update_cli
because pandoc-common is not in ELN.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The ISO or HDD image won't be attached to the post-upgrade
tests, and these are not tests of the image anyway, they're
tests of the compose.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
As of yesterday's Rawhide, the modular repos are not installed by
default, so of course all the modular tests fail. So, install
the repos before running the tests.
This isn't conditionalized on release version as I don't think
we ever run this test on anything other than Rawhide and
Branched.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
desktop tests are very very fail-y on aarch64, with all kinds of
random failure cases. I suspect the worker hosts are just *slow*,
but they do have plenty of RAM per worker instance, so let's try
throwing a bit more RAM at each one and see if that helps at all.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 'universal' flavor has been kinda pointless for some time
now. It dates back to the earliest days of openQA, before Pungi
4 was a thing, when composes were very different; we only built
a boot.iso and some live images nightly for Rawhide, these
weren't even formally grouped as a 'compose' at all (fedfind had
to invent the concept). The TCs/RCs had DVD installer images
(not *Server* DVD, at the time, just a universal DVD installer).
We wanted to run some tests on the DVD image if it was available,
but we still wanted to run them for the nightlies, so we invented
a whole mechanism for that - this 'universal' flavor, with some
complicated logic in fedora_openqa which schedules universal on
the 'best available' image it can find in the compose.
All this is functionally obsolete now. All composes we test are
now run through Pungi (except the live respins, but they aren't
relevant here). In current config, the Server DVD is non-failable
on x86_64 and aarch64, which means it will *always be there* -
if it fails to build, the compose itself fails, so we won't test
it. It's failable for ppc64le, but we don't care that much about
ppc64le; I'm fine with these tests just not running if the Server
DVD happens to fail in a ppc64le compose.
As a cherry on top, some of the 'universal' tests aren't really
universal anyway, they fail if you run them on a netinst (off
the top of my head, all the NFS install tests are like this, as
we use the ISO to populate the NFS share on the server end).
So let's just move all the tests that actually need an installer
image to the Server-dvd-iso flavor. Left over in the 'universal'
flavor are upgrade tests, which don't need an ISO at all - they
boot from hard disk images and run an upgrade using repos. We
can change the scheduler logic to be more simple for these, and
just always schedule them, with no ISO attached. We could even
rename this flavor 'upgrade', but it might not be worth it.
One slight complication is that the split happened to be helping
us avoid too many tests in a single support_server cluster; we
have a cluster of five support_server tests on Server-dvd-iso
and five support_server tests on universal. I try to avoid the
clusters getting too big as you need as many worker instances on
at least one worker host as your largest cluster; if you don't
have that many, the cluster's tests simply never get scheduled.
Requiring folks to have at least ten worker instances on one
host to run these tests is a bit of a big ask. So, to handle
that, we create a support_server_2 and have the former universal
tests use that one instead, so we'll have two separate clusters
on Server-dvd-iso now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We no longer build 32-bit ARM images and we hadn't had a working
setup for testing them for years anyway.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This PR adds a small test suite to test the Characters applications.
It displays several different groups of characters and then tries
to copy one of the characters and place it into a text editor.
This makes the two rpm-ostree tests written for IoT - overlaying
and rebasing - work across all rpm-ostree-based flavors we
currently test (IoT, CoreOS and Silverblue) and runs them on
all those flavors.
This requires some other changes. For the Workstation ostree
installer update tests, we have install_default_update_ostree
upload a disk image and run these tests on that image. That means
install_default_update_ostree cannot use a scratch disk (as if
we boot it with two disks but only upload one, the subsequent
tests fail to boot, looking for the missing second disk), but its
specified disk size should be large enough for all updates.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
These actually *do* need it because they have START_AFTER_TEST
set, but they're still install tests.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
When I enabled _advisory_post for live and ostree install tests,
the point was to check that updated packages were included in
the install media and used during installation. We shouldn't run
a system update in _repo_setup_updates on this path because it
will hide the problem if the updated packages weren't included.
The INSTALL variable is for this purpose - it was previously
used to skip _advisory_post on the same path. At the same time
let's remove some stray settings of this var on non-update tests
as it serves no purpose there.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We currently snapshot after every run of _console_wait_login or
_graphical_wait_login, which means we snapshot *twice* on most
update tests as those modules get run twice. However, we almost
never use those snapshots. Snapshotting takes quite some time,
and hits the disk pretty hard, so we should avoid it unless it
is really needed.
We only have a few modules that are not fatal (and so might use
the snapshots), and most of those don't run after one of these
tests, or run after a later module that's also a milestone. Best
I can tell, only two test suites really need to use a snapshot
from a login test: server_cockpit_updates and modularity_tests.
To handle these and potential future cases, we'll add a new
module that does nothing, but is marked 'milestone', so it will
take a snapshot, and load that test after the login test if the
var LOGIN_SNAPSHOT is set, and set that var for those two suites.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is intended to allow us to split the workload across
multiple tap workers, as right now, sometimes we wind up in a
situation where all non-tap jobs are done and the non-tap
workers are sitting idle, but the poor tap worker has a backlog.
This way the tap jobs for updates can be split across two
worker hosts, which should help out.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Hopefully https://bugzilla.redhat.com/show_bug.cgi?id=2133829
is fully resolved/worked around by now, and we shouldn't need
these any more. We could drop back to 2G but let's just do
3G for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was from F35 era when Cloud_Base worked on UEFI on Rawhide
but not F33 or F34. Now F34 is EOL, we can just follow the same
policy for every release (run two tests on both BIOS and UEFI,
the rest only on UEFI).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is getting messy, but even the install_default_upload test
is failing out with 3G now. Let's try putting 4G here so it
reaches that test and see how that goes.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We keep running into 2119970 and it's really a pain restarting
tests manually all the time. So let's just retry the affected
tests three times. Big hammer time!
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Looking through more openQA failures I saw other desktop update
tests which were clearly caused by OOM kills, so let's give all
the desktop tests we run on updates 4G for now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We're seeing a lot of failures recently in desktop_browser caused
by the browser or desktop getting OOM killed. I've filed a bug
on this: https://bugzilla.redhat.com/show_bug.cgi?id=2133829
but we need to try and work around it, having the test failing
constantly is no good. Let's see if 4G of RAM is enough.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We see tests periodically come up incomplete with this:
"Migrate to file failed, it has been running for more than 240 seconds"
Turns out there's a setting for this, so...let's set it. I don't
think there's really a bug here, just when our workers are busy,
these migrations can take longer than expected.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Workstation tests on Rawhide are getting OOM killed. Let's see
if this helps. All the other arches have this much already, and
the worker hosts can handle it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
