RHEL 8's PAM has some weird bug where it rejects the *current* password
when changing an expired one. This isn't trivial to reproduce on the CLI
and thus has no bugzilla yet.
Cockpit 221 fixed storage test classes to all have different names, so
the "TestStorage" pattern now catches them all.
Also move the `image-download cirros` to simplify conditional blocks.
This provides retries on flakes, integrates tests-policy for known
issues, and outputs proper TAP format.
Once we work out the remaining kinks (unstable tests and proper test
class names), it will also be much shorter.
We currently have 3 versions of this script (one in fedora, one in rhel
cockpit and one in rhel cockpit-appstream).
With this patch we can keep this same file in all 3.
Also backporting hack in verify.sh.
Dynamically set $TEST_OS from /etc/os-release, and install chromium
directly on Fedora and from EPEL on RHEL. With these we can keep the
test in sync between RHEL and Fedora.
Also drop the obsolete resetting of system changes, that was fixed in
cockpit's tests.
Apparently when encountering some cleanup errors (at least when podman
fails to remove its temporary network), podman exits with 0 instead of
with the exit code of the container workload. This hides failing tests.
Pass the test exit code to the host via the logs directory instead.
As the firewalld tests expect the libvirt "default" network, add
cockpit-machines test dependency, which pulls in the libvirt stack. We
will soon test the Machines page anyway.
Upstream now grows more and more @nondestructive test, which we can run
in our single test VM, so add the first batch.
This uncovers some issues in the tests that assume that there is no root
login in the VM, skip them. They are being fixed in
https://github.com/cockpit-project/cockpit/pull/13773
check-menu is a very robust test that also does not change the host too
much. This does not cover a lot of functionality to actually find
regressions in subsystems, but exposes the missing pieces for
running more integration tests in the dist-git pipeline.
Re-use the cockpit/tests container for this, which has the necessary
test dependencies -- most importantly, chromium-browser. Aside from not
being available in RHEL, this also better shields the tests from the
tested system. The container talks to Cockpit on the host, i. e. the
container's gateway.
Use a fixed tag to avoid running into failures with "latest", as we
don't gate container updates on the dist-git tests for now.
- Move from the deprecated standard-test-scripts to the better
supported standard-test-basic. This can install test dependencies.
- Drop the obsolete standard-test-overlay, it does not even exist any
more.
- Drop test-canary. This was being used to test the CI gating, but this
has been done now.
Later once the upstream no longer uses phantomjs, we should
be able to execute the entire integration test suite.
This includes the standard-test-overlay role, and the
standard-test-scripts role. These will be suggested to the
standard-test-roles repo.
