Commit Graph

67 Commits

Author SHA1 Message Date
Martin Pitt
96f32077a5 tests: Adjust for renamed upstream test 2021-02-19 06:06:50 +01:00
Martin Pitt
13869944f3 test: Check os-release instead of /dev/kvm for Machines tests
Commit e17149cb did not work as intended -- the RHEL gating machines
also don't have /dev/kvm, but are still powerful enough to run all
Machines tests. So consult /etc/os-release instead of /dev/kvm.
2021-02-18 10:12:58 +01:00
Martin Pitt
6b9fb734bb Revert "test: Exclude TestLogin.testServer"
This reverts commit 6f3becab15.

This test works fine in 33, and lots of other tests fail, there is a
more fundamental reason for this.
2021-02-17 08:30:08 +01:00
Martin Pitt
e17149cb3b tests: Only run one Machines test on VM instance without /dev/kvm
Each Machines test takes about 4 minutes with QEMU emulation, and they
are very prone to failing. Due to the automatic retry, the Machines
tests currently take about 6 hours on Fedora gating infrastructure, and
are just about impossible to get succeeding.
2021-02-16 14:45:21 +01:00
Martin Pitt
6f3becab15 test: Exclude TestLogin.testServer
It fails in Fedora for an unknown reason.
2021-02-16 08:49:39 +01:00
Martin Pitt
f5c3319f6f Run tests less verbosely 2021-02-16 08:42:49 +01:00
Martin Pitt
a0f5493eef tests: Fix browser.sh path detection to be less STI specific
Only set `$SOURCE` to ./source/ if that actually exists, which is when
it is being called through Standard Test Interface.

Provide fallbacks for using FMF or when calling the script manually.
2021-02-10 06:46:21 +01:00
Martin Pitt
168e0397aa tests: Pull in older version on chromium
To work around https://bugs.chromium.org/p/chromium/issues/detail?id=1170634
2021-02-10 06:44:57 +01:00
Martin Pitt
18a7b9c999 tests: Drop obsolete cockpit-dashboard
It is not built any more.
2021-02-04 17:26:34 +01:00
Martin Pitt
0040d33066 tests: Drop npm
The necessary NPM modules for testing are part of the
release tarball since Cockpit 237:
https://github.com/cockpit-project/cockpit/commit/c00710475
2021-02-02 07:20:03 +01:00
Martin Pitt
e88473cf0c tests: Run TestLogin.testExpired again
Fixed in 237 in https://github.com/cockpit-project/cockpit/pull/15164
2021-02-02 07:15:22 +01:00
Martin Pitt
8504d70463 tests: Sync with RHEL 8 cockpit-appstream
Version 236 fixed a lot of Machines tests, and also split/renamed them.
2021-01-22 16:35:56 +01:00
Katerina Koukiou
e229288441 test: testAddDisk and testCreate don't exist upstream any more
These tests have been split into smaller chunks for ex. by the upstream
commit:
c485484189
2020-11-25 07:23:27 +00:00
Katerina Koukiou
14a0914014 Set root password to 'foobar' in test Vm
TestAccounts.testRootLogin logs in as root.
2020-11-16 11:20:42 +01:00
Martin Pitt
437bc76686 test: Avoid npm install downloading everything
In the presence of a package-lock.json, `npm install` apparently
downloads all packages even when giving explicit ones on the CLI. Avoid
that by removing package-lock.json before.
2020-09-03 14:34:30 +02:00
Martin Pitt
25eef42c91 tests: Cockpit 227 knows about Fedora 33 now 2020-09-02 16:47:28 +02:00
Martin Pitt
e55b16ec59 tests: Adjust for Fedora 33 split 2020-08-25 07:13:40 +02:00
Martin Pitt
a464c2534b tests: Add sssd-dbus test dependency
Required for new TestLogin-testClientCertAuthentication. Normally this
gets pulled in through freeipa-client, but we don't run IdM tests.
2020-07-09 07:39:55 +02:00
Martin Pitt
3a21750157 test: Exclude two unstable Machines tests 2020-07-01 06:04:41 +02:00
Martin Pitt
711e67fb14 test: Add missing test dependencies 2020-07-01 06:04:35 +02:00
Martin Pitt
1ebb3575a5 test: PAM bug affects Fedora 31 as well 2020-07-01 06:04:30 +02:00
Matej Marusak
8cfa228f5a Add kdump test and remove obsoleted excludes
The mentioned PR was merged and thus these tests should be good to go.
2020-06-26 07:12:28 +02:00
Martin Pitt
f894e9dea6 tests: Enable more Machines tests
These seem work fine now, especially with run-tests flake retrying.

testCreatePXE() and some others will be fixed in the next upstream
release.

testCreate() still seems to give trouble, though.
2020-06-16 17:37:42 +02:00
Martin Pitt
9b9428e82e tests: Enable more basic tests
With that, all upstream tests applicable to RHEL BaseOS now run in
gating.
2020-06-16 13:58:54 +02:00
Martin Pitt
bc3c660e8b tests: Drop run-tests download hack
Cockpit 221.1 includes what the test needs.
2020-06-15 22:02:18 +02:00
Martin Pitt
db96fd730f test: Skip some new Machines tests
They didn't run before, and still need some adjustment for the gating
environment.
2020-06-15 15:06:29 +02:00
Martin Pitt
c7c30d7162 tests: Skip TestAccounts.testBasic
The "Terminate session" button does not work in CI, this needs to be
investigated. Skip that test for now.
2020-06-15 14:45:52 +02:00
Martin Pitt
12407e7f2f tests: Skip TestLogin.testTally
This doesn't work on our CI environment, still needs debugging. It
didn't run before.
2020-06-15 13:48:52 +02:00
Martin Pitt
5f7e778846 tests: Adjust unexpected message pattern
This also hits for a related message about COPR.
2020-06-15 13:46:27 +02:00
Martin Pitt
1c54526baf tests: Use run-tests --exclude option
This is simpler, more robust, and fixes the silently skipped tests as
the `{}` did not get expanded by the shell any more.
2020-06-15 13:46:27 +02:00
Martin Pitt
029dfa3410 tests: Exclude TestLogin.testExpired on RHEL
RHEL 8's PAM has some weird bug where it rejects the *current* password
when changing an expired one. This isn't trivial to reproduce on the CLI
and thus has no bugzilla yet.
2020-06-15 09:35:27 +02:00
Martin Pitt
8cdec9d69e tests: Grab latest upstream run-tests to fix BlockingIOError crash
This can be reverted once version 222 gets packaged, which includes the
fix.
2020-06-15 07:48:30 +02:00
Martin Pitt
2734e8d5fe tests: Fix cockpit-appstream detection
cwd in run-tests.sh is just the test user's home directory. Look into
the original test unpack directory for the cockpit-appstream spec file.
2020-06-15 07:43:19 +02:00
Martin Pitt
bbd0810e02 tests: Make libpwquality less aggressive
So that our standard "foobar" password works.
2020-06-15 07:12:44 +02:00
Martin Pitt
ebd03582ec tests: Properly set up ssh access
Allow the unprivileged `runtest` account to ssh to localhost as root.
2020-06-15 07:06:33 +02:00
Martin Pitt
a6f0e0a3b0 tests: Ignore message about wrong beakerlib app-info permissions
This is unrelated to the test, and really uninteresting.
2020-06-15 07:06:30 +02:00
Martin Pitt
a043a83424 tests: Use run-tests for everything
Cockpit 221 fixed storage test classes to all have different names, so
the "TestStorage" pattern now catches them all.

Also move the `image-download cirros` to simplify conditional blocks.
2020-06-14 16:13:00 +02:00
Martin Pitt
6c679bed82 tests: Fix nonexisting variable access in run-test.sh 2020-05-30 12:41:30 +02:00
Martin Pitt
ecd7b62189 tests: Use run-tests
This provides retries on flakes, integrates tests-policy for known
issues, and outputs proper TAP format.

Once we work out the remaining kinks (unstable tests and proper test
class names), it will also be much shorter.
2020-05-28 18:57:06 +02:00
Matej Marusak
b6bc45dfef test: 'run-test.sh' can decide which tests to run
We currently have 3 versions of this script (one in fedora, one in rhel
cockpit and one in rhel cockpit-appstream).
With this patch we can keep this same file in all 3.

Also backporting hack in verify.sh.
2020-05-28 15:43:32 +00:00
Martin Pitt
e929045b34 tests: Adjust to version 219
- check-machines-dbus got renamed to check-machines
 - More Machines tests should be stable now
2020-05-14 14:00:31 +02:00
Martin Pitt
63f5fd2065 test: Re-disable TestFirewall.testNetworkingPage
Still too unstable.
2020-04-28 16:29:07 +02:00
Martin Pitt
5c652c5d08 test: Re-disable TestMachinesDBus.test{Libvirt,NetworkSettings}
They are still too unstable.
2020-04-28 13:59:04 +02:00
Martin Pitt
6d1be3eb74 test: Run browser and tests directly on the host
Dynamically set $TEST_OS from /etc/os-release, and install chromium
directly on Fedora and from EPEL on RHEL. With these we can keep the
test in sync between RHEL and Fedora.

Also drop the obsolete resetting of system changes, that was fixed in
cockpit's tests.
2020-04-28 13:59:04 +02:00
Martin Pitt
a791ac3ba7 tests: Re-drop testNICAdd
It's too unstable: https://github.com/cockpit-project/cockpit/issues/13861
2020-04-07 20:13:35 +02:00
Martin Pitt
365ca755d7 tests: Fix syntax error 2020-04-07 13:43:40 +02:00
Martin Pitt
cc5550a75f test: Enable first libvirt tests
Enable the ones corresponding to the selenium tests [1], except for
some which are still too unstable.

[1] https://github.com/cockpit-project/cockpit/issues/13481
2020-04-07 07:21:17 +02:00
Martin Pitt
15c2932687 tests: Split host and container run scripts
This fixes trap handling and also is easier to write and debug.
2020-04-06 20:44:29 +02:00
Martin Pitt
46f292cc92 tests: Robustify exit code handling from podman
Apparently when encountering some cleanup errors (at least when podman
fails to remove its temporary network), podman exits with 0 instead of
with the exit code of the container workload. This hides failing tests.

Pass the test exit code to the host via the logs directory instead.
2020-04-06 20:26:11 +02:00
Martin Pitt
259131798c tests: Fix PackageKit tests 2020-04-06 20:25:45 +02:00