Commit Graph

2098 Commits

Author SHA1 Message Date
Alexander Todorov
ed08a283c4 tests: unskip Qcow2 and Live-ISO scenarios
previously relying on nested virt

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
7945d00aa1 tests: Ensure failure if beakerlib results file not found
In example here
https://209.132.184.41:8493/logs/pull-887-20191025-100505-509b9eae-weldr-lorax--fedora-31-tar/log.html

we're seeing stuff like:
/tests/test_cli.sh: line 109: /tmp/composer-test.wIAITH/TestResults: No such file or directory

but the result is PASS

Cherry-picked from 2b578dcee8

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
1dbc7ffaf3 tests: Documentation updates
Cherry-picked from c43ba9e78f

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
daf71fa267 tests: Use host repositories for make vm
to help with running the tests by hand on downstream snapshots.
In that scenario we want TEST_OS/VM_IMAGE to look as closely as
possibly like the snapshot that we'd like to test.

Cherry-picked from 1351c4dc63

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
1a89141d9d Remove unused make targets
these were only used by Jenkins but we don't have Jenkins anymore

Cherry-picked from e7e70313ef

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
3409d0a949 DRY when setting up, running & parsing results for beakerlib tests
Cherry-picked from b197e448ff

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
bb4b6b7059 tests: Use journalctl -g to check for failed login
/var/log/audit/audit.log isn't always available (eg. tar liveimg
install), but it is logged to the journal, which can be grepped with
'journalctl -g' so use that instead.

Cherry-picked from 89539a7bf4

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
a7c9c049eb tests: Fix check_root_account when used with tar liveimg test
passwd and openssh-clients need to be installed, and there is no need to
use sudo when you are connecting to the system as root.

Cherry-picked from 454af0c5d5

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
4a052314f4 tests: Use the same asserts as before
after the changes around live-iso and qcow2 test cases the asserts
used have also changed. This commit utilizes the existing test lib.

Cherry-picked from d67745d755

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
471adbd8de tests: switch to using podman instead of docker
Note: use podman-docker to avoid changing tests too much. This
is also what we have on the RHEL branches.

There's no service to be started/restarted so remove everything
related to docker service.

Cherry-picked from e1b45958f4

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
493055c03c tests: Remove nested vm from tar liveimg kickstart test
Use anaconda to install the root.tar.xz into a disk image.  Copy the
disk image to the host, and use the cockpit vm support to boot it and
test it.

Cherry-picked from 4705ff3b1e

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
020fd300a4 tests: Use --http0.9 for curl ssh test
Cherry-picked from a0ae10c2d7

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
f87db8844f test: Boot the live-iso faster, and login using ssh key
On Fedora 31 passworless root login is no longer working. We already
install a ssh key, may as well use it.

This also reduces the live boot timeout to 2s from 60s, which should
help with timeout problems when booting.

Cherry-picked from e3654e8297

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
e7cfd20ca3 tests: Split testing the image into a separate script
Nested virt is not reliable enough, especially on other arches, to rely
on for testing the created images. This moves the test code into
test_boot_* scripts to be run from inside the booted images.

It also adds copying the results of the build into
/var/tmp/test-results/, and includes the generated ssh key so that
whatever boots the image can also log in.

The tests/test_image.sh script has been added to handle running the
test_boot_* scripts without any of the extra lorax-composer specific
setup.

Cherry-picked from b8bf258a3c

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
5a3a6fb942 test: Split up the test class to allow booting other images
This splits out the lorax-composer specific execution so that the built
image can be downloaded from the build vm and booted by the host instead
of using nested virt to try and boot it inside the build vm.

Also adds copying the ssh key from the build vm so that it can log into
the image and run the test_boot_* scripts.

Cherry-picked from df7a018ee2

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
29cfda8eae [tests] Collect compose logs after each build
- save compose logs under /var/log/$TEST
- save qemu logs under /var/log/$TEST/qemu.log
- download everything to $TEST_ATTACHMENTS so it can be saved
  in CI results

Cherry-picked from 8af9723c

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Jiri Kortus
16daa25137 [tests] Use a function to wait for compose to finish
The function is meant to replace the duplicate pieces of code used in various
tests, the polling interval is unified to 20 seconds.

Cherry-picked from 836f0ddf

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
e7f2706461 [tests] Use functions for starting and stopping lorax-composer
Cherry-picked from f978d4b01c

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Jiri Kortus
71a666507b tests: Add kickstart tar installation test
Related: rhbz#1733975
2019-11-05 11:30:56 +02:00
Jiri Kortus
1f7efc3bba tests: add option to disable kernel command line parameters check
Related: rhbz#1733975
2019-11-05 11:30:56 +02:00
Brian C. Lane
3b0bb5718c tests: Use a loop to wait for VM and sshd to start
Related: rhbz#1733975
2019-11-05 11:30:56 +02:00
Brian C. Lane
1a08cbf374 tests: Drop an old bug reference
These weren't built until after 8.0.0, which they were approved for, and
even though they have all acks, it's for the wrong release so the build
system complains when they end up back in lorax.spec
2019-10-31 15:12:39 -07:00
Brian C. Lane
199aa67a13 Automatic commit of package [lorax] release [28.14.32-1].
Created by command:

/bin/tito tag
2019-10-31 15:02:48 -07:00
Brian C. Lane
c11a8b8018 tests: Add tests for recipe checking functions
(cherry picked from commit f1733369fa)

Related: rhbz#1716596
2019-10-31 10:11:13 -07:00
Brian C. Lane
7270511379 lorax-composer: Add basic case check to check_recipe_dict
This will detect if an expected field is not all lower case as is
required.

(cherry picked from commit 13a7dcf2b1)

Related: rhbz#1716596
2019-10-31 10:11:13 -07:00
Brian C. Lane
229253b0cb lorax-composer: Add basic recipe checker function
This makes sure that required fields are included, and that sections are
not empty. It does not check for all optional fields.

If there are errors it will gather up all of them and then raise a
RecipeError with a string of all the errors.

(cherry picked from commit 61059a2699)

Resolves: rhbz#1716596
2019-10-31 10:11:13 -07:00
Brian C. Lane
94d0c036ea Add live iso support to s390
With this patch lmc and lorax-composer can be used to create a live iso
when running on s390 systems.

Related: rhbz#1746424
2019-10-31 09:53:13 -07:00
Brian C. Lane
140f40f6be Disable some compose types on other architectures
The 'enabled' field in the /compose/types output now reflects whether or
not the type is supported on the current architecture. Disabled types
are not allowed to be built, and will raise an error like:

Compose type 'alibaba' is disabled on this architecture

Resolves: rhbz#1751998
2019-10-28 17:03:18 -07:00
Brian C. Lane
115bf6c17f aarch64: Fix live-iso creation on aarch64
Drop unneeded uboot-tools, and remove iso-graft from the aarch64.tmpl

Resolves: rhbz#1752002
(cherry picked from commit 5efaa876e1)
2019-10-21 15:51:32 -07:00
Brian C. Lane
79847076b0 Automatic commit of package [lorax] release [28.14.31-1].
Created by command:

/bin/tito tag
2019-10-21 13:32:22 -07:00
Alexander Todorov
305dc9b7f6 tests: Document Azure setup 2019-10-21 20:50:22 +03:00
Alexander Todorov
f1e75264b6 tests: unskip Azure scenario 2019-10-21 19:46:45 +02:00
Martin Pitt
edde42af98 Support CI testing against a bots project PR
* If `$COCKPIT_BOTS_REF` is set, check out that bots version instead of
   master.

 * Use git cache in $XDG_CACHE_HOME if available. Our CI uses that to
   save downloads, and it does not get in the way for local developers.

Adapted from https://github.com/cockpit-project/starter-kit/pull/233

Cherry-picked from master commit a2c67385e4.
2019-10-15 08:15:53 +02:00
Sanne Raymaekers
9527e65fd0 Makefile: Update bots target for moved GitHub project
Cockpit bots are in their own project now.

Cherry-picked from master commits 4643afa58 and 29c308ae73.
2019-10-15 08:15:53 +02:00
Alexander Todorov
a0c5cacae2 Skip azure, live-iso & qcow2 tests due to infra issues
Related: rhbz#1724073
2019-09-11 14:16:27 +03:00
Brian C. Lane
ecc1ab705b tests: Add check to make sure the compose actually finished
Previously it was looping, waiting for FINISHED|FAILED but was not
actually failing the test if the compose failed to build.
This adds a function to check the status of the compose and calls it
after each compose.

(cherry picked from commit d0e947fe3b)

Related: rhbz#1724073
2019-09-11 14:16:27 +03:00
Alexander Todorov
dcb4c1a0e5 Backport Cockpit CI changes for RHEL 8
Notes:

- also install podman-docker
- on RHEL 8 there's no docker.service to start

Related: rhbz#1724073
2019-09-11 14:16:27 +03:00
Brian C. Lane
c96beac237 tests: Drop sort from compose types test
The output from the API is already sorted, and 'sort' changes this
ordering in some situations.

Related: rhbz#1714103
2019-09-02 15:00:25 +03:00
Alexander Todorov
d538aa9176 New test: assert toml files in git workspace
export BLUEPRINTS_DIR for use in tests

Depending on the way the tests are run the directory may be a temporary
dir, or it may be the standard /var/lib/lorax/... path.

Related: rhbz#1714103
2019-09-02 15:00:25 +03:00
Jiri Kortus
91cfdb277e Use passwd --status for locked root account check
Related: rhbz#1728571
2019-07-11 11:55:22 +03:00
Brian C. Lane
1dee478ac5 Automatic commit of package [lorax] release [28.14.30-1].
Created by command:

/bin/tito tag
2019-06-27 12:09:27 -07:00
Brian C. Lane
11c111c9d3 New lorax documentation - 28.14.30
Related: rhbz#1709594
2019-06-27 11:56:19 -07:00
Evan Goode
4ac29da334 Assert that RuntimeErrors have correct messages
Related: rhbz#1709594

Signed-off-by: Brian C. Lane <bcl@redhat.com>
2019-06-27 08:54:20 -07:00
Evan Goode
12bdd8ba95 More descriptive error for a bad ref in repos.git
Fixes #771

Related: rhbz#1709594

Signed-off-by: Brian C. Lane <bcl@redhat.com>
2019-06-27 08:53:44 -07:00
Brian C. Lane
a2fcd9c387 Add more tests for gitrpm.py
Make sure that dotfiles are installed when not directly under /
Make sure / is not packaged in the rpm (it will cause a conflict with
the filesystem package).
Make sure that using destination="/" works

(cherry picked from commit 6fd06c6931)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane
ff9421c848 lorax-composer: Fix installing files from [[repos.git]] to /
rpmfluff was including / in the rpm, which conflicts with
filesystem.rpm

The rpm globs are pretty limited, and we don't actually know the file
paths until later, so we have to use a glob or a directory.

So when the destination is / it now uses /* to select all the files and
sub-directories in the archive. The limitation of this is that it cannot
support dotfiles directly under /, they will cause a rpmbuild error.

For destinations other than / it uses the name of the directory, so
dotfiles are fine in that situation.

(cherry picked from commit 049f68cb60)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane
666e1fa9f0 Add tests using repos.git in blueprints
- Check final-kickstart.ks for the rpm source
- Check final-kickstart.ks for the rpm package name and version
- Make sure depsolve works
- Make sure errors from a bad repo are returned correctly
- Make sure errors from a bad reference are returned correctly

This moves _wait_for_status into a helper function so it can be shared
between the test classes.

(cherry picked from commit 8c2184d59e)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane
8bf423592f Move git repo creation into tests/lib.py
This way it can be shared with test_server.py

(cherry picked from commit 02f757d231)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane
adede63a95 rpmgit: catch potential errors while running git
Log them and report them as RuntimeError. Also add a couple tests for
them.

(cherry picked from commit 61efa91a03)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane
b3b1180b7b tests: Add test for Recipe.freeze() function
The freeze function was not being tested. Add a test for it using the
repos.git test recipe.

(cherry picked from commit c26477a63c)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00