History

Lars Karlitski 42c7c0691c Monkey-patch beakerlib to fail on first assert Beakerlib upstream can't do this yet, but might at some point: https://github.com/beakerlib/beakerlib/issues/42 This is only enabled in combination with the `--sit` option of the `test/check-*` scripts. It leaves the system in exacly the state it was in when an assertion failed. Finishing the test run would run cleanup as well (such as deleting created images). It also takes longer.		2019-06-04 20:12:54 +02:00
..
check-api	Don't send CORS headers	2019-06-04 20:10:03 +02:00
check-cli	Split live-iso and qcow2 and update test scenario execution	2019-05-31 12:57:23 +02:00
check-cloud	Use Cockpit's test images and infrastructure	2019-05-20 20:24:01 +02:00
composertest.py	Monkey-patch beakerlib to fail on first assert	2019-06-04 20:12:54 +02:00
README.md	Don't execute compose/blueprint sanity tests in Travis CI	2019-05-31 12:57:23 +02:00
run	Add API integration test	2019-06-04 11:50:25 +03:00
vm.install	test: Install beakerlib on non-RHEL images	2019-05-23 12:00:30 +02:00

README.md

Integration Tests

lorax uses Cockpit's integration test framework and infrastructure. To do this, we're checking out Cockpit's bots/ subdirectory. It contains links to test images and tools to manipulate and start virtual machines from them.

Each test is run on a new instance of a virtual machine. Branch/test matrix is configured in bots/tests-scan inside the cockpit repository.

Dependencies

These dependencies are needed on Fedora to run tests locally:

$ sudo dnf install curl expect \
    libvirt libvirt-client libvirt-daemon libvirt-python \
    python python-libguestfs python-lxml libguestfs-xfs \
    python3 libvirt-python3 \
    libguestfs-tools qemu qemu-kvm rpm-build rsync xz

Building a test VM

To build a test VM, run

$ make vm

This downloads a base image from Cockpit's infrastructure. You can control which image is downloaded with the TEST_OS environment variable. Cockpit's documentation lists accepted values. It then creates a new image based on that (a qemu snapshot) in tests/images, which contain the current tests/ directory and have newly built rpms from the current checkout installed.

To delete the generated image, run

$ make vm-reset

Base images are stored in bots/images. Set TEST_DATA to override this directory.

Running tests

After building a test image, run

$ ./test/check-cli [TESTNAME]

or any of the other check-* scripts. To debug a test failure, pass --sit. This will keep the test machine running after the first failure and print an ssh line to connect to it.

Run make vm after changing tests or lorax source to recreate the test machine. It is usually not necessary to reset the VM.

Updating images

The bots/ directory is checked out from Cockpit when make vm is first run. To get the latest images you need to update it manually (in order not to poll GitHub every time):

$ make -B bots