Commit Graph

76 Commits

Author SHA1 Message Date
Chris Roberts
3e95ebfbc7 Switch VMware testing env to improve stability results
Related: rhbz#1825190
2020-08-04 20:05:34 +03:00
Alexander Todorov
99dec118aa tests: Overwrite rhel-8.json when testing with osbuild-composer
Provides a well-known location of internal repositories and does
not depend on the URLs configured in osbuild-composer itself.

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
b9e5fa5e7b tests: Allow skipping image build in compose sanity test
when $SKIP_IMAGE_BUILD is set the test skips 2 phases:
- start the compose again (after it has been cancelled) and
- waiting for it to finish & downloading the resulting image

This will allow to enable this test script in downstream gating
jobs.

Cherry-picked from cc29b99659,
https://github.com/weldr/lorax/pull/1053/

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
b3ec7af1d1 Set BACKEND=osbuild-composer if running that test scenario
- this will execute cli sanity tests in parallel with the other
- make sure to pass BACKED to vm.install too
- more checks in lib.sh

Cherry-picked from cc29b99659

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
359b99a178 tests: Don't check info after compose cancel with osbuild-composer
osbuild-composer doesn't remove cancelled composes and even if we
wait for the cancellation to take place the UUID is still there
and `compose info` returns an exit code of 0 instead of 1

Cherry-picked from 27c4c67a59

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
de3f548025 tests: Compare blueprints as TOML objects, not strings
this is to avoid differences in ordering and white space. The same
approach is used on rhel7-extras branch.

Cherry-picked from 04bc5e9e86

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
45d227a189 tests: Remove lorax-composer specific checks
Cherry-picked from eb7748236c

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
76b52daa97 tests: Remove compose after we're done
Cherry-picked from 1c218ecbae

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Lars Karlitski
3734129e67 tests: don't use beakerlib in blueprint
b/c for osbuild-composer beakerlib is not included in the
distributions we test on.

Cherry-picked from abcbbf3194

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Lars Karlitski
8f860cc8a1 tests: don't depend on internal state of composer
Cherry-picked from 123827c90a

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
6b2efa0da1 tests: Do not rely on example blueprints
- avoids having to modify them in the first place so less changes
  to the SUT
- will help with transition to osbuild-composer backend
- each test which needs a blueprint either declares it on its own
  (some already do this) or may use a shared blueprints from the
  tests lib dir

Cherry-picked from 7b08fa8838

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
6240df6caf tests: Special case compose types for osbuild-composer
- alibaba - will be added later
- google - will be added later, low priority b/c
  GCE doesn't fully support RHEL8
- live-iso - not supported for now
- ext4 & partitioned disk - removed from osbuild-composer
- hyper-v & liveimg-tar are available only on master branch

Cherry-picked from 4dd7cf8798

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
b72bb63599 tests: Don't check example blueprints if we don't have to
ATM osbuild-composer doesn't ship example blueprints

Cherry-picked from d68e01b7e1

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Alexander Todorov
9193384bb8 tests: Use BACKEND env variable instead of hard-coded values
- default BACKEND to lorax-composer
- pass BACKEND everywhere we need to

Cherry-picked from 14a3c8d5a7

Related: rhbz#1825190
2020-07-28 14:21:47 -07:00
Chris Roberts
9dfc0e6baf Update datastore for VMware testing
Related: rhbz#1825190
Signed-off-by: Brian C. Lane <bcl@redhat.com>
2020-04-28 13:57:10 -07:00
Jakub Rusz
0f20c8f5cf tests: Add new test to run 'test' and 'check' in cockpit ci
(cherry picked from commit ac01d52fa6c27de274394dfaa144990f05d9b8df)

Related: rhbz#1785154
2020-02-07 11:51:13 +01:00
Alexander Todorov
cb92c839f2 Test & cleanup script for Alibaba cloud
Cherry-picked from 718ac31c47

Related: rhbz#1785154
2020-02-04 09:51:40 +01:00
Jakub Rusz
4dec82ae20 tests: run ssh commands in batch mode
(cherry picked from commit 711668dc99)

Related: rhbz#1785154
2020-01-24 14:27:40 +02:00
Jakub Rusz
2a310efdcf tests: remove --test=2 from compose_sanity
This ensures that after canceling a compose, the next one can be finished.

(cherry picked from commit 2ee92b75b0)

Related: rhbz#1788461
2020-01-21 15:27:13 +02:00
Jakub Rusz
14d399fe0c tests: Check for cloud-init presence in azure image
This tests if the cloud-init is installed and running.
Cherry-picked from: bae3cb06bc

Related: rhbz#1754711
2020-01-14 19:20:38 +02:00
Alexander Todorov
5189a97f0a tests: Enable Elastic Network Adapter support for AWS
- this is required for arm64 but is present in all latest kernels
  so doesn't seem to hurt
- when registering the AMI mark its architecture properly

Cherry-picked from 6c35448feb

Related: rhbz#1789308
2020-01-13 15:35:54 -08:00
Alexander Todorov
69589123b7 AWS test: take into account different instance type for non x86
- also fix equality comparison to use == for consistency

Cherry-picked from d3d7e072d9

Related: rhbz#1789308
2020-01-13 15:35:54 -08:00
Brian C. Lane
17ce6ee4e4 lorax-composer: Enable ami on aarch64
Resolves: rhbz#1789308
2020-01-13 15:10:16 -08:00
Jakub Rusz
23d528c705 Add test for canceling a running compose
This is a modification of an existing test for compose sanity.
Cherry-picked from master 8c9f528cd4

Related: rhbz#1788461
2020-01-11 13:52:33 +02:00
Jiri Kortus
d20b2f3309 Add test for running composer with --no-system-repos option
- add test coverage for rhbz#1650363
- cherry-picked from 5322664432

Related: rhbz#1785154
2020-01-06 09:31:57 +02:00
Brian C. Lane
054669d027 composer-cli: Only display the available compose types
The enabled bool is now being used so the cli should only show the types
actually available on the architecture.

Also modifies the test in test_compose_sanity.sh

Related: rhbz#1751998
2019-12-10 15:52:39 -08:00
Jakub Rusz
86ea9adbab tests: enable compose_ext4 test to use CDN repos
Related: rhbz#1777265
2019-11-27 15:08:27 +02:00
Alexander Todorov
7945d00aa1 tests: Ensure failure if beakerlib results file not found
In example here
https://209.132.184.41:8493/logs/pull-887-20191025-100505-509b9eae-weldr-lorax--fedora-31-tar/log.html

we're seeing stuff like:
/tests/test_cli.sh: line 109: /tmp/composer-test.wIAITH/TestResults: No such file or directory

but the result is PASS

Cherry-picked from 2b578dcee8

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
3409d0a949 DRY when setting up, running & parsing results for beakerlib tests
Cherry-picked from b197e448ff

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
bb4b6b7059 tests: Use journalctl -g to check for failed login
/var/log/audit/audit.log isn't always available (eg. tar liveimg
install), but it is logged to the journal, which can be grepped with
'journalctl -g' so use that instead.

Cherry-picked from 89539a7bf4

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
a7c9c049eb tests: Fix check_root_account when used with tar liveimg test
passwd and openssh-clients need to be installed, and there is no need to
use sudo when you are connecting to the system as root.

Cherry-picked from 454af0c5d5

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
4a052314f4 tests: Use the same asserts as before
after the changes around live-iso and qcow2 test cases the asserts
used have also changed. This commit utilizes the existing test lib.

Cherry-picked from d67745d755

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
471adbd8de tests: switch to using podman instead of docker
Note: use podman-docker to avoid changing tests too much. This
is also what we have on the RHEL branches.

There's no service to be started/restarted so remove everything
related to docker service.

Cherry-picked from e1b45958f4

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
493055c03c tests: Remove nested vm from tar liveimg kickstart test
Use anaconda to install the root.tar.xz into a disk image.  Copy the
disk image to the host, and use the cockpit vm support to boot it and
test it.

Cherry-picked from 4705ff3b1e

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
020fd300a4 tests: Use --http0.9 for curl ssh test
Cherry-picked from a0ae10c2d7

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Brian C. Lane
e7cfd20ca3 tests: Split testing the image into a separate script
Nested virt is not reliable enough, especially on other arches, to rely
on for testing the created images. This moves the test code into
test_boot_* scripts to be run from inside the booted images.

It also adds copying the results of the build into
/var/tmp/test-results/, and includes the generated ssh key so that
whatever boots the image can also log in.

The tests/test_image.sh script has been added to handle running the
test_boot_* scripts without any of the extra lorax-composer specific
setup.

Cherry-picked from b8bf258a3c

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
29cfda8eae [tests] Collect compose logs after each build
- save compose logs under /var/log/$TEST
- save qemu logs under /var/log/$TEST/qemu.log
- download everything to $TEST_ATTACHMENTS so it can be saved
  in CI results

Cherry-picked from 8af9723c

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Jiri Kortus
16daa25137 [tests] Use a function to wait for compose to finish
The function is meant to replace the duplicate pieces of code used in various
tests, the polling interval is unified to 20 seconds.

Cherry-picked from 836f0ddf

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Alexander Todorov
e7f2706461 [tests] Use functions for starting and stopping lorax-composer
Cherry-picked from f978d4b01c

Related: rhbz#1769525
2019-11-08 13:57:35 +02:00
Jiri Kortus
71a666507b tests: Add kickstart tar installation test
Related: rhbz#1733975
2019-11-05 11:30:56 +02:00
Jiri Kortus
1f7efc3bba tests: add option to disable kernel command line parameters check
Related: rhbz#1733975
2019-11-05 11:30:56 +02:00
Brian C. Lane
3b0bb5718c tests: Use a loop to wait for VM and sshd to start
Related: rhbz#1733975
2019-11-05 11:30:56 +02:00
Alexander Todorov
305dc9b7f6 tests: Document Azure setup 2019-10-21 20:50:22 +03:00
Brian C. Lane
ecc1ab705b tests: Add check to make sure the compose actually finished
Previously it was looping, waiting for FINISHED|FAILED but was not
actually failing the test if the compose failed to build.
This adds a function to check the status of the compose and calls it
after each compose.

(cherry picked from commit d0e947fe3b)

Related: rhbz#1724073
2019-09-11 14:16:27 +03:00
Alexander Todorov
dcb4c1a0e5 Backport Cockpit CI changes for RHEL 8
Notes:

- also install podman-docker
- on RHEL 8 there's no docker.service to start

Related: rhbz#1724073
2019-09-11 14:16:27 +03:00
Brian C. Lane
c96beac237 tests: Drop sort from compose types test
The output from the API is already sorted, and 'sort' changes this
ordering in some situations.

Related: rhbz#1714103
2019-09-02 15:00:25 +03:00
Alexander Todorov
d538aa9176 New test: assert toml files in git workspace
export BLUEPRINTS_DIR for use in tests

Depending on the way the tests are run the directory may be a temporary
dir, or it may be the standard /var/lib/lorax/... path.

Related: rhbz#1714103
2019-09-02 15:00:25 +03:00
Jiri Kortus
91cfdb277e Use passwd --status for locked root account check
Related: rhbz#1728571
2019-07-11 11:55:22 +03:00
Jiri Kortus
755e71b542 Add test for passing custom option on kernel command line
Related: rhbz#1687743
2019-04-29 12:52:22 +03:00
Jiri Kortus
81ad0509f2 Use verify_image function as a helper for generic tests
Related: rhbz#1704172
2019-04-29 12:52:22 +03:00