Commit Graph

181 Commits

Author SHA1 Message Date
Alexander Todorov
8b8f5355bf [tests] Collect compose logs after each build
- save compose logs under /var/log/$TEST
- save qemu logs under /var/log/$TEST/qemu.log
- download everything to $TEST_ATTACHMENTS so it can be saved
  in CI results

Cherry-picked from 8af9723c

Related: rhbz#1770193
2019-11-19 09:59:58 -08:00
Jiri Kortus
a8d7841bc1 [tests] Use a function to wait for compose to finish
The function is meant to replace the duplicate pieces of code used in various
tests, the polling interval is unified to 20 seconds.

Cherry-picked from 836f0ddf

Related: rhbz#1770193
2019-11-19 09:59:58 -08:00
Alexander Todorov
78fbb7c49e [tests] Use functions for starting and stopping lorax-composer
Cherry-picked from f978d4b01c

Related: rhbz#1770193
2019-11-19 09:59:58 -08:00
Brian C. Lane
8e650d1d07 Update for differences from py3 in the backported code
In py3 dicts are not predictable, make sure the keys are sorted so that
the tests are useful.

Use StringIO.StringIO instead of io.StringIO which requires unicode
text.

kickstart timezone.ntpservers is a set() so adjust the test for it.

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
406dddafef Remove repos.git related tests
Support is not included in this patchset.

(cherry picked from commit a643522769)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
39f464d46b composer-cli: Update diff support for customizations and repos.git
composer-cli will now output information about changes to customizations
entries and the repos.git entries.

(cherry picked from commit 156ef0acfd)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
f0efcd9b0f Add support for customizations and repos.git to /blueprints/diff/
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.

(cherry picked from commit 850c490b6e)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
e2262e57e3 tests: Update custom-base with customizations
(cherry picked from commit 9724731b8d)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
b4a95b8d2b Change customizations.firewall to append items instead of replace
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.

Also updated the docs, added a new test for it, and sorted the output
for consistency.

(cherry picked from commit 3e08389a0f)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
f2209932a6 lorax-composer: Add services support to blueprints
Add support for enabling and disabling systemd services in the
blueprint. It works like this:

    [customizations.services]
    enabled = ["sshd", "cockpit.socket", "httpd"]
    disabled = ["postfix", "telnetd"]

They are *added* to any existing settings in the kickstart templates.

(cherry picked from commit 1111aee92d)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
780f00d81a lorax-composer: Add firewall support to blueprints
You can now open ports in the firewall, using port numbers or service
names:

    [customizations.firewall]
    ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]

Or enable/disable services registered with firewalld:

     [customizations.firewall.services]
     enabled = ["ftp", "ntp", "dhcp"]
     disabled = ["telnet"]

If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.

(cherry picked from commit 4d35668ab5)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
a9c5581aa9 lorax-composer: Add locale support to blueprints
You can now set the keyboard layout and language. Eg.

[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"

Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'

Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.

(cherry picked from commit e5a8700bdf)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Brian C. Lane
956ebfc77c lorax-composer: Add timezone support to blueprint
For example:

[customizations.timezone]
timezone = "US/Samoa"
ntpservers = ["0.pool.ntp.org"]

Also includes tests.

This removes the timezone kickstart command from all of the templates
except for google.ks which needs to set it's own ntp servers and timezone.

If timezone isn't included in the blueprint, and it is not already in a
template, it will be set to 'timezone UTC' by default.

If timezone is set in a template it is left as-is, under the assumption
that the image type requires it to boot correctly.

(cherry picked from commit 9bdbb29662)

Related: rhbz#1718473
2019-11-18 12:36:18 +02:00
Alexander Todorov
e34201c0bd tests: Document Azure setup 2019-10-21 20:51:20 +03:00
Jiri Kortus
22acf067cd tests: Add kickstart tar installation test
Related: rhbz#1733504
2019-10-04 12:09:12 +03:00
Jiri Kortus
64f00d94b8 tests: add option to disable kernel command line parameters check
Related: rhbz#1733504
2019-10-04 12:09:12 +03:00
Brian C. Lane
c2c6bfa458 tests: Use a loop to wait for VM and sshd to start
Related: rhbz#1733504
2019-10-04 12:09:12 +03:00
Brian C. Lane
9d7fcc8642 tests: Drop sort from compose types test
The output from the API is already sorted, and 'sort' changes this
ordering in some situations.

Related: rhbz#1749802
2019-09-13 14:05:42 +03:00
Alexander Todorov
80c69f0621 New test: assert toml files in git workspace
export BLUEPRINTS_DIR for use in tests

Depending on the way the tests are run the directory may be a temporary
dir, or it may be the standard /var/lib/lorax/... path.

Related: rhbz#1749802
2019-09-13 14:05:42 +03:00
Alexander Todorov
22de8ed603 Ignore Cockpit CI files when linting
- these files are executed under Python 3 while the linter
  is Python 2 for RHEL 7 so we just ingore them
- also reverts the changes introduced to make these files Py2
  compatible, making them the same as on master branch

Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Alexander Todorov
b38a358ea3 Use optional repository in Live ISO test
live-iso image type needs grub2-efi-*-cdboot from optional.

Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Alexander Todorov
77649cbf76 Install test dependencies inside virtualenv
Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Alexander Todorov
89e56aa3b4 Install python2-pip in the test VM and disable EPEL repo
so we don't use EPEL when actually building the images

Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Alexander Todorov
2136d88cd6 Use qemu-kvm in tests instead of qemu-system
Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Alexander Todorov
3d27053605 Don't hard-code the path to toml-compare
Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Alexander Todorov
d97ff118d9 Prevent ssh asking for password when testing on interactive terminal
Related: rhbz#1704209
2019-09-13 11:27:12 +03:00
Alexander Todorov
9bce583fc2 Fix typo from test backport
Related: rhbz#1698366
2019-09-13 11:27:12 +03:00
Jiri Kortus
b0b603dc2e Use passwd --status for locked root account check
Related: rhbz#1687595
2019-07-10 09:17:58 +02:00
Alexander Todorov
8734f662e9 Backport changes for Cockpit CI
Related: rhbz#1698366
2019-06-25 11:57:33 +03:00
Lars Karlitski
fd2e1738ab test_compose_tar: Fix docker test
The docker phase always failed because `-ti` was passed even though the
the output was not a terminal. Moreover docker service isn't running by
default on RHEL-7, so it's necessary to start it first explicitly.

Related: rhbz#1720224
2019-06-24 10:33:37 +03:00
Brian C. Lane
cdfdb651f7 tests: kill the qemu process name used to start it
Related: rhbz#1710877
2019-06-24 10:33:37 +03:00
Alexander Todorov
63c50c08eb [tests] Handle blueprints in setup_tests/teardown_tests correctly
It's necessary to make sure the blueprints directory doesn't contain
the git/ directory before the tests are run, so that we can just simply
modify the blueprint files without using blueprints push.

Related: rhbz#1698366
2019-06-12 21:50:08 +03:00
Lars Karlitski
7c0d2577a8 tests: Set BLUEPRINTS_DIR in all cases
`setup_tests()` expected BLUEPRINTS_DIR to be set, but it wasn't when
running in automated mode (with $CLI set).

Fix this and move share and blueprint dirs to function arguments.

Related: rhbz#1698366
2019-06-12 21:50:08 +03:00
Alexander Todorov
9b5554aa1e tests: Change the way how we remove pyOpenSSL
the version coming from RPM is older and prevents installing
ansible[azure] via pip. OTOH removing with yum also removes
lorax-composer and breaks tests when running against the RPM
instead of git checkout (which will be the default with Cockpit CI).

Related: rhbz#1715003
2019-06-12 11:21:56 +03:00
Jiri Kortus
94892e56dd Use a less strict regex for disabled root account check
Related: rhbz#1687595
2019-05-13 15:31:27 +03:00
Jiri Kortus
481f33b898 Add test for passing custom option on kernel command line
Related: rhbz#1688335
2019-05-13 15:31:27 +03:00
Jiri Kortus
0980ddfc54 Use verify_image function as a helper for generic tests
Related: rhbz#1704209
2019-05-13 15:31:27 +03:00
Alexander Todorov
1bf7708830 Change [[modules]] to [[packages]] in tests
> bcl: this should really be [[packages]] since [[modules]] is
> going to change to actual modules at some point

Related: rhbz#1698366
2019-05-07 19:19:27 +03:00
Alexander Todorov
83707bfa06 Add new test to verify compose paths exist
because they are mentioned in the official documentation

Related: rhbz#1698366
2019-05-07 19:19:27 +03:00
Alexander Todorov
ff7d7c2d49 Add new sanity tests for blueprints
- verify SemVer .patch number will be automatically updated when
  we push the blueprint a second time without changing version
- verify show displays the content in TOML format and it matches
  what is on disk. Because of that also start with empty packages
  and groups fields in the initial toml. If they are missing they
  will be added automatically by lorax-composer and this simplifies
  the test
- verify delete works

Related: rhbz#1698366
2019-05-07 19:19:27 +03:00
Brian C. Lane
41d822f754 tests: Add a test for using [[customizations]] with [customizations.kernel]
Make sure this is really fixed by running the template tests with both
variations. NOTE: [customizations] is the correct form. An early
documentation bug led to both being accepted.

Related: rhbz#1688335
2019-04-29 16:01:30 -07:00
Brian C. Lane
0b8a5ac87c lorax-composer: pass customization.kernel append to extra_boot_args
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.

Also adds tests to make sure it is written to config.toml in the build
metadata.

Resolves: rhbz#1688335
2019-04-29 10:41:57 -07:00
Brian C. Lane
9cebd1ddaf lorax-composer: Add the ability to append to the kernel command-line
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:

[customizations.kernel]
append = "nosmt=force"

This will be appended to the kickstart's bootloader --append argument.

Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.

Related: rhbz#1688335
2019-04-29 10:41:57 -07:00
Jiri Kortus
5b45593f3c Add test for starting compose with deleted blueprint
Related: rhbz#1683442
2019-04-24 14:14:40 +03:00
Brian C. Lane
a81964a603 lorax-composer: Return UnknownBlueprint errors when using deleted blueprints
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.

One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.

This also adds tests for the various operations.

(cherry picked from commit d32f477e0b)

Resolves: rhbz#1683442
2019-04-22 10:17:25 -07:00
Brian C. Lane
2eb4437159 lorax-composer: Delete workspace copy when deleting blueprint
Also extends the blueprint delete test to also check the workspace.

(cherry picked from commit 26bd2c1378)

Related: rhbz#1683442
2019-04-22 10:16:40 -07:00
Jan Stodola
73de195334 Use existing storage account
To avoid creating a new storage account for every new VM.

Related: rhbz#1673012
2019-04-22 15:57:07 +03:00
Jan Stodola
fabf428f00 Record date/time of VM creation
Related: rhbz#1673012
2019-04-22 15:57:07 +03:00
Chris Roberts
2ebf60f28d Update datastore for VMware testing
Cluster and Network changed due to new hardware in RDU2 DC

Related: rhbz#1656105
2019-04-18 10:35:53 +03:00
Jiri Kortus
6fb94326e7 Fixes for locked root account test
Fixes related to a few issues in the locked root account test I somehow
managed to overlook in the initial commit.

Related: rhbz#1687595
2019-04-09 18:54:03 +03:00