Commit Graph

212 Commits

Author SHA1 Message Date
Brian C. Lane abe7df34fc Switch to new toml library
The previous library, pytoml from https://github.com/avakar/pytoml is no
longer supported. So this adds a compatibility layer on top of the
suggested replacement, toml from https://github.com/uiri/toml
2019-06-18 14:01:15 -07:00
Brian C. Lane 932ff5812c composer-cli: Update diff support for customizations and repos.git
composer-cli will now output information about changes to customizations
entries and the repos.git entries.
2019-06-13 14:16:48 -07:00
Brian C. Lane 9011a564e8 Add support for customizations and repos.git to /blueprints/diff/
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.
2019-06-13 14:16:48 -07:00
Brian C. Lane 316257fbc0 tests: Update custom-base with customizations 2019-06-13 14:16:48 -07:00
Alexander Todorov f61dfcc2c7 Increase retry count/sleep times when waiting for lorax to start
the biggest slow down is fetching data for many repositories
over a slow network. The previous retry count and sleep times
sometimes are not enough on Fedora.
2019-06-10 13:26:11 +03:00
Alexander Todorov 688f70eefa Revert "remove the check for qemu-kvm"
Partially reverts commit 303a69bcbd.
2019-06-10 13:26:11 +03:00
Alexander Todorov 22b0eb28bc Revert "remove the check for /usr/bin/docker in the setup phase"
this is still helpful when executing these test scripts manually
outside Cockpit CI or in Jenkins (for gating).

Partially reverts commit 5f5a2d5337.
2019-06-10 13:26:11 +03:00
Alexander Todorov b725eb141a [tests] Define unbound variables in test scripts 2019-06-06 14:56:40 +02:00
Alexander Todorov 649a9e2239 [tests] Handle blueprints in setup_tests/teardown_tests correctly
It's necessary to make sure the blueprints directory doesn't contain
the git/ directory before the tests are run, so that we can just simply
modify the blueprint files without using blueprints push.

Related: rhbz#1714298
2019-06-06 14:56:40 +02:00
Alexander Todorov 16fd1a2340 [tests] grep|cut for IP address in a more robust way
because sometimes these VMs may get an IPv6 address assigned
and we don't want to cut only the first part of it
2019-06-06 14:56:40 +02:00
Lars Karlitski 42c7c0691c Monkey-patch beakerlib to fail on first assert
Beakerlib upstream can't do this yet, but might at some point:

https://github.com/beakerlib/beakerlib/issues/42

This is only enabled in combination with the `--sit` option of the
`test/check-*` scripts. It leaves the system in exacly the state it was
in when an assertion failed. Finishing the test run would run cleanup as
well (such as deleting created images). It also takes longer.
2019-06-04 20:12:54 +02:00
Lars Karlitski a4dcc34396 test_cli.sh: Return beakerlib's exit code
Read the return status directly from beakerlib's TestResults file. This
is more robust when running the same test multiple times.
2019-06-04 20:12:54 +02:00
Lars Karlitski 4bd03cb8f6 Don't send CORS headers
These are meant for web applications that are accessed by browsers, not
REST APIs.
2019-06-04 20:10:03 +02:00
Lars Karlitski a164ed3d7d tests: Set BLUEPRINTS_DIR in all cases
`setup_tests()` expected BLUEPRINTS_DIR to be set, but it wasn't when
running in automated mode (with $CLI set).

Fix this and move share and blueprint dirs to function arguments.
2019-06-04 14:07:36 +03:00
Lars Karlitski ce3a277100 tests: Fail on script errors
Fixes #727
2019-06-04 14:07:36 +03:00
Lars Karlitski 8ed910b29a composer: Set up a custom HTTP error handler
Override flask's default error handler, because that return html. Return
JSON instead with the usual { "status": false, "errors": [ ... ] }
pattern.
2019-06-04 11:50:25 +03:00
Alexander Todorov b71e8f74d8 Configure $PACKAGE for beakerlib reports
this makes it more clear that we are testing the installed RPM
instead of sources
2019-05-31 12:57:23 +02:00
Alexander Todorov b67ce5379c Use cloud credentials during test if they exist 2019-05-31 12:57:23 +02:00
Jan Stodola ed8a21a627 Use ansible instead of awscli
Not all parts of the script has been switched from awscli to ansible yet,
because the ansible aws modules do not support importing s3 object as snapshots.
(https://github.com/ansible/ansible/issues/53453)
Workaround using the image_location parameter of the ec2_ami ansible module
would mean adding extra code for generating AMI manifest with pre-signed
URLs.
2019-05-24 12:44:30 +03:00
Lars Karlitski 1e5a5921e3 tests: Source lib.sh from the right directory
When testing in one of the VMs, tests are in /tests. Source the lib from
a relative directory instead.
2019-05-24 00:23:21 +02:00
Brian C. Lane 6120f74c73 Revert "Add rpmfluff temporarily"
This reverts commit 6ea1c45734.

python-rpmfluff-0.5.7.1-1 is now available.
2019-05-23 14:57:46 -07:00
Brian C. Lane 795cc3400c tests: Update tmux version to 2.9a 2019-05-23 14:57:46 -07:00
Lars Karlitski c261cec33d tests: Fail immediately when image build fails
We were checking for composer's FINISHED status only, which meant that
when a compose failed, the test ran until it timed out.

Check for failed as well. Also, always time out after 30 minutes.
2019-05-22 08:24:49 +02:00
Lars Karlitski bdffaa02f4 tests: Re-enable kvm
In 303a69bc, we stopped using kvm, because it doesn't exist on all test
builders. Enable it again on those that have it.
2019-05-21 22:45:32 +02:00
Lars Karlitski 303a69bcbd tests: Don't depend on kvm for tar and qcow2 tests
Some test runners don't have nested virtualization enabled. Because
these checks are only checking that a boot works, kvm doesn't give us
that much. Disable for now.

Also remove the check for qemu-kvm. It doesn't abort the test
prematurely anyway.
2019-05-20 20:24:01 +02:00
Lars Karlitski 77acc8972b test_compose_tar: Work around selinux policy change
A compose can change the hosts' policy, which can lead to docker
crashing if the container-selinux policy is not included. Add a
workaround and bug link.
2019-05-20 20:24:01 +02:00
Lars Karlitski ca171424eb test_compose_tar: Be less verbose
A huge chunk of the output log is tar's file list. Remove that to make
the log more readable.
2019-05-20 20:24:01 +02:00
Lars Karlitski 5f5a2d5337 test_compose_tar: Fix docker test
The docker phase always failed because `-ti` was passed even though the
the output was not a terminal.

Also remove the check for /usr/bin/docker in the setup phase. It didn't
test that the daemon was running. More importantly, it didn't abort the
test anwyay (and there doesn't seem to be a good way to do this in
beakerlib).
2019-05-20 20:24:01 +02:00
Lars Karlitski 1ba2e6fd4b tests: Extract images to /var/tmp, not /tmp
Test VMs don't have enough memory (/tmp is a tmpfs).
2019-05-20 20:24:01 +02:00
Lars Karlitski 250f49f78d Use Cockpit's test images and infrastructure
Allows to run the tests on multiple operating systems and on the
infrastructure that the Cockpit team maintains.

`make vm` downloads one of Cockpit's test images (override which one
with TEST_OS) and installs rpms build from the local checkout of lorax.
The resulting image is placed in `test/images/$TEST_OS`.

TEST_OS can be set to any of Cockpit's supported images (default:
fedora-30).

Run `make check-vm` to run the CLI checks in the VM. The bulk of the
work is done in `test/check-cli`, which uses Cockpit's `bots` library to
start the VM and run the script in it.

Also included is a `test/run` script, which is the entrypoint for
Cockpit's test infrastructure.
2019-05-20 20:24:01 +02:00
Lars Karlitski aa9c2e13b1 pylint: Remove unused false positive 2019-05-20 20:24:01 +02:00
Jiri Kortus 123b5c269d Add test for passing custom option on kernel command line 2019-05-07 12:36:32 +03:00
Jiri Kortus 596cba2255 Use verify_image function as a helper for generic tests 2019-05-07 12:36:32 +03:00
Brian C. Lane fa4bda8317 tests: Update openssh-server to v8.* 2019-05-02 17:02:30 -07:00
Brian C. Lane 3e08389a0f Change customizations.firewall to append items instead of replace
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.

Also updated the docs, added a new test for it, and sorted the output
for consistency.
2019-05-02 16:21:36 -07:00
Brian C. Lane 1111aee92d lorax-composer: Add services support to blueprints
Add support for enabling and disabling systemd services in the
blueprint. It works like this:

    [customizations.services]
    enabled = ["sshd", "cockpit.socket", "httpd"]
    disabled = ["postfix", "telnetd"]

They are *added* to any existing settings in the kickstart templates.
2019-05-02 16:21:36 -07:00
Brian C. Lane 6ea1c45734 Add rpmfluff temporarily
There is a problem with rpmfluff and the current version of rpm in
rawhide.

Changes are upstream, but no new build has been done yet.
2019-05-02 16:21:36 -07:00
Brian C. Lane 4d35668ab5 lorax-composer: Add firewall support to blueprints
You can now open ports in the firewall, using port numbers or service
names:

    [customizations.firewall]
    ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]

Or enable/disable services registered with firewalld:

     [customizations.firewall.services]
     enabled = ["ftp", "ntp", "dhcp"]
     disabled = ["telnet"]

If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.
2019-05-02 16:21:36 -07:00
Brian C. Lane e5a8700bdf lorax-composer: Add locale support to blueprints
You can now set the keyboard layout and language. Eg.

[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"

Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'

Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.
2019-05-02 16:21:36 -07:00
Brian C. Lane 9bdbb29662 lorax-composer: Add timezone support to blueprint
For example:

[customizations.timezone]
timezone = "US/Samoa"
ntpservers = ["0.pool.ntp.org"]

Also includes tests.

This removes the timezone kickstart command from all of the templates
except for google.ks which needs to set it's own ntp servers and timezone.

If timezone isn't included in the blueprint, and it is not already in a
template, it will be set to 'timezone UTC' by default.

If timezone is set in a template it is left as-is, under the assumption
that the image type requires it to boot correctly.
2019-05-02 16:21:36 -07:00
Jiri Kortus 21e141a84c Add test for starting compose with deleted blueprint 2019-04-23 11:37:03 +03:00
Chris Roberts 0fa5af2e30 Update VMware info for VMware testing 2019-04-18 10:32:33 +03:00
Brian C. Lane 65b8a2be68 tests: Cleanup on failure of in_tempdir
Otherwise other tests will also fail when they try to run from the wrong
directory.
2019-04-15 14:06:48 -07:00
Alexander Todorov 933fad9ded Change [[modules]] to [[packages]] in tests
> bcl: this should really be [[packages]] since [[modules]] is
> going to change to actual modules at some point
2019-04-15 21:53:24 +03:00
Alexander Todorov e45e645fd0 Add new test to verify compose paths exist
because they are mentioned in the official documentation
2019-04-15 21:53:24 +03:00
Alexander Todorov 7cbb324ae5 Add new sanity tests for blueprints
- verify SemVer .patch number will be automatically updated when
  we push the blueprint a second time without changing version
- verify show displays the content in TOML format and it matches
  what is on disk. Because of that also start with empty packages
  and groups fields in the initial toml. If they are missing they
  will be added automatically by lorax-composer and this simplifies
  the test
- verify delete works
2019-04-15 21:53:24 +03:00
Jiri Kortus 209bdd6911 Fixes for locked root account test
Fixes related to a few issues in the locked root account test I somehow
managed to overlook in the initial commit.
2019-04-09 18:53:15 +03:00
Jan Stodola b57de93468 Fix keeping files on Amazon s3 2019-04-04 15:07:46 +03:00
Jan Stodola db7b1e4fcd Allow to keep objects in AWS
Instances, Volumes, Snapshots, AMIs and s3 objects with the "keep_me" tag will
not be deleted automatically even if they are older then the specified time limit.
2019-04-03 23:01:15 +03:00
David Shea 44e14176bb Add a compose type for alibaba.
This compose type creates a partitioned disk as a qcow2 file, but with
only one partition instead of using a separate /boot.
2019-04-03 13:05:31 -04:00