Commit Graph

121 Commits

Author SHA1 Message Date
Jan Stodola 6680e2ffec Add script for removing old artifacts from OpenStack 2019-01-24 14:12:27 +02:00
Jan Stodola 677c858f60 Add script for removing old artifacts from AWS 2019-01-21 10:54:59 +02:00
Brian C. Lane 2950f2641b Remove unneeded else from for/else loop. It confuses pylint 2019-01-08 11:38:26 -08:00
David Shea 35ab6a1336 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.
2019-01-08 10:21:07 -05:00
Dan Horák 3817671587 drop ppc/ppc64 from tests 2019-01-07 13:52:46 -08:00
Alexander Todorov 883a959442 tests: use the first IP address if more than 1 retruned from OpenStack 2018-12-08 02:31:06 +02:00
Alexander Todorov 6563a33f63 tests: remove a debugging command 2018-12-08 02:31:06 +02:00
Brian C. Lane d18934775c lorax-composer: Handle packages with multiple builds
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.

This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.

Resolves: rhbz#1656642
2018-12-06 15:48:58 -08:00
Brian C. Lane 4dd9004d13 lorax-composer: Check the queue and results at startup
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
 * Removes broken symlinks from queue/new and queue/run
 * Removes symlinks from run and sets the build to FAILED
 * Sets builds w/o a STATUS to FAILED
 * Sets builds with STATUS of RUNNING to FAILED
 * Creates missing queue/new symlinks to results with STATUS of WAITING

So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.

Resolves: rhbz#1647985
2018-12-06 15:13:28 -08:00
Alexander Todorov 642b909d24 Teach OpenStack test to distinguish between RHEL and Fedora 2018-12-06 22:15:49 +02:00
Alexander Todorov 543a4d88d9 Use full path for Azure playbook as well 2018-12-06 22:15:49 +02:00
Alexander Todorov 16260c1569 Use a temporary dir for ssh keys during testing
resolves path problems with sudo, we already have the same change
in the AWS test script.
2018-12-06 22:15:49 +02:00
Alexander Todorov 59050df710 Update V_DATASTORE b/c defaults appear to have been changed 2018-12-06 22:15:49 +02:00
Alexander Todorov e0c8cdc5a8 Clone pyvmomi samples in the correct directory 2018-12-06 22:15:49 +02:00
Alexander Todorov 8f430be2a6 Use full path when pushing toml files during testing 2018-12-06 22:15:49 +02:00
Alexander Todorov 4b4f4bd0e0 Implicitly specify ssh key directory/files for testing
this is to work around identity/sudo problems
2018-12-03 11:36:29 +02:00
Alexander Todorov ed63611cb7 [test] Clean up containers.json 2018-12-03 11:36:29 +02:00
Alexander Todorov ab048947cb Teach AWS test to distinguish between RHEL and Fedora 2018-12-01 18:06:20 +02:00
Alexander Todorov cd6f4fdb67 Drop minor version from php package in blueprint
there's 7.2 in Fedora 29 (which we use in Jenkins) and 7.3 in
Rawhide which causes test jobs to fail with unsolved dependencies.
Alternatively we can switch to another recipe for cloud image tests.
2018-11-29 09:46:06 +02:00
Alexander Todorov b88466fd74 Use a temporary shared dir when testing
otherwise composer-cli is unable to glob() the kickstart
files and we're left without supported compose types. Seen
during AWS testing for example.

Helps with running some of the tests via sudo b/c this is
what Jenkins requires.
2018-11-29 09:46:06 +02:00
Alexander Todorov c9d706a382 Copy blueprints used for testing to temporary directory
If trying to execute test_cli.sh inside a git checkout
we are going to get the following exception:

Traceback (most recent call last):
  File "./src/sbin/lorax-composer", line 251, in <module>
    repo = open_or_create_repo(server.config["REPO_DIR"])
  File "/home/jenkins/lorax/src/pylorax/api/recipes.py", line 306, in open_or_create_repo
gi.repository.GLib.Error: ggit-error: failed to stat '/home/jenkins/lorax/tests/pylorax/blueprints': Permission denied (-1)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/popen_fork.py", line 54, in _send_signal
    os.kill(self.pid, sig)

From what I can tell open_or_create_repo() is trying to initialize
a git repository inside the blueprints directory which fails when
we have an active git checkout.

This doesn't happen when we run the tests in Travis CI because
rsync excludes .git/ inside the Docker container.
2018-11-29 09:46:06 +02:00
Alexander Todorov 366ae55abe Add make targets for Jenkins
these targets help hooking up things in Jenkins and enable us to
perform build & deploy tests for cloud images.

NOTE: use sudo -E to preserve the environment
2018-11-29 09:46:06 +02:00
Alexander Todorov af2ae790ce Teach test_cli.sh to execute test scripts via arguments
this will be used to invoke scripts that build/push cloud images
without having to duplicate the setup/teardown/report parts!
2018-11-26 14:22:43 +02:00
Alexander Todorov 13fbd0861b new test: build an image and deploy it on Azure 2018-11-21 11:39:15 +02:00
Alexander Todorov d6dcb38a78 Fix typo in comment 2018-11-21 11:39:15 +02:00
Alexander Todorov 6ae78fbd47 For OpenStack build image with rng-tools installed
appears to be a decent workaround for slow boot likely caused
by lack of entropy for ssh-keygen, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1639326
2018-11-13 12:09:07 +02:00
Brian C. Lane fb4e6f2588 Add tests for partitioned disk images
The setup for this is a bit complex, so it really ends up testing things
twice.
2018-11-12 11:13:37 -08:00
Brian C. Lane 063a1770e1 Add tests for pylorax.imgutils
Some of these can only run as root on a real system with access to loop
devices. They are skipped when running in a container.
2018-11-12 11:13:37 -08:00
Brian C. Lane bab4b20d0d Add tests to test_creator.py
Add a /.in-container file to the container root so that tests requiring root
and loop device support will be skipped when running in a container.
2018-11-12 11:13:37 -08:00
Brian C. Lane 134a333d92 Add some tests for creator.py
This is complicated by the fact that much of this module requires mount.
So for now just test the things that don't need mount.
2018-11-12 11:13:37 -08:00
Brian C. Lane fb87db0e1e tests: Add executils test 2018-11-12 11:13:37 -08:00
Brian C. Lane cc69478249 tests: Add sysutils test 2018-11-12 11:13:37 -08:00
Brian C. Lane 2e36e941ce tests: Add discinfo test 2018-11-12 11:13:37 -08:00
Brian C. Lane e8fd8a496b tests: Add treeinfo test 2018-11-12 11:13:37 -08:00
Alexander Todorov 6a42570eea new test: build and deploy an image in OpenStack 2018-11-09 10:25:26 +02:00
Alexander Todorov 6005f98123 Fix typos in VM_NAME and cleanup command 2018-11-09 10:25:26 +02:00
Alexander Todorov 9906ad6da3 new test: build and deploy images on vSphere 2018-11-08 10:33:20 +02:00
Alexander Todorov efaf2bf793 new test: build and deploy images on AWS 2018-10-30 00:28:18 +02:00
Alexander Todorov 7d2b9d2d30 Disable execution of new tests which need Docker privileged mode 2018-10-29 23:15:06 +02:00
Alexander Todorov 2eb62014db New tests: build ext4-filesystem and partitioned-disk composes
for the moment we just make sure these two can be built without
visible errors.
2018-10-29 23:15:06 +02:00
Alexander Todorov 453b1c1236 Update tmux version in tests 2018-10-29 23:15:06 +02:00
Brian C. Lane a4783ba29f Add tests for ltmpl.py
This covers things like installing globbed package names from multiple
repos, pinned package versions, and ltmpl functions

Related: rhbz#1548586
2018-10-29 13:55:19 -07:00
Brian C. Lane df70e3d677 Update php version to 7.3.* 2018-10-24 10:22:40 -07:00
Brian C. Lane 27aff75aa3 Update the projects tests to use DNF Repo object
Stop using fake dnf object, use the real thing. This will help catch
problems with dnf returning unexpected types like VectorString.
2018-10-24 10:22:40 -07:00
Brian C. Lane 75644689de Update the tests for new make_dnf_dir arguments.
Use the uid and gid that the test is running as instead of hard-coding
0.
2018-10-12 11:59:32 -07:00
Brian C. Lane 65b769984b Change make_dnf_dirs to be run as root
It needs to be root in order to set the ownership and permissions on the
directories that are under /var/lib/lorax/composer/

Refactor the directory creation into a utility function, and use a umask
of 0o006 to ensure that the parent directories created do not have o+rw
set on them (makedirs behavior is different between Python 3.6 and 3.7
so umask of 0 doesn't work consistently).
2018-10-12 11:59:32 -07:00
Brian C. Lane 98f8b23129 Add an openstack image type
This is a qcow2 image with cloud-init in the template.
2018-10-09 10:17:14 -07:00
Brian C. Lane 2a7a323ddf Update cli tests to use composer-cli name 2018-10-08 09:48:34 -07:00
Brian C. Lane 6da3079349 Report an error if the blueprint doesn't exist
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.

composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
2018-10-03 16:41:49 -07:00
Brian C. Lane e7bfab8b4a Write a rootpw line if no root customizations in the blueprint
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'

Also adds tests for this.

Resolves: rhbz#1626122
2018-10-03 08:30:31 -07:00