Commit Graph

142 Commits

Author SHA1 Message Date
Alexander Todorov
3f000b1fe3 New test: Build qcow2 compose and test it with QEMU-KVM
(cherry picked from commit b936dfb198)
2019-05-03 13:55:41 -07:00
Brian C. Lane
2861bdb95e lorax-composer: pass customization.kernel append to extra_boot_args
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.

Also adds tests to make sure it is written to config.toml in the build
metadata.

(cherry picked from commit 5dea308080)
2019-04-05 15:41:44 -07:00
Brian C. Lane
c5f4dfe113 lorax-composer: Add the ability to append to the kernel command-line
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:

[customizations.kernel]
append = "nosmt=force"

This will be appended to the kickstart's bootloader --append argument.

Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.

(cherry picked from commit 59464286f9)
2019-04-05 15:41:38 -07:00
Brian C. Lane
d513747384 tests: Make it easier to update version globs
Use constants so we won't have to edit a dozen places in the test when
the package versions are bumped.

Also switch to using Fedora 31 GPG key now that it has branched for
Fedora 30.
2019-03-15 12:20:02 -07:00
Brian C. Lane
fc9ccb51ee Fix pylint problems with vmware_list_vms.py 2019-02-25 14:38:37 -08:00
Brian C. Lane
3b8de2a233 Move the package requirements for live-iso setup out of the template
In order to support iso creation on multiple arches with the templates
we need to be able to select different packages based on arch.
lorax-composer uses the arch-specific Lorax templates in order to
generate the output iso so this patch:

1. Creates a new template and type to parse it, live-install.tmpl
   which contains only installpkg commands and #if clauses for arch
2. Removes bootloader related packages from the live-iso.ks
3. Remove dracut-config-rescue exclusion because it can cause problems
   with some blueprints.
4. Switch logo requirement to system-logos which is satisfied by
   generic-logos or fedora-logos. This prevents conflicts when a blueprint
   installs fedora-release-workstation.

So in the future, if x86.tmpl, etc. need a new package to support
creating the iso it should be added to the correct section in
./share/live/live-install.tmpl
2019-02-25 13:49:12 -08:00
Brian C. Lane
b47554d716 livemedia-creator: Add support for reqpart kickstart command
reqpart can be used to make kickstarts more platform agnostic, creating
needed partitions without lmc having to keep track of the arch-specific
needs. eg. ppc64 needs prepboot and /boot

This increases the size of the disk based on whether reqpart or
reqpart --add-boot is in the kickstart.

Note that this is only valid for partitioned disk output types, not
for filesystem images or live iso output.
2019-02-25 13:49:12 -08:00
Jan Stodola
ce110afd22 Add script for removing old artifacts from VMware 2019-02-25 11:40:03 +02:00
Brian C. Lane
28bd68793b tests: Fix makeFakeRPM calls 2019-02-21 10:10:11 -08:00
Brian C. Lane
2272b2b189 Update bash to 5.0.*
so that the tests will pass again on rawhide.
2019-02-21 08:55:23 -08:00
Alexander Todorov
ea78cce882
New test: Verify tar images with Docker and systemd-nspawn
- on some arches (also Fedora x86_64) systemd-nspawn may not be
  available
- delete composes from other tests in rlPhaseStartCleanup because
  we're seeing the tar compose kind of hanging in Jenkins and that
  test script is executed last so the slave may be running out of
  disk space. Be a good citizen and clean up after the previous
  tests.
2019-02-18 19:14:56 +02:00
Alexander Todorov
c95d7084a6 Update OpenStack flavor and network settings in tests
b/c we've migrated to Upshift we must use different instance type,
specify the desired network to connect to and update how we get
the ip address of the launched VM.
2019-02-16 01:19:43 +02:00
Alexander Todorov
5dc895fbee Keep OpenStack VMs with Tag keep_me
so we can have some systems for debugging
2019-02-08 22:02:41 +01:00
Jiri Kortus
57be7313c4 Make sure compose build tests run with SELinux in enforcing mode 2019-02-06 18:54:16 +01:00
Jan Stodola
f26fcba902 Add script for removing old artifacts from Azure 2019-02-06 09:38:18 +01:00
Jan Stodola
8a8802aab7 Use existing storage account
To avoid creating a new storage account for every new VM.
2019-02-05 21:03:57 +01:00
Jan Stodola
e918ff807c Record date/time of VM creation 2019-02-05 21:03:57 +01:00
Brian C. Lane
98482e444d Remove duplicate repositories from the sources list
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.

So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.
2019-01-30 08:30:40 -08:00
Brian C. Lane
3676cb65bb Clarify the ks repo only error message
This also moves the run_creator kickstart checks into check_kickstart
so that tests may be added.

This will close #164
2019-01-29 14:03:11 -08:00
Jan Stodola
cc6fdb2fac Export OS_PROJECT_NAME variable in openstack scripts
The OS_PROJECT_NAME (or OS_TENANT_NAME) environment variable needs to be defined.
Use the OS_PROJECT_NAME, since it is recommended in the documentation instead of
the older OS_TENANT_NAME.
2019-01-29 12:51:30 +02:00
Jan Stodola
961ea18145 Expand parameters as separate words 2019-01-25 21:34:25 +02:00
Jan Stodola
6680e2ffec Add script for removing old artifacts from OpenStack 2019-01-24 14:12:27 +02:00
Jan Stodola
677c858f60 Add script for removing old artifacts from AWS 2019-01-21 10:54:59 +02:00
Brian C. Lane
2950f2641b Remove unneeded else from for/else loop. It confuses pylint 2019-01-08 11:38:26 -08:00
David Shea
35ab6a1336 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.
2019-01-08 10:21:07 -05:00
Dan Horák
3817671587 drop ppc/ppc64 from tests 2019-01-07 13:52:46 -08:00
Alexander Todorov
883a959442 tests: use the first IP address if more than 1 retruned from OpenStack 2018-12-08 02:31:06 +02:00
Alexander Todorov
6563a33f63 tests: remove a debugging command 2018-12-08 02:31:06 +02:00
Brian C. Lane
d18934775c lorax-composer: Handle packages with multiple builds
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.

This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.

Resolves: rhbz#1656642
2018-12-06 15:48:58 -08:00
Brian C. Lane
4dd9004d13 lorax-composer: Check the queue and results at startup
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
 * Removes broken symlinks from queue/new and queue/run
 * Removes symlinks from run and sets the build to FAILED
 * Sets builds w/o a STATUS to FAILED
 * Sets builds with STATUS of RUNNING to FAILED
 * Creates missing queue/new symlinks to results with STATUS of WAITING

So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.

Resolves: rhbz#1647985
2018-12-06 15:13:28 -08:00
Alexander Todorov
642b909d24 Teach OpenStack test to distinguish between RHEL and Fedora 2018-12-06 22:15:49 +02:00
Alexander Todorov
543a4d88d9 Use full path for Azure playbook as well 2018-12-06 22:15:49 +02:00
Alexander Todorov
16260c1569 Use a temporary dir for ssh keys during testing
resolves path problems with sudo, we already have the same change
in the AWS test script.
2018-12-06 22:15:49 +02:00
Alexander Todorov
59050df710 Update V_DATASTORE b/c defaults appear to have been changed 2018-12-06 22:15:49 +02:00
Alexander Todorov
e0c8cdc5a8 Clone pyvmomi samples in the correct directory 2018-12-06 22:15:49 +02:00
Alexander Todorov
8f430be2a6 Use full path when pushing toml files during testing 2018-12-06 22:15:49 +02:00
Alexander Todorov
4b4f4bd0e0 Implicitly specify ssh key directory/files for testing
this is to work around identity/sudo problems
2018-12-03 11:36:29 +02:00
Alexander Todorov
ed63611cb7 [test] Clean up containers.json 2018-12-03 11:36:29 +02:00
Alexander Todorov
ab048947cb Teach AWS test to distinguish between RHEL and Fedora 2018-12-01 18:06:20 +02:00
Alexander Todorov
cd6f4fdb67 Drop minor version from php package in blueprint
there's 7.2 in Fedora 29 (which we use in Jenkins) and 7.3 in
Rawhide which causes test jobs to fail with unsolved dependencies.
Alternatively we can switch to another recipe for cloud image tests.
2018-11-29 09:46:06 +02:00
Alexander Todorov
b88466fd74 Use a temporary shared dir when testing
otherwise composer-cli is unable to glob() the kickstart
files and we're left without supported compose types. Seen
during AWS testing for example.

Helps with running some of the tests via sudo b/c this is
what Jenkins requires.
2018-11-29 09:46:06 +02:00
Alexander Todorov
c9d706a382 Copy blueprints used for testing to temporary directory
If trying to execute test_cli.sh inside a git checkout
we are going to get the following exception:

Traceback (most recent call last):
  File "./src/sbin/lorax-composer", line 251, in <module>
    repo = open_or_create_repo(server.config["REPO_DIR"])
  File "/home/jenkins/lorax/src/pylorax/api/recipes.py", line 306, in open_or_create_repo
gi.repository.GLib.Error: ggit-error: failed to stat '/home/jenkins/lorax/tests/pylorax/blueprints': Permission denied (-1)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/popen_fork.py", line 54, in _send_signal
    os.kill(self.pid, sig)

From what I can tell open_or_create_repo() is trying to initialize
a git repository inside the blueprints directory which fails when
we have an active git checkout.

This doesn't happen when we run the tests in Travis CI because
rsync excludes .git/ inside the Docker container.
2018-11-29 09:46:06 +02:00
Alexander Todorov
366ae55abe Add make targets for Jenkins
these targets help hooking up things in Jenkins and enable us to
perform build & deploy tests for cloud images.

NOTE: use sudo -E to preserve the environment
2018-11-29 09:46:06 +02:00
Alexander Todorov
af2ae790ce Teach test_cli.sh to execute test scripts via arguments
this will be used to invoke scripts that build/push cloud images
without having to duplicate the setup/teardown/report parts!
2018-11-26 14:22:43 +02:00
Alexander Todorov
13fbd0861b new test: build an image and deploy it on Azure 2018-11-21 11:39:15 +02:00
Alexander Todorov
d6dcb38a78 Fix typo in comment 2018-11-21 11:39:15 +02:00
Alexander Todorov
6ae78fbd47 For OpenStack build image with rng-tools installed
appears to be a decent workaround for slow boot likely caused
by lack of entropy for ssh-keygen, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1639326
2018-11-13 12:09:07 +02:00
Brian C. Lane
fb4e6f2588 Add tests for partitioned disk images
The setup for this is a bit complex, so it really ends up testing things
twice.
2018-11-12 11:13:37 -08:00
Brian C. Lane
063a1770e1 Add tests for pylorax.imgutils
Some of these can only run as root on a real system with access to loop
devices. They are skipped when running in a container.
2018-11-12 11:13:37 -08:00
Brian C. Lane
bab4b20d0d Add tests to test_creator.py
Add a /.in-container file to the container root so that tests requiring root
and loop device support will be skipped when running in a container.
2018-11-12 11:13:37 -08:00