the biggest slow down is fetching data for many repositories
over a slow network. The previous retry count and sleep times
sometimes are not enough on Fedora.
It's necessary to make sure the blueprints directory doesn't contain
the git/ directory before the tests are run, so that we can just simply
modify the blueprint files without using blueprints push.
Related: rhbz#1714298
Beakerlib upstream can't do this yet, but might at some point:
https://github.com/beakerlib/beakerlib/issues/42
This is only enabled in combination with the `--sit` option of the
`test/check-*` scripts. It leaves the system in exacly the state it was
in when an assertion failed. Finishing the test run would run cleanup as
well (such as deleting created images). It also takes longer.
`setup_tests()` expected BLUEPRINTS_DIR to be set, but it wasn't when
running in automated mode (with $CLI set).
Fix this and move share and blueprint dirs to function arguments.
Not all parts of the script has been switched from awscli to ansible yet,
because the ansible aws modules do not support importing s3 object as snapshots.
(https://github.com/ansible/ansible/issues/53453)
Workaround using the image_location parameter of the ec2_ami ansible module
would mean adding extra code for generating AMI manifest with pre-signed
URLs.
We were checking for composer's FINISHED status only, which meant that
when a compose failed, the test ran until it timed out.
Check for failed as well. Also, always time out after 30 minutes.
Some test runners don't have nested virtualization enabled. Because
these checks are only checking that a boot works, kvm doesn't give us
that much. Disable for now.
Also remove the check for qemu-kvm. It doesn't abort the test
prematurely anyway.
A compose can change the hosts' policy, which can lead to docker
crashing if the container-selinux policy is not included. Add a
workaround and bug link.
The docker phase always failed because `-ti` was passed even though the
the output was not a terminal.
Also remove the check for /usr/bin/docker in the setup phase. It didn't
test that the daemon was running. More importantly, it didn't abort the
test anwyay (and there doesn't seem to be a good way to do this in
beakerlib).
Allows to run the tests on multiple operating systems and on the
infrastructure that the Cockpit team maintains.
`make vm` downloads one of Cockpit's test images (override which one
with TEST_OS) and installs rpms build from the local checkout of lorax.
The resulting image is placed in `test/images/$TEST_OS`.
TEST_OS can be set to any of Cockpit's supported images (default:
fedora-30).
Run `make check-vm` to run the CLI checks in the VM. The bulk of the
work is done in `test/check-cli`, which uses Cockpit's `bots` library to
start the VM and run the script in it.
Also included is a `test/run` script, which is the entrypoint for
Cockpit's test infrastructure.
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.
Also updated the docs, added a new test for it, and sorted the output
for consistency.
Add support for enabling and disabling systemd services in the
blueprint. It works like this:
[customizations.services]
enabled = ["sshd", "cockpit.socket", "httpd"]
disabled = ["postfix", "telnetd"]
They are *added* to any existing settings in the kickstart templates.
You can now open ports in the firewall, using port numbers or service
names:
[customizations.firewall]
ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
Or enable/disable services registered with firewalld:
[customizations.firewall.services]
enabled = ["ftp", "ntp", "dhcp"]
disabled = ["telnet"]
If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.
You can now set the keyboard layout and language. Eg.
[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"
Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'
Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.
For example:
[customizations.timezone]
timezone = "US/Samoa"
ntpservers = ["0.pool.ntp.org"]
Also includes tests.
This removes the timezone kickstart command from all of the templates
except for google.ks which needs to set it's own ntp servers and timezone.
If timezone isn't included in the blueprint, and it is not already in a
template, it will be set to 'timezone UTC' by default.
If timezone is set in a template it is left as-is, under the assumption
that the image type requires it to boot correctly.
- verify SemVer .patch number will be automatically updated when
we push the blueprint a second time without changing version
- verify show displays the content in TOML format and it matches
what is on disk. Because of that also start with empty packages
and groups fields in the initial toml. If they are missing they
will be added automatically by lorax-composer and this simplifies
the test
- verify delete works
Instances, Volumes, Snapshots, AMIs and s3 objects with the "keep_me" tag will
not be deleted automatically even if they are older then the specified time limit.
This is based on the VHD compose type, with the following differences:
* Use the vhdx format instead of vhd
* No WALinuxAgent
* Install hyperv-daemons
The hyperv-daemons are activated through udev rules, so there is no need
to add them to the services line.
If provided, round the disk image size up to a multiple of the value.
This allows for image formats with specific size-alignment requirements
(e.g., disk size must be in GiB).
Make sure that dotfiles are installed when not directly under /
Make sure / is not packaged in the rpm (it will cause a conflict with
the filesystem package).
Make sure that using destination="/" works
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.
Also adds tests to make sure it is written to config.toml in the build
metadata.
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:
[customizations.kernel]
append = "nosmt=force"
This will be appended to the kickstart's bootloader --append argument.
Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.
- Check final-kickstart.ks for the rpm source
- Check final-kickstart.ks for the rpm package name and version
- Make sure depsolve works
- Make sure errors from a bad repo are returned correctly
- Make sure errors from a bad reference are returned correctly
This moves _wait_for_status into a helper function so it can be shared
between the test classes.
This hooks up creation of the rpm to the build, adds it to the
kickstart, and passes the url to Anaconda. The dnf repo with the rpms is
created under the results directory so it will be included when
downloading the build's results.
This adds support, documentation, and testing for a [[repos.git]]
blueprint section that can be used to install files from a git
repository. It will create an rpm that will be added to the build,
and included in the metadata that can be downloaded. This allows you to
accurately keep track of the source of configuration files and extra
metadata that is added to the build.
The source repo and reference will be listed in the rpm's summary making
it easy to discover on the installed system.
this will allow you to test against installed RPM like so:
# export CLI="/usr/bin/composer-cli"
# make test_images
If you already have lorax-composer running then you can directly
execute test scripts:
# ./tests/cli/test_build_and_deploy_aws.sh
Use constants so we won't have to edit a dozen places in the test when
the package versions are bumped.
Also switch to using Fedora 31 GPG key now that it has branched for
Fedora 30.
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.
One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.
This also adds tests for the various operations.
Resolves: rhbz#1682113
In order to support iso creation on multiple arches with the templates
we need to be able to select different packages based on arch.
lorax-composer uses the arch-specific Lorax templates in order to
generate the output iso so this patch:
1. Creates a new template and type to parse it, live-install.tmpl
which contains only installpkg commands and #if clauses for arch
2. Removes bootloader related packages from the live-iso.ks
3. Remove dracut-config-rescue exclusion because it can cause problems
with some blueprints.
4. Switch logo requirement to system-logos which is satisfied by
generic-logos or fedora-logos. This prevents conflicts when a blueprint
installs fedora-release-workstation.
So in the future, if x86.tmpl, etc. need a new package to support
creating the iso it should be added to the correct section in
./share/live/live-install.tmpl
reqpart can be used to make kickstarts more platform agnostic, creating
needed partitions without lmc having to keep track of the arch-specific
needs. eg. ppc64 needs prepboot and /boot
This increases the size of the disk based on whether reqpart or
reqpart --add-boot is in the kickstart.
Note that this is only valid for partitioned disk output types, not
for filesystem images or live iso output.
- on some arches (also Fedora x86_64) systemd-nspawn may not be
available
- delete composes from other tests in rlPhaseStartCleanup because
we're seeing the tar compose kind of hanging in Jenkins and that
test script is executed last so the slave may be running out of
disk space. Be a good citizen and clean up after the previous
tests.
b/c we've migrated to Upshift we must use different instance type,
specify the desired network to connect to and update how we get
the ip address of the launched VM.
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.
So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.
The OS_PROJECT_NAME (or OS_TENANT_NAME) environment variable needs to be defined.
Use the OS_PROJECT_NAME, since it is recommended in the documentation instead of
the older OS_TENANT_NAME.
Support both
[customizations]
hostname = "whatever"
and
[[customizations]]
hostname = "whatever"
in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.
Add some tests for converting customizations to kickstarts.
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.
This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.
Resolves: rhbz#1656642
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
* Removes broken symlinks from queue/new and queue/run
* Removes symlinks from run and sets the build to FAILED
* Sets builds w/o a STATUS to FAILED
* Sets builds with STATUS of RUNNING to FAILED
* Creates missing queue/new symlinks to results with STATUS of WAITING
So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.
Resolves: rhbz#1647985
there's 7.2 in Fedora 29 (which we use in Jenkins) and 7.3 in
Rawhide which causes test jobs to fail with unsolved dependencies.
Alternatively we can switch to another recipe for cloud image tests.
otherwise composer-cli is unable to glob() the kickstart
files and we're left without supported compose types. Seen
during AWS testing for example.
Helps with running some of the tests via sudo b/c this is
what Jenkins requires.
If trying to execute test_cli.sh inside a git checkout
we are going to get the following exception:
Traceback (most recent call last):
File "./src/sbin/lorax-composer", line 251, in <module>
repo = open_or_create_repo(server.config["REPO_DIR"])
File "/home/jenkins/lorax/src/pylorax/api/recipes.py", line 306, in open_or_create_repo
gi.repository.GLib.Error: ggit-error: failed to stat '/home/jenkins/lorax/tests/pylorax/blueprints': Permission denied (-1)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib64/python3.7/multiprocessing/popen_fork.py", line 54, in _send_signal
os.kill(self.pid, sig)
From what I can tell open_or_create_repo() is trying to initialize
a git repository inside the blueprints directory which fails when
we have an active git checkout.
This doesn't happen when we run the tests in Travis CI because
rsync excludes .git/ inside the Docker container.
these targets help hooking up things in Jenkins and enable us to
perform build & deploy tests for cloud images.
NOTE: use sudo -E to preserve the environment
It needs to be root in order to set the ownership and permissions on the
directories that are under /var/lib/lorax/composer/
Refactor the directory creation into a utility function, and use a umask
of 0o006 to ensure that the parent directories created do not have o+rw
set on them (makedirs behavior is different between Python 3.6 and 3.7
so umask of 0 doesn't work consistently).
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.
composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'
Also adds tests for this.
Resolves: rhbz#1626122
Also kill the lorax-composer process and remove /run/weldr/api.socket
so that when this is run with podman you don't get an error about
attempting to tar up the socket.
This tests to make sure that the metadata timer is working (by setting
it to 10s and adding a new package to the repo), and that
DNFLock.lock_check immediately picks up a new package.
This depends on rpmfluff which is available from Fedora or EPEL repos.
Related: rhbz#1631561
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.
Remove `except` block which immediately raises the same exception again (it's
not a subclass of another caught exception, so this is safe).
Remove a false positive, because it is not emitted from the code base.
Disable subprocess-popen-preexec-fn in startProgram, which is not used
internally.
- need to specify --sharedir so lorax-composer can find its
kickstart files
- each test script writes results into a separate directory to
avoid a passing test overwriting the results from a failing one.
To avoid reporting failures in case of previously failing tests
(e.g. during development) remove the temporary directories holding
tets results before execution!
these are built on top of beakerlib and we use its internal
protocol to figure out the result without relying on the full
test runner that is tipically used inside of a RHEL environment!
Includes a disabled test snippet for Issue #460
