In py3 dicts are not predictable, make sure the keys are sorted so that
the tests are useful.
Use StringIO.StringIO instead of io.StringIO which requires unicode
text.
kickstart timezone.ntpservers is a set() so adjust the test for it.
Related: rhbz#1718473
composer-cli will now output information about changes to customizations
entries and the repos.git entries.
(cherry picked from commit 156ef0acfd)
Related: rhbz#1718473
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.
(cherry picked from commit 850c490b6e)
Related: rhbz#1718473
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.
Also updated the docs, added a new test for it, and sorted the output
for consistency.
(cherry picked from commit 3e08389a0f)
Related: rhbz#1718473
Add support for enabling and disabling systemd services in the
blueprint. It works like this:
[customizations.services]
enabled = ["sshd", "cockpit.socket", "httpd"]
disabled = ["postfix", "telnetd"]
They are *added* to any existing settings in the kickstart templates.
(cherry picked from commit 1111aee92d)
Related: rhbz#1718473
You can now open ports in the firewall, using port numbers or service
names:
[customizations.firewall]
ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
Or enable/disable services registered with firewalld:
[customizations.firewall.services]
enabled = ["ftp", "ntp", "dhcp"]
disabled = ["telnet"]
If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.
(cherry picked from commit 4d35668ab5)
Related: rhbz#1718473
You can now set the keyboard layout and language. Eg.
[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"
Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'
Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.
(cherry picked from commit e5a8700bdf)
Related: rhbz#1718473
For example:
[customizations.timezone]
timezone = "US/Samoa"
ntpservers = ["0.pool.ntp.org"]
Also includes tests.
This removes the timezone kickstart command from all of the templates
except for google.ks which needs to set it's own ntp servers and timezone.
If timezone isn't included in the blueprint, and it is not already in a
template, it will be set to 'timezone UTC' by default.
If timezone is set in a template it is left as-is, under the assumption
that the image type requires it to boot correctly.
(cherry picked from commit 9bdbb29662)
Related: rhbz#1718473
These are used when running tests, so catch-up with changes in lorax.
Includes adding rootfs-size, extra kernel args, and using locking to
prevent temporary files from being deleted by systemd tmpfiles handling.
Related: rhbz#1668520
Related: rhbz#1715116
Related: rhbz#1689314
If a repository has `sslcacert`, `sslclientcert`, or `ssclientkey` set,
pass them to anaconda through the kickstart file. This is mostly the
case when using RHEL repositories that are accessed through a
subscription.
Resolves: rhbz#1701033
This fixes the customizations list problem earlier than in
add_customizations.
In the recipe it should be [customizations] not [[customizations]]
which creates a list. If it was used that way grab the first element and
replace the list with it.
Related: rhbz#1688335
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.
Also adds tests to make sure it is written to config.toml in the build
metadata.
Resolves: rhbz#1688335
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:
[customizations.kernel]
append = "nosmt=force"
This will be appended to the kickstart's bootloader --append argument.
Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.
Related: rhbz#1688335
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.
One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.
This also adds tests for the various operations.
(cherry picked from commit d32f477e0b)
Resolves: rhbz#1683442
When there is no run or new symlink do one last check to make sure no
STATUS file was written. If it is missing, go ahead and remove the
results directory.
(cherry picked from commit a8f616c6da)
Related: rhbz#1659129
If another CANCEL request has already been made just exit from
uuid_cancel. If the build is FINISHED before it times out just exit,
don't remove the finished results.
(cherry picked from commit 962d10b3b7)
Related: rhbz#1659129
In addition to monitoring the logs for errors, call a function (or
functions) that tell it to cancel the anaconda process and cleanup.
Also check for a cancel after creating the squashfs image for live-iso
since that's a long running process.
This required adding a new argument to a number of existing functions,
passing it down to VirtualInstall and novirt_install where the function
is called.
(cherry picked from commit 4b84475612)
Resolves: rhbz#1659129
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.
So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.
Resolves: rhbz#1664128
Support both
[customizations]
hostname = "whatever"
and
[[customizations]]
hostname = "whatever"
in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.
Add some tests for converting customizations to kickstarts.
(cherry picked from commit 35ab6a1336)
Resolves: rhbz#1666517
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.
composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
pylint has trouble with Flask response objects, thinking they are tuples
and returning no-member errors. It also doesn't recognize gevent.socket
members like AF_UNIX.
The projects_list function uses a different yum request, and doesn't
return a full project dict. Updated it to also return only the unique
project names.
Related: rhbz#1657055
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.
This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.
Resolves: rhbz#1657055
(cherry picked from commit 663a0dcd73)
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
* Removes broken symlinks from queue/new and queue/run
* Removes symlinks from run and sets the build to FAILED
* Sets builds w/o a STATUS to FAILED
* Sets builds with STATUS of RUNNING to FAILED
* Creates missing queue/new symlinks to results with STATUS of WAITING
So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.
Resolves: rhbz#1657054
(cherry picked from commit f0bac40d7f)
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning
(cherry picked from commit 1056bfc25b)
Resolves: rhbz#1656105
This is similar to the AMI compose type, with a handful of additional
changes specific to Azure:
* Add waagent (but leave NetworkManager enabled, despite some of the
docs)
* Disable cloud-init
* Add Hyper-V modules into initrams.
Fixes specific for RHEL:
* Create ifcfg-eth0 required by waagent.
* Install python3 and net-tools required by waagent.
Recommended changes:
* Use recommended kernel boot args.
* Disable kdump.
(cherry picked from commit e0c236ff36)
(cherry picked from commit da0435bc90)
(cherry picked from commit b594fa99bc)
Resolves: rhbz#1656105
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.
(cherry picked from commit 18188bf6cf)
(cherry picked from commit 81d38b6445)
Resolves: rhbz#1656105
Anaconda, Lorax, lorax-composer, and livemedia-creator can all now run
with SELinux in Enforcing mode. It does not need to be disabled and if
there are denials they should be reported as a bug.
Log the current state of SELinux when starting, update the
documentation.
Resolves: rhbz#1654795
Some kickstart templates use globbing to match packages, searchNames
doesn't support this.
Add fallback code to find packages that are just a dep, matching the
behavior of yum.install()
This fixes things so that it can depsolve package names with globs in
them like grub2-efi-*-cdboot, and deps like shim and grub2-efi.
Results are still filtered by the version glob, with the highest NEVRA
selected for installation.
Resolves: rhbz#1641601
The problem this solves is that yum really isn't designed to be part of\
a long running daemon. So when repodata changes upstream, even when
you force it to download the new metadata, it doesn't change in memory
so you end up with lorax-composer depsolving against old versions, and
anaconda depsolving against new versions (because it sets up its own
YumBase and cache) and then the kickstart is no longer valid.
To solve this I have
- Added a 6h timeout to the metadata check (because yum's doesn't work
in this situation).
- Added a metadata check to the YumLock .lock property, but only when
the timeout expires.
- Added a new .lock_check property to YumLock that always checks the
metadata and resets the timeout.
If it has changed it does its best to tear down the existing YumBase,
deleting as much as it can in hopes it doesn't leak memory. And then it
sets up a totally new YumBase with the new repodata.
Resolves: rhbz#1632962
Use a common _depsolve function for projects_depsolve and
projects_depsolve_with_size so that it always uses the correct version
glob support when depsolving blueprints and templates.
Resolves: rhbz#1628114
Ends up you cannot use the kickstart user command on root, since it
already exists, so we have to translate that into a rootpw command.
So [[customizations.user]] with name = "root" only support key, which
will set the ssh key, and password which will use rootpw to set the
password. plain text or encrypted are supported.
Related: rhbz#1626120
The blueprint version glob was being applied to the whole package NEVRA
by yum (it lacks a separate API for just globbing versions), so this
implements that in filterVersionGlob using fnmatchcase on the package
names, and the yum package verGT comparison on the versions for the
selected package.
Also includes tests.
Resolves: rhbz#1628114
blueprints/changes is different, each blueprint has it's own total,
limited by the call's limit. So it needs to find the max total of all
the requested blueprints.
The blueprints/changes API is a bit different from the others, the total
that it includes is for each blueprint, not one total for all of them,
since there will be a different number of commits for each.
The function is passed the dict, and it can be used to select the total
to use for retrieving all of the results. If it isn't included it will
use data["total"] which works fine in most cases.
