The host system can run out of space while running mksquashfs, and while
this is logged to program.log it isn't detected by lorax or
livemedia-creator so it will continue running, possibly reporting
unrelated errors and causing confusion.
This adds checks for the return status when calling mksquashfs, logs it
to the log, and either exits or raises an error immediately.
Previously the release package was chosen by picking the first package
that provides 'system-release' that isn't named generic. This can cause
branding issues with repos containing multiple system-release packages.
This patch changes _install_branding so that it will give preference to
a system-release package that ends with lowercase --variant name. If
there isn't one it will fall back to the previous behavior.
Resolves: rhbz#1472622
epoch wasn't included in the lorax-packages.log, but arch is. And for
the debuginfo the epoch, including 0, *is* included. This should match
the previous output.
Resolves: rhbz#1815000
A change in glibc now requires /proc be mounted in order to run mknod
which is needed in order to run rpm from runtime-postinstall.
This drops that code from the template and moves writing the package
list into run_transaction, which already has all of the needed
information to generate the list.
Resolves: rhbz#1812895
It isn't always obvious what happened when the rootfs runs out of space,
especially when using lorax via pungi. So this checks for the out of
space error string when building the runtime image and logs it to the
primary logfile and console as an error with the rootfs size.
eg.
2020-01-20 18:52:58,920: The rootfs ran out of space with size=1
startProgram is running in binary mode, so bufsize=1 is invalid. The
ExecLineReader class already breaks it up into lines using readline() so
there is no need to change the default buffering.
On some platforms (aarch64, ppc64le) toolchain limitations/optimizations
can break anaconda startup, as discussed in rhbz#1722181. The workaround
is to preload libgomp.so before starting anaconda.
When the timeout is too short the http library sends the request again.
We return the last response to the user so even if the cancel works they
get an error about the UUID not being valid.
Resolves: rhbz#1788461
To distinguish from the base images, a different
volume label is useful.
Add a commandline option, pass it to the iso tool,
and replace the occurences of the label in isolinux
and grub config files.
Signed-off-by: Brian C. Lane <bcl@redhat.com>
The enabled bool is now being used so the cli should only show the types
actually available on the architecture.
Also modifies the test in test_compose_sanity.sh
Related: rhbz#1751998
The callers, and the documentation, all expect int 0/1 to use as the
exit status for the program. Not True/False, even though that works most
of the time.
Graft them from tmpdir instead of trying to use the source iso for some,
and tmpdir for the ones needing changes. This will prevent problems with
trying to remove files likes TRANS.TBL from a read-only filesystem.
This tool will add the kickstart to the boot.iso, edit the kernel boot
arguments so that the kickstart is used when the iso boots, as well as
allow adding extra files and directories to the / of the iso which can
then be used by the kickstart (they are found under /run/install/repo
while Anaconda is running).
The 'enabled' field in the /compose/types output now reflects whether or
not the type is supported on the current architecture. Disabled types
are not allowed to be built, and will raise an error like:
Compose type 'alibaba' is disabled on this architecture
This outputs a TOML template of the settings needed for setting the
upload credentials. It can be passed to 'upload start' and to 'compose
start', as well as used to set the profile for 'providers push'
Since we have both compose uuids and upload uuids they need to be
clearly named. This updates the upload naming to use 'upload_uuid' in
the inputs, and 'upload_id' in the output (_id instead of _uuid for
consistency with build_id naming in the status responses).
This also adds 'upload_id' to the /upload/log response.
This allows deleting a provider's profile. Pass the provider and profile
name like:
/api/v1/upload/providers/delete/azure/test-settings
A standard json response will be returned.
This extends the /compose/ route to support uploading with either an
existing profile, or with one-time use settings passed in the POST.
To select a profile include the provider and profile, as returned by
`/uploads/providers`:
"upload": {
"image_name": "My Image",
"provider": "azure",
"profile": "production-azure-settings"
}
uploads should only be included in the V1 API routes, add `api`
selection to the relevant helper functions and calls to them from v0.py
Add new V1 routes with api=1 to include the uploads information in the
results.
Also add tests to ensure that V0 requests do not include uploads.
Currently, Azure, vSphere, and OpenStack are supported. See
https://github.com/weldr/lorax/pull/826 for more details about this new
feature.
I've called the upload library "lifted" as a reference to Seuss'
The Lorax -- in the book, the Lorax lifts himself up by the seat of his
pants through a hole in the smog clouds, and they start calling him the
"Lifted Lorax."
This adds new features to the /compose route under API v1
A recipe that is valid TOML can still be an invalid recipe (eg. missing
the 'name' field) so this should also catch RecipeError.
Also added tests for this, as well as making sure commit_recipe_file()
raises the correct errors.
Resolves: rhbz#1755068
Add dmsquash-live-ntfs to the default dracut modules in livecd builds.
The omission of this is probably why:
https://bugzilla.redhat.com/show_bug.cgi?id=1449410
didn't work out as intended (I suspect it was closed unfixed).
Without this, initramfs winds up with the ntfs-3g tools lying
around, but no hooks cause them to actually get used.
AFAICT, this is a legit bugfix, not a "proposed enhancement"
so to speak. To test that this actually fixed the problem (well,
/a/ problem, at least), I used some scripts:
https://github.com/gmt/test-respin-fedora-ntfsable-iso
Signed-off-by: Greg Turner <gmt@be-evil.net>
Signed-off-by: Brian C. Lane <bcl@redhat.com>
This loads the system dnf vars from /etc/dnf/vars at startup if
system repos are enabled, and it substitutes the values in the sources
when loaded, and when a new source is added.
Also includes tests.
This changes the source 'name' field to match the DNF usage of it as a
descriptive string. 'id' is now used as the short name to refer to the
source. The v0 API remains unchanged.
Tests for v1 behavior have been added.
Now that the v1 API is in use the status message will return api: 1
This creates a tar suitable for use with the anaconda kickstart liveimg
command. It adds the kernel, grub2, and grub2-tools packages to the tar
template.
Normally the runtime image is an ext4 filesystem image that is
compressed with squashfs. dracut now supports setting up an overlayfs
when it detects a bare filesystem tree inside the squashfs.
This commit adds support for a --squashfs-only option which places the
root tree directly in the squashfs.img
In python 3 f.seek() on text doesn't work like it does in py2/C because
text is now unicode. So change read_tail to use byte mode and take
unicode into account. Also add tests for it.
The VALID_API_STRING function allows for characters that should not be
allowed in blueprint names. VALID_BLUEPRINT_NAME allows us to
specifically check if a blueprint contains a valid name.
dnf seems to have changed the default for skip_if_unavailable. Some
mock repositories are still around in later tests, which then fail
because metadata cannot be synced.
Also expose skip_if_unavailable in dnf_repo_to_file_repo(), so that
tests checking for equality of repo files continue to pass.
This makes sure that required fields are included, and that sections are
not empty. It does not check for all optional fields.
If there are errors it will gather up all of them and then raise a
RecipeError with a string of all the errors.
The new toml library, introduced with abe7df34f, outputs different
whitespace from the old one. Fix the test expectation and strip()
results from toml.dumps(), because it contains superfluous newlines at
the end.
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.
Instead of setting up the routes inside a function we can now use a
BlueprintSkip class, which allows us to register them at different
routes (eg. /api/v0/ and /api/v1/) and override any routes that will be
replaced by the new API version.
When adding a new API we want to use the old code for any routes that
aren't being overridden.
This modifies the Flask Blueprint class so that a skip_rules list can be
passed to server.register_blueprint()
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.
Also updated the docs, added a new test for it, and sorted the output
for consistency.
Add support for enabling and disabling systemd services in the
blueprint. It works like this:
[customizations.services]
enabled = ["sshd", "cockpit.socket", "httpd"]
disabled = ["postfix", "telnetd"]
They are *added* to any existing settings in the kickstart templates.
You can now open ports in the firewall, using port numbers or service
names:
[customizations.firewall]
ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
Or enable/disable services registered with firewalld:
[customizations.firewall.services]
enabled = ["ftp", "ntp", "dhcp"]
disabled = ["telnet"]
If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.
You can now set the keyboard layout and language. Eg.
[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"
Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'
Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.
This fixes the customizations list problem earlier than in
add_customizations.
In the recipe it should be [customizations] not [[customizations]]
which creates a list. If it was used that way grab the first element and
replace the list with it.
For example:
[customizations.timezone]
timezone = "US/Samoa"
ntpservers = ["0.pool.ntp.org"]
Also includes tests.
This removes the timezone kickstart command from all of the templates
except for google.ks which needs to set it's own ntp servers and timezone.
If timezone isn't included in the blueprint, and it is not already in a
template, it will be set to 'timezone UTC' by default.
If timezone is set in a template it is left as-is, under the assumption
that the image type requires it to boot correctly.
This is based on the VHD compose type, with the following differences:
* Use the vhdx format instead of vhd
* No WALinuxAgent
* Install hyperv-daemons
The hyperv-daemons are activated through udev rules, so there is no need
to add them to the services line.
This option will create an optionally compressed tarball containing a
disk image. This format is used by Google's Compute Engine.
This also adds a new option, tar_disk_name, to set the name of the disk
image that will be wrapped in the final tarball. opts.image_name
continues to be the final output file name.
If provided, round the disk image size up to a multiple of the value.
This allows for image formats with specific size-alignment requirements
(e.g., disk size must be in GiB).
rpmfluff was including / in the rpm, which conflicts with
filesystem.rpm
The rpm globs are pretty limited, and we don't actually know the file
paths until later, so we have to use a glob or a directory.
So when the destination is / it now uses /* to select all the files and
sub-directories in the archive. The limitation of this is that it cannot
support dotfiles directly under /, they will cause a rpmbuild error.
For destinations other than / it uses the name of the directory, so
dotfiles are fine in that situation.
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.
Also adds tests to make sure it is written to config.toml in the build
metadata.
