Depsolve the packages included in the templates and report any errors
using the /api/status 'msgs' field. This should help narrow down
problems with package sources not being setup correctly.
It appears that sometimes the loop device doesn't get setup properly,
this may be a race with other users of loop devices on the system, or
some other mechanism that isn't understood.
To try and prevent total failure when this happens this patch retries
the loop setup 3 times before giving up. Previously it would wait for
the loop device to appear (checking 5 times), that operation is now
executed 3 times with a new losetup attempt each time.
Resolves: rhbz#1589084
(cherry picked from commit c746e8b0c3)
Make it possible to manipulate the simple and regexp
tests the LogRequestHandler class uses to check error
messages for potential error states.
This is accomplished by moving the simple and regexp test
strings to class members, where they can be easily
manipulated by users of the pylorax module.
It's also now possible to set the log request handler class
for a LogMonitor.
This functionality can then be used for example like this:
customized_log_request_handler = monitor.LogRequestHandler
customized_log_request_handler.simple_tests.remove("Call Trace:")
log_monitor = monitor.LogMonitor(install_log,
timeout=opts.timeout,
log_request_handler_class = customized_log_request_handler)
This way installation will continue even if there was a call
trace in the logs. In a similar way additional tests and regexps can be
also added.
(cherry picked from commit d5d3dd3be3)
Use it to override the default dracut arguments (displayed as part of
the --help output). If you want to extend the default arguments they
all need to be passed in on the cmdline as well. eg.
--dracut-arg='--xz' --dracut-arg='--install /.buildstamp' ...
Resolves: rhbz#1452220
(cherry picked from commit d8ce013a2b)
This adds the sources command which can be used to list, add, change,
and delete sources using the TOML formatted source file.
(cherry picked from commit 6f6ce410c0)
DNF Repo.dump() function cannot be used as a .repo file for dnf due to
it writing baseurl and gpgkey as a list instead of a string. Add a new
function to write this in the correct format, and limited to the fields
we use.
Add a test for the new function.
Fix /projects/source/info to return an error 400 if a nonexistant TOML
source is requested. If JSON is used the error is part of the standard
response.
Update test_server.py to check for the correct error code.
(cherry picked from commit afa89ea657)
When adding a source failed it wasn't being removed from the dnf object.
This fixes that, and returns an error when setting up the source fails.
Also adds a test for it.
This also includes detecting rawhide vs. non-rawhide releases and
adjusting the tests accordingly (some of the source names change).
(cherry picked from commit dd8e4d9e99)
It was chopping off an extra directory level due to realpath removing
the trailing / from the paths when they are setup.
(cherry picked from commit 23f4b2a3ec)
We had only been indirectly pulling in GConf, and anyways
nothing was listening to these keys.
<kalev> I still think it's a fallout from 27a90d973f?branch=master
Really in general, if we wanted to make changes like this
it'd probably be a lot simpler to do them on boot or so.
https://bugzilla.redhat.com/show_bug.cgi?id=1581838
(cherry picked from commit bb3d8edd06)
This uses dnf's version__glob filter to implement it. It amounts to '*'
wildcards and '?' for single character matching.
(cherry picked from commit 095829171a)
First is Anaconda uses 6k blocks per file for its estimate, and it
fudges by 10% so adjust for those with an extra 10% of headroom just in
case.
Second is an Anaconda bug that won't allow it to do a kickstart install
to a disk smaller than 3000 MB. There is a PR to fix it upstream, but
for now the minimum size has to be 3000e9
This adds support for the optional blueprint section [customizations].
Use it like this:
[customizations]
hostname = yourhostnamehere
[[customizations.sshkey]]
user = root
key = root user key
Different versions of libgit2 act differently. Using TIME results in
some commits (like a revert) being listed correctly, but the rest being
listed in reverse order. Leaving it at the default works for
libgit2-0.26.3
This moves everything except the cmdline checking into run_creator in
pylorax.creator
It also rearranges some functions to prevent import loops, and adds a
utility function to imgutils (mkfsimage_from_disk for copying a
partition into a filesystem image).
This no longer uses the enabled configuration setting to select repos to
use. It uses everything in the repo_dir, and if system repos have not
been disabled it copies them into the repo_dir at startup, overwriting
the previous copy.
This reduces the amount of code in livemedia-creator to the cmdline
parsing and calling of the installer functions. Moving them into other
modules will allow them to be used by other projects, like the
lorax-composer API server.
filter(provides=...) doesn't work with paths. The release packages
provide system-release so just look for that instead of a file.
Now it finds the release package and selects it along with the
corresponding logos package.
Note, this has been broken since commit 431ca6ce
Commit 8edaefd4d1 added the ability to install specific NVR's of
packages, but it did not adjust the exclude operation to account for
this.
This patch fixes that, applying the exclude only to the name part of the
package NVR, and changes some variable names to pkgnvr/pkgnvrs to make
it more clear that the content has changed to <name>-<version>-<release>
Some lorax users run it from inside mock, which isn't able to detect
whether the host is in Permissive mode. This can lead to confusing
error messages, so this points them in the right direction.
When multiple units are passed to systemctl and one fails it doesn't
finish the others. Change the template command to call systemctl for
each unit individually.
This also removes the lvm2-activation-generator in runtime-cleanup.tmpl
This will allow anaconda to fetch kickstarts using https when installing
with fips=1
Leave vmlinuz and .vmlinuz.hmac in /boot
dracut-fips module needs the vmlinuz.hmac file in order to boot.
It seems that on rare occasions losetup can return before the /dev/loopX
is ready for use, causing problems with mkfs. This tries to make sure
that the loop device really is associated with the backing file before
continuing.
NOTE that using losetup --list -O to return the backing store
associated with the loop device can fail due to losetup truncating
the output filename if sysfs isn't setup. Instead of printing the full
path it will truncate it to 64 characters with a * at the end.
See util-linux lib/loopdev.c for the code that does this.
Use the existing get_loop_name function, which uses losetup -j, to lookup
the loop device associated with the backing store which should work the
same, just in the opposite direction.
For historical reasons, lorax used the 'anaconda' package as a
touchstone to determine the architecture for the build. At some
point, this package became a metapackage that pulls in both the
GUI and headless installers.
In the modular world, it's possible that only the core and TUI bits
may be available for use. The only subpackage of anaconda that is
guaranteed to be on any viable system is anaconda-core, so let's
switch to using that for the touchstone instead of the metapackage.
Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
Also sort the expanded list of packages so that any failures will
be consistent instead of depending on the randomness of a set().
And add better logging when things fail.
The core issue is that repodata may have packages that match globs, but
they cannot actually be installed (eg. sigrok-firmware). This can cause
*some* of the globbed packages to be installed before hitting the
failure package.
With this change it will log the expanded list of packages if a glob is
used. It will skip any packages that fail to install when using
--optional with the glob, and continue to install the rest.
Related: rhbz#1440417
Previously lorax had no way to use repos with self-signed certificates.
This adds the --noverifyssl cmdline option which will ignore certificate
errors.
Resolves: rhbz#1430483
OSTree is a deduplicating hardlink store using a new file path
`/ostree`, which SELinux policy doesn't know about. However, OSTree
has SELinux support built in, and rpm-ostree (for example) uses this
to ensure the attributes on files stored there are simply always
correct. Relabeling it will corrupt it.
Hence, let's skip it.
Right now we dump all subprocess output to `program.log`. Unfortunately,
The pungi/koji stack doesn't know how to scrape out the lorax logs.
And even when running interactively, it's annoying that *some* fatal
errors show up on stderr, but if it's from a subprocess, I need to go
over and `tail program.log`.
Let's output the subprocess stderr directly, since the user is
going to want it prominently anyways.
anaconda-26.1 changed how package scriptlet failures are handled. They
are now fatal, and anaconda hangs after logging an Installation failure.
ERR packaging: Installation failed: PayloadInstallError('DNF error:
Non-fatal POSTIN scriptlet failure in rpm package mlocate',)
Catch this (the 'packaging: Installation failed' part) and terminate the
image creation.
This controls how big the root filesystem is for the squashfs used in
the boot.iso, the default is 2GiB.
Note that larger rootfs sizes will require more memory and may cause the
build to fail.
I'm working on
https://fedoraproject.org/wiki/Changes/WorkstationOstree and when
using lorax to make an installer ISO with content embedded, I run out
of disk space since the desktop+various apps is large.
Since this ends up being compressed anyways, let's just bump the
currently arbitrary `2` to `10` - the only real cost I can think of is
going to be a few more superblock entries.
If the query filter doesn't return anything it would just ignore the
install request instead of logging and raising an error when
required=True.
This checks for no packages matching, and if required is True raises an
error after all of the requested packages have been processed, instead
of after the first one to fail.
Previous versions of lorax assumed that installpkg was optional, and
would continue on if the PKGGLOB didn't match anything. But the majority
of the packages are required so this allows the boot.iso to be built
with missing packages that are hard to track down.
It makes more sense to make the PKGGLOB required and to flag the
few exceptions to this with --optional.
DNF doesn't want users to access base.logging anymore.
Lorax already takes over the "dnf" logger and directs it to ./dnf.log,
so it wasn't really being used.
This raises the debug level to DNF's custom DDEBUG, and sets it up so
that dnf.librepo.log and hawkey.log are next to dnf.log
Before attempting to cleanup any dangling anaconda mounts copy the
anaconda logs to their final location.
Also, catch failures to cleanup the mounts, log it, and continue trying
the other mountpoints. A cleanup failure will result in an InstallError
instead of a CalledProcessError.
Fedora now has a edk2 package so use the OVMF code from there. This also
adds using a copy of OVMF_VARS for each boot instead of reusing the one
provided by the package.
In some cases the initramfs may not be present in /boot to save space.
Use it if present, otherwise use the kernel version to recreate the name
of it.
This also fixes problems with dracut running out of space when not using
--live-rootfs-keep-size
There's no reason to require the initramfs when we can rebuild it using
the version from the kernel. This adds handling of missing initramfs so
that lmc kickstarts can remove it from the squashfs, saving about 40M on
the iso.
This makes sure the contents of /boot are at the expected locations in
/boot and in sys_root. For partitioned images it mounts the separate
/boot partition on /boot. For both fsimage and partitioned images ir
binf mounts it to sys_root so that the kernel+initrd can be found.
The boot directory isn't always named boot.0, so wildcard it and let the
count check handle failure if there is more than 1.
umount tries to delete a mountpoint if it has lorax.imgutils in the
path. This doesn't work right if you try to umount something mounted
deeper on the path.
This adds a delete option, which is True by default, to skip the delete.
If an anaconda no-virt run crashes it can leave things mounted under
/mnt/sysimage. Previously anaconda-cleanup was used to handle this, but
it will also try to cleanup host mountpoints which isn't desired.
When using the template install command copying the same file to itself
shouldn't crash. Just log the error and continue.
Also copy the s390 configuration files for use with livemedia-creator
Resolves: rhbz#1269213
When an image name hasn't been passed, and the compression type is
something other than xz, the default image name should use the user
specified compression suffix.
Resolves: rhbz#1318958
Some cases of mksquashfs were not using -Xbcj when it is available for
the arch. This adds a function to return the correct args based on the
arch and the cmdline args.
Allow the template to select a different compression type or arguments
for the installimg command.
On 32bit builds running inside a mock xz sees the full amount of system
memory which can result in xz failing with a memory error. This allows
the template to limit the amount of memory it tries to use.
lmc --no-virt was switching selinux to permissive if it was enforcing
and restore it when done. This works fine when it is the only session
running, but would cause problems if it was run in parallel.
It now only checks the state and exits with an error if it isn't already
disabled or in Permissive mode.
Users will need to run setenforce 0 before running lmc.
If there isn't enough space for DNF to download packages it will log:
"Not enough disk space to download the packages."
So add this to the messages in monitor that trigger an error.
This makes package selection a little more roundabout, but it allows for
unused packages (and their dependencies) to be removed from globs during
the install phase.
dnf.subject.Subject is the class used by dnf's Base.install to select
packages, so the behavior of installpkg without --except options is the
same as it was before.
Brian C. Lane
Martin Kolman
Colin Walters
Yuval Turgeman
Claudio Zumbo
Alexander Todorov
Stephen Gallagher
Jaroslav Mracek
Pat Riehecky
Igor Gnatenko
Manjunath A Kumatagi
Friedrich Schöller
David Shea