Also sort the expanded list of packages so that any failures will
be consistent instead of depending on the randomness of a set().
And add better logging when things fail.
The core issue is that repodata may have packages that match globs, but
they cannot actually be installed (eg. sigrok-firmware). This can cause
*some* of the globbed packages to be installed before hitting the
failure package.
With this change it will log the expanded list of packages if a glob is
used. It will skip any packages that fail to install when using
--optional with the glob, and continue to install the rest.
Related: rhbz#1440417
Previously lorax had no way to use repos with self-signed certificates.
This adds the --noverifyssl cmdline option which will ignore certificate
errors.
Resolves: rhbz#1430483
OSTree is a deduplicating hardlink store using a new file path
`/ostree`, which SELinux policy doesn't know about. However, OSTree
has SELinux support built in, and rpm-ostree (for example) uses this
to ensure the attributes on files stored there are simply always
correct. Relabeling it will corrupt it.
Hence, let's skip it.
Right now we dump all subprocess output to `program.log`. Unfortunately,
The pungi/koji stack doesn't know how to scrape out the lorax logs.
And even when running interactively, it's annoying that *some* fatal
errors show up on stderr, but if it's from a subprocess, I need to go
over and `tail program.log`.
Let's output the subprocess stderr directly, since the user is
going to want it prominently anyways.
anaconda-26.1 changed how package scriptlet failures are handled. They
are now fatal, and anaconda hangs after logging an Installation failure.
ERR packaging: Installation failed: PayloadInstallError('DNF error:
Non-fatal POSTIN scriptlet failure in rpm package mlocate',)
Catch this (the 'packaging: Installation failed' part) and terminate the
image creation.
This controls how big the root filesystem is for the squashfs used in
the boot.iso, the default is 2GiB.
Note that larger rootfs sizes will require more memory and may cause the
build to fail.
I'm working on
https://fedoraproject.org/wiki/Changes/WorkstationOstree and when
using lorax to make an installer ISO with content embedded, I run out
of disk space since the desktop+various apps is large.
Since this ends up being compressed anyways, let's just bump the
currently arbitrary `2` to `10` - the only real cost I can think of is
going to be a few more superblock entries.
If the query filter doesn't return anything it would just ignore the
install request instead of logging and raising an error when
required=True.
This checks for no packages matching, and if required is True raises an
error after all of the requested packages have been processed, instead
of after the first one to fail.
Previous versions of lorax assumed that installpkg was optional, and
would continue on if the PKGGLOB didn't match anything. But the majority
of the packages are required so this allows the boot.iso to be built
with missing packages that are hard to track down.
It makes more sense to make the PKGGLOB required and to flag the
few exceptions to this with --optional.
Fedora now has a edk2 package so use the OVMF code from there. This also
adds using a copy of OVMF_VARS for each boot instead of reusing the one
provided by the package.
There's no reason to require the initramfs when we can rebuild it using
the version from the kernel. This adds handling of missing initramfs so
that lmc kickstarts can remove it from the squashfs, saving about 40M on
the iso.
umount tries to delete a mountpoint if it has lorax.imgutils in the
path. This doesn't work right if you try to umount something mounted
deeper on the path.
This adds a delete option, which is True by default, to skip the delete.
When using the template install command copying the same file to itself
shouldn't crash. Just log the error and continue.
Also copy the s390 configuration files for use with livemedia-creator
Resolves: rhbz#1269213
Allow the template to select a different compression type or arguments
for the installimg command.
On 32bit builds running inside a mock xz sees the full amount of system
memory which can result in xz failing with a memory error. This allows
the template to limit the amount of memory it tries to use.
If there isn't enough space for DNF to download packages it will log:
"Not enough disk space to download the packages."
So add this to the messages in monitor that trigger an error.
This makes package selection a little more roundabout, but it allows for
unused packages (and their dependencies) to be removed from globs during
the install phase.
dnf.subject.Subject is the class used by dnf's Base.install to select
packages, so the behavior of installpkg without --except options is the
same as it was before.
When using no-virt the runtime filesystem size comes from the kickstart.
For virt installs lmc was creating a runtime filesystem that was just
slightly larger than the space used by the files installed by anaconda.
This can run into problems with larger filesystem. It is also
inconsistent behavior between virt and no-virt installations.
With this commit the virt runtime filesystem will also come from the
kickstart.
This allows lorax to support multiple templates.
If there is no templates.d under the sharedir (/usr/share/lorax or the
directory passed by --sharedir) then the templates in that directory
will be used as they were previously.
If there are directories under templates.d the first one will be used,
unless --sharedir points to a specific one.
Use 4k blocks for the ext4 filesystem. Run fsck on the filesystem to
make sure deleted blocks are actually zeroed, and pass -Xbcj to
mksquashfs.
4k blocks and -Xbcj decreases the size by 2-6% depending on the
filesystem size. Zeroing the blocks of the ext4 fs improves things
dramatically. The problem is that DNF downloads the rpms before
installing them. In addition to forcing us to use a larger filesystem
than we would like it leaves data that is difficult to compress on the
image. The downloaded files are removed, but need to be zeroed out so
that mksquashfs can compress it.
Sometimes debugging a boot.iso requires using gdb, and finding the
corresponding debuginfo packages can be difficult. This writes the
matching -debuginfo package names and full ENVR to a file on the iso.
This can then be fed to dnf to install the correct debug packages.
Sometimes you don't want to include the selinux xattrs in the tar (eg.
bsdtar has problems extracting them). They are still included by
default, but pass selinux=False to remove '--selinux --acls --xattrs'
from the tar cmdline.
This allows the partition to be mounted on a directory underneath the
temporary directory, eg. /rootfs/, to help support creating other image
types without needed to move the files around.
When running the transaction in a separate process it crashes if you use
a https repo source. There's really no need for threads or processes in
lorax so drop it.
Also switched to using the DNF TransactionProgress API for progress
reporting.
The system the image boots on will likely not match the host where lorax
was run, and in some cases this can cause systems to hang.
Resolves: rhbz#1258498
fedup is deprecated and abandoned. Let's save time and disk by not
building `upgrade.img` when nothing is going to use it anymore.
For the record, performing upgrades using an initramfs from the new
system turns out to be fragile and hard to support:
* dracut initramfs isn't generic enough to handle booting all systems
(e.g. missing vconsole.conf means you get keymaps wrong, so users
can't unlock encrypted disks)
* The ABI differences between the two versions of plymouth, systemd,
etc. requires nasty workarounds at best and causes nightmarish
systemd crashes at worst
This patch removes all the code that built and installed `upgrade.img`.
For backwards compatibility, the API retains the `doupgrade` keyword
argument, and the `--noupgrade` flag is still accepted.
Some callers expect CalledProcessError.output to have the output, so
pass up the stdout + stderr output.
This means failed runcmd template commands will log to program.log and
lorax.log
After the cleanup step, check that everything in /usr/bin and /usr/sbin
can still run. Currently, this just checks that ELF files have
everything they need to link, and scripts have an interpreter.
Verifying is on by default but can be skipped with --noverify
When this is too small the rootfs can run into problems when used with a
live system. Doubling it leaves enough space for the system to run
properly during the installation and since it's all compresses it
doesn't make the image noticeably bigger.
