This makes package selection a little more roundabout, but it allows for
unused packages (and their dependencies) to be removed from globs during
the install phase.
dnf.subject.Subject is the class used by dnf's Base.install to select
packages, so the behavior of installpkg without --except options is the
same as it was before.
commit 4699c88109 changed how the disk
size is estimated and not all users took into account that the return
value is in MiB.
This would result in qemu based iso installations having a rootfs.img
that was 1024x too large.
Something is causing problems with the ext4 rootfs.img when running with
no-virt inside koji. This results in a failed image that looks good
until you try to boot it.
make_squashfs will now return False if it fails, and make_live_image
will return None (instead of the result path). lmc will exit with a 1
and log an error.
When using no-virt the runtime filesystem size comes from the kickstart.
For virt installs lmc was creating a runtime filesystem that was just
slightly larger than the space used by the files installed by anaconda.
This can run into problems with larger filesystem. It is also
inconsistent behavior between virt and no-virt installations.
With this commit the virt runtime filesystem will also come from the
kickstart.
Switching to using qemu directly allows lmc to be more flexible. It can
now run from inside a mock chroot for creation of all image types,
inculding disk images, and can take advantage of KVM on the host system
if /dev/kvm device is present inside the mock.
It should also be possible to create cross-arch images, but without kvm
available this is likely to be a very slow option.
When running a no-virt installation it was parsing the kickstart url
method and passing it to anaconda using --repo which prevents it from
working with url --mirrorlist method. There is no good reason to do
this, anaconda gets the method directly from the kickstart when it isn't
on the cmdline.
This allows lorax to support multiple templates.
If there is no templates.d under the sharedir (/usr/share/lorax or the
directory passed by --sharedir) then the templates in that directory
will be used as they were previously.
If there are directories under templates.d the first one will be used,
unless --sharedir points to a specific one.
Use 4k blocks for the ext4 filesystem. Run fsck on the filesystem to
make sure deleted blocks are actually zeroed, and pass -Xbcj to
mksquashfs.
4k blocks and -Xbcj decreases the size by 2-6% depending on the
filesystem size. Zeroing the blocks of the ext4 fs improves things
dramatically. The problem is that DNF downloads the rpms before
installing them. In addition to forcing us to use a larger filesystem
than we would like it leaves data that is difficult to compress on the
image. The downloaded files are removed, but need to be zeroed out so
that mksquashfs can compress it.
Instead of reusing --image-name add a new argument to name the iso. This
way the disk image can be given a unique name with --image-name and the
iso can be named something different.
This option removes all the extra build artifacts from --make-iso,
leaving only the boot.iso
It also supports naming of the final iso with --image-name
If the kickstart includes multiple definitions for the same mount point,
the last one defined is used. The current code includes all of them in
size calculation, and the image file that livemedia-creator makes is big
enough to hold all of the partitions, even though the duplicates are
ignored by Anaconda.
Also alias --qcow2 to --image-type=qcow2
This allows --make-disk to be used to create any disk image that
qemu-img supports, not just raw or qcow2. See qemu-img --help for a list
of the supported image types.
Sometimes debugging a boot.iso requires using gdb, and finding the
corresponding debuginfo packages can be difficult. This writes the
matching -debuginfo package names and full ENVR to a file on the iso.
This can then be fed to dnf to install the correct debug packages.
Because livemedia-creator is using a media based installation by default,
no networking is brought up automatically. If then the url installation
method is used, it fails with an unclear reason.
This patch adds a check to raise a clear error if the url installation
method is used insisde the kickstart but no networking is configured.
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>
This could help to keep the disk size down during installation,
if the FS within the VM is also supporting TRIM.
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>
This adds the --repo command which can be added multiple times to point
to dnf .repo files.
--enablerepo and --disablerepo can be used multiple times to control
which repos from the .repo files are actually used for the boot.iso
creation.
--repo can be used instead of --source, or in addition to it.
This requires OVMF to be setup on the system, and for the kickstart to
create a /boot/efi/ partition. You can then use it to create UEFI
bootable partitioned disk images.
Make the metavars useful, not STRING. Simplify some of the error
checking, let the parser handle it. Add type=os.path.abspath to several
path arguments so that relative paths will be converted to absolute
paths when they are processed.
One of the most useful things to override is the path to the templates,
this adds a cmdline option to do that instead of needing to create a
whole configuration file and pass it.
This adds support for creating Vagrant boxes using virt-install. It also
includes an example kickstart that sets up the vagrant user with the
default ssh key.
The default result, without passing --image-name, is in
/var/tmp/vagrant.tar.xz
Sometimes you don't want to include the selinux xattrs in the tar (eg.
bsdtar has problems extracting them). They are still included by
default, but pass selinux=False to remove '--selinux --acls --xattrs'
from the tar cmdline.
This implements the bundle spec from:
https://github.com/opencontainers/specs
It creates a tar with the filesystem under /rootfs/ and includes user
provided config.json and runtime.json files.
This allows the partition to be mounted on a directory underneath the
temporary directory, eg. /rootfs/, to help support creating other image
types without needed to move the files around.
The size estimate was counting the /EFI/BOOT/ contents twice and then
doubling that. Only count things once, then double it for the
System/Library/CoreServices/ copy.
hard-links don't work. With CoreServices hardlinked to /EFI/BOOT/ the
Mac won't boot. With /EFI/BOOT/ hardlinked to CoreServices grub2 cannot
read the config file so there are 2 real copies.
This reduces the image size from 21M to about 12M
When running the transaction in a separate process it crashes if you use
a https repo source. There's really no need for threads or processes in
lorax so drop it.
Also switched to using the DNF TransactionProgress API for progress
reporting.
The system the image boots on will likely not match the host where lorax
was run, and in some cases this can cause systems to hang.
Resolves: rhbz#1258498
fedup is deprecated and abandoned. Let's save time and disk by not
building `upgrade.img` when nothing is going to use it anymore.
For the record, performing upgrades using an initramfs from the new
system turns out to be fragile and hard to support:
* dracut initramfs isn't generic enough to handle booting all systems
(e.g. missing vconsole.conf means you get keymaps wrong, so users
can't unlock encrypted disks)
* The ABI differences between the two versions of plymouth, systemd,
etc. requires nasty workarounds at best and causes nightmarish
systemd crashes at worst
This patch removes all the code that built and installed `upgrade.img`.
For backwards compatibility, the API retains the `doupgrade` keyword
argument, and the `--noupgrade` flag is still accepted.
Some callers expect CalledProcessError.output to have the output, so
pass up the stdout + stderr output.
This means failed runcmd template commands will log to program.log and
lorax.log
After the cleanup step, check that everything in /usr/bin and /usr/sbin
can still run. Currently, this just checks that ELF files have
everything they need to link, and scripts have an interpreter.
Verifying is on by default but can be skipped with --noverify
When this is too small the rootfs can run into problems when used with a
live system. Doubling it leaves enough space for the system to run
properly during the installation and since it's all compresses it
doesn't make the image noticeably bigger.
Add a 'lower' filter to the templates to replace string.lower which no
longer exists. Fix udev_escape, the strings are already unicode, and
drop --chdir from runcmd. It wasn't ever used, and passing cwd to the
new runcmd isn't supported.
Fix up 2to3 complaints. I've decided to do with wrapping list
comprehension inside list() to get the generators to run in several
places instead of list(map( or list(filter( which seem less readable to
me.
If it terminates really badly (e.g. with SIGSEGV), it doesn't report any error,
just doesn't put anything to the queue. So instead of just blindly waiting on
the queue forever, check that the process is still alive if we don't get any
message in long time interval.
I originally added --add-template to support doing something similar
to pungi, which injects content into the system to be used by default.
However, this causes the content to be part of the squashfs, which
means PXE installations have to download significantly more data that
they may not need (if they actually want to pull the tree data from
the network, which is not an unusual case).
What I actually need is to be able to modify *both* the runtime image
and the arch-specific content. For the runtime, I need to change
/usr/share/anaconda/interactive-defaults.ks to point to the new
content. (Although, potentially we could patch Anaconda itself to
auto-detect an ostree repository configured in disk image, similar to
what it does for yum repositories)
For the arch-specfic image, I want to drop my content into the ISO
root.
So this patch adds --add-arch-template and --add-arch-template-var
in order to do the latter, while preserving the --add-template
to affect the runtime image.
Further, the templates will automatically graft in a directory named
"iso-graft/" from the working directory (if it exists).
(I suggest that external templates create a subdirectory named
"content" to avoid clashes with any future lorax work)
Thus, this will be used by the Atomic Host lorax templates to inject
content/repo, but could be used by e.g. pungi to add content/rpms as
well.
I tried to avoid code deduplication by creating a new template for the
product.img bits and this, but that broke because the parent boot.iso
code needs access to the `${imggraft}` variable. I think a real fix
here would involve turning the product.img, content/, *and* boot.iso
into a new template.
pylorax unconditionally calls reset() on the dbo, so provide an empty
method to keep it happy.
The lmc dbo is minimal because it is only used for creating the iso, not
anything related to package installation.
The stage2 image can be either LiveOS/squashfs.img or it can be
images/install.img, adjust the IsoMountpoint for this and rename the
flag to .stage2 instead of .liveos
removekmod GLOB [GLOB...] --allbut KEEPGLOB [KEEPGLOB...]
This can be used to remove kernel modules from under
/lib/modules/*/kernel/ while keeping specific items. This should be
easier than constructing find arguments to select the right things to
save.
It appears that reset+fill_sack will now do the right thing and load the
state of the installed packages. Drop the hack with deleting the object.
Also add a double-check to make sure there really is a list of files
for anaconda-core before we run off and make an image without removing
anything.
--cachedir allows the user to specify where the DNF cache is located.
This doesn't actually appear to do much since dnf erases the cache when
it is done. May be useful in the future.
--workdir sets the top level directory for lorax to use for installing
packages, creating installtree and installroot. Normally a temporary
directory under /var/tmp.
Note that the workdir will *not* be removed if there is an error setting
up the DNF object.
--force skips checking if the output directory exists, allowing things
like pungi to use lorax to place the output next to the repo tree it has
already created.
This is a workaround for a current dnf bug, it doesn't update the state
of the packages after they are installed so we tear down the base dnf
object and create a new one pointing to the installroot.
There is an additional issue with the list of files returned, hawkey and
dnf don't appear to make a distinction between files, dirs and ghosted
dirs like yum did, this can result in too much being removed (eg. all of
/etc/selinux/) so we only remove files not directories.
pylorax users will need to change to using dnf and pass a dnf.Base()
object as the dbo argument instead of a yum object as the yum or ybo
argument. See the lorax script for an example of how to do this.
The lorax cmdline argument --excludepkgs has been removed since dnf
doesn't appear to have any way to support it and packages should be
controlled using templates anyway.
The directory where the --logfile is located is also used for other log
files and for the anaconda logs when using --no-virt. Create the parent
directories if they don't exist.
--make-pxe-live target generate live squashfs and initrd for pxe boot.
Also generates pxe config template.
--make-ostree-live is used for installations of Atomic Host. Additionally to
--make-pxe-live it ensures using deployment root instead of physical root of
installed disk image where needed. Atomic installation needs to be virt
installation with /boot on separate partition (the only way supported by
Anaconda currently). Content of boot partition is added to live root fs so that
ostree can find deployment by boot configuration.
Previously if there was an error during a novirt installation that
didn't exit the process there was no way to detect it. This uses the new
--remotelog option for anaconda to monitor the logs for errors using the
same criteria as it does when monitoring a virt install. If there is an
error the anaconda process will be terminated and the logs will be
gathered up into ./anaconda/
Recently, Fedora has been trying to do a 3 product split. As part of
that, lorax was changed to do "installpkg lorax-product-*" via
provides.
I think that approach is awkward; a much simpler approach is to simply
specify the product package as input to lorax on the command line, via
external rel-eng scripts.
This patch therefore adds --includepkg (and we should probably add an
option to remove the implicit lorax-product-* glob).
David Shea
Brian C. Lane
Scott K Logan
Fabian Deutsch
Will Woods
Chris Lumens
Vratislav Podzimek
Colin Walters
Radek Vykydal