The queue is in /var/lib/weldr/queue/new by default. It watches the
directory for new symlinks (to /var/lib/weldr/results/<dirname>) and
handles running anaconda on the kickstart found in final-kickstart.ks
inside the symlinked directory.
Also move default_image_name into imgutils so it can be used in other
places.
When running from lorax-composer the wait() call wasn't waiting until
the tar was finished. I think this is due to gevent monkey-patching
something. Using communicate() solves this problem.
This drops support for the TCP port and switches to using a socket at
/var/run/weldr/api.socket
Also add the start of some docs for lorax-composer.
--host and --port argument have been removed.
--group sets the group name to use for access to the socket and its
parent directory. Defaults to 'weldr'
--socket sets the full path to the socket to create. Defaults to
'/var/run/weldr/api.socket'
Passing ?branch=<branch-name> will use the specified branch instead of
master.
The new branch will not exist until a /recipes/new?branch=new-branch
POST is made. At that time the branch will be created based on the
current master branch and the new commit will be added to it.
- Fix `projects_depsolve()` to not consider a successful empty response
(rc == 0) as an error.
- Fix recipe_from_dict() to default modules and packages to empty lists
instead of `None`, to avoid a Python-ism in the API for consumers and
stay compatible to the bdcs API.
Fixes#290
Returns a simple string to indicate that the API server is running.
/api/v0/status should be used instead, it provides more detailed info in
JSON format.
This includes a new configuration file at /etc/lorax/composer.conf with
built-in defaults. It also adds a YUMLOCK server config object so that
request handlers can access the yum base object without interfering
with each other.
This requires that the docs be at /usr/share/doc/lorax-*/html/ or if
running from the source tree, at ./docs/html/
They can be re-created by running 'make docs'
Recipe should have its version bumped based on the version from the
previous commit, and not be bumped on the first commit. Fix the code and
the tests.
It appears that with libgit2 v0.24.6 reverse causes it to list them
newest first. In 0.25.1 it lists them oldest first. On both versions
just using SortMode.TIME gives the desired result of oldest first.
when default value is list or dict the default arguments are
instantiated as objects at the time of definition. This is significant
(exposing visible semantics) when the object is mutable. There’s no way
of re-binding that default argument name in the function’s closure. When
function is executed multiple times with its default value the value
will change between executions, possibly leading to strange side effects.
For more information see:
http://satran.in/2012/01/12/python-dangerous-default-value-as-argument.html
The lorax-composer program will launch a BDCS compatible API server
using Flask and Gevent. Currently this is a skeleton application with
only one active route (/api/v0/status).
The API code lives in ./src/pylorax/api/v0.py with related code in other
pylorax/api/* modules.
This reduces the amount of code in livemedia-creator to the cmdline
parsing and calling of the installer functions. Moving them into other
modules will allow them to be used by other projects, like the
lorax-composer API server.
It appears that sometimes the loop device doesn't get setup properly,
this may be a race with other users of loop devices on the system, or
some other mechanism that isn't understood.
To try and prevent total failure when this happens this patch retries
the loop setup 3 times before giving up. Previously it would wait for
the loop device to appear (checking 5 times), that operation is now
executed 3 times with a new losetup attempt each time.
Resolves: rhbz#1589084
This requires OVMF to be setup on the system, and for the kickstart to
create a /boot/efi/ partition. You can then use it to create UEFI
bootable partitioned disk images.
The UEFI firmware needs to be installed manually on the system, either
in the default location of /usr/share/OVMF/ or use --ovmf-path to point
to the location.
Resolves: rhbz#1546715
Resolves: rhbz#1544805
Use it to override the default dracut arguments (displayed as part of
the --help output). If you want to extend the default arguments they
all need to be passed in on the cmdline as well. eg.
--dracut-arg='--xz' --dracut-arg='--install /.buildstamp' ...
Resolves: rhbz#1452220
This can't be done the same way as on master because there is no rpm
database inside the installroot to run rpm -qa against. Do it at the end
of the yum transaction.
Resolves: rhbz#1416155
This uses the --release value as the yum releasever so that $releasever
in a --repo will work.
It also turns on assumeyes so that any gpgkey entries in the .repo file
will be installed and used automatically if gpgcheck is enabled for the
repo.
Related: rhbz#1430479
When multiple units are passed to systemctl and one fails it doesn't
finish the others. Change the template command to call systemctl for
each unit individually.
This also removes the lvm2-activation-generator in runtime-cleanup.tmpl
Resolves: rhbz#1478247
This will allow anaconda to fetch kickstarts using https when installing
with fips=1
Leave vmlinuz and .vmlinuz.hmac in /boot
dracut-fips module needs the vmlinuz.hmac file in order to boot.
Resolves: rhbz#1341280
The previous code used losetup --list -O to return the backing store
associated with the loop device. This can fail due to losetup truncating
the output filename if sysfs isn't setup. Instead of printing the full
path it will truncate it to 64 characters with a * at the end.
See util-linux lib/loopdev.c for the code that does this.
This commit changes it to use the existing get_loop_name function, which
uses losetup -j to lookup the loop device associated with the backing
store which avoids the truncation problem.
Resolves: rhbz#1462150
This makes it easier to specify existing repos with extra args, eg.
/etc/yum.repos.d/redhat.repo generated by subscription-manager.
Resolves: rhbz#1430479
It seems that on rare occasions losetup can return before the /dev/loopX
is ready for use, causing problems with mkfs. This tries to make sure
that the loop device really is associated with the backing file before
continuing.
Resolves: rhbz#1462150
Previously lorax had no way to use repos with self-signed certificates.
This adds the --noverifyssl cmdline option which will ignore certificate
errors.
Resolves: rhbz#1430483
commit 5da9f5c179 cleaned up the resultdir
handling, but didn't take into account that on master shutil.copytree
was switched to imgutil.copytree which doesn't care if the empty
directory exists or not.
Resolves: rhbz#1362157
The cherry-pick for commit 0d28b9e09c
dropped the patch into the middle of setup_logging and added a (unused)
main() function so things would still run, but logging wasn't being
setup properly.
This removes the main() and moves the default_image_name() function
after all of the logging setup code.
Resolves: rhbz#1361031
The correct way with the current dracut (and it should be backwards
compatible) is to reference it as stage2=hd:LABEL=...
(cherry picked from commit 4bc4b4c5a5)
Resolves: rhbz#1355882
If the --config file doesn't exist lorax will fall back to its defaults,
which are probably not what you want. Now it will exit if it doesn't
exist.
Resolves: rhbz#1348302
(cherry picked from commit 748259619b)
When using the template install command copying the same file to itself
shouldn't crash. Just log the error and continue.
Also copy the s390 configuration files for use with livemedia-creator
Resolves: rhbz#1269213
(cherry picked from commit 701ab02619)
When an image name hasn't been passed, and the compression type is
something other than xz, the default image name should use the user
specified compression suffix.
Resolves: rhbz#1318958
(cherry picked from commit f753a064b8)
Recently, Fedora has been trying to do a 3 product split. As part of
that, lorax was changed to do "installpkg lorax-product-*" via
provides.
I think that approach is awkward; a much simpler approach is to simply
specify the product package as input to lorax on the command line, via
external rel-eng scripts.
This patch therefore adds --installpkgs (and we should probably add an
option to remove the implicit lorax-product-* glob).
(cherry picked from commit 52d962d613)
Resolves: rhbz#1272222
The system the image boots on will likely not match the host where lorax
was run, and in some cases this can cause systems to hang.
Resolves: rhbz#1258498
installimg SRCDIR DESTFILE
Create a compressed cpio archive of the contents of SRCDIR and place
it in DESTFILE.
If SRCDIR doesn't exist or is empty nothing is created.
Examples:
installimg ${LORAXDIR}/product/ images/product.img
(cherry picked from commit b064ae6166)
Related: rhbz#1202278
The anaconda rebase changed the repo download failure to ERR, fix this
in the log watching code so that it will shut down the VM when it hits
it.
Related: rhbz#1196721
I originally added --add-template to support doing something similar
to pungi, which injects content into the system to be used by default.
However, this causes the content to be part of the squashfs, which
means PXE installations have to download significantly more data that
they may not need (if they actually want to pull the tree data from
the network, which is not an unusual case).
What I actually need is to be able to modify *both* the runtime image
and the arch-specific content. For the runtime, I need to change
/usr/share/anaconda/interactive-defaults.ks to point to the new
content. (Although, potentially we could patch Anaconda itself to
auto-detect an ostree repository configured in disk image, similar to
what it does for yum repositories)
For the arch-specfic image, I want to drop my content into the ISO
root.
So this patch adds --add-arch-template and --add-arch-template-var
in order to do the latter, while preserving the --add-template
to affect the runtime image.
Further, the templates will automatically graft in a directory named
"iso-graft/" from the working directory (if it exists).
(I suggest that external templates create a subdirectory named
"content" to avoid clashes with any future lorax work)
Thus, this will be used by the Atomic Host lorax templates to inject
content/repo, but could be used by e.g. pungi to add content/rpms as
well.
I tried to avoid code deduplication by creating a new template for the
product.img bits and this, but that broke because the parent boot.iso
code needs access to the `${imggraft}` variable. I think a real fix
here would involve turning the product.img, content/, *and* boot.iso
into a new template.
Resolves: rhbz#1202278
This adds the --qcow2 option to create qcow2 disk images in virt and
no-virt modes. You can pass extra options to qemu-img with --qcow2-arg
(cherry picked from commit b73aeb92a9)
Resolves: rhbz#1210413
removekmod GLOB [GLOB...] --allbut KEEPGLOB [KEEPGLOB...]
This can be used to remove kernel modules from under
/lib/modules/*/kernel/ while keeping specific items. This should be
easier than constructing find arguments to select the right things to
save.
(cherry picked from commit 11c9e0e8ee)
Resolves: rhbz#1230356
Resolves: rhbz#1184021
--make-pxe-live target generate live squashfs and initrd for pxe boot.
Also generates pxe config template.
--make-ostree-live is used for installations of Atomic Host. Additionally to
--make-pxe-live it ensures using deployment root instead of physical root of
installed disk image where needed. Atomic installation needs to be virt
installation with /boot on separate partition (the only way supported by
Anaconda currently). Content of boot partition is added to live root fs so that
ostree can find deployment by boot configuration.
In this case, we don't need to pass a repo parameter to anaconda. It's running
in a VM and knows how to deal with a kickstart file.
Related: rhbz#1184021
What I need is to make something like the traditional DVD which also
includes packages. At present this is apparently handled by the
entirely separate pungi tool.
At the moment for me, it's the least bad option to modify lorax to
inject data from an external source than to create a new tool, or
attempt to also modify pungi to do this.
This would also allow pungi's DVD creation to eventually be a set of
external templates for Lorax.
(cherry picked from commit 66359415be)
Resolves: rhbz#1157777
This is used as a kickstart %post interpreter to streamline
modifications to images.
Also adds an example kickstart.
This Obsoletes the old appliance-tools-minimizer and includes a Provide
so that the transition will be seamless.
(cherry picked from commit 99f2ab9137)
(cherry picked from commit b090a09dca)
Resolves: rhbz#1082642
The aarch64 change to use shim (6907567) also stopped using mixed-case
names for BOOT${efiarch}.efi (so it's always .EFI and ${efiarch} is X64
IA32 AA64 etc. now), and mkefiboot needs to match that.
Related: rhbz#1100048
Incidentally, this should fix are aych bee zee one zero four three two
seven four.
tar recurses into directories by default, but find is feeding it all the
files and directories so the tar it produces is considerably larger than
it needs to be due to duplicate files. Add --no-recursion flag so that
tar will only add the specific files and directories piped to it by find.
Related: rhbz#1144140
This adds the --make-tar option which will produce a xz compressed tar
of the root filesystem. This works with either virt-install or no-virt
modes. Use --image-name to set the output filename.
--compression is used to set the compression type to use, which defaults
to xz. Supported types are xz, lzma, gzip and bzip2.
--compress-arg is used to pass arguments to the compression utility.
(cherry picked from commit d04a99e8f4)
Resolves: rhbz#1144140
If the user supplies a fsimage to use for the iso it may not be on the
same filesystem as the working directory. Try to hardlink first, but if
that fails copy the image.
(cherry picked from commit b3e80b7564)
Related: rhbz#1144140
--make-fsimage was only working with --no-virt, this re-structures
things so that virt-install partitioned disk images can be converted to
a fsimage. --make-ami was actually already doing this, so change it to
use --make-fsimage and set the default image name to "ami-root.img" with
a label of "AMI".
This also adds the ability to set the fs label on iso fsimage and
fsimages created with --make-fsimage and --make-ami by passing
--fs-label, but note that bootable iso's expect the Anaconda label.
(cherry picked from commit 19d8304070)
Related: rhbz#1144140
The 32MiB size limit does not apply to upgrade.img since it's installed
to /boot by redhat-upgrade-tool instead of downloaded through TFTP. The
warning in rebuild_initrds will still be triggered by an upgrade.img
over the limit, but this doesn't halt the compose and it's probably not
a bad thing to know about.
Resolves: rhbz#1069671
basearch is ppc64le so we needs to check for that in addition to ppc64.
Resolves:rhbz#1136490
(cherry picked from commit 72357bf96b6b016c3a39b2af51eaf5cf724a0928)
When running lmc with --no-virt yum will use /tmp/ to cache some things.
If the kickstart or repos change between runs this cache can become
stale so remove it when lmc is finished.
Resolves: 1073502
The ppc config files were missing from the live config_files directory
and ppc needs the correct lib directory so lmc has been switched to use
ArchData driven from the installed kernel arch.
Resolves: rhbz#1102318
When doing an image install there is no .buildstamp file to pull the
information from so use the cmdline variables (or their defaults) and
pass them to anaconda in the environment.
Resolves: rhbz#1067746
Remove more drivers and remove plymouth and drm dracut modules. Only on
PPC64 initrd, all other arches have the full set of drivers and modules.
Resolves: rhbz#1060691
Make sure the data is written before we do anything else with the disk
image. This shouldn't be needed, umount should take care of it, but it
also can't hurt.
Resolves: rhbz#1052175
Use redhat-upgrade-dracut for the RHEL version of the system-upgrade
dracut module. upgrade.img also needs the convertfs module for upgrades
from RHEL6.
Resolves: rhbz#1029999
Also adds a check for a bad url repo, and fix ram calculation for
appliance mode. Updates the README.livemedia-creator documentation.
Resolves: rhbz#1019728
Fix "lorax -V" and add a "version is ???" to the log file so we can easily know
what version of lorax is used to build an iso.
Changed to try/import for versioning because the version file is autogenerated.
We should probably let the user know if something goes wrong with the
transaction or if a scriptlet fails. So: log the messages so we can find
(and, one hopes, fix) them later.
Some package scripts may call utilities using dbus. Since this is just a
chroot that will fail. This unsets DESKTOP and DBUS_SESSION_BUS_ADDRESS
to keep them from crashing.
This adds support for installing to a filesystem image instead of a
partitioned disk image. It requires Anaconda's --dirinstall support.
Also re-organized the code to break it up into smaller methods.
This speeds up iso creation in no-virt mode by removing the need to copy
the filesystem from the partitioned disk image to the filesystem image
that is used to make the squashfs image.
It used to give us (x/y) packagename, but now it only sends package
name. This was changed in yum commit 7e8c76173. For now default to
showing the zeros along with the package name so that it doesn't look
like an error.
kpartx can return before the devices are created. Use -s to wait.
Also remove -p p and let kpartx handle adding pX if needed, we use
whatever it outputs so there is no need to force a name.
This makes findkernels() look for any image named something like:
$PREFIX-$KERNELVER.img
and adds a corresponding entry to its returned data like:
kernel.$PREFIX.path = [path]
As a special backwards-compatibility case we use 'initrd' for the
attribute name if $PREFIX is 'initramfs'.
This gives us any extra initramfs images that may have been built using
the 'prefix' argument to rebuild_initrds().
If 'prefix' is passed to rebuild_initrds(), it will build a *new*
initramfs with a name like $PREFIX-$KERNELVER.img, rather than
overwriting the existing initramfs.
Dracut now makes the initrd with 600 permissions
for security reasons. These reasons do not apply
to install images, and we want the other tools
that use lorax to be able to read the initrd file.
When I switched execution over to execWith* functions I failed to
account for the use of CalledProcessError in various places. This
patch restores that behavior. All places that used check_call or
check_output now pass raise_err=True to the execWith* call.
--location specifies an iso directory tree to be used by virt-install
instead of the iso. This allows you to update the initrd in the tree for
debugging.
virt-install uses the images/pxeboot/ directory for initrd.img and
vmlinux.
An iso is still required for the LiveOS/squashfs.img stage2 file.
live media isn't exactly the same as the Anaconda install media. Right
now this amounts to needing a root= cmdline argument but in the future
there may be other differences.
This also reverts 5437557846 on the new copies of the templates.
Switch to using execWith* so that the command and its output can be
logged. To capture the output setup a logger named "program"
livemedia-creator captures all of this into program.log
- Add a check to livemedia-creator for /usr/bin/virt-install (#841566)
- Suggest the correct package for livemedia-creator's libvirt (#841552)
- Add to list of packages needed to build a livemedia-creator iso (#841594)
The 'systemctl' command can be used to enable, disable, or mask systemd
units inside the runtime being modified. Modify runtime-postinstall.tmpl
to use the 'systemctl' command.
We also no longer remove quota*.service or kexec*.service, since
these aren't enabled by default. And systemd-remount-api-vfs.service
should work correctly now, so we can leave it alone as well.
The '-cmd' functionality depends on the individual lorax template
commands raising errors, so they shouldn't do sys.exit().
Also, capture stderr along with stdout, and put both in the log.
There's something strange going on where unmounting a hfsplus volume
immediately after mounting it will fail with EBUSY.
This makes the umount fail, which makes the rmdir fail, which causes a
traceback, which breaks mkefiboot --apple.
It works fine if you wait a second and retry.. so do that.
Also, add the "lazy" argument so you can do lazy unmounts if you like.
Generally it's not a good idea for python libraries to set up loggers in
the body of the library.
Set up a NullHandler by default (as the logging module suggests), and
add a function to do the current logging setup during run().
From Harald Hoyer <harald@redhat.com>:
pollcdrom is used to poll for the install medium. A lot of CDROM drives
are not polled by the kernel correctly, so we have to actively poll for
the medium.
Some packages are critical to the compose. If --required
is specified in the template's installpkg command, lorax
will exit if the package is not available.
This adds support for creating an appliance description file for the
disk image. Mako templates are used to make it easy to support other
appliance targets. The included example works with virt-image.
Mac boot images are optional. Don't require hfsplus-tools
by default, but warn the user that he needs to install them
if he wants to create mac boot images.
When running with no-virt mode libvirt and virt-install are not needed
so make the import optional and update the usage info reflect that the
virt options are disabled.
There's a small amount of additional metadata required for the Mac boot
images to appear as bootable devices in the startup preferencs, so add
support for generating that.
Signed-off-by: Brian C. Lane <bcl@redhat.com>
Install the anaconda dracut module during 'install', use it when
rebuilding initramfs, and clean it up afterward.
Also install '.buildstamp' into the initramfs (the anconda module wants
it).
AMI images are un-partitioned filesystem images with a grub.conf that
is read by the pv-grub bootloader used by EC2. Most of the actual work
making the AMI is done in the kickstart. This just creates the image
file.
Fedora 17 changes top level directories like /bin, /lib, etc. to
symlinks to the corresponding dirs in /usr/
dracut can convert old systems to the new layout using its convertfs
module.
The installer no longer has access to the initrd's root. We need to
copy any needed files over to /sysroot before switching root. This
copies *.cfg and *.ks files.
It also adds the ability to add dracut hook scripts to the initramfs
from /usr/share/lorax/dracut_hooks/
This re-adds commit af6d4e2c50 which was
lost during the switch to the treebuilder branch.
In the latest method for booting the rootfs is in the LiveOS
directory of the media, not appended to the initrd. Detect this
and mount the iso and pass the CDLABEL to virt-install.
Some repos may contain anaconda packages for more architectures,
so it's not possible to get the right buildarch.
This patch allows optional specifying of the buildarch on the
command line when running lorax.
If the buildarch is not specified manually, lorax tries to get
it from the anaconda package as before.
If there's no anaconda package in the repository, don't fallback
to the system architecture and continue, because this is pointless.
We need the anaconda package.
When considering a package's filelist (e.g. for things like removefrom
--allbut), we might need to also include the %ghost files, so make
_filelist() add them to its returned list.
