Commit Graph

223 Commits

Author SHA1 Message Date
Brian C. Lane
ca2eb38d92 Always update repo metadata when building an image
When the kickstart is handed off to Anaconda for building it will
download its own copy of the metadata and re-run the depsolve. So if the
dnf cache isn't current there will be a mismatch and the build will
fail to find some of the versions in final-kickstart.ks

This adds a new context to DNFLock, .lock_check, that will force a check
of the metadata. It also implements its own timeout and forces a
refresh of the metadata when that expires because the dnf expiration
doesn't always work as expected.

Resolves: rhbz#1631561
2018-10-01 14:50:11 -07:00
Adam Williamson
d1aa8676ab Ditch all use of pyanaconda's simpleconfig
lorax uses pyanaconda's SimpleConfigParser in three different
places (twice with a copy that's been dumped into pylorax, once
by importing it), just to do a fairly simple job: read some
values out of /etc/os-release. The only value SimpleConfigParser
is adding over Python's own ConfigParser here is to read a file
with no section headers, and to unquote the values. The cost is
either a dependency on pyanaconda, or needing to copy the whole
of simpleparser plus some other utility bits from pyanaconda
into lorax. This seems like a bad trade-off.

This changes the approach: we copy one very simple utility
function from pyanaconda (`unquote`), and do some very simple
wrapping of ConfigParser to handle reading a file without any
section headers, and returning unquoted values. This way we can
read what we need out of os-release without needing a dep on
pyanaconda or to copy lots of things from it into pylorax.

Resolves: #449
Resolves: #450

Signed-off-by: Adam Williamson <awilliam@redhat.com>

Related: rhbz#1613058
2018-09-04 11:13:20 -07:00
Brian C. Lane
3a9d39f397 Add support for DNF 3.2 module_platform_id config value
This borrows simpleconfig.py from Anaconda to make parsing os-release
easier.

It defaults to platform:el8
2018-08-29 10:48:02 -07:00
Lars Karlitski
39837fb7f8 Set TCP listen backlog for API socket to SOMAXCONN
A value of 1 is too low for heavy users of the API, such as the weldr-web
interface.

This is also systemd's default for sockets it opens. Using lorax-composer with
socket activation already results in a backlog of SOMAXCONN connections.

(cherry picked from commit be5d50e6f3)

Related: rhbz#1613058
2018-08-20 15:19:17 -07:00
Chris Lumens
dbc4d08de4 Run as root/weldr by default.
We need to be root to read the certificates that give access to the
package repos.  Right now, the alternative seems to be changing
permissions on the certs themselves, which seems less good.  We're
running anaconda as root anyway.
2018-08-08 17:07:59 -07:00
Brian C. Lane
619026ff3d Update to use only qemu-kvm
We only have qemu-kvm available, so use that. This also means that there
will not me any support for using qemu with arches that are different
from the host.
2018-07-27 15:10:22 -07:00
Brian C. Lane
d50321239c Log and exit on metadata update errors at startup
A bad system repo can cause lorax-composer to fail to start. Instead of
a traceback log the error and exit.

(note that the exit still results in an OSError traceback due to part of
it running as root, this needs to be addressed in another commit).

(cherry picked from commit 49380b4b49)
2018-07-19 11:01:08 -07:00
Brian C. Lane
3f054d6a3a Check the compose templates at startup
Depsolve the packages included in the templates and report any errors
using the /api/status 'msgs' field. This should help narrow down
problems with package sources not being setup correctly.

(cherry picked from commit d92f2f5b04)
2018-07-13 10:17:08 -07:00
Brian C. Lane
a1905c3b4b Add --dracut-arg support to lorax
Use it to override the default dracut arguments (displayed as part of
the --help output). If you want to extend the default arguments they
all need to be passed in on the cmdline as well. eg.

--dracut-arg='--xz' --dracut-arg='--install /.buildstamp' ...

Resolves: rhbz#1452220
(cherry picked from commit d8ce013a2b)
2018-06-07 09:01:42 -07:00
Brian C. Lane
b9fe90000e gevent has deprecated .wsgi, should use .pywsgi instead
https://github.com/gevent/gevent/blob/master/doc/api/gevent.wsgi.rst
(cherry picked from commit c9ca451568)
2018-06-05 16:58:25 -07:00
Brian C. Lane
55b89d1aeb Add support for systemd socket activation
Instead of enabling lorax-composer.service enable lorax-composer.socket
and it will start lorax-composer on first access to
/run/weldr/api.socket
2018-05-17 10:34:35 -07:00
Brian C. Lane
e7f9028fca Move lorax-composer and composer-cli argument parsing into modules
This allows sphinx-argparse to document them automatically.
2018-05-17 10:34:35 -07:00
Brian C. Lane
611f5e62f3 Add --proxy to lorax-composer cmdline
Overrides the [dnf] proxy setting in the config file.
2018-05-17 10:34:35 -07:00
Brian C. Lane
d9c4a6c951 Pass the --tmp value into run_creator and cleanup after a crash
Crashing can sometimes leave directories in /var/tmp/lmc-* so clean
those up after run_creator is finished.
2018-05-17 10:34:35 -07:00
Brian C. Lane
559b66b7e3 Add --tmp to lorax-composer and set default tempdir
It was using /tmp/ which can fill up quickly when building images.
Default to /var/tmp unless the user passes --tmp /tmp/
2018-05-17 10:34:35 -07:00
Brian C. Lane
b6e0ba2e45 Add selinux check to lorax-composer
anaconda needs to have SELinux set to disabled or permissive in order to
run correctly. Check at startup and exit with an error.
2018-05-17 10:34:34 -07:00
Brian C. Lane
cb0158ca22 Add lorax-composer and the composer kickstart templates 2018-05-17 10:34:34 -07:00
Brian C. Lane
7fa009c66e Move core of livemedia-creator into pylorax.creator
This moves everything except the cmdline checking into run_creator in
pylorax.creator

It also rearranges some functions to prevent import loops, and adds a
utility function to imgutils (mkfsimage_from_disk for copying a
partition into a filesystem image).
2018-05-17 10:34:34 -07:00
Brian C. Lane
fc56671638 livemedia-creator: Move core functions into pylorax modules
This reduces the amount of code in livemedia-creator to the cmdline
parsing and calling of the installer functions. Moving them into other
modules will allow them to be used by other projects, like the
lorax-composer API server.
2018-05-17 10:34:34 -07:00
Brian C. Lane
39673941a8 Check selinux state before creating output directory
This closes issue #163

(cherry picked from commit dc348fac25)
2018-05-17 10:34:34 -07:00
Brian C. Lane
3b1933b400 Update default releasever to Fedora 28 2018-03-15 13:52:20 -07:00
Yuval Turgeman
68d1f4208d LMC: Make the QEMU RNG device optional
Added a new QEMU argument '--with-rng' to specify a RNG device other
than /dev/random or "none".

Signed-off-by: Yuval Turgeman <yturgema@redhat.com>
2018-01-23 08:22:38 -08:00
Brian C. Lane
e3dcb1e329 Add version output (#1335456)
Add -V to livemedia-creator, and log the running version when lorax or
lmc is started.
2017-08-14 13:54:02 -07:00
Brian C. Lane
cc56a5b5c9 Add --noverifyssl to lorax (#1430483)
Previously lorax had no way to use repos with self-signed certificates.
This adds the --noverifyssl cmdline option which will ignore certificate
errors.

Resolves: rhbz#1430483
2017-03-08 13:27:22 -08:00
Yuval Turgeman
46277f36db Fixed NameError on result_dir when calling with --image-only
Signed-off-by: Brian C. Lane <bcl@redhat.com>
2017-01-13 13:38:33 -08:00
Pat Riehecky
7ca356845b Added option to remove packages (parallel to installpkgs) 2016-11-14 09:06:10 -08:00
Igor Gnatenko
ef5475e552 adapt to DNF 2.0 API changes
References: https://bugzilla.redhat.com/show_bug.cgi?id=1380566
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2016-09-30 09:09:41 -07:00
Brian C. Lane
37d36daa9c lorax: Add --rootfs-size (#1368743)
This controls how big the root filesystem is for the squashfs used in
the boot.iso, the default is 2GiB.

Note that larger rootfs sizes will require more memory and may cause the
build to fail.
2016-08-22 17:20:48 -07:00
Friedrich Schöller
c60338464b Allow supplying a disk image for PXE live systems
Fixes that the combination of command line arguments
"--make-ostree-live" and "--disk-image" lead to a program error.
2016-08-12 13:12:54 -07:00
Brian C. Lane
748259619b Make sure cmdline config file exists (#1348304)
If the --config file doesn't exist lorax will fall back to its defaults,
which are probably not what you want. Now it will exit if it doesn't
exist.
2016-06-20 16:13:45 -07:00
Brian C. Lane
5ef7c093b1 Stop using undocumented DNF logging API
DNF doesn't want users to access base.logging anymore.

Lorax already takes over the "dnf" logger and directs it to ./dnf.log,
so it wasn't really being used.

This raises the debug level to DNF's custom DDEBUG, and sets it up so
that dnf.librepo.log and hawkey.log are next to dnf.log
2016-06-03 16:44:30 -07:00
Brian C. Lane
3485fe462c livemedia-creator: Always copy novirt logs before cleanup
Before attempting to cleanup any dangling anaconda mounts copy the
anaconda logs to their final location.

Also, catch failures to cleanup the mounts, log it, and continue trying
the other mountpoints. A cleanup failure will result in an InstallError
instead of a CalledProcessError.
2016-06-01 11:48:41 -07:00
Brian C. Lane
551b27a8fa Update lmc UEFI support to use the edk2-ovmf package
Fedora now has a edk2 package so use the OVMF code from there. This also
adds using a copy of OVMF_VARS for each boot instead of reusing the one
provided by the package.
2016-05-17 16:44:22 -07:00
Brian C. Lane
2f66212c0f livemedia-creator: Update make-pxe-live to support missing initramfs
In some cases the initramfs may not be present in /boot to save space.
Use it if present, otherwise use the kernel version to recreate the name
of it.

This also fixes problems with dracut running out of space when not using
--live-rootfs-keep-size
2016-05-05 08:22:59 -07:00
Brian C. Lane
9b151ec2bd Update ostree boot handling
This makes sure the contents of /boot are at the expected locations in
/boot and in sys_root. For partitioned images it mounts the separate
/boot partition on /boot. For both fsimage and partitioned images ir
binf mounts it to sys_root so that the kernel+initrd can be found.

The boot directory isn't always named boot.0, so wildcard it and let the
count check handle failure if there is more than 1.
2016-04-29 16:09:03 -07:00
Brian C. Lane
6ed2aaed69 Refactor PXE live creation code
This moves the mounting inside make_live_images so that it isn't trying
to copy the fsimage while having it mounted.
2016-04-29 16:08:57 -07:00
Brian C. Lane
efb0cce9b8 Change --make-pxe-live --no-virt use a fsimage
Instead of a partitioned disk image. This will allow a single pass
of lmc to be used to create an ostree live PXE image.
2016-04-29 16:08:54 -07:00
Brian C. Lane
b72b41e690 Allow ostreesetup kickstart
Also make sure that a --make_pxe_live --no-virt doesn't use autopart or
other mountpoints.
2016-04-29 16:08:50 -07:00
Brian C. Lane
607d7c1eeb livemedia-creator: Make sure make-iso kickstart includes dracut-live
iso creation requires the dracut-live package, otherwise rebuilding the
initrd will crash. Since it takes a long time to discover let's fail
early.
2016-04-06 17:10:58 -07:00
Brian C. Lane
bae111d5a3 livemedia-creator: Simplify cleanup for no-virt
If an anaconda no-virt run crashes it can leave things mounted under
/mnt/sysimage. Previously anaconda-cleanup was used to handle this, but
it will also try to cleanup host mountpoints which isn't desired.
2016-04-06 17:04:42 -07:00
Brian C. Lane
f753a064b8 livemedia-creator: Use correct suffix on default image names (#1318958)
When an image name hasn't been passed, and the compression type is
something other than xz, the default image name should use the user
specified compression suffix.

Resolves: rhbz#1318958
2016-03-29 09:56:32 -07:00
Brian C. Lane
3740df3756 livemedia-creator: Pass -Xbcj to mksquashfs
Some cases of mksquashfs were not using -Xbcj when it is available for
the arch. This adds a function to return the correct args based on the
arch and the cmdline args.
2016-03-29 09:48:33 -07:00
Brian C. Lane
b91e79d9bc livemedia-creator: Check selinux state and exit
lmc --no-virt was switching selinux to permissive if it was enforcing
and restore it when done. This works fine when it is the only session
running, but would cause problems if it was run in parallel.

It now only checks the state and exits with an error if it isn't already
disabled or in Permissive mode.

Users will need to run setenforce 0 before running lmc.
2016-03-28 16:40:16 -07:00
Brian C. Lane
0b9077ceaa Move argument parsers into pylorax.cmdline
This will make it possible to use sphinx-argparse to generate all the
documentation and manual pages.
2016-03-21 14:21:55 -07:00
Brian C. Lane
6798f7c5ec livemedia-creator: Fix off by 1024 error
commit 4699c88109 changed how the disk
size is estimated and not all users took into account that the return
value is in MiB.

This would result in qemu based iso installations having a rootfs.img
that was 1024x too large.
2016-03-21 12:14:28 -07:00
Brian C. Lane
3e9efdcf48 livemedia-creator: Change fsck.ext4 discard failures to errors
Something is causing problems with the ext4 rootfs.img when running with
no-virt inside koji. This results in a failed image that looks good
until you try to boot it.

make_squashfs will now return False if it fails, and make_live_image
will return None (instead of the result path). lmc will exit with a 1
and log an error.
2016-03-09 16:41:11 -08:00
Brian C. Lane
4699c88109 livemedia-creator: Create runtime using kickstart partition size
When using no-virt the runtime filesystem size comes from the kickstart.
For virt installs lmc was creating a runtime filesystem that was just
slightly larger than the space used by the files installed by anaconda.
This can run into problems with larger filesystem. It is also
inconsistent behavior between virt and no-virt installations.

With this commit the virt runtime filesystem will also come from the
kickstart.
2016-03-09 16:41:11 -08:00
Brian C. Lane
a72e9aed0c livemedia-creator: Bump default releasever to 25 2016-03-04 08:57:55 -08:00
Brian C. Lane
f8316a7b89 livemedia-creator: Use qemu instead of virt-install
Switching to using qemu directly allows lmc to be more flexible. It can
now run from inside a mock chroot for creation of all image types,
inculding disk images, and can take advantage of KVM on the host system
if /dev/kvm device is present inside the mock.

It should also be possible to create cross-arch images, but without kvm
available this is likely to be a very slow option.
2016-03-03 17:22:50 -08:00
Brian C. Lane
7c0a6dffd6 livemedia-creator: Stop passing --repo to anaconda (#1304802)
When running a no-virt installation it was parsing the kickstart url
method and passing it to anaconda using --repo which prevents it from
working with url --mirrorlist method. There is no good reason to do
this, anaconda gets the method directly from the kickstart when it isn't
on the cmdline.
2016-02-12 10:22:52 -08:00