Commit Graph

1066 Commits

Author SHA1 Message Date
Brian C. Lane 0485ef22f2 Add a context manager for dracut
dracut needs a reasonable chroot environment in order to run correctly.
This adds the DracutChroot context manager that sets up and tears down
mounts inside the root directory tree.

Switch to using it in creator.rebuild_initrds_for_live() and
TreeBuilder.rebuild_initrds()

(cherry picked from commit 322a810403)

Resolves: rhbz#1982271
2021-07-14 11:40:00 -07:00
Brian C. Lane d70f0629f9 mkksiso: cmdline should default to empty string
Related: rhbz#1975844
2021-06-25 13:15:15 -07:00
Brian C. Lane 5fa949a423 livemedia-creator: Check for mkfs.hfsplus
mkfs.hfsplus may not be installed so instead of crashing near the end of
the install check for it and print a useful error message.

Resolves: rhbz#1969743
2021-06-10 11:13:48 -07:00
Radek Vykydal 17f4a79207 Add prefixdevname to Anaconda initramfs
Related: rhbz#1958173
2021-05-24 08:21:54 -07:00
Brian C. Lane 65cb81e7de livemedia-creator: Use inst.ks on cmdline for virt
Related: rhbz#1955674
2021-05-05 11:26:09 -07:00
Brian C. Lane 17cb4f879f image-minimizer: Fix decode() usage
Related: rhbz#1955674
2021-05-05 11:26:09 -07:00
Brian C. Lane b240c9bf1a livemedia-creator: RHEL9 only supports qemu-kvm
Return an error if --arch is passed, and drop the mapping to
qemu-system-<arch> from installer.py and replace it with
/usr/libexec/qemu-kvm

Resolves: rhbz#1955674
2021-05-05 11:26:09 -07:00
Brian C. Lane 980e628835 treebuilder: Add branding package to template variables
The branding package name doesn't always match the product name.  This
saves the branding package names as discovered in the enabled repos and
exposes it to the templates as branding.release and branding.logos --
which could potentially be None so the template needs to take that into
account.

Related: rhbz#1956205
2021-05-05 11:19:09 -07:00
Brian C. Lane d16e2579d5 composer-cli: Remove all traces of composer-cli
weldr-client has replaced composer-cli so remove all of the code and
tests, adjust various things so they don't expect it to be available,
and rename some things like test/composertest.py to reflect its
exclusive use by lorax.

(cherry picked from commit b75b692607)

Resolves: rhbz#1952978
2021-04-29 11:13:51 -07:00
Brian C. Lane e1caa5624e Remove LD_PRELOAD libgomp.so.1 from lmc --no-virt
The libgomp bug rhbz#1722181 has been closed since August.

(cherry picked from commit b0318efead)
2021-02-08 11:00:23 -08:00
Brian C. Lane 929d2585eb Add POSTIN scriptlet error to the log monitor list
This will cause livemedia-creator to terminate anaconda and exit when an
install hits an error that often causes anaconda to get stuck and not
exit.

Resolves: rhbz#1900596
(cherry picked from commit a33efe7c51)
2021-02-08 11:00:23 -08:00
Brian C. Lane e6a2cc605c Improve lmc no-virt error handling
When monitoring log output in livemedia-creator --no-virt it could get
stuck if the output from anaconda stops for some reason.

This changes execReadlines so that it will only read output when it is
available, will monitor the process state, and continue to call the
callback function.

It also adds a final timeout on proc.communicate() so that if Anaconda
becomes stuck and won't exit livemedia-creator will eventually exit.

When the no-virt callback terminates anaconda on an error it now sends a
TERM signal to all of the unshare process' children because just sending
it to unshare doesn't cause anaconda to exit.

(cherry picked from commit 6400515880)
2021-02-08 11:00:23 -08:00
Vojtech Trefny d8c125fec2 Do not use '--loglevel' option when running Anaconda
This option has been removed, see https://github.com/rhinstaller/anaconda/pull/2864
2020-12-02 11:36:30 -08:00
Brian C. Lane eff7ab4009 Update the default release version to 34 2020-12-02 11:33:14 -08:00
Brian C. Lane 67d16ebc32 Switch to using upstream mk-s390image for s390 cdboot.img creation
mk-s390-cdboot has stopped working because the kernel outgrew the
hard-coded offset it used when creating cdboot.img. IBM now has a script
in s390utils that can do the same thing so use the upstream script
instead.

This drops mk-s390-cdboot script, switches the s390 templates to use
mk-s390image from s390utils.

It adds @ROOT@ to cdboot.prm, and sets inst.stage2 so that the installer
image will be found when booting the iso.

Resolves: rhbz#1891778
2020-12-02 11:33:14 -08:00
Brian C. Lane 4d14c4bed0 lorax: Strip ' from product cmdline argument
Temporary work around pungi issue:
https://pagure.io/pungi/pull-request/1463
2020-12-02 11:11:12 -08:00
Stephen Gallagher c14e6b627c Change rootfs default size to 3GiB
Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
2020-12-02 11:11:02 -08:00
Brian C. Lane 46ab9e55f5 composer: Fix open file warnings 2020-10-07 10:51:41 -07:00
Brian C. Lane abbd86a2f6 ltmpl: Fix deprecated escape in docstring 2020-10-07 10:51:41 -07:00
Brian C. Lane c3c3605b70 Cleanup imgutil open files and processes
Because of the way that compress pipes things between processes it would
leave open file descriptors, and running processes around for a bit.
This waits for them to exit and closes the fds before returning.
2020-10-07 10:51:41 -07:00
Brian C. Lane 9d6211f1b3 Fix unclosed files
Python will eventually close open files, but it is recommended to
explicitly close them instead of waiting for the gc or program exit.

This fixes all the uses of open...read/write in the codebase, mostly in
tests.
2020-10-07 10:51:41 -07:00
Brian C. Lane e3e8ff884b doc: Add Blueprint documentation and example to composer-cli.rst 2020-10-01 15:06:14 -07:00
Brian C. Lane 7616a10373 Remove lorax-composer, it has been replaced by osbuild-composer
Remove the code, related files, and tests.
2020-09-30 15:42:46 -07:00
Brian C. Lane b7e69596e3 composer-cli: Make start-ostree parent and ref optional
This changes the start-ostree command to:

compose start-ostree [--size XXXX] [--parent PARENT] [--ref REF] <BLUEPRINT> <TYPE> [<IMAGE-NAME> <PROFILE.TOML>]

Both of them are optional, and if missing a "" is passed to
osbuild-composer. Also adds more tests for all the various possible
options and removes the provider and profile arguments.
2020-07-23 10:35:10 -07:00
Brian C. Lane d830829929 composer-cli: Add a get_arg function
This is in preperation for adding more optional arguments. Adds a
generic get_arg function, tests for it, and converts get_size to use it.
2020-07-23 10:35:10 -07:00
Brian C. Lane be2c9333fc Add log entry about dracut and /proc 2020-07-17 09:38:13 -07:00
Brian C. Lane 68c26bf87b Skip creating empty /proc/modules for dracut
Creating /proc/modules just quiets one dracut warning, it doesn't
actually effect the initrd that is created.

Now that mock is being run with netsharedpath set to exclude /proc is
causes problems for running lorax inside a mock, so just remove this to
prevent a crash when it doesn't exist.

Related: rhbz#1848201
2020-07-17 09:38:13 -07:00
Michel Alexandre Salim 0b9841d5f7 Fix EFI booting for ISOs generated by `mkksiso`
If the original ISO contains both `efiboot.img` and `macboot.img`,
the current `xorrisofs` invocation does not configure `efiboot.img`
to be bootable, only `macboot.img`.

This fix uses `>=` to compare `efimode` against `EFIBOOT`, since
`MACBOOT` implies `EFIBOOT` as well.

This is similar to the logic for `mkisofs`.

Fixes #1046.

Generated image confirmed to work, when comparing the reports:

  `xorriso -indev $PATH_TO_ISO -report_el_torito cmd`

```
❯ diff -u ~/xorriso-report-{orig,new}.log
--- /home/michel/xorriso-report-orig.log        2020-07-15 20:01:01.405289018 -0700
+++ /home/michel/xorriso-report-new.log 2020-07-15 20:01:19.183555423 -0700
@@ -1,6 +1,6 @@
--volid 'Fedora-S-dvd-x86_64-32'
--volume_date uuid '2020042219432600'
--boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'/home/michel/Downloads/Fedora-Server-netinst-x86_64-32-1.6.iso'
+-volid 'Fedora-32-btrfs'
+-volume_date uuid '2020071602410900'
+-boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'/home/michel/src/github/michel-slm/luks-kickstarts/iso/Fedora-32-WS-20200715-3.iso'
 -boot_image any partition_cyl_align=on
 -boot_image any partition_offset=0
 -boot_image any partition_hd_cyl=64
@@ -14,12 +14,6 @@
 -boot_image any load_size=2048
 -boot_image any boot_info_table=on
 -boot_image any next
--boot_image any efi_path='/images/efiboot.img'
--boot_image any platform_id=0xef
--boot_image any emul_type=no_emulation
--boot_image any load_size=11462656
--boot_image isolinux partition_entry=gpt_basdat
--boot_image any next
 -boot_image any efi_path='/images/macboot.img'
 -boot_image any platform_id=0xef
 -boot_image any emul_type=no_emulation
```

```
❯ diff -u ~/xorriso-report-{orig,new5}.log
--- /home/michel/xorriso-report-orig.log        2020-07-15 20:01:01.405289018 -0700
+++ /home/michel/xorriso-report-new5.log        2020-07-15 20:30:27.571644770 -0700
@@ -1,6 +1,6 @@
--volid 'Fedora-S-dvd-x86_64-32'
--volume_date uuid '2020042219432600'
--boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'/home/michel/Downloads/Fedora-Server-netinst-x86_64-32-1.6.iso'
+-volid 'Fedora-32-btrfs'
+-volume_date uuid '2020071603283100'
+-boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'./iso/Fedora-32-WS-20200715-5.iso'
 -boot_image any partition_cyl_align=on
 -boot_image any partition_offset=0
 -boot_image any partition_hd_cyl=64
@@ -17,7 +17,7 @@
 -boot_image any efi_path='/images/efiboot.img'
 -boot_image any platform_id=0xef
 -boot_image any emul_type=no_emulation
--boot_image any load_size=11462656
+-boot_image any load_size=11460608
 -boot_image isolinux partition_entry=gpt_basdat
 -boot_image any next
 -boot_image any efi_path='/images/macboot.img'
```

Signed-off-by: Michel Alexandre Salim <michel@michel-slm.name>
2020-07-16 08:48:58 -07:00
Brian C. Lane 03ff3bb53a composer-cli: Disable retry counter on connection timeout
urllib3 has a retry system that send the request again when the
connection fails, times out, is redircted, etc. This turns off retries,
which aren't very useful anyway.
2020-06-17 09:52:49 -07:00
Brian C. Lane 65d7397cf2 composer-cli: Change timeout to 5 minutes
In some cases osbuild-composer with an empty cache can take longer than
a minute to depsolve and start a compose.
2020-06-16 16:36:42 -07:00
Brian C. Lane bf1be846e5 composer-cli: Return a better error with no value
And add tests for the get_size function.
2020-06-11 09:36:32 -07:00
Brian C. Lane e9b088068c composer-cli: Add new start-ostree command
This is used to start an ostree build, it is only supported on
ostree-composer, and requires the ostree ref and parent. It may also
include --size and optionally be uploaded.
2020-06-11 09:36:32 -07:00
Brian C. Lane d97687a496 composer-cli: Add support for --size to compose start
osbuild-composer can support user specified image sizes, this adds an
optional argument, after start, to specify the size in MiB. eg.

    composer-cli compose start --size 2048 example-http-server qcow2

This only works when the backend is not 'lorax-composer', when it is the
user will get a warning that it will be ignored.
2020-06-11 09:36:32 -07:00
Brian C. Lane 4b1a4f8f16 composer-cli: Add osbuild-composer to connection failure message 2020-06-01 11:27:57 -07:00
Brian C. Lane 019cac8a7c lorax-composer: Check compose/status for invalid characters 2020-05-28 14:23:57 -07:00
Brian C. Lane 9a76c20c6b lorax-composer: deleting an unknown workspace should return an error
This changes the workspace delete behavior to match osbuild-composer's,
returning an error if the workspace doesn't exist.
2020-05-28 11:57:21 -07:00
Brian C. Lane 74f8cd4f34 lorax-composer: Check for valid characters in the undo commit
Return an error 400 with INVALID_CHARS if the commit characters are not
in the allowed list.
2020-05-28 09:39:23 -07:00
Brian C. Lane f83ae2fed3 mksquashfs: Catch errors with mksquashfs and report them
The host system can run out of space while running mksquashfs, and while
this is logged to program.log it isn't detected by lorax or
livemedia-creator so it will continue running, possibly reporting
unrelated errors and causing confusion.

This adds checks for the return status when calling mksquashfs, logs it
to the log, and either exits or raises an error immediately.
2020-05-07 14:35:55 -07:00
Alexander Todorov 2e585c98f3 Don't use f-string without interpolation
resolves pylint W1309(f-string-without-interpolation)
2020-05-01 10:43:07 -07:00
Brian C. Lane c9c1283c56 lorax: Add --skip-branding cmdline argument
Also document how branding currently works. See docs/lorax.rst

Resolves: rhbz#1826479
2020-04-21 15:57:54 -07:00
Brian C. Lane 91d71b09a4 lorax: Update how the release package is chosen
Previously the release package was chosen by picking the first package
that provides 'system-release' that isn't named generic. This can cause
branding issues with repos containing multiple system-release packages.

This patch changes _install_branding so that it will give preference to
a system-release package that ends with lowercase --variant name. If
there isn't one it will fall back to the previous behavior.

Resolves: rhbz#1472622
2020-03-20 08:28:11 -07:00
Brian C. Lane d6ce9c907d ltmpl: Fix package logging format
epoch wasn't included in the lorax-packages.log, but arch is. And for
the debuginfo the epoch, including 0, *is* included. This should match
the previous output.

Resolves: rhbz#1815000
2020-03-19 14:03:08 -07:00
Brian C. Lane e2e8deb906 lorax: Write package lists in run_transaction
A change in glibc now requires /proc be mounted in order to run mknod
which is needed in order to run rpm from runtime-postinstall.

This drops that code from the template and moves writing the package
list into run_transaction, which already has all of the needed
information to generate the list.

Resolves: rhbz#1812895
2020-03-16 11:51:11 -07:00
Brian C. Lane fe45fa3610 lorax: Catch rootfs out of space failures
It isn't always obvious what happened when the rootfs runs out of space,
especially when using lorax via pungi. So this checks for the out of
space error string when building the runtime image and logs it to the
primary logfile and console as an error with the rootfs size.

eg.
2020-01-20 18:52:58,920: The rootfs ran out of space with size=1
2020-02-05 15:27:41 -08:00
Brian C. Lane 210fe3d8ef lorax: Log dnf solver debug data in ./debugdata/ 2020-01-20 14:04:00 -08:00
Brian C. Lane a0fce98109 Add --dracut-conf cmdline argument to lorax and livemedia-creator
This adds the ability to use a dracut.conf file instead of passing
--dracut-arg on the cmdline multiple times.
2020-01-16 09:26:27 -08:00
Brian C. Lane 5561bbb3bc executils: Drop bufsize=1 from execReadlines
startProgram is running in binary mode, so bufsize=1 is invalid. The
ExecLineReader class already breaks it up into lines using readline() so
there is no need to change the default buffering.
2020-01-15 10:40:17 -08:00
Brian C. Lane 9bb1a0916e Update to_timeval usage to use format_iso8601
GLib.DateTime.to_timeval is deprecated.
2020-01-15 10:40:17 -08:00
Brian C. Lane 0d3cff0dde ltmpl: Update to use collections.abc 2020-01-15 10:40:17 -08:00
Brian C. Lane b69e6d725d lorax-composer: Enable ami on aarch64 2020-01-13 14:55:41 -08:00