Commit Graph

822 Commits

Author SHA1 Message Date
Brian C. Lane
710e65fa6f mount: Add s390x support to IsoMountopoint
kernel and initrd are under images/ on the iso.

Resolves: rhbz#2028089
2021-12-07 10:33:11 -08:00
Brian C. Lane
6160d340c4 Fix monitor problem with split UTF8 characters
Sometimes, depending on the buffering, or length of data being received,
the end of the data could be the 1st byte or so of a UTF8 character.
This would cause a crash when trying to decode the raw data buffer.

This switches it to only decode once a full line has been found.
It also adds tests for the LogMonitor class.
2021-11-17 15:37:59 -08:00
Brian C. Lane
c4aba2e47f mount: Switch to using pycdio instead of pycdlib
Also add a root only test for IsoMountpoint.
2021-11-09 12:02:54 -08:00
Brian C. Lane
87f9f77e24 Move default releasever into pylorax DEFAULT_RELEASEVER
This makes it easier to update in one place, and hopefully not forget
for new releases.

Also update default platform id to f36.
2021-11-08 14:41:26 -08:00
Brian C. Lane
c811ba98d4 dnfbase: Handle defaults better 2021-10-28 14:19:50 -07:00
Brian C. Lane
1adbeef14c ltmpl: Add version compare support to installpkg
This adds support for enforcing version requirements on installed
packages. See the documentation in ltmpl.installpkg for details.
2021-10-28 14:19:50 -07:00
Brian C. Lane
a3810024ac Add fstrim to disk and filesystem image creation
Tests show that it may reduce disk usage a bit vs. just doing the
fallocate --dig-holes.
2021-09-24 15:11:44 -07:00
Jiri Kucera
35cb8f71a7 mount.py: Fix docstring 2021-06-15 15:50:38 -07:00
Brian C. Lane
ea72ace122 pylorax: Fix mksparse ftruncate size handling
Make sure that os.ftruncate() is called with an int() size. Before python
3.10 it used to handle this internally, but in 3.9 it was deprecated and
in 3.10 it now raises an error:

    TypeError: 'float' object cannot be interpreted as an integer

This makes sure that the size is truncated to an int(). The value is in
bytes, so truncation does not lose anything.
2021-06-11 09:28:16 -07:00
Brian C. Lane
322a810403 Add a context manager for dracut
dracut needs a reasonable chroot environment in order to run correctly.
This adds the DracutChroot context manager that sets up and tears down
mounts inside the root directory tree.

Switch to using it in creator.rebuild_initrds_for_live() and
TreeBuilder.rebuild_initrds()
2021-05-25 09:00:18 -07:00
Brian C. Lane
9cb34c5520 treebuilder: Add branding package to template variables
The branding package name doesn't always match the product name.  This
saves the branding package names as discovered in the enabled repos and
exposes it to the templates as branding.release and branding.logos --
which could potentially be None so the template needs to take that into
account.

Related: rhbz#1956205
2021-05-05 11:16:31 -07:00
Brian C. Lane
79a1fd9c46 livemedia-creator: Use inst.ks on cmdline for virt 2021-05-04 15:17:55 -07:00
Brian C. Lane
b75b692607 composer-cli: Remove all traces of composer-cli
weldr-client has replaced composer-cli so remove all of the code and
tests, adjust various things so they don't expect it to be available,
and rename some things like test/composertest.py to reflect its
exclusive use by lorax.
2021-04-26 15:59:35 -07:00
Brian C. Lane
b0318efead Remove LD_PRELOAD libgomp.so.1 from lmc --no-virt
The libgomp bug rhbz#1722181 has been closed since August.
2020-12-15 15:01:24 -08:00
Brian C. Lane
a33efe7c51 Add POSTIN scriptlet error to the log monitor list
This will cause livemedia-creator to terminate anaconda and exit when an
install hits an error that often causes anaconda to get stuck and not
exit.

Resolves: rhbz#1900596
2020-12-15 15:01:24 -08:00
Brian C. Lane
6400515880 Improve lmc no-virt error handling
When monitoring log output in livemedia-creator --no-virt it could get
stuck if the output from anaconda stops for some reason.

This changes execReadlines so that it will only read output when it is
available, will monitor the process state, and continue to call the
callback function.

It also adds a final timeout on proc.communicate() so that if Anaconda
becomes stuck and won't exit livemedia-creator will eventually exit.

When the no-virt callback terminates anaconda on an error it now sends a
TERM signal to all of the unshare process' children because just sending
it to unshare doesn't cause anaconda to exit.
2020-12-15 15:01:24 -08:00
Vojtech Trefny
78eec89cda Do not use '--loglevel' option when running Anaconda
This option has been removed, see https://github.com/rhinstaller/anaconda/pull/2864
2020-11-18 11:18:06 -08:00
Brian C. Lane
cd77f6d4d5 Update the default release version to 34 2020-11-02 13:13:36 -08:00
Brian C. Lane
abbd86a2f6 ltmpl: Fix deprecated escape in docstring 2020-10-07 10:51:41 -07:00
Brian C. Lane
c3c3605b70 Cleanup imgutil open files and processes
Because of the way that compress pipes things between processes it would
leave open file descriptors, and running processes around for a bit.
This waits for them to exit and closes the fds before returning.
2020-10-07 10:51:41 -07:00
Brian C. Lane
9d6211f1b3 Fix unclosed files
Python will eventually close open files, but it is recommended to
explicitly close them instead of waiting for the gc or program exit.

This fixes all the uses of open...read/write in the codebase, mostly in
tests.
2020-10-07 10:51:41 -07:00
Brian C. Lane
7616a10373 Remove lorax-composer, it has been replaced by osbuild-composer
Remove the code, related files, and tests.
2020-09-30 15:42:46 -07:00
Brian C. Lane
be2c9333fc Add log entry about dracut and /proc 2020-07-17 09:38:13 -07:00
Brian C. Lane
68c26bf87b Skip creating empty /proc/modules for dracut
Creating /proc/modules just quiets one dracut warning, it doesn't
actually effect the initrd that is created.

Now that mock is being run with netsharedpath set to exclude /proc is
causes problems for running lorax inside a mock, so just remove this to
prevent a crash when it doesn't exist.

Related: rhbz#1848201
2020-07-17 09:38:13 -07:00
Brian C. Lane
019cac8a7c lorax-composer: Check compose/status for invalid characters 2020-05-28 14:23:57 -07:00
Brian C. Lane
9a76c20c6b lorax-composer: deleting an unknown workspace should return an error
This changes the workspace delete behavior to match osbuild-composer's,
returning an error if the workspace doesn't exist.
2020-05-28 11:57:21 -07:00
Brian C. Lane
74f8cd4f34 lorax-composer: Check for valid characters in the undo commit
Return an error 400 with INVALID_CHARS if the commit characters are not
in the allowed list.
2020-05-28 09:39:23 -07:00
Brian C. Lane
f83ae2fed3 mksquashfs: Catch errors with mksquashfs and report them
The host system can run out of space while running mksquashfs, and while
this is logged to program.log it isn't detected by lorax or
livemedia-creator so it will continue running, possibly reporting
unrelated errors and causing confusion.

This adds checks for the return status when calling mksquashfs, logs it
to the log, and either exits or raises an error immediately.
2020-05-07 14:35:55 -07:00
Brian C. Lane
c9c1283c56 lorax: Add --skip-branding cmdline argument
Also document how branding currently works. See docs/lorax.rst

Resolves: rhbz#1826479
2020-04-21 15:57:54 -07:00
Brian C. Lane
91d71b09a4 lorax: Update how the release package is chosen
Previously the release package was chosen by picking the first package
that provides 'system-release' that isn't named generic. This can cause
branding issues with repos containing multiple system-release packages.

This patch changes _install_branding so that it will give preference to
a system-release package that ends with lowercase --variant name. If
there isn't one it will fall back to the previous behavior.

Resolves: rhbz#1472622
2020-03-20 08:28:11 -07:00
Brian C. Lane
d6ce9c907d ltmpl: Fix package logging format
epoch wasn't included in the lorax-packages.log, but arch is. And for
the debuginfo the epoch, including 0, *is* included. This should match
the previous output.

Resolves: rhbz#1815000
2020-03-19 14:03:08 -07:00
Brian C. Lane
e2e8deb906 lorax: Write package lists in run_transaction
A change in glibc now requires /proc be mounted in order to run mknod
which is needed in order to run rpm from runtime-postinstall.

This drops that code from the template and moves writing the package
list into run_transaction, which already has all of the needed
information to generate the list.

Resolves: rhbz#1812895
2020-03-16 11:51:11 -07:00
Brian C. Lane
fe45fa3610 lorax: Catch rootfs out of space failures
It isn't always obvious what happened when the rootfs runs out of space,
especially when using lorax via pungi. So this checks for the out of
space error string when building the runtime image and logs it to the
primary logfile and console as an error with the rootfs size.

eg.
2020-01-20 18:52:58,920: The rootfs ran out of space with size=1
2020-02-05 15:27:41 -08:00
Brian C. Lane
210fe3d8ef lorax: Log dnf solver debug data in ./debugdata/ 2020-01-20 14:04:00 -08:00
Brian C. Lane
a0fce98109 Add --dracut-conf cmdline argument to lorax and livemedia-creator
This adds the ability to use a dracut.conf file instead of passing
--dracut-arg on the cmdline multiple times.
2020-01-16 09:26:27 -08:00
Brian C. Lane
5561bbb3bc executils: Drop bufsize=1 from execReadlines
startProgram is running in binary mode, so bufsize=1 is invalid. The
ExecLineReader class already breaks it up into lines using readline() so
there is no need to change the default buffering.
2020-01-15 10:40:17 -08:00
Brian C. Lane
9bb1a0916e Update to_timeval usage to use format_iso8601
GLib.DateTime.to_timeval is deprecated.
2020-01-15 10:40:17 -08:00
Brian C. Lane
0d3cff0dde ltmpl: Update to use collections.abc 2020-01-15 10:40:17 -08:00
Brian C. Lane
b69e6d725d lorax-composer: Enable ami on aarch64 2020-01-13 14:55:41 -08:00
Dan Horák
c038bb53c6 livemedia-creator: workaround glibc limitation when starting anaconda
On some platforms (aarch64, ppc64le) toolchain limitations/optimizations
can break anaconda startup, as discussed in rhbz#1722181. The workaround
is to preload libgomp.so before starting anaconda.
2020-01-10 11:10:48 +01:00
Brian C. Lane
cd65758ebe anaconda_cleanup: Remove anaconda.pid if it is left behind
When a novirt-install fails or is canceled anaconda may leave its pid
file behind. Add it to the cleanup function.

Resolves: rhbz#1788501
2020-01-07 15:42:16 -08:00
Ondřej Budai
045bddb37a fix typo in api docstring 2019-11-27 12:24:03 +02:00
Brian C. Lane
e365e476af Disable some compose types on other architectures
The 'enabled' field in the /compose/types output now reflects whether or
not the type is supported on the current architecture. Disabled types
are not allowed to be built, and will raise an error like:

Compose type 'alibaba' is disabled on this architecture
2019-10-28 16:42:36 -07:00
Brian C. Lane
af3494fc46 lorax: Drop unused --title option
It has never been hooked up to anything, let alone substitution @TITLE@
(which also hasn't ever been used in the config files).

Closes #863
2019-10-25 11:35:21 -07:00
Brian C. Lane
009c051003 Bump default platform and releasever to 32 2019-10-16 14:22:58 -07:00
Brian C. Lane
3fdc783850 docs: Fix Sphinx errors in docstrings 2019-10-16 13:52:54 -07:00
Brian C. Lane
146560b959 Add upload status examples to compose route docstrings 2019-10-16 09:17:53 -07:00
Brian C. Lane
b23c73e52c Add docstrings to the new upload functions in pylorax.api.queue 2019-10-16 09:17:53 -07:00
Brian C. Lane
d16e8f5bfc Change /compose/uploads/delete to /upload/delete
Remove the requirement to pass the compose uuid and the upload uuid --
only the upload uuid is needed.
2019-10-16 09:17:53 -07:00
Brian C. Lane
0eda252d61 Add profile support to /uploads/schedule/
Pass in the name of a profile in the `profile` field, or pass in
one-time use settings in the `settings` object.
2019-10-16 09:17:53 -07:00