Commit Graph

1034 Commits

Author SHA1 Message Date
Michel Alexandre Salim 265303de63 Fix EFI booting for ISOs generated by `mkksiso`
If the original ISO contains both `efiboot.img` and `macboot.img`,
the current `xorrisofs` invocation does not configure `efiboot.img`
to be bootable, only `macboot.img`.

This fix uses `>=` to compare `efimode` against `EFIBOOT`, since
`MACBOOT` implies `EFIBOOT` as well.

This is similar to the logic for `mkisofs`.

Fixes #1046.

Generated image confirmed to work, when comparing the reports:

  `xorriso -indev $PATH_TO_ISO -report_el_torito cmd`

```
❯ diff -u ~/xorriso-report-{orig,new}.log
--- /home/michel/xorriso-report-orig.log        2020-07-15 20:01:01.405289018 -0700
+++ /home/michel/xorriso-report-new.log 2020-07-15 20:01:19.183555423 -0700
@@ -1,6 +1,6 @@
--volid 'Fedora-S-dvd-x86_64-32'
--volume_date uuid '2020042219432600'
--boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'/home/michel/Downloads/Fedora-Server-netinst-x86_64-32-1.6.iso'
+-volid 'Fedora-32-btrfs'
+-volume_date uuid '2020071602410900'
+-boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'/home/michel/src/github/michel-slm/luks-kickstarts/iso/Fedora-32-WS-20200715-3.iso'
 -boot_image any partition_cyl_align=on
 -boot_image any partition_offset=0
 -boot_image any partition_hd_cyl=64
@@ -14,12 +14,6 @@
 -boot_image any load_size=2048
 -boot_image any boot_info_table=on
 -boot_image any next
--boot_image any efi_path='/images/efiboot.img'
--boot_image any platform_id=0xef
--boot_image any emul_type=no_emulation
--boot_image any load_size=11462656
--boot_image isolinux partition_entry=gpt_basdat
--boot_image any next
 -boot_image any efi_path='/images/macboot.img'
 -boot_image any platform_id=0xef
 -boot_image any emul_type=no_emulation
```

```
❯ diff -u ~/xorriso-report-{orig,new5}.log
--- /home/michel/xorriso-report-orig.log        2020-07-15 20:01:01.405289018 -0700
+++ /home/michel/xorriso-report-new5.log        2020-07-15 20:30:27.571644770 -0700
@@ -1,6 +1,6 @@
--volid 'Fedora-S-dvd-x86_64-32'
--volume_date uuid '2020042219432600'
--boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'/home/michel/Downloads/Fedora-Server-netinst-x86_64-32-1.6.iso'
+-volid 'Fedora-32-btrfs'
+-volume_date uuid '2020071603283100'
+-boot_image isolinux system_area=--interval:imported_iso:0s-15s:zero_mbrpt,zero_gpt:'./iso/Fedora-32-WS-20200715-5.iso'
 -boot_image any partition_cyl_align=on
 -boot_image any partition_offset=0
 -boot_image any partition_hd_cyl=64
@@ -17,7 +17,7 @@
 -boot_image any efi_path='/images/efiboot.img'
 -boot_image any platform_id=0xef
 -boot_image any emul_type=no_emulation
--boot_image any load_size=11462656
+-boot_image any load_size=11460608
 -boot_image isolinux partition_entry=gpt_basdat
 -boot_image any next
 -boot_image any efi_path='/images/macboot.img'
```

Signed-off-by: Michel Alexandre Salim <michel@michel-slm.name>
(cherry picked from commit 0b9841d5f7)
2020-07-20 10:31:10 -07:00
Brian C. Lane d34263a885 composer-cli: Disable retry counter on connection timeout
urllib3 has a retry system that send the request again when the
connection fails, times out, is redircted, etc. This turns off retries,
which aren't very useful anyway.
2020-06-18 14:31:33 -07:00
Brian C. Lane 2ff1646e31 composer-cli: Change timeout to 5 minutes
In some cases osbuild-composer with an empty cache can take longer than
a minute to depsolve and start a compose.
2020-06-18 14:31:33 -07:00
Brian C. Lane 7a9ec10fc7 composer-cli: Return a better error with no value
And add tests for the get_size function.
2020-06-10 11:11:35 -07:00
Brian C. Lane abf4bfe4b5 composer-cli: Add new start-ostree command
This is used to start an ostree build, it is only supported on
ostree-composer, and requires the ostree ref and parent. It may also
include --size and optionally be uploaded.
2020-06-09 15:10:02 -07:00
Brian C. Lane aa7b09944d composer-cli: Add support for --size to compose start
osbuild-composer can support user specified image sizes, this adds an
optional argument, after start, to specify the size in MiB. eg.

    composer-cli compose start --size 2048 example-http-server qcow2

This only works when the backend is not 'lorax-composer', when it is the
user will get a warning that it will be ignored.
2020-06-09 15:09:49 -07:00
Alexander Todorov b7ce3e5685 Don't use f-string without interpolation
resolves pylint W1309(f-string-without-interpolation)

(cherry picked from commit 2e585c98f3)
2020-05-04 11:53:17 -07:00
Brian C. Lane 8045a95c63 lorax: Add --skip-branding cmdline argument
Also document how branding currently works. See docs/lorax.rst

Resolves: rhbz#1826479
(cherry picked from commit c9c1283c56)
2020-05-04 11:41:01 -07:00
Brian C. Lane e732e244ca lorax: Update how the release package is chosen
Previously the release package was chosen by picking the first package
that provides 'system-release' that isn't named generic. This can cause
branding issues with repos containing multiple system-release packages.

This patch changes _install_branding so that it will give preference to
a system-release package that ends with lowercase --variant name. If
there isn't one it will fall back to the previous behavior.

Resolves: rhbz#1472622
(cherry picked from commit 91d71b09a4)
2020-05-04 11:40:56 -07:00
Brian C. Lane 05c9cb8649 ltmpl: Fix package logging format
epoch wasn't included in the lorax-packages.log, but arch is. And for
the debuginfo the epoch, including 0, *is* included. This should match
the previous output.

Resolves: rhbz#1815000
(cherry picked from commit d6ce9c907d)
2020-03-20 08:23:08 -07:00
Brian C. Lane e2e8deb906 lorax: Write package lists in run_transaction
A change in glibc now requires /proc be mounted in order to run mknod
which is needed in order to run rpm from runtime-postinstall.

This drops that code from the template and moves writing the package
list into run_transaction, which already has all of the needed
information to generate the list.

Resolves: rhbz#1812895
2020-03-16 11:51:11 -07:00
Brian C. Lane fe45fa3610 lorax: Catch rootfs out of space failures
It isn't always obvious what happened when the rootfs runs out of space,
especially when using lorax via pungi. So this checks for the out of
space error string when building the runtime image and logs it to the
primary logfile and console as an error with the rootfs size.

eg.
2020-01-20 18:52:58,920: The rootfs ran out of space with size=1
2020-02-05 15:27:41 -08:00
Brian C. Lane 210fe3d8ef lorax: Log dnf solver debug data in ./debugdata/ 2020-01-20 14:04:00 -08:00
Brian C. Lane a0fce98109 Add --dracut-conf cmdline argument to lorax and livemedia-creator
This adds the ability to use a dracut.conf file instead of passing
--dracut-arg on the cmdline multiple times.
2020-01-16 09:26:27 -08:00
Brian C. Lane 5561bbb3bc executils: Drop bufsize=1 from execReadlines
startProgram is running in binary mode, so bufsize=1 is invalid. The
ExecLineReader class already breaks it up into lines using readline() so
there is no need to change the default buffering.
2020-01-15 10:40:17 -08:00
Brian C. Lane 9bb1a0916e Update to_timeval usage to use format_iso8601
GLib.DateTime.to_timeval is deprecated.
2020-01-15 10:40:17 -08:00
Brian C. Lane 0d3cff0dde ltmpl: Update to use collections.abc 2020-01-15 10:40:17 -08:00
Brian C. Lane b69e6d725d lorax-composer: Enable ami on aarch64 2020-01-13 14:55:41 -08:00
Dan Horák c038bb53c6 livemedia-creator: workaround glibc limitation when starting anaconda
On some platforms (aarch64, ppc64le) toolchain limitations/optimizations
can break anaconda startup, as discussed in rhbz#1722181. The workaround
is to preload libgomp.so before starting anaconda.
2020-01-10 11:10:48 +01:00
Brian C. Lane ed15b81818 composer-cli: Increase DELETE timeout to 120s
When the timeout is too short the http library sends the request again.
We return the last response to the user so even if the cancel works they
get an error about the UUID not being valid.

Resolves: rhbz#1788461
2020-01-08 11:57:43 -08:00
Brian C. Lane cd65758ebe anaconda_cleanup: Remove anaconda.pid if it is left behind
When a novirt-install fails or is canceled anaconda may leave its pid
file behind. Add it to the cleanup function.

Resolves: rhbz#1788501
2020-01-07 15:42:16 -08:00
Florian Achleitner e82ae6b82a mkksiso: Add the option to set the ISO volume label
To distinguish from the base images, a different
volume label is useful.
Add a commandline option, pass it to the iso tool,
and replace the occurences of the label in isolinux
and grub config files.

Signed-off-by: Brian C. Lane <bcl@redhat.com>
2019-12-19 11:30:50 -08:00
Brian C. Lane 80dd997b9c composer-cli: Only display the available compose types
The enabled bool is now being used so the cli should only show the types
actually available on the architecture.

Also modifies the test in test_compose_sanity.sh

Related: rhbz#1751998
2019-12-10 12:17:34 -08:00
Ondřej Budai 045bddb37a fix typo in api docstring 2019-11-27 12:24:03 +02:00
Brian C. Lane fad9b324f7 composer-cli: Return int from handle_api_result not bool
The callers, and the documentation, all expect int 0/1 to use as the
exit status for the program. Not True/False, even though that works most
of the time.
2019-11-14 11:58:56 -08:00
Brian C. Lane 95fd12e1f0 mkksiso: copy all the directories over to tmpdir
Graft them from tmpdir instead of trying to use the source iso for some,
and tmpdir for the ones needing changes. This will prevent problems with
trying to remove files likes TRANS.TBL from a read-only filesystem.
2019-11-14 11:57:35 -08:00
Brian C. Lane f74a5cc4a6 mkksiso: Add a tool to add a kickstart to an existing boot.iso
This tool will add the kickstart to the boot.iso, edit the kernel boot
arguments so that the kickstart is used when the iso boots, as well as
allow adding extra files and directories to the / of the iso which can
then be used by the kickstart (they are found under /run/install/repo
while Anaconda is running).
2019-11-05 11:09:42 -08:00
Brian C. Lane e365e476af Disable some compose types on other architectures
The 'enabled' field in the /compose/types output now reflects whether or
not the type is supported on the current architecture. Disabled types
are not allowed to be built, and will raise an error like:

Compose type 'alibaba' is disabled on this architecture
2019-10-28 16:42:36 -07:00
Brian C. Lane af3494fc46 lorax: Drop unused --title option
It has never been hooked up to anything, let alone substitution @TITLE@
(which also hasn't ever been used in the config files).

Closes #863
2019-10-25 11:35:21 -07:00
Brian C. Lane 009c051003 Bump default platform and releasever to 32 2019-10-16 14:22:58 -07:00
Brian C. Lane 3fdc783850 docs: Fix Sphinx errors in docstrings 2019-10-16 13:52:54 -07:00
Brian C. Lane 2fdcfc2e4c composer-cli: Add providers info <PROVIDER> command
This will print details about the settings for the provider, as well as
the supported image types.
2019-10-16 09:17:54 -07:00
Brian C. Lane 63816d9764 composer-cli: Fix error handling in providers push 2019-10-16 09:17:54 -07:00
Brian C. Lane 4256a1d4f2 composer-cli: Fix upload log output 2019-10-16 09:17:54 -07:00
Brian C. Lane 3e9b56e66d composer-cli: Add starting an upload to compose start
This allows starting an upload once the image has been built.
2019-10-16 09:17:54 -07:00
Brian C. Lane 6a2f465217 composer-cli: Add providers template command
This outputs a TOML template of the settings needed for setting the
upload credentials. It can be passed to 'upload start' and to 'compose
start', as well as used to set the profile for 'providers push'
2019-10-16 09:17:54 -07:00
Brian C. Lane 4276a68cf3 composer-cli: Add support for providers command
Add commands for listing, saving, and adding new provider profiles.
Also adds a list command to upload.
2019-10-16 09:17:54 -07:00
Brian C. Lane a414988ee8 composer-cli: Add support for upload command
This allows the user to start an upload, get info about a specific
upload, and to cancel, reset, or delete an upload.
2019-10-16 09:17:54 -07:00
Brian C. Lane 27a4165089 Increase ansible verbosity to 2 2019-10-16 09:17:54 -07:00
Brian C. Lane a59c0241c4 lifted: Improve logging for upload playbooks
playbook errors are now logged, as well as errors encountered while
executing the playbook.
2019-10-16 09:17:53 -07:00
Brian C. Lane 146560b959 Add upload status examples to compose route docstrings 2019-10-16 09:17:53 -07:00
Brian C. Lane b23c73e52c Add docstrings to the new upload functions in pylorax.api.queue 2019-10-16 09:17:53 -07:00
Brian C. Lane d16e8f5bfc Change /compose/uploads/delete to /upload/delete
Remove the requirement to pass the compose uuid and the upload uuid --
only the upload uuid is needed.
2019-10-16 09:17:53 -07:00
Brian C. Lane 0eda252d61 Add profile support to /uploads/schedule/
Pass in the name of a profile in the `profile` field, or pass in
one-time use settings in the `settings` object.
2019-10-16 09:17:53 -07:00
Brian C. Lane acbf63013a lifted: Make sure inputs cannot have path elements
This processes the inputs with os.path.basename to strip off any path
elements and prevent potential path traversal attacks. Also adds a test.
2019-10-16 09:17:53 -07:00
Brian C. Lane 7396c272b2 Use consistent naming for upload uuids
Since we have both compose uuids and upload uuids they need to be
clearly named. This updates the upload naming to use 'upload_uuid' in
the inputs, and 'upload_id' in the output (_id instead of _uuid for
consistency with build_id naming in the status responses).

This also adds 'upload_id' to the /upload/log response.
2019-10-16 09:17:53 -07:00
Brian C. Lane f8f06ccd23 Fix some docstrings in the v1 API 2019-10-16 09:17:53 -07:00
Brian C. Lane fd55bee85d lifted: Make sure providers list is always sorted 2019-10-16 09:17:53 -07:00
Brian C. Lane c384b687ba Add /upload/providers/delete route
This allows deleting a provider's profile. Pass the provider and profile
name like:

/api/v1/upload/providers/delete/azure/test-settings

A standard json response will be returned.
2019-10-16 09:17:53 -07:00
Brian C. Lane b2fc391677 lifted: Add delete_profile function and tests
Also adds a helper to providers, _get_profile_path, so that the code
doesn't need to be repeated in all the functions.
2019-10-16 09:17:53 -07:00