Commit Graph

960 Commits

Author SHA1 Message Date
Brian C. Lane a1787458b1 mkksiso: Check the length of the filenames
With -joliet-long it allows longer filenames, but silently drops the
ones longer than 253 characters. Check for this and raise an error.

Related: rhbz#2028048
2021-12-09 09:47:12 -08:00
Brian C. Lane ddd02891b0 mkksiso: Check the iso's arch against the host's
mkksiso cannot be run on an iso from another arch, some of the files and
tools are arch specific. Catch this problem and tell the user that the
iso doesn't match the host's architecture.

Related: rhbz#2028048
2021-12-09 09:47:12 -08:00
Brian C. Lane b4a95c3584 mkksiso: Add missing implantisomd5 tool requirements
Related: rhbz#2028048
2021-12-09 09:47:12 -08:00
Brian C. Lane 86e492d680 mkksiso: Raise error if no volume id is found
It is possible the source iso doesn't have an id, or there is an errorr
reading it. Raise an error when this happens to make it more clear what
the problem is.

Related: rhbz#2028048
2021-12-09 09:47:12 -08:00
Brian C. Lane c7f22717a8 mount: Add aarch64 and s390x support to IsoMountopoint
kernel and initrd are under images/ on these isos.

Resolves: rhbz#2028089
2021-12-09 09:47:12 -08:00
Brian C. Lane c966f26d6d mkksiso: Skip mkefiboot for non-UEFI isos
Resolves: rhbz#2028097
2021-12-09 09:47:12 -08:00
Brian C. Lane 236e1cab7a mkksiso: Add -joliet-long
DVD's or user content may use filenames > 64 characters. This increases
the limit to 103.

Resolves: rhbz#2028044
2021-12-09 09:47:12 -08:00
Brian C. Lane 5e88b6ede5 mkksiso: Return 1 on errors
Resolves: rhbz#2028048
2021-12-09 09:47:12 -08:00
Brian C. Lane 12815c5679 Increase boot.iso rootfs to 4GiB
Free space was < 500M again, increase the default.

Resolves: rhbz#2028059
2021-12-06 11:50:57 -08:00
Brian C. Lane 900cd436f7 Handle all possible dnf group_install errors
dnf has changed what it will raise when trying to group_install a
nonexistent group, this adds handling for all of them.

Related: rhbz#1947958
2021-10-11 10:24:50 -07:00
Brian C. Lane 6c07a08be0 Backport mkksiso to RHEL 8.5.0
This allows you to modify the boot.iso to run a kickstart and add
additional files to the iso.

Resolves: rhbz#1955355
2021-06-29 09:21:03 -07:00
Brian C. Lane 206d9a34a6 composer-cli: Add support for start-ostree --url URL
Some ostree builds, like edge, require passing a url to it along with
the ref and parent arguments. This adds an optional --url argument to
the 'composer-cli compose start-ostree' command.

Resolves: rhbz#1929381
2021-02-16 13:37:23 -08:00
Anthony F McInerney 4d7dc6f1de fixes #543 qemu -nodefconfig deprecated
Use -no-user-config instead.

Resolves: rhbz#1904168

(cherry picked from commit f66bff5aa7)
Signed-off-by: Brian C. Lane <bcl@redhat.com>
2020-12-09 16:40:53 -08:00
Brian C. Lane be3aa59daa Switch to using upstream mk-s390image for s390 cdboot.img creation
mk-s390-cdboot has stopped working because the kernel outgrew the
hard-coded offset it used when creating cdboot.img. IBM now has a script
in s390utils that can do the same thing so use the upstream script
instead.

This drops mk-s390-cdboot script, switches the s390 templates to use
mk-s390image from s390utils.

It adds @ROOT@ to cdboot.prm, and sets inst.stage2 so that the installer
image will be found when booting the iso.

Related: rhbz#1892404
2020-12-03 08:37:23 -08:00
Marek Marczykowski-Górecki 6291373794 Add --squashfs-only option to drop inner rootfs.img layer
Make runtime directly into squashfs image. This reduces largely
unreproducible ext4 layer, but requires anaconda's dracut module
modification to properly mount the image.

(cherry picked from commit 27e611629f)
Signed-off-by: Brian C. Lane <bcl@redhat.com>

Resolves: rhbz#1846282
2020-12-02 16:14:35 -08:00
Brian C. Lane 088111f087 composer-cli: Make start-ostree parent and ref optional
This changes the start-ostree command to:

compose start-ostree [--size XXXX] [--parent PARENT] [--ref REF] <BLUEPRINT> <TYPE> [<IMAGE-NAME> <PROFILE.TOML>]

Both of them are optional, and if missing a "" is passed to
osbuild-composer. Also adds more tests for all the various possible
options and removes the provider and profile arguments.

(cherry picked from commit 5b0487f47c)

Resolves: rhbz#1859680
2020-07-23 10:35:28 -07:00
Brian C. Lane dcb2f36c2d composer-cli: Add a get_arg function
This is in preperation for adding more optional arguments. Adds a
generic get_arg function, tests for it, and converts get_size to use it.

(cherry picked from commit 4a4128af23)

Related: rhbz#1859680
2020-07-23 10:35:28 -07:00
Brian C. Lane 952b34dbca composer-cli: Disable retry counter on connection timeout
urllib3 has a retry system that send the request again when the
connection fails, times out, is redircted, etc. This turns off retries,
which aren't very useful anyway.

Related: rhbz#1844649
2020-06-17 10:21:00 -07:00
Brian C. Lane 800fc3b18b composer-cli: Change timeout to 5 minutes
Related: rhbz#1844649

In some cases osbuild-composer with an empty cache can take longer than
a minute to depsolve and start a compose.
2020-06-16 17:04:47 -07:00
Brian C. Lane c300ac6000 composer-cli: Remove the upload and providers commands
These have never been supported on RHEL8 so remove them. Uploads are
supported by osbuild-composer using the 'compose' command.

Related: rhbz#1844649
2020-06-11 09:14:11 -07:00
Brian C. Lane ba5a36c982 composer-cli: Return a better error with no value
And add tests for the get_size function.

Related: rhbz#1844649
2020-06-11 09:14:11 -07:00
Brian C. Lane 7da012a92b composer-cli: Add new start-ostree command
This is used to start an ostree build, it is only supported on
ostree-composer, and requires the ostree ref and parent. It may also
include --size and optionally be uploaded.

Resolves: rhbz#1844649
2020-06-09 13:52:44 -07:00
Brian C. Lane 7c705c17d1 composer-cli: Add support for --size to compose start
osbuild-composer can support user specified image sizes, this adds an
optional argument, after start, to specify the size in MiB. eg.

    composer-cli compose start --size 2048 example-http-server qcow2

This only works when the backend is not 'lorax-composer', when it is the
user will get a warning that it will be ignored.

Related: rhbz#1844649
2020-06-09 13:47:40 -07:00
Brian C. Lane b75a4e2e3c composer-cli: pytoml load() needs an open file not a string
Resolves: rhbz#1843704
2020-06-03 15:16:37 -07:00
Brian C. Lane 3613c12464 lorax: Update how the release package is chosen
Previously the release package was chosen by picking the first package
that provides 'system-release' that isn't named generic. This can cause
branding issues with repos containing multiple system-release packages.

This patch changes _install_branding so that it will give preference to
a system-release package that ends with lowercase --variant name. If
there isn't one it will fall back to the previous behavior.

Resolves: rhbz#1826479
2020-04-27 16:02:15 -07:00
Brian C. Lane b310f3f788 Move get_dnf_base_object into a module
This allows it to be imported by tests.

Related: rhbz#1826479
2020-04-27 16:01:56 -07:00
Brian C. Lane db8f198304 lorax: Add --skip-branding cmdline argument
Also document how branding currently works. See docs/lorax.rst

Resolves: rhbz#1826479
2020-04-27 16:01:35 -07:00
Brian C. Lane a08bd603d0 tests: Disable pylint preexec-fn warning
Related: rhbz#1785154
2020-02-05 14:35:16 -08:00
Brian C. Lane 17ce6ee4e4 lorax-composer: Enable ami on aarch64
Resolves: rhbz#1789308
2020-01-13 15:10:16 -08:00
Brian C. Lane 9636a9b1db composer-cli: Increase DELETE timeout to 120s
When the timeout is too short the http library sends the request again.
We return the last response to the user so even if the cancel works they
get an error about the UUID not being valid.

Resolves: rhbz#1788461
2020-01-07 14:12:27 -08:00
Brian C. Lane 15fbef8f97 composer-cli: Remove unneeded % from API WARNING message
Related: rhbz#1779301
2019-12-12 09:32:40 -08:00
Brian C. Lane 054669d027 composer-cli: Only display the available compose types
The enabled bool is now being used so the cli should only show the types
actually available on the architecture.

Also modifies the test in test_compose_sanity.sh

Related: rhbz#1751998
2019-12-10 15:52:39 -08:00
Brian C. Lane b6b842943d composer-cli: Return int from handle_api_result not bool
The callers, and the documentation, all expect int 0/1 to use as the
exit status for the program. Not True/False, even though that works most
of the time.

(cherry picked from commit fad9b324f7)
Related: rhbz#1779301
2019-12-03 15:13:05 -08:00
Brian C. Lane 32d1451a22 Add support for API v1 commands upload, and providers
This adds the ability to execute API v1 commands when the backend server
supports them. It detects the API version supported by the server and
adjusts the available commands accordingly.

Resolves: rhbz#1779301
2019-12-03 15:13:05 -08:00
Brian C. Lane 7270511379 lorax-composer: Add basic case check to check_recipe_dict
This will detect if an expected field is not all lower case as is
required.

(cherry picked from commit 13a7dcf2b1)

Related: rhbz#1716596
2019-10-31 10:11:13 -07:00
Brian C. Lane 229253b0cb lorax-composer: Add basic recipe checker function
This makes sure that required fields are included, and that sections are
not empty. It does not check for all optional fields.

If there are errors it will gather up all of them and then raise a
RecipeError with a string of all the errors.

(cherry picked from commit 61059a2699)

Resolves: rhbz#1716596
2019-10-31 10:11:13 -07:00
Brian C. Lane 140f40f6be Disable some compose types on other architectures
The 'enabled' field in the /compose/types output now reflects whether or
not the type is supported on the current architecture. Disabled types
are not allowed to be built, and will raise an error like:

Compose type 'alibaba' is disabled on this architecture

Resolves: rhbz#1751998
2019-10-28 17:03:18 -07:00
Evan Goode 12bdd8ba95 More descriptive error for a bad ref in repos.git
Fixes #771

Related: rhbz#1709594

Signed-off-by: Brian C. Lane <bcl@redhat.com>
2019-06-27 08:53:44 -07:00
Brian C. Lane ff9421c848 lorax-composer: Fix installing files from [[repos.git]] to /
rpmfluff was including / in the rpm, which conflicts with
filesystem.rpm

The rpm globs are pretty limited, and we don't actually know the file
paths until later, so we have to use a glob or a directory.

So when the destination is / it now uses /* to select all the files and
sub-directories in the archive. The limitation of this is that it cannot
support dotfiles directly under /, they will cause a rpmbuild error.

For destinations other than / it uses the name of the directory, so
dotfiles are fine in that situation.

(cherry picked from commit 049f68cb60)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane adede63a95 rpmgit: catch potential errors while running git
Log them and report them as RuntimeError. Also add a couple tests for
them.

(cherry picked from commit 61efa91a03)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane 20389c912f Add repos.git support to lorax-composer builds
This hooks up creation of the rpm to the build, adds it to the
kickstart, and passes the url to Anaconda. The dnf repo with the rpms is
created under the results directory so it will be included when
downloading the build's results.

(cherry picked from commit cd8c884adb)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane 3b288f0779 Add pylorax.api.gitrpm module and tests
This handles creating the rpm from the dictionary describing the
repository and rpm. Also adds tests for archive and rpm creation.

(cherry picked from commit f6f2308765)

Related: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane f5113542b1 Add support for [[repos.git]] section to blueprints
This adds support, documentation, and testing for a [[repos.git]]
blueprint section that can be used to install files from a git
repository. It will create an rpm that will be added to the build,
and included in the metadata that can be downloaded. This allows you to
accurately keep track of the source of configuration files and extra
metadata that is added to the build.

The source repo and reference will be listed in the rpm's summary making
it easy to discover on the installed system.

(cherry picked from commit d7b96c8f0f)

Resolves: rhbz#1709594
2019-06-26 16:14:40 -07:00
Brian C. Lane 724ca8b56d composer-cli: Update diff support for customizations and repos.git
composer-cli will now output information about changes to customizations
entries and the repos.git entries.

(cherry picked from commit 156ef0acfd)

Related: rhbz#1709595
2019-05-29 16:35:42 -07:00
Brian C. Lane eb59a1a26e Add support for customizations and repos.git to /blueprints/diff/
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.

(cherry picked from commit 850c490b6e)

Related: rhbz#1709595
2019-05-29 16:35:07 -07:00
Brian C. Lane 9f0dd17e2f Change customizations.firewall to append items instead of replace
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.

Also updated the docs, added a new test for it, and sorted the output
for consistency.

(cherry picked from commit 3e08389a0f)

Resolves: rhbz#1709595
2019-05-29 16:29:17 -07:00
Brian C. Lane cdf0cbbc5e lorax-composer: Add services support to blueprints
Add support for enabling and disabling systemd services in the
blueprint. It works like this:

    [customizations.services]
    enabled = ["sshd", "cockpit.socket", "httpd"]
    disabled = ["postfix", "telnetd"]

They are *added* to any existing settings in the kickstart templates.

(cherry picked from commit 1111aee92d)

Related: rhbz#1709595
2019-05-29 15:22:30 -07:00
Brian C. Lane 59bad712ad lorax-composer: Add firewall support to blueprints
You can now open ports in the firewall, using port numbers or service
names:

    [customizations.firewall]
    ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]

Or enable/disable services registered with firewalld:

     [customizations.firewall.services]
     enabled = ["ftp", "ntp", "dhcp"]
     disabled = ["telnet"]

If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.

(cherry picked from commit 4d35668ab5)

Related: rhbz#1709595
2019-05-29 15:21:34 -07:00
Brian C. Lane ad6fe6cffd lorax-composer: Add locale support to blueprints
You can now set the keyboard layout and language. Eg.

[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"

Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'

Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.

(cherry picked from commit e5a8700bdf)

Related: rhbz#1709595
2019-05-29 15:20:28 -07:00
Brian C. Lane 86b595e13e lorax-composer: Fix customizations when creating a recipe
This fixes the customizations list problem earlier than in
add_customizations.

In the recipe it should be [customizations] not [[customizations]]
which creates a list. If it was used that way grab the first element and
replace the list with it.

(cherry picked from commit 67007dfa60)

Related: rhbz#1709595
2019-05-29 15:18:41 -07:00