Commit Graph

2740 Commits

Author SHA1 Message Date
Lubomír Sedlář
d546a49299 createiso: Block reuse if unsigned packages are allowed
We can have a compose with unsigned packages.

By the time the next compose is generated, the packages could have been
signed. However, the new compose would still reuse the ISO with unsigned
copies.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-08-14 11:09:37 +02:00
Lubomír Sedlář
c59f2371a3 Allow live_images phase to still be skipped
Without this fix existing configurations break even though they don't
use the phase.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-08-07 13:34:02 +02:00
Lubomír Sedlář
3b2c6ae72a createiso: Recompute .treeinfo checksums for images
Running xorriso to modify an ISO image can update content of included
images such as images/eltorito.img, unless we explicitly update the
image, which is undesirable (https://pagure.io/pungi/issue/1647).

However, when the file is changed, the checksum changes and .treeinfo no
longer matches.

This patch implements a workaround: once the DVD is written, it looks
for incorrect checksums, recalculates them and updates the .treeinfo on
the DVD. Since only the checksum is changing and the size of the file
remains the same, this seems to help fix the issue.

An additional step for implanting MD5 is needed again, as that gets
erased by the workaround.

JIRA: RHELCMP-13664
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-08-06 09:15:17 +02:00
Lubomír Sedlář
0726a4dca7 Drop support for signing rpm-wrapped artifacts
This was only usable in live_images phase that doesn't exist anymore,
and wasn't used much in the first place.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-08-05 10:55:08 +00:00
Adam Williamson
531f0ef389 Remove live_images.py (LiveImagesPhase)
This phase was used to create live images with livecd-creator
and 32-bit ARM images with appliance-creator. We also remove
get_create_image_cmd from the Koji wrapper as it was only used
for this phase, remove associated tests, and remove related
configuration settings and documentation.

Fixes: https://pagure.io/pungi/issue/1753
Merges: https://pagure.io/pungi/pull-request/1774
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-08-05 10:55:08 +00:00
Lubomír Sedlář
c96b5358ba Clean up requirements
* dict.sorted and funcsigs are not used anywhere anymore
* urlgrabber is used only in the yum based gather.py module, and thus
  only needed on Python 2
* py3 doesn't need to reinstall mock as that is part of stdlib now

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-07-29 10:29:31 +02:00
Haibo Lin
d1c0276761 Update pungi.spec for py3
Signed-off-by: Haibo Lin <hlin@redhat.com>
2024-07-12 17:03:09 +08:00
Haibo Lin
0cb18bfa24 Release 4.6.3
JIRA: RHELCMP-13724

Signed-off-by: Haibo Lin <hlin@redhat.com>
2024-07-12 09:30:12 +08:00
Lubomír Sedlář
f72adc03b1 Fix formatting of long line
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-06-17 10:20:10 +02:00
Lubomír Sedlář
8ced384540 unified-isos: Resolve symlinks
If the compose is configured to use symlinks for packages, the unified
ISO would include the symlinks which is useless.

Instead, let's check and replace any symlinks pointing outside of the
compose with the actual file.

JIRA: RHELCMP-13802
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-06-07 15:18:45 +02:00
Lubomír Sedlář
4a5106375e gather: Skip lookaside packages from local lookaside repo
When variant X depends on variant A, Pungi creates a temporary local
lookaside with packages from A. If there's an external lookaside
configured, the list of package for variant A can contain URLs to the
external repo.

Newer versions of createrepo fail when pkglist specifies an unreachable
package, and it doesn't do downloading.

JIRA: RHELCMP-13648
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-06-05 09:45:02 +02:00
Haibo Lin
627b72597e pkgset: Avoid adding modules to unavailable arches
If a module is not built for specific arches, pungi will skip adding it
to these arches in pkgset phase.

JIRA: RHELCMP-13625
Signed-off-by: Haibo Lin <hlin@redhat.com>
2024-05-09 15:42:06 +08:00
Lubomír Sedlář
bc0334cc09 iso: Extract volume id with xorriso if available
Pungi can use either genisoimage or xorriso to create ISOs.

It also needed isoinfo utility for querying volume ID from the ISO
image. However, the utility is part of the genisoimage suite of tools.

On systems that no longer provide genisoimage, the image would be
successfully generate with xorriso, but then pungi would fail to extract
the volume id leading to metadata with missing values.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-04-23 09:13:43 +02:00
Adam Williamson
5c9e79f535 De-duplicate log messages for ostree and ostree_container phases
The ostree and ostree_container phases both log messages in the
exact same form, which is rather confusing. This will make it
much clearer which message comes from which phase.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-22 06:39:50 +00:00
Lubomír Sedlář
29c166ab99 Handle tracebacks as str or bytes
Kobo 0.36.0 changed how tracebacks are handled. Instead of `bytes`, it
returns a `str`. That makes pungi fail to write it into a file opened as
binary.

Relates: https://github.com/release-engineering/kobo/pull/246
Fixes: https://pagure.io/pungi/issue/1756
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-04-15 14:02:13 +02:00
Adam Williamson
51d58322f2 ostree/container: add missing --version arg
https://pagure.io/pungi/pull-request/1726 tries to use
`self.args.version`, but the `pungi-make-ostree container`
subcommand does not actually have a `--version` arg, so that is
not going to work. This adds the required arg.

We *could* make it optional by still setting an empty update
dict if it's not specified, I guess, but not sure if that's worth
the effort.

Fixes: https://pagure.io/pungi/issue/1751

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-14 12:28:39 -07:00
Lubomír Sedlář
0ef1c102b8 Block pkgset reuse on module defaults change
JIRA: RHELCMP-13463
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 12:36:55 +00:00
Adam Williamson
b6cfd8c5d4 Include task ID in DONE message for OSBS phase
Again, composetracker expects the message in this format.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-13 12:17:11 +00:00
Adam Williamson
9f8377abab Various phases: consistent format of failure message
composetracker expects the failure message to be in a specific
form, but some phases weren't using it. They were phrasing it
slightly differently, which throws off composetracker's parsing.
We could extend composetracker to handle both forms, but it seems
simpler to just make all the phases use a consistent form.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-13 12:17:11 +00:00
Lubomír Sedlář
949add0dac Update tests to exercise kiwi specific metadata
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 13:06:43 +01:00
Adam Williamson
8fb694f000 Kiwi: translate virtualbox and azure productmd formats
As discussed in
https://pagure.io/releng/failed-composes/issue/6047#comment-899622
the list of 'acceptable' types and formats (in productmd terms)
is locked down in productmd, we cannot just 'declare' new formats
in pungi as we kinda wound up doing by adding these Kiwi
extensions to the EXTENSIONS dict in image_build phase. So
instead, let's return the image_build phase to the way it was,
and add an additional layer of handling in kiwibuild phase for
these awkward cases, which 'translates' the file suffix to a
format productmd knows about already. This is actually how we
would rather behave anyway, because a Kiwi-produced
`vagrant.libvirt.box` file really is the same kind of thing as an
ImageFactory-produced `vagrant-libvirt.box` file; we want them to
have compatible metadata, we don't want them to look like
different things.

Merges: https://pagure.io/pungi/pull-request/1740
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-13 13:06:43 +01:00
Lubomír Sedlář
8a3b64e5b8 kiwibuild: Add tests for the basic functionality
Merges: https://pagure.io/pungi/pull-request/1739
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 13:06:43 +01:00
Lubomír Sedlář
c80ebb029b kiwibuild: Remove repos as dicts
The task needs just URLs, the dics don't bring anything here.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 13:06:43 +01:00
Lubomír Sedlář
e2ceb48450 Fix additional image metadata
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 13:06:43 +01:00
Lubomír Sedlář
242d7d951f Drop kiwibuild_version option
Version in kiwibuild is embedded in the definition file. The option
makes no sense.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 13:06:43 +01:00
Lubomír Sedlář
04d4e1d585 Update docs with kiwibuild options
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-13 13:06:43 +01:00
Adam Williamson
e90ffdfd93 kiwibuild: allow setting description scm and path at phase level
Neal wanted this to work - he tried using global_description_scm
and global_description_path in the initial PR - but it wasn't
wired up to work. This should make it possible to set
`kiwibuild_description_scm` and `kiwibuild_description_path`.
It also technically lets you set `global_` for both, since the
`get_config` implementation is very generic, but it doesn't add
it to the checks, so you'd still get an "unrecognized config
option" warning, I think. It seems appropriate to encourage
setting this as a phase-level option rather than a global one
since it seems quite specific to the kiwibuild phase.

Merges: https://pagure.io/pungi/pull-request/1737
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-13 13:06:43 +01:00
Lubomír Sedlář
0d310fb3b3 Use latest Fedora for python 3 test environment
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-12 09:26:01 +01:00
Lubomír Sedlář
5172d7e5eb Install unittest2 only on python 2
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-12 08:58:39 +01:00
Adam Williamson
0d306d4964 Fix 'failable' handling for kiwibuild phase
The mechanisms here are a bit subtle and the kiwibuild phase
didn't quite get them right. The arg passed to `util.failable`
is supposed to be a boolean, but kiwibuild was passing it the
list of failable arches (which will always evaluate True).

How this is meant to work is that we only make *the Koji task
as a whole* failable (by passing `True` to `util.failable`) if
*all* the arches in it are failable. If *any* arch in the task
is not failable, the task should not be failable.

We allow a subset of arches to fail by passing the Koji task a
list of `optional_arches`, later. If an arch is 'optional', that
arch failing won't cause the Koji task itself to be considered
failed.

This commit fixes the logic (I hope), renames all the variables
and adds a couple of comments to make it clearer what's going on,
and does a bit of making the code simpler.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-08 15:12:01 -08:00
Jeremy Cline
1494f203ce image_build: Accept Kiwi extension for Azure VHD images
Kiwi builds for Azure fixed VHD images are suffixed with "vhdfixed"
instead of plain "vhd". Add that to the list of suffixes.

Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
2024-03-07 08:07:38 +00:00
Adam Williamson
93b4b4ae0f image_build: accept Kiwi vagrant image name format
According to Neal, Vagrant images produced by Kiwi end in e.g.
`vagrant.libvirt.box` and `vagrant.virtualbox.box` - with a
period between `vagrant` and the image type, not a dash as with
oz. We should accept this slightly different format so we can
correctly derive the productmd `type` and `format` for these.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-06 10:38:45 -08:00
Lubomír Sedlář
b8e26bfb64 Release 4.6.2
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-05 15:01:24 +01:00
Tomáš Hozza
e738f65458 Phases/osbuild: support passing 'customizations' for image builds
The osbuild Koji plugin supports passing customizations for an image
build. This is also supported in the Koji CLI plugin. Some teams want to
pass image customizations for images built as part of Pungi composes.
Extend the osbuild phase to support passing customizations in the Pungi
configuration.

Merges: https://pagure.io/pungi/pull-request/1733
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-05 14:46:56 +01:00
Lubomír Sedlář
209d308e1c dnf: Load filelists for actual solver too
Not just in tests.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-05 13:38:45 +00:00
Lubomír Sedlář
be410d9fd5 kiwibuild: Tell Koji which arches are allowed to fail
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-04 11:15:39 +01:00
Lubomír Sedlář
1f819ee08a kiwibuild: Update documentation with more details
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-04 11:15:39 +01:00
Lubomír Sedlář
b9d94970b5 kiwibuild: Add kiwibuild global options
This is already supported by code, just missing in the schema.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-04 11:14:37 +01:00
Lubomír Sedlář
b032425f30 kiwibuild: Process images same as image-build
Getting the images from task is less hacky then matching on filenames.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-04 11:14:37 +01:00
Lubomír Sedlář
bcd937d16d kiwibuild: Add subvariant configuration
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-04 11:14:37 +01:00
Lubomír Sedlář
f0137fd9b9 kiwibuild: Work around missing arch in build data
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-03-04 11:14:37 +01:00
Haibo Lin
3d630d3e8e Support KiwiBuild
Adding kiwibuild phase which is similar to osbuild.

Fixes: https://pagure.io/pungi/issue/1710
Merges: https://pagure.io/pungi/pull-request/1720
JIRA: RHELCMP-13348
Signed-off-by: Haibo Lin <hlin@redhat.com>
2024-03-04 11:14:23 +01:00
Timothée Ravier
8412890640 ostree/container: Set version in treefile 'automatic-version-prefix'
In the non container path, we're setting the version for the build using
the `--add-metadata-string=version=XYZ` argument passed to `rpm-ostree
compose tree ...`.

The `rpm-ostree compose image` path does not expose this option yet so
modify the treefile directly as we are already doing it to set the
repos used for the compose.

See: https://github.com/coreos/rpm-ostree/issues/4829
See: https://pagure.io/workstation-ostree-config/pull-request/472
Merges: https://pagure.io/pungi/pull-request/1726
Signed-off-by: Timothée Ravier <tim@siosm.fr>
2024-02-26 15:51:48 +01:00
Lubomír Sedlář
42befba0b1 dnf: Explicitly load filelists
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2264414
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-20 07:12:06 +00:00
Lubomír Sedlář
52c2cea0ef Fix buildinstall reuse with pungi_buildinstall plugin
The keys may not exist anymore. If there's nothing to delete, it's fine.

JIRA: RHELCMP-13464
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-14 13:56:14 +00:00
Lubomír Sedlář
d2e9ccefde Fix filters for DNF query
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-14 14:41:12 +01:00
Lubomír Sedlář
2c61416423 gather-dnf: Support dotarch in filter_packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-14 11:38:16 +00:00
Lubomír Sedlář
986099f8b5 gather: Support dotarch notation for debuginfo packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-14 11:38:16 +00:00
Lubomír Sedlář
947ddf0a1a Correctly set input and fultree_exclude flags for debuginfo
This only matters for composes that use the functionality for trimming
addon packages from parent variants.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-14 11:38:16 +00:00
Lubomír Sedlář
e46393263e 4.6.1 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2024-02-08 08:58:37 +01:00