Commit Graph

2863 Commits

Author SHA1 Message Date
Tomáš Hozza
a97488721d
Phases/osbuild: support passing 'customizations' for image builds
The osbuild Koji plugin supports passing customizations for an image
build. This is also supported in the Koji CLI plugin. Some teams want to
pass image customizations for images built as part of Pungi composes.
Extend the osbuild phase to support passing customizations in the Pungi
configuration.

Merges: https://pagure.io/pungi/pull-request/1733
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit e738f65458)
2024-08-30 13:39:16 +03:00
Lubomír Sedlář
4d858ef958
dnf: Load filelists for actual solver too
Not just in tests.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 209d308e1c)
2024-08-30 13:39:14 +03:00
Lubomír Sedlář
744b00499d
kiwibuild: Tell Koji which arches are allowed to fail
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit be410d9fd5)
2024-08-30 13:39:13 +03:00
Lubomír Sedlář
583547c6ee
kiwibuild: Update documentation with more details
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 1f819ee08a)
2024-08-30 13:39:11 +03:00
Lubomír Sedlář
f28053eecc
kiwibuild: Add kiwibuild global options
This is already supported by code, just missing in the schema.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit b9d94970b5)
2024-08-30 13:39:10 +03:00
Lubomír Sedlář
a196e9c895
kiwibuild: Process images same as image-build
Getting the images from task is less hacky then matching on filenames.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit b032425f30)
2024-08-30 13:39:08 +03:00
Lubomír Sedlář
a6f6199910
kiwibuild: Add subvariant configuration
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit bcd937d16d)
2024-08-30 13:39:07 +03:00
Lubomír Sedlář
a3dcec5059
kiwibuild: Work around missing arch in build data
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit f0137fd9b9)
2024-08-30 13:39:05 +03:00
Haibo Lin
6aa674fbb3
Support KiwiBuild
Adding kiwibuild phase which is similar to osbuild.

Fixes: https://pagure.io/pungi/issue/1710
Merges: https://pagure.io/pungi/pull-request/1720
JIRA: RHELCMP-13348
Signed-off-by: Haibo Lin <hlin@redhat.com>
(cherry picked from commit 3d630d3e8e)
2024-08-30 13:39:04 +03:00
Timothée Ravier
05d9651eba
ostree/container: Set version in treefile 'automatic-version-prefix'
In the non container path, we're setting the version for the build using
the `--add-metadata-string=version=XYZ` argument passed to `rpm-ostree
compose tree ...`.

The `rpm-ostree compose image` path does not expose this option yet so
modify the treefile directly as we are already doing it to set the
repos used for the compose.

See: https://github.com/coreos/rpm-ostree/issues/4829
See: https://pagure.io/workstation-ostree-config/pull-request/472
Merges: https://pagure.io/pungi/pull-request/1726
Signed-off-by: Timothée Ravier <tim@siosm.fr>
(cherry picked from commit 8412890640)
2024-08-30 13:39:02 +03:00
Lubomír Sedlář
75ab6a14b2
dnf: Explicitly load filelists
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2264414
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 42befba0b1)
2024-08-30 13:39:01 +03:00
Lubomír Sedlář
533ea641d8
Fix buildinstall reuse with pungi_buildinstall plugin
The keys may not exist anymore. If there's nothing to delete, it's fine.

JIRA: RHELCMP-13464
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 52c2cea0ef)
2024-08-30 13:38:59 +03:00
Lubomír Sedlář
185a53d56b
Fix filters for DNF query
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit d2e9ccefde)
2024-08-30 13:38:58 +03:00
Lubomír Sedlář
305deab9ed
gather-dnf: Support dotarch in filter_packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 2c61416423)
2024-08-30 13:38:57 +03:00
Lubomír Sedlář
6af11d5747
gather: Support dotarch notation for debuginfo packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 986099f8b5)
2024-08-30 13:38:56 +03:00
Lubomír Sedlář
58f96531c7
Correctly set input and fultree_exclude flags for debuginfo
This only matters for composes that use the functionality for trimming
addon packages from parent variants.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 947ddf0a1a)
2024-08-30 13:38:55 +03:00
Lubomír Sedlář
e570aa7726
4.6.1 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

(cherry picked from commit e46393263e)
2024-08-30 13:38:53 +03:00
Lubomír Sedlář
d8a553163f
Make python3-mock dependency optional
https://fedoraproject.org/wiki/Changes/RemovePythonMockUsage

Prefer using unittest.mock to a standalone package. The separate
packages should only really be needed on Python 2.7 these days.

The test requirements file is updated to only require mock on old
Python, and the dependency is removed from setup.py to avoid issues
there.

Relates: https://src.fedoraproject.org/rpms/pungi/pull-request/9

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

(cherry picked from commit ff5a7e6377)
2024-08-30 13:38:43 +03:00
Lubomír Sedlář
a9839d8078
Make latest black happy
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit dd7ecbd5fd)
2024-08-30 13:31:29 +03:00
Lubomír Sedlář
dc05d1fbba
Update tox configuration
The whitelist_externals option has been renamed to allowlist_externals.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit ba613563f6)
2024-08-30 13:31:28 +03:00
Lubomír Sedlář
dc4e8b2fb7
Fix scm tests to not use user configuration
If you configure default branch name in new repos to anything else than
master, there will be failures in tests. The test expects the branch to
be called master, but does not ensure it in any way.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit c8d16e6978)
2024-08-30 13:31:26 +03:00
Lubomír Sedlář
27d055992e
Add workaround for old requests in kojiwrapper
When running with requests<2.18 (i.e. on RHEL 7), streaming responses
are not a context manager and need to be wrapped in contextlib.closing.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 860360629d)
2024-08-30 13:31:25 +03:00
Lubomír Sedlář
34fcd550b6
Use pungi_buildinstall without NFS
The plugin supports two modes of operation:
1. Mount a shared storage volume into the runroot and have the output
   written there.
2. Have the plugin create a tar.gz with the outputs and upload them to
   the hub, from where they can be downloaded.

This patch switches from option 1 to option 2.

This requires all input repositories to be passes in as URLs and not
paths. Once the task finishes, Pungi will download the output archives
and unpack them into the expected locations.

JIRA: RHELCMP-13284
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit f25489d060)
2024-08-30 13:31:24 +03:00
Adam Williamson
4c0059e91b
checks: don't require "repo" in the "ostree" schema
Per @siosm in https://pagure.io/pungi-fedora/pull-request/1227
this option "is deprecated and not needed anymore", so Pungi
should not be requiring it.

Merges: https://pagure.io/pungi/pull-request/1714
Signed-off-by: Adam Williamson <awilliam@redhat.com>
(cherry picked from commit 432b0bce04)
2024-08-30 13:31:23 +03:00
Lubomír Sedlář
bb2e32132e
ostree_container: Use unique temporary directory
The config repository is cloned into a path that conflicts with the
regular ostree phase. Let's use a unique name to avoid that problem.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 7e779aa90f)
2024-08-30 13:31:22 +03:00
Lubomír Sedlář
dca3be5861
4.6.0 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

(cherry picked from commit f4bf0739aa)
2024-08-30 13:31:20 +03:00
Lubomír Sedlář
38ec4ca159
Add ostree container to image metadata
This requires https://github.com/release-engineering/productmd/pull/172

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 119b212241)
2024-08-30 13:30:44 +03:00
Lubomír Sedlář
c589ccb56f
Updates for ostree-container phase
This patch connects the phase into the main script, and adds other
modifications:

* The archive is now stored in the images/ subdirectory in the compose.
* Documentation is updated to correctly mention that variant repos are
  not available.
* Configuration for path and name of the final archive is dropped. There
  are reasonable defaults for this and there's no point in having users
  configure it.
* The extra message for the archive is no longer sent.
* The pungi-make-ostree utility is no longer required in the buildroot.

The pungi-make-ostree utility doesn't do any significant work. It
modifies configuration files (which can happen on the compose host), and
it starts other processes.

This patch changes the ostree-container phase to no longer need the
script in the buildroot. Instead, the utility is called on the compose
host to do the config manipulation and output the needed commands. Those
are then passed into the runroot task.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 081c31238b)
2024-08-30 13:30:42 +03:00
Timothée Ravier
e413955849
Add ostree native container support
Add a new `ostree_container` stage to create ostree native container
images as OCI archives, using rpm-ostree compose image.

See: https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable
See: https://gitlab.com/CentOS/cloud/issue-tracker/-/issues/1

Fixes: https://pagure.io/pungi/issue/1698
Merges: https://pagure.io/pungi/pull-request/1699

Signed-off-by: Timothée Ravier <tim@siosm.fr>
(cherry picked from commit 95497d2676)
2024-08-30 13:30:41 +03:00
Adam Williamson
e70e1841c7
Improve autodetection of productmd image type for osbuild images
I don't love inferring the type from the filename like this -
it's kinda backwards - but it's an improvement on the current
logic (I don't think 'dvd' is ever currently the correct value
here, I don't think osbuild *can* currently build the type of
image that 'dvd' is meant to indicate). I can't immediately see
any better source of data here (we could use the 'name' or
'package_name' from 'build_info', but those are pretty much
just inputs to the filenames anyway).

Types that are possible in productmd but not covered here are
'cd' (never likely to be used again in Fedora at least, not sure
about RHEL), 'dvd-debuginfo' (again not used in Fedora, may be
used in RHEL), 'ec2', 'kvm' (not sure about those), 'netinst'
(this is a synonym for 'boot', we use 'boot' in practice in
Fedora metadata), 'p2v' and 'rescue' (not sure).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
(cherry picked from commit aa7fcc1c20)
2024-08-30 13:30:40 +03:00
Lubomír Sedlář
fc86e03e44
pkgset: ignore events for modular content tags
Generally we want all packages to come from particular event.

There are two exceptions: packages configured via `pkgset_koji_builds`
are pulled in by exact NVR and skip event; and modules in
`pkgset_koji_modules` are pulled in by NSVC and also ignore events.

However, the modular content tag did honor event, and could lead to a
crashed compose if the content tag did not exist at the configured
event.

This patch is a slightly too big hammer. It ignores events for all
modules, not just ones configured by explicit NSVC. It's not a huge deal
as the content tags are created before the corresponding module build is
created, and once all rpm builds are tagged into the content tag, MBS
will never change it again.

JIRA: RHELCMP-12765
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit b32c8f3e5e)
2024-08-30 13:30:38 +03:00
Lubomír Sedlář
548441644b
pkgset: Ignore duplicated module builds
If the module tag contains the same module build multiple times (because
it's in multiple tags in the inheritance), Pungi will not process that
correctly and try to include the same NSVC in the compose multiple
times. That leads to a crash.

This patch adds another step to the inheritance filter to ensure the
result contains each module only once.

JIRA: RHELCMP-12768
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 935da7c246)
2024-08-30 13:30:36 +03:00
Aditya Bisoi
ca369df0df
Drop buildinstall method
JIRA: RHELCMP-12388

Signed-off-by: Aditya Bisoi <abisoi@redhat.com>
(cherry picked from commit b513c8cd00)
2024-08-30 13:30:35 +03:00
Lingyan Zhuang
67ae4202c4
Add step to send UMB message
If reuse old ISO finished, send out UMB message.

Signed-off-by: Lingyan Zhuang <lzhuang@redhat.com>
(cherry picked from commit 8cf1d98312)
2024-08-30 13:30:33 +03:00
Timothée Ravier
aba5a7a093
Fix minor Ruff/flake8 warnings
```
pungi/checks.py:575:17: F601 [*] Dictionary key literal `"type"` repeated
pungi/phases/pkgset/pkgsets.py:617:12: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:241:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:244:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:370:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:374:20: E721 Do not compare types, use `isinstance()`
```

Signed-off-by: Timothée Ravier <tim@siosm.fr>
(cherry picked from commit 2534ddee99)
2024-08-30 13:30:32 +03:00
Simon de Vlieger
323d1c1eb6
osbuild: manifest type in config
Allow the manifest type used to be specified in the pungi configuration
instead of always selecting the manifest type based on the koji output.

Signed-off-by: Simon de Vlieger <cmdr@supakeen.com>
(cherry picked from commit f30a8b4d15)
2024-08-30 13:30:31 +03:00
Lubomír Sedlář
b0964ff555
4.5.1 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

(cherry picked from commit 3ffb991bac)
2024-08-30 13:30:30 +03:00
Ozan Unsal
79bc4e0c3a
gather_dnf.py: Do not raise error when the downloaded package is exists.
If the packages are pulled from different repos and a package is already
exists in target directory, pungi raises File exists error and breaks. This
behavior can be suspended and skipped if the package is already available.

Merges: https://pagure.io/pungi/pull-request/1696
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
(cherry picked from commit dbc0e531b2)
2024-08-30 13:30:05 +03:00
Lubomír Sedlář
8772ccca23
New upstream release 4.7.0
(cherry picked from commit e0600a2abac9e0e9b8a3b15b51eb44e3cd467bd3)
2024-08-30 13:29:32 +03:00
Fedora Release Engineering
3bb34225a9
Rebuilt for https://fedoraproject.org/wiki/Fedora_41_Mass_Rebuild
(cherry picked from commit 192a8ef731fbc134bf5337dfb3d60ba6c5ad7bd5)
2024-08-30 13:29:32 +03:00
Haibo Lin
daea6cabdf
Upstream release 4.6.3
(cherry picked from commit 9a24cfff1bfccbafde32a4a34805d9d0aeff5650)
2024-08-30 13:29:30 +03:00
Python Maint
35b720e87a
Rebuilt for Python 3.13
(cherry picked from commit 639bb6214433a96a6275817baf893ab4850a3309)
2024-08-30 13:29:29 +03:00
Lubomír Sedlář
5a6ee9f8eb
Bump release over f40-infra build
(cherry picked from commit 1ad8b6fa2edeb91316dd1d1e33a9c234800e28d9)
2024-08-30 13:29:28 +03:00
Lubomír Sedlář
9a64db0485
Require xorriso for bug#2278677
(cherry picked from commit 22214e03b888c9b5f85919815f2825ad176c5370)
2024-08-30 13:29:27 +03:00
Lubomír Sedlář
de7210f69a
Upstream release 4.6.2
(cherry picked from commit f24f577c89647dc80a84bfa76f3055d24ced55a5)
2024-08-30 13:29:05 +03:00
Lubomír Sedlář
24418ef74d
New upstream release 4.6.1
(cherry picked from commit 98b4f26e0972a2bea2d46f2c74c1db94ed087477)
2024-08-30 13:29:03 +03:00
f4765fbe3a
Remove python3-mock dependency
Merges: https://src.fedoraproject.org/rpms/pungi/pull-request/9

(cherry picked from commit 67a11d878b04bd46a0d9fb98036467bca6ffed92)
2024-08-30 13:28:01 +03:00
Fedora Release Engineering
80b9add9f7
Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild
(cherry picked from commit 40fd963a495689a2a3a0279760f5a4024e7e5857)
2024-08-30 13:27:24 +03:00
Fedora Release Engineering
b241545ca6
Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild
(cherry picked from commit 5cfb290545fdd5b18bb1691218e5e8e732e351e4)
2024-08-30 13:27:00 +03:00
Lubomír Sedlář
2e536228ae
Backport: Stop requiring repo option in ostree phase
(cherry picked from commit 6778cae05afb2b5784a46ed72ee2703785756dde)
2024-08-30 13:26:39 +03:00