Commit Graph

219 Commits

Author SHA1 Message Date
Lubomír Sedlář
cb9dede604
kiwibuild: Add support for type, type attr and bundle format
This is a very basic support. Whatever users specify in the new option
will be passed to the koji task.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=2270197
Related: https://pagure.io/koji/pull-request/4157
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit e43cf68f08)
2024-08-30 13:40:50 +03:00
Lubomír Sedlář
be4fd75a7a
Allow live_images phase to still be skipped
Without this fix existing configurations break even though they don't
use the phase.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit c59f2371a3)
2024-08-30 13:40:48 +03:00
Lubomír Sedlář
aef48c0ab4
Drop support for signing rpm-wrapped artifacts
This was only usable in live_images phase that doesn't exist anymore,
and wasn't used much in the first place.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 0726a4dca7)
2024-08-30 13:40:15 +03:00
Adam Williamson
bd91ef1d10
Remove live_images.py (LiveImagesPhase)
This phase was used to create live images with livecd-creator
and 32-bit ARM images with appliance-creator. We also remove
get_create_image_cmd from the Koji wrapper as it was only used
for this phase, remove associated tests, and remove related
configuration settings and documentation.

Fixes: https://pagure.io/pungi/issue/1753
Merges: https://pagure.io/pungi/pull-request/1774
Signed-off-by: Adam Williamson <awilliam@redhat.com>

(cherry picked from commit 531f0ef389)
2024-08-30 13:40:14 +03:00
Lubomír Sedlář
36133b71da
Drop kiwibuild_version option
Version in kiwibuild is embedded in the definition file. The option
makes no sense.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 242d7d951f)
2024-08-30 13:39:32 +03:00
Adam Williamson
57f2b428d5
kiwibuild: allow setting description scm and path at phase level
Neal wanted this to work - he tried using global_description_scm
and global_description_path in the initial PR - but it wasn't
wired up to work. This should make it possible to set
`kiwibuild_description_scm` and `kiwibuild_description_path`.
It also technically lets you set `global_` for both, since the
`get_config` implementation is very generic, but it doesn't add
it to the checks, so you'd still get an "unrecognized config
option" warning, I think. It seems appropriate to encourage
setting this as a phase-level option rather than a global one
since it seems quite specific to the kiwibuild phase.

Merges: https://pagure.io/pungi/pull-request/1737
Signed-off-by: Adam Williamson <awilliam@redhat.com>
(cherry picked from commit e90ffdfd93)
2024-08-30 13:39:29 +03:00
Tomáš Hozza
a97488721d
Phases/osbuild: support passing 'customizations' for image builds
The osbuild Koji plugin supports passing customizations for an image
build. This is also supported in the Koji CLI plugin. Some teams want to
pass image customizations for images built as part of Pungi composes.
Extend the osbuild phase to support passing customizations in the Pungi
configuration.

Merges: https://pagure.io/pungi/pull-request/1733
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit e738f65458)
2024-08-30 13:39:16 +03:00
Lubomír Sedlář
f28053eecc
kiwibuild: Add kiwibuild global options
This is already supported by code, just missing in the schema.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit b9d94970b5)
2024-08-30 13:39:10 +03:00
Lubomír Sedlář
a6f6199910
kiwibuild: Add subvariant configuration
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit bcd937d16d)
2024-08-30 13:39:07 +03:00
Lubomír Sedlář
a3dcec5059
kiwibuild: Work around missing arch in build data
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit f0137fd9b9)
2024-08-30 13:39:05 +03:00
Haibo Lin
6aa674fbb3
Support KiwiBuild
Adding kiwibuild phase which is similar to osbuild.

Fixes: https://pagure.io/pungi/issue/1710
Merges: https://pagure.io/pungi/pull-request/1720
JIRA: RHELCMP-13348
Signed-off-by: Haibo Lin <hlin@redhat.com>
(cherry picked from commit 3d630d3e8e)
2024-08-30 13:39:04 +03:00
Adam Williamson
4c0059e91b
checks: don't require "repo" in the "ostree" schema
Per @siosm in https://pagure.io/pungi-fedora/pull-request/1227
this option "is deprecated and not needed anymore", so Pungi
should not be requiring it.

Merges: https://pagure.io/pungi/pull-request/1714
Signed-off-by: Adam Williamson <awilliam@redhat.com>
(cherry picked from commit 432b0bce04)
2024-08-30 13:31:23 +03:00
Lubomír Sedlář
c589ccb56f
Updates for ostree-container phase
This patch connects the phase into the main script, and adds other
modifications:

* The archive is now stored in the images/ subdirectory in the compose.
* Documentation is updated to correctly mention that variant repos are
  not available.
* Configuration for path and name of the final archive is dropped. There
  are reasonable defaults for this and there's no point in having users
  configure it.
* The extra message for the archive is no longer sent.
* The pungi-make-ostree utility is no longer required in the buildroot.

The pungi-make-ostree utility doesn't do any significant work. It
modifies configuration files (which can happen on the compose host), and
it starts other processes.

This patch changes the ostree-container phase to no longer need the
script in the buildroot. Instead, the utility is called on the compose
host to do the config manipulation and output the needed commands. Those
are then passed into the runroot task.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 081c31238b)
2024-08-30 13:30:42 +03:00
Timothée Ravier
e413955849
Add ostree native container support
Add a new `ostree_container` stage to create ostree native container
images as OCI archives, using rpm-ostree compose image.

See: https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable
See: https://gitlab.com/CentOS/cloud/issue-tracker/-/issues/1

Fixes: https://pagure.io/pungi/issue/1698
Merges: https://pagure.io/pungi/pull-request/1699

Signed-off-by: Timothée Ravier <tim@siosm.fr>
(cherry picked from commit 95497d2676)
2024-08-30 13:30:41 +03:00
Aditya Bisoi
ca369df0df
Drop buildinstall method
JIRA: RHELCMP-12388

Signed-off-by: Aditya Bisoi <abisoi@redhat.com>
(cherry picked from commit b513c8cd00)
2024-08-30 13:30:35 +03:00
Timothée Ravier
aba5a7a093
Fix minor Ruff/flake8 warnings
```
pungi/checks.py:575:17: F601 [*] Dictionary key literal `"type"` repeated
pungi/phases/pkgset/pkgsets.py:617:12: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:241:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:244:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:370:16: E721 Do not compare types, use `isinstance()`
tests/test_pkgset_source_koji.py:374:20: E721 Do not compare types, use `isinstance()`
```

Signed-off-by: Timothée Ravier <tim@siosm.fr>
(cherry picked from commit 2534ddee99)
2024-08-30 13:30:32 +03:00
Simon de Vlieger
323d1c1eb6
osbuild: manifest type in config
Allow the manifest type used to be specified in the pungi configuration
instead of always selecting the manifest type based on the koji output.

Signed-off-by: Simon de Vlieger <cmdr@supakeen.com>
(cherry picked from commit f30a8b4d15)
2024-08-30 13:30:31 +03:00
Lubomír Sedlář
f6f54b56ca
Add support for not having koji volume mounted locally
With this patch, Pungi can be configured with a local directory to be
used as a cache for RPMs, and it will download packages from Koji over
HTTP instead of reading them from filesystem directly.

The files from the cache can then be hardlink as usual.

There is locking in place to avoid different composes running at the
same time to step on each other.

This is now supported for RPMs only, be it real builds or scratch
builds.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 631bb01d8f)
2023-11-10 16:55:19 +02:00
Lubomír Sedlář
0f8cae69b7
ostree: Add configuration for custom runroot packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit bd64894a03)
2023-11-10 16:55:01 +02:00
Lubomír Sedlář
f3485410ad
Add support for git-credential-helper
This patch adds an additional field `options` to scm_dict, which can be
used to provide additional information to the backends.

It implements a single new option for GitWrapper. This option allows
setting a custom git credentials wrapper. This can be useful if Pungi
needs to get files from a git repository that requires authentication.

The helper can be as simple as this (assuming the username is already
provided in the url):

    #!/bin/sh
    echo password=i-am-secret

The helper would need to be referenced by an absolute path from the
pungi configuration, or prefixed with ! to have git interpret it as a
shell script and look it up in PATH.

See https://git-scm.com/docs/gitcredentials for more details.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: RHELCMP-11808
(cherry picked from commit ada8f4e346)
2023-11-10 16:54:59 +02:00
Haibo Lin
cccfaea14e
Support OIDC Client Credentials authentication to CTS
JIRA: RHELCMP-11324
Signed-off-by: Haibo Lin <hlin@redhat.com>
(cherry picked from commit e4c525ecbf)
2023-11-10 16:54:58 +02:00
Lubomír Sedlář
997e372f25
Add back compatibility with jsonschema <3.0.0
Resolves: https://pagure.io/pungi/issue/1667
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

(cherry picked from commit e888e76992 (centos_master))
2023-11-10 16:54:00 +02:00
Lubomír Sedlář
4cb8671fe4
Allow disabling inheriting ExcludeArch to noarch packages
Copying ExcludeArch/ExclusiveArch from source rpm to noarch is an easy
option to block shipping that particular noarch package from a certain
architecture. However, there is no way to bypass it, and it is rather
confusing and not discoverable.

An alternative way to remove an unwanted package is to use the good old
`filter_packages`, which has enough granularity to remove pretty much
anything from anywhere. The only downside is that it requires a change
in configuration, so it can't be done by a packager directly from a spec
file.

When we decide to break backwards compatibility, this option should be
removed and the entire ExcludeArch/ExclusiveArch inheritance removed
completely.

JIRA: ENGCMP-2606
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit ab508c1511)
2023-11-10 16:51:37 +02:00
soksanichenko
1c3e5dce5e - CLI option --label can be passed through a Pungi config file
- Bump version
- Update changelog
2023-04-13 00:57:39 +03:00
Tomáš Hozza
9e014fed6a osbuild: support specifying package_sets for repos
The `koji-osbuild` plugin supports additional formats for the `repo`
property since v4 [1]. Specifically, a repo can be specified as a
dictionary with `baseurl` key and `package_sets` list containing
specific package set names, that the repository should be used for.

Extend the configuration schema to reflect the plugin change.
Extend the documentation to cover the new repository format.
Extend an existing unit test to specify additional repository using the
added format.

[1] https://github.com/osbuild/koji-osbuild/pull/82

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit 8f0906be53)
2023-03-17 11:58:11 +02:00
Tomáš Hozza
abec28256d osbuild: update schema and config documentation
The `koji-osbuild` Hub schema has been relaxed a bit in the latest
release (v11). Adjust the schema in Pungi to reflect changes in
`koji-osbuild`.

For more information on the changes in `koji-osbuild`, see:
https://github.com/osbuild/koji-osbuild/pull/108

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit ef6d40dce4)
2023-03-17 11:58:11 +02:00
soksanichenko
4e0d2d14c9 - Unify branch for both RHEL versions 2022-11-11 16:31:43 +02:00
soksanichenko
d914626d92 - "kojimock" is valid value for option "pkgset_source" 2022-11-09 17:59:50 +02:00
soksanichenko
9ae49dae5b - Fix unittests 2022-11-08 01:43:53 +02:00
soksanichenko
c82cbfdc32 - Fix unittests 2022-11-08 00:59:10 +02:00
soksanichenko
323d31df2b Merge branch 'master' into a8_updated
# Conflicts:
#	pungi.spec
#	pungi/wrappers/kojiwrapper.py
#	setup.py
#	tests/test_extra_isos_phase.py
#	tests/test_pkgset_pkgsets.py
2022-11-07 23:38:38 +02:00
Tomas Hozza
57739c238f
osbuild: support specifying upload_options
Since version 9, the `koji-osbuild` plugin supports specifying upload
options as part of a Koji build. This enables one to upload the built
image directly to the cloud environment as part of the image build in
Koji.

Extend the configuration schema with `upload_options`.
Extend the documentation and describe valid `upload_options` values.
Add a unit test testing a scenario when `upload_options` are specified.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-09-06 11:20:01 +02:00
Tomas Hozza
805a1083a2
osbuild: accept only a single image type in the configuration
Modify the osbuild configuration schema to accept only an array with a
single value as the `image_types`, in addition to a single string. The
single string was supported by the schema also before, but this fact was
not mentioned in the documentation, nor it was supported by the
`koji-osbuild` plugin of version lower than `9`.

Update the documentation accordingly.

Add unit test for invalid configuration containing more than one image
type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-09-06 10:55:25 +02:00
Ondřej Budai
779793386c osbuild: add support for building ostree artifacts
In order to start building Fedora IoT images with osbuild, we need to be able
to pass ostree options from pungi to the koji's osbuildImage task.

This commit adds support for it via new configuration options: ostree_url,
ostree_url and ostree_parent.

A test was added to cover these new options and they are were also added
into the documentation.

JIRA: COMPOSER-1702
Merges: https://pagure.io/pungi/pull-request/1624
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-08-25 15:35:17 +02:00
Timothée Ravier
603c61a033 ostree: Add unified core mode for compose in rpm-ostree
rpm-ostree is moving to unified core composes and this is now working
for Silverblue & Kinoite.

This is untested for IoT but they should move to os-build with Fedora
37.

See: https://github.com/coreos/rpm-ostree/issues/729
Merges: https://pagure.io/pungi/pull-request/1626
Signed-off-by: Timothée Ravier <tim@siosm.fr>
2022-08-23 10:52:45 +02:00
Lubomír Sedlář
11fa342507 createiso: Make ISO level more granular
Make it possible to set the level separately for each variant and
architecture.

JIRA: RHELCMP-9341

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-08-16 14:38:18 +02:00
Lubomír Sedlář
0abf937b0e Fix compatibility with jsonschema >= 4.0.0
Fedora Rawhide (to be 37) packages jsonschema 4.9.0 at the moment, so we
can no longer get by with limiting the requirements. This patch makes
the validation work with both old and new version.

Fixes: rhbz#2113607
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-08-10 09:52:23 +02:00
Lingyan Zhuang
c4aa45beab Add skip_branding to ostree_installer.
Fixes: #1594
Merges: https://pagure.io/pungi/pull-request/1609
Signed-off-by: Lingyan Zhuang <lzhuang@redhat.com>
2022-05-11 15:19:53 +02:00
Christopher O'Brien
d55770898c nomacboot option for livemedia koji tasks
Merges: https://pagure.io/pungi/pull-request/1591
Signed-off-by: Christopher O'Brien <cobrien@redhat.com>
2022-03-23 09:36:51 +01:00
Filip Valder
fe986d68b9 Add module obsoletes feature
JIRA: MODULAR-113
Merges: https://pagure.io/pungi/pull-request/1578
Signed-off-by: Filip Valder <fvalder@redhat.com>
2022-01-12 12:51:37 +01:00
Ozan Unsal
42f668d969 buildinstall: Add ability to install extra packages in runroot
Resolves: https://pagure.io/pungi/issue/1461
Merges: https://pagure.io/pungi/pull-request/1580
JIRA: RHELCMP-2911
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2022-01-05 14:35:37 +01:00
Lubomír Sedlář
80bd254347 Check dependencies after config validation
This way the checks can rely on default values from the config.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-04 13:37:51 +00:00
Ken Dreyer
94ffa1c5c6 default "with_jigdo" to False
Fedora has not composed with jigdo in a long time. Disable it by
default.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
Merges: https://pagure.io/pungi/pull-request/1561
Fixes: https://pagure.io/pungi/issue/1560
2021-11-04 13:37:51 +00:00
Lubomír Sedlář
e8305f3978 extra_isos: Allow reusing old images
When nothing in configuration or the image itself changed, let's just
copy the older one.

JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-02 07:08:00 +00:00
Lubomír Sedlář
ac66c3d7f3 createiso: Allow reusing old images
This patch allows Pungi to reuse ISO image created in previous compose
if a list of assumptions proves to hold:

 * If image is bootable, buildinstall phase must have been reused too.
 * Compose configuration must have not changed (except for a few
   whitelisted options).
 * Volume ID of the ISO much not have changed.
 * No RPM on the ISO must have changed.

The ISO also contains other files. Changes in extra files and product ID
certificates should be visible in configuration (the SHA will differ).
Similarly any repodata configuration would be reflected in
configuration.

JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-02 07:08:00 +00:00
Haibo Lin
8133676270 osbs: Reuse images from old compose
JIRA: RHELCMP-5972
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-10-20 14:22:19 +08:00
Haibo Lin
e42e65783d image_build: Allow reusing old image_build results
JIRA: RHELCMP-5970
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-10-20 14:07:02 +08:00
Ozan Unsal
7475d2a3a9 Allow ISO-Level configuration within the config file
In order to enable this feature set "iso_level=<Value from 1 to 4>"
in config file

Jira: RHELCMP-6880

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-10-18 16:32:04 +02:00
Ozan Unsal
b7666ba4a4 Enable pungi to send compose_url patches to CTS
If cts_keytab is also enabled then the HTTP requests are handled with
Kerberos Authentication otherwise no authentication is used.

If cts_url is defined in the configuration, translate_paths is required.
This is needed in order to get the host and the path of the composes.

Jira: RHELCMP-6318

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-09-07 16:41:35 +02:00
Haibo Lin
56a55db966 Use cachedir when createrepo
Then createrepo can reuse checksum values from cache to make it faster.

JIRA: RHELCMP-5984
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-07-27 09:23:47 +08:00